2024-09-21T05:53:48.265 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T05:53:48.296 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T05:53:48.387 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913523 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913523' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_export_ephemeral_random_max: 0.1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBb+CWlYqfY++Rv0gy/69nP2+iaPOUkEv6SRjYyXH8UQfdsjImahPJgegO5j4NWcfWwMG3AQGei+90YxCQu69z4= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI3kCkpxe1EAOxPQkY35c3AoAWd8HI52cWVco95onNw2UNtQQm5xWvFDVe12vbHbo2RykMMHVvEPEV1nc9TV2eY= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKGkvrR6Qac95dhHTss/nmKuA41ARTZhq6vMoGkDmwcygHOOzTDw88y+b9w7TEJqXh3mVV52ay9bk3pZJBNHCoY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T05:53:48.387 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T05:53:48.387 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T05:53:48.387 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T05:53:48.389 INFO:teuthology.task.internal:Checking packages... 2024-09-21T05:53:48.407 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T05:53:48.407 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T05:53:48.407 INFO:teuthology.packaging:ref: None 2024-09-21T05:53:48.407 INFO:teuthology.packaging:tag: None 2024-09-21T05:53:48.407 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T05:53:48.407 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:53:48.408 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T05:53:48.550 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T05:53:48.551 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T05:53:48.557 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T05:53:48.558 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T05:53:48.611 INFO:teuthology.task.internal:Saving configuration 2024-09-21T05:53:48.630 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T05:53:48.636 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T05:53:48.661 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913523', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 05:48:39.184848', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBb+CWlYqfY++Rv0gy/69nP2+iaPOUkEv6SRjYyXH8UQfdsjImahPJgegO5j4NWcfWwMG3AQGei+90YxCQu69z4='} 2024-09-21T05:53:48.684 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913523', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 05:48:39.187439', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI3kCkpxe1EAOxPQkY35c3AoAWd8HI52cWVco95onNw2UNtQQm5xWvFDVe12vbHbo2RykMMHVvEPEV1nc9TV2eY='} 2024-09-21T05:53:48.707 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913523', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 05:48:39.186230', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKGkvrR6Qac95dhHTss/nmKuA41ARTZhq6vMoGkDmwcygHOOzTDw88y+b9w7TEJqXh3mVV52ay9bk3pZJBNHCoY='} 2024-09-21T05:53:48.707 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T05:53:48.714 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T05:53:48.714 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T05:53:48.714 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T05:53:48.714 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T05:53:48.790 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f018eecc160>, signals=[15]) 2024-09-21T05:53:48.791 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T05:53:48.796 INFO:teuthology.task.internal:Opening connections... 2024-09-21T05:53:48.797 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2024-09-21T05:53:48.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:53:48.875 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-09-21T05:53:48.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:53:48.947 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-09-21T05:53:48.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:53:49.021 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T05:53:49.027 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2024-09-21T05:53:49.046 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2024-09-21T05:53:49.046 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2024-09-21T05:53:49.103 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream" 2024-09-21T05:53:49.103 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="9" 2024-09-21T05:53:49.103 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos" 2024-09-21T05:53:49.104 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora" 2024-09-21T05:53:49.104 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="9" 2024-09-21T05:53:49.104 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el9" 2024-09-21T05:53:49.104 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T05:53:49.104 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2024-09-21T05:53:49.104 INFO:teuthology.orchestra.run.smithi003.stdout:LOGO="fedora-logo-icon" 2024-09-21T05:53:49.104 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T05:53:49.104 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/" 2024-09-21T05:53:49.104 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T05:53:49.104 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T05:53:49.104 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T05:53:49.105 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2024-09-21T05:53:49.149 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-09-21T05:53:49.167 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-09-21T05:53:49.167 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-09-21T05:53:49.225 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-09-21T05:53:49.225 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-09-21T05:53:49.225 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-09-21T05:53:49.225 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-09-21T05:53:49.225 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-09-21T05:53:49.225 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-09-21T05:53:49.225 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T05:53:49.225 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-09-21T05:53:49.225 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-09-21T05:53:49.226 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T05:53:49.226 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-09-21T05:53:49.226 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T05:53:49.226 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T05:53:49.226 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T05:53:49.226 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-09-21T05:53:49.248 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-09-21T05:53:49.267 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-09-21T05:53:49.267 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-09-21T05:53:49.325 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-09-21T05:53:49.325 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-09-21T05:53:49.325 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-09-21T05:53:49.325 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-09-21T05:53:49.325 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-09-21T05:53:49.326 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-09-21T05:53:49.326 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T05:53:49.326 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-09-21T05:53:49.326 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-09-21T05:53:49.326 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T05:53:49.326 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-09-21T05:53:49.326 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T05:53:49.326 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T05:53:49.326 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T05:53:49.326 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-09-21T05:53:49.348 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T05:53:49.365 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T05:53:49.390 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T05:53:49.390 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2024-09-21T05:53:49.392 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-09-21T05:53:49.395 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-09-21T05:53:49.411 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T05:53:49.431 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T05:53:49.431 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2024-09-21T05:53:49.449 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-09-21T05:53:49.452 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-09-21T05:53:49.606 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T05:53:49.706 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T05:53:49.720 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T05:53:49.721 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-21T05:53:49.721 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-21T05:53:49.721 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T05:53:49.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:53:49.738 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-21T05:53:49.796 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-503.el9.x86_64 2024-09-21T05:53:49.797 INFO:teuthology.task.kernel:Running kernel on smithi003: 5.14.0-503.el9.x86_64 2024-09-21T05:53:49.797 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum install -y kernel 2024-09-21T05:53:53.255 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-21T05:53:55.306 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-21T05:53:59.773 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-21T05:54:01.482 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-21T05:54:02.370 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-21T05:54:08.034 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T05:54:10.024 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T05:54:10.024 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T05:54:10.109 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-21T05:54:10.116 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-21T05:54:10.116 INFO:teuthology.orchestra.run.smithi003.stdout: Package Architecture Version Repository Size 2024-09-21T05:54:10.116 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-21T05:54:10.116 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-21T05:54:10.116 INFO:teuthology.orchestra.run.smithi003.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T05:54:10.116 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-21T05:54:10.117 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T05:54:10.117 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T05:54:10.117 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T05:54:10.117 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T05:54:10.117 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-21T05:54:10.117 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-21T05:54:10.117 INFO:teuthology.orchestra.run.smithi003.stdout:Install 4 Packages 2024-09-21T05:54:10.117 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T05:54:10.118 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 79 M 2024-09-21T05:54:10.118 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 126 M 2024-09-21T05:54:10.118 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-21T05:54:10.669 INFO:teuthology.orchestra.run.smithi003.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 123 kB/s | 36 kB 00:00 2024-09-21T05:54:12.544 INFO:teuthology.orchestra.run.smithi003.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 7.2 MB/s | 16 MB 00:02 2024-09-21T05:54:13.235 INFO:teuthology.orchestra.run.smithi003.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 12 MB/s | 35 MB 00:02 2024-09-21T05:54:13.794 INFO:teuthology.orchestra.run.smithi003.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 9.1 MB/s | 29 MB 00:03 2024-09-21T05:54:13.794 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:54:13.794 INFO:teuthology.orchestra.run.smithi003.stdout:Total 21 MB/s | 79 MB 00:03 2024-09-21T05:54:14.501 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-21T05:54:14.877 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-21T05:54:14.877 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-21T05:54:15.886 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-21T05:54:15.887 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-21T05:54:18.439 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-21T05:54:19.108 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T05:54:19.196 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:54:21.927 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:54:22.049 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:54:27.169 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:54:27.208 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:54:32.356 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:55:11.602 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:55:11.643 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:55:11.976 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:55:11.976 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T05:55:11.976 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:55:11.976 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:55:12.840 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:55:12.840 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T05:55:12.840 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-21T05:55:12.840 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T05:55:12.840 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T05:55:12.840 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T05:55:12.840 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T05:55:12.840 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T05:55:12.841 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-21T05:55:13.204 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T05:55:13.298 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T05:55:13.298 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T05:55:13.298 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T05:55:13.298 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T05:55:13.299 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T05:55:13.299 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T05:55:13.299 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T05:55:13.299 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T05:55:13.358 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T05:55:13.358 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T05:55:13.358 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -qi grub2-tools 2024-09-21T05:55:13.398 INFO:teuthology.orchestra.run.smithi003.stdout:Name : grub2-tools 2024-09-21T05:55:13.398 INFO:teuthology.orchestra.run.smithi003.stdout:Epoch : 1 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:Version : 2.06 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:Release : 82.el9 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: x86_64 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:Group : Unspecified 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:Size : 8274670 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:License : GPLv3+ 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:Packager : builder@centos.org 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:Vendor : CentOS 2024-09-21T05:55:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T05:55:13.400 INFO:teuthology.orchestra.run.smithi003.stdout:Summary : Support tools for GRUB. 2024-09-21T05:55:13.400 INFO:teuthology.orchestra.run.smithi003.stdout:Description : 2024-09-21T05:55:13.400 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T05:55:13.400 INFO:teuthology.orchestra.run.smithi003.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T05:55:13.400 INFO:teuthology.orchestra.run.smithi003.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T05:55:13.400 INFO:teuthology.orchestra.run.smithi003.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T05:55:13.400 INFO:teuthology.orchestra.run.smithi003.stdout:hardware devices. 2024-09-21T05:55:13.400 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T05:55:13.400 INFO:teuthology.orchestra.run.smithi003.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T05:55:13.401 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T05:55:13.401 INFO:teuthology.task.kernel:Updating grub on smithi003 to boot 5.14.0-508.el9.x86_64 2024-09-21T05:55:13.401 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T05:55:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2024-09-21T05:55:15.334 INFO:teuthology.orchestra.run.smithi003.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T05:55:15.364 INFO:teuthology.orchestra.run.smithi003.stderr:done 2024-09-21T05:55:15.366 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-09-21T05:55:15.381 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.SmnombRlaA 2024-09-21T05:55:15.382 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.SmnombRlaA 2024-09-21T05:55:15.455 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.SmnombRlaA 2024-09-21T05:55:15.595 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.SmnombRlaA is 6KB 2024-09-21T05:55:15.643 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.SmnombRlaA 2024-09-21T05:55:15.658 DEBUG:teuthology.orchestra.run.smithi003:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T05:55:15.719 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T05:55:15.720 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T05:55:15.720 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T05:55:15.720 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T05:55:15.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T05:55:15.884 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2024-09-21T05:55:45.886 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T05:55:45.887 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-21T05:55:45.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:56:04.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2024-09-21T05:56:13.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-21T05:56:13.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:57:13.209 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T05:57:25.210 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-21T05:57:25.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:57:25.584 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-21T05:57:25.939 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-21T05:57:25.939 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T05:57:25.940 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-21T05:57:25.957 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-508.el9.x86_64 2024-09-21T05:57:25.957 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T05:57:25.957 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T05:57:25.957 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T05:57:26.958 DEBUG:teuthology.parallel:result is None 2024-09-21T05:57:26.958 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T05:57:26.966 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T05:57:26.966 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T05:57:26.969 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T05:57:26.971 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T05:57:26.990 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T05:57:26.997 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T05:57:27.003 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T05:57:27.003 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T05:57:27.027 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T05:57:27.030 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T05:57:27.098 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T05:57:27.104 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T05:57:27.104 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T05:57:27.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:57:27.122 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T05:57:27.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:57:27.139 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T05:57:27.141 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T05:57:27.164 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T05:57:27.206 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:57:27.220 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:57:27.221 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:57:27.239 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:57:27.246 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:57:27.265 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:57:27.266 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T05:57:27.273 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T05:57:27.274 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T05:57:27.276 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T05:57:27.279 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T05:57:27.307 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T05:57:27.315 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T05:57:27.315 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T05:57:27.346 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T05:57:27.349 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T05:57:27.366 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:57:27.445 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:57:27.499 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:57:27.583 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:57:27.626 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T05:57:27.626 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T05:57:27.692 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:57:27.736 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:57:27.788 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:57:27.872 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:57:27.915 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-21T05:57:27.915 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T05:57:27.980 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:57:28.028 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:57:28.091 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:57:28.138 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:57:28.177 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-21T05:57:28.177 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T05:57:28.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2024-09-21T05:57:28.246 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-09-21T05:57:28.249 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-09-21T05:57:28.286 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:57:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:57:28.333 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:57:28.680 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T05:57:28.689 INFO:teuthology.task.internal:Starting timer... 2024-09-21T05:57:28.689 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T05:57:28.699 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T05:57:28.711 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T05:57:28.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2024-09-21T05:57:28.783 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: 2024-09-21T05:57:28.785 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-09-21T05:57:28.821 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-09-21T05:57:28.823 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-09-21T05:57:28.875 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-09-21T05:57:28.877 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T05:57:28.877 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2024-09-21T05:57:28.897 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2024-09-21T05:57:28.897 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-09-21T05:57:28.922 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-09-21T05:57:28.922 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-09-21T05:57:28.955 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-09-21T05:57:28.955 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-09-21T05:57:28.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T05:57:28.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:57:28.985 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T05:57:29.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:57:29.012 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T05:57:29.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:57:29.040 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T05:57:29.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2024-09-21T05:57:29.075 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-09-21T05:57:29.103 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-09-21T05:57:29.139 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T05:57:29.147 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T05:57:29.157 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T05:57:29.158 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi137.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T06:01:44.880 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-09-21T06:01:44.881 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-21T06:01:44.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:01:44.962 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-21T06:01:45.038 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-21T06:01:45.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-09-21T06:01:45.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:01:45.112 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-09-21T06:01:45.190 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-09-21T06:01:45.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-21T06:01:45.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:01:45.269 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-09-21T06:01:45.351 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-21T06:01:45.352 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T06:01:45.363 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T06:01:45.364 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:01:45.364 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:01:45.366 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:01:45.366 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:01:45.369 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:01:45.369 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:01:45.398 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:01:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:01:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:01:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:01:45.423 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:01:45.438 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:01:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-09-21T06:01:45.443 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2024-09-21T06:01:45.450 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-09-21T06:01:45.456 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2024-09-21T06:01:45.464 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-09-21T06:01:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:01:45.472 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:01:45.475 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-09-21T06:01:45.479 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:01:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:01:45.490 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:01:45.504 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:01:45.537 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:01:45.540 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:01:45.540 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-09-21T06:01:45.540 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:01:45.541 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:01:45.541 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:01:45.541 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:01:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:01:45.553 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:01:45.553 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-09-21T06:01:45.553 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:01:45.553 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:01:45.553 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:01:45.553 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:01:45.561 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:01:45.566 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:01:45.566 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-09-21T06:01:45.566 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:01:45.566 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:01:45.566 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:01:45.566 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:01:45.567 INFO:teuthology.run_tasks:Running task install... 2024-09-21T06:01:45.576 DEBUG:teuthology.task.install:project ceph 2024-09-21T06:01:45.576 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T06:01:45.576 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T06:01:45.576 INFO:teuthology.task.install:Using flavor: default 2024-09-21T06:01:45.581 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T06:01:45.582 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T06:01:45.582 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:01:45.582 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:01:45.584 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:01:45.584 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:01:45.585 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:01:45.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:01:45.730 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:01:45.730 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:01:45.733 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:01:45.733 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:01:45.739 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:01:45.740 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:01:45.839 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:01:45.839 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:01:45.839 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:01:45.845 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:01:45.845 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-21T06:01:45.845 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:01:45.852 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:01:45.852 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-21T06:01:45.853 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:01:45.866 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:01:45.867 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:01:45.871 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:01:45.872 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:01:45.878 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:01:45.878 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:01:45.934 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:01:45.938 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:01:45.944 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:01:46.011 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:01:46.012 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:01:46.019 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:01:46.077 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-09-21T06:01:46.077 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2024-09-21T06:01:46.078 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-09-21T06:01:46.079 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2024-09-21T06:01:46.083 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-09-21T06:01:46.085 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-09-21T06:01:46.306 INFO:teuthology.orchestra.run.smithi204.stdout:45 files removed 2024-09-21T06:01:46.310 INFO:teuthology.orchestra.run.smithi137.stdout:45 files removed 2024-09-21T06:01:46.318 INFO:teuthology.orchestra.run.smithi003.stdout:45 files removed 2024-09-21T06:01:46.333 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:01:46.336 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:01:46.345 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:01:46.878 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 318 kB/s | 84 kB 00:00 2024-09-21T06:01:46.883 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-09-21T06:01:46.918 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-21T06:01:47.163 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-21T06:01:47.175 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T06:01:47.211 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-21T06:01:47.435 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-21T06:01:47.438 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T06:01:47.495 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-21T06:01:47.885 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.3 MB 00:00 2024-09-21T06:01:50.028 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.3 MB 00:02 2024-09-21T06:01:51.977 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 20 MB 00:03 2024-09-21T06:01:53.446 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 1.4 MB/s | 8.3 MB 00:05 2024-09-21T06:01:54.112 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 6.9 MB/s | 20 MB 00:02 2024-09-21T06:01:56.515 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-21T06:01:58.019 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.5 MB 00:01 2024-09-21T06:01:58.446 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-21T06:01:59.753 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 19 kB 00:00 2024-09-21T06:02:00.188 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-21T06:02:00.533 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-21T06:02:00.976 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-21T06:02:01.406 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.5 MB 00:01 2024-09-21T06:02:02.849 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 19 kB 00:00 2024-09-21T06:02:03.611 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-21T06:02:05.980 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-09-21T06:02:06.632 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T06:02:08.159 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:02:08.160 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:02:08.173 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:02:08.174 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:02:08.175 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:02:08.176 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:02:08.178 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:02:08.181 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:02:08.182 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:02:08.246 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-09-21T06:02:08.259 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-09-21T06:02:08.259 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-09-21T06:02:08.259 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-09-21T06:02:08.259 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-09-21T06:02:08.259 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:02:08.259 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:02:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:02:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:02:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:02:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:02:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:02:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:02:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:02:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:02:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:02:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:02:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:02:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:02:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:02:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:02:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:02:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:02:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:02:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:02:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:02:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:02:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:02:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:02:08.262 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-09-21T06:02:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:02:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:02:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:02:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:02:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:02:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:02:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:02:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:02:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:02:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:02:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:02:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:02:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:02:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:02:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:02:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:02:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:02:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:02:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:02:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:02:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:02:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:02:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:02:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:02:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:02:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:02:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:02:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:02:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:02:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:02:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:02:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:02:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:02:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:02:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout:Install 149 Packages 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 8 Packages 2024-09-21T06:02:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:02:08.272 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 228 M 2024-09-21T06:02:08.272 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-09-21T06:02:08.831 INFO:teuthology.orchestra.run.smithi137.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 50 kB/s | 6.6 kB 00:00 2024-09-21T06:02:08.928 INFO:teuthology.orchestra.run.smithi003.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:02:08.929 INFO:teuthology.orchestra.run.smithi003.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:02:08.943 INFO:teuthology.orchestra.run.smithi003.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:02:08.944 INFO:teuthology.orchestra.run.smithi003.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:02:08.945 INFO:teuthology.orchestra.run.smithi003.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:02:08.946 INFO:teuthology.orchestra.run.smithi003.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:02:08.948 INFO:teuthology.orchestra.run.smithi003.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:02:08.951 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:02:08.952 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:02:09.022 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-21T06:02:09.036 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-09-21T06:02:09.036 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-21T06:02:09.036 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-09-21T06:02:09.036 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-21T06:02:09.037 INFO:teuthology.orchestra.run.smithi003.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:02:09.037 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:02:09.037 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:02:09.037 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:02:09.037 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:02:09.037 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:02:09.038 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:02:09.038 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:02:09.038 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:02:09.038 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:02:09.038 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:02:09.038 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:02:09.039 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:02:09.039 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:02:09.039 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:02:09.039 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:02:09.039 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:02:09.039 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:02:09.040 INFO:teuthology.orchestra.run.smithi003.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:02:09.040 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:02:09.040 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:02:09.040 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:02:09.040 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:02:09.040 INFO:teuthology.orchestra.run.smithi003.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:02:09.041 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:02:09.041 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:02:09.041 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:02:09.041 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:02:09.041 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:02:09.041 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:02:09.042 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:02:09.042 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:02:09.042 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:02:09.042 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:02:09.042 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:02:09.042 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:02:09.043 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:02:09.043 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2024-09-21T06:02:09.043 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:02:09.043 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:02:09.043 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:02:09.043 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:02:09.044 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:02:09.044 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:02:09.044 INFO:teuthology.orchestra.run.smithi003.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:02:09.044 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:02:09.044 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-21T06:02:09.044 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:02:09.044 INFO:teuthology.orchestra.run.smithi003.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:02:09.045 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:02:09.045 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:02:09.045 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:02:09.045 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:02:09.045 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:02:09.045 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:02:09.046 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:02:09.046 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:02:09.046 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:02:09.046 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:02:09.046 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:02:09.046 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:02:09.047 INFO:teuthology.orchestra.run.smithi003.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:02:09.047 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:02:09.047 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:02:09.047 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:02:09.047 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:02:09.047 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:02:09.048 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:02:09.048 INFO:teuthology.orchestra.run.smithi003.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:02:09.048 INFO:teuthology.orchestra.run.smithi003.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:02:09.048 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:02:09.048 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:02:09.048 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:02:09.049 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:02:09.049 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:02:09.049 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:02:09.049 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:02:09.049 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:02:09.049 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:02:09.050 INFO:teuthology.orchestra.run.smithi003.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:02:09.050 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:02:09.050 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:02:09.050 INFO:teuthology.orchestra.run.smithi003.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:02:09.050 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:02:09.050 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:02:09.050 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:02:09.051 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:02:09.051 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:02:09.051 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:02:09.051 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:02:09.051 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:02:09.051 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:02:09.051 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:02:09.052 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:02:09.052 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:02:09.052 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:02:09.052 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:02:09.052 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:02:09.052 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:02:09.053 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:02:09.053 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:02:09.053 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:02:09.053 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:02:09.053 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:02:09.053 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:02:09.054 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:02:09.054 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:02:09.054 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:02:09.054 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:02:09.054 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:02:09.054 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:02:09.055 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:02:09.055 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:02:09.055 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:02:09.055 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:02:09.055 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:02:09.055 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:02:09.055 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:02:09.056 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:02:09.056 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:02:09.056 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:02:09.056 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:02:09.056 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:02:09.056 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:02:09.057 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:02:09.057 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:02:09.057 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:02:09.057 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:02:09.057 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:02:09.057 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:02:09.058 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:02:09.058 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:02:09.058 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:02:09.058 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:02:09.058 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:02:09.058 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:02:09.058 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:02:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:02:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:02:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:02:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:02:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:02:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:02:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:02:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:02:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:02:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:02:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:02:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:02:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:02:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:02:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:02:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:02:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:02:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:02:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:02:09.062 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:02:09.062 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:02:09.062 INFO:teuthology.orchestra.run.smithi003.stdout:Installing weak dependencies: 2024-09-21T06:02:09.062 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:02:09.062 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:02:09.062 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-21T06:02:09.063 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-09-21T06:02:09.063 INFO:teuthology.orchestra.run.smithi003.stdout:Install 149 Packages 2024-09-21T06:02:09.063 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 8 Packages 2024-09-21T06:02:09.063 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:02:09.064 INFO:teuthology.orchestra.run.smithi137.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.8 MB/s | 873 kB 00:00 2024-09-21T06:02:09.064 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 228 M 2024-09-21T06:02:09.065 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-21T06:02:09.123 INFO:teuthology.orchestra.run.smithi137.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.2 MB/s | 148 kB 00:00 2024-09-21T06:02:09.209 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-21T06:02:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 8.2 MB/s | 5.5 MB 00:00 2024-09-21T06:02:09.424 INFO:teuthology.orchestra.run.smithi137.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 7.6 MB/s | 2.3 MB 00:00 2024-09-21T06:02:09.491 INFO:teuthology.orchestra.run.smithi137.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 13 MB/s | 1.5 MB 00:00 2024-09-21T06:02:09.600 INFO:teuthology.orchestra.run.smithi003.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 46 kB/s | 6.6 kB 00:00 2024-09-21T06:02:09.683 INFO:teuthology.orchestra.run.smithi137.stdout:(7/157): ceph-common-19.3.0-5120.g18432218.el9. 20 MB/s | 20 MB 00:00 2024-09-21T06:02:10.086 INFO:teuthology.orchestra.run.smithi137.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 7.5 MB/s | 5.0 MB 00:00 2024-09-21T06:02:10.125 INFO:teuthology.orchestra.run.smithi003.stdout:(2/157): ceph-base-19.3.0-5120.g18432218.el9.x8 8.3 MB/s | 5.5 MB 00:00 2024-09-21T06:02:10.142 INFO:teuthology.orchestra.run.smithi137.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 433 kB/s | 25 kB 00:00 2024-09-21T06:02:10.184 INFO:teuthology.orchestra.run.smithi003.stdout:(3/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.5 MB/s | 873 kB 00:00 2024-09-21T06:02:10.209 INFO:teuthology.orchestra.run.smithi003.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.7 MB/s | 148 kB 00:00 2024-09-21T06:02:10.359 INFO:teuthology.orchestra.run.smithi003.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 10 MB/s | 1.5 MB 00:00 2024-09-21T06:02:10.426 INFO:teuthology.orchestra.run.smithi137.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 14 MB/s | 10 MB 00:00 2024-09-21T06:02:10.526 INFO:teuthology.orchestra.run.smithi137.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 2.2 MB/s | 226 kB 00:00 2024-09-21T06:02:10.676 INFO:teuthology.orchestra.run.smithi003.stdout:(6/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 16 MB/s | 5.0 MB 00:00 2024-09-21T06:02:10.701 INFO:teuthology.orchestra.run.smithi137.stdout:(12/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 15 MB/s | 18 MB 00:01 2024-09-21T06:02:10.726 INFO:teuthology.orchestra.run.smithi137.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 135 kB/s | 27 kB 00:00 2024-09-21T06:02:10.768 INFO:teuthology.orchestra.run.smithi137.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 477 kB/s | 32 kB 00:00 2024-09-21T06:02:10.860 INFO:teuthology.orchestra.run.smithi137.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.7 MB/s | 158 kB 00:00 2024-09-21T06:02:10.885 INFO:teuthology.orchestra.run.smithi137.stdout:(16/157): libcephfs2-19.3.0-5120.g18432218.el9. 4.7 MB/s | 766 kB 00:00 2024-09-21T06:02:10.910 INFO:teuthology.orchestra.run.smithi003.stdout:(7/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 3.1 MB/s | 2.3 MB 00:00 2024-09-21T06:02:10.936 INFO:teuthology.orchestra.run.smithi137.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 1.6 MB/s | 127 kB 00:00 2024-09-21T06:02:10.969 INFO:teuthology.orchestra.run.smithi137.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 5.9 MB/s | 507 kB 00:00 2024-09-21T06:02:11.061 INFO:teuthology.orchestra.run.smithi137.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 492 kB/s | 45 kB 00:00 2024-09-21T06:02:11.161 INFO:teuthology.orchestra.run.smithi137.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.5 MB/s | 148 kB 00:00 2024-09-21T06:02:11.262 INFO:teuthology.orchestra.run.smithi137.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 1.6 MB/s | 161 kB 00:00 2024-09-21T06:02:11.345 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:02:11.345 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:02:11.356 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:02:11.357 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:02:11.358 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:02:11.359 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:02:11.361 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:02:11.363 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:02:11.364 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:02:11.395 INFO:teuthology.orchestra.run.smithi137.stdout:(22/157): librgw2-19.3.0-5120.g18432218.el9.x86 11 MB/s | 5.3 MB 00:00 2024-09-21T06:02:11.420 INFO:teuthology.orchestra.run.smithi137.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 2.0 MB/s | 321 kB 00:00 2024-09-21T06:02:11.427 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-21T06:02:11.446 INFO:teuthology.orchestra.run.smithi137.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T06:02:11.447 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-09-21T06:02:11.448 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-21T06:02:11.448 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-09-21T06:02:11.448 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-21T06:02:11.448 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:02:11.448 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:02:11.448 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:02:11.448 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:02:11.448 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:02:11.448 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:02:11.448 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:02:11.448 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:02:11.448 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:02:11.449 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:02:11.449 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:02:11.449 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:02:11.449 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:02:11.449 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:02:11.449 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:02:11.449 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:02:11.449 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:02:11.449 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:02:11.449 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:02:11.449 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:02:11.449 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:02:11.449 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:02:11.450 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:02:11.450 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:02:11.450 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:02:11.450 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:02:11.450 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:02:11.450 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:02:11.450 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:02:11.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:02:11.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:02:11.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:02:11.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:02:11.450 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:02:11.450 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:02:11.451 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:02:11.451 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:02:11.451 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-09-21T06:02:11.451 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:02:11.451 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:02:11.451 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:02:11.451 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:02:11.451 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:02:11.451 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:02:11.451 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:02:11.451 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:02:11.451 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-21T06:02:11.452 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:02:11.452 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:02:11.452 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:02:11.452 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:02:11.452 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:02:11.452 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:02:11.452 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:02:11.452 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:02:11.452 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:02:11.452 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:02:11.452 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:02:11.452 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:02:11.453 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:02:11.453 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:02:11.453 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:02:11.453 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:02:11.453 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:02:11.453 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:02:11.453 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:02:11.453 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:02:11.453 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:02:11.453 INFO:teuthology.orchestra.run.smithi204.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:02:11.454 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:02:11.454 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:02:11.454 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:02:11.454 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:02:11.454 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:02:11.454 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:02:11.454 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:02:11.454 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:02:11.454 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:02:11.454 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:02:11.454 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:02:11.454 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:02:11.455 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:02:11.455 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:02:11.455 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:02:11.455 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:02:11.455 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:02:11.455 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:02:11.455 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:02:11.455 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:02:11.455 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:02:11.455 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:02:11.455 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:02:11.456 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:02:11.457 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:02:11.457 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:02:11.457 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:02:11.457 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:02:11.457 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:02:11.457 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:02:11.457 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:02:11.457 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:02:11.457 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:02:11.457 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:02:11.457 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:02:11.457 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:02:11.457 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:02:11.458 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:02:11.458 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:02:11.458 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:02:11.458 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:02:11.458 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:02:11.458 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:02:11.458 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:02:11.458 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:02:11.458 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:02:11.458 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:02:11.458 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:02:11.459 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:02:11.459 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:02:11.459 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:02:11.459 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:02:11.459 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:02:11.459 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:02:11.459 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:02:11.459 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:02:11.459 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:02:11.459 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:02:11.460 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:02:11.460 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:02:11.460 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:02:11.460 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:02:11.460 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:02:11.460 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:02:11.460 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:02:11.460 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:02:11.460 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:02:11.460 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:02:11.461 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:02:11.461 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:02:11.461 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:02:11.461 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:02:11.461 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:02:11.461 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:02:11.461 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:02:11.461 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:02:11.461 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-09-21T06:02:11.461 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:02:11.461 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:02:11.462 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-21T06:02:11.462 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-09-21T06:02:11.462 INFO:teuthology.orchestra.run.smithi204.stdout:Install 149 Packages 2024-09-21T06:02:11.462 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 8 Packages 2024-09-21T06:02:11.462 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:02:11.462 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 228 M 2024-09-21T06:02:11.463 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-21T06:02:11.471 INFO:teuthology.orchestra.run.smithi137.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T06:02:11.505 INFO:teuthology.orchestra.run.smithi137.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.4 MB/s | 86 kB 00:00 2024-09-21T06:02:11.580 INFO:teuthology.orchestra.run.smithi137.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.2 MB/s | 170 kB 00:00 2024-09-21T06:02:11.639 INFO:teuthology.orchestra.run.smithi137.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 587 kB/s | 34 kB 00:00 2024-09-21T06:02:11.679 INFO:teuthology.orchestra.run.smithi003.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 9.0 MB/s | 20 MB 00:02 2024-09-21T06:02:11.697 INFO:teuthology.orchestra.run.smithi137.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 14 MB/s | 3.1 MB 00:00 2024-09-21T06:02:11.723 INFO:teuthology.orchestra.run.smithi137.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.8 MB/s | 154 kB 00:00 2024-09-21T06:02:11.726 INFO:teuthology.orchestra.run.smithi003.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 531 kB/s | 25 kB 00:00 2024-09-21T06:02:12.035 INFO:teuthology.orchestra.run.smithi003.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 13 MB/s | 18 MB 00:01 2024-09-21T06:02:12.136 INFO:teuthology.orchestra.run.smithi204.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-09-21T06:02:12.144 INFO:teuthology.orchestra.run.smithi003.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 2.0 MB/s | 226 kB 00:00 2024-09-21T06:02:12.252 INFO:teuthology.orchestra.run.smithi003.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 250 kB/s | 27 kB 00:00 2024-09-21T06:02:12.315 INFO:teuthology.orchestra.run.smithi137.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 12 MB/s | 7.4 MB 00:00 2024-09-21T06:02:12.327 INFO:teuthology.orchestra.run.smithi003.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 424 kB/s | 32 kB 00:00 2024-09-21T06:02:12.424 INFO:teuthology.orchestra.run.smithi137.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.6 MB/s | 292 kB 00:00 2024-09-21T06:02:12.524 INFO:teuthology.orchestra.run.smithi137.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 494 kB/s | 49 kB 00:00 2024-09-21T06:02:12.599 INFO:teuthology.orchestra.run.smithi137.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 219 kB/s | 16 kB 00:00 2024-09-21T06:02:12.667 INFO:teuthology.orchestra.run.smithi137.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 4.2 MB/s | 285 kB 00:00 2024-09-21T06:02:12.678 INFO:teuthology.orchestra.run.smithi003.stdout:(14/157): ceph-radosgw-19.3.0-5120.g18432218.el 5.9 MB/s | 10 MB 00:01 2024-09-21T06:02:12.703 INFO:teuthology.orchestra.run.smithi204.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.5 MB/s | 873 kB 00:00 2024-09-21T06:02:12.734 INFO:teuthology.orchestra.run.smithi137.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 12 MB/s | 784 kB 00:00 2024-09-21T06:02:12.778 INFO:teuthology.orchestra.run.smithi003.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.5 MB/s | 158 kB 00:00 2024-09-21T06:02:12.817 INFO:teuthology.orchestra.run.smithi137.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 221 kB/s | 18 kB 00:00 2024-09-21T06:02:12.828 INFO:teuthology.orchestra.run.smithi204.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.2 MB/s | 148 kB 00:00 2024-09-21T06:02:12.845 INFO:teuthology.orchestra.run.smithi003.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.9 MB/s | 127 kB 00:00 2024-09-21T06:02:12.945 INFO:teuthology.orchestra.run.smithi003.stdout:(17/157): libcephfs2-19.3.0-5120.g18432218.el9. 1.2 MB/s | 766 kB 00:00 2024-09-21T06:02:12.960 INFO:teuthology.orchestra.run.smithi137.stdout:(38/157): ceph-mgr-dashboard-19.3.0-5120.g18432 11 MB/s | 14 MB 00:01 2024-09-21T06:02:12.971 INFO:teuthology.orchestra.run.smithi003.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 4.0 MB/s | 507 kB 00:00 2024-09-21T06:02:13.046 INFO:teuthology.orchestra.run.smithi003.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 602 kB/s | 45 kB 00:00 2024-09-21T06:02:13.051 INFO:teuthology.orchestra.run.smithi137.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 341 kB/s | 80 kB 00:00 2024-09-21T06:02:13.121 INFO:teuthology.orchestra.run.smithi003.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.9 MB/s | 148 kB 00:00 2024-09-21T06:02:13.126 INFO:teuthology.orchestra.run.smithi137.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.7 MB/s | 801 kB 00:00 2024-09-21T06:02:13.160 INFO:teuthology.orchestra.run.smithi137.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-21T06:02:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-21T06:02:13.188 INFO:teuthology.orchestra.run.smithi003.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.4 MB/s | 161 kB 00:00 2024-09-21T06:02:13.236 INFO:teuthology.orchestra.run.smithi137.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-09-21T06:02:13.261 INFO:teuthology.orchestra.run.smithi137.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-21T06:02:13.272 INFO:teuthology.orchestra.run.smithi003.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 3.8 MB/s | 321 kB 00:00 2024-09-21T06:02:13.286 INFO:teuthology.orchestra.run.smithi137.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T06:02:13.312 INFO:teuthology.orchestra.run.smithi137.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T06:02:13.337 INFO:teuthology.orchestra.run.smithi137.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-21T06:02:13.356 INFO:teuthology.orchestra.run.smithi003.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 3.6 MB/s | 307 kB 00:00 2024-09-21T06:02:13.464 INFO:teuthology.orchestra.run.smithi003.stdout:(24/157): librgw2-19.3.0-5120.g18432218.el9.x86 10 MB/s | 5.3 MB 00:00 2024-09-21T06:02:13.498 INFO:teuthology.orchestra.run.smithi003.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 701 kB/s | 99 kB 00:00 2024-09-21T06:02:13.520 INFO:teuthology.orchestra.run.smithi137.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-21T06:02:13.531 INFO:teuthology.orchestra.run.smithi003.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.3 MB/s | 86 kB 00:00 2024-09-21T06:02:13.598 INFO:teuthology.orchestra.run.smithi003.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.5 MB/s | 170 kB 00:00 2024-09-21T06:02:13.612 INFO:teuthology.orchestra.run.smithi204.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 2.9 MB/s | 2.3 MB 00:00 2024-09-21T06:02:13.657 INFO:teuthology.orchestra.run.smithi003.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 588 kB/s | 34 kB 00:00 2024-09-21T06:02:13.724 INFO:teuthology.orchestra.run.smithi003.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.3 MB/s | 154 kB 00:00 2024-09-21T06:02:13.879 INFO:teuthology.orchestra.run.smithi137.stdout:(49/157): ceph-test-19.3.0-5120.g18432218.el9.x 11 MB/s | 40 MB 00:03 2024-09-21T06:02:13.887 INFO:teuthology.orchestra.run.smithi204.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 2.7 MB/s | 5.5 MB 00:02 2024-09-21T06:02:13.913 INFO:teuthology.orchestra.run.smithi137.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 273 kB/s | 107 kB 00:00 2024-09-21T06:02:13.946 INFO:teuthology.orchestra.run.smithi137.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 1.5 MB/s | 943 kB 00:00 2024-09-21T06:02:13.971 INFO:teuthology.orchestra.run.smithi137.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-21T06:02:13.988 INFO:teuthology.orchestra.run.smithi137.stdout:(53/157): elfutils-libelf-devel-0.191-4.el9.x86 199 kB/s | 22 kB 00:00 2024-09-21T06:02:14.005 INFO:teuthology.orchestra.run.smithi137.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 557 kB/s | 33 kB 00:00 2024-09-21T06:02:14.020 INFO:teuthology.orchestra.run.smithi204.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 3.7 MB/s | 1.5 MB 00:00 2024-09-21T06:02:14.031 INFO:teuthology.orchestra.run.smithi137.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 435 kB/s | 18 kB 00:00 2024-09-21T06:02:14.048 INFO:teuthology.orchestra.run.smithi137.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 265 kB/s | 11 kB 00:00 2024-09-21T06:02:14.081 INFO:teuthology.orchestra.run.smithi137.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T06:02:14.149 INFO:teuthology.orchestra.run.smithi003.stdout:(30/157): ceph-test-19.3.0-5120.g18432218.el9.x 16 MB/s | 40 MB 00:02 2024-09-21T06:02:14.157 INFO:teuthology.orchestra.run.smithi137.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-09-21T06:02:14.182 INFO:teuthology.orchestra.run.smithi137.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-09-21T06:02:14.199 INFO:teuthology.orchestra.run.smithi003.stdout:(31/157): rbd-mirror-19.3.0-5120.g18432218.el9. 4.4 MB/s | 3.1 MB 00:00 2024-09-21T06:02:14.207 INFO:teuthology.orchestra.run.smithi137.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-21T06:02:14.249 INFO:teuthology.orchestra.run.smithi137.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-09-21T06:02:14.274 INFO:teuthology.orchestra.run.smithi137.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 367 kB/s | 24 kB 00:00 2024-09-21T06:02:14.299 INFO:teuthology.orchestra.run.smithi137.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-21T06:02:14.325 INFO:teuthology.orchestra.run.smithi003.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.3 MB/s | 292 kB 00:00 2024-09-21T06:02:14.333 INFO:teuthology.orchestra.run.smithi137.stdout:(64/157): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-09-21T06:02:14.359 INFO:teuthology.orchestra.run.smithi137.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-21T06:02:14.384 INFO:teuthology.orchestra.run.smithi137.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 765 kB/s | 38 kB 00:00 2024-09-21T06:02:14.391 INFO:teuthology.orchestra.run.smithi003.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 743 kB/s | 49 kB 00:00 2024-09-21T06:02:14.409 INFO:teuthology.orchestra.run.smithi137.stdout:(67/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-21T06:02:14.459 INFO:teuthology.orchestra.run.smithi003.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 247 kB/s | 16 kB 00:00 2024-09-21T06:02:14.510 INFO:teuthology.orchestra.run.smithi137.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 33 MB/s | 4.9 MB 00:00 2024-09-21T06:02:14.534 INFO:teuthology.orchestra.run.smithi003.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 3.7 MB/s | 285 kB 00:00 2024-09-21T06:02:14.577 INFO:teuthology.orchestra.run.smithi137.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-09-21T06:02:14.618 INFO:teuthology.orchestra.run.smithi137.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-09-21T06:02:14.626 INFO:teuthology.orchestra.run.smithi003.stdout:(36/157): ceph-mgr-diskprediction-local-19.3.0- 15 MB/s | 7.4 MB 00:00 2024-09-21T06:02:14.644 INFO:teuthology.orchestra.run.smithi137.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T06:02:14.668 INFO:teuthology.orchestra.run.smithi003.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 5.7 MB/s | 784 kB 00:00 2024-09-21T06:02:14.671 INFO:teuthology.orchestra.run.smithi204.stdout:(7/157): ceph-common-19.3.0-5120.g18432218.el9. 7.0 MB/s | 20 MB 00:02 2024-09-21T06:02:14.677 INFO:teuthology.orchestra.run.smithi137.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-21T06:02:14.693 INFO:teuthology.orchestra.run.smithi003.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 276 kB/s | 18 kB 00:00 2024-09-21T06:02:14.719 INFO:teuthology.orchestra.run.smithi137.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-21T06:02:14.753 INFO:teuthology.orchestra.run.smithi137.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-21T06:02:14.786 INFO:teuthology.orchestra.run.smithi137.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-21T06:02:14.793 INFO:teuthology.orchestra.run.smithi003.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 637 kB/s | 80 kB 00:00 2024-09-21T06:02:14.870 INFO:teuthology.orchestra.run.smithi137.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-21T06:02:14.895 INFO:teuthology.orchestra.run.smithi137.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-21T06:02:14.921 INFO:teuthology.orchestra.run.smithi137.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T06:02:14.946 INFO:teuthology.orchestra.run.smithi137.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-21T06:02:14.952 INFO:teuthology.orchestra.run.smithi003.stdout:(40/157): ceph-mgr-dashboard-19.3.0-5120.g18432 12 MB/s | 14 MB 00:01 2024-09-21T06:02:14.971 INFO:teuthology.orchestra.run.smithi137.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-21T06:02:14.977 INFO:teuthology.orchestra.run.smithi003.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-21T06:02:14.997 INFO:teuthology.orchestra.run.smithi137.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-21T06:02:15.002 INFO:teuthology.orchestra.run.smithi003.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-21T06:02:15.004 INFO:teuthology.orchestra.run.smithi204.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 4.4 MB/s | 5.0 MB 00:01 2024-09-21T06:02:15.022 INFO:teuthology.orchestra.run.smithi137.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-21T06:02:15.094 INFO:teuthology.orchestra.run.smithi003.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-21T06:02:15.106 INFO:teuthology.orchestra.run.smithi137.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-21T06:02:15.131 INFO:teuthology.orchestra.run.smithi137.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-21T06:02:15.138 INFO:teuthology.orchestra.run.smithi204.stdout:(9/157): ceph-radosgw-19.3.0-5120.g18432218.el9 22 MB/s | 10 MB 00:00 2024-09-21T06:02:15.145 INFO:teuthology.orchestra.run.smithi003.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 749 kB/s | 106 kB 00:00 2024-09-21T06:02:15.156 INFO:teuthology.orchestra.run.smithi137.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-21T06:02:15.163 INFO:teuthology.orchestra.run.smithi204.stdout:(10/157): ceph-selinux-19.3.0-5120.g18432218.el 159 kB/s | 25 kB 00:00 2024-09-21T06:02:15.178 INFO:teuthology.orchestra.run.smithi003.stdout:(45/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-21T06:02:15.181 INFO:teuthology.orchestra.run.smithi137.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T06:02:15.204 INFO:teuthology.orchestra.run.smithi003.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-21T06:02:15.206 INFO:teuthology.orchestra.run.smithi137.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-21T06:02:15.229 INFO:teuthology.orchestra.run.smithi003.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-21T06:02:15.240 INFO:teuthology.orchestra.run.smithi137.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-21T06:02:15.262 INFO:teuthology.orchestra.run.smithi003.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-21T06:02:15.263 INFO:teuthology.orchestra.run.smithi204.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 2.2 MB/s | 226 kB 00:00 2024-09-21T06:02:15.265 INFO:teuthology.orchestra.run.smithi137.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-21T06:02:15.291 INFO:teuthology.orchestra.run.smithi137.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T06:02:15.314 INFO:teuthology.orchestra.run.smithi204.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 540 kB/s | 27 kB 00:00 2024-09-21T06:02:15.316 INFO:teuthology.orchestra.run.smithi137.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T06:02:15.341 INFO:teuthology.orchestra.run.smithi137.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T06:02:15.364 INFO:teuthology.orchestra.run.smithi204.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 639 kB/s | 32 kB 00:00 2024-09-21T06:02:15.380 INFO:teuthology.orchestra.run.smithi003.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-21T06:02:15.405 INFO:teuthology.orchestra.run.smithi003.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 753 kB/s | 107 kB 00:00 2024-09-21T06:02:15.417 INFO:teuthology.orchestra.run.smithi137.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T06:02:15.430 INFO:teuthology.orchestra.run.smithi003.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-21T06:02:15.450 INFO:teuthology.orchestra.run.smithi137.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-21T06:02:15.472 INFO:teuthology.orchestra.run.smithi003.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-09-21T06:02:15.476 INFO:teuthology.orchestra.run.smithi137.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 755 kB/s | 44 kB 00:00 2024-09-21T06:02:15.497 INFO:teuthology.orchestra.run.smithi003.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-09-21T06:02:15.501 INFO:teuthology.orchestra.run.smithi137.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T06:02:15.522 INFO:teuthology.orchestra.run.smithi003.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-21T06:02:15.526 INFO:teuthology.orchestra.run.smithi137.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-21T06:02:15.548 INFO:teuthology.orchestra.run.smithi003.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-21T06:02:15.551 INFO:teuthology.orchestra.run.smithi137.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-21T06:02:15.573 INFO:teuthology.orchestra.run.smithi003.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-21T06:02:15.577 INFO:teuthology.orchestra.run.smithi137.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T06:02:15.581 INFO:teuthology.orchestra.run.smithi204.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 3.5 MB/s | 766 kB 00:00 2024-09-21T06:02:15.602 INFO:teuthology.orchestra.run.smithi137.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T06:02:15.607 INFO:teuthology.orchestra.run.smithi003.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.9 MB/s | 171 kB 00:00 2024-09-21T06:02:15.627 INFO:teuthology.orchestra.run.smithi137.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T06:02:15.632 INFO:teuthology.orchestra.run.smithi003.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-21T06:02:15.653 INFO:teuthology.orchestra.run.smithi137.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T06:02:15.656 INFO:teuthology.orchestra.run.smithi204.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.1 MB/s | 158 kB 00:00 2024-09-21T06:02:15.658 INFO:teuthology.orchestra.run.smithi003.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-21T06:02:15.678 INFO:teuthology.orchestra.run.smithi137.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-21T06:02:15.724 INFO:teuthology.orchestra.run.smithi003.stdout:(60/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-21T06:02:15.731 INFO:teuthology.orchestra.run.smithi204.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.7 MB/s | 127 kB 00:00 2024-09-21T06:02:15.749 INFO:teuthology.orchestra.run.smithi003.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-21T06:02:15.775 INFO:teuthology.orchestra.run.smithi003.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-21T06:02:15.800 INFO:teuthology.orchestra.run.smithi003.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-21T06:02:15.825 INFO:teuthology.orchestra.run.smithi003.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-21T06:02:15.859 INFO:teuthology.orchestra.run.smithi003.stdout:(65/157): libtool-2.4.6-46.el9.x86_64.rpm 2.8 MB/s | 581 kB 00:00 2024-09-21T06:02:15.884 INFO:teuthology.orchestra.run.smithi003.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 659 kB/s | 38 kB 00:00 2024-09-21T06:02:15.909 INFO:teuthology.orchestra.run.smithi003.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-21T06:02:15.923 INFO:teuthology.orchestra.run.smithi204.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 2.6 MB/s | 507 kB 00:00 2024-09-21T06:02:15.953 INFO:teuthology.orchestra.run.smithi137.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-21T06:02:15.978 INFO:teuthology.orchestra.run.smithi137.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 68 kB/s | 22 kB 00:00 2024-09-21T06:02:15.993 INFO:teuthology.orchestra.run.smithi003.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-21T06:02:16.029 INFO:teuthology.orchestra.run.smithi137.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 6.0 MB/s | 308 kB 00:00 2024-09-21T06:02:16.035 INFO:teuthology.orchestra.run.smithi003.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-21T06:02:16.054 INFO:teuthology.orchestra.run.smithi137.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 843 kB/s | 19 kB 00:00 2024-09-21T06:02:16.068 INFO:teuthology.orchestra.run.smithi003.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-21T06:02:16.094 INFO:teuthology.orchestra.run.smithi003.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T06:02:16.096 INFO:teuthology.orchestra.run.smithi137.stdout:(108/157): abseil-cpp-20211102.0-3.el9.x86_64.r 3.8 MB/s | 548 kB 00:00 2024-09-21T06:02:16.121 INFO:teuthology.orchestra.run.smithi137.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T06:02:16.146 INFO:teuthology.orchestra.run.smithi137.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T06:02:16.161 INFO:teuthology.orchestra.run.smithi003.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 16 MB/s | 4.9 MB 00:00 2024-09-21T06:02:16.171 INFO:teuthology.orchestra.run.smithi137.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-21T06:02:16.197 INFO:teuthology.orchestra.run.smithi137.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-21T06:02:16.203 INFO:teuthology.orchestra.run.smithi003.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-21T06:02:16.228 INFO:teuthology.orchestra.run.smithi003.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-21T06:02:16.255 INFO:teuthology.orchestra.run.smithi137.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-21T06:02:16.278 INFO:teuthology.orchestra.run.smithi003.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T06:02:16.298 INFO:teuthology.orchestra.run.smithi137.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-21T06:02:16.315 INFO:teuthology.orchestra.run.smithi204.stdout:(18/157): ceph-test-19.3.0-5120.g18432218.el9.x 34 MB/s | 40 MB 00:01 2024-09-21T06:02:16.323 INFO:teuthology.orchestra.run.smithi137.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T06:02:16.348 INFO:teuthology.orchestra.run.smithi137.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T06:02:16.354 INFO:teuthology.orchestra.run.smithi003.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-09-21T06:02:16.366 INFO:teuthology.orchestra.run.smithi204.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 907 kB/s | 45 kB 00:00 2024-09-21T06:02:16.373 INFO:teuthology.orchestra.run.smithi137.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-09-21T06:02:16.379 INFO:teuthology.orchestra.run.smithi003.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-21T06:02:16.398 INFO:teuthology.orchestra.run.smithi137.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-21T06:02:16.404 INFO:teuthology.orchestra.run.smithi003.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T06:02:16.416 INFO:teuthology.orchestra.run.smithi204.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.9 MB/s | 148 kB 00:00 2024-09-21T06:02:16.429 INFO:teuthology.orchestra.run.smithi003.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-21T06:02:16.455 INFO:teuthology.orchestra.run.smithi003.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-21T06:02:16.465 INFO:teuthology.orchestra.run.smithi137.stdout:(119/157): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-21T06:02:16.466 INFO:teuthology.orchestra.run.smithi204.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 3.2 MB/s | 161 kB 00:00 2024-09-21T06:02:16.480 INFO:teuthology.orchestra.run.smithi003.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-21T06:02:16.499 INFO:teuthology.orchestra.run.smithi137.stdout:(120/157): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-09-21T06:02:16.505 INFO:teuthology.orchestra.run.smithi003.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-09-21T06:02:16.525 INFO:teuthology.orchestra.run.smithi204.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 5.4 MB/s | 321 kB 00:00 2024-09-21T06:02:16.533 INFO:teuthology.orchestra.run.smithi137.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-09-21T06:02:16.567 INFO:teuthology.orchestra.run.smithi137.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-09-21T06:02:16.575 INFO:teuthology.orchestra.run.smithi204.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T06:02:16.600 INFO:teuthology.orchestra.run.smithi137.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-21T06:02:16.626 INFO:teuthology.orchestra.run.smithi204.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T06:02:16.634 INFO:teuthology.orchestra.run.smithi137.stdout:(124/157): python3-google-auth-2.34.0-1.el9.noa 1.6 MB/s | 220 kB 00:00 2024-09-21T06:02:16.659 INFO:teuthology.orchestra.run.smithi137.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-09-21T06:02:16.661 INFO:teuthology.orchestra.run.smithi003.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-21T06:02:16.684 INFO:teuthology.orchestra.run.smithi204.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.5 MB/s | 86 kB 00:00 2024-09-21T06:02:16.685 INFO:teuthology.orchestra.run.smithi137.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-21T06:02:16.689 INFO:teuthology.orchestra.run.smithi003.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-09-21T06:02:16.710 INFO:teuthology.orchestra.run.smithi137.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-21T06:02:16.714 INFO:teuthology.orchestra.run.smithi003.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.1 MB/s | 222 kB 00:00 2024-09-21T06:02:16.735 INFO:teuthology.orchestra.run.smithi137.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-21T06:02:16.739 INFO:teuthology.orchestra.run.smithi003.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T06:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-21T06:02:16.765 INFO:teuthology.orchestra.run.smithi003.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-21T06:02:16.786 INFO:teuthology.orchestra.run.smithi137.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T06:02:16.790 INFO:teuthology.orchestra.run.smithi003.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-21T06:02:16.811 INFO:teuthology.orchestra.run.smithi137.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T06:02:16.815 INFO:teuthology.orchestra.run.smithi003.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T06:02:16.818 INFO:teuthology.orchestra.run.smithi204.stdout:(26/157): rbd-mirror-19.3.0-5120.g18432218.el9. 23 MB/s | 3.1 MB 00:00 2024-09-21T06:02:16.836 INFO:teuthology.orchestra.run.smithi137.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T06:02:16.841 INFO:teuthology.orchestra.run.smithi003.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T06:02:16.866 INFO:teuthology.orchestra.run.smithi003.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T06:02:16.871 INFO:teuthology.orchestra.run.smithi204.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.2 MB/s | 170 kB 00:00 2024-09-21T06:02:16.878 INFO:teuthology.orchestra.run.smithi137.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-21T06:02:16.891 INFO:teuthology.orchestra.run.smithi003.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T06:02:16.903 INFO:teuthology.orchestra.run.smithi137.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-21T06:02:16.921 INFO:teuthology.orchestra.run.smithi204.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 688 kB/s | 34 kB 00:00 2024-09-21T06:02:16.928 INFO:teuthology.orchestra.run.smithi137.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-21T06:02:16.967 INFO:teuthology.orchestra.run.smithi003.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-09-21T06:02:16.970 INFO:teuthology.orchestra.run.smithi137.stdout:(136/157): python3-kubernetes-26.1.0-2.el9.noar 4.3 MB/s | 1.0 MB 00:00 2024-09-21T06:02:16.972 INFO:teuthology.orchestra.run.smithi204.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T06:02:16.992 INFO:teuthology.orchestra.run.smithi003.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-21T06:02:16.996 INFO:teuthology.orchestra.run.smithi137.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-09-21T06:02:17.017 INFO:teuthology.orchestra.run.smithi003.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-09-21T06:02:17.021 INFO:teuthology.orchestra.run.smithi137.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-21T06:02:17.043 INFO:teuthology.orchestra.run.smithi003.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T06:02:17.046 INFO:teuthology.orchestra.run.smithi137.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T06:02:17.047 INFO:teuthology.orchestra.run.smithi204.stdout:(30/157): librgw2-19.3.0-5120.g18432218.el9.x86 4.7 MB/s | 5.3 MB 00:01 2024-09-21T06:02:17.059 INFO:teuthology.orchestra.run.smithi003.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 861 kB/s | 36 kB 00:00 2024-09-21T06:02:17.076 INFO:teuthology.orchestra.run.smithi003.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-21T06:02:17.096 INFO:teuthology.orchestra.run.smithi137.stdout:(140/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-09-21T06:02:17.110 INFO:teuthology.orchestra.run.smithi003.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T06:02:17.122 INFO:teuthology.orchestra.run.smithi137.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-09-21T06:02:17.126 INFO:teuthology.orchestra.run.smithi003.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T06:02:17.147 INFO:teuthology.orchestra.run.smithi137.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-21T06:02:17.152 INFO:teuthology.orchestra.run.smithi003.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-21T06:02:17.169 INFO:teuthology.orchestra.run.smithi003.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-21T06:02:17.172 INFO:teuthology.orchestra.run.smithi137.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-21T06:02:17.186 INFO:teuthology.orchestra.run.smithi003.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-21T06:02:17.198 INFO:teuthology.orchestra.run.smithi137.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T06:02:17.223 INFO:teuthology.orchestra.run.smithi137.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-21T06:02:17.228 INFO:teuthology.orchestra.run.smithi003.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 383 kB/s | 22 kB 00:00 2024-09-21T06:02:17.248 INFO:teuthology.orchestra.run.smithi137.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-21T06:02:17.274 INFO:teuthology.orchestra.run.smithi137.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T06:02:17.287 INFO:teuthology.orchestra.run.smithi003.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T06:02:17.299 INFO:teuthology.orchestra.run.smithi137.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T06:02:17.329 INFO:teuthology.orchestra.run.smithi003.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 5.9 MB/s | 863 kB 00:00 2024-09-21T06:02:17.349 INFO:teuthology.orchestra.run.smithi137.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T06:02:17.362 INFO:teuthology.orchestra.run.smithi003.stdout:(107/157): gperftools-libs-2.9.1-3.el9.x86_64.r 4.0 MB/s | 308 kB 00:00 2024-09-21T06:02:17.387 INFO:teuthology.orchestra.run.smithi003.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-09-21T06:02:17.412 INFO:teuthology.orchestra.run.smithi003.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T06:02:17.438 INFO:teuthology.orchestra.run.smithi003.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T06:02:17.463 INFO:teuthology.orchestra.run.smithi003.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T06:02:17.473 INFO:teuthology.orchestra.run.smithi204.stdout:(31/157): ceph-mgr-dashboard-19.3.0-5120.g18432 28 MB/s | 14 MB 00:00 2024-09-21T06:02:17.531 INFO:teuthology.orchestra.run.smithi204.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.9 MB/s | 292 kB 00:00 2024-09-21T06:02:17.541 INFO:teuthology.orchestra.run.smithi137.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 13 MB/s | 3.4 MB 00:00 2024-09-21T06:02:17.573 INFO:teuthology.orchestra.run.smithi204.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 1.2 MB/s | 49 kB 00:00 2024-09-21T06:02:17.583 INFO:teuthology.orchestra.run.smithi137.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 9.9 MB/s | 2.3 MB 00:00 2024-09-21T06:02:17.615 INFO:teuthology.orchestra.run.smithi204.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 395 kB/s | 16 kB 00:00 2024-09-21T06:02:17.625 INFO:teuthology.orchestra.run.smithi137.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 985 kB/s | 41 kB 00:00 2024-09-21T06:02:17.630 INFO:teuthology.orchestra.run.smithi003.stdout:(112/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-09-21T06:02:17.664 INFO:teuthology.orchestra.run.smithi003.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 756 kB/s | 151 kB 00:00 2024-09-21T06:02:17.665 INFO:teuthology.orchestra.run.smithi204.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T06:02:17.684 INFO:teuthology.orchestra.run.smithi137.stdout:(153/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.2 MB/s | 3.2 MB 00:00 2024-09-21T06:02:17.714 INFO:teuthology.orchestra.run.smithi003.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-21T06:02:17.717 INFO:teuthology.orchestra.run.smithi137.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.6 MB/s | 249 kB 00:00 2024-09-21T06:02:17.724 INFO:teuthology.orchestra.run.smithi204.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 13 MB/s | 784 kB 00:00 2024-09-21T06:02:17.739 INFO:teuthology.orchestra.run.smithi003.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T06:02:17.765 INFO:teuthology.orchestra.run.smithi003.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T06:02:17.774 INFO:teuthology.orchestra.run.smithi204.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 368 kB/s | 18 kB 00:00 2024-09-21T06:02:17.790 INFO:teuthology.orchestra.run.smithi003.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 570 kB/s | 14 kB 00:00 2024-09-21T06:02:17.793 INFO:teuthology.orchestra.run.smithi137.stdout:(155/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-09-21T06:02:17.832 INFO:teuthology.orchestra.run.smithi003.stdout:(118/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 839 kB 00:00 2024-09-21T06:02:17.843 INFO:teuthology.orchestra.run.smithi137.stdout:(156/157): NetworkManager-libnm-1.51.0-1.el9.x8 6.0 MB/s | 1.8 MB 00:00 2024-09-21T06:02:17.865 INFO:teuthology.orchestra.run.smithi003.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-09-21T06:02:17.899 INFO:teuthology.orchestra.run.smithi003.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-21T06:02:17.924 INFO:teuthology.orchestra.run.smithi003.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T06:02:17.949 INFO:teuthology.orchestra.run.smithi003.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T06:02:17.950 INFO:teuthology.orchestra.run.smithi204.stdout:(38/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 4.5 MB/s | 18 MB 00:03 2024-09-21T06:02:17.959 INFO:teuthology.orchestra.run.smithi137.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 4.9 MB/s | 1.4 MB 00:00 2024-09-21T06:02:17.966 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:02:17.966 INFO:teuthology.orchestra.run.smithi137.stdout:Total 23 MB/s | 228 MB 00:09 2024-09-21T06:02:18.008 INFO:teuthology.orchestra.run.smithi003.stdout:(123/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-09-21T06:02:18.033 INFO:teuthology.orchestra.run.smithi003.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-21T06:02:18.059 INFO:teuthology.orchestra.run.smithi003.stdout:(125/157): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-09-21T06:02:18.075 INFO:teuthology.orchestra.run.smithi204.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 266 kB/s | 80 kB 00:00 2024-09-21T06:02:18.084 INFO:teuthology.orchestra.run.smithi003.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-21T06:02:18.109 INFO:teuthology.orchestra.run.smithi003.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-21T06:02:18.134 INFO:teuthology.orchestra.run.smithi003.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-21T06:02:18.160 INFO:teuthology.orchestra.run.smithi003.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-21T06:02:18.183 INFO:teuthology.orchestra.run.smithi204.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-21T06:02:18.185 INFO:teuthology.orchestra.run.smithi003.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T06:02:18.209 INFO:teuthology.orchestra.run.smithi204.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.0 MB/s | 801 kB 00:00 2024-09-21T06:02:18.210 INFO:teuthology.orchestra.run.smithi003.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T06:02:18.235 INFO:teuthology.orchestra.run.smithi003.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T06:02:18.251 INFO:teuthology.orchestra.run.smithi204.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-21T06:02:18.277 INFO:teuthology.orchestra.run.smithi003.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-21T06:02:18.293 INFO:teuthology.orchestra.run.smithi204.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-09-21T06:02:18.303 INFO:teuthology.orchestra.run.smithi003.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-21T06:02:18.318 INFO:teuthology.orchestra.run.smithi204.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-21T06:02:18.337 INFO:teuthology.orchestra.run.smithi003.stdout:(135/157): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-09-21T06:02:18.343 INFO:teuthology.orchestra.run.smithi204.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T06:02:18.362 INFO:teuthology.orchestra.run.smithi003.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T06:02:18.387 INFO:teuthology.orchestra.run.smithi003.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-21T06:02:18.412 INFO:teuthology.orchestra.run.smithi003.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-21T06:02:18.427 INFO:teuthology.orchestra.run.smithi204.stdout:(46/157): ceph-mgr-diskprediction-local-19.3.0- 5.3 MB/s | 7.4 MB 00:01 2024-09-21T06:02:18.438 INFO:teuthology.orchestra.run.smithi003.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T06:02:18.453 INFO:teuthology.orchestra.run.smithi204.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 931 kB/s | 125 kB 00:00 2024-09-21T06:02:18.463 INFO:teuthology.orchestra.run.smithi003.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-21T06:02:18.478 INFO:teuthology.orchestra.run.smithi204.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-09-21T06:02:18.480 INFO:teuthology.orchestra.run.smithi003.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 856 kB/s | 36 kB 00:00 2024-09-21T06:02:18.505 INFO:teuthology.orchestra.run.smithi003.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-21T06:02:18.547 INFO:teuthology.orchestra.run.smithi003.stdout:(143/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-09-21T06:02:18.573 INFO:teuthology.orchestra.run.smithi003.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-21T06:02:18.606 INFO:teuthology.orchestra.run.smithi003.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-21T06:02:18.631 INFO:teuthology.orchestra.run.smithi003.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-09-21T06:02:18.645 INFO:teuthology.orchestra.run.smithi204.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 641 kB/s | 107 kB 00:00 2024-09-21T06:02:18.665 INFO:teuthology.orchestra.run.smithi003.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-21T06:02:18.698 INFO:teuthology.orchestra.run.smithi003.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-21T06:02:18.703 INFO:teuthology.orchestra.run.smithi204.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-21T06:02:18.737 INFO:teuthology.orchestra.run.smithi204.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-21T06:02:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-21T06:02:18.771 INFO:teuthology.orchestra.run.smithi204.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-09-21T06:02:18.796 INFO:teuthology.orchestra.run.smithi204.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-21T06:02:18.847 INFO:teuthology.orchestra.run.smithi204.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-21T06:02:18.847 INFO:teuthology.orchestra.run.smithi204.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-21T06:02:18.857 INFO:teuthology.orchestra.run.smithi003.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 21 MB/s | 2.3 MB 00:00 2024-09-21T06:02:18.872 INFO:teuthology.orchestra.run.smithi204.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-21T06:02:18.897 INFO:teuthology.orchestra.run.smithi204.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T06:02:18.922 INFO:teuthology.orchestra.run.smithi204.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-21T06:02:18.948 INFO:teuthology.orchestra.run.smithi204.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-09-21T06:02:18.958 INFO:teuthology.orchestra.run.smithi003.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 18 MB/s | 1.8 MB 00:00 2024-09-21T06:02:19.015 INFO:teuthology.orchestra.run.smithi204.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-09-21T06:02:19.016 INFO:teuthology.orchestra.run.smithi003.stdout:(152/157): librados2-19.3.0-5120.g18432218.el9. 9.7 MB/s | 3.4 MB 00:00 2024-09-21T06:02:19.042 INFO:teuthology.orchestra.run.smithi003.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 491 kB/s | 41 kB 00:00 2024-09-21T06:02:19.048 INFO:teuthology.orchestra.run.smithi204.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 5.7 MB/s | 581 kB 00:00 2024-09-21T06:02:19.065 INFO:teuthology.orchestra.run.smithi204.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 487 kB/s | 24 kB 00:00 2024-09-21T06:02:19.091 INFO:teuthology.orchestra.run.smithi204.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-21T06:02:19.092 INFO:teuthology.orchestra.run.smithi003.stdout:(154/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.1 MB/s | 3.2 MB 00:00 2024-09-21T06:02:19.124 INFO:teuthology.orchestra.run.smithi204.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-21T06:02:19.125 INFO:teuthology.orchestra.run.smithi003.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.3 MB/s | 249 kB 00:00 2024-09-21T06:02:19.167 INFO:teuthology.orchestra.run.smithi003.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-21T06:02:19.183 INFO:teuthology.orchestra.run.smithi204.stdout:(65/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.3 MB/s | 3.0 MB 00:00 2024-09-21T06:02:19.200 INFO:teuthology.orchestra.run.smithi204.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 512 kB/s | 38 kB 00:00 2024-09-21T06:02:19.216 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-09-21T06:02:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-21T06:02:19.233 INFO:teuthology.orchestra.run.smithi003.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-09-21T06:02:19.240 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:02:19.240 INFO:teuthology.orchestra.run.smithi003.stdout:Total 22 MB/s | 228 MB 00:10 2024-09-21T06:02:19.370 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-09-21T06:02:19.370 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-09-21T06:02:19.384 INFO:teuthology.orchestra.run.smithi204.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-21T06:02:19.484 INFO:teuthology.orchestra.run.smithi204.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-21T06:02:19.543 INFO:teuthology.orchestra.run.smithi204.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T06:02:19.610 INFO:teuthology.orchestra.run.smithi204.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-21T06:02:19.710 INFO:teuthology.orchestra.run.smithi204.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-21T06:02:19.777 INFO:teuthology.orchestra.run.smithi204.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-21T06:02:19.869 INFO:teuthology.orchestra.run.smithi204.stdout:(74/157): openblas-openmp-0.3.26-2.el9.x86_64.r 7.2 MB/s | 4.9 MB 00:00 2024-09-21T06:02:19.919 INFO:teuthology.orchestra.run.smithi204.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T06:02:20.036 INFO:teuthology.orchestra.run.smithi204.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-09-21T06:02:20.070 INFO:teuthology.orchestra.run.smithi204.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-09-21T06:02:20.095 INFO:teuthology.orchestra.run.smithi204.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-21T06:02:20.129 INFO:teuthology.orchestra.run.smithi204.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-21T06:02:20.154 INFO:teuthology.orchestra.run.smithi204.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 326 kB/s | 19 kB 00:00 2024-09-21T06:02:20.188 INFO:teuthology.orchestra.run.smithi204.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-21T06:02:20.213 INFO:teuthology.orchestra.run.smithi204.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-09-21T06:02:20.280 INFO:teuthology.orchestra.run.smithi204.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-21T06:02:20.305 INFO:teuthology.orchestra.run.smithi204.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-21T06:02:20.347 INFO:teuthology.orchestra.run.smithi204.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-09-21T06:02:20.372 INFO:teuthology.orchestra.run.smithi204.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-21T06:02:20.398 INFO:teuthology.orchestra.run.smithi204.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-21T06:02:20.431 INFO:teuthology.orchestra.run.smithi204.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-21T06:02:20.465 INFO:teuthology.orchestra.run.smithi204.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-21T06:02:20.490 INFO:teuthology.orchestra.run.smithi204.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-21T06:02:20.493 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-21T06:02:20.515 INFO:teuthology.orchestra.run.smithi204.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T06:02:20.541 INFO:teuthology.orchestra.run.smithi204.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T06:02:20.591 INFO:teuthology.orchestra.run.smithi204.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-09-21T06:02:20.650 INFO:teuthology.orchestra.run.smithi204.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-09-21T06:02:20.662 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-21T06:02:20.662 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-21T06:02:20.725 INFO:teuthology.orchestra.run.smithi204.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-21T06:02:20.783 INFO:teuthology.orchestra.run.smithi204.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-09-21T06:02:20.842 INFO:teuthology.orchestra.run.smithi204.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-09-21T06:02:20.909 INFO:teuthology.orchestra.run.smithi204.stdout:(98/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.1 MB/s | 5.1 MB 00:00 2024-09-21T06:02:20.934 INFO:teuthology.orchestra.run.smithi204.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-09-21T06:02:20.968 INFO:teuthology.orchestra.run.smithi204.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-09-21T06:02:20.994 INFO:teuthology.orchestra.run.smithi204.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-21T06:02:21.027 INFO:teuthology.orchestra.run.smithi204.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-21T06:02:21.053 INFO:teuthology.orchestra.run.smithi204.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-09-21T06:02:21.180 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-09-21T06:02:21.181 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-09-21T06:02:21.278 INFO:teuthology.orchestra.run.smithi204.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-09-21T06:02:21.329 INFO:teuthology.orchestra.run.smithi204.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T06:02:21.362 INFO:teuthology.orchestra.run.smithi204.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-21T06:02:21.387 INFO:teuthology.orchestra.run.smithi204.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 789 kB/s | 19 kB 00:00 2024-09-21T06:02:21.563 INFO:teuthology.orchestra.run.smithi204.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-21T06:02:21.588 INFO:teuthology.orchestra.run.smithi204.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-21T06:02:21.613 INFO:teuthology.orchestra.run.smithi204.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T06:02:21.638 INFO:teuthology.orchestra.run.smithi204.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T06:02:21.664 INFO:teuthology.orchestra.run.smithi204.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-21T06:02:21.706 INFO:teuthology.orchestra.run.smithi204.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-21T06:02:21.739 INFO:teuthology.orchestra.run.smithi204.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-21T06:02:21.764 INFO:teuthology.orchestra.run.smithi204.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T06:02:21.790 INFO:teuthology.orchestra.run.smithi204.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T06:02:21.807 INFO:teuthology.orchestra.run.smithi204.stdout:(117/157): lua-devel-5.4.4-4.el9.x86_64.rpm 29 kB/s | 22 kB 00:00 2024-09-21T06:02:21.824 INFO:teuthology.orchestra.run.smithi204.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 431 kB/s | 14 kB 00:00 2024-09-21T06:02:21.857 INFO:teuthology.orchestra.run.smithi204.stdout:(119/157): python3-cherrypy-18.6.1-2.el9.noarch 13 MB/s | 358 kB 00:00 2024-09-21T06:02:21.899 INFO:teuthology.orchestra.run.smithi204.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 7.4 MB/s | 303 kB 00:00 2024-09-21T06:02:21.925 INFO:teuthology.orchestra.run.smithi204.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-21T06:02:21.950 INFO:teuthology.orchestra.run.smithi204.stdout:(122/157): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-09-21T06:02:21.984 INFO:teuthology.orchestra.run.smithi204.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-21T06:02:22.009 INFO:teuthology.orchestra.run.smithi204.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-21T06:02:22.034 INFO:teuthology.orchestra.run.smithi204.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 706 kB/s | 18 kB 00:00 2024-09-21T06:02:22.059 INFO:teuthology.orchestra.run.smithi204.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-21T06:02:22.085 INFO:teuthology.orchestra.run.smithi204.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-09-21T06:02:22.110 INFO:teuthology.orchestra.run.smithi204.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 785 kB/s | 20 kB 00:00 2024-09-21T06:02:22.160 INFO:teuthology.orchestra.run.smithi204.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-21T06:02:22.203 INFO:teuthology.orchestra.run.smithi204.stdout:(130/157): protobuf-compiler-3.14.0-13.el9.x86_ 751 kB/s | 863 kB 00:01 2024-09-21T06:02:22.236 INFO:teuthology.orchestra.run.smithi204.stdout:(131/157): python3-logutils-0.3.5-21.el9.noarch 612 kB/s | 46 kB 00:00 2024-09-21T06:02:22.270 INFO:teuthology.orchestra.run.smithi204.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-21T06:02:22.303 INFO:teuthology.orchestra.run.smithi204.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.5 MB/s | 272 kB 00:00 2024-09-21T06:02:22.329 INFO:teuthology.orchestra.run.smithi204.stdout:(134/157): python3-more-itertools-8.12.0-2.el9. 626 kB/s | 79 kB 00:00 2024-09-21T06:02:22.354 INFO:teuthology.orchestra.run.smithi204.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-21T06:02:22.379 INFO:teuthology.orchestra.run.smithi204.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T06:02:22.405 INFO:teuthology.orchestra.run.smithi204.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T06:02:22.430 INFO:teuthology.orchestra.run.smithi204.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-21T06:02:22.464 INFO:teuthology.orchestra.run.smithi204.stdout:(139/157): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-09-21T06:02:22.489 INFO:teuthology.orchestra.run.smithi204.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-21T06:02:22.514 INFO:teuthology.orchestra.run.smithi204.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-21T06:02:22.539 INFO:teuthology.orchestra.run.smithi204.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-21T06:02:22.564 INFO:teuthology.orchestra.run.smithi204.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 990 kB/s | 74 kB 00:00 2024-09-21T06:02:22.590 INFO:teuthology.orchestra.run.smithi204.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T06:02:22.615 INFO:teuthology.orchestra.run.smithi204.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T06:02:22.640 INFO:teuthology.orchestra.run.smithi204.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-21T06:02:22.666 INFO:teuthology.orchestra.run.smithi204.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T06:02:22.691 INFO:teuthology.orchestra.run.smithi204.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T06:02:22.741 INFO:teuthology.orchestra.run.smithi204.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T06:02:22.892 INFO:teuthology.orchestra.run.smithi204.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 15 MB/s | 2.3 MB 00:00 2024-09-21T06:02:22.959 INFO:teuthology.orchestra.run.smithi204.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-09-21T06:02:23.009 INFO:teuthology.orchestra.run.smithi204.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 822 kB/s | 41 kB 00:00 2024-09-21T06:02:23.012 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-21T06:02:23.013 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-21T06:02:23.059 INFO:teuthology.orchestra.run.smithi204.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.9 MB/s | 249 kB 00:00 2024-09-21T06:02:23.126 INFO:teuthology.orchestra.run.smithi204.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-21T06:02:23.201 INFO:teuthology.orchestra.run.smithi204.stdout:(155/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-09-21T06:02:23.252 INFO:teuthology.orchestra.run.smithi204.stdout:(156/157): librbd1-19.3.0-5120.g18432218.el9.x8 5.7 MB/s | 3.2 MB 00:00 2024-09-21T06:02:23.302 INFO:teuthology.orchestra.run.smithi204.stdout:(157/157): librados2-19.3.0-5120.g18432218.el9. 5.3 MB/s | 3.4 MB 00:00 2024-09-21T06:02:23.309 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:02:23.309 INFO:teuthology.orchestra.run.smithi204.stdout:Total 19 MB/s | 228 MB 00:11 2024-09-21T06:02:24.552 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-21T06:02:24.706 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-21T06:02:24.706 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-21T06:02:26.522 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-21T06:02:26.522 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-21T06:02:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-09-21T06:02:27.622 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:02:27.660 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:02:27.714 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:02:27.747 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:02:27.773 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:02:27.816 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:02:28.163 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:02:28.274 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:02:28.400 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:02:28.433 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:02:28.525 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:02:28.554 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:02:28.584 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:02:28.737 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:02:28.793 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:02:28.822 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:02:28.926 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:02:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:02:29.001 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:02:29.023 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:02:29.045 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:02:29.199 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:02:29.333 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:02:29.419 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-21T06:02:29.462 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:02:29.546 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:02:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:02:29.631 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:02:29.662 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:02:29.840 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:02:29.848 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:02:29.881 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:02:29.905 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:02:29.951 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:02:29.965 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:02:29.985 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:02:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:02:30.096 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:02:30.141 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:02:30.181 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:02:30.233 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:02:30.266 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:02:30.293 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:02:30.362 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:02:30.393 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:02:30.425 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:02:30.455 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:02:30.465 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:02:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:02:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:02:30.595 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:02:30.618 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:02:30.633 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:02:30.671 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:02:30.698 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:02:30.725 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:02:30.754 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:02:30.769 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:02:30.784 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:02:30.847 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:02:30.921 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:02:30.974 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:02:31.035 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:02:31.063 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:02:31.064 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:02:31.123 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:02:31.170 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:02:31.197 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:02:31.268 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:02:31.298 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:02:31.318 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:02:31.475 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:02:31.647 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:02:31.653 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:02:31.736 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:02:31.770 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:02:31.773 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:02:31.816 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:02:31.854 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:02:31.878 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:02:31.895 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:02:31.963 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:02:32.061 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:02:32.124 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:02:32.194 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:02:32.215 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:02:32.281 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:02:32.307 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:02:32.352 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:02:32.393 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:02:32.429 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:02:32.443 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:02:32.476 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:02:32.503 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:02:32.511 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:02:32.555 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:02:32.586 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:02:32.596 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:02:32.638 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:02:32.678 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:02:32.721 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:02:32.732 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:02:32.754 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:02:32.773 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:02:32.781 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:02:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:02:32.852 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:02:32.856 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-21T06:02:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:02:32.973 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:02:32.990 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:02:33.011 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:02:33.030 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:02:33.039 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:02:33.056 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:02:33.064 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:02:33.068 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:02:33.080 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:02:33.106 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:02:33.112 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:02:33.132 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:02:33.156 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:02:33.174 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:02:33.194 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:02:33.230 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:02:33.282 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:02:33.283 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:02:33.310 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:02:33.312 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:02:33.348 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:02:33.385 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:02:33.434 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:02:33.507 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:02:33.511 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:02:33.592 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:02:33.631 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:02:33.633 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:02:33.656 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:02:33.671 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:02:33.700 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:02:33.735 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:02:33.742 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:02:33.768 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:02:33.784 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:02:33.810 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:02:33.837 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:02:33.844 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:02:33.875 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:02:33.906 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:02:33.935 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:02:33.938 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:02:33.991 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:02:34.022 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:02:34.094 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:02:34.124 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:02:34.145 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:02:34.173 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:02:34.182 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:02:34.279 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:02:34.307 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:02:34.370 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:02:34.391 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:02:34.392 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:02:34.416 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:02:34.475 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:02:34.510 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:02:34.554 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:02:34.592 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:02:34.610 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:02:34.634 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:02:34.767 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:02:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:02:34.869 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:02:34.882 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:02:34.904 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:02:34.962 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:02:34.964 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:02:34.991 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:02:35.004 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:02:35.058 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:02:35.102 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:02:35.159 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:02:35.203 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:02:35.221 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:02:35.291 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:02:35.296 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:02:35.332 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:02:35.359 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:02:35.396 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:02:35.414 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:02:35.433 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:02:35.441 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:02:35.477 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:02:35.480 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:02:35.515 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:02:35.524 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:02:35.546 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:02:35.551 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:02:35.575 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:02:35.602 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:02:35.617 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:02:35.623 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:02:35.642 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:02:35.686 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:02:35.727 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:02:35.755 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:02:35.760 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:02:35.802 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:02:35.824 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:02:35.844 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:02:35.878 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:02:35.908 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:02:35.943 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:02:35.950 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:02:35.994 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:02:36.025 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:02:36.042 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:02:36.062 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:02:36.083 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:02:36.106 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:02:36.108 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:02:36.143 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:02:36.205 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:02:36.215 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:02:36.266 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:02:36.273 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:02:36.334 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:02:36.341 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:02:36.377 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:02:36.394 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:02:36.413 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:02:36.458 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:02:36.477 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:02:36.495 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:02:36.527 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:02:36.527 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:02:36.565 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:02:36.599 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:02:36.633 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:02:36.675 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:02:36.708 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:02:36.798 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:02:36.848 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:02:36.887 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:02:36.995 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:02:37.047 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:02:37.059 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:02:37.157 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:02:37.184 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:02:37.227 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:02:37.265 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:02:37.308 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:02:37.476 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:02:37.547 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:02:37.626 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:02:37.648 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:02:37.710 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:02:37.711 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:02:37.737 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:02:37.771 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:02:37.784 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:02:37.815 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:02:37.821 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:02:37.851 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:02:37.876 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:02:37.899 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:02:37.918 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:02:37.934 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:02:37.977 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:02:38.020 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:02:38.027 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:02:38.082 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:02:38.098 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:02:38.151 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:02:38.152 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:02:38.165 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:02:38.207 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:02:38.244 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:02:38.250 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:02:38.280 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:02:38.281 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:02:38.316 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:02:38.457 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:02:38.470 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:02:38.503 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:02:38.582 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:02:38.685 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:02:38.723 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:02:38.763 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:02:38.789 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:02:38.813 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:02:38.813 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:02:38.854 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:02:38.890 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:02:38.908 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:02:38.939 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:02:39.024 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:02:39.063 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:02:39.091 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:02:39.125 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:02:39.173 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:02:39.212 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:02:39.242 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:02:39.266 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:02:39.277 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:02:39.310 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:02:39.352 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:02:39.386 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:02:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:02:39.474 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:02:39.517 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:02:39.557 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:02:39.661 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:02:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:02:40.461 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:02:40.514 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:02:40.555 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:02:40.664 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:02:40.711 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:02:40.728 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:02:40.773 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:02:40.806 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:02:40.834 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:02:40.861 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:02:40.867 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:02:40.900 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:02:41.085 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:02:41.123 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:02:41.209 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:02:41.219 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:02:41.271 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:02:41.364 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:02:41.393 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:02:41.429 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:02:41.492 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:02:41.526 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:02:41.860 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:02:41.955 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:02:41.980 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:02:41.983 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:02:42.054 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:02:42.084 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:02:42.099 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:02:42.147 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:02:42.221 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:02:42.221 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:02:42.221 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:02:42.221 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:02:42.248 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:02:43.215 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:02:43.303 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:02:43.331 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:02:43.364 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:02:43.689 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:02:43.735 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:02:43.827 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:02:43.957 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:02:44.010 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:02:44.428 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:02:44.464 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:02:44.517 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:02:44.555 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:02:44.627 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:02:44.657 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:02:45.080 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:02:45.117 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:02:45.209 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:02:45.210 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:02:45.210 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:02:45.210 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:02:45.243 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:02:46.972 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:02:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:02:47.591 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:02:47.632 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:02:47.724 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:02:47.724 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:02:47.725 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:02:47.725 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:02:47.758 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:02:54.057 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:02:54.057 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-09-21T06:02:54.058 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-09-21T06:02:54.058 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-09-21T06:02:54.058 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-09-21T06:02:54.058 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-09-21T06:02:54.058 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-09-21T06:02:54.058 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-09-21T06:02:54.058 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:02:54.259 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:02:54.275 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:02:54.275 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:02:54.275 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:02:54.276 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:02:55.879 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:02:55.906 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:02:55.906 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:02:55.906 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:02:55.906 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:02:55.906 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:02:55.906 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:02:56.146 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:02:56.170 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:02:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:02:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:02:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:02:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:02:56.170 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:02:56.638 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:02:56.662 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:02:56.662 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:02:56.662 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:02:56.663 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:02:56.663 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:02:56.663 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:02:56.768 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:02:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:02:56.880 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:02:57.064 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:02:57.113 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:02:57.199 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:02:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:02:57.427 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:02:57.472 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:02:57.504 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:02:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:02:57.510 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /sys 2024-09-21T06:02:57.510 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /proc 2024-09-21T06:02:57.510 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /mnt 2024-09-21T06:02:57.510 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /var/tmp 2024-09-21T06:02:57.511 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /home 2024-09-21T06:02:57.511 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /root 2024-09-21T06:02:57.511 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /tmp 2024-09-21T06:02:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:02:57.550 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:02:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:02:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:02:57.710 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:02:57.710 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:02:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:02:57.975 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:02:58.014 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:02:58.046 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:02:58.121 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:02:58.154 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:02:59.355 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:02:59.383 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:02:59.383 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:02:59.383 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:02:59.383 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:02:59.383 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:02:59.383 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:02:59.623 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:02:59.642 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:02:59.642 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-09-21T06:02:59.642 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-09-21T06:02:59.642 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-09-21T06:02:59.642 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-09-21T06:02:59.642 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-09-21T06:02:59.642 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-09-21T06:02:59.642 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-09-21T06:02:59.642 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:02:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:02:59.648 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:02:59.648 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:02:59.648 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:02:59.648 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:02:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:02:59.820 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:02:59.837 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:02:59.837 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:02:59.837 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:02:59.837 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:03:00.140 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:03:00.165 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:03:00.165 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:00.165 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:03:00.165 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:03:00.165 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:03:00.165 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:03:00.272 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:03:00.329 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:03:00.401 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:03:00.596 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:03:00.650 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:03:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:03:00.829 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:03:00.980 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:03:01.017 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:03:01.050 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:03:01.078 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:03:01.330 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:03:01.393 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:03:01.450 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:03:01.477 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:03:01.477 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:01.477 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:03:01.477 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:03:01.478 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:03:01.478 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:03:01.560 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:03:01.592 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:03:01.624 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:03:01.693 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:03:01.710 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:03:01.723 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:03:01.733 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:03:01.734 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:01.734 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:03:01.734 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:03:01.734 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:03:01.734 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:03:02.201 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:03:02.225 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:03:02.225 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:02.225 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:03:02.225 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:03:02.225 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:03:02.225 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:03:02.340 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:03:02.356 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:03:02.385 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:03:02.396 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:03:02.452 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:03:02.498 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:03:02.646 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:03:02.678 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:03:02.702 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:03:02.712 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:03:02.737 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:03:02.737 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:02.737 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:03:02.737 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:03:02.737 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:03:02.737 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:03:02.772 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:03:02.788 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:03:02.800 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:03:02.861 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:03:02.898 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:03:02.952 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:03:03.010 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:03:03.052 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:03:03.085 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:03:03.121 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:03:03.523 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:03:03.553 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:03:03.585 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:03:03.652 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:03:03.676 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:03:03.917 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:03:03.948 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:03:03.972 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:03:03.973 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:03.973 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:03:03.973 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:03:03.973 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:03:03.973 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:03:04.016 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:03:04.039 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:03:04.040 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:04.040 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:03:04.040 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:03:04.091 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:03:04.115 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:03:04.115 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:04.115 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:03:04.116 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:03:04.116 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:03:04.116 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:03:04.442 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:03:04.466 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:03:04.466 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:04.466 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:03:04.466 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:03:04.466 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:03:04.467 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:03:05.097 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:03:05.162 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:03:06.174 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:03:06.212 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:03:06.336 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:03:06.511 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:03:06.550 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:03:06.572 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:03:06.572 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:06.572 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:03:06.572 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:03:06.572 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:03:06.573 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:03:06.632 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:03:06.644 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:03:06.746 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:03:06.804 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:03:06.912 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:03:06.974 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:03:07.823 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:03:07.859 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:03:07.884 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:03:07.884 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:07.884 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:03:07.884 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:03:07.884 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:03:07.884 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:03:07.926 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:03:07.941 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:03:07.950 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:03:07.950 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:07.950 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:03:07.950 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:03:07.966 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:03:08.001 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:03:08.026 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:03:08.026 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:08.026 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:03:08.026 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:03:08.026 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:03:08.026 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:03:08.083 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:03:08.258 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:03:08.293 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:03:08.317 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:03:08.318 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:08.318 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:03:08.318 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:03:08.318 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:03:08.318 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:03:08.360 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:03:08.369 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:03:08.381 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:03:08.385 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:03:08.385 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:08.385 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:03:08.385 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:03:08.385 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:03:08.385 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:03:08.463 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:03:08.479 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:03:08.513 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:03:08.526 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:03:08.553 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:03:08.585 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:03:08.612 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:03:08.795 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:03:08.834 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:03:08.864 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:03:08.894 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:03:08.941 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:03:09.491 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:03:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:03:09.521 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:03:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:03:09.546 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:03:09.546 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:09.546 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:03:09.546 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:03:09.546 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:03:09.546 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:03:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:03:09.597 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:03:09.620 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:03:09.621 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:09.621 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:03:09.621 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:03:09.674 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:03:09.696 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:03:09.696 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:09.697 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:03:09.697 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:03:09.697 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:03:09.697 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:03:10.024 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:03:10.047 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:03:10.047 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:03:10.048 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:03:10.048 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:03:10.048 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:03:10.048 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:03:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:03:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:03:10.533 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:03:10.614 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:03:10.662 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:03:10.688 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:03:10.722 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:03:10.722 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:03:10.823 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:03:10.823 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:03:10.940 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:03:10.966 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:03:10.971 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:03:10.971 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:03:11.150 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:03:11.150 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:03:11.234 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:03:11.299 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:03:12.675 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:03:12.724 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:03:12.765 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:03:12.796 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:03:12.823 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:03:13.038 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:03:13.095 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:03:13.125 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:03:13.155 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:03:13.201 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:03:13.787 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:03:13.831 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:03:13.853 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:03:14.045 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:03:14.086 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:03:14.127 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:03:14.159 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:03:14.186 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:03:14.383 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:03:14.433 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:03:14.463 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:03:14.493 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:03:14.548 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:03:14.765 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:03:14.860 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:03:14.913 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:03:15.030 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:03:15.063 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:03:15.089 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:03:15.115 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:03:15.115 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:03:15.191 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:03:15.216 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:03:15.216 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:03:15.237 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:03:15.258 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:03:15.300 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:03:15.333 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:03:15.339 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:03:15.339 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:03:15.527 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:03:15.527 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:03:15.602 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:03:15.671 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:03:16.120 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:03:16.209 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:03:16.251 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:03:16.332 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:03:16.371 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:03:16.405 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:03:16.431 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:03:16.431 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:03:16.507 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:03:16.507 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:03:16.599 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:03:16.625 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:03:16.630 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:03:16.630 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:03:16.809 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:03:16.810 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:03:16.885 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:03:16.949 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:03:17.810 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:03:17.810 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:03:17.810 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:03:17.810 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:03:17.810 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:03:17.810 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:03:17.810 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:03:17.811 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:03:17.812 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:03:17.812 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:03:17.812 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:03:17.812 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:03:17.812 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:03:17.812 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:03:17.812 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:03:17.812 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:03:17.812 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:03:17.825 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:03:17.825 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:03:17.825 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:03:17.826 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:03:17.826 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:03:17.826 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:03:17.826 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:03:17.826 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:03:17.826 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:03:17.826 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:03:17.826 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:03:17.826 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:03:17.826 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:03:17.826 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:03:17.826 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:03:17.826 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:03:17.827 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:03:17.827 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:03:17.827 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:03:17.827 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:03:17.827 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:03:17.827 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:03:17.827 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:03:17.827 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:03:17.827 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:03:17.827 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:03:17.827 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:03:17.827 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:03:17.827 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:03:17.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:03:17.829 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:03:17.830 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:03:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:03:17.832 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:03:17.832 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:03:17.832 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:03:17.832 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:03:17.832 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:03:17.832 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:03:17.832 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:03:17.832 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:03:17.832 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:03:17.832 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:03:17.832 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:03:17.832 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:03:17.833 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:03:17.833 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:03:17.833 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:03:17.833 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:03:17.833 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:03:17.833 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:03:17.833 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:03:17.833 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:03:17.833 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:03:17.833 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:03:17.833 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:03:17.833 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:03:17.833 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:03:17.834 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:03:17.834 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:03:17.834 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:03:17.834 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:03:17.834 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:03:17.834 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:03:17.834 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:03:17.834 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:03:17.834 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:03:17.834 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:03:17.834 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:03:17.834 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:03:17.834 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:03:17.835 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:03:17.835 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:03:17.835 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:03:17.835 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:03:17.835 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:03:17.835 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:03:20.307 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:03:20.307 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:03:20.307 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-09-21T06:03:20.307 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:03:20.307 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:03:20.307 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:03:20.308 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:03:20.308 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.308 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.308 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:03:20.308 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:03:20.308 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-09-21T06:03:20.308 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:03:20.309 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:03:20.309 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:03:20.309 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:03:20.309 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.309 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.309 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.309 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.310 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:20.310 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.310 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.310 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.310 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:20.310 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:20.311 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:20.311 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:20.311 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:20.311 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.311 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.311 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:20.311 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.312 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.312 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.312 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:20.312 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:20.312 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.312 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.313 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:20.313 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:03:20.313 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:03:20.313 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:03:20.313 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:03:20.313 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:03:20.314 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:03:20.314 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:03:20.314 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:03:20.314 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:03:20.314 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:03:20.314 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:03:20.315 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.315 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.315 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.315 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:03:20.315 INFO:teuthology.orchestra.run.smithi137.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:03:20.315 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:03:20.315 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:03:20.316 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:03:20.316 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:03:20.316 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.316 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.316 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:03:20.316 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.317 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:03:20.317 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:03:20.317 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:03:20.317 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:03:20.317 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:03:20.317 INFO:teuthology.orchestra.run.smithi137.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:03:20.317 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:03:20.318 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:03:20.318 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:03:20.318 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:03:20.318 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:03:20.318 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:03:20.318 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:03:20.319 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:03:20.319 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:03:20.319 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:03:20.319 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:03:20.319 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:03:20.319 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:03:20.319 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:03:20.320 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:03:20.320 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:03:20.320 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:03:20.320 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.320 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.320 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.320 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:03:20.321 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:03:20.321 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:03:20.321 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:03:20.321 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:03:20.321 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:03:20.321 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:03:20.321 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:03:20.322 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:03:20.322 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:03:20.322 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:03:20.322 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:03:20.322 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:03:20.322 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:03:20.322 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:03:20.323 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:03:20.323 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:03:20.323 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:03:20.323 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:03:20.323 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:03:20.323 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:03:20.324 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:03:20.324 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:03:20.324 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:03:20.324 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:03:20.324 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:03:20.324 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:03:20.324 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:03:20.325 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:03:20.325 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:03:20.325 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:03:20.325 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:03:20.325 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:03:20.325 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:03:20.325 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:03:20.326 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:03:20.326 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:03:20.326 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:03:20.326 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:03:20.326 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:03:20.326 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.326 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:03:20.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:03:20.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:03:20.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:03:20.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:03:20.328 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:03:20.328 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:03:20.328 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:03:20.328 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:03:20.328 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:03:20.328 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:03:20.329 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:03:20.329 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:03:20.329 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:03:20.329 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:03:20.329 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:03:20.329 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:03:20.329 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:03:20.330 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.330 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.330 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:20.330 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:03:20.330 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:03:20.330 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:03:20.330 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:03:20.331 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:03:20.331 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:03:20.331 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:03:20.331 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:03:20.331 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-09-21T06:03:20.731 DEBUG:teuthology.parallel:result is None 2024-09-21T06:03:21.979 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:03:21.979 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:03:21.979 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:03:21.979 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:03:21.979 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:03:21.979 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:03:21.979 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:03:21.979 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:03:21.979 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:03:21.979 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:03:21.980 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:03:21.980 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:03:21.980 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:03:21.980 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:03:21.980 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:03:21.980 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:03:21.980 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:03:21.980 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:03:21.980 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:03:21.980 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:03:21.980 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:03:21.980 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:03:21.980 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:03:21.981 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:03:21.982 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:03:21.983 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:03:21.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:03:21.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:03:21.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:03:21.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:03:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:03:21.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:03:21.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:03:21.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:03:23.428 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:03:23.428 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:03:23.428 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:03:23.428 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:03:23.428 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:03:23.428 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:03:23.428 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:03:23.428 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:03:23.428 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:03:23.428 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:03:23.428 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:03:23.428 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:03:23.429 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:03:23.429 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:03:23.429 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:03:23.429 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:03:23.429 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:03:23.429 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:03:23.429 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:03:23.429 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:03:23.429 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:03:23.429 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:03:23.429 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:03:23.429 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:03:23.429 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:03:23.441 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:03:23.441 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:03:23.441 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:03:23.441 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:03:23.441 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:03:23.441 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:03:23.441 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:03:23.441 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:03:23.441 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:03:23.441 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:03:23.441 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:03:23.441 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:03:23.441 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:03:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:03:23.443 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:03:23.444 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:03:23.445 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:03:23.446 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:03:23.446 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:03:23.446 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:03:23.446 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:03:23.446 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:03:23.446 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:03:23.446 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:03:23.446 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:03:23.446 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:03:23.446 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:03:23.446 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:03:23.446 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:03:23.446 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:03:23.447 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:03:23.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:03:23.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:03:23.450 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:03:23.450 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:03:23.450 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:03:23.450 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:03:23.450 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:03:23.450 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:03:23.450 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:03:23.450 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:03:23.450 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:03:23.450 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:03:23.450 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:03:23.450 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:03:23.450 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:03:24.639 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:03:24.639 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:03:24.639 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2024-09-21T06:03:24.639 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:03:24.639 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:03:24.639 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:03:24.639 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:03:24.639 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.639 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-21T06:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:03:24.642 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:03:24.643 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.644 INFO:teuthology.orchestra.run.smithi003.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:03:24.644 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:03:24.644 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:03:24.644 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:03:24.644 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:03:24.644 INFO:teuthology.orchestra.run.smithi003.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:03:24.644 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:03:24.644 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:03:24.644 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:03:24.644 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:03:24.644 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:03:24.644 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:03:24.644 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:03:24.645 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:03:24.645 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:03:24.645 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:03:24.645 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:03:24.645 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:03:24.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:03:24.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:03:24.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:03:24.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:03:24.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:03:24.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:03:24.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:03:24.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:03:24.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:03:24.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:03:24.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:03:24.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:03:24.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:03:24.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:03:24.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:03:24.647 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:03:24.648 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:03:24.648 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:03:24.648 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:03:24.648 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:03:24.649 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:03:24.649 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:03:24.649 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:03:24.649 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:03:24.649 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:03:24.649 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:03:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.651 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.652 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:24.652 INFO:teuthology.orchestra.run.smithi003.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:03:24.652 INFO:teuthology.orchestra.run.smithi003.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:03:24.652 INFO:teuthology.orchestra.run.smithi003.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:03:24.652 INFO:teuthology.orchestra.run.smithi003.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:03:24.652 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:03:24.652 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:03:24.652 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:03:24.652 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:03:24.652 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-21T06:03:25.066 DEBUG:teuthology.parallel:result is None 2024-09-21T06:03:26.237 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:03:26.237 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:03:26.237 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-09-21T06:03:26.238 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:03:26.238 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:03:26.238 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:03:26.238 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:03:26.238 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.238 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.238 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:03:26.238 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:03:26.238 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-21T06:03:26.238 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:03:26.239 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:03:26.239 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:03:26.239 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:03:26.239 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.239 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.239 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.239 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.239 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:26.239 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.239 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.239 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.239 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:26.239 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.240 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:03:26.241 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:03:26.241 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:03:26.241 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:03:26.241 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:03:26.241 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:03:26.241 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:03:26.241 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:03:26.241 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:03:26.241 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:03:26.241 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:03:26.241 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:03:26.241 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.242 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.242 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.242 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:03:26.242 INFO:teuthology.orchestra.run.smithi204.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:03:26.242 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:03:26.242 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:03:26.242 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:03:26.242 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:03:26.242 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.242 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.242 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:03:26.242 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.242 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:03:26.243 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:03:26.244 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:03:26.244 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:03:26.244 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:03:26.244 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:03:26.244 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:03:26.244 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:03:26.244 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:03:26.244 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:03:26.244 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.244 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.244 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.244 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:03:26.244 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:03:26.245 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:03:26.245 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:03:26.245 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:03:26.245 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:03:26.245 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:03:26.245 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:03:26.245 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:03:26.245 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:03:26.245 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:03:26.245 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:03:26.245 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:03:26.245 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:03:26.245 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:03:26.246 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:03:26.246 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:03:26.246 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:03:26.246 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:03:26.246 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:03:26.246 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:03:26.246 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:03:26.246 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:03:26.246 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:03:26.246 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:03:26.246 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:03:26.246 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:03:26.246 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:03:26.247 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:03:26.247 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:03:26.247 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:03:26.247 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:03:26.247 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:03:26.247 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:03:26.247 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:03:26.247 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:03:26.247 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:03:26.247 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:03:26.247 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:03:26.247 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:03:26.247 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:03:26.249 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:03:26.250 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:03:26.250 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:03:26.250 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:03:26.250 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-21T06:03:26.694 DEBUG:teuthology.parallel:result is None 2024-09-21T06:03:26.694 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:03:26.838 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:03:26.865 INFO:teuthology.orchestra.run.smithi003.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:03:27.223 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:03:27.223 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:03:27.225 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:03:27.371 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:03:27.397 INFO:teuthology.orchestra.run.smithi137.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:03:27.398 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:03:27.398 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:03:27.399 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:03:27.538 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:03:27.564 INFO:teuthology.orchestra.run.smithi204.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:03:27.564 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:03:27.564 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:03:27.566 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T06:03:27.566 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:03:27.566 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:03:27.600 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-21T06:03:27.600 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:03:27.634 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-21T06:03:27.634 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:03:27.667 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T06:03:27.667 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:03:27.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:03:27.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:03:27.766 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-21T06:03:27.767 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:03:27.799 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:03:27.869 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-21T06:03:27.869 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:03:27.901 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:03:27.972 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T06:03:27.973 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:03:27.973 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:03:28.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:03:28.072 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-21T06:03:28.072 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:03:28.102 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:03:28.172 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-21T06:03:28.172 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:03:28.201 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:03:28.270 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T06:03:28.270 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:03:28.270 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:03:28.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:03:28.371 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-21T06:03:28.371 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:03:28.399 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:03:28.468 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-21T06:03:28.468 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:03:28.498 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:03:28.568 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T06:03:28.658 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug', 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_export_ephemeral_random_max': 0.1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-21T06:03:28.658 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:03:28.659 INFO:tasks.cephadm:Cluster fsid is 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:03:28.659 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T06:03:28.659 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.3', 'mon.b': '172.21.15.137', 'mon.c': '172.21.15.204'} 2024-09-21T06:03:28.659 INFO:tasks.cephadm:First mon is mon.a on smithi003 2024-09-21T06:03:28.659 INFO:tasks.cephadm:First mgr is x 2024-09-21T06:03:28.659 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T06:03:28.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname $(hostname -s) 2024-09-21T06:03:28.691 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2024-09-21T06:03:28.722 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2024-09-21T06:03:28.754 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T06:03:28.754 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:03:28.893 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T06:03:29.035 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:03:29.036 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T06:03:29.036 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T06:03:29.037 DEBUG:teuthology.orchestra.run.smithi003:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:03:29.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:03 /home/ubuntu/cephtest/cephadm 2024-09-21T06:03:29.569 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:03:30.143 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:03 /home/ubuntu/cephtest/cephadm 2024-09-21T06:03:30.143 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:03:30.695 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:03 /home/ubuntu/cephtest/cephadm 2024-09-21T06:03:30.695 DEBUG:teuthology.orchestra.run.smithi003:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:03:30.716 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:03:30.735 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:03:30.773 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T06:03:30.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:03:30.775 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:03:30.778 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:03:32.382 INFO:teuthology.orchestra.run.smithi137.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:03:32.429 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:03:32.582 INFO:teuthology.orchestra.run.smithi003.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:04:11.355 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-21T06:04:11.355 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:04:11.355 INFO:teuthology.orchestra.run.smithi137.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T06:04:11.355 INFO:teuthology.orchestra.run.smithi137.stdout: "repo_digests": [ 2024-09-21T06:04:11.355 INFO:teuthology.orchestra.run.smithi137.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T06:04:11.355 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-21T06:04:11.355 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-21T06:04:14.682 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:04:14.682 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:04:14.682 INFO:teuthology.orchestra.run.smithi003.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T06:04:14.682 INFO:teuthology.orchestra.run.smithi003.stdout: "repo_digests": [ 2024-09-21T06:04:14.682 INFO:teuthology.orchestra.run.smithi003.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T06:04:14.682 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-21T06:04:14.682 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:04:15.045 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-21T06:04:15.045 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:04:15.045 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T06:04:15.045 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2024-09-21T06:04:15.045 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T06:04:15.045 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2024-09-21T06:04:15.046 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-21T06:04:15.072 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph 2024-09-21T06:04:15.101 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2024-09-21T06:04:15.134 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2024-09-21T06:04:15.161 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /etc/ceph 2024-09-21T06:04:15.187 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2024-09-21T06:04:15.216 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2024-09-21T06:04:15.241 INFO:tasks.cephadm:Writing seed config... 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-21T06:04:15.243 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-21T06:04:15.244 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T06:04:15.244 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T06:04:15.244 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-21T06:04:15.244 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T06:04:15.244 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T06:04:15.244 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T06:04:15.244 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T06:04:15.244 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T06:04:15.244 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T06:04:15.244 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-21T06:04:15.245 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:04:15.245 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T06:04:15.260 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 38d3c75c-77df-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_export_ephemeral_random_max = 0.1 2024-09-21T06:04:15.261 DEBUG:teuthology.orchestra.run.smithi003:mon.a> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T06:04:15.304 DEBUG:teuthology.orchestra.run.smithi003:mgr.x> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T06:04:15.347 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T06:04:15.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.3 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:04:15.558 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:04:15.558 INFO:teuthology.orchestra.run.smithi003.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.3', '--skip-admin-label'] 2024-09-21T06:04:15.558 INFO:teuthology.orchestra.run.smithi003.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T06:04:15.558 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying podman|docker is present... 2024-09-21T06:04:15.584 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 5.2.2 2024-09-21T06:04:15.584 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying lvm2 is present... 2024-09-21T06:04:15.584 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying time synchronization is in place... 2024-09-21T06:04:15.592 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T06:04:15.592 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T06:04:15.599 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T06:04:15.599 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-21T06:04:15.606 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2024-09-21T06:04:15.612 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2024-09-21T06:04:15.613 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2024-09-21T06:04:15.613 INFO:teuthology.orchestra.run.smithi003.stdout:Repeating the final host check... 2024-09-21T06:04:15.636 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 5.2.2 2024-09-21T06:04:15.636 INFO:teuthology.orchestra.run.smithi003.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T06:04:15.636 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl is present 2024-09-21T06:04:15.636 INFO:teuthology.orchestra.run.smithi003.stdout:lvcreate is present 2024-09-21T06:04:15.643 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T06:04:15.643 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T06:04:15.649 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T06:04:15.649 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-21T06:04:15.656 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2024-09-21T06:04:15.663 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2024-09-21T06:04:15.663 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2024-09-21T06:04:15.663 INFO:teuthology.orchestra.run.smithi003.stdout:Host looks OK 2024-09-21T06:04:15.666 INFO:teuthology.orchestra.run.smithi003.stdout:Cluster fsid: 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:04:15.666 INFO:teuthology.orchestra.run.smithi003.stdout:Acquiring lock 139655053007504 on /run/cephadm/38d3c75c-77df-11ef-baf5-efdc52797490.lock 2024-09-21T06:04:15.667 INFO:teuthology.orchestra.run.smithi003.stdout:Lock 139655053007504 acquired on /run/cephadm/38d3c75c-77df-11ef-baf5-efdc52797490.lock 2024-09-21T06:04:15.667 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 3300 ... 2024-09-21T06:04:15.667 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 6789 ... 2024-09-21T06:04:15.668 INFO:teuthology.orchestra.run.smithi003.stdout:Base mon IP(s) is [172.21.15.3:3300, 172.21.15.3:6789], mon addrv is [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-09-21T06:04:15.668 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.3 metric 100 2024-09-21T06:04:15.668 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.3 metric 100 2024-09-21T06:04:15.670 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T06:04:15.670 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-21T06:04:15.670 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1786sec hoplimit 64 pref medium 2024-09-21T06:04:15.672 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T06:04:15.672 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T06:04:15.673 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T06:04:15.673 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 3: ens1f0: mtu 1500 state UP qlen 1000 2024-09-21T06:04:15.673 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-21T06:04:15.673 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T06:04:15.673 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2024-09-21T06:04:15.673 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2024-09-21T06:04:15.673 INFO:teuthology.orchestra.run.smithi003.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T06:04:15.674 INFO:teuthology.orchestra.run.smithi003.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T06:04:15.674 INFO:teuthology.orchestra.run.smithi003.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:04:16.167 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T06:04:16.168 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:04:16.168 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T06:04:16.168 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T06:04:16.168 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T06:04:16.168 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T06:04:16.168 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T06:04:17.080 INFO:teuthology.orchestra.run.smithi003.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T06:04:17.080 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T06:04:17.080 INFO:teuthology.orchestra.run.smithi003.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T06:04:18.128 INFO:teuthology.orchestra.run.smithi003.stdout:stat: stdout 167 167 2024-09-21T06:04:18.128 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial keys... 2024-09-21T06:04:19.035 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQDiYe5mncJQHhAA3MJOJGnAhoU/7p+4EaCqPA== 2024-09-21T06:04:19.939 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQDjYe5mfY01FRAAZ9gkfe7VEsZP1JQrUzFvsw== 2024-09-21T06:04:20.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQDkYe5m3t18ERAAZFouEW/IED1d5+fFTF9a+g== 2024-09-21T06:04:20.845 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial monmap... 2024-09-21T06:04:21.759 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T06:04:21.759 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T06:04:21.759 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:04:21.759 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T06:04:21.759 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool for a [v2:172.21.15.3:3300,v1:172.21.15.3:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T06:04:21.759 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = quincy 2024-09-21T06:04:21.759 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: set fsid to 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:04:21.760 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T06:04:21.760 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:04:21.760 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mon... 2024-09-21T06:04:22.831 INFO:teuthology.orchestra.run.smithi003.stdout:create mon.a on 2024-09-21T06:04:23.051 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T06:04:23.201 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T06:04:23.379 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-38d3c75c-77df-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-38d3c75c-77df-11ef-baf5-efdc52797490.target. 2024-09-21T06:04:23.379 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-38d3c75c-77df-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-38d3c75c-77df-11ef-baf5-efdc52797490.target. 2024-09-21T06:04:23.606 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.a 2024-09-21T06:04:23.606 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T06:04:23.772 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-38d3c75c-77df-11ef-baf5-efdc52797490.target.wants/ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-38d3c75c-77df-11ef-baf5-efdc52797490@.service. 2024-09-21T06:04:24.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:24 smithi003 podman[30845]: 2024-09-21 06:04:24.243396676 +0000 UTC m=+0.345282343 container start 76e059b1796812709346f4a8d3b1407e115b75963fb6855cb2e3f59ca615c92e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T06:04:24.348 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:04:24.348 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-09-21T06:04:24.355 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:04:24.355 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-21T06:04:24.355 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-09-21T06:04:24.355 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T06:04:24.356 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon to start... 2024-09-21T06:04:24.356 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon... 2024-09-21T06:04:24.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:24 smithi003 ceph-mon[30864]: mkfs 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:04:24.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:24 smithi003 ceph-mon[30864]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T06:04:24.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:24 smithi003 bash[30845]: 76e059b1796812709346f4a8d3b1407e115b75963fb6855cb2e3f59ca615c92e 2024-09-21T06:04:24.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:24 smithi003 systemd[1]: Started Ceph mon.a for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:04:25.506 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:25 smithi003 ceph-mon[30864]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T06:04:25.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:25 smithi003 ceph-mon[30864]: monmap epoch 1 2024-09-21T06:04:25.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:25 smithi003 ceph-mon[30864]: fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:04:25.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:25 smithi003 ceph-mon[30864]: last_changed 2024-09-21T06:04:21.188902+0000 2024-09-21T06:04:25.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:25 smithi003 ceph-mon[30864]: created 2024-09-21T06:04:21.188902+0000 2024-09-21T06:04:25.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:25 smithi003 ceph-mon[30864]: min_mon_release 19 (squid) 2024-09-21T06:04:25.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:25 smithi003 ceph-mon[30864]: election_strategy: 1 2024-09-21T06:04:25.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:25 smithi003 ceph-mon[30864]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-21T06:04:25.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:25 smithi003 ceph-mon[30864]: fsmap 2024-09-21T06:04:25.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:25 smithi003 ceph-mon[30864]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T06:04:25.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:25 smithi003 ceph-mon[30864]: mgrmap e1: no daemons active 2024-09-21T06:04:25.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:25 smithi003 ceph-mon[30864]: from='client.? 172.21.15.3:0/556011602' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T06:04:25.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout id: 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout services: 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.688588s) 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout data: 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T06:04:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-21T06:04:25.512 INFO:teuthology.orchestra.run.smithi003.stdout:mon is available 2024-09-21T06:04:25.512 INFO:teuthology.orchestra.run.smithi003.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T06:04:26.538 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:26 smithi003 ceph-mon[30864]: from='client.? 172.21.15.3:0/2453337970' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T06:04:26.539 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:26 smithi003 ceph-mon[30864]: from='client.? 172.21.15.3:0/2453337970' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T06:04:26.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-21T06:04:26.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T06:04:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T06:04:26.545 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T06:04:26.545 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T06:04:26.545 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T06:04:26.545 INFO:teuthology.orchestra.run.smithi003.stdout:Generating new minimal ceph.conf... 2024-09-21T06:04:27.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:27 smithi003 ceph-mon[30864]: from='client.? 172.21.15.3:0/3235862678' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:27.681 INFO:teuthology.orchestra.run.smithi003.stdout:Restarting the monitor... 2024-09-21T06:04:27.973 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:27 smithi003 systemd[1]: Stopping Ceph mon.a for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:04:28.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:27 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a[30860]: 2024-09-21T06:04:27.970+0000 7ff4ab305640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:04:28.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:27 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a[30860]: 2024-09-21T06:04:27.970+0000 7ff4ab305640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T06:04:28.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:27 smithi003 podman[31106]: 2024-09-21 06:04:27.982063194 +0000 UTC m=+0.120439873 container died 76e059b1796812709346f4a8d3b1407e115b75963fb6855cb2e3f59ca615c92e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T06:04:28.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:28 smithi003 podman[31106]: 2024-09-21 06:04:28.287962156 +0000 UTC m=+0.426338867 container cleanup 76e059b1796812709346f4a8d3b1407e115b75963fb6855cb2e3f59ca615c92e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T06:04:28.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:28 smithi003 bash[31106]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a 2024-09-21T06:04:28.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:28 smithi003 podman[31118]: 2024-09-21 06:04:28.438791519 +0000 UTC m=+0.450818466 container remove 76e059b1796812709346f4a8d3b1407e115b75963fb6855cb2e3f59ca615c92e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T06:04:28.976 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:28 smithi003 systemd[1]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T06:04:28.976 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:28 smithi003 systemd[1]: Stopped Ceph mon.a for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:04:28.976 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:28 smithi003 systemd[1]: Starting Ceph mon.a for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:04:28.976 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:28 smithi003 podman[31208]: 2024-09-21 06:04:28.864557464 +0000 UTC m=+0.107491865 container create 6ceb763afa3c479746e0c332d336fa89a4ab399f9c9303235dfcf01e622ac614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T06:04:28.976 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:28 smithi003 podman[31208]: 2024-09-21 06:04:28.773377809 +0000 UTC m=+0.016312210 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:04:29.147 INFO:teuthology.orchestra.run.smithi003.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T06:04:29.271 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 podman[31208]: 2024-09-21 06:04:29.058607953 +0000 UTC m=+0.301542357 container init 6ceb763afa3c479746e0c332d336fa89a4ab399f9c9303235dfcf01e622ac614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:04:29.271 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 podman[31208]: 2024-09-21 06:04:29.061912974 +0000 UTC m=+0.304847376 container start 6ceb763afa3c479746e0c332d336fa89a4ab399f9c9303235dfcf01e622ac614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:04:29.272 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T06:04:29.272 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T06:04:29.272 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: pidfile_write: ignore empty --pid-file 2024-09-21T06:04:29.272 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: load: jerasure load: lrc 2024-09-21T06:04:29.272 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: RocksDB version: 7.9.2 2024-09-21T06:04:29.272 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Git sha 0 2024-09-21T06:04:29.272 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T06:04:29.272 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: DB SUMMARY 2024-09-21T06:04:29.272 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: DB Session ID: 1GHFRK7O62DQGRZJ1MWC 2024-09-21T06:04:29.272 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: CURRENT file: CURRENT 2024-09-21T06:04:29.272 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T06:04:29.272 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T06:04:29.272 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T06:04:29.273 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94810 ; 2024-09-21T06:04:29.273 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.error_if_exists: 0 2024-09-21T06:04:29.273 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.create_if_missing: 0 2024-09-21T06:04:29.273 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.paranoid_checks: 1 2024-09-21T06:04:29.273 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T06:04:29.273 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T06:04:29.273 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T06:04:29.273 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.env: 0x55b80f591d60 2024-09-21T06:04:29.273 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T06:04:29.273 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.info_log: 0x55b8156c7360 2024-09-21T06:04:29.273 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T06:04:29.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.statistics: (nil) 2024-09-21T06:04:29.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.use_fsync: 0 2024-09-21T06:04:29.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_log_file_size: 0 2024-09-21T06:04:29.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T06:04:29.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T06:04:29.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T06:04:29.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T06:04:29.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.allow_fallocate: 1 2024-09-21T06:04:29.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T06:04:29.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T06:04:29.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.use_direct_reads: 0 2024-09-21T06:04:29.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T06:04:29.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.db_log_dir: 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.wal_dir: 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.write_buffer_manager: 0x55b8156cbd60 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T06:04:29.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.unordered_write: 0 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.row_cache: None 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.wal_filter: None 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T06:04:29.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.two_write_queues: 0 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.wal_compression: 0 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.atomic_flush: 0 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.log_readahead_size: 0 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T06:04:29.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T06:04:29.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_background_jobs: 2 2024-09-21T06:04:29.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_background_compactions: -1 2024-09-21T06:04:29.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_subcompactions: 1 2024-09-21T06:04:29.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T06:04:29.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T06:04:29.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T06:04:29.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T06:04:29.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T06:04:29.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T06:04:29.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T06:04:29.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_open_files: -1 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_background_flushes: -1 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Compression algorithms supported: 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: kZSTD supported: 0 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: kXpressCompression supported: 0 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: kBZip2Compression supported: 0 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: kLZ4Compression supported: 1 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: kZlibCompression supported: 1 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: kSnappyCompression supported: 1 2024-09-21T06:04:29.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.merge_operator: 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compaction_filter: None 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b8156c6fa0) 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks: 1 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T06:04:29.281 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_top_level_index_and_filter: 1 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_type: 0 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_index_type: 0 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_shortening: 1 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: checksum: 4 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: no_block_cache: 0 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache: 0x55b81168b610 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_name: BinnedLRUCache 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_options: 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: capacity : 536870912 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_shard_bits : 4 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: strict_capacity_limit : 0 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: high_pri_pool_ratio: 0.000 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_compressed: (nil) 2024-09-21T06:04:29.282 INFO:journalctl@ceph.mon.a.smithi003.stdout: persistent_cache: (nil) 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size: 4096 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size_deviation: 10 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_restart_interval: 16 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_block_restart_interval: 1 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: metadata_block_size: 4096 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: partition_filters: 0 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: use_delta_encoding: 1 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: filter_policy: bloomfilter 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: whole_key_filtering: 1 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: verify_compression: 0 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: read_amp_bytes_per_bit: 0 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: format_version: 5 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_index_compression: 1 2024-09-21T06:04:29.283 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_align: 0 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout: max_auto_readahead_size: 262144 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout: prepopulate_block_cache: 0 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout: initial_auto_readahead_size: 8192 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compression: NoCompression 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.num_levels: 7 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T06:04:29.285 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T06:04:29.286 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:04:29.286 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T06:04:29.286 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T06:04:29.286 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T06:04:29.286 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T06:04:29.286 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T06:04:29.286 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T06:04:29.286 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T06:04:29.286 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T06:04:29.286 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T06:04:29.286 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T06:04:29.286 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T06:04:29.286 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T06:04:29.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T06:04:29.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T06:04:29.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T06:04:29.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T06:04:29.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T06:04:29.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T06:04:29.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.inplace_update_support: 0 2024-09-21T06:04:29.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T06:04:29.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T06:04:29.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T06:04:29.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T06:04:29.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.bloom_locality: 0 2024-09-21T06:04:29.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.max_successive_merges: 0 2024-09-21T06:04:29.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.ttl: 2592000 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.enable_blob_files: false 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.min_blob_size: 0 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T06:04:29.289 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T06:04:29.290 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T06:04:29.290 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T06:04:29.290 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T06:04:29.290 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T06:04:29.290 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ba26b746-01b0-4bf7-b327-a1d220889e6d 2024-09-21T06:04:29.290 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898669098472, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T06:04:29.290 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T06:04:29.290 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898669099649, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91023, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 283, "table_properties": {"data_size": 89071, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11605, "raw_average_key_size": 47, "raw_value_size": 82502, "raw_average_value_size": 340, "num_data_blocks": 12, "num_entries": 242, "num_filter_entries": 242, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726898669, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ba26b746-01b0-4bf7-b327-a1d220889e6d", "db_session_id": "1GHFRK7O62DQGRZJ1MWC", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T06:04:29.290 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898669099757, "job": 1, "event": "recovery_finished"} 2024-09-21T06:04:29.290 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T06:04:29.290 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T06:04:29.290 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b8156eaa80 2024-09-21T06:04:29.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: DB pointer 0x55b816490000 2024-09-21T06:04:29.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T06:04:29.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T06:04:29.291 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** DB Stats ** 2024-09-21T06:04:29.291 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:04:29.291 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T06:04:29.291 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:04:29.291 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:04:29.291 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T06:04:29.291 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:04:29.291 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:04:29.291 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-21T06:04:29.291 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: L0 2/0 90.75 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: Sum 2/0 90.75 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 120.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:04:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative compaction: 0.00 GB write, 25.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval compaction: 0.00 GB write, 25.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache BinnedLRUCache@0x55b81168b610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: starting mon.a rank 0 at public addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] at bind addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: mon.a@-1(???) e1 preinit fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:04:29.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: mon.a@-1(???).mds e1 new map 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: mon.a@-1(???).mds e1 print_map 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout: e1 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout: btime 2024-09-21T06:04:24:293546+0000 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout: legacy client fscid: -1 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout: No filesystems configured 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T06:04:29.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 bash[31208]: 6ceb763afa3c479746e0c332d336fa89a4ab399f9c9303235dfcf01e622ac614 2024-09-21T06:04:29.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 systemd[1]: Started Ceph mon.a for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:04:29.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T06:04:29.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: monmap epoch 1 2024-09-21T06:04:29.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:04:29.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: last_changed 2024-09-21T06:04:21.188902+0000 2024-09-21T06:04:29.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: created 2024-09-21T06:04:21.188902+0000 2024-09-21T06:04:29.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: min_mon_release 19 (squid) 2024-09-21T06:04:29.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: election_strategy: 1 2024-09-21T06:04:29.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-21T06:04:29.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: fsmap 2024-09-21T06:04:29.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T06:04:29.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:29 smithi003 ceph-mon[31227]: mgrmap e1: no daemons active 2024-09-21T06:04:31.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:30 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1933140761' entity='client.admin' 2024-09-21T06:04:35.835 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T06:04:36.072 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:04:36.072 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mgr... 2024-09-21T06:04:36.073 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T06:04:36.073 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T06:04:37.115 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.x 2024-09-21T06:04:37.115 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T06:04:37.615 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-38d3c75c-77df-11ef-baf5-efdc52797490.target.wants/ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-38d3c75c-77df-11ef-baf5-efdc52797490@.service. 2024-09-21T06:04:37.990 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:37 smithi003 podman[31443]: 2024-09-21 06:04:37.749757792 +0000 UTC m=+0.016482463 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:04:37.991 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:37 smithi003 podman[31443]: 2024-09-21 06:04:37.989442153 +0000 UTC m=+0.256166807 container create 46bb2d2a398fad101e868f7080948ec841c9eefb9b0c50466bc06f13e0b4867a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:04:38.233 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:04:38.233 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-09-21T06:04:38.241 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:04:38.241 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-21T06:04:38.241 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-09-21T06:04:38.241 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T06:04:38.252 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:04:38.252 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-09-21T06:04:38.261 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:04:38.261 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-21T06:04:38.261 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-09-21T06:04:38.262 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T06:04:38.262 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr to start... 2024-09-21T06:04:38.262 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr... 2024-09-21T06:04:38.340 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:38 smithi003 podman[31443]: 2024-09-21 06:04:38.15830414 +0000 UTC m=+0.425028806 container init 46bb2d2a398fad101e868f7080948ec841c9eefb9b0c50466bc06f13e0b4867a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:04:38.340 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:38 smithi003 podman[31443]: 2024-09-21 06:04:38.161082093 +0000 UTC m=+0.427806747 container start 46bb2d2a398fad101e868f7080948ec841c9eefb9b0c50466bc06f13e0b4867a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:04:38.340 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:38 smithi003 bash[31443]: 46bb2d2a398fad101e868f7080948ec841c9eefb9b0c50466bc06f13e0b4867a 2024-09-21T06:04:38.340 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:38 smithi003 systemd[1]: Started Ceph mgr.x for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:04:39.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:38 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/395373497' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T06:04:39.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-21T06:04:39.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:04:39.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "38d3c75c-77df-11ef-baf5-efdc52797490", 2024-09-21T06:04:39.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T06:04:39.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T06:04:39.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T06:04:39.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T06:04:39.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:39.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T06:04:39.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T06:04:39.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-09-21T06:04:39.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:04:39.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T06:04:39.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T06:04:39.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:04:39.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-21T06:04:39.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T06:04:39.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:04:39.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T06:04:39.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T06:04:39.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:39.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T06:04:39.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:04:39.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T06:04:39.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T06:04:39.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T06:04:39.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T06:04:39.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T06:04:39.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T06:04:39.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:39.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T06:04:39.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T06:04:39.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T06:04:24:293546+0000", 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:39.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T06:04:24.294537+0000", 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T06:04:39.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:04:39.354 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (1/15)... 2024-09-21T06:04:42.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:41 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3089203506' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T06:04:42.444 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-21T06:04:42.444 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:04:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "38d3c75c-77df-11ef-baf5-efdc52797490", 2024-09-21T06:04:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T06:04:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T06:04:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T06:04:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T06:04:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T06:04:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T06:04:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-09-21T06:04:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:04:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T06:04:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T06:04:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T06:04:42.446 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T06:04:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T06:04:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T06:04:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T06:04:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T06:04:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T06:04:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T06:04:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T06:04:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T06:04:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T06:04:24:293546+0000", 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T06:04:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T06:04:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:04:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:04:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T06:04:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:04:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T06:04:24.294537+0000", 2024-09-21T06:04:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:04:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T06:04:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:04:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (2/15)... 2024-09-21T06:04:43.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:43 smithi003 ceph-mon[31227]: Activating manager daemon x 2024-09-21T06:04:43.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:43 smithi003 ceph-mon[31227]: mgrmap e2: x(active, starting, since 0.0035682s) 2024-09-21T06:04:43.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:43 smithi003 ceph-mon[31227]: from='mgr.14100 172.21.15.3:0/3600896335' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:04:43.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:43 smithi003 ceph-mon[31227]: from='mgr.14100 172.21.15.3:0/3600896335' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:04:43.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:43 smithi003 ceph-mon[31227]: from='mgr.14100 172.21.15.3:0/3600896335' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:04:43.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:43 smithi003 ceph-mon[31227]: from='mgr.14100 172.21.15.3:0/3600896335' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:04:43.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:43 smithi003 ceph-mon[31227]: from='mgr.14100 172.21.15.3:0/3600896335' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:04:43.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:43 smithi003 ceph-mon[31227]: Manager daemon x is now available 2024-09-21T06:04:43.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:43 smithi003 ceph-mon[31227]: from='mgr.14100 172.21.15.3:0/3600896335' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:04:43.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:43 smithi003 ceph-mon[31227]: from='mgr.14100 172.21.15.3:0/3600896335' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:04:43.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:43 smithi003 ceph-mon[31227]: from='mgr.14100 172.21.15.3:0/3600896335' entity='mgr.x' 2024-09-21T06:04:43.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:43 smithi003 ceph-mon[31227]: from='mgr.14100 172.21.15.3:0/3600896335' entity='mgr.x' 2024-09-21T06:04:43.743 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:43 smithi003 ceph-mon[31227]: from='mgr.14100 172.21.15.3:0/3600896335' entity='mgr.x' 2024-09-21T06:04:45.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:45 smithi003 ceph-mon[31227]: mgrmap e3: x(active, since 1.00737s) 2024-09-21T06:04:46.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:46 smithi003 ceph-mon[31227]: mgrmap e4: x(active, since 2s) 2024-09-21T06:04:47.899 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:47 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2755258204' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T06:04:48.188 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-21T06:04:48.188 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:04:48.188 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "38d3c75c-77df-11ef-baf5-efdc52797490", 2024-09-21T06:04:48.188 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T06:04:48.188 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T06:04:48.188 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T06:04:48.188 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T06:04:48.188 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:48.188 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:04:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T06:04:48.190 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T06:04:48.190 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T06:04:48.193 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T06:04:48.193 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T06:04:48.193 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T06:04:48.193 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:48.193 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T06:04:48.193 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T06:04:48.193 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T06:04:48.193 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T06:04:48.193 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T06:04:24:293546+0000", 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T06:04:48.194 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T06:04:24.294537+0000", 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:04:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:mgr is available 2024-09-21T06:04:49.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:48 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3785270578' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T06:04:49.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-21T06:04:49.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T06:04:49.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:04:49.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T06:04:49.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-09-21T06:04:49.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T06:04:49.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T06:04:49.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T06:04:49.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T06:04:49.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-21T06:04:49.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T06:04:49.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T06:04:49.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-21T06:04:49.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T06:04:49.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T06:04:49.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T06:04:49.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T06:04:49.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T06:04:49.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T06:04:49.339 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling cephadm module... 2024-09-21T06:04:50.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:50 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3755827807' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T06:04:50.341 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:50 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: ignoring --setuser ceph since I am not root 2024-09-21T06:04:50.341 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:50 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: ignoring --setgroup ceph since I am not root 2024-09-21T06:04:51.334 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:51 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3755827807' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T06:04:51.334 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:51 smithi003 ceph-mon[31227]: mgrmap e5: x(active, since 6s) 2024-09-21T06:04:52.330 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:52 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2528342315' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T06:04:52.333 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:04:52.333 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T06:04:52.333 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T06:04:52.333 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T06:04:52.333 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T06:04:52.333 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:04:52.333 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2024-09-21T06:04:52.334 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 5... 2024-09-21T06:04:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:55 smithi003 ceph-mon[31227]: Active manager daemon x restarted 2024-09-21T06:04:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:55 smithi003 ceph-mon[31227]: Activating manager daemon x 2024-09-21T06:04:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:55 smithi003 ceph-mon[31227]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T06:04:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:55 smithi003 ceph-mon[31227]: mgrmap e6: x(active, starting, since 0.00359315s) 2024-09-21T06:04:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:55 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:04:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:55 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:04:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:55 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:04:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:55 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:04:55.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:55 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:04:55.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:55 smithi003 ceph-mon[31227]: Manager daemon x is now available 2024-09-21T06:04:56.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:04:56.848 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T06:04:56.848 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T06:04:56.848 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:04:56.848 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 5 is available 2024-09-21T06:04:56.848 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying orchestrator module is enabled... 2024-09-21T06:04:57.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:57 smithi003 ceph-mon[31227]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T06:04:57.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:57 smithi003 ceph-mon[31227]: mgrmap e7: x(active, since 1.00554s) 2024-09-21T06:04:57.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:57 smithi003 ceph-mon[31227]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T06:04:58.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:58 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2969181062' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T06:04:58.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:58 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:04:58.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:58 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:04:58.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:58 smithi003 ceph-mon[31227]: Found migration_current of "None". Setting to last migration. 2024-09-21T06:04:58.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:58 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:04:58.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:58 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:04:58.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:58 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:58.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:58 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:58.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:58 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:04:58.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:58 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:04:58.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T06:04:58.986 INFO:teuthology.orchestra.run.smithi003.stdout:Setting orchestrator backend to cephadm... 2024-09-21T06:04:59.239 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:04:58.999+0000 7fcf6b330640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:06:04:59] ENGINE Error in HTTPServer.serve 2024-09-21T06:04:59.239 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: Traceback (most recent call last): 2024-09-21T06:04:59.239 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:04:59.239 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: self._connections.run(self.expiration_interval) 2024-09-21T06:04:59.239 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:04:59.239 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: self._run(expiration_interval) 2024-09-21T06:04:59.239 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:04:59.239 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:04:59.239 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:04:59.239 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:04:59.240 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:04:59.240 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: s = self.context.wrap_socket( 2024-09-21T06:04:59.240 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:04:59.240 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: return self.sslsocket_class._create( 2024-09-21T06:04:59.240 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:04:59.240 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: self.do_handshake() 2024-09-21T06:04:59.240 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:04:59.240 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: self._sslobj.do_handshake() 2024-09-21T06:04:59.240 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:04:59.240 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:04:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2969181062' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T06:04:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-mon[31227]: mgrmap e8: x(active, since 3s) 2024-09-21T06:04:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-mon[31227]: [21/Sep/2024:06:04:58] ENGINE Bus STARTING 2024-09-21T06:04:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-mon[31227]: [21/Sep/2024:06:04:58] ENGINE Serving on https://172.21.15.3:7150 2024-09-21T06:04:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-mon[31227]: [21/Sep/2024:06:04:59] ENGINE Error in HTTPServer.serve 2024-09-21T06:04:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout: Traceback (most recent call last): 2024-09-21T06:04:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:04:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:04:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:04:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._run(expiration_interval) 2024-09-21T06:04:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:04:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:04:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:04:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:04:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:04:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout: s = self.context.wrap_socket( 2024-09-21T06:04:59.592 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:04:59.592 INFO:journalctl@ceph.mon.a.smithi003.stdout: return self.sslsocket_class._create( 2024-09-21T06:04:59.592 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:04:59.592 INFO:journalctl@ceph.mon.a.smithi003.stdout: self.do_handshake() 2024-09-21T06:04:59.592 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:04:59.592 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._sslobj.do_handshake() 2024-09-21T06:04:59.592 INFO:journalctl@ceph.mon.a.smithi003.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:04:59.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-mon[31227]: [21/Sep/2024:06:04:59] ENGINE Serving on http://172.21.15.3:8765 2024-09-21T06:04:59.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-mon[31227]: [21/Sep/2024:06:04:59] ENGINE Bus STARTED 2024-09-21T06:04:59.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:04:59 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:00.501 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:00 smithi003 ceph-mon[31227]: mgrmap e9: x(active, since 4s) 2024-09-21T06:05:00.501 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:00 smithi003 ceph-mon[31227]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:00.501 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:00 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:05:00.501 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:00 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:01.463 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:01 smithi003 ceph-mon[31227]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:01.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T06:05:01.466 INFO:teuthology.orchestra.run.smithi003.stdout:Generating ssh key... 2024-09-21T06:05:02.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-mon[31227]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:02.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-mon[31227]: Generating ssh key... 2024-09-21T06:05:02.769 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: Generating public/private rsa key pair. 2024-09-21T06:05:02.769 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: Your identification has been saved in /tmp/tmpjkptm_6a/key 2024-09-21T06:05:02.769 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: Your public key has been saved in /tmp/tmpjkptm_6a/key.pub 2024-09-21T06:05:02.769 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: The key fingerprint is: 2024-09-21T06:05:02.770 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: SHA256:2P6RrrViEvPRsPGP3TDW9758sD3oMmpY3eYLt42ZmE8 ceph-38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:05:02.770 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: The key's randomart image is: 2024-09-21T06:05:02.770 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: +---[RSA 3072]----+ 2024-09-21T06:05:02.770 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: | | 2024-09-21T06:05:02.770 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: | | 2024-09-21T06:05:02.770 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: | | 2024-09-21T06:05:02.770 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: | oo | 2024-09-21T06:05:02.770 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: | . S*. .. | 2024-09-21T06:05:02.770 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: | o.o.oo+oo .| 2024-09-21T06:05:02.770 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: | ++.+*oE.=.| 2024-09-21T06:05:02.770 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: | ..+=.=OoX.+| 2024-09-21T06:05:02.770 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: | oo+=o=O.==| 2024-09-21T06:05:02.770 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:02 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: +----[SHA256]-----+ 2024-09-21T06:05:03.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:03 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:05:03.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:03 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:05:04.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXWbfmCbiHqU5+hJRPEeLlPFVsxMW4jsw/87KZ3y6DzvsdAddIuzg0vektCJvKaT31hyDK7qBB4bEFPfR9NzQjPUjcfjFNQPIPJaPtRyOeOH3GcrZ/Agj749WYi5jPK0uVVnpFKiDWYfbiGU3JNXde6ox+5f41X8uM/dQiyXn7ZCvQOsWyrXPl37FxI/RXgCpr+nJNNbUxzxi0QzETeeJZ4F6HOwgVJjXTH7Yn3HXMHMJ3dqnh90NTgBTM4BS2GAL41DgeL0skYupLSYMET+xs0f/87og16Oyu6LS1cuFgV1rgb/ZvT32ywTy1NfAECKVp2ZpfHWfSRCinD/PvUKs578jLrqAajLBN4bdg6iQk5W0oceWwyqC+66bGHfXGmzLI9AawJQnTu1wlr6bUsqyX7chJJpF15Gq40xf2C4uao30qAzKIGchJe+Z0FsOHFUaOHT4VyXk9dQSfzGbfJCNdljtl0RcJoSzEAcpMgFyt5huGKUcbOqZBSFCwvYj+43M= ceph-38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:05:04.120 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:05:04.120 INFO:teuthology.orchestra.run.smithi003.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T06:05:04.125 INFO:teuthology.orchestra.run.smithi003.stdout:Adding host smithi003... 2024-09-21T06:05:04.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:04 smithi003 ceph-mon[31227]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:05.693 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:05 smithi003 ceph-mon[31227]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi003", "addr": "172.21.15.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:06.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:06 smithi003 ceph-mon[31227]: Deploying cephadm binary to smithi003 2024-09-21T06:05:09.011 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Added host 'smithi003' with addr '172.21.15.3' 2024-09-21T06:05:09.011 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mon service... 2024-09-21T06:05:09.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:09 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:05:09.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:09 smithi003 ceph-mon[31227]: Added host smithi003 2024-09-21T06:05:09.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:09 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:10.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T06:05:10.767 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mgr service... 2024-09-21T06:05:10.897 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:10 smithi003 ceph-mon[31227]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:10.897 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:10 smithi003 ceph-mon[31227]: Saving service mon spec with placement count:5 2024-09-21T06:05:10.897 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:10 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:05:12.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T06:05:12.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:12 smithi003 ceph-mon[31227]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:12.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:12 smithi003 ceph-mon[31227]: Saving service mgr spec with placement count:2 2024-09-21T06:05:12.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:12 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:05:12.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:12 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:05:12.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:12 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:05:13.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:13 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2940967297' entity='client.admin' 2024-09-21T06:05:13.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:13 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:05:15.083 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling the dashboard module... 2024-09-21T06:05:15.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:15 smithi003 ceph-mon[31227]: from='mgr.14118 172.21.15.3:0/713632573' entity='mgr.x' 2024-09-21T06:05:15.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:15 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2817951368' entity='client.admin' 2024-09-21T06:05:16.631 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:16 smithi003 ceph-mon[31227]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:16.632 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:16 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/255583956' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T06:05:16.632 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:16 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: ignoring --setuser ceph since I am not root 2024-09-21T06:05:16.632 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:16 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: ignoring --setgroup ceph since I am not root 2024-09-21T06:05:17.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:17 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/255583956' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T06:05:17.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:17 smithi003 ceph-mon[31227]: mgrmap e10: x(active, since 21s) 2024-09-21T06:05:18.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:18 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1143002334' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T06:05:18.595 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:05:18.595 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 10, 2024-09-21T06:05:18.595 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T06:05:18.595 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T06:05:18.595 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T06:05:18.595 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:05:18.595 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2024-09-21T06:05:18.595 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 10... 2024-09-21T06:05:22.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:21 smithi003 ceph-mon[31227]: Active manager daemon x restarted 2024-09-21T06:05:22.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:21 smithi003 ceph-mon[31227]: Activating manager daemon x 2024-09-21T06:05:22.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:21 smithi003 ceph-mon[31227]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T06:05:22.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:21 smithi003 ceph-mon[31227]: mgrmap e11: x(active, starting, since 0.00434409s) 2024-09-21T06:05:22.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:21 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:05:22.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:21 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:05:22.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:21 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:05:22.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:21 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:05:22.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:21 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:05:22.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:21 smithi003 ceph-mon[31227]: Manager daemon x is now available 2024-09-21T06:05:23.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:22 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:23.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:22 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:05:23.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:22 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:05:23.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:22 smithi003 ceph-mon[31227]: mgrmap e12: x(active, since 1.00732s) 2024-09-21T06:05:23.627 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:05:23.627 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 12, 2024-09-21T06:05:23.627 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T06:05:23.627 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:05:23.627 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 10 is available 2024-09-21T06:05:23.627 INFO:teuthology.orchestra.run.smithi003.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T06:05:23.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:23 smithi003 ceph-mon[31227]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T06:05:23.878 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:23 smithi003 ceph-mon[31227]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T06:05:24.195 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:05:24.054+0000 7f46f68b9640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:06:05:24] ENGINE Error in HTTPServer.serve 2024-09-21T06:05:24.195 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: Traceback (most recent call last): 2024-09-21T06:05:24.195 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:05:24.196 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: self._connections.run(self.expiration_interval) 2024-09-21T06:05:24.196 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:05:24.196 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: self._run(expiration_interval) 2024-09-21T06:05:24.196 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:05:24.196 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:05:24.196 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:05:24.196 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:05:24.196 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:05:24.196 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: s = self.context.wrap_socket( 2024-09-21T06:05:24.196 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:05:24.196 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: return self.sslsocket_class._create( 2024-09-21T06:05:24.197 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:05:24.197 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: self.do_handshake() 2024-09-21T06:05:24.197 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:05:24.197 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: self._sslobj.do_handshake() 2024-09-21T06:05:24.197 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:05:24.197 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:05:25.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-mon[31227]: mgrmap e13: x(active, since 2s) 2024-09-21T06:05:25.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-mon[31227]: [21/Sep/2024:06:05:23] ENGINE Bus STARTING 2024-09-21T06:05:25.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-mon[31227]: [21/Sep/2024:06:05:24] ENGINE Serving on https://172.21.15.3:7150 2024-09-21T06:05:25.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-mon[31227]: [21/Sep/2024:06:05:24] ENGINE Error in HTTPServer.serve 2024-09-21T06:05:25.090 INFO:journalctl@ceph.mon.a.smithi003.stdout: Traceback (most recent call last): 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._run(expiration_interval) 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: s = self.context.wrap_socket( 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: return self.sslsocket_class._create( 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: self.do_handshake() 2024-09-21T06:05:25.091 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:05:25.092 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._sslobj.do_handshake() 2024-09-21T06:05:25.092 INFO:journalctl@ceph.mon.a.smithi003.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:05:25.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-mon[31227]: [21/Sep/2024:06:05:24] ENGINE Serving on http://172.21.15.3:8765 2024-09-21T06:05:25.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-mon[31227]: [21/Sep/2024:06:05:24] ENGINE Bus STARTED 2024-09-21T06:05:25.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-mon[31227]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:25.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:24 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:26.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:26 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:27.076 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+jCCAuKgAwIBAgIUBzlytVD1kq5cfd29E7z04vutgiIwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTA2MDUyNloXDTM0\nMDkyMjA2MDUyNlowFjEUMBIGA1UEAwwLMTcyLjIxLjE1LjMwggIiMA0GCSqGSIb3\nDQEBAQUAA4ICDwAwggIKAoICAQDaxkXjxnR7nsZfwQR+U06uAfos2nacpE+JzCEb\njge1ZgETuBF9rH25Ncci5c1I9AASmNcxKLQeAYPwIHb5hoz0s8nWM2joETXrAvXR\nwOlp4YfOigqa3+rh4psfposjiT76ZwSIfLBuwa1ZGbzKbIkGi4gbDS0dKHITUv7k\noaQFrhsOKXss0KiiK+RDxQwUmkTdjUjHEXD6CGEM4QJXjWg1nO7aAh744ntt3rIu\nesHmeGDtAJ47FIly3K+zK2wFO0RSa59PXF4EhzGl3Ybax1P6GKZ/Ll9j4ZVqXqAj\nQZ2Hdnrr0HYU9Dzqg2bYLM9qXyNuZ3nVNs+ezmg1PKpOjexumTHdlUP5xwujFoi5\nsx0omCyUx7TMHO0eqlqGCBy8GLEpbSFYd0yDE5ZfFVNXlcKYCYB0hb1Am5seuVia\n/kPzD2uTUsYoYvFT6f0VdQoMlSyUHy2nVaeV4xqUwz9vgbZ32L12VSzt7QIjYD58\nHPn0zJtQYykc/ndkQmcnqeWkP+xAe9p2Av0nbYEWykX67qohUFmAv9bfjjkIfEQ8\nszmANYbatTouXAvpovSm1+5o81o3bj7p5Wz+lP5KLhsgkL8JB1JQO2m4Zl+De6YB\n0OU7tnW9XeOQzhIAUNPwJlWYvvTmpz8VW9kfS+wCzD9H2p8z3qP6B4GrjWcgQCBi\nDahsWQIDAQABoz8wPTAtBgNVHREEJjAkgglzbWl0aGkwMDOCEWRhc2hib2FyZF9z\nZXJ2ZXJzhwSsFQ8DMAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQELBQADggIBAHwy\nQzrVmVt3zKH4GW+a916D/bxLqvfOJa/6MZyFgvgj2TQNY8hatbfXZbDr/3mFRO1V\nJMRg1MiyfDxE1/4RbcXlQbV+0zjSw+DOvKNPoEf7Il33h+bJnxFzxaNj9KKSL4C9\nRBXg+s80zwppHHQW9vQlD9EBJPnfx6o1d7Du/kkJ6HNiXWp5b273xqpAng1Si1+p\n5+JQFO/TN3gFLva2iPj0GLkxuB513ZNCXJuFHCzxJ/ZOtbMThapI9CnxQTDnFmMU\nN1ddxVOuXSPXn9OTpguN+gURf8II4m3JwR1yIDm9vt9UBdsCO4SEibNT4KP5kZVN\n7Ea5S8KzfDOy+NF32D3e7OBT99AdIDFK0FaG1xaq/+uX0WPw5D84djTlcLcdaAhT\noQOwljQ6URcbm+jiERXPoWC4Uv5dGapUigGNHx085HI4EugbU2cLoav0OVgxHgnG\n97hATY4dRdJdGtk0KcxCZj9WpYp1bhhQrrlKEZfBSki9tcU2Hhl1j5W6BgwpDsT7\nPGSPCi3YiZRlA/YfviGu5HbUfHPx49Am76D6ITFHYc9Kvn0vPFHKYncqbuQ6PRcp\nTyebaEqNXJWUg2zY5h2rhZboFnN2PbLrzSzIlF5ttkXuezeBOgZrSAl47asSzxrK\nndk648J4lp8T3lPz3Ixd9ff4P6WPvsTPrwI66xg6\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJJwIBAAKCAgEA2sZF48Z0e57GX8EEflNOrgH6LNp2nKRPicwhG44HtWYBE7gR\nfax9uTXHIuXNSPQAEpjXMSi0HgGD8CB2+YaM9LPJ1jNo6BE16wL10cDpaeGHzooK\nmt/q4eKbH6aLI4k++mcEiHywbsGtWRm8ymyJBouIGw0tHShyE1L+5KGkBa4bDil7\nLNCooivkQ8UMFJpE3Y1IxxFw+ghhDOECV41oNZzu2gIe+OJ7bd6yLnrB5nhg7QCe\nOxSJctyvsytsBTtEUmufT1xeBIcxpd2G2sdT+himfy5fY+GVal6gI0Gdh3Z669B2\nFPQ86oNm2CzPal8jbmd51TbPns5oNTyqTo3sbpkx3ZVD+ccLoxaIubMdKJgslMe0\nzBztHqpahggcvBixKW0hWHdMgxOWXxVTV5XCmAmAdIW9QJubHrlYmv5D8w9rk1LG\nKGLxU+n9FXUKDJUslB8tp1WnleMalMM/b4G2d9i9dlUs7e0CI2A+fBz59MybUGMp\nHP53ZEJnJ6nlpD/sQHvadgL9J22BFspF+u6qIVBZgL/W3445CHxEPLM5gDWG2rU6\nLlwL6aL0ptfuaPNaN24+6eVs/pT+Si4bIJC/CQdSUDtpuGZfg3umAdDlO7Z1vV3j\nkM4SAFDT8CZVmL705qc/FVvZH0vsAsw/R9qfM96j+geBq41nIEAgYg2obFkCAwEA\nAQKCAgAFqqQNycqjlxG5UjUNFphPnoJ93ey7ku/yeESVQ6EEfE9j1N0zHaAmB3QQ\nfdZK4PMNiELupYjAzDZAJn8eaHNHNU7xiuRZOnTkkO+QDsic8vgDDAVJTKp3TljJ\nQZWSBpCIiNmPtXcWb1LPn4CJ17TW5A9S8Azul+kYggpTHrKyaPePbtifaQotJSjk\n7nopIHPPL0UmztdXypbE8CEQ5ntABEAVin+F38vlpSKFXK2gDBcqnFz818PVo0yz\nvdiu7AGRhvMZ6aEPyfapvqqrCQWuUSwi2C8lVab6mdfZAe6UHygA6qXcVhUr19p2\nl/aJYUx4JBr+QEyxZtRBKMCah0eL64uHS1YrQBSOBrx9JbBJ5qooMt6RKkQGjxRe\nLOgQ9oN8RKoi5mpxMJnAMsE9ydUttzVfhUSAwdzP5FyjuRDZJpCaarHHDUKlJ7il\nd3toIRyHrw+G5Tm/jzXxA5U6Vd/5bgFvFz6jH6H5CbVfZYZ1XBXqKyq/hk7WBhpg\n+puP11cmSunZYbgzg62CkyOEc9saNpEMUNXzHSWXxk3WNWHwDBQlr/rvuvIk9RsJ\nE2DPcGcYaK4VMM8QV5mXyftzJpFBWilxJPT4EiBIARQTgg/8J1dRlLiQerupiCcb\nk9ZTnRA+eM7oOkE4UBMri4Hwxuv0+fevkuX9JmboAoMu9Iey1wKCAQEA9wnqtrZR\nHE984DPaP5KRtt7nA5A2qcuN2PCxIMUphZKoBom3UFVpAKgKie2Ic6YG2Gr8J2up\nR+4RvnMsXmnc00Hnh+fhBH7jlMyy3JyT0ciblRhaYZLyRgVeYWk2jiYb4cBFMMuD\nsYgBHsBLF1QlX2R0pSsSDUAjzFyxuPYv5Q9qftdXb25nPHo6oiZ60y6V2ffYRFbS\nNukmWAo/G2iotp69txvH9J4J9n5FxdQiQDYi7h/soxa5AUkQq56pdUUfWhcjjhhY\neHLtf7zcqMszdrJc2PTxfj4wrcoSTvg6PR+J6tvAM30Kwn200KJlupUQ9cDJnKGl\n9xZzE1tyorq9cwKCAQEA4rXimiqjk8QM+6oFzkNkvifr5Vs/SXqR7uhl8Qhqv7D5\nkt6Ta67ZEYHaT+mI/tO394lVSKyGm3N6cSBi1Iyc/zJ/CEtG/HdaVavexKJQV+oS\nY77koVbz2cikeEVhuuNVjvNf8RIb2If7i7R3ndY81SgKZ9El2nbrQzuJeRxUBVrR\nTb/jmW08U48A20V9o1qoZHiZV/HFPUQhzIOCMoZaK44KH0c836TIfJTQwjDGQL8Q\n1unZKA7D2w0DSDzNq3BF6NUHqx0wtiWSiVu/ahUF3MK1y93G2scHFmMcgtmk4Gmt\ntoZQrzEW1HYg8HR/qr/gtyvMRFVrjRt0LlLaMln8AwKCAQAN2F9wOA2ZsZbFFgLg\nVz2Vay61U2rM31ym0O6McKBgHRUPsL42YWMOJ1zX2BkyqhZuIKjbKLyHDH+wFdJh\nxDsdF+96/XLuBpKIr2RC2v+p3T1jAMdn+o1bgUeF3BizpM0hPN5j2rQZyg6OA6bx\nusSRcRbcfup3FaYr+PH49MtAYjnMvBLpat2cVRleJxvxJAY//hT/14HWlACTJJfe\n2GHEQmdNkOAHBKh4SSFt8fKteJtCj0V66nYZ7Vdi1vSWXvZf3TYf65thvMc0xofz\nadgLifmMK0YxlhiyVMXr6RuSe2cGY6SE/jgRKGxu2PssltgpfdGO1rxvebGfXxuw\npRijAoIBADG3eW4dk6nAAKjf7E4DcYBR+OVfUU2n14Dw7gohsmc/RDv1G5ctzOSN\n+XY03Joli954+37odsx4Uo2kcIf0HinIxRrohzSuZ90Jx9yPS9c7ubSuFC1diDRm\nT8sgTAJUAFtM++k982VUzNEiTmkKpmSIV1/U+dwi4AzqbYdlsvdJp0TJo/vtfKFG\nZwWvA5JjOzJtUBN9Laf13QOgg1jZJcUPbSDW2n6hAGZxrCYp1BA21laLCdhDBKhw\n/htFWkUSZgJdVUUtLZWEP9JnkF6PJa339S40AHvR0bgTHBmF6SZdhn7UjbUdqsgE\n78xLgAPN+FWzASSrJt7W+m9+M+N+4EkCggEAIqmdQhsyaz1+PMrTAs4uwR1PFbuT\nCBnCnQWJkRpKwSnNQYtgZmJl73NvQjXGrXqWCOUCK6Um/KMkmXw09C4f2GGiWwGE\n2mEulR5JCvnIu8EgHu4ScDbBvFyZdSCnPYWJPMWBWt87wfvYfKeVzzVUFVZGzKv7\nWQf1RNd967J4EFBbbKRPHk0Nph97xbm4WCmwclIlK+ombYTBjRvELANDpQQaVE6A\nMi/CWeK29yWO3x3Ds4Jv8Haplq8DhsyKgShzGY+QvqoCOIsDpR5mPG34klvB6bHp\n+R9eTn7tmsfJdP2UNUwXJTvxumgD6VIC8wjcc5wcy8OJIs3g8lPPEw+SDQ==\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T06:05:28.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T06:05:28.950 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:28 smithi003 ceph-mon[31227]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:28.950 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:28 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:29.505 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T06:05:29.505 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial admin user... 2024-09-21T06:05:30.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:29 smithi003 ceph-mon[31227]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:30.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:29 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:30.920 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$0NW/jS6a9CtJB53TN7tMDeN4bHZvEK7dPtldISlY3lNga9u6yLrx6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726898730, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T06:05:30.921 INFO:teuthology.orchestra.run.smithi003.stdout:Fetching dashboard port number... 2024-09-21T06:05:31.564 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:31 smithi003 ceph-mon[31227]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:31.564 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:31 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:32.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T06:05:32.311 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:05:32.311 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-09-21T06:05:32.319 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:05:32.319 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-21T06:05:32.319 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-09-21T06:05:32.319 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T06:05:32.321 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph Dashboard is now available at: 2024-09-21T06:05:32.321 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:05:32.321 INFO:teuthology.orchestra.run.smithi003.stdout: URL: https://smithi003.front.sepia.ceph.com:8443/ 2024-09-21T06:05:32.321 INFO:teuthology.orchestra.run.smithi003.stdout: User: admin 2024-09-21T06:05:32.321 INFO:teuthology.orchestra.run.smithi003.stdout: Password: jgrca8as60 2024-09-21T06:05:32.321 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:05:32.322 INFO:teuthology.orchestra.run.smithi003.stdout:Saving cluster configuration to /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config directory 2024-09-21T06:05:32.572 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:32 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/976034132' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T06:05:33.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T06:05:33.676 INFO:teuthology.orchestra.run.smithi003.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T06:05:33.676 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:05:33.676 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:05:33.676 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:05:33.676 INFO:teuthology.orchestra.run.smithi003.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T06:05:33.676 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:05:33.676 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T06:05:33.676 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:05:33.676 INFO:teuthology.orchestra.run.smithi003.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T06:05:33.676 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:05:33.677 INFO:teuthology.orchestra.run.smithi003.stdout: ceph telemetry on 2024-09-21T06:05:33.677 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:05:33.677 INFO:teuthology.orchestra.run.smithi003.stdout:For more information see: 2024-09-21T06:05:33.677 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:05:33.677 INFO:teuthology.orchestra.run.smithi003.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T06:05:33.677 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:05:33.677 INFO:teuthology.orchestra.run.smithi003.stdout:Bootstrap complete. 2024-09-21T06:05:33.725 INFO:tasks.cephadm:Fetching config... 2024-09-21T06:05:33.725 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:05:33.725 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T06:05:33.742 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T06:05:33.742 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:05:33.743 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T06:05:33.799 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T06:05:33.799 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:05:33.799 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T06:05:33.866 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T06:05:33.866 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:05:33.866 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T06:05:33.922 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T06:05:33.922 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXWbfmCbiHqU5+hJRPEeLlPFVsxMW4jsw/87KZ3y6DzvsdAddIuzg0vektCJvKaT31hyDK7qBB4bEFPfR9NzQjPUjcfjFNQPIPJaPtRyOeOH3GcrZ/Agj749WYi5jPK0uVVnpFKiDWYfbiGU3JNXde6ox+5f41X8uM/dQiyXn7ZCvQOsWyrXPl37FxI/RXgCpr+nJNNbUxzxi0QzETeeJZ4F6HOwgVJjXTH7Yn3HXMHMJ3dqnh90NTgBTM4BS2GAL41DgeL0skYupLSYMET+xs0f/87og16Oyu6LS1cuFgV1rgb/ZvT32ywTy1NfAECKVp2ZpfHWfSRCinD/PvUKs578jLrqAajLBN4bdg6iQk5W0oceWwyqC+66bGHfXGmzLI9AawJQnTu1wlr6bUsqyX7chJJpF15Gq40xf2C4uao30qAzKIGchJe+Z0FsOHFUaOHT4VyXk9dQSfzGbfJCNdljtl0RcJoSzEAcpMgFyt5huGKUcbOqZBSFCwvYj+43M= ceph-38d3c75c-77df-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T06:05:33.997 INFO:teuthology.orchestra.run.smithi003.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXWbfmCbiHqU5+hJRPEeLlPFVsxMW4jsw/87KZ3y6DzvsdAddIuzg0vektCJvKaT31hyDK7qBB4bEFPfR9NzQjPUjcfjFNQPIPJaPtRyOeOH3GcrZ/Agj749WYi5jPK0uVVnpFKiDWYfbiGU3JNXde6ox+5f41X8uM/dQiyXn7ZCvQOsWyrXPl37FxI/RXgCpr+nJNNbUxzxi0QzETeeJZ4F6HOwgVJjXTH7Yn3HXMHMJ3dqnh90NTgBTM4BS2GAL41DgeL0skYupLSYMET+xs0f/87og16Oyu6LS1cuFgV1rgb/ZvT32ywTy1NfAECKVp2ZpfHWfSRCinD/PvUKs578jLrqAajLBN4bdg6iQk5W0oceWwyqC+66bGHfXGmzLI9AawJQnTu1wlr6bUsqyX7chJJpF15Gq40xf2C4uao30qAzKIGchJe+Z0FsOHFUaOHT4VyXk9dQSfzGbfJCNdljtl0RcJoSzEAcpMgFyt5huGKUcbOqZBSFCwvYj+43M= ceph-38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:05:34.009 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXWbfmCbiHqU5+hJRPEeLlPFVsxMW4jsw/87KZ3y6DzvsdAddIuzg0vektCJvKaT31hyDK7qBB4bEFPfR9NzQjPUjcfjFNQPIPJaPtRyOeOH3GcrZ/Agj749WYi5jPK0uVVnpFKiDWYfbiGU3JNXde6ox+5f41X8uM/dQiyXn7ZCvQOsWyrXPl37FxI/RXgCpr+nJNNbUxzxi0QzETeeJZ4F6HOwgVJjXTH7Yn3HXMHMJ3dqnh90NTgBTM4BS2GAL41DgeL0skYupLSYMET+xs0f/87og16Oyu6LS1cuFgV1rgb/ZvT32ywTy1NfAECKVp2ZpfHWfSRCinD/PvUKs578jLrqAajLBN4bdg6iQk5W0oceWwyqC+66bGHfXGmzLI9AawJQnTu1wlr6bUsqyX7chJJpF15Gq40xf2C4uao30qAzKIGchJe+Z0FsOHFUaOHT4VyXk9dQSfzGbfJCNdljtl0RcJoSzEAcpMgFyt5huGKUcbOqZBSFCwvYj+43M= ceph-38d3c75c-77df-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T06:05:34.050 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXWbfmCbiHqU5+hJRPEeLlPFVsxMW4jsw/87KZ3y6DzvsdAddIuzg0vektCJvKaT31hyDK7qBB4bEFPfR9NzQjPUjcfjFNQPIPJaPtRyOeOH3GcrZ/Agj749WYi5jPK0uVVnpFKiDWYfbiGU3JNXde6ox+5f41X8uM/dQiyXn7ZCvQOsWyrXPl37FxI/RXgCpr+nJNNbUxzxi0QzETeeJZ4F6HOwgVJjXTH7Yn3HXMHMJ3dqnh90NTgBTM4BS2GAL41DgeL0skYupLSYMET+xs0f/87og16Oyu6LS1cuFgV1rgb/ZvT32ywTy1NfAECKVp2ZpfHWfSRCinD/PvUKs578jLrqAajLBN4bdg6iQk5W0oceWwyqC+66bGHfXGmzLI9AawJQnTu1wlr6bUsqyX7chJJpF15Gq40xf2C4uao30qAzKIGchJe+Z0FsOHFUaOHT4VyXk9dQSfzGbfJCNdljtl0RcJoSzEAcpMgFyt5huGKUcbOqZBSFCwvYj+43M= ceph-38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:05:34.063 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXWbfmCbiHqU5+hJRPEeLlPFVsxMW4jsw/87KZ3y6DzvsdAddIuzg0vektCJvKaT31hyDK7qBB4bEFPfR9NzQjPUjcfjFNQPIPJaPtRyOeOH3GcrZ/Agj749WYi5jPK0uVVnpFKiDWYfbiGU3JNXde6ox+5f41X8uM/dQiyXn7ZCvQOsWyrXPl37FxI/RXgCpr+nJNNbUxzxi0QzETeeJZ4F6HOwgVJjXTH7Yn3HXMHMJ3dqnh90NTgBTM4BS2GAL41DgeL0skYupLSYMET+xs0f/87og16Oyu6LS1cuFgV1rgb/ZvT32ywTy1NfAECKVp2ZpfHWfSRCinD/PvUKs578jLrqAajLBN4bdg6iQk5W0oceWwyqC+66bGHfXGmzLI9AawJQnTu1wlr6bUsqyX7chJJpF15Gq40xf2C4uao30qAzKIGchJe+Z0FsOHFUaOHT4VyXk9dQSfzGbfJCNdljtl0RcJoSzEAcpMgFyt5huGKUcbOqZBSFCwvYj+43M= ceph-38d3c75c-77df-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T06:05:34.105 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXWbfmCbiHqU5+hJRPEeLlPFVsxMW4jsw/87KZ3y6DzvsdAddIuzg0vektCJvKaT31hyDK7qBB4bEFPfR9NzQjPUjcfjFNQPIPJaPtRyOeOH3GcrZ/Agj749WYi5jPK0uVVnpFKiDWYfbiGU3JNXde6ox+5f41X8uM/dQiyXn7ZCvQOsWyrXPl37FxI/RXgCpr+nJNNbUxzxi0QzETeeJZ4F6HOwgVJjXTH7Yn3HXMHMJ3dqnh90NTgBTM4BS2GAL41DgeL0skYupLSYMET+xs0f/87og16Oyu6LS1cuFgV1rgb/ZvT32ywTy1NfAECKVp2ZpfHWfSRCinD/PvUKs578jLrqAajLBN4bdg6iQk5W0oceWwyqC+66bGHfXGmzLI9AawJQnTu1wlr6bUsqyX7chJJpF15Gq40xf2C4uao30qAzKIGchJe+Z0FsOHFUaOHT4VyXk9dQSfzGbfJCNdljtl0RcJoSzEAcpMgFyt5huGKUcbOqZBSFCwvYj+43M= ceph-38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:05:34.118 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T06:05:34.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:34 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/632970930' entity='client.admin' 2024-09-21T06:05:34.406 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:35.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:35 smithi003 ceph-mon[31227]: mgrmap e14: x(active, since 12s) 2024-09-21T06:05:36.264 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T06:05:36.264 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T06:05:36.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:36 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3548293180' entity='client.admin' 2024-09-21T06:05:36.859 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:38.622 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi137 2024-09-21T06:05:38.622 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-21T06:05:38.622 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.conf 2024-09-21T06:05:38.642 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-21T06:05:38.642 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:05:38.700 INFO:tasks.cephadm:Adding host smithi137 to orchestrator... 2024-09-21T06:05:38.700 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch host add smithi137 2024-09-21T06:05:38.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:38.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:38.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:38.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:38.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi003", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:38.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:38.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:38.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: Updating smithi003:/etc/ceph/ceph.conf 2024-09-21T06:05:38.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: Updating smithi003:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:05:38.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:05:38.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: Updating smithi003:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:05:38.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:38.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:38.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:38.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:38.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:38.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:38.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:38 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:40.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:40 smithi003 ceph-mon[31227]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:41.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:41 smithi003 ceph-mon[31227]: Deploying cephadm binary to smithi137 2024-09-21T06:05:42.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:42 smithi003 ceph-mon[31227]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:43.337 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi137' with addr '172.21.15.137' 2024-09-21T06:05:43.934 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T06:05:44.191 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:44.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:44 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:44.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:44 smithi003 ceph-mon[31227]: Added host smithi137 2024-09-21T06:05:44.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:44 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:44.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:44 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:44.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:44 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:44.846 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:05:44.846 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}] 2024-09-21T06:05:45.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:45 smithi003 ceph-mon[31227]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:45.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:45 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:45.412 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2024-09-21T06:05:45.412 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-21T06:05:45.413 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2024-09-21T06:05:45.431 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-21T06:05:45.431 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:05:45.491 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2024-09-21T06:05:45.491 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch host add smithi204 2024-09-21T06:05:45.748 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:46.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:46 smithi003 ceph-mon[31227]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:05:47.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:47 smithi003 ceph-mon[31227]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:47.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:47 smithi003 ceph-mon[31227]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:48.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:48 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:48.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:48 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:48.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:48 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:48.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:48 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:48.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:48 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:48.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:48 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:48.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:48 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:48.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:48 smithi003 ceph-mon[31227]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-21T06:05:48.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:48 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:48.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:48 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:48.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:48 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:49.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:49 smithi003 ceph-mon[31227]: Updating smithi137:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:05:49.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:49 smithi003 ceph-mon[31227]: Deploying cephadm binary to smithi204 2024-09-21T06:05:49.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:49 smithi003 ceph-mon[31227]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:05:49.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:49 smithi003 ceph-mon[31227]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:49.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:49 smithi003 ceph-mon[31227]: Updating smithi137:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:05:50.264 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi204' with addr '172.21.15.204' 2024-09-21T06:05:50.913 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T06:05:51.170 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:51.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:51 smithi003 ceph-mon[31227]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:51.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:51 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:51.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:51 smithi003 ceph-mon[31227]: Added host smithi204 2024-09-21T06:05:51.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:51 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:51.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:51 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:51.588 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:51 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:05:51.881 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2024-09-21T06:05:52.520 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T06:05:52.521 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T06:05:52.776 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:52.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:52 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:52.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:52 smithi003 ceph-mon[31227]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:52.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:52 smithi003 ceph-mon[31227]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:05:53.584 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2024-09-21T06:05:53.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:53 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3764589064' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T06:05:54.167 INFO:tasks.cephadm:Adding mon.a on smithi003 2024-09-21T06:05:54.167 INFO:tasks.cephadm:Adding mon.b on smithi137 2024-09-21T06:05:54.168 INFO:tasks.cephadm:Adding mon.c on smithi204 2024-09-21T06:05:54.168 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi003:172.21.15.3=a;smithi137:172.21.15.137=b;smithi204:172.21.15.204=c' 2024-09-21T06:05:54.424 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T06:05:54.470 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T06:05:55.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:54 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3764589064' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T06:05:55.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:54 smithi003 ceph-mon[31227]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:05:55.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:54 smithi003 ceph-mon[31227]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:55.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:54 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:55.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:54 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:55.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:54 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:55.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:54 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:55.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:54 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:55.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:54 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:55.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:54 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:55.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:54 smithi003 ceph-mon[31227]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-21T06:05:55.233 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2024-09-21T06:05:55.830 DEBUG:teuthology.orchestra.run.smithi137:mon.b> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T06:05:55.833 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T06:05:55.835 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:05:55.835 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:05:56.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: Updating smithi204:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:05:56.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:05:56.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: Updating smithi204:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:05:56.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:56.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:56.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:56.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi137:172.21.15.137=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:56.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi137:172.21.15.137=b;smithi204:172.21.15.204=c;count:3 2024-09-21T06:05:56.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:56.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:56.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:56.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:56.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:56.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:05:56.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:56.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:55 smithi003 ceph-mon[31227]: Deploying daemon mon.c on smithi204 2024-09-21T06:05:57.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:57 smithi003 ceph-mon[31227]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:57.739 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:05:58.556 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:05:58.556 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":1,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","modified":"2024-09-21T06:04:21.188902Z","created":"2024-09-21T06:04:21.188902Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T06:05:58.556 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 1 2024-09-21T06:05:59.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 podman[34633]: 2024-09-21 06:05:59.380029335 +0000 UTC m=+0.106581121 container create ee491905d6881487a7f21646a380aafcee8e61ed2edbc898b1101b9a0357a9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-c, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:05:59.389 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 podman[34633]: 2024-09-21 06:05:59.289811259 +0000 UTC m=+0.016363055 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:05:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:59 smithi003 ceph-mon[31227]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:05:59 smithi003 ceph-mon[31227]: from='client.? 172.21.15.204:0/1605495446' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:05:59.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 podman[34633]: 2024-09-21 06:05:59.641090782 +0000 UTC m=+0.367642574 container init ee491905d6881487a7f21646a380aafcee8e61ed2edbc898b1101b9a0357a9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-21T06:05:59.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 podman[34633]: 2024-09-21 06:05:59.644754622 +0000 UTC m=+0.371306399 container start ee491905d6881487a7f21646a380aafcee8e61ed2edbc898b1101b9a0357a9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:05:59.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T06:05:59.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T06:05:59.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: pidfile_write: ignore empty --pid-file 2024-09-21T06:05:59.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: load: jerasure load: lrc 2024-09-21T06:05:59.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: RocksDB version: 7.9.2 2024-09-21T06:05:59.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Git sha 0 2024-09-21T06:05:59.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T06:05:59.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: DB SUMMARY 2024-09-21T06:05:59.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: DB Session ID: 6RHCJWRXWPCVBIFKEJJZ 2024-09-21T06:05:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: CURRENT file: CURRENT 2024-09-21T06:05:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T06:05:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T06:05:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-21T06:05:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-21T06:05:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.error_if_exists: 0 2024-09-21T06:05:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.create_if_missing: 0 2024-09-21T06:05:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.paranoid_checks: 1 2024-09-21T06:05:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T06:05:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T06:05:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T06:05:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.env: 0x55c62f4f7d60 2024-09-21T06:05:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T06:05:59.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.info_log: 0x55c635a11360 2024-09-21T06:05:59.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T06:05:59.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.statistics: (nil) 2024-09-21T06:05:59.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.use_fsync: 0 2024-09-21T06:05:59.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_log_file_size: 0 2024-09-21T06:05:59.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T06:05:59.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T06:05:59.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T06:05:59.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T06:05:59.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.allow_fallocate: 1 2024-09-21T06:05:59.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T06:05:59.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T06:05:59.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.use_direct_reads: 0 2024-09-21T06:05:59.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T06:05:59.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T06:05:59.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.db_log_dir: 2024-09-21T06:05:59.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.wal_dir: 2024-09-21T06:05:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T06:05:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T06:05:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T06:05:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T06:05:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T06:05:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T06:05:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T06:05:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T06:05:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.write_buffer_manager: 0x55c635a15d60 2024-09-21T06:05:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T06:05:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T06:05:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T06:05:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.unordered_write: 0 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.row_cache: None 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.wal_filter: None 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T06:05:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.two_write_queues: 0 2024-09-21T06:05:59.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T06:05:59.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.wal_compression: 0 2024-09-21T06:05:59.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.atomic_flush: 0 2024-09-21T06:05:59.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T06:05:59.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T06:05:59.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T06:05:59.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.log_readahead_size: 0 2024-09-21T06:05:59.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T06:05:59.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T06:05:59.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T06:05:59.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T06:05:59.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_background_jobs: 2 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_background_compactions: -1 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_subcompactions: 1 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_open_files: -1 2024-09-21T06:05:59.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_background_flushes: -1 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Compression algorithms supported: 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: kZSTD supported: 0 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: kXpressCompression supported: 0 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: kBZip2Compression supported: 0 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: kLZ4Compression supported: 1 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: kZlibCompression supported: 1 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: kSnappyCompression supported: 1 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T06:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.merge_operator: 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compaction_filter: None 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c635a10fa0) 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks: 1 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T06:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_top_level_index_and_filter: 1 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_type: 0 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_index_type: 0 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_shortening: 1 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: checksum: 4 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: no_block_cache: 0 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache: 0x55c6319d5610 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_name: BinnedLRUCache 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_options: 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: capacity : 536870912 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_shard_bits : 4 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: strict_capacity_limit : 0 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: high_pri_pool_ratio: 0.000 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_compressed: (nil) 2024-09-21T06:05:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout: persistent_cache: (nil) 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size: 4096 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size_deviation: 10 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_restart_interval: 16 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_block_restart_interval: 1 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: metadata_block_size: 4096 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: partition_filters: 0 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: use_delta_encoding: 1 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: filter_policy: bloomfilter 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: whole_key_filtering: 1 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: verify_compression: 0 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: read_amp_bytes_per_bit: 0 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: format_version: 5 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_index_compression: 1 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_align: 0 2024-09-21T06:05:59.916 INFO:journalctl@ceph.mon.c.smithi204.stdout: max_auto_readahead_size: 262144 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout: prepopulate_block_cache: 0 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout: initial_auto_readahead_size: 8192 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compression: NoCompression 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.num_levels: 7 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T06:05:59.917 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:05:59.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:05:59.919 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T06:05:59.919 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T06:05:59.919 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:05:59.919 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T06:05:59.919 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T06:05:59.919 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T06:05:59.919 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T06:05:59.919 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T06:05:59.919 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T06:05:59.919 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T06:05:59.919 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T06:05:59.919 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T06:05:59.920 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T06:05:59.920 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T06:05:59.920 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T06:05:59.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.inplace_update_support: 0 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.bloom_locality: 0 2024-09-21T06:05:59.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.max_successive_merges: 0 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.ttl: 2592000 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.enable_blob_files: false 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.min_blob_size: 0 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T06:05:59.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a1e90a8c-87af-4ea1-9839-a4a8e786aac0 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898759681984, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898759682452, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726898759, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a1e90a8c-87af-4ea1-9839-a4a8e786aac0", "db_session_id": "6RHCJWRXWPCVBIFKEJJZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898759682551, "job": 1, "event": "recovery_finished"} 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c635a34a80 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: DB pointer 0x55c6367da000 2024-09-21T06:05:59.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** DB Stats ** 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T06:05:59.925 INFO:journalctl@ceph.mon.c.smithi204.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T06:05:59.926 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache BinnedLRUCache@0x55c6319d5610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: using public_addr v2:172.21.15.204:0/0 -> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: starting mon.c rank -1 at public addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] at bind addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(???) e0 preinit fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout: e1 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout: btime 2024-09-21T06:04:24:293546+0000 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout: legacy client fscid: -1 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout: No filesystems configured 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T06:05:59.928 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: Added host smithi137 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:59.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: Updating smithi137:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: Deploying cephadm binary to smithi204 2024-09-21T06:05:59.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: Updating smithi137:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: Added host smithi204 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3764589064' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3764589064' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T06:05:59.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: Updating smithi204:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: Updating smithi204:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi137:172.21.15.137=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi137:172.21.15.137=b;smithi204:172.21.15.204=c;count:3 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: Deploying daemon mon.c on smithi204 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: from='client.? 172.21.15.204:0/1605495446' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:05:59.933 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 ceph-mon[34659]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T06:05:59.934 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 bash[34633]: ee491905d6881487a7f21646a380aafcee8e61ed2edbc898b1101b9a0357a9be 2024-09-21T06:05:59.934 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:05:59 smithi204 systemd[1]: Started Ceph mon.c for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:06:00.166 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:06:00.167 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:06:00.429 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:06:05.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:05.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:06:05.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: mon.a calling monitor election 2024-09-21T06:06:05.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:05.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:05.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:05.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:05.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: mon.c calling monitor election 2024-09-21T06:06:05.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:05.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:05.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:05.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:05.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:06:05.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: monmap epoch 2 2024-09-21T06:06:05.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:06:05.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: last_changed 2024-09-21T06:05:59.710800+0000 2024-09-21T06:06:05.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: created 2024-09-21T06:04:21.188902+0000 2024-09-21T06:06:05.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: min_mon_release 19 (squid) 2024-09-21T06:06:05.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: election_strategy: 1 2024-09-21T06:06:05.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-21T06:06:05.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-21T06:06:05.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: fsmap 2024-09-21T06:06:05.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:06:05.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: mgrmap e14: x(active, since 43s) 2024-09-21T06:06:05.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: overall HEALTH_OK 2024-09-21T06:06:05.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:05.093 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:05.093 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:05.093 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:05.093 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:04 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:05.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:05.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:06:05.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: mon.a calling monitor election 2024-09-21T06:06:05.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:05.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:05.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:05.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:05.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: mon.c calling monitor election 2024-09-21T06:06:05.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:05.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: monmap epoch 2 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: last_changed 2024-09-21T06:05:59.710800+0000 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: created 2024-09-21T06:04:21.188902+0000 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: min_mon_release 19 (squid) 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: election_strategy: 1 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: fsmap 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:06:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: mgrmap e14: x(active, since 43s) 2024-09-21T06:06:05.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: overall HEALTH_OK 2024-09-21T06:06:05.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:05.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:05.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:05.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:05.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:04 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:05.579 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:06:05.579 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","modified":"2024-09-21T06:05:59.710800Z","created":"2024-09-21T06:04:21.188902Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T06:06:05.579 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-09-21T06:06:05.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:05 smithi204 ceph-mon[34659]: Deploying daemon mon.b on smithi137 2024-09-21T06:06:05.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:05 smithi204 ceph-mon[34659]: from='client.? 172.21.15.204:0/3783360487' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:06:05.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:05 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:06.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:05 smithi003 ceph-mon[31227]: Deploying daemon mon.b on smithi137 2024-09-21T06:06:06.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:05 smithi003 ceph-mon[31227]: from='client.? 172.21.15.204:0/3783360487' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:06:06.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:05 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:07.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:06 smithi003 ceph-mon[31227]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:07.090 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:06 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:06:06.710+0000 7f472a2ee640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T06:06:07.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:06 smithi204 ceph-mon[34659]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:07.185 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:06:07.186 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:06:07.432 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:06:08.108 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:06:08.108 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","modified":"2024-09-21T06:05:59.710800Z","created":"2024-09-21T06:04:21.188902Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T06:06:08.108 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-09-21T06:06:08.494 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 podman[34445]: 2024-09-21 06:06:08.462009071 +0000 UTC m=+0.353879604 container start cc17861276f7c89b1bfe9c0a632db4f73036dd75c0a296c5d75f0a8b238246a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:06:08.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T06:06:08.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pidfile_write: ignore empty --pid-file 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: load: jerasure load: lrc 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: RocksDB version: 7.9.2 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Git sha 0 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: DB SUMMARY 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: DB Session ID: 553Q9XVKSXLCL0M7ZW65 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: CURRENT file: CURRENT 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.error_if_exists: 0 2024-09-21T06:06:08.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.create_if_missing: 0 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.paranoid_checks: 1 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.env: 0x564e355a0d60 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.info_log: 0x564e3bdb3360 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.statistics: (nil) 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.use_fsync: 0 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_log_file_size: 0 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T06:06:08.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T06:06:08.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T06:06:08.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.allow_fallocate: 1 2024-09-21T06:06:08.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T06:06:08.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T06:06:08.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.use_direct_reads: 0 2024-09-21T06:06:08.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T06:06:08.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T06:06:08.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.db_log_dir: 2024-09-21T06:06:08.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.wal_dir: 2024-09-21T06:06:08.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T06:06:08.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T06:06:08.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T06:06:08.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T06:06:08.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T06:06:08.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T06:06:08.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T06:06:08.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T06:06:08.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.write_buffer_manager: 0x564e3bdb7d60 2024-09-21T06:06:08.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T06:06:08.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T06:06:08.901 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T06:06:08.901 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T06:06:08.901 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T06:06:08.901 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T06:06:08.901 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T06:06:08.901 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T06:06:08.901 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.unordered_write: 0 2024-09-21T06:06:08.901 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T06:06:08.901 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T06:06:08.901 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T06:06:08.901 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T06:06:08.901 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.row_cache: None 2024-09-21T06:06:08.901 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.wal_filter: None 2024-09-21T06:06:08.902 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T06:06:08.902 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T06:06:08.902 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.two_write_queues: 0 2024-09-21T06:06:08.902 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T06:06:08.902 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.wal_compression: 0 2024-09-21T06:06:08.902 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.atomic_flush: 0 2024-09-21T06:06:08.902 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T06:06:08.903 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T06:06:08.903 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T06:06:08.903 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.log_readahead_size: 0 2024-09-21T06:06:08.903 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T06:06:08.903 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T06:06:08.903 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T06:06:08.903 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T06:06:08.903 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_background_jobs: 2 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_background_compactions: -1 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_subcompactions: 1 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_open_files: -1 2024-09-21T06:06:08.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_background_flushes: -1 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Compression algorithms supported: 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: kZSTD supported: 0 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: kXpressCompression supported: 0 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: kBZip2Compression supported: 0 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: kLZ4Compression supported: 1 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: kZlibCompression supported: 1 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: kSnappyCompression supported: 1 2024-09-21T06:06:08.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.merge_operator: 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compaction_filter: None 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564e3bdb2fa0) 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout: cache_index_and_filter_blocks: 1 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T06:06:08.906 INFO:journalctl@ceph.mon.b.smithi137.stdout: pin_top_level_index_and_filter: 1 2024-09-21T06:06:08.907 INFO:journalctl@ceph.mon.b.smithi137.stdout: index_type: 0 2024-09-21T06:06:08.907 INFO:journalctl@ceph.mon.b.smithi137.stdout: data_block_index_type: 0 2024-09-21T06:06:08.907 INFO:journalctl@ceph.mon.b.smithi137.stdout: index_shortening: 1 2024-09-21T06:06:08.907 INFO:journalctl@ceph.mon.b.smithi137.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T06:06:08.907 INFO:journalctl@ceph.mon.b.smithi137.stdout: checksum: 4 2024-09-21T06:06:08.907 INFO:journalctl@ceph.mon.b.smithi137.stdout: no_block_cache: 0 2024-09-21T06:06:08.907 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_cache: 0x564e37d77610 2024-09-21T06:06:08.907 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_cache_name: BinnedLRUCache 2024-09-21T06:06:08.907 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_cache_options: 2024-09-21T06:06:08.907 INFO:journalctl@ceph.mon.b.smithi137.stdout: capacity : 536870912 2024-09-21T06:06:08.907 INFO:journalctl@ceph.mon.b.smithi137.stdout: num_shard_bits : 4 2024-09-21T06:06:08.907 INFO:journalctl@ceph.mon.b.smithi137.stdout: strict_capacity_limit : 0 2024-09-21T06:06:08.907 INFO:journalctl@ceph.mon.b.smithi137.stdout: high_pri_pool_ratio: 0.000 2024-09-21T06:06:08.908 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_cache_compressed: (nil) 2024-09-21T06:06:08.908 INFO:journalctl@ceph.mon.b.smithi137.stdout: persistent_cache: (nil) 2024-09-21T06:06:08.908 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_size: 4096 2024-09-21T06:06:08.908 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_size_deviation: 10 2024-09-21T06:06:08.908 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_restart_interval: 16 2024-09-21T06:06:08.908 INFO:journalctl@ceph.mon.b.smithi137.stdout: index_block_restart_interval: 1 2024-09-21T06:06:08.908 INFO:journalctl@ceph.mon.b.smithi137.stdout: metadata_block_size: 4096 2024-09-21T06:06:08.908 INFO:journalctl@ceph.mon.b.smithi137.stdout: partition_filters: 0 2024-09-21T06:06:08.908 INFO:journalctl@ceph.mon.b.smithi137.stdout: use_delta_encoding: 1 2024-09-21T06:06:08.908 INFO:journalctl@ceph.mon.b.smithi137.stdout: filter_policy: bloomfilter 2024-09-21T06:06:08.908 INFO:journalctl@ceph.mon.b.smithi137.stdout: whole_key_filtering: 1 2024-09-21T06:06:08.908 INFO:journalctl@ceph.mon.b.smithi137.stdout: verify_compression: 0 2024-09-21T06:06:08.908 INFO:journalctl@ceph.mon.b.smithi137.stdout: read_amp_bytes_per_bit: 0 2024-09-21T06:06:08.909 INFO:journalctl@ceph.mon.b.smithi137.stdout: format_version: 5 2024-09-21T06:06:08.909 INFO:journalctl@ceph.mon.b.smithi137.stdout: enable_index_compression: 1 2024-09-21T06:06:08.909 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_align: 0 2024-09-21T06:06:08.909 INFO:journalctl@ceph.mon.b.smithi137.stdout: max_auto_readahead_size: 262144 2024-09-21T06:06:08.909 INFO:journalctl@ceph.mon.b.smithi137.stdout: prepopulate_block_cache: 0 2024-09-21T06:06:08.909 INFO:journalctl@ceph.mon.b.smithi137.stdout: initial_auto_readahead_size: 8192 2024-09-21T06:06:08.909 INFO:journalctl@ceph.mon.b.smithi137.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T06:06:08.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T06:06:08.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T06:06:08.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compression: NoCompression 2024-09-21T06:06:08.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T06:06:08.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T06:06:08.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.num_levels: 7 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:06:08.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T06:06:08.911 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T06:06:08.912 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.inplace_update_support: 0 2024-09-21T06:06:08.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.bloom_locality: 0 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.max_successive_merges: 0 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.ttl: 2592000 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T06:06:08.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.enable_blob_files: false 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.min_blob_size: 0 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0223f795-0afa-4441-98af-295b3080fe79 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898768499249, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T06:06:08.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T06:06:08.916 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898768499855, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726898768, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0223f795-0afa-4441-98af-295b3080fe79", "db_session_id": "553Q9XVKSXLCL0M7ZW65", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T06:06:08.916 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898768499986, "job": 1, "event": "recovery_finished"} 2024-09-21T06:06:08.916 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T06:06:08.916 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T06:06:08.916 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564e3bdd6a80 2024-09-21T06:06:08.916 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: DB pointer 0x564e3cb7c000 2024-09-21T06:06:08.916 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T06:06:08.916 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T06:06:08.916 INFO:journalctl@ceph.mon.b.smithi137.stdout: ** DB Stats ** 2024-09-21T06:06:08.916 INFO:journalctl@ceph.mon.b.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:06:08.916 INFO:journalctl@ceph.mon.b.smithi137.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T06:06:08.916 INFO:journalctl@ceph.mon.b.smithi137.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:06:08.916 INFO:journalctl@ceph.mon.b.smithi137.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: ** Compaction Stats [default] ** 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: ** Compaction Stats [default] ** 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:06:08.917 INFO:journalctl@ceph.mon.b.smithi137.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: Cumulative compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: Interval compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T06:06:08.918 INFO:journalctl@ceph.mon.b.smithi137.stdout: Block cache BinnedLRUCache@0x564e37d77610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: using public_addr v2:172.21.15.137:0/0 -> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: starting mon.b rank -1 at public addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] at bind addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(???) e0 preinit fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(synchronizing).mds e1 new map 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout: e1 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout: btime 2024-09-21T06:04:24:293546+0000 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:06:08.919 INFO:journalctl@ceph.mon.b.smithi137.stdout: legacy client fscid: -1 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout: No filesystems configured 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Added host smithi137 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Updating smithi137:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Deploying cephadm binary to smithi204 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Updating smithi137:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Added host smithi204 2024-09-21T06:06:08.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3764589064' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3764589064' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:08.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Updating smithi204:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Updating smithi204:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi137:172.21.15.137=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi137:172.21.15.137=b;smithi204:172.21.15.204=c;count:3 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:08.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:08.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:08.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Deploying daemon mon.c on smithi204 2024-09-21T06:06:08.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:08.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:08.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='client.? 172.21.15.204:0/1605495446' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:06:08.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:08.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:06:08.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.a calling monitor election 2024-09-21T06:06:08.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:08.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:08.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:08.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.c calling monitor election 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: monmap epoch 2 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: last_changed 2024-09-21T06:05:59.710800+0000 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: created 2024-09-21T06:04:21.188902+0000 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: min_mon_release 19 (squid) 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: election_strategy: 1 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-21T06:06:08.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: fsmap 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mgrmap e14: x(active, since 43s) 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: overall HEALTH_OK 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: Deploying daemon mon.b on smithi137 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='client.? 172.21.15.204:0/3783360487' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:08.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 ceph-mon[34464]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T06:06:08.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 bash[34445]: cc17861276f7c89b1bfe9c0a632db4f73036dd75c0a296c5d75f0a8b238246a0 2024-09-21T06:06:08.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:08 smithi137 systemd[1]: Started Ceph mon.b for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:06:09.716 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:06:09.717 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:06:09.971 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:06:13.896 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:06:13.896 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","modified":"2024-09-21T06:06:08.529830Z","created":"2024-09-21T06:04:21.188902Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T06:06:13.896 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2024-09-21T06:06:13.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: mon.a calling monitor election 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: mon.c calling monitor election 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: monmap epoch 3 2024-09-21T06:06:13.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: last_changed 2024-09-21T06:06:08.529830+0000 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: created 2024-09-21T06:04:21.188902+0000 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: min_mon_release 19 (squid) 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: election_strategy: 1 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: fsmap 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: mgrmap e14: x(active, since 51s) 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: overall HEALTH_OK 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:13.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:13.899 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:13 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:14.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:06:14.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:14.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: mon.a calling monitor election 2024-09-21T06:06:14.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: mon.c calling monitor election 2024-09-21T06:06:14.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:14.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:14.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:14.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:14.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:14.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:14.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:14.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:14.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: monmap epoch 3 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: last_changed 2024-09-21T06:06:08.529830+0000 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: created 2024-09-21T06:04:21.188902+0000 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: min_mon_release 19 (squid) 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: election_strategy: 1 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: fsmap 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: mgrmap e14: x(active, since 51s) 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: overall HEALTH_OK 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:14.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:14.093 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:13 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:14.485 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T06:06:14.486 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T06:06:14.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:14 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:14.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:14 smithi204 ceph-mon[34659]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:14.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:14 smithi204 ceph-mon[34659]: from='client.? 172.21.15.204:0/2150370592' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:06:14.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:14 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:14.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:14 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:14.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:14 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:14.750 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:14.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:14 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:14.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:14 smithi003 ceph-mon[31227]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:14.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:14 smithi003 ceph-mon[31227]: from='client.? 172.21.15.204:0/2150370592' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:06:14.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:14 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:14.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:14 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:14.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:14 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:15.406 INFO:teuthology.orchestra.run.smithi003.stdout:# minimal ceph.conf for 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:06:15.406 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-21T06:06:15.406 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:06:15.406 INFO:teuthology.orchestra.run.smithi003.stdout: mon_host = [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-21T06:06:15.876 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:06:15.876 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:15.876 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: mon.a calling monitor election 2024-09-21T06:06:15.876 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: mon.c calling monitor election 2024-09-21T06:06:15.876 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:15.876 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:15.876 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:15.876 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:15.876 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:15.876 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: monmap epoch 3 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: last_changed 2024-09-21T06:06:08.529830+0000 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: created 2024-09-21T06:04:21.188902+0000 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: min_mon_release 19 (squid) 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: election_strategy: 1 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: fsmap 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:06:15.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: mgrmap e14: x(active, since 51s) 2024-09-21T06:06:15.878 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: overall HEALTH_OK 2024-09-21T06:06:15.878 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:15.878 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:15.878 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:15.878 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:15.878 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:15.878 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:15.878 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='client.? 172.21.15.204:0/2150370592' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:06:15.878 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:15.878 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:15.878 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:15 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.104 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T06:06:16.104 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:06:16.104 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T06:06:16.132 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:06:16.132 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:16.204 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-21T06:06:16.204 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T06:06:16.231 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-21T06:06:16.231 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:16.296 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-21T06:06:16.296 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T06:06:16.324 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-21T06:06:16.324 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:16.390 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-21T06:06:16.390 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T06:06:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: mon.b calling monitor election 2024-09-21T06:06:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: mon.b calling monitor election 2024-09-21T06:06:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: mon.c calling monitor election 2024-09-21T06:06:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: mon.a calling monitor election 2024-09-21T06:06:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T06:06:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: monmap epoch 3 2024-09-21T06:06:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:06:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: last_changed 2024-09-21T06:06:08.529830+0000 2024-09-21T06:06:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: created 2024-09-21T06:04:21.188902+0000 2024-09-21T06:06:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: min_mon_release 19 (squid) 2024-09-21T06:06:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: election_strategy: 1 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: fsmap 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: mgrmap e14: x(active, since 53s) 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: overall HEALTH_OK 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T06:06:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:06:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: Reconfiguring daemon mon.a on smithi003 2024-09-21T06:06:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:16 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:16.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: mon.b calling monitor election 2024-09-21T06:06:16.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: mon.b calling monitor election 2024-09-21T06:06:16.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: mon.c calling monitor election 2024-09-21T06:06:16.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: mon.a calling monitor election 2024-09-21T06:06:16.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T06:06:16.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: monmap epoch 3 2024-09-21T06:06:16.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:06:16.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: last_changed 2024-09-21T06:06:08.529830+0000 2024-09-21T06:06:16.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: created 2024-09-21T06:04:21.188902+0000 2024-09-21T06:06:16.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: min_mon_release 19 (squid) 2024-09-21T06:06:16.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: election_strategy: 1 2024-09-21T06:06:16.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-21T06:06:16.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-21T06:06:16.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-09-21T06:06:16.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: fsmap 2024-09-21T06:06:16.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:06:16.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: mgrmap e14: x(active, since 53s) 2024-09-21T06:06:16.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: overall HEALTH_OK 2024-09-21T06:06:16.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T06:06:16.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:16.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:06:16.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:16.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: Reconfiguring daemon mon.a on smithi003 2024-09-21T06:06:16.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:16.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:16 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:16.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: mon.b calling monitor election 2024-09-21T06:06:16.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: mon.b calling monitor election 2024-09-21T06:06:16.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: mon.c calling monitor election 2024-09-21T06:06:16.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: mon.a calling monitor election 2024-09-21T06:06:16.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: monmap epoch 3 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: last_changed 2024-09-21T06:06:08.529830+0000 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: created 2024-09-21T06:04:21.188902+0000 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: min_mon_release 19 (squid) 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: election_strategy: 1 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: fsmap 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: mgrmap e14: x(active, since 53s) 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: overall HEALTH_OK 2024-09-21T06:06:16.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:16.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T06:06:16.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:16.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:06:16.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:16.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: Reconfiguring daemon mon.a on smithi003 2024-09-21T06:06:16.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:16.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:16 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:17.840 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:17 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:06:17.529+0000 7f472a2ee640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T06:06:18.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:18 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:18.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:18 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:18.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:18 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:18.373 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:18 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:18.374 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:18 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:18.374 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:18 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:18.374 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:18 smithi003 ceph-mon[31227]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:06:18.374 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:18 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:18.374 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:18 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:06:18.374 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:18 smithi003 ceph-mon[31227]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:18.374 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:18 smithi003 ceph-mon[31227]: Reconfiguring daemon mon.b on smithi137 2024-09-21T06:06:18.374 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:18 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3287563604' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T06:06:18.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:18 smithi137 ceph-mon[34464]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:06:18.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:18 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:18.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:18 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:06:18.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:18 smithi137 ceph-mon[34464]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:18.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:18 smithi137 ceph-mon[34464]: Reconfiguring daemon mon.b on smithi137 2024-09-21T06:06:18.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:18 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3287563604' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T06:06:18.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:18 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:18.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:18 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:18.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:18 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' 2024-09-21T06:06:18.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:18 smithi204 ceph-mon[34659]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:06:18.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:18 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:18.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:18 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:06:18.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:18 smithi204 ceph-mon[34659]: from='mgr.14152 172.21.15.3:0/1405141755' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:18.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:18 smithi204 ceph-mon[34659]: Reconfiguring daemon mon.b on smithi137 2024-09-21T06:06:18.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:18 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3287563604' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T06:06:18.840 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:18 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: ignoring --setuser ceph since I am not root 2024-09-21T06:06:18.840 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:18 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: ignoring --setgroup ceph since I am not root 2024-09-21T06:06:18.935 INFO:tasks.cephadm:Adding mgr.x on smithi003 2024-09-21T06:06:18.936 INFO:tasks.cephadm:Adding mgr.y on smithi137 2024-09-21T06:06:18.936 INFO:tasks.cephadm:Adding mgr.z on smithi204 2024-09-21T06:06:18.936 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi003=x;smithi137=y;smithi204=z' 2024-09-21T06:06:19.187 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:06:19.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:19 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3287563604' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T06:06:19.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:19 smithi204 ceph-mon[34659]: mgrmap e15: x(active, since 56s) 2024-09-21T06:06:19.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:19 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3287563604' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T06:06:19.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:19 smithi003 ceph-mon[31227]: mgrmap e15: x(active, since 56s) 2024-09-21T06:06:19.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:19 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3287563604' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T06:06:19.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:19 smithi137 ceph-mon[34464]: mgrmap e15: x(active, since 56s) 2024-09-21T06:06:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:23 smithi003 ceph-mon[31227]: Active manager daemon x restarted 2024-09-21T06:06:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:23 smithi003 ceph-mon[31227]: Activating manager daemon x 2024-09-21T06:06:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:23 smithi003 ceph-mon[31227]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T06:06:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:23 smithi003 ceph-mon[31227]: mgrmap e16: x(active, starting, since 0.00461329s) 2024-09-21T06:06:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:06:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:06:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:06:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:06:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:06:23.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:23 smithi003 ceph-mon[31227]: Manager daemon x is now available 2024-09-21T06:06:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:23 smithi137 ceph-mon[34464]: Active manager daemon x restarted 2024-09-21T06:06:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:23 smithi137 ceph-mon[34464]: Activating manager daemon x 2024-09-21T06:06:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:23 smithi137 ceph-mon[34464]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T06:06:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:23 smithi137 ceph-mon[34464]: mgrmap e16: x(active, starting, since 0.00461329s) 2024-09-21T06:06:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:06:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:06:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:06:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:06:23.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:06:23.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:23 smithi137 ceph-mon[34464]: Manager daemon x is now available 2024-09-21T06:06:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:23 smithi204 ceph-mon[34659]: Active manager daemon x restarted 2024-09-21T06:06:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:23 smithi204 ceph-mon[34659]: Activating manager daemon x 2024-09-21T06:06:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:23 smithi204 ceph-mon[34659]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T06:06:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:23 smithi204 ceph-mon[34659]: mgrmap e16: x(active, starting, since 0.00461329s) 2024-09-21T06:06:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:06:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:06:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:06:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:06:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:06:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:06:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:06:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:23 smithi204 ceph-mon[34659]: Manager daemon x is now available 2024-09-21T06:06:24.591 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2024-09-21T06:06:24.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:24.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:06:24.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:06:24.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:24 smithi003 ceph-mon[31227]: mgrmap e17: x(active, since 1.01063s) 2024-09-21T06:06:24.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:24.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:24.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:06:24.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:06:24.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:24 smithi137 ceph-mon[34464]: mgrmap e17: x(active, since 1.01063s) 2024-09-21T06:06:24.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:24.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:24.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:06:24.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:06:24.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:24 smithi204 ceph-mon[34659]: mgrmap e17: x(active, since 1.01063s) 2024-09-21T06:06:24.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:25.590 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:06:25.331+0000 7fc4d72ef640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:06:06:25] ENGINE Error in HTTPServer.serve 2024-09-21T06:06:25.590 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: Traceback (most recent call last): 2024-09-21T06:06:25.590 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: self._connections.run(self.expiration_interval) 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: self._run(expiration_interval) 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: s = self.context.wrap_socket( 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: return self.sslsocket_class._create( 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: self.do_handshake() 2024-09-21T06:06:25.591 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:06:25.592 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: self._sslobj.do_handshake() 2024-09-21T06:06:25.592 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:06:25.592 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:06:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:06:25.845 DEBUG:teuthology.orchestra.run.smithi137:mgr.y> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T06:06:25.848 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T06:06:25.850 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T06:06:25.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T06:06:25.852 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T06:06:25.852 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T06:06:25.894 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T06:06:25.895 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T06:06:25.937 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T06:06:25.938 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T06:06:25.982 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T06:06:25.982 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T06:06:26.026 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T06:06:26.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T06:06:26.070 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T06:06:26.070 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T06:06:26.112 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T06:06:26.113 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T06:06:26.156 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T06:06:26.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T06:06:26.200 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T06:06:26.201 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T06:06:26.210 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:26.212 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T06:06:26.212 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T06:06:26.262 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T06:06:26.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T06:06:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:26.294 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T06:06:26.294 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T06:06:26.350 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T06:06:26.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T06:06:26.397 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T06:06:26.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T06:06:26.402 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: Saving service mgr spec with placement smithi003=x;smithi137=y;smithi204=z;count:3 2024-09-21T06:06:26.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: [21/Sep/2024:06:06:25] ENGINE Bus STARTING 2024-09-21T06:06:26.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: [21/Sep/2024:06:06:25] ENGINE Serving on https://172.21.15.3:7150 2024-09-21T06:06:26.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: [21/Sep/2024:06:06:25] ENGINE Error in HTTPServer.serve 2024-09-21T06:06:26.403 INFO:journalctl@ceph.mon.a.smithi003.stdout: Traceback (most recent call last): 2024-09-21T06:06:26.403 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:06:26.403 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:06:26.404 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:06:26.404 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._run(expiration_interval) 2024-09-21T06:06:26.404 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:06:26.404 INFO:journalctl@ceph.mon.a.smithi003.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:06:26.404 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:06:26.404 INFO:journalctl@ceph.mon.a.smithi003.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:06:26.404 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:06:26.405 INFO:journalctl@ceph.mon.a.smithi003.stdout: s = self.context.wrap_socket( 2024-09-21T06:06:26.405 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:06:26.405 INFO:journalctl@ceph.mon.a.smithi003.stdout: return self.sslsocket_class._create( 2024-09-21T06:06:26.405 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:06:26.405 INFO:journalctl@ceph.mon.a.smithi003.stdout: self.do_handshake() 2024-09-21T06:06:26.406 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:06:26.406 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._sslobj.do_handshake() 2024-09-21T06:06:26.406 INFO:journalctl@ceph.mon.a.smithi003.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:06:26.406 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.406 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.406 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.406 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:26.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi003", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:26.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.408 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:26.408 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:26.408 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:26.409 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T06:06:26.409 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T06:06:26.459 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T06:06:26.459 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T06:06:26.507 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T06:06:26.507 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T06:06:26.559 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T06:06:26.559 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T06:06:26.622 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T06:06:26.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T06:06:26.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: Saving service mgr spec with placement smithi003=x;smithi137=y;smithi204=z;count:3 2024-09-21T06:06:26.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: [21/Sep/2024:06:06:25] ENGINE Bus STARTING 2024-09-21T06:06:26.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: [21/Sep/2024:06:06:25] ENGINE Serving on https://172.21.15.3:7150 2024-09-21T06:06:26.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: [21/Sep/2024:06:06:25] ENGINE Error in HTTPServer.serve 2024-09-21T06:06:26.646 INFO:journalctl@ceph.mon.b.smithi137.stdout: Traceback (most recent call last): 2024-09-21T06:06:26.646 INFO:journalctl@ceph.mon.b.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:06:26.646 INFO:journalctl@ceph.mon.b.smithi137.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:06:26.646 INFO:journalctl@ceph.mon.b.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:06:26.646 INFO:journalctl@ceph.mon.b.smithi137.stdout: self._run(expiration_interval) 2024-09-21T06:06:26.646 INFO:journalctl@ceph.mon.b.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:06:26.646 INFO:journalctl@ceph.mon.b.smithi137.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:06:26.647 INFO:journalctl@ceph.mon.b.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:06:26.647 INFO:journalctl@ceph.mon.b.smithi137.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:06:26.647 INFO:journalctl@ceph.mon.b.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:06:26.647 INFO:journalctl@ceph.mon.b.smithi137.stdout: s = self.context.wrap_socket( 2024-09-21T06:06:26.647 INFO:journalctl@ceph.mon.b.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:06:26.647 INFO:journalctl@ceph.mon.b.smithi137.stdout: return self.sslsocket_class._create( 2024-09-21T06:06:26.647 INFO:journalctl@ceph.mon.b.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:06:26.647 INFO:journalctl@ceph.mon.b.smithi137.stdout: self.do_handshake() 2024-09-21T06:06:26.647 INFO:journalctl@ceph.mon.b.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:06:26.647 INFO:journalctl@ceph.mon.b.smithi137.stdout: self._sslobj.do_handshake() 2024-09-21T06:06:26.647 INFO:journalctl@ceph.mon.b.smithi137.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:06:26.647 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:26.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi003", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:26.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:26.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:26.649 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:26.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: Saving service mgr spec with placement smithi003=x;smithi137=y;smithi204=z;count:3 2024-09-21T06:06:26.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: [21/Sep/2024:06:06:25] ENGINE Bus STARTING 2024-09-21T06:06:26.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: [21/Sep/2024:06:06:25] ENGINE Serving on https://172.21.15.3:7150 2024-09-21T06:06:26.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: [21/Sep/2024:06:06:25] ENGINE Error in HTTPServer.serve 2024-09-21T06:06:26.655 INFO:journalctl@ceph.mon.c.smithi204.stdout: Traceback (most recent call last): 2024-09-21T06:06:26.655 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:06:26.655 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:06:26.655 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:06:26.655 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._run(expiration_interval) 2024-09-21T06:06:26.655 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:06:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:06:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:06:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:06:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:06:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout: s = self.context.wrap_socket( 2024-09-21T06:06:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:06:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout: return self.sslsocket_class._create( 2024-09-21T06:06:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:06:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout: self.do_handshake() 2024-09-21T06:06:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:06:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._sslobj.do_handshake() 2024-09-21T06:06:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:06:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:26.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi003", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:26.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:26.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:26.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:26.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:26.692 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-21T06:06:26.692 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mgr debug_cephsqlite 20 2024-09-21T06:06:26.752 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T06:06:26.752 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T06:06:26.801 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T06:06:26.801 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T06:06:26.848 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T06:06:26.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T06:06:26.892 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-21T06:06:26.893 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T06:06:26.952 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-21T06:06:26.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T06:06:26.996 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-21T06:06:26.997 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-21T06:06:27.050 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T06:06:27.051 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T06:06:27.104 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T06:06:27.104 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T06:06:27.155 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T06:06:27.155 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T06:06:27.209 DEBUG:tasks.cephadm:set 30 configs 2024-09-21T06:06:27.209 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi003') 2024-09-21T06:06:27.579 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:27 smithi003 ceph-mon[31227]: [21/Sep/2024:06:06:25] ENGINE Serving on http://172.21.15.3:8765 2024-09-21T06:06:27.579 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:27 smithi003 ceph-mon[31227]: [21/Sep/2024:06:06:25] ENGINE Bus STARTED 2024-09-21T06:06:27.579 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:27 smithi003 ceph-mon[31227]: Updating smithi003:/etc/ceph/ceph.conf 2024-09-21T06:06:27.579 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:27 smithi003 ceph-mon[31227]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-21T06:06:27.579 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:27 smithi003 ceph-mon[31227]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-21T06:06:27.579 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:27 smithi003 ceph-mon[31227]: mgrmap e18: x(active, since 2s) 2024-09-21T06:06:27.579 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:27.579 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:27.579 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:27.579 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:27 smithi137 ceph-mon[34464]: [21/Sep/2024:06:06:25] ENGINE Serving on http://172.21.15.3:8765 2024-09-21T06:06:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:27 smithi137 ceph-mon[34464]: [21/Sep/2024:06:06:25] ENGINE Bus STARTED 2024-09-21T06:06:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:27 smithi137 ceph-mon[34464]: Updating smithi003:/etc/ceph/ceph.conf 2024-09-21T06:06:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:27 smithi137 ceph-mon[34464]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-21T06:06:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:27 smithi137 ceph-mon[34464]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-21T06:06:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:27 smithi137 ceph-mon[34464]: mgrmap e18: x(active, since 2s) 2024-09-21T06:06:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:27.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:27 smithi204 ceph-mon[34659]: [21/Sep/2024:06:06:25] ENGINE Serving on http://172.21.15.3:8765 2024-09-21T06:06:27.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:27 smithi204 ceph-mon[34659]: [21/Sep/2024:06:06:25] ENGINE Bus STARTED 2024-09-21T06:06:27.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:27 smithi204 ceph-mon[34659]: Updating smithi003:/etc/ceph/ceph.conf 2024-09-21T06:06:27.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:27 smithi204 ceph-mon[34659]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-21T06:06:27.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:27 smithi204 ceph-mon[34659]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-21T06:06:27.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:27 smithi204 ceph-mon[34659]: mgrmap e18: x(active, since 2s) 2024-09-21T06:06:27.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:27.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:27.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:27.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:28.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: Updating smithi137:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:06:28.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: Updating smithi204:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:06:28.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:28.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:28.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: Updating smithi204:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:06:28.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: Updating smithi137:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:06:28.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: from='client.? ' entity='client.admin' 2024-09-21T06:06:28.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:28.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:28.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:28.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:06:28.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:06:28.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:06:28.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:28.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:28 smithi003 ceph-mon[31227]: from='client.? ' entity='client.admin' 2024-09-21T06:06:28.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: Updating smithi137:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:06:28.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: Updating smithi204:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:06:28.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:28.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:28.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: Updating smithi204:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:06:28.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: Updating smithi137:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:06:28.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: from='client.? ' entity='client.admin' 2024-09-21T06:06:28.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:28.641 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:28.642 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:28.642 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:06:28.642 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:06:28.642 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:06:28.642 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:28.642 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:28 smithi204 ceph-mon[34659]: from='client.? ' entity='client.admin' 2024-09-21T06:06:28.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: Updating smithi137:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:06:28.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: Updating smithi204:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:06:28.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:28.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:28.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: Updating smithi204:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:06:28.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: Updating smithi137:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:06:28.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: from='client.? ' entity='client.admin' 2024-09-21T06:06:28.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:28.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:28.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:28.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:06:28.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:06:28.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:06:28.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:28.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:28 smithi137 ceph-mon[34464]: from='client.? ' entity='client.admin' 2024-09-21T06:06:29.448 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:29 smithi003 ceph-mon[31227]: Updating smithi003:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:06:29.448 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:29 smithi003 ceph-mon[31227]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:29.448 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:29 smithi003 ceph-mon[31227]: Updating smithi003:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:06:29.448 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:29 smithi003 ceph-mon[31227]: Deploying daemon mgr.z on smithi204 2024-09-21T06:06:29.465 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:29.478 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:29.527 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi003') 2024-09-21T06:06:29.527 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi003') 2024-09-21T06:06:29.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:29 smithi137 ceph-mon[34464]: Updating smithi003:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:06:29.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:29 smithi137 ceph-mon[34464]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:29.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:29 smithi137 ceph-mon[34464]: Updating smithi003:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:06:29.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:29 smithi137 ceph-mon[34464]: Deploying daemon mgr.z on smithi204 2024-09-21T06:06:29.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:29 smithi204 ceph-mon[34659]: Updating smithi003:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:06:29.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:29 smithi204 ceph-mon[34659]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:06:29.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:29 smithi204 ceph-mon[34659]: Updating smithi003:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:06:29.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:29 smithi204 ceph-mon[34659]: Deploying daemon mgr.z on smithi204 2024-09-21T06:06:30.099 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 21 06:06:29 smithi204 systemd[1]: Starting Ceph mgr.z for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:06:30.154 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:30.352 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 21 06:06:30 smithi204 podman[36515]: 2024-09-21 06:06:30.000845836 +0000 UTC m=+0.016230081 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:06:30.352 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 21 06:06:30 smithi204 podman[36515]: 2024-09-21 06:06:30.138791015 +0000 UTC m=+0.154175239 container create 42ac69c4e6c161929373714e465800f7daa94918803f9205d999f98ec8a72477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T06:06:30.352 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 21 06:06:30 smithi204 podman[36515]: 2024-09-21 06:06:30.348214647 +0000 UTC m=+0.363598871 container init 42ac69c4e6c161929373714e465800f7daa94918803f9205d999f98ec8a72477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-z, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:06:30.352 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 21 06:06:30 smithi204 podman[36515]: 2024-09-21 06:06:30.351119559 +0000 UTC m=+0.366503783 container start 42ac69c4e6c161929373714e465800f7daa94918803f9205d999f98ec8a72477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-z, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T06:06:30.654 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 21 06:06:30 smithi204 ceph-mgr[36534]: -- 172.21.15.204:0/519910951 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f7bbab0e0 con 0x561f8094e000 2024-09-21T06:06:30.654 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 21 06:06:30 smithi204 bash[36515]: 42ac69c4e6c161929373714e465800f7daa94918803f9205d999f98ec8a72477 2024-09-21T06:06:30.654 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 21 06:06:30 smithi204 systemd[1]: Started Ceph mgr.z for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:06:31.012 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.012 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.015 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.016 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.019 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.020 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.021 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.025 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.052 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.058 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.059 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.059 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:31.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.242+0000 7f32b928d640 1 -- 172.21.15.3:0/3378082303 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f329c03fb70 msgr2=0x7f329c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:31.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.242+0000 7f32b928d640 1 --2- 172.21.15.3:0/3378082303 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f329c03fb70 0x7f329c042080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f32a4005fd0 tx=0x7f32a4009680 comp rx=0 tx=0).stop 2024-09-21T06:06:31.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.242+0000 7f32b928d640 1 -- 172.21.15.3:0/3378082303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32b4108d00 msgr2=0x7f32b419fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:31.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.242+0000 7f32b928d640 1 --2- 172.21.15.3:0/3378082303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32b4108d00 0x7f32b419fa40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00d750 tx=0x7f32ac00dc10 comp rx=0 tx=0).stop 2024-09-21T06:06:31.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.242+0000 7f32b928d640 1 -- 172.21.15.3:0/3378082303 shutdown_connections 2024-09-21T06:06:31.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.242+0000 7f32b928d640 1 --2- 172.21.15.3:0/3378082303 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f329c03fb70 0x7f329c042080 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:31.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.242+0000 7f32b928d640 1 --2- 172.21.15.3:0/3378082303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b41083b0 0x7f32b419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:31.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.242+0000 7f32b928d640 1 --2- 172.21.15.3:0/3378082303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32b4108d00 0x7f32b419fa40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:31.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.242+0000 7f32b928d640 1 --2- 172.21.15.3:0/3378082303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32b4102a70 0x7f32b419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:31.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.242+0000 7f32b928d640 1 -- 172.21.15.3:0/3378082303 >> 172.21.15.3:0/3378082303 conn(0x7f32b4079730 msgr2=0x7f32b41044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:31.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.243+0000 7f32b928d640 1 -- 172.21.15.3:0/3378082303 shutdown_connections 2024-09-21T06:06:31.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.243+0000 7f32b928d640 1 -- 172.21.15.3:0/3378082303 wait complete. 2024-09-21T06:06:31.536 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:31 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2228534217' entity='client.admin' 2024-09-21T06:06:31.537 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:31.537 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:31.537 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:31.537 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:06:31.537 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:06:31.537 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:06:31.537 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:31.537 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:31 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3378082303' entity='client.admin' 2024-09-21T06:06:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:31 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2228534217' entity='client.admin' 2024-09-21T06:06:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:06:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:06:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:06:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:31.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:31 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3378082303' entity='client.admin' 2024-09-21T06:06:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:31 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2228534217' entity='client.admin' 2024-09-21T06:06:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:06:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:06:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:06:31.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:31.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:31 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3378082303' entity='client.admin' 2024-09-21T06:06:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.830+0000 7f4fae087640 1 -- 172.21.15.3:0/484865568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa81083a0 msgr2=0x7f4fa81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.830+0000 7f4fae087640 1 --2- 172.21.15.3:0/484865568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa81083a0 0x7f4fa81087b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009900 tx=0x7f4f9c02a410 comp rx=0 tx=0).stop 2024-09-21T06:06:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.830+0000 7f4fae087640 1 -- 172.21.15.3:0/484865568 shutdown_connections 2024-09-21T06:06:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.830+0000 7f4fae087640 1 --2- 172.21.15.3:0/484865568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa8108cf0 0x7f4fa810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.830+0000 7f4fae087640 1 --2- 172.21.15.3:0/484865568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa81083a0 0x7f4fa81087b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.830+0000 7f4fae087640 1 --2- 172.21.15.3:0/484865568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa8102a60 0x7f4fa8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.830+0000 7f4fae087640 1 -- 172.21.15.3:0/484865568 >> 172.21.15.3:0/484865568 conn(0x7f4fa8079730 msgr2=0x7f4fa8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.830+0000 7f4fae087640 1 -- 172.21.15.3:0/484865568 shutdown_connections 2024-09-21T06:06:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.830+0000 7f4fae087640 1 -- 172.21.15.3:0/484865568 wait complete. 2024-09-21T06:06:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.831+0000 7f4fae087640 1 Processor -- start 2024-09-21T06:06:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.831+0000 7f4fae087640 1 -- start start 2024-09-21T06:06:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.831+0000 7f4fae087640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa8102a60 0x7f4fa81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.831+0000 7f4fae087640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa81083a0 0x7f4fa8199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fae087640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa8108cf0 0x7f4fa819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fae087640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8112720 con 0x7f4fa8102a60 2024-09-21T06:06:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fae087640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81125a0 con 0x7f4fa81083a0 2024-09-21T06:06:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fae087640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81128a0 con 0x7f4fa8108cf0 2024-09-21T06:06:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa8102a60 0x7f4fa81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa8102a60 0x7f4fa81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51784/0 (socket says 172.21.15.3:51784) 2024-09-21T06:06:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa77fe640 1 -- 172.21.15.3:0/2044170761 learned_addr learned my addr 172.21.15.3:0/2044170761 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa6ffd640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa81083a0 0x7f4fa8199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa7fff640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa8108cf0 0x7f4fa819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa77fe640 1 -- 172.21.15.3:0/2044170761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa8108cf0 msgr2=0x7f4fa819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa77fe640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa8108cf0 0x7f4fa819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa77fe640 1 -- 172.21.15.3:0/2044170761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa81083a0 msgr2=0x7f4fa8199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa77fe640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa81083a0 0x7f4fa8199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa77fe640 1 -- 172.21.15.3:0/2044170761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa819f260 con 0x7f4fa8102a60 2024-09-21T06:06:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa6ffd640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa81083a0 0x7f4fa8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa77fe640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa8102a60 0x7f4fa81992f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800d950 tx=0x7f4f9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa4ff9640 1 -- 172.21.15.3:0/2044170761 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98017070 con 0x7f4fa8102a60 2024-09-21T06:06:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fa4ff9640 1 -- 172.21.15.3:0/2044170761 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f4f98009c30 con 0x7f4fa8102a60 2024-09-21T06:06:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.832+0000 7f4fae087640 1 -- 172.21.15.3:0/2044170761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa819f550 con 0x7f4fa8102a60 2024-09-21T06:06:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.833+0000 7f4fa4ff9640 1 -- 172.21.15.3:0/2044170761 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9800f620 con 0x7f4fa8102a60 2024-09-21T06:06:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.833+0000 7f4fae087640 1 -- 172.21.15.3:0/2044170761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa81a5cb0 con 0x7f4fa8102a60 2024-09-21T06:06:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.834+0000 7f4fae087640 1 -- 172.21.15.3:0/2044170761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa8102e70 con 0x7f4fa8102a60 2024-09-21T06:06:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.834+0000 7f4fa4ff9640 1 -- 172.21.15.3:0/2044170761 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 51914+0+0 (secure 0 0 0) 0x7f4f9800f7c0 con 0x7f4fa8102a60 2024-09-21T06:06:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.834+0000 7f4fa4ff9640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f9003fb70 0x7f4f90042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.834+0000 7f4fa4ff9640 1 -- 172.21.15.3:0/2044170761 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4f98054200 con 0x7f4fa8102a60 2024-09-21T06:06:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.834+0000 7f4fa6ffd640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f9003fb70 0x7f4f90042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.835+0000 7f4fa6ffd640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f9003fb70 0x7f4f90042080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c000c00 tx=0x7f4f9c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:31.839+0000 7f4fa4ff9640 1 -- 172.21.15.3:0/2044170761 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4f98010e20 con 0x7f4fa8102a60 2024-09-21T06:06:32.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.027+0000 7f4fae087640 1 -- 172.21.15.3:0/2044170761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f4fa810aae0 con 0x7f4fa8102a60 2024-09-21T06:06:32.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.031+0000 7f4fa4ff9640 1 -- 172.21.15.3:0/2044170761 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v14)=0 v14) ==== 117+0+0 (secure 0 0 0) 0x7f4f9801e120 con 0x7f4fa8102a60 2024-09-21T06:06:32.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.033+0000 7f4fae087640 1 -- 172.21.15.3:0/2044170761 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f9003fb70 msgr2=0x7f4f90042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.033+0000 7f4fae087640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f9003fb70 0x7f4f90042080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c000c00 tx=0x7f4f9c031400 comp rx=0 tx=0).stop 2024-09-21T06:06:32.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.033+0000 7f4fae087640 1 -- 172.21.15.3:0/2044170761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa8102a60 msgr2=0x7f4fa81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.033+0000 7f4fae087640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa8102a60 0x7f4fa81992f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800d950 tx=0x7f4f9800de10 comp rx=0 tx=0).stop 2024-09-21T06:06:32.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.033+0000 7f4fae087640 1 -- 172.21.15.3:0/2044170761 shutdown_connections 2024-09-21T06:06:32.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.033+0000 7f4fae087640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f9003fb70 0x7f4f90042080 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.033+0000 7f4fae087640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa8108cf0 0x7f4fa819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.033+0000 7f4fae087640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa81083a0 0x7f4fa8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.033+0000 7f4fae087640 1 --2- 172.21.15.3:0/2044170761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa8102a60 0x7f4fa81992f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.033+0000 7f4fae087640 1 -- 172.21.15.3:0/2044170761 >> 172.21.15.3:0/2044170761 conn(0x7f4fa8079730 msgr2=0x7f4fa8104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:32.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.034+0000 7f4fae087640 1 -- 172.21.15.3:0/2044170761 shutdown_connections 2024-09-21T06:06:32.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.034+0000 7f4fae087640 1 -- 172.21.15.3:0/2044170761 wait complete. 2024-09-21T06:06:32.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:32 smithi137 ceph-mon[34464]: Deploying daemon mgr.y on smithi137 2024-09-21T06:06:32.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:32 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2044170761' entity='client.admin' 2024-09-21T06:06:32.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:32 smithi003 ceph-mon[31227]: Deploying daemon mgr.y on smithi137 2024-09-21T06:06:32.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:32 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2044170761' entity='client.admin' 2024-09-21T06:06:32.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:32 smithi204 ceph-mon[34659]: Deploying daemon mgr.y on smithi137 2024-09-21T06:06:32.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:32 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2044170761' entity='client.admin' 2024-09-21T06:06:32.762 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 21 06:06:32 smithi137 podman[35967]: 2024-09-21 06:06:32.726335575 +0000 UTC m=+0.349773260 container start f23cce124baf2232f20577c596f5ac28908c9d2dc26d71669aa57e8dad6f53fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-y, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T06:06:33.145 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 21 06:06:32 smithi137 ceph-mgr[35986]: -- 172.21.15.137:0/3466693745 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ce58d10e0 con 0x561cea674000 2024-09-21T06:06:33.145 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 21 06:06:32 smithi137 bash[35967]: f23cce124baf2232f20577c596f5ac28908c9d2dc26d71669aa57e8dad6f53fd 2024-09-21T06:06:33.145 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 21 06:06:32 smithi137 systemd[1]: Started Ceph mgr.y for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:06:33.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.495+0000 7f6d19515640 1 -- 172.21.15.3:0/2085753749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d14109e00 msgr2=0x7f6d14110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.495+0000 7f6d19515640 1 --2- 172.21.15.3:0/2085753749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d14109e00 0x7f6d14110630 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009900 tx=0x7f6d0c02a410 comp rx=0 tx=0).stop 2024-09-21T06:06:33.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.495+0000 7f6d19515640 1 -- 172.21.15.3:0/2085753749 shutdown_connections 2024-09-21T06:06:33.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.495+0000 7f6d19515640 1 --2- 172.21.15.3:0/2085753749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d141094b0 0x7f6d141098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.495+0000 7f6d19515640 1 --2- 172.21.15.3:0/2085753749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d14109e00 0x7f6d14110630 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.495+0000 7f6d19515640 1 --2- 172.21.15.3:0/2085753749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d14103b70 0x7f6d14103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.495+0000 7f6d19515640 1 -- 172.21.15.3:0/2085753749 >> 172.21.15.3:0/2085753749 conn(0x7f6d140ff5e0 msgr2=0x7f6d14101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:33.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.495+0000 7f6d19515640 1 -- 172.21.15.3:0/2085753749 shutdown_connections 2024-09-21T06:06:33.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.495+0000 7f6d19515640 1 -- 172.21.15.3:0/2085753749 wait complete. 2024-09-21T06:06:33.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.496+0000 7f6d19515640 1 Processor -- start 2024-09-21T06:06:33.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.496+0000 7f6d19515640 1 -- start start 2024-09-21T06:06:33.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.496+0000 7f6d19515640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d14103b70 0x7f6d1419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.496+0000 7f6d19515640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d141094b0 0x7f6d1419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.496+0000 7f6d19515640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d14109e00 0x7f6d1419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.496+0000 7f6d19515640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6d14113880 con 0x7f6d14103b70 2024-09-21T06:06:33.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.496+0000 7f6d19515640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6d14113700 con 0x7f6d141094b0 2024-09-21T06:06:33.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.496+0000 7f6d19515640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d14113a00 con 0x7f6d14109e00 2024-09-21T06:06:33.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.496+0000 7f6d12ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d14103b70 0x7f6d1419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.496+0000 7f6d12ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d14103b70 0x7f6d1419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51804/0 (socket says 172.21.15.3:51804) 2024-09-21T06:06:33.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.496+0000 7f6d12ffd640 1 -- 172.21.15.3:0/570860206 learned_addr learned my addr 172.21.15.3:0/570860206 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.497+0000 7f6d127fc640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d141094b0 0x7f6d1419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.497+0000 7f6d137fe640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d14109e00 0x7f6d1419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.497+0000 7f6d12ffd640 1 -- 172.21.15.3:0/570860206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d14109e00 msgr2=0x7f6d1419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.497+0000 7f6d12ffd640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d14109e00 0x7f6d1419faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.497+0000 7f6d12ffd640 1 -- 172.21.15.3:0/570860206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d141094b0 msgr2=0x7f6d1419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.497+0000 7f6d12ffd640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d141094b0 0x7f6d1419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.497+0000 7f6d12ffd640 1 -- 172.21.15.3:0/570860206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d141a0220 con 0x7f6d14103b70 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.497+0000 7f6d12ffd640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d14103b70 0x7f6d1419a2b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400d950 tx=0x7f6d0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.497+0000 7f6cfbfff640 1 -- 172.21.15.3:0/570860206 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04017070 con 0x7f6d14103b70 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.497+0000 7f6cfbfff640 1 -- 172.21.15.3:0/570860206 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f6d04009c30 con 0x7f6d14103b70 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.497+0000 7f6d19515640 1 -- 172.21.15.3:0/570860206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d141a0510 con 0x7f6d14103b70 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.497+0000 7f6cfbfff640 1 -- 172.21.15.3:0/570860206 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0400f620 con 0x7f6d14103b70 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.497+0000 7f6d19515640 1 -- 172.21.15.3:0/570860206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d141a6c70 con 0x7f6d14103b70 2024-09-21T06:06:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.498+0000 7f6cfbfff640 1 -- 172.21.15.3:0/570860206 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 51914+0+0 (secure 0 0 0) 0x7f6d0400f7c0 con 0x7f6d14103b70 2024-09-21T06:06:33.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.498+0000 7f6cfbfff640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6cfc03fb70 0x7f6cfc042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.498+0000 7f6d19515640 1 -- 172.21.15.3:0/570860206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d14103f80 con 0x7f6d14103b70 2024-09-21T06:06:33.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.499+0000 7f6cfbfff640 1 -- 172.21.15.3:0/570860206 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6d040550b0 con 0x7f6d14103b70 2024-09-21T06:06:33.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.499+0000 7f6d127fc640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6cfc03fb70 0x7f6cfc042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.499+0000 7f6d127fc640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6cfc03fb70 0x7f6cfc042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6d14108020 tx=0x7f6d08009760 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:33.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.504+0000 7f6cfbfff640 1 -- 172.21.15.3:0/570860206 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6d04010e20 con 0x7f6d14103b70 2024-09-21T06:06:33.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.699+0000 7f6d19515640 1 -- 172.21.15.3:0/570860206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f6d1410bbf0 con 0x7f6d14103b70 2024-09-21T06:06:33.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.713+0000 7f6cfbfff640 1 -- 172.21.15.3:0/570860206 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7f6d0401e120 con 0x7f6d14103b70 2024-09-21T06:06:33.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.715+0000 7f6d19515640 1 -- 172.21.15.3:0/570860206 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6cfc03fb70 msgr2=0x7f6cfc042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.715+0000 7f6d19515640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6cfc03fb70 0x7f6cfc042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6d14108020 tx=0x7f6d08009760 comp rx=0 tx=0).stop 2024-09-21T06:06:33.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.715+0000 7f6d19515640 1 -- 172.21.15.3:0/570860206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d14103b70 msgr2=0x7f6d1419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.715+0000 7f6d19515640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d14103b70 0x7f6d1419a2b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400d950 tx=0x7f6d0400de10 comp rx=0 tx=0).stop 2024-09-21T06:06:33.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.715+0000 7f6d19515640 1 -- 172.21.15.3:0/570860206 shutdown_connections 2024-09-21T06:06:33.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.715+0000 7f6d19515640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6cfc03fb70 0x7f6cfc042080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.715+0000 7f6d19515640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d14109e00 0x7f6d1419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.715+0000 7f6d19515640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d141094b0 0x7f6d1419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.715+0000 7f6d19515640 1 --2- 172.21.15.3:0/570860206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d14103b70 0x7f6d1419a2b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.715+0000 7f6d19515640 1 -- 172.21.15.3:0/570860206 >> 172.21.15.3:0/570860206 conn(0x7f6d140ff5e0 msgr2=0x7f6d14101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:33.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.715+0000 7f6d19515640 1 -- 172.21.15.3:0/570860206 shutdown_connections 2024-09-21T06:06:33.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:32.715+0000 7f6d19515640 1 -- 172.21.15.3:0/570860206 wait complete. 2024-09-21T06:06:33.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:33 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/570860206' entity='client.admin' 2024-09-21T06:06:33.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:33 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:33.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:33 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:33.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:33 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:33.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:33 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:33.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:33 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:33.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.772+0000 7fc46c313640 1 -- 172.21.15.3:0/3182595439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4641094d0 msgr2=0x7fc4641098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.772+0000 7fc46c313640 1 --2- 172.21.15.3:0/3182595439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4641094d0 0x7fc4641098e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc458009900 tx=0x7fc45802a410 comp rx=0 tx=0).stop 2024-09-21T06:06:33.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.772+0000 7fc46c313640 1 -- 172.21.15.3:0/3182595439 shutdown_connections 2024-09-21T06:06:33.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.772+0000 7fc46c313640 1 --2- 172.21.15.3:0/3182595439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc464109e20 0x7fc464110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.772+0000 7fc46c313640 1 --2- 172.21.15.3:0/3182595439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4641094d0 0x7fc4641098e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.772+0000 7fc46c313640 1 --2- 172.21.15.3:0/3182595439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc464103b90 0x7fc464103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.772+0000 7fc46c313640 1 -- 172.21.15.3:0/3182595439 >> 172.21.15.3:0/3182595439 conn(0x7fc4640ff5e0 msgr2=0x7fc464101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:33.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.772+0000 7fc46c313640 1 -- 172.21.15.3:0/3182595439 shutdown_connections 2024-09-21T06:06:33.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.772+0000 7fc46c313640 1 -- 172.21.15.3:0/3182595439 wait complete. 2024-09-21T06:06:33.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.773+0000 7fc46c313640 1 Processor -- start 2024-09-21T06:06:33.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.773+0000 7fc46c313640 1 -- start start 2024-09-21T06:06:33.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.773+0000 7fc46c313640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc464103b90 0x7fc46419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.773+0000 7fc46c313640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4641094d0 0x7fc46419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.773+0000 7fc46c313640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc464109e20 0x7fc46419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.773+0000 7fc46c313640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641138a0 con 0x7fc4641094d0 2024-09-21T06:06:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.773+0000 7fc46c313640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc464113720 con 0x7fc464103b90 2024-09-21T06:06:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.773+0000 7fc46c313640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc464113a20 con 0x7fc464109e20 2024-09-21T06:06:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.773+0000 7fc4698ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4641094d0 0x7fc46419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.773+0000 7fc4698ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4641094d0 0x7fc46419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51830/0 (socket says 172.21.15.3:51830) 2024-09-21T06:06:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.773+0000 7fc46a0af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc464103b90 0x7fc46419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.773+0000 7fc4698ae640 1 -- 172.21.15.3:0/811445179 learned_addr learned my addr 172.21.15.3:0/811445179 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.773+0000 7fc46a8b0640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc464109e20 0x7fc46419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.774+0000 7fc4698ae640 1 -- 172.21.15.3:0/811445179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc464109e20 msgr2=0x7fc46419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.774+0000 7fc4698ae640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc464109e20 0x7fc46419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.774+0000 7fc4698ae640 1 -- 172.21.15.3:0/811445179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc464103b90 msgr2=0x7fc46419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.774+0000 7fc4698ae640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc464103b90 0x7fc46419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.774+0000 7fc4698ae640 1 -- 172.21.15.3:0/811445179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4641a02a0 con 0x7fc4641094d0 2024-09-21T06:06:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.774+0000 7fc46a0af640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc464103b90 0x7fc46419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.774+0000 7fc4698ae640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4641094d0 0x7fc46419a8d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc45802a930 tx=0x7fc458003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.774+0000 7fc4577fe640 1 -- 172.21.15.3:0/811445179 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45803f070 con 0x7fc4641094d0 2024-09-21T06:06:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.774+0000 7fc4577fe640 1 -- 172.21.15.3:0/811445179 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fc458031b10 con 0x7fc4641094d0 2024-09-21T06:06:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.774+0000 7fc46c313640 1 -- 172.21.15.3:0/811445179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4641a0530 con 0x7fc4641094d0 2024-09-21T06:06:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.774+0000 7fc4577fe640 1 -- 172.21.15.3:0/811445179 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458035030 con 0x7fc4641094d0 2024-09-21T06:06:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.774+0000 7fc46c313640 1 -- 172.21.15.3:0/811445179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4640755f0 con 0x7fc4641094d0 2024-09-21T06:06:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.775+0000 7fc46c313640 1 -- 172.21.15.3:0/811445179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc464104e10 con 0x7fc4641094d0 2024-09-21T06:06:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.776+0000 7fc4577fe640 1 -- 172.21.15.3:0/811445179 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 51914+0+0 (secure 0 0 0) 0x7fc458042070 con 0x7fc4641094d0 2024-09-21T06:06:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.776+0000 7fc4577fe640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc45003fb70 0x7fc450042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.776+0000 7fc4577fe640 1 -- 172.21.15.3:0/811445179 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc458074e00 con 0x7fc4641094d0 2024-09-21T06:06:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.776+0000 7fc46a0af640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc45003fb70 0x7fc450042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.776+0000 7fc46a0af640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc45003fb70 0x7fc450042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc460005fd0 tx=0x7fc460012040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:33.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.781+0000 7fc4577fe640 1 -- 172.21.15.3:0/811445179 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc45803b460 con 0x7fc4641094d0 2024-09-21T06:06:33.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:33 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/570860206' entity='client.admin' 2024-09-21T06:06:33.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:33 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:33.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:33 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:33.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:33 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:33.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:33 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:33.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:33 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:33 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/570860206' entity='client.admin' 2024-09-21T06:06:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:33 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:33 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:33 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:33 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:33 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:33.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.988+0000 7fc46c313640 1 -- 172.21.15.3:0/811445179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fc46410bc10 con 0x7fc4641094d0 2024-09-21T06:06:33.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.992+0000 7fc4577fe640 1 -- 172.21.15.3:0/811445179 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v16)=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fc4580316a0 con 0x7fc4641094d0 2024-09-21T06:06:33.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.995+0000 7fc46c313640 1 -- 172.21.15.3:0/811445179 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc45003fb70 msgr2=0x7fc450042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.995+0000 7fc46c313640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc45003fb70 0x7fc450042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc460005fd0 tx=0x7fc460012040 comp rx=0 tx=0).stop 2024-09-21T06:06:33.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.995+0000 7fc46c313640 1 -- 172.21.15.3:0/811445179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4641094d0 msgr2=0x7fc46419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.995+0000 7fc46c313640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4641094d0 0x7fc46419a8d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc45802a930 tx=0x7fc458003b90 comp rx=0 tx=0).stop 2024-09-21T06:06:33.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.995+0000 7fc46c313640 1 -- 172.21.15.3:0/811445179 shutdown_connections 2024-09-21T06:06:33.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.995+0000 7fc46c313640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc45003fb70 0x7fc450042080 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.995+0000 7fc46c313640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc464109e20 0x7fc46419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.995+0000 7fc46c313640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4641094d0 0x7fc46419a8d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.995+0000 7fc46c313640 1 --2- 172.21.15.3:0/811445179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc464103b90 0x7fc46419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.995+0000 7fc46c313640 1 -- 172.21.15.3:0/811445179 >> 172.21.15.3:0/811445179 conn(0x7fc4640ff5e0 msgr2=0x7fc464101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.995+0000 7fc46c313640 1 -- 172.21.15.3:0/811445179 shutdown_connections 2024-09-21T06:06:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:33.995+0000 7fc46c313640 1 -- 172.21.15.3:0/811445179 wait complete. 2024-09-21T06:06:35.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.028+0000 7f0603e69640 1 -- 172.21.15.3:0/2818647053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05fc103c30 msgr2=0x7f05fc104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.028+0000 7f0603e69640 1 --2- 172.21.15.3:0/2818647053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05fc103c30 0x7f05fc104040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009900 tx=0x7f05ec02a410 comp rx=0 tx=0).stop 2024-09-21T06:06:35.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.028+0000 7f0603e69640 1 -- 172.21.15.3:0/2818647053 shutdown_connections 2024-09-21T06:06:35.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.028+0000 7f0603e69640 1 --2- 172.21.15.3:0/2818647053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05fc109570 0x7f05fc109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.028+0000 7f0603e69640 1 --2- 172.21.15.3:0/2818647053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc109ec0 0x7f05fc1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.028+0000 7f0603e69640 1 --2- 172.21.15.3:0/2818647053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05fc103c30 0x7f05fc104040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.028+0000 7f0603e69640 1 -- 172.21.15.3:0/2818647053 >> 172.21.15.3:0/2818647053 conn(0x7f05fc07b8d0 msgr2=0x7f05fc07bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:35.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.028+0000 7f0603e69640 1 -- 172.21.15.3:0/2818647053 shutdown_connections 2024-09-21T06:06:35.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.028+0000 7f0603e69640 1 -- 172.21.15.3:0/2818647053 wait complete. 2024-09-21T06:06:35.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/811445179' entity='client.admin' 2024-09-21T06:06:35.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:35.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:35.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T06:06:35.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:06:35.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:06:35.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:35.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:34 smithi003 ceph-mon[31227]: Reconfiguring daemon mgr.x on smithi003 2024-09-21T06:06:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.029+0000 7f0603e69640 1 Processor -- start 2024-09-21T06:06:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.029+0000 7f0603e69640 1 -- start start 2024-09-21T06:06:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.029+0000 7f0603e69640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05fc103c30 0x7f05fc19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.029+0000 7f0603e69640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc109570 0x7f05fc19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0603e69640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05fc109ec0 0x7f05fc19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0603e69640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc1138a0 con 0x7f05fc103c30 2024-09-21T06:06:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0603e69640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc113720 con 0x7f05fc109570 2024-09-21T06:06:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0603e69640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc113a20 con 0x7f05fc109ec0 2024-09-21T06:06:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0601c05640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05fc103c30 0x7f05fc19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0601404640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc109570 0x7f05fc19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0601404640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc109570 0x7f05fc19a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:45792/0 (socket says 172.21.15.3:45792) 2024-09-21T06:06:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0601404640 1 -- 172.21.15.3:0/123848365 learned_addr learned my addr 172.21.15.3:0/123848365 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0602406640 1 --2- 172.21.15.3:0/123848365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05fc109ec0 0x7f05fc19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0601404640 1 -- 172.21.15.3:0/123848365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05fc109ec0 msgr2=0x7f05fc19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0601404640 1 --2- 172.21.15.3:0/123848365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05fc109ec0 0x7f05fc19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0601404640 1 -- 172.21.15.3:0/123848365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05fc103c30 msgr2=0x7f05fc19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0601404640 1 --2- 172.21.15.3:0/123848365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05fc103c30 0x7f05fc19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0601404640 1 -- 172.21.15.3:0/123848365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc1a0280 con 0x7f05fc109570 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.030+0000 7f0601404640 1 --2- 172.21.15.3:0/123848365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc109570 0x7f05fc19a8b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f05f800bcd0 tx=0x7f05f8009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.031+0000 7f05f2ffd640 1 -- 172.21.15.3:0/123848365 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8004220 con 0x7f05fc109570 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.031+0000 7f0603e69640 1 -- 172.21.15.3:0/123848365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc1a0570 con 0x7f05fc109570 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.031+0000 7f05f2ffd640 1 -- 172.21.15.3:0/123848365 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f05f80043c0 con 0x7f05fc109570 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.031+0000 7f05f2ffd640 1 -- 172.21.15.3:0/123848365 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8005520 con 0x7f05fc109570 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.031+0000 7f0603e69640 1 -- 172.21.15.3:0/123848365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc0755f0 con 0x7f05fc109570 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.032+0000 7f05f2ffd640 1 -- 172.21.15.3:0/123848365 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 51914+0+0 (secure 0 0 0) 0x7f05f80047a0 con 0x7f05fc109570 2024-09-21T06:06:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.032+0000 7f0603e69640 1 -- 172.21.15.3:0/123848365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05fc104040 con 0x7f05fc109570 2024-09-21T06:06:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.032+0000 7f05f2ffd640 1 --2- 172.21.15.3:0/123848365 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f05e803fb70 0x7f05e8042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.032+0000 7f05f2ffd640 1 -- 172.21.15.3:0/123848365 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f05f8011550 con 0x7f05fc109570 2024-09-21T06:06:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.032+0000 7f0601c05640 1 --2- 172.21.15.3:0/123848365 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f05e803fb70 0x7f05e8042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.032+0000 7f0601c05640 1 --2- 172.21.15.3:0/123848365 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f05e803fb70 0x7f05e8042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f05ec031440 tx=0x7f05ec009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:35.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.037+0000 7f05f2ffd640 1 -- 172.21.15.3:0/123848365 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f05f8016070 con 0x7f05fc109570 2024-09-21T06:06:35.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.233+0000 7f0603e69640 1 -- 172.21.15.3:0/123848365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f05fc10bc40 con 0x7f05fc109570 2024-09-21T06:06:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.240+0000 7f05f2ffd640 1 -- 172.21.15.3:0/123848365 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v17)=0 v17) ==== 115+0+0 (secure 0 0 0) 0x7f05f801adb0 con 0x7f05fc109570 2024-09-21T06:06:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.242+0000 7f0603e69640 1 -- 172.21.15.3:0/123848365 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f05e803fb70 msgr2=0x7f05e8042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.242+0000 7f0603e69640 1 --2- 172.21.15.3:0/123848365 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f05e803fb70 0x7f05e8042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f05ec031440 tx=0x7f05ec009560 comp rx=0 tx=0).stop 2024-09-21T06:06:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.243+0000 7f0603e69640 1 -- 172.21.15.3:0/123848365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc109570 msgr2=0x7f05fc19a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.243+0000 7f0603e69640 1 --2- 172.21.15.3:0/123848365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc109570 0x7f05fc19a8b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f05f800bcd0 tx=0x7f05f8009610 comp rx=0 tx=0).stop 2024-09-21T06:06:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.243+0000 7f0603e69640 1 -- 172.21.15.3:0/123848365 shutdown_connections 2024-09-21T06:06:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.243+0000 7f0603e69640 1 --2- 172.21.15.3:0/123848365 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f05e803fb70 0x7f05e8042080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.243+0000 7f0603e69640 1 --2- 172.21.15.3:0/123848365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05fc109ec0 0x7f05fc19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.243+0000 7f0603e69640 1 --2- 172.21.15.3:0/123848365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc109570 0x7f05fc19a8b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.243+0000 7f0603e69640 1 --2- 172.21.15.3:0/123848365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05fc103c30 0x7f05fc19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.243+0000 7f0603e69640 1 -- 172.21.15.3:0/123848365 >> 172.21.15.3:0/123848365 conn(0x7f05fc07b8d0 msgr2=0x7f05fc107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.243+0000 7f0603e69640 1 -- 172.21.15.3:0/123848365 shutdown_connections 2024-09-21T06:06:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.243+0000 7f0603e69640 1 -- 172.21.15.3:0/123848365 wait complete. 2024-09-21T06:06:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/811445179' entity='client.admin' 2024-09-21T06:06:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T06:06:35.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:06:35.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:06:35.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:35.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:34 smithi137 ceph-mon[34464]: Reconfiguring daemon mgr.x on smithi003 2024-09-21T06:06:35.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/811445179' entity='client.admin' 2024-09-21T06:06:35.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:35.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:35.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:35.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T06:06:35.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:06:35.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:06:35.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:35.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:34 smithi204 ceph-mon[34659]: Reconfiguring daemon mgr.x on smithi003 2024-09-21T06:06:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.646+0000 7fab43c32640 1 -- 172.21.15.3:0/3643449847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab3c102a80 msgr2=0x7fab3c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.646+0000 7fab43c32640 1 --2- 172.21.15.3:0/3643449847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab3c102a80 0x7fab3c102e90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fab38009900 tx=0x7fab3802a410 comp rx=0 tx=0).stop 2024-09-21T06:06:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.646+0000 7fab43c32640 1 -- 172.21.15.3:0/3643449847 shutdown_connections 2024-09-21T06:06:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.646+0000 7fab43c32640 1 --2- 172.21.15.3:0/3643449847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab3c108d10 0x7fab3c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.646+0000 7fab43c32640 1 --2- 172.21.15.3:0/3643449847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab3c1083c0 0x7fab3c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.646+0000 7fab43c32640 1 --2- 172.21.15.3:0/3643449847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab3c102a80 0x7fab3c102e90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.646+0000 7fab43c32640 1 -- 172.21.15.3:0/3643449847 >> 172.21.15.3:0/3643449847 conn(0x7fab3c079740 msgr2=0x7fab3c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.646+0000 7fab43c32640 1 -- 172.21.15.3:0/3643449847 shutdown_connections 2024-09-21T06:06:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.646+0000 7fab43c32640 1 -- 172.21.15.3:0/3643449847 wait complete. 2024-09-21T06:06:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.647+0000 7fab43c32640 1 Processor -- start 2024-09-21T06:06:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.647+0000 7fab43c32640 1 -- start start 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.647+0000 7fab43c32640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab3c102a80 0x7fab3c104270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.647+0000 7fab43c32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab3c1083c0 0x7fab3c1047b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.647+0000 7fab43c32640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab3c108d10 0x7fab3c104cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.647+0000 7fab43c32640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c112740 con 0x7fab3c108d10 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.647+0000 7fab43c32640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1125c0 con 0x7fab3c1083c0 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.647+0000 7fab43c32640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1128c0 con 0x7fab3c102a80 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.647+0000 7fab421cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab3c108d10 0x7fab3c104cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.647+0000 7fab419ce640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab3c102a80 0x7fab3c104270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.647+0000 7fab421cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab3c108d10 0x7fab3c104cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36112/0 (socket says 172.21.15.3:36112) 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.647+0000 7fab421cf640 1 -- 172.21.15.3:0/498690744 learned_addr learned my addr 172.21.15.3:0/498690744 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.647+0000 7fab411cd640 1 --2- 172.21.15.3:0/498690744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab3c1083c0 0x7fab3c1047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.648+0000 7fab421cf640 1 -- 172.21.15.3:0/498690744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab3c102a80 msgr2=0x7fab3c104270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.648+0000 7fab421cf640 1 --2- 172.21.15.3:0/498690744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab3c102a80 0x7fab3c104270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.648+0000 7fab421cf640 1 -- 172.21.15.3:0/498690744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab3c1083c0 msgr2=0x7fab3c1047b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.648+0000 7fab421cf640 1 --2- 172.21.15.3:0/498690744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab3c1083c0 0x7fab3c1047b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.648+0000 7fab421cf640 1 -- 172.21.15.3:0/498690744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c1a8580 con 0x7fab3c108d10 2024-09-21T06:06:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.648+0000 7fab421cf640 1 --2- 172.21.15.3:0/498690744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab3c108d10 0x7fab3c104cf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fab3400eb10 tx=0x7fab3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.648+0000 7fab32ffd640 1 -- 172.21.15.3:0/498690744 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab34004b20 con 0x7fab3c108d10 2024-09-21T06:06:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.648+0000 7fab32ffd640 1 -- 172.21.15.3:0/498690744 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fab34004cc0 con 0x7fab3c108d10 2024-09-21T06:06:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.648+0000 7fab43c32640 1 -- 172.21.15.3:0/498690744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3c1a8870 con 0x7fab3c108d10 2024-09-21T06:06:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.648+0000 7fab32ffd640 1 -- 172.21.15.3:0/498690744 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab34010690 con 0x7fab3c108d10 2024-09-21T06:06:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.648+0000 7fab43c32640 1 -- 172.21.15.3:0/498690744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab3c1a8db0 con 0x7fab3c108d10 2024-09-21T06:06:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.649+0000 7fab43c32640 1 -- 172.21.15.3:0/498690744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab3c103d00 con 0x7fab3c108d10 2024-09-21T06:06:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.649+0000 7fab32ffd640 1 -- 172.21.15.3:0/498690744 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 51914+0+0 (secure 0 0 0) 0x7fab34010830 con 0x7fab3c108d10 2024-09-21T06:06:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.650+0000 7fab32ffd640 1 --2- 172.21.15.3:0/498690744 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fab2803fb70 0x7fab28042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.650+0000 7fab32ffd640 1 -- 172.21.15.3:0/498690744 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fab34054c40 con 0x7fab3c108d10 2024-09-21T06:06:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.650+0000 7fab419ce640 1 --2- 172.21.15.3:0/498690744 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fab2803fb70 0x7fab28042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.650+0000 7fab419ce640 1 --2- 172.21.15.3:0/498690744 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fab2803fb70 0x7fab28042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fab38000c00 tx=0x7fab38031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:35.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.655+0000 7fab32ffd640 1 -- 172.21.15.3:0/498690744 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab340059f0 con 0x7fab3c108d10 2024-09-21T06:06:35.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.846+0000 7fab43c32640 1 -- 172.21.15.3:0/498690744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fab3c102e90 con 0x7fab3c108d10 2024-09-21T06:06:35.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.849+0000 7fab32ffd640 1 -- 172.21.15.3:0/498690744 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7fab34016330 con 0x7fab3c108d10 2024-09-21T06:06:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.852+0000 7fab43c32640 1 -- 172.21.15.3:0/498690744 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fab2803fb70 msgr2=0x7fab28042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.852+0000 7fab43c32640 1 --2- 172.21.15.3:0/498690744 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fab2803fb70 0x7fab28042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fab38000c00 tx=0x7fab38031400 comp rx=0 tx=0).stop 2024-09-21T06:06:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.852+0000 7fab43c32640 1 -- 172.21.15.3:0/498690744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab3c108d10 msgr2=0x7fab3c104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.852+0000 7fab43c32640 1 --2- 172.21.15.3:0/498690744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab3c108d10 0x7fab3c104cf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fab3400eb10 tx=0x7fab3400c2d0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.852+0000 7fab43c32640 1 -- 172.21.15.3:0/498690744 shutdown_connections 2024-09-21T06:06:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.852+0000 7fab43c32640 1 --2- 172.21.15.3:0/498690744 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fab2803fb70 0x7fab28042080 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.852+0000 7fab43c32640 1 --2- 172.21.15.3:0/498690744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab3c108d10 0x7fab3c104cf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.852+0000 7fab43c32640 1 --2- 172.21.15.3:0/498690744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab3c1083c0 0x7fab3c1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.852+0000 7fab43c32640 1 --2- 172.21.15.3:0/498690744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab3c102a80 0x7fab3c104270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.852+0000 7fab43c32640 1 -- 172.21.15.3:0/498690744 >> 172.21.15.3:0/498690744 conn(0x7fab3c079740 msgr2=0x7fab3c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:35.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.852+0000 7fab43c32640 1 -- 172.21.15.3:0/498690744 shutdown_connections 2024-09-21T06:06:35.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.852+0000 7fab43c32640 1 -- 172.21.15.3:0/498690744 wait complete. 2024-09-21T06:06:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:36.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.350+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/3686552374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48108d20 msgr2=0x7f2a4810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:36.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.350+0000 7f2a4e9cb640 1 --2- 172.21.15.3:0/3686552374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48108d20 0x7f2a4810f550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009ca0 tx=0x7f2a4002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:36.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.351+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/3686552374 shutdown_connections 2024-09-21T06:06:36.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.351+0000 7f2a4e9cb640 1 --2- 172.21.15.3:0/3686552374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48108d20 0x7f2a4810f550 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:36.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.351+0000 7f2a4e9cb640 1 --2- 172.21.15.3:0/3686552374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a481083d0 0x7f2a481087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:36.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.351+0000 7f2a4e9cb640 1 --2- 172.21.15.3:0/3686552374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a48102a90 0x7f2a48102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:36.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.351+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/3686552374 >> 172.21.15.3:0/3686552374 conn(0x7f2a48079750 msgr2=0x7f2a48079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:36.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.351+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/3686552374 shutdown_connections 2024-09-21T06:06:36.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.351+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/3686552374 wait complete. 2024-09-21T06:06:36.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.351+0000 7f2a4e9cb640 1 Processor -- start 2024-09-21T06:06:36.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.351+0000 7f2a4e9cb640 1 -- start start 2024-09-21T06:06:36.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4e9cb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a48102a90 0x7f2a4819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:36.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4e9cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a481083d0 0x7f2a4819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:36.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4e9cb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48108d20 0x7f2a4819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:36.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4e9cb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481138a0 con 0x7f2a48108d20 2024-09-21T06:06:36.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4e9cb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48113720 con 0x7f2a481083d0 2024-09-21T06:06:36.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4e9cb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48113a20 con 0x7f2a48102a90 2024-09-21T06:06:36.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4cf68640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48108d20 0x7f2a4819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:36.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4cf68640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48108d20 0x7f2a4819fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36094/0 (socket says 172.21.15.3:36094) 2024-09-21T06:06:36.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4cf68640 1 -- 172.21.15.3:0/71889362 learned_addr learned my addr 172.21.15.3:0/71889362 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:36.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a47fff640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a48102a90 0x7f2a4819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:36.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a477fe640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a481083d0 0x7f2a4819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:36.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4cf68640 1 -- 172.21.15.3:0/71889362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a48102a90 msgr2=0x7f2a4819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:36.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4cf68640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a48102a90 0x7f2a4819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:36.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4cf68640 1 -- 172.21.15.3:0/71889362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a481083d0 msgr2=0x7f2a4819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4cf68640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a481083d0 0x7f2a4819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4cf68640 1 -- 172.21.15.3:0/71889362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a481a0260 con 0x7f2a48108d20 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.352+0000 7f2a4cf68640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48108d20 0x7f2a4819fb30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2a40033010 tx=0x7f2a40009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.353+0000 7f2a457fa640 1 -- 172.21.15.3:0/71889362 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40003cb0 con 0x7f2a48108d20 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.353+0000 7f2a457fa640 1 -- 172.21.15.3:0/71889362 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f2a40003e50 con 0x7f2a48108d20 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.353+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/71889362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a481a04f0 con 0x7f2a48108d20 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.353+0000 7f2a457fa640 1 -- 172.21.15.3:0/71889362 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a400309d0 con 0x7f2a48108d20 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.353+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/71889362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a481a6cb0 con 0x7f2a48108d20 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.354+0000 7f2a457fa640 1 -- 172.21.15.3:0/71889362 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 51914+0+0 (secure 0 0 0) 0x7f2a40030bb0 con 0x7f2a48108d20 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.354+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/71889362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a48103d10 con 0x7f2a48108d20 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.354+0000 7f2a457fa640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a3003fb70 0x7f2a30042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.354+0000 7f2a457fa640 1 -- 172.21.15.3:0/71889362 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f2a400759e0 con 0x7f2a48108d20 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.354+0000 7f2a47fff640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a3003fb70 0x7f2a30042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.355+0000 7f2a47fff640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a3003fb70 0x7f2a30042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009bd0 tx=0x7f2a38005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.359+0000 7f2a457fa640 1 -- 172.21.15.3:0/71889362 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2a40044ca0 con 0x7f2a48108d20 2024-09-21T06:06:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.554+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/71889362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7f2a4810ab10 con 0x7f2a48108d20 2024-09-21T06:06:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.557+0000 7f2a457fa640 1 -- 172.21.15.3:0/71889362 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v18)=0 v18) ==== 115+0+0 (secure 0 0 0) 0x7f2a4003b410 con 0x7f2a48108d20 2024-09-21T06:06:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.560+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/71889362 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a3003fb70 msgr2=0x7f2a30042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.560+0000 7f2a4e9cb640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a3003fb70 0x7f2a30042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009bd0 tx=0x7f2a38005f90 comp rx=0 tx=0).stop 2024-09-21T06:06:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.560+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/71889362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48108d20 msgr2=0x7f2a4819fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.560+0000 7f2a4e9cb640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48108d20 0x7f2a4819fb30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2a40033010 tx=0x7f2a40009880 comp rx=0 tx=0).stop 2024-09-21T06:06:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.560+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/71889362 shutdown_connections 2024-09-21T06:06:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.560+0000 7f2a4e9cb640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a3003fb70 0x7f2a30042080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.560+0000 7f2a4e9cb640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48108d20 0x7f2a4819fb30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.560+0000 7f2a4e9cb640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a481083d0 0x7f2a4819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.560+0000 7f2a4e9cb640 1 --2- 172.21.15.3:0/71889362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a48102a90 0x7f2a4819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.560+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/71889362 >> 172.21.15.3:0/71889362 conn(0x7f2a48079750 msgr2=0x7f2a48104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.560+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/71889362 shutdown_connections 2024-09-21T06:06:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.560+0000 7f2a4e9cb640 1 -- 172.21.15.3:0/71889362 wait complete. 2024-09-21T06:06:36.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:36 smithi003 ceph-mon[31227]: from='client.? ' entity='client.admin' 2024-09-21T06:06:36.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:36 smithi003 ceph-mon[31227]: Standby manager daemon z started 2024-09-21T06:06:36.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:36 smithi003 ceph-mon[31227]: from='mgr.? 172.21.15.204:0/3957024751' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T06:06:36.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:36 smithi003 ceph-mon[31227]: from='mgr.? 172.21.15.204:0/3957024751' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:06:36.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:36 smithi003 ceph-mon[31227]: from='mgr.? 172.21.15.204:0/3957024751' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T06:06:36.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:36 smithi003 ceph-mon[31227]: from='mgr.? 172.21.15.204:0/3957024751' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:06:36.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:36 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/71889362' entity='client.admin' 2024-09-21T06:06:36.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:36 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/498690744' entity='client.admin' 2024-09-21T06:06:36.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:36 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1404730018' entity='client.admin' 2024-09-21T06:06:36.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:36 smithi137 ceph-mon[34464]: from='client.? ' entity='client.admin' 2024-09-21T06:06:36.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:36 smithi137 ceph-mon[34464]: Standby manager daemon z started 2024-09-21T06:06:36.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:36 smithi137 ceph-mon[34464]: from='mgr.? 172.21.15.204:0/3957024751' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T06:06:36.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:36 smithi137 ceph-mon[34464]: from='mgr.? 172.21.15.204:0/3957024751' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:06:36.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:36 smithi137 ceph-mon[34464]: from='mgr.? 172.21.15.204:0/3957024751' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T06:06:36.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:36 smithi137 ceph-mon[34464]: from='mgr.? 172.21.15.204:0/3957024751' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:06:36.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:36 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/71889362' entity='client.admin' 2024-09-21T06:06:36.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:36 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/498690744' entity='client.admin' 2024-09-21T06:06:36.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:36 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1404730018' entity='client.admin' 2024-09-21T06:06:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:36 smithi204 ceph-mon[34659]: from='client.? ' entity='client.admin' 2024-09-21T06:06:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:36 smithi204 ceph-mon[34659]: Standby manager daemon z started 2024-09-21T06:06:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:36 smithi204 ceph-mon[34659]: from='mgr.? 172.21.15.204:0/3957024751' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T06:06:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:36 smithi204 ceph-mon[34659]: from='mgr.? 172.21.15.204:0/3957024751' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:06:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:36 smithi204 ceph-mon[34659]: from='mgr.? 172.21.15.204:0/3957024751' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T06:06:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:36 smithi204 ceph-mon[34659]: from='mgr.? 172.21.15.204:0/3957024751' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:06:36.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:36 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/71889362' entity='client.admin' 2024-09-21T06:06:36.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:36 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/498690744' entity='client.admin' 2024-09-21T06:06:36.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:36 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1404730018' entity='client.admin' 2024-09-21T06:06:37.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.262+0000 7f703aca0640 1 -- 172.21.15.3:0/2733223072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70341094c0 msgr2=0x7f70341098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.262+0000 7f703aca0640 1 --2- 172.21.15.3:0/2733223072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70341094c0 0x7f70341098d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7020009900 tx=0x7f702002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.262+0000 7f703aca0640 1 -- 172.21.15.3:0/2733223072 shutdown_connections 2024-09-21T06:06:37.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.262+0000 7f703aca0640 1 --2- 172.21.15.3:0/2733223072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7034109e10 0x7f7034110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.262+0000 7f703aca0640 1 --2- 172.21.15.3:0/2733223072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70341094c0 0x7f70341098d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.262+0000 7f703aca0640 1 --2- 172.21.15.3:0/2733223072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7034103b80 0x7f7034103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.262+0000 7f703aca0640 1 -- 172.21.15.3:0/2733223072 >> 172.21.15.3:0/2733223072 conn(0x7f70340ff610 msgr2=0x7f7034101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:37.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.262+0000 7f703aca0640 1 -- 172.21.15.3:0/2733223072 shutdown_connections 2024-09-21T06:06:37.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.262+0000 7f703aca0640 1 -- 172.21.15.3:0/2733223072 wait complete. 2024-09-21T06:06:37.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.263+0000 7f703aca0640 1 Processor -- start 2024-09-21T06:06:37.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.263+0000 7f703aca0640 1 -- start start 2024-09-21T06:06:37.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.263+0000 7f703aca0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7034103b80 0x7f703419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.263+0000 7f703aca0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70341094c0 0x7f703419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.263+0000 7f703aca0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7034109e10 0x7f703419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.263+0000 7f703aca0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f70341138e0 con 0x7f70341094c0 2024-09-21T06:06:37.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.263+0000 7f703aca0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7034113760 con 0x7f7034109e10 2024-09-21T06:06:37.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.263+0000 7f703aca0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7034113a60 con 0x7f7034103b80 2024-09-21T06:06:37.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.264+0000 7f703923d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7034109e10 0x7f703419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.264+0000 7f7033fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70341094c0 0x7f703419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.264+0000 7f7033fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70341094c0 0x7f703419a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36142/0 (socket says 172.21.15.3:36142) 2024-09-21T06:06:37.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.264+0000 7f703923d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7034109e10 0x7f703419fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:41620/0 (socket says 172.21.15.3:41620) 2024-09-21T06:06:37.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.264+0000 7f7033fff640 1 -- 172.21.15.3:0/796682064 learned_addr learned my addr 172.21.15.3:0/796682064 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:37.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.264+0000 7f7038a3c640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7034103b80 0x7f703419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.264+0000 7f7038a3c640 1 -- 172.21.15.3:0/796682064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7034109e10 msgr2=0x7f703419fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.264+0000 7f7038a3c640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7034109e10 0x7f703419fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.264+0000 7f7038a3c640 1 -- 172.21.15.3:0/796682064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70341094c0 msgr2=0x7f703419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.264+0000 7f7038a3c640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70341094c0 0x7f703419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.264+0000 7f7038a3c640 1 -- 172.21.15.3:0/796682064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70341a0280 con 0x7f7034103b80 2024-09-21T06:06:37.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.264+0000 7f703923d640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7034109e10 0x7f703419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:37.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.264+0000 7f7038a3c640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7034103b80 0x7f703419a310 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f702800e9f0 tx=0x7f702800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:37.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.265+0000 7f7031ffb640 1 -- 172.21.15.3:0/796682064 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702800ccc0 con 0x7f7034103b80 2024-09-21T06:06:37.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.265+0000 7f703aca0640 1 -- 172.21.15.3:0/796682064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70341a0570 con 0x7f7034103b80 2024-09-21T06:06:37.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.265+0000 7f7031ffb640 1 -- 172.21.15.3:0/796682064 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f70280108a0 con 0x7f7034103b80 2024-09-21T06:06:37.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.265+0000 7f7031ffb640 1 -- 172.21.15.3:0/796682064 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028018640 con 0x7f7034103b80 2024-09-21T06:06:37.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.265+0000 7f703aca0640 1 -- 172.21.15.3:0/796682064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70341a6cd0 con 0x7f7034103b80 2024-09-21T06:06:37.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.267+0000 7f703aca0640 1 -- 172.21.15.3:0/796682064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7034103f90 con 0x7f7034103b80 2024-09-21T06:06:37.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.267+0000 7f7031ffb640 1 -- 172.21.15.3:0/796682064 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 19) ==== 103193+0+0 (secure 0 0 0) 0x7f70280187e0 con 0x7f7034103b80 2024-09-21T06:06:37.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.267+0000 7f7031ffb640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f702407b020 0x7f702407d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.268+0000 7f7031ffb640 1 -- 172.21.15.3:0/796682064 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f702809d2d0 con 0x7f7034103b80 2024-09-21T06:06:37.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.268+0000 7f7033fff640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f702407b020 0x7f702407d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.268+0000 7f7033fff640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f702407b020 0x7f702407d530 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7020006010 tx=0x7f7020005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:37.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.273+0000 7f7031ffb640 1 -- 172.21.15.3:0/796682064 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7028069bc0 con 0x7f7034103b80 2024-09-21T06:06:37.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.462+0000 7f703aca0640 1 -- 172.21.15.3:0/796682064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f703410bc00 con 0x7f7034103b80 2024-09-21T06:06:37.464 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:37 smithi003 ceph-mon[31227]: mgrmap e19: x(active, since 12s), standbys: z 2024-09-21T06:06:37.464 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:37 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T06:06:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.470+0000 7f7031ffb640 1 -- 172.21.15.3:0/796682064 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v21)=0 v21) ==== 108+0+0 (secure 0 0 0) 0x7f702806ea70 con 0x7f7034103b80 2024-09-21T06:06:37.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.472+0000 7f703aca0640 1 -- 172.21.15.3:0/796682064 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f702407b020 msgr2=0x7f702407d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.472+0000 7f703aca0640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f702407b020 0x7f702407d530 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7020006010 tx=0x7f7020005a20 comp rx=0 tx=0).stop 2024-09-21T06:06:37.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.472+0000 7f703aca0640 1 -- 172.21.15.3:0/796682064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7034103b80 msgr2=0x7f703419a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.472+0000 7f703aca0640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7034103b80 0x7f703419a310 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f702800e9f0 tx=0x7f702800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.472+0000 7f703aca0640 1 -- 172.21.15.3:0/796682064 shutdown_connections 2024-09-21T06:06:37.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.472+0000 7f703aca0640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f702407b020 0x7f702407d530 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.472+0000 7f703aca0640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7034109e10 0x7f703419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.472+0000 7f703aca0640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70341094c0 0x7f703419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.472+0000 7f703aca0640 1 --2- 172.21.15.3:0/796682064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7034103b80 0x7f703419a310 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.472+0000 7f703aca0640 1 -- 172.21.15.3:0/796682064 >> 172.21.15.3:0/796682064 conn(0x7f70340ff610 msgr2=0x7f7034101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:37.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.472+0000 7f703aca0640 1 -- 172.21.15.3:0/796682064 shutdown_connections 2024-09-21T06:06:37.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:37.472+0000 7f703aca0640 1 -- 172.21.15.3:0/796682064 wait complete. 2024-09-21T06:06:37.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:37 smithi137 ceph-mon[34464]: mgrmap e19: x(active, since 12s), standbys: z 2024-09-21T06:06:37.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:37 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T06:06:37.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:37 smithi204 ceph-mon[34659]: mgrmap e19: x(active, since 12s), standbys: z 2024-09-21T06:06:37.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:37 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T06:06:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.154+0000 7ff098104640 1 -- 172.21.15.3:0/706966841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff090105340 msgr2=0x7ff09010d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.154+0000 7ff098104640 1 --2- 172.21.15.3:0/706966841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff090105340 0x7ff09010d890 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff080009900 tx=0x7ff08002a410 comp rx=0 tx=0).stop 2024-09-21T06:06:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.154+0000 7ff098104640 1 -- 172.21.15.3:0/706966841 shutdown_connections 2024-09-21T06:06:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.154+0000 7ff098104640 1 --2- 172.21.15.3:0/706966841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff09010de50 0x7ff090110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.154+0000 7ff098104640 1 --2- 172.21.15.3:0/706966841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff090105340 0x7ff09010d890 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.154+0000 7ff098104640 1 --2- 172.21.15.3:0/706966841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0901049f0 0x7ff090104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.154+0000 7ff098104640 1 -- 172.21.15.3:0/706966841 >> 172.21.15.3:0/706966841 conn(0x7ff0900797e0 msgr2=0x7ff090079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.154+0000 7ff098104640 1 -- 172.21.15.3:0/706966841 shutdown_connections 2024-09-21T06:06:38.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.154+0000 7ff098104640 1 -- 172.21.15.3:0/706966841 wait complete. 2024-09-21T06:06:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff098104640 1 Processor -- start 2024-09-21T06:06:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff098104640 1 -- start start 2024-09-21T06:06:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff098104640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0901049f0 0x7ff090102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff098104640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff090105340 0x7ff090102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff098104640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff09010de50 0x7ff0901031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff098104640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff090113870 con 0x7ff0901049f0 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff098104640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901136f0 con 0x7ff09010de50 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff098104640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901139f0 con 0x7ff090105340 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff095ea0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0901049f0 0x7ff090102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff095ea0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0901049f0 0x7ff090102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36150/0 (socket says 172.21.15.3:36150) 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff095ea0640 1 -- 172.21.15.3:0/4025591086 learned_addr learned my addr 172.21.15.3:0/4025591086 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff0966a1640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff09010de50 0x7ff0901031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff09569f640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff090105340 0x7ff090102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff095ea0640 1 -- 172.21.15.3:0/4025591086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff090105340 msgr2=0x7ff090102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff095ea0640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff090105340 0x7ff090102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff095ea0640 1 -- 172.21.15.3:0/4025591086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff09010de50 msgr2=0x7ff0901031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff095ea0640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff09010de50 0x7ff0901031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.155+0000 7ff095ea0640 1 -- 172.21.15.3:0/4025591086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0901a8590 con 0x7ff0901049f0 2024-09-21T06:06:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.156+0000 7ff095ea0640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0901049f0 0x7ff090102730 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c970 tx=0x7ff08c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.156+0000 7ff086ffd640 1 -- 172.21.15.3:0/4025591086 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c007c10 con 0x7ff0901049f0 2024-09-21T06:06:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.156+0000 7ff098104640 1 -- 172.21.15.3:0/4025591086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0901a8880 con 0x7ff0901049f0 2024-09-21T06:06:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.156+0000 7ff086ffd640 1 -- 172.21.15.3:0/4025591086 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7ff08c007db0 con 0x7ff0901049f0 2024-09-21T06:06:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.156+0000 7ff086ffd640 1 -- 172.21.15.3:0/4025591086 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00f6b0 con 0x7ff0901049f0 2024-09-21T06:06:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.156+0000 7ff098104640 1 -- 172.21.15.3:0/4025591086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0901a8d90 con 0x7ff0901049f0 2024-09-21T06:06:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.157+0000 7ff098104640 1 -- 172.21.15.3:0/4025591086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff090104e00 con 0x7ff0901049f0 2024-09-21T06:06:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.158+0000 7ff086ffd640 1 -- 172.21.15.3:0/4025591086 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 103193+0+0 (secure 0 0 0) 0x7ff08c018480 con 0x7ff0901049f0 2024-09-21T06:06:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.158+0000 7ff086ffd640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff07c07b020 0x7ff07c07d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.158+0000 7ff086ffd640 1 -- 172.21.15.3:0/4025591086 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff08c09d1b0 con 0x7ff0901049f0 2024-09-21T06:06:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.158+0000 7ff09569f640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff07c07b020 0x7ff07c07d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.159+0000 7ff09569f640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff07c07b020 0x7ff07c07d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff080009230 tx=0x7ff080031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:38.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.163+0000 7ff086ffd640 1 -- 172.21.15.3:0/4025591086 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff08c065760 con 0x7ff0901049f0 2024-09-21T06:06:38.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.354+0000 7ff098104640 1 -- 172.21.15.3:0/4025591086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7ff090108f50 con 0x7ff0901049f0 2024-09-21T06:06:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.358+0000 7ff086ffd640 1 -- 172.21.15.3:0/4025591086 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v22)=0 v22) ==== 120+0+0 (secure 0 0 0) 0x7ff08c069b40 con 0x7ff0901049f0 2024-09-21T06:06:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.360+0000 7ff098104640 1 -- 172.21.15.3:0/4025591086 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff07c07b020 msgr2=0x7ff07c07d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.360+0000 7ff098104640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff07c07b020 0x7ff07c07d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff080009230 tx=0x7ff080031400 comp rx=0 tx=0).stop 2024-09-21T06:06:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.360+0000 7ff098104640 1 -- 172.21.15.3:0/4025591086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0901049f0 msgr2=0x7ff090102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.360+0000 7ff098104640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0901049f0 0x7ff090102730 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c970 tx=0x7ff08c00ce30 comp rx=0 tx=0).stop 2024-09-21T06:06:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.360+0000 7ff098104640 1 -- 172.21.15.3:0/4025591086 shutdown_connections 2024-09-21T06:06:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.360+0000 7ff098104640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff07c07b020 0x7ff07c07d530 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.360+0000 7ff098104640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff090105340 0x7ff090102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.360+0000 7ff098104640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff09010de50 0x7ff0901031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.360+0000 7ff098104640 1 --2- 172.21.15.3:0/4025591086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0901049f0 0x7ff090102730 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.360+0000 7ff098104640 1 -- 172.21.15.3:0/4025591086 >> 172.21.15.3:0/4025591086 conn(0x7ff0900797e0 msgr2=0x7ff090100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:38.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.360+0000 7ff098104640 1 -- 172.21.15.3:0/4025591086 shutdown_connections 2024-09-21T06:06:38.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:38.360+0000 7ff098104640 1 -- 172.21.15.3:0/4025591086 wait complete. 2024-09-21T06:06:38.616 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:38 smithi003 ceph-mon[31227]: from='client.? ' entity='client.admin' 2024-09-21T06:06:38.617 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:38 smithi003 ceph-mon[31227]: Standby manager daemon y started 2024-09-21T06:06:38.617 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:38 smithi003 ceph-mon[31227]: from='mgr.? 172.21.15.137:0/1783641546' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T06:06:38.617 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:38 smithi003 ceph-mon[31227]: from='mgr.? 172.21.15.137:0/1783641546' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:06:38.617 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:38 smithi003 ceph-mon[31227]: from='mgr.? 172.21.15.137:0/1783641546' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T06:06:38.617 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:38 smithi003 ceph-mon[31227]: from='mgr.? 172.21.15.137:0/1783641546' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:06:38.617 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:38 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/4025591086' entity='client.admin' 2024-09-21T06:06:38.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:38 smithi137 ceph-mon[34464]: from='client.? ' entity='client.admin' 2024-09-21T06:06:38.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:38 smithi137 ceph-mon[34464]: Standby manager daemon y started 2024-09-21T06:06:38.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:38 smithi137 ceph-mon[34464]: from='mgr.? 172.21.15.137:0/1783641546' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T06:06:38.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:38 smithi137 ceph-mon[34464]: from='mgr.? 172.21.15.137:0/1783641546' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:06:38.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:38 smithi137 ceph-mon[34464]: from='mgr.? 172.21.15.137:0/1783641546' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T06:06:38.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:38 smithi137 ceph-mon[34464]: from='mgr.? 172.21.15.137:0/1783641546' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:06:38.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:38 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/4025591086' entity='client.admin' 2024-09-21T06:06:38.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:38 smithi204 ceph-mon[34659]: from='client.? ' entity='client.admin' 2024-09-21T06:06:38.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:38 smithi204 ceph-mon[34659]: Standby manager daemon y started 2024-09-21T06:06:38.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:38 smithi204 ceph-mon[34659]: from='mgr.? 172.21.15.137:0/1783641546' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T06:06:38.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:38 smithi204 ceph-mon[34659]: from='mgr.? 172.21.15.137:0/1783641546' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:06:38.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:38 smithi204 ceph-mon[34659]: from='mgr.? 172.21.15.137:0/1783641546' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T06:06:38.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:38 smithi204 ceph-mon[34659]: from='mgr.? 172.21.15.137:0/1783641546' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:06:38.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:38 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/4025591086' entity='client.admin' 2024-09-21T06:06:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.009+0000 7effdd006640 1 -- 172.21.15.3:0/3934449345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effd8104840 msgr2=0x7effd8104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.009+0000 7effdd006640 1 --2- 172.21.15.3:0/3934449345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effd8104840 0x7effd8104c50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7effc8009ca0 tx=0x7effc802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.009+0000 7effdd006640 1 -- 172.21.15.3:0/3934449345 shutdown_connections 2024-09-21T06:06:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.009+0000 7effdd006640 1 --2- 172.21.15.3:0/3934449345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effd8105270 0x7effd810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.009+0000 7effdd006640 1 --2- 172.21.15.3:0/3934449345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effd810ddf0 0x7effd8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.009+0000 7effdd006640 1 --2- 172.21.15.3:0/3934449345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effd8104840 0x7effd8104c50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.009+0000 7effdd006640 1 -- 172.21.15.3:0/3934449345 >> 172.21.15.3:0/3934449345 conn(0x7effd8079730 msgr2=0x7effd8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.009+0000 7effdd006640 1 -- 172.21.15.3:0/3934449345 shutdown_connections 2024-09-21T06:06:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.009+0000 7effdd006640 1 -- 172.21.15.3:0/3934449345 wait complete. 2024-09-21T06:06:39.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.010+0000 7effdd006640 1 Processor -- start 2024-09-21T06:06:39.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.010+0000 7effdd006640 1 -- start start 2024-09-21T06:06:39.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.010+0000 7effdd006640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effd8104840 0x7effd819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.010+0000 7effdd006640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effd8105270 0x7effd819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.010+0000 7effdd006640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effd810ddf0 0x7effd819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.010+0000 7effdd006640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7effd81137e0 con 0x7effd810ddf0 2024-09-21T06:06:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.010+0000 7effdd006640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7effd8113660 con 0x7effd8104840 2024-09-21T06:06:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.010+0000 7effdd006640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7effd8113960 con 0x7effd8105270 2024-09-21T06:06:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.010+0000 7effd759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effd810ddf0 0x7effd819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.010+0000 7effd759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effd810ddf0 0x7effd819fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36162/0 (socket says 172.21.15.3:36162) 2024-09-21T06:06:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.010+0000 7effd759e640 1 -- 172.21.15.3:0/3553403854 learned_addr learned my addr 172.21.15.3:0/3553403854 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.010+0000 7effd6d9d640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effd8104840 0x7effd819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.010+0000 7effd659c640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effd8105270 0x7effd819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.011+0000 7effd759e640 1 -- 172.21.15.3:0/3553403854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effd8105270 msgr2=0x7effd819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.011+0000 7effd759e640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effd8105270 0x7effd819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.011+0000 7effd759e640 1 -- 172.21.15.3:0/3553403854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effd8104840 msgr2=0x7effd819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.011+0000 7effd759e640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effd8104840 0x7effd819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.011+0000 7effd759e640 1 -- 172.21.15.3:0/3553403854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd81a02a0 con 0x7effd810ddf0 2024-09-21T06:06:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.011+0000 7effd6d9d640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effd8104840 0x7effd819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.011+0000 7effd759e640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effd810ddf0 0x7effd819fb70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7effd000d750 tx=0x7effd000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.011+0000 7effcffff640 1 -- 172.21.15.3:0/3553403854 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0017070 con 0x7effd810ddf0 2024-09-21T06:06:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.011+0000 7effcffff640 1 -- 172.21.15.3:0/3553403854 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7effd0005490 con 0x7effd810ddf0 2024-09-21T06:06:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.011+0000 7effdd006640 1 -- 172.21.15.3:0/3553403854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd81a0590 con 0x7effd810ddf0 2024-09-21T06:06:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.011+0000 7effcffff640 1 -- 172.21.15.3:0/3553403854 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0004b40 con 0x7effd810ddf0 2024-09-21T06:06:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.011+0000 7effdd006640 1 -- 172.21.15.3:0/3553403854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd81a6cf0 con 0x7effd810ddf0 2024-09-21T06:06:39.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.012+0000 7effdd006640 1 -- 172.21.15.3:0/3553403854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd8104c50 con 0x7effd810ddf0 2024-09-21T06:06:39.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.014+0000 7effcffff640 1 -- 172.21.15.3:0/3553403854 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7effd0004ce0 con 0x7effd810ddf0 2024-09-21T06:06:39.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.015+0000 7effcffff640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7effc40b64d0 0x7effc40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:39.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.015+0000 7effd6d9d640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7effc40b64d0 0x7effc40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:39.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.015+0000 7effcffff640 1 -- 172.21.15.3:0/3553403854 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7effd00e55e0 con 0x7effd810ddf0 2024-09-21T06:06:39.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.015+0000 7effd6d9d640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7effc40b64d0 0x7effc40b89e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7effc80094e0 tx=0x7effc80093f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:39.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.018+0000 7effcffff640 1 -- 172.21.15.3:0/3553403854 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7effd00ada70 con 0x7effd810ddf0 2024-09-21T06:06:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.862+0000 7f12ccfd2640 1 -- 172.21.15.3:0/2024912668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c8102a60 msgr2=0x7f12c8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.862+0000 7f12ccfd2640 1 --2- 172.21.15.3:0/2024912668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c8102a60 0x7f12c8102e70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009900 tx=0x7f12b802a410 comp rx=0 tx=0).stop 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.862+0000 7f12ccfd2640 1 -- 172.21.15.3:0/2024912668 shutdown_connections 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.862+0000 7f12ccfd2640 1 --2- 172.21.15.3:0/2024912668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12c81083a0 0x7f12c81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.862+0000 7f12ccfd2640 1 --2- 172.21.15.3:0/2024912668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12c8108cf0 0x7f12c810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.862+0000 7f12ccfd2640 1 --2- 172.21.15.3:0/2024912668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c8102a60 0x7f12c8102e70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.862+0000 7f12ccfd2640 1 -- 172.21.15.3:0/2024912668 >> 172.21.15.3:0/2024912668 conn(0x7f12c8079730 msgr2=0x7f12c8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.863+0000 7f12ccfd2640 1 -- 172.21.15.3:0/2024912668 shutdown_connections 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.863+0000 7f12ccfd2640 1 -- 172.21.15.3:0/2024912668 wait complete. 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.863+0000 7f12ccfd2640 1 Processor -- start 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.863+0000 7f12ccfd2640 1 -- start start 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12ccfd2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c8102a60 0x7f12c81044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12ccfd2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12c81083a0 0x7f12c8104a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12ccfd2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12c8108cf0 0x7f12c8104f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12ccfd2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f12c8112720 con 0x7f12c8102a60 2024-09-21T06:06:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12ccfd2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12c81125a0 con 0x7f12c8108cf0 2024-09-21T06:06:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12ccfd2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f12c81128a0 con 0x7f12c81083a0 2024-09-21T06:06:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12c659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c8102a60 0x7f12c81044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12c659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c8102a60 0x7f12c81044e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36132/0 (socket says 172.21.15.3:36132) 2024-09-21T06:06:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12c659c640 1 -- 172.21.15.3:0/1404730018 learned_addr learned my addr 172.21.15.3:0/1404730018 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12c5d9b640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12c81083a0 0x7f12c8104a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12c6d9d640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12c8108cf0 0x7f12c8104f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12c659c640 1 -- 172.21.15.3:0/1404730018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12c81083a0 msgr2=0x7f12c8104a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12c659c640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12c81083a0 0x7f12c8104a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12c659c640 1 -- 172.21.15.3:0/1404730018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12c8108cf0 msgr2=0x7f12c8104f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12c659c640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12c8108cf0 0x7f12c8104f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.864+0000 7f12c659c640 1 -- 172.21.15.3:0/1404730018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c81a85a0 con 0x7f12c8102a60 2024-09-21T06:06:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.865+0000 7f12c659c640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c8102a60 0x7f12c81044e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f12b8005940 tx=0x7f12b8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.865+0000 7f12bf7fe640 1 -- 172.21.15.3:0/1404730018 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b803f070 con 0x7f12c8102a60 2024-09-21T06:06:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.865+0000 7f12bf7fe640 1 -- 172.21.15.3:0/1404730018 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f12b8031b20 con 0x7f12c8102a60 2024-09-21T06:06:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.865+0000 7f12ccfd2640 1 -- 172.21.15.3:0/1404730018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c81a8830 con 0x7f12c8102a60 2024-09-21T06:06:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.865+0000 7f12bf7fe640 1 -- 172.21.15.3:0/1404730018 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8035030 con 0x7f12c8102a60 2024-09-21T06:06:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.865+0000 7f12ccfd2640 1 -- 172.21.15.3:0/1404730018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c81a8d60 con 0x7f12c8102a60 2024-09-21T06:06:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.866+0000 7f12bf7fe640 1 -- 172.21.15.3:0/1404730018 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 51914+0+0 (secure 0 0 0) 0x7f12b8042070 con 0x7f12c8102a60 2024-09-21T06:06:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.866+0000 7f12bf7fe640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12b403fb70 0x7f12b4042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.866+0000 7f12bf7fe640 1 -- 172.21.15.3:0/1404730018 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f12b8031cd0 con 0x7f12c8102a60 2024-09-21T06:06:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.866+0000 7f12c5d9b640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12b403fb70 0x7f12b4042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.866+0000 7f12ccfd2640 1 -- 172.21.15.3:0/1404730018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c8102e70 con 0x7f12c8102a60 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.867+0000 7f12c5d9b640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12b403fb70 0x7f12b4042080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f12c8100e00 tx=0x7f12b000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:35.872+0000 7f12bf7fe640 1 -- 172.21.15.3:0/1404730018 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12b8042360 con 0x7f12c8102a60 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.078+0000 7f12ccfd2640 1 -- 172.21.15.3:0/1404730018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f12c81a9080 con 0x7f12c8102a60 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.082+0000 7f12bf7fe640 1 -- 172.21.15.3:0/1404730018 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v20)=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f12b8041db0 con 0x7f12c8102a60 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.084+0000 7f12ccfd2640 1 -- 172.21.15.3:0/1404730018 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12b403fb70 msgr2=0x7f12b4042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.084+0000 7f12ccfd2640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12b403fb70 0x7f12b4042080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f12c8100e00 tx=0x7f12b000b390 comp rx=0 tx=0).stop 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.084+0000 7f12ccfd2640 1 -- 172.21.15.3:0/1404730018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c8102a60 msgr2=0x7f12c81044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.084+0000 7f12ccfd2640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c8102a60 0x7f12c81044e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f12b8005940 tx=0x7f12b8003c10 comp rx=0 tx=0).stop 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.084+0000 7f12ccfd2640 1 -- 172.21.15.3:0/1404730018 shutdown_connections 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.084+0000 7f12ccfd2640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12b403fb70 0x7f12b4042080 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.084+0000 7f12ccfd2640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12c81083a0 0x7f12c8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.084+0000 7f12ccfd2640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12c8108cf0 0x7f12c8104f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.084+0000 7f12ccfd2640 1 --2- 172.21.15.3:0/1404730018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c8102a60 0x7f12c81044e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.084+0000 7f12ccfd2640 1 -- 172.21.15.3:0/1404730018 >> 172.21.15.3:0/1404730018 conn(0x7f12c8079730 msgr2=0x7f12c8106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:39.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.084+0000 7f12ccfd2640 1 -- 172.21.15.3:0/1404730018 shutdown_connections 2024-09-21T06:06:39.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:36.084+0000 7f12ccfd2640 1 -- 172.21.15.3:0/1404730018 wait complete. 2024-09-21T06:06:39.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.209+0000 7effdd006640 1 -- 172.21.15.3:0/3553403854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7effd8108f70 con 0x7effd810ddf0 2024-09-21T06:06:39.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.213+0000 7effcffff640 1 -- 172.21.15.3:0/3553403854 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v23)=0 v23) ==== 125+0+0 (secure 0 0 0) 0x7effd00b1e50 con 0x7effd810ddf0 2024-09-21T06:06:39.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.216+0000 7effdd006640 1 -- 172.21.15.3:0/3553403854 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7effc40b64d0 msgr2=0x7effc40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:39.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.216+0000 7effdd006640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7effc40b64d0 0x7effc40b89e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7effc80094e0 tx=0x7effc80093f0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.216+0000 7effdd006640 1 -- 172.21.15.3:0/3553403854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effd810ddf0 msgr2=0x7effd819fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:39.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.216+0000 7effdd006640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effd810ddf0 0x7effd819fb70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7effd000d750 tx=0x7effd000dc10 comp rx=0 tx=0).stop 2024-09-21T06:06:39.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.216+0000 7effdd006640 1 -- 172.21.15.3:0/3553403854 shutdown_connections 2024-09-21T06:06:39.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.216+0000 7effdd006640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7effc40b64d0 0x7effc40b89e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.216+0000 7effdd006640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effd8105270 0x7effd819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.216+0000 7effdd006640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effd810ddf0 0x7effd819fb70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.216+0000 7effdd006640 1 --2- 172.21.15.3:0/3553403854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effd8104840 0x7effd819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:39.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.216+0000 7effdd006640 1 -- 172.21.15.3:0/3553403854 >> 172.21.15.3:0/3553403854 conn(0x7effd8079730 msgr2=0x7effd80ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:39.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.216+0000 7effdd006640 1 -- 172.21.15.3:0/3553403854 shutdown_connections 2024-09-21T06:06:39.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:39.216+0000 7effdd006640 1 -- 172.21.15.3:0/3553403854 wait complete. 2024-09-21T06:06:39.762 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:39 smithi003 ceph-mon[31227]: mgrmap e20: x(active, since 14s), standbys: z, y 2024-09-21T06:06:39.762 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:39 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T06:06:39.762 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:39 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3553403854' entity='client.admin' 2024-09-21T06:06:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:39 smithi137 ceph-mon[34464]: mgrmap e20: x(active, since 14s), standbys: z, y 2024-09-21T06:06:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:39 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T06:06:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:39 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3553403854' entity='client.admin' 2024-09-21T06:06:39.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:39 smithi204 ceph-mon[34659]: mgrmap e20: x(active, since 14s), standbys: z, y 2024-09-21T06:06:39.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:39 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T06:06:39.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:39 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3553403854' entity='client.admin' 2024-09-21T06:06:40.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.967+0000 7f3097700640 1 -- 172.21.15.3:0/3996402076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3090102ae0 msgr2=0x7f3090102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.967+0000 7f3097700640 1 --2- 172.21.15.3:0/3996402076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3090102ae0 0x7f3090102ef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3084009900 tx=0x7f308402a410 comp rx=0 tx=0).stop 2024-09-21T06:06:40.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.967+0000 7f3097700640 1 -- 172.21.15.3:0/3996402076 shutdown_connections 2024-09-21T06:06:40.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.967+0000 7f3097700640 1 --2- 172.21.15.3:0/3996402076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30901083d0 0x7f30901087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.967+0000 7f3097700640 1 --2- 172.21.15.3:0/3996402076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090108d20 0x7f309010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.967+0000 7f3097700640 1 --2- 172.21.15.3:0/3996402076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3090102ae0 0x7f3090102ef0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.967+0000 7f3097700640 1 -- 172.21.15.3:0/3996402076 >> 172.21.15.3:0/3996402076 conn(0x7f30900797b0 msgr2=0x7f3090079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:40.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.967+0000 7f3097700640 1 -- 172.21.15.3:0/3996402076 shutdown_connections 2024-09-21T06:06:40.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.967+0000 7f3097700640 1 -- 172.21.15.3:0/3996402076 wait complete. 2024-09-21T06:06:40.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.968+0000 7f3097700640 1 Processor -- start 2024-09-21T06:06:40.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.968+0000 7f3097700640 1 -- start start 2024-09-21T06:06:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.968+0000 7f3097700640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3090102ae0 0x7f309019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.968+0000 7f3097700640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30901083d0 0x7f309019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.968+0000 7f3097700640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090108d20 0x7f309019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.968+0000 7f3097700640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f30901127a0 con 0x7f3090102ae0 2024-09-21T06:06:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.968+0000 7f3097700640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3090112620 con 0x7f3090108d20 2024-09-21T06:06:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.968+0000 7f3097700640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3090112920 con 0x7f30901083d0 2024-09-21T06:06:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.969+0000 7f309549c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3090102ae0 0x7f309019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.969+0000 7f309549c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3090102ae0 0x7f309019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36176/0 (socket says 172.21.15.3:36176) 2024-09-21T06:06:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.969+0000 7f309549c640 1 -- 172.21.15.3:0/3700646329 learned_addr learned my addr 172.21.15.3:0/3700646329 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.969+0000 7f3095c9d640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090108d20 0x7f309019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.969+0000 7f3094c9b640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30901083d0 0x7f309019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.969+0000 7f309549c640 1 -- 172.21.15.3:0/3700646329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30901083d0 msgr2=0x7f309019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.969+0000 7f309549c640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30901083d0 0x7f309019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.969+0000 7f309549c640 1 -- 172.21.15.3:0/3700646329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090108d20 msgr2=0x7f309019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.969+0000 7f309549c640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090108d20 0x7f309019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.969+0000 7f309549c640 1 -- 172.21.15.3:0/3700646329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30901a03d0 con 0x7f3090102ae0 2024-09-21T06:06:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.969+0000 7f309549c640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3090102ae0 0x7f309019a460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3084005940 tx=0x7f3084003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.969+0000 7f30827fc640 1 -- 172.21.15.3:0/3700646329 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308403f070 con 0x7f3090102ae0 2024-09-21T06:06:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.969+0000 7f30827fc640 1 -- 172.21.15.3:0/3700646329 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f3084031d20 con 0x7f3090102ae0 2024-09-21T06:06:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.970+0000 7f3097700640 1 -- 172.21.15.3:0/3700646329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30901a0660 con 0x7f3090102ae0 2024-09-21T06:06:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.970+0000 7f30827fc640 1 -- 172.21.15.3:0/3700646329 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084035030 con 0x7f3090102ae0 2024-09-21T06:06:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.970+0000 7f3097700640 1 -- 172.21.15.3:0/3700646329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30901a6e20 con 0x7f3090102ae0 2024-09-21T06:06:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.971+0000 7f3097700640 1 -- 172.21.15.3:0/3700646329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3090102ef0 con 0x7f3090102ae0 2024-09-21T06:06:40.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.973+0000 7f30827fc640 1 -- 172.21.15.3:0/3700646329 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f3084042070 con 0x7f3090102ae0 2024-09-21T06:06:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.974+0000 7f30827fc640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f307c0b6550 0x7f307c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.974+0000 7f3094c9b640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f307c0b6550 0x7f307c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.974+0000 7f30827fc640 1 -- 172.21.15.3:0/3700646329 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f30841064c0 con 0x7f3090102ae0 2024-09-21T06:06:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.975+0000 7f3094c9b640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f307c0b6550 0x7f307c0b8a60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3090079d50 tx=0x7f308c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:40.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:40.977+0000 7f30827fc640 1 -- 172.21.15.3:0/3700646329 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30840d2d30 con 0x7f3090102ae0 2024-09-21T06:06:41.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.178+0000 7f3097700640 1 -- 172.21.15.3:0/3700646329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f309019b590 con 0x7f3090102ae0 2024-09-21T06:06:41.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.182+0000 7f30827fc640 1 -- 172.21.15.3:0/3700646329 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v24)=0 v24) ==== 106+0+0 (secure 0 0 0) 0x7f309019b590 con 0x7f3090102ae0 2024-09-21T06:06:41.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.184+0000 7f3097700640 1 -- 172.21.15.3:0/3700646329 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f307c0b6550 msgr2=0x7f307c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:41.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.184+0000 7f3097700640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f307c0b6550 0x7f307c0b8a60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3090079d50 tx=0x7f308c00b390 comp rx=0 tx=0).stop 2024-09-21T06:06:41.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.184+0000 7f3097700640 1 -- 172.21.15.3:0/3700646329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3090102ae0 msgr2=0x7f309019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:41.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.184+0000 7f3097700640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3090102ae0 0x7f309019a460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3084005940 tx=0x7f3084003d50 comp rx=0 tx=0).stop 2024-09-21T06:06:41.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.184+0000 7f3097700640 1 -- 172.21.15.3:0/3700646329 shutdown_connections 2024-09-21T06:06:41.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.184+0000 7f3097700640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f307c0b6550 0x7f307c0b8a60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:41.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.184+0000 7f3097700640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30901083d0 0x7f309019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:41.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.184+0000 7f3097700640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090108d20 0x7f309019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:41.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.184+0000 7f3097700640 1 --2- 172.21.15.3:0/3700646329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3090102ae0 0x7f309019a460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:41.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.184+0000 7f3097700640 1 -- 172.21.15.3:0/3700646329 >> 172.21.15.3:0/3700646329 conn(0x7f30900797b0 msgr2=0x7f3090101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:41.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.184+0000 7f3097700640 1 -- 172.21.15.3:0/3700646329 shutdown_connections 2024-09-21T06:06:41.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:41.184+0000 7f3097700640 1 -- 172.21.15.3:0/3700646329 wait complete. 2024-09-21T06:06:42.437 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:42 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3700646329' entity='client.admin' 2024-09-21T06:06:42.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:42 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3700646329' entity='client.admin' 2024-09-21T06:06:42.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:42 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3700646329' entity='client.admin' 2024-09-21T06:06:43.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.275+0000 7f284a182640 1 -- 172.21.15.3:0/3581856215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f284410de10 msgr2=0x7f2844110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.275+0000 7f284a182640 1 --2- 172.21.15.3:0/3581856215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f284410de10 0x7f2844110300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f283c009ca0 tx=0x7f283c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.275+0000 7f284a182640 1 -- 172.21.15.3:0/3581856215 shutdown_connections 2024-09-21T06:06:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.275+0000 7f284a182640 1 --2- 172.21.15.3:0/3581856215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2844105300 0x7f284410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.275+0000 7f284a182640 1 --2- 172.21.15.3:0/3581856215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f284410de10 0x7f2844110300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.275+0000 7f284a182640 1 --2- 172.21.15.3:0/3581856215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28441049b0 0x7f2844104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.275+0000 7f284a182640 1 -- 172.21.15.3:0/3581856215 >> 172.21.15.3:0/3581856215 conn(0x7f2844079730 msgr2=0x7f2844079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.275+0000 7f284a182640 1 -- 172.21.15.3:0/3581856215 shutdown_connections 2024-09-21T06:06:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.275+0000 7f284a182640 1 -- 172.21.15.3:0/3581856215 wait complete. 2024-09-21T06:06:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.275+0000 7f284a182640 1 Processor -- start 2024-09-21T06:06:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f284a182640 1 -- start start 2024-09-21T06:06:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f284a182640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28441049b0 0x7f28441026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f284a182640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844105300 0x7f2844102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f284a182640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f284410de10 0x7f2844103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f284a182640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f28441137e0 con 0x7f2844105300 2024-09-21T06:06:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f284a182640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2844113660 con 0x7f284410de10 2024-09-21T06:06:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f284a182640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2844113960 con 0x7f28441049b0 2024-09-21T06:06:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f2842ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844105300 0x7f2844102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f2842ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844105300 0x7f2844102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36246/0 (socket says 172.21.15.3:36246) 2024-09-21T06:06:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f2842ffd640 1 -- 172.21.15.3:0/2366784587 learned_addr learned my addr 172.21.15.3:0/2366784587 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f28437fe640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28441049b0 0x7f28441026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f2843fff640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f284410de10 0x7f2844103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f2842ffd640 1 -- 172.21.15.3:0/2366784587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28441049b0 msgr2=0x7f28441026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f2842ffd640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28441049b0 0x7f28441026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f2842ffd640 1 -- 172.21.15.3:0/2366784587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f284410de10 msgr2=0x7f2844103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f2842ffd640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f284410de10 0x7f2844103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f2842ffd640 1 -- 172.21.15.3:0/2366784587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28441a8570 con 0x7f2844105300 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.276+0000 7f2843fff640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f284410de10 0x7f2844103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.277+0000 7f2842ffd640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844105300 0x7f2844102c30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f282c00e9f0 tx=0x7f282c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.277+0000 7f2840ff9640 1 -- 172.21.15.3:0/2366784587 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c00ccc0 con 0x7f2844105300 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.277+0000 7f2840ff9640 1 -- 172.21.15.3:0/2366784587 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f282c0108a0 con 0x7f2844105300 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.277+0000 7f284a182640 1 -- 172.21.15.3:0/2366784587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28441a8860 con 0x7f2844105300 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.277+0000 7f2840ff9640 1 -- 172.21.15.3:0/2366784587 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c018760 con 0x7f2844105300 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.277+0000 7f284a182640 1 -- 172.21.15.3:0/2366784587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28441a8df0 con 0x7f2844105300 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.278+0000 7f284a182640 1 -- 172.21.15.3:0/2366784587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2844104dc0 con 0x7f2844105300 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.280+0000 7f2840ff9640 1 -- 172.21.15.3:0/2366784587 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f282c010430 con 0x7f2844105300 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.281+0000 7f2840ff9640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f28300b6550 0x7f28300b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.281+0000 7f28437fe640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f28300b6550 0x7f28300b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.281+0000 7f2840ff9640 1 -- 172.21.15.3:0/2366784587 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f282c0e5ec0 con 0x7f2844105300 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.282+0000 7f28437fe640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f28300b6550 0x7f28300b8a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2844104140 tx=0x7f2834005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.284+0000 7f2840ff9640 1 -- 172.21.15.3:0/2366784587 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f282c0ae350 con 0x7f2844105300 2024-09-21T06:06:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.481+0000 7f284a182640 1 -- 172.21.15.3:0/2366784587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f28441039f0 con 0x7f2844105300 2024-09-21T06:06:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.486+0000 7f2840ff9640 1 -- 172.21.15.3:0/2366784587 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v28)=0 v28) ==== 127+0+0 (secure 0 0 0) 0x7f28441039f0 con 0x7f2844105300 2024-09-21T06:06:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.489+0000 7f284a182640 1 -- 172.21.15.3:0/2366784587 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f28300b6550 msgr2=0x7f28300b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.489+0000 7f284a182640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f28300b6550 0x7f28300b8a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2844104140 tx=0x7f2834005f90 comp rx=0 tx=0).stop 2024-09-21T06:06:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.489+0000 7f284a182640 1 -- 172.21.15.3:0/2366784587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844105300 msgr2=0x7f2844102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.489+0000 7f284a182640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844105300 0x7f2844102c30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f282c00e9f0 tx=0x7f282c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:06:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.489+0000 7f284a182640 1 -- 172.21.15.3:0/2366784587 shutdown_connections 2024-09-21T06:06:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.489+0000 7f284a182640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f28300b6550 0x7f28300b8a60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.489+0000 7f284a182640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f284410de10 0x7f2844103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.489+0000 7f284a182640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844105300 0x7f2844102c30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.489+0000 7f284a182640 1 --2- 172.21.15.3:0/2366784587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28441049b0 0x7f28441026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.489+0000 7f284a182640 1 -- 172.21.15.3:0/2366784587 >> 172.21.15.3:0/2366784587 conn(0x7f2844079730 msgr2=0x7f2844100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.489+0000 7f284a182640 1 -- 172.21.15.3:0/2366784587 shutdown_connections 2024-09-21T06:06:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.489+0000 7f284a182640 1 -- 172.21.15.3:0/2366784587 wait complete. 2024-09-21T06:06:44.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.388+0000 7f84dc758640 1 -- 172.21.15.3:0/730543741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84d41094d0 msgr2=0x7f84d41098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.388+0000 7f84dc758640 1 --2- 172.21.15.3:0/730543741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84d41094d0 0x7f84d41098e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009900 tx=0x7f84d002a410 comp rx=0 tx=0).stop 2024-09-21T06:06:44.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.388+0000 7f84dc758640 1 -- 172.21.15.3:0/730543741 shutdown_connections 2024-09-21T06:06:44.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.388+0000 7f84dc758640 1 --2- 172.21.15.3:0/730543741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4109e20 0x7f84d4110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.388+0000 7f84dc758640 1 --2- 172.21.15.3:0/730543741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84d41094d0 0x7f84d41098e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.388+0000 7f84dc758640 1 --2- 172.21.15.3:0/730543741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d4103b90 0x7f84d4103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.388+0000 7f84dc758640 1 -- 172.21.15.3:0/730543741 >> 172.21.15.3:0/730543741 conn(0x7f84d40ff600 msgr2=0x7f84d4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.388+0000 7f84dc758640 1 -- 172.21.15.3:0/730543741 shutdown_connections 2024-09-21T06:06:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.388+0000 7f84dc758640 1 -- 172.21.15.3:0/730543741 wait complete. 2024-09-21T06:06:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.389+0000 7f84dc758640 1 Processor -- start 2024-09-21T06:06:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.389+0000 7f84dc758640 1 -- start start 2024-09-21T06:06:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.389+0000 7f84dc758640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84d4103b90 0x7f84d419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.389+0000 7f84dc758640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d41094d0 0x7f84d419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.389+0000 7f84dc758640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4109e20 0x7f84d419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.389+0000 7f84dc758640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f84d41138a0 con 0x7f84d4103b90 2024-09-21T06:06:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.389+0000 7f84dc758640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f84d4113720 con 0x7f84d41094d0 2024-09-21T06:06:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.389+0000 7f84da4f4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84d4103b90 0x7f84d419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.389+0000 7f84dc758640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f84d4113a20 con 0x7f84d4109e20 2024-09-21T06:06:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.389+0000 7f84da4f4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84d4103b90 0x7f84d419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36262/0 (socket says 172.21.15.3:36262) 2024-09-21T06:06:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.389+0000 7f84da4f4640 1 -- 172.21.15.3:0/4285136130 learned_addr learned my addr 172.21.15.3:0/4285136130 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:44.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.389+0000 7f84dacf5640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4109e20 0x7f84d419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.390+0000 7f84d9cf3640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d41094d0 0x7f84d419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.390+0000 7f84da4f4640 1 -- 172.21.15.3:0/4285136130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4109e20 msgr2=0x7f84d419fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.390+0000 7f84da4f4640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4109e20 0x7f84d419fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.390+0000 7f84da4f4640 1 -- 172.21.15.3:0/4285136130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d41094d0 msgr2=0x7f84d419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.390+0000 7f84da4f4640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d41094d0 0x7f84d419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.390+0000 7f84da4f4640 1 -- 172.21.15.3:0/4285136130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d41a0260 con 0x7f84d4103b90 2024-09-21T06:06:44.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.390+0000 7f84da4f4640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84d4103b90 0x7f84d419a2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f84c800d950 tx=0x7f84c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:44.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.390+0000 7f84c77fe640 1 -- 172.21.15.3:0/4285136130 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8017070 con 0x7f84d4103b90 2024-09-21T06:06:44.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.390+0000 7f84c77fe640 1 -- 172.21.15.3:0/4285136130 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f84c8009c30 con 0x7f84d4103b90 2024-09-21T06:06:44.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.390+0000 7f84dc758640 1 -- 172.21.15.3:0/4285136130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d41a0550 con 0x7f84d4103b90 2024-09-21T06:06:44.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.390+0000 7f84c77fe640 1 -- 172.21.15.3:0/4285136130 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c800f620 con 0x7f84d4103b90 2024-09-21T06:06:44.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.390+0000 7f84dc758640 1 -- 172.21.15.3:0/4285136130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d41a6cb0 con 0x7f84d4103b90 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.391+0000 7f84dc758640 1 -- 172.21.15.3:0/4285136130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d4103fa0 con 0x7f84d4103b90 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.393+0000 7f84c77fe640 1 -- 172.21.15.3:0/4285136130 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f84c8010460 con 0x7f84d4103b90 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.394+0000 7f84c77fe640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f84c00b6550 0x7f84c00b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.394+0000 7f84d9cf3640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f84c00b6550 0x7f84c00b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.394+0000 7f84c77fe640 1 -- 172.21.15.3:0/4285136130 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f84c80e5300 con 0x7f84d4103b90 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.395+0000 7f84d9cf3640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f84c00b6550 0x7f84c00b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009230 tx=0x7f84d0031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.398+0000 7f84c77fe640 1 -- 172.21.15.3:0/4285136130 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f84c80ad790 con 0x7f84d4103b90 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.599+0000 7f84dc758640 1 -- 172.21.15.3:0/4285136130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f84d410bba0 con 0x7f84d4103b90 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.599+0000 7f84c77fe640 1 -- 172.21.15.3:0/4285136130 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v28)=0 v28) ==== 108+0+0 (secure 0 0 0) 0x7f84c8010e20 con 0x7f84d4103b90 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.601+0000 7f84dc758640 1 -- 172.21.15.3:0/4285136130 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f84c00b6550 msgr2=0x7f84c00b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.601+0000 7f84dc758640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f84c00b6550 0x7f84c00b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009230 tx=0x7f84d0031400 comp rx=0 tx=0).stop 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.601+0000 7f84dc758640 1 -- 172.21.15.3:0/4285136130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84d4103b90 msgr2=0x7f84d419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.601+0000 7f84dc758640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84d4103b90 0x7f84d419a2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f84c800d950 tx=0x7f84c800de10 comp rx=0 tx=0).stop 2024-09-21T06:06:44.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.601+0000 7f84dc758640 1 -- 172.21.15.3:0/4285136130 shutdown_connections 2024-09-21T06:06:44.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.601+0000 7f84dc758640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f84c00b6550 0x7f84c00b8a60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.601+0000 7f84dc758640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4109e20 0x7f84d419fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.601+0000 7f84dc758640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d41094d0 0x7f84d419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.601+0000 7f84dc758640 1 --2- 172.21.15.3:0/4285136130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84d4103b90 0x7f84d419a2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.601+0000 7f84dc758640 1 -- 172.21.15.3:0/4285136130 >> 172.21.15.3:0/4285136130 conn(0x7f84d40ff600 msgr2=0x7f84d4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:44.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.602+0000 7f84dc758640 1 -- 172.21.15.3:0/4285136130 shutdown_connections 2024-09-21T06:06:44.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.602+0000 7f84dc758640 1 -- 172.21.15.3:0/4285136130 wait complete. 2024-09-21T06:06:44.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:43 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1593098017' entity='client.admin' 2024-09-21T06:06:44.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:43 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3634885047' entity='client.admin' 2024-09-21T06:06:44.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:43 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1096098911' entity='client.admin' 2024-09-21T06:06:44.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:43 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2366784587' entity='client.admin' 2024-09-21T06:06:44.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:43 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1593098017' entity='client.admin' 2024-09-21T06:06:44.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:43 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3634885047' entity='client.admin' 2024-09-21T06:06:44.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:43 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1096098911' entity='client.admin' 2024-09-21T06:06:44.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:43 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2366784587' entity='client.admin' 2024-09-21T06:06:44.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:43 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1593098017' entity='client.admin' 2024-09-21T06:06:44.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:43 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3634885047' entity='client.admin' 2024-09-21T06:06:44.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:43 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1096098911' entity='client.admin' 2024-09-21T06:06:44.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:43 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2366784587' entity='client.admin' 2024-09-21T06:06:45.288 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:44 smithi003 ceph-mon[31227]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:45.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:44 smithi137 ceph-mon[34464]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.136+0000 7f2abec8f640 1 -- 172.21.15.3:0/2400070098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8102080 msgr2=0x7f2ab8102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.136+0000 7f2abec8f640 1 --2- 172.21.15.3:0/2400070098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8102080 0x7f2ab8102490 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009900 tx=0x7f2aac02a410 comp rx=0 tx=0).stop 2024-09-21T06:06:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.136+0000 7f2abec8f640 1 -- 172.21.15.3:0/2400070098 shutdown_connections 2024-09-21T06:06:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.136+0000 7f2abec8f640 1 --2- 172.21.15.3:0/2400070098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab8109eb0 0x7f2ab81106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.136+0000 7f2abec8f640 1 --2- 172.21.15.3:0/2400070098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ab8109560 0x7f2ab8109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.136+0000 7f2abec8f640 1 --2- 172.21.15.3:0/2400070098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8102080 0x7f2ab8102490 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.136+0000 7f2abec8f640 1 -- 172.21.15.3:0/2400070098 >> 172.21.15.3:0/2400070098 conn(0x7f2ab8079ec0 msgr2=0x7f2ab807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.137+0000 7f2abec8f640 1 -- 172.21.15.3:0/2400070098 shutdown_connections 2024-09-21T06:06:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.137+0000 7f2abec8f640 1 -- 172.21.15.3:0/2400070098 wait complete. 2024-09-21T06:06:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.137+0000 7f2abec8f640 1 Processor -- start 2024-09-21T06:06:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.137+0000 7f2abec8f640 1 -- start start 2024-09-21T06:06:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.137+0000 7f2abec8f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8102080 0x7f2ab819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.137+0000 7f2abec8f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ab8109560 0x7f2ab819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abec8f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab8109eb0 0x7f2ab819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abec8f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8113930 con 0x7f2ab8102080 2024-09-21T06:06:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abec8f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81137b0 con 0x7f2ab8109560 2024-09-21T06:06:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abec8f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8113ab0 con 0x7f2ab8109eb0 2024-09-21T06:06:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abca2b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8102080 0x7f2ab819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abca2b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8102080 0x7f2ab819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36228/0 (socket says 172.21.15.3:36228) 2024-09-21T06:06:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abca2b640 1 -- 172.21.15.3:0/1096098911 learned_addr learned my addr 172.21.15.3:0/1096098911 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:45.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2ab7fff640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ab8109560 0x7f2ab819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abd22c640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab8109eb0 0x7f2ab819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abca2b640 1 -- 172.21.15.3:0/1096098911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab8109eb0 msgr2=0x7f2ab819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abca2b640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab8109eb0 0x7f2ab819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abca2b640 1 -- 172.21.15.3:0/1096098911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ab8109560 msgr2=0x7f2ab819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abca2b640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ab8109560 0x7f2ab819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abca2b640 1 -- 172.21.15.3:0/1096098911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab81a0310 con 0x7f2ab8102080 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abca2b640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8102080 0x7f2ab819a3a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2aac005940 tx=0x7f2aac003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/1096098911 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac03f070 con 0x7f2ab8102080 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/1096098911 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f2aac031d20 con 0x7f2ab8102080 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2abec8f640 1 -- 172.21.15.3:0/1096098911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab81a05a0 con 0x7f2ab8102080 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.138+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/1096098911 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac035030 con 0x7f2ab8102080 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.139+0000 7f2abec8f640 1 -- 172.21.15.3:0/1096098911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab81a6d60 con 0x7f2ab8102080 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.140+0000 7f2abec8f640 1 -- 172.21.15.3:0/1096098911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab8102490 con 0x7f2ab8102080 2024-09-21T06:06:45.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.141+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/1096098911 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f2aac042070 con 0x7f2ab8102080 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.142+0000 7f2ab5ffb640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2aa80b65d0 0x7f2aa80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.142+0000 7f2ab7fff640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2aa80b65d0 0x7f2aa80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.142+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/1096098911 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f2aac106560 con 0x7f2ab8102080 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.143+0000 7f2ab7fff640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2aa80b65d0 0x7f2aa80b8ae0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2ab807a760 tx=0x7f2aa400b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.145+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/1096098911 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2aac0d2d30 con 0x7f2ab8102080 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.339+0000 7f2abec8f640 1 -- 172.21.15.3:0/1096098911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f2ab810bc30 con 0x7f2ab8102080 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.346+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/1096098911 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v27)=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7f2aac0d7c70 con 0x7f2ab8102080 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.349+0000 7f2abec8f640 1 -- 172.21.15.3:0/1096098911 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2aa80b65d0 msgr2=0x7f2aa80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.349+0000 7f2abec8f640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2aa80b65d0 0x7f2aa80b8ae0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2ab807a760 tx=0x7f2aa400b410 comp rx=0 tx=0).stop 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.349+0000 7f2abec8f640 1 -- 172.21.15.3:0/1096098911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8102080 msgr2=0x7f2ab819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.349+0000 7f2abec8f640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8102080 0x7f2ab819a3a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2aac005940 tx=0x7f2aac003d50 comp rx=0 tx=0).stop 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.349+0000 7f2abec8f640 1 -- 172.21.15.3:0/1096098911 shutdown_connections 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.349+0000 7f2abec8f640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2aa80b65d0 0x7f2aa80b8ae0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.349+0000 7f2abec8f640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab8109eb0 0x7f2ab819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.349+0000 7f2abec8f640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ab8109560 0x7f2ab819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.349+0000 7f2abec8f640 1 --2- 172.21.15.3:0/1096098911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8102080 0x7f2ab819a3a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.349+0000 7f2abec8f640 1 -- 172.21.15.3:0/1096098911 >> 172.21.15.3:0/1096098911 conn(0x7f2ab8079ec0 msgr2=0x7f2ab8107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:45.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.349+0000 7f2abec8f640 1 -- 172.21.15.3:0/1096098911 shutdown_connections 2024-09-21T06:06:45.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.349+0000 7f2abec8f640 1 -- 172.21.15.3:0/1096098911 wait complete. 2024-09-21T06:06:45.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:44 smithi204 ceph-mon[34659]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:45.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.191+0000 7fafe1224640 1 -- 172.21.15.3:0/2455776093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafdc10de10 msgr2=0x7fafdc110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.191+0000 7fafe1224640 1 --2- 172.21.15.3:0/2455776093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafdc10de10 0x7fafdc110300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009ca0 tx=0x7fafd402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.191+0000 7fafe1224640 1 -- 172.21.15.3:0/2455776093 shutdown_connections 2024-09-21T06:06:45.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.191+0000 7fafe1224640 1 --2- 172.21.15.3:0/2455776093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafdc105300 0x7fafdc10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.191+0000 7fafe1224640 1 --2- 172.21.15.3:0/2455776093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafdc10de10 0x7fafdc110300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.191+0000 7fafe1224640 1 --2- 172.21.15.3:0/2455776093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafdc1049b0 0x7fafdc104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.191+0000 7fafe1224640 1 -- 172.21.15.3:0/2455776093 >> 172.21.15.3:0/2455776093 conn(0x7fafdc079730 msgr2=0x7fafdc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:45.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.191+0000 7fafe1224640 1 -- 172.21.15.3:0/2455776093 shutdown_connections 2024-09-21T06:06:45.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.191+0000 7fafe1224640 1 -- 172.21.15.3:0/2455776093 wait complete. 2024-09-21T06:06:45.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.192+0000 7fafe1224640 1 Processor -- start 2024-09-21T06:06:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.192+0000 7fafe1224640 1 -- start start 2024-09-21T06:06:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.192+0000 7fafe1224640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafdc1049b0 0x7fafdc19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.192+0000 7fafe1224640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafdc105300 0x7fafdc19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.192+0000 7fafe1224640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafdc10de10 0x7fafdc19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.192+0000 7fafe1224640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1137e0 con 0x7fafdc105300 2024-09-21T06:06:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.192+0000 7fafe1224640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc113660 con 0x7fafdc1049b0 2024-09-21T06:06:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.192+0000 7fafe1224640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc113960 con 0x7fafdc10de10 2024-09-21T06:06:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.192+0000 7fafda59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafdc105300 0x7fafdc19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.192+0000 7fafda59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafdc105300 0x7fafdc19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36280/0 (socket says 172.21.15.3:36280) 2024-09-21T06:06:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.192+0000 7fafda59c640 1 -- 172.21.15.3:0/1553294083 learned_addr learned my addr 172.21.15.3:0/1553294083 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafdad9d640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafdc1049b0 0x7fafdc19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafdb59e640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafdc10de10 0x7fafdc19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafda59c640 1 -- 172.21.15.3:0/1553294083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafdc10de10 msgr2=0x7fafdc19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafda59c640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafdc10de10 0x7fafdc19faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafda59c640 1 -- 172.21.15.3:0/1553294083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafdc1049b0 msgr2=0x7fafdc19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafda59c640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafdc1049b0 0x7fafdc19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafda59c640 1 -- 172.21.15.3:0/1553294083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc1a0160 con 0x7fafdc105300 2024-09-21T06:06:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafdb59e640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafdc10de10 0x7fafdc19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafda59c640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafdc105300 0x7fafdc19a850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fafd000d740 tx=0x7fafd000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafc3fff640 1 -- 172.21.15.3:0/1553294083 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0017070 con 0x7fafdc105300 2024-09-21T06:06:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafe1224640 1 -- 172.21.15.3:0/1553294083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc1a0450 con 0x7fafdc105300 2024-09-21T06:06:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafc3fff640 1 -- 172.21.15.3:0/1553294083 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fafd000de50 con 0x7fafdc105300 2024-09-21T06:06:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafc3fff640 1 -- 172.21.15.3:0/1553294083 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd000f620 con 0x7fafdc105300 2024-09-21T06:06:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.193+0000 7fafe1224640 1 -- 172.21.15.3:0/1553294083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc1a6c70 con 0x7fafdc105300 2024-09-21T06:06:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.194+0000 7fafe1224640 1 -- 172.21.15.3:0/1553294083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafdc104dc0 con 0x7fafdc105300 2024-09-21T06:06:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.196+0000 7fafc3fff640 1 -- 172.21.15.3:0/1553294083 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7fafd001a070 con 0x7fafdc105300 2024-09-21T06:06:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.197+0000 7fafc3fff640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fafc40b6500 0x7fafc40b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.197+0000 7fafc3fff640 1 -- 172.21.15.3:0/1553294083 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fafd00e51a0 con 0x7fafdc105300 2024-09-21T06:06:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.197+0000 7fafdad9d640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fafc40b6500 0x7fafc40b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.197+0000 7fafdad9d640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fafc40b6500 0x7fafc40b8a10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0058c0 tx=0x7fafcc005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.204+0000 7fafc3fff640 1 -- 172.21.15.3:0/1553294083 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fafd00ad630 con 0x7fafdc105300 2024-09-21T06:06:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.402+0000 7fafe1224640 1 -- 172.21.15.3:0/1553294083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fafdc108f10 con 0x7fafdc105300 2024-09-21T06:06:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.407+0000 7fafc3fff640 1 -- 172.21.15.3:0/1553294083 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v29)=0 v29) ==== 111+0+0 (secure 0 0 0) 0x7fafd00b1a10 con 0x7fafdc105300 2024-09-21T06:06:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.410+0000 7fafe1224640 1 -- 172.21.15.3:0/1553294083 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fafc40b6500 msgr2=0x7fafc40b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.410+0000 7fafe1224640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fafc40b6500 0x7fafc40b8a10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0058c0 tx=0x7fafcc005850 comp rx=0 tx=0).stop 2024-09-21T06:06:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.410+0000 7fafe1224640 1 -- 172.21.15.3:0/1553294083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafdc105300 msgr2=0x7fafdc19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.410+0000 7fafe1224640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafdc105300 0x7fafdc19a850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fafd000d740 tx=0x7fafd000dc00 comp rx=0 tx=0).stop 2024-09-21T06:06:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.410+0000 7fafe1224640 1 -- 172.21.15.3:0/1553294083 shutdown_connections 2024-09-21T06:06:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.410+0000 7fafe1224640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fafc40b6500 0x7fafc40b8a10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.410+0000 7fafe1224640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafdc10de10 0x7fafdc19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.410+0000 7fafe1224640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafdc105300 0x7fafdc19a850 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.410+0000 7fafe1224640 1 --2- 172.21.15.3:0/1553294083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafdc1049b0 0x7fafdc19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.410+0000 7fafe1224640 1 -- 172.21.15.3:0/1553294083 >> 172.21.15.3:0/1553294083 conn(0x7fafdc079730 msgr2=0x7fafdc100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.410+0000 7fafe1224640 1 -- 172.21.15.3:0/1553294083 shutdown_connections 2024-09-21T06:06:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:45.410+0000 7fafe1224640 1 -- 172.21.15.3:0/1553294083 wait complete. 2024-09-21T06:06:46.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.746+0000 7fded4f17640 1 -- 172.21.15.3:0/1582035432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fded0108d10 msgr2=0x7fded010f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:46.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.746+0000 7fded4f17640 1 --2- 172.21.15.3:0/1582035432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fded0108d10 0x7fded010f540 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009cc0 tx=0x7fdec802cad0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.746+0000 7fded4f17640 1 -- 172.21.15.3:0/1582035432 shutdown_connections 2024-09-21T06:06:46.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.746+0000 7fded4f17640 1 --2- 172.21.15.3:0/1582035432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fded0108d10 0x7fded010f540 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.746+0000 7fded4f17640 1 --2- 172.21.15.3:0/1582035432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fded01083c0 0x7fded01087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.746+0000 7fded4f17640 1 --2- 172.21.15.3:0/1582035432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded0102a80 0x7fded0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.746+0000 7fded4f17640 1 -- 172.21.15.3:0/1582035432 >> 172.21.15.3:0/1582035432 conn(0x7fded0079740 msgr2=0x7fded0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:46.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.746+0000 7fded4f17640 1 -- 172.21.15.3:0/1582035432 shutdown_connections 2024-09-21T06:06:46.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.746+0000 7fded4f17640 1 -- 172.21.15.3:0/1582035432 wait complete. 2024-09-21T06:06:46.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.747+0000 7fded4f17640 1 Processor -- start 2024-09-21T06:06:46.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.747+0000 7fded4f17640 1 -- start start 2024-09-21T06:06:46.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fded4f17640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded0102a80 0x7fded01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:46.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fded4f17640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fded01083c0 0x7fded0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:46.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fded4f17640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fded0108d10 0x7fded019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:46.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fded4f17640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fded0111640 con 0x7fded01083c0 2024-09-21T06:06:46.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fded4f17640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fded01114c0 con 0x7fded0108d10 2024-09-21T06:06:46.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fded4f17640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fded01117c0 con 0x7fded0102a80 2024-09-21T06:06:46.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fdecdd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fded01083c0 0x7fded0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:46.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fdecdd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fded01083c0 0x7fded0198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36188/0 (socket says 172.21.15.3:36188) 2024-09-21T06:06:46.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fdecdd9b640 1 -- 172.21.15.3:0/1593098017 learned_addr learned my addr 172.21.15.3:0/1593098017 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:46.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fdece59c640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded0102a80 0x7fded01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:46.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fdeced9d640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fded0108d10 0x7fded019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:46.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fdecdd9b640 1 -- 172.21.15.3:0/1593098017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded0102a80 msgr2=0x7fded01980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:46.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fdecdd9b640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded0102a80 0x7fded01980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fdecdd9b640 1 -- 172.21.15.3:0/1593098017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fded0108d10 msgr2=0x7fded019d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:46.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fdecdd9b640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fded0108d10 0x7fded019d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fdecdd9b640 1 -- 172.21.15.3:0/1593098017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded019e040 con 0x7fded01083c0 2024-09-21T06:06:46.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fdecdd9b640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fded01083c0 0x7fded0198670 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdec0005fd0 tx=0x7fdec000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:46.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.748+0000 7fdec77fe640 1 -- 172.21.15.3:0/1593098017 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0017070 con 0x7fded01083c0 2024-09-21T06:06:46.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.749+0000 7fdec77fe640 1 -- 172.21.15.3:0/1593098017 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fdec0009e20 con 0x7fded01083c0 2024-09-21T06:06:46.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.749+0000 7fded4f17640 1 -- 172.21.15.3:0/1593098017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded019e330 con 0x7fded01083c0 2024-09-21T06:06:46.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.749+0000 7fdec77fe640 1 -- 172.21.15.3:0/1593098017 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0003bb0 con 0x7fded01083c0 2024-09-21T06:06:46.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.749+0000 7fded4f17640 1 -- 172.21.15.3:0/1593098017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded01a4a90 con 0x7fded01083c0 2024-09-21T06:06:46.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.750+0000 7fded4f17640 1 -- 172.21.15.3:0/1593098017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded0102e90 con 0x7fded01083c0 2024-09-21T06:06:46.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.752+0000 7fdec77fe640 1 -- 172.21.15.3:0/1593098017 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7fdec00099b0 con 0x7fded01083c0 2024-09-21T06:06:46.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.753+0000 7fdec77fe640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdebc0b6550 0x7fdebc0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:46.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.753+0000 7fdece59c640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdebc0b6550 0x7fdebc0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:46.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.753+0000 7fdec77fe640 1 -- 172.21.15.3:0/1593098017 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdec00e59f0 con 0x7fded01083c0 2024-09-21T06:06:46.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.753+0000 7fdece59c640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdebc0b6550 0x7fdebc0b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8005fd0 tx=0x7fdeb800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:46.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.755+0000 7fdec77fe640 1 -- 172.21.15.3:0/1593098017 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdec0020030 con 0x7fded01083c0 2024-09-21T06:06:46.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.952+0000 7fded4f17640 1 -- 172.21.15.3:0/1593098017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fded0199200 con 0x7fded01083c0 2024-09-21T06:06:46.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.956+0000 7fdec77fe640 1 -- 172.21.15.3:0/1593098017 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v25)=0 v25) ==== 129+0+0 (secure 0 0 0) 0x7fded0199200 con 0x7fded01083c0 2024-09-21T06:06:46.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.958+0000 7fded4f17640 1 -- 172.21.15.3:0/1593098017 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdebc0b6550 msgr2=0x7fdebc0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:46.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.958+0000 7fded4f17640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdebc0b6550 0x7fdebc0b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8005fd0 tx=0x7fdeb800b390 comp rx=0 tx=0).stop 2024-09-21T06:06:46.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.958+0000 7fded4f17640 1 -- 172.21.15.3:0/1593098017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fded01083c0 msgr2=0x7fded0198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:46.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.958+0000 7fded4f17640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fded01083c0 0x7fded0198670 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdec0005fd0 tx=0x7fdec000dde0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.959+0000 7fded4f17640 1 -- 172.21.15.3:0/1593098017 shutdown_connections 2024-09-21T06:06:46.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.959+0000 7fded4f17640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdebc0b6550 0x7fdebc0b8a60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.959+0000 7fded4f17640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fded0108d10 0x7fded019d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.959+0000 7fded4f17640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fded01083c0 0x7fded0198670 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.959+0000 7fded4f17640 1 --2- 172.21.15.3:0/1593098017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded0102a80 0x7fded01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.959+0000 7fded4f17640 1 -- 172.21.15.3:0/1593098017 >> 172.21.15.3:0/1593098017 conn(0x7fded0079740 msgr2=0x7fded0104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:46.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.959+0000 7fded4f17640 1 -- 172.21.15.3:0/1593098017 shutdown_connections 2024-09-21T06:06:46.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.959+0000 7fded4f17640 1 -- 172.21.15.3:0/1593098017 wait complete. 2024-09-21T06:06:46.475 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:46 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1553294083' entity='client.admin' 2024-09-21T06:06:46.475 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:46 smithi003 ceph-mon[31227]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.473+0000 7f12865c8640 1 -- 172.21.15.3:0/4107944682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1280102a60 msgr2=0x7f1280102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.473+0000 7f12865c8640 1 --2- 172.21.15.3:0/4107944682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1280102a60 0x7f1280102e70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1270009900 tx=0x7f127002a410 comp rx=0 tx=0).stop 2024-09-21T06:06:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.473+0000 7f12865c8640 1 -- 172.21.15.3:0/4107944682 shutdown_connections 2024-09-21T06:06:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.473+0000 7f12865c8640 1 --2- 172.21.15.3:0/4107944682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1280108cf0 0x7f128010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.473+0000 7f12865c8640 1 --2- 172.21.15.3:0/4107944682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12801083a0 0x7f12801087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.473+0000 7f12865c8640 1 --2- 172.21.15.3:0/4107944682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1280102a60 0x7f1280102e70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.473+0000 7f12865c8640 1 -- 172.21.15.3:0/4107944682 >> 172.21.15.3:0/4107944682 conn(0x7f1280079730 msgr2=0x7f1280079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.473+0000 7f12865c8640 1 -- 172.21.15.3:0/4107944682 shutdown_connections 2024-09-21T06:06:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.473+0000 7f12865c8640 1 -- 172.21.15.3:0/4107944682 wait complete. 2024-09-21T06:06:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.473+0000 7f12865c8640 1 Processor -- start 2024-09-21T06:06:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f12865c8640 1 -- start start 2024-09-21T06:06:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f12865c8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1280102a60 0x7f1280104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f12865c8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12801083a0 0x7f1280104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f12865c8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1280108cf0 0x7f1280104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f12865c8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1280112720 con 0x7f1280108cf0 2024-09-21T06:06:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f12865c8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12801125a0 con 0x7f12801083a0 2024-09-21T06:06:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f12865c8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f12801128a0 con 0x7f1280102a60 2024-09-21T06:06:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f1284b65640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1280108cf0 0x7f1280104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f1284b65640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1280108cf0 0x7f1280104cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38748/0 (socket says 172.21.15.3:38748) 2024-09-21T06:06:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f1284b65640 1 -- 172.21.15.3:0/2007294728 learned_addr learned my addr 172.21.15.3:0/2007294728 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f1277fff640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12801083a0 0x7f1280104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f127ffff640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1280102a60 0x7f1280104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f1284b65640 1 -- 172.21.15.3:0/2007294728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1280102a60 msgr2=0x7f1280104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f1284b65640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1280102a60 0x7f1280104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f1284b65640 1 -- 172.21.15.3:0/2007294728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12801083a0 msgr2=0x7f1280104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f1284b65640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12801083a0 0x7f1280104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.474+0000 7f1284b65640 1 -- 172.21.15.3:0/2007294728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12801a85e0 con 0x7f1280108cf0 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.475+0000 7f1284b65640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1280108cf0 0x7f1280104cd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f127800eb10 tx=0x7f127800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.475+0000 7f127dffb640 1 -- 172.21.15.3:0/2007294728 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278004b20 con 0x7f1280108cf0 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.475+0000 7f127dffb640 1 -- 172.21.15.3:0/2007294728 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f1278004cc0 con 0x7f1280108cf0 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.475+0000 7f12865c8640 1 -- 172.21.15.3:0/2007294728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12801a88d0 con 0x7f1280108cf0 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.475+0000 7f127dffb640 1 -- 172.21.15.3:0/2007294728 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278010690 con 0x7f1280108cf0 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.475+0000 7f12865c8640 1 -- 172.21.15.3:0/2007294728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12801a8e60 con 0x7f1280108cf0 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.476+0000 7f12865c8640 1 -- 172.21.15.3:0/2007294728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1280102e70 con 0x7f1280108cf0 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.477+0000 7f127dffb640 1 -- 172.21.15.3:0/2007294728 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f1278005020 con 0x7f1280108cf0 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.479+0000 7f127dffb640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f126c0b6550 0x7f126c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.479+0000 7f127ffff640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f126c0b6550 0x7f126c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.479+0000 7f127dffb640 1 -- 172.21.15.3:0/2007294728 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f12780e5bd0 con 0x7f1280108cf0 2024-09-21T06:06:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.479+0000 7f127ffff640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f126c0b6550 0x7f126c0b8a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1270000c00 tx=0x7f1270031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.483+0000 7f127dffb640 1 -- 172.21.15.3:0/2007294728 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12780ae060 con 0x7f1280108cf0 2024-09-21T06:06:46.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.674+0000 7f12865c8640 1 -- 172.21.15.3:0/2007294728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f12801a90f0 con 0x7f1280108cf0 2024-09-21T06:06:46.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.678+0000 7f127dffb640 1 -- 172.21.15.3:0/2007294728 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f12780059f0 con 0x7f1280108cf0 2024-09-21T06:06:46.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.681+0000 7f12865c8640 1 -- 172.21.15.3:0/2007294728 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f126c0b6550 msgr2=0x7f126c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:46.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.681+0000 7f12865c8640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f126c0b6550 0x7f126c0b8a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1270000c00 tx=0x7f1270031400 comp rx=0 tx=0).stop 2024-09-21T06:06:46.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.681+0000 7f12865c8640 1 -- 172.21.15.3:0/2007294728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1280108cf0 msgr2=0x7f1280104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:46.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.681+0000 7f12865c8640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1280108cf0 0x7f1280104cd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f127800eb10 tx=0x7f127800c2d0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.681+0000 7f12865c8640 1 -- 172.21.15.3:0/2007294728 shutdown_connections 2024-09-21T06:06:46.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.681+0000 7f12865c8640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f126c0b6550 0x7f126c0b8a60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.681+0000 7f12865c8640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1280108cf0 0x7f1280104cd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.681+0000 7f12865c8640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12801083a0 0x7f1280104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.681+0000 7f12865c8640 1 --2- 172.21.15.3:0/2007294728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1280102a60 0x7f1280104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:46.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.681+0000 7f12865c8640 1 -- 172.21.15.3:0/2007294728 >> 172.21.15.3:0/2007294728 conn(0x7f1280079730 msgr2=0x7f1280101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:46.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.681+0000 7f12865c8640 1 -- 172.21.15.3:0/2007294728 shutdown_connections 2024-09-21T06:06:46.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.681+0000 7f12865c8640 1 -- 172.21.15.3:0/2007294728 wait complete. 2024-09-21T06:06:46.712 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:46 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1553294083' entity='client.admin' 2024-09-21T06:06:46.712 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:46 smithi137 ceph-mon[34464]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:46.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:46 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1553294083' entity='client.admin' 2024-09-21T06:06:46.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:46 smithi204 ceph-mon[34659]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:47.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.797+0000 7f522ac65640 1 -- 172.21.15.3:0/4206895010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5224105380 msgr2=0x7f522410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.797+0000 7f522ac65640 1 --2- 172.21.15.3:0/4206895010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5224105380 0x7f522410d8d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5210009900 tx=0x7f521002a410 comp rx=0 tx=0).stop 2024-09-21T06:06:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.797+0000 7f522ac65640 1 -- 172.21.15.3:0/4206895010 shutdown_connections 2024-09-21T06:06:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.797+0000 7f522ac65640 1 --2- 172.21.15.3:0/4206895010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5224105380 0x7f522410d8d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.797+0000 7f522ac65640 1 --2- 172.21.15.3:0/4206895010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522410de60 0x7f5224110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.797+0000 7f522ac65640 1 --2- 172.21.15.3:0/4206895010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5224104a30 0x7f5224104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.797+0000 7f522ac65640 1 -- 172.21.15.3:0/4206895010 >> 172.21.15.3:0/4206895010 conn(0x7f52240797b0 msgr2=0x7f5224079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.797+0000 7f522ac65640 1 -- 172.21.15.3:0/4206895010 shutdown_connections 2024-09-21T06:06:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.797+0000 7f522ac65640 1 -- 172.21.15.3:0/4206895010 wait complete. 2024-09-21T06:06:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.798+0000 7f522ac65640 1 Processor -- start 2024-09-21T06:06:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.798+0000 7f522ac65640 1 -- start start 2024-09-21T06:06:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.798+0000 7f522ac65640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5224104a30 0x7f522419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.798+0000 7f522ac65640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5224105380 0x7f522419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.798+0000 7f522ac65640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f522410de60 0x7f522419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.798+0000 7f522ac65640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5224113870 con 0x7f5224105380 2024-09-21T06:06:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.798+0000 7f522ac65640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f52241136f0 con 0x7f5224104a30 2024-09-21T06:06:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.798+0000 7f522ac65640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f52241139f0 con 0x7f522410de60 2024-09-21T06:06:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.798+0000 7f5223fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5224105380 0x7f522419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.798+0000 7f5223fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5224105380 0x7f522419a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38778/0 (socket says 172.21.15.3:38778) 2024-09-21T06:06:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.798+0000 7f5223fff640 1 -- 172.21.15.3:0/2539378539 learned_addr learned my addr 172.21.15.3:0/2539378539 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.799+0000 7f5228a01640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5224104a30 0x7f522419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.799+0000 7f5229202640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f522410de60 0x7f522419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.799+0000 7f5223fff640 1 -- 172.21.15.3:0/2539378539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f522410de60 msgr2=0x7f522419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.799+0000 7f5223fff640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f522410de60 0x7f522419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.799+0000 7f5223fff640 1 -- 172.21.15.3:0/2539378539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5224104a30 msgr2=0x7f522419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.799+0000 7f5223fff640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5224104a30 0x7f522419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.799+0000 7f5223fff640 1 -- 172.21.15.3:0/2539378539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52241a0310 con 0x7f5224105380 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.799+0000 7f5223fff640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5224105380 0x7f522419a940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5210005ab0 tx=0x7f5210003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.799+0000 7f5221ffb640 1 -- 172.21.15.3:0/2539378539 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521003f070 con 0x7f5224105380 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.799+0000 7f522ac65640 1 -- 172.21.15.3:0/2539378539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52241a05a0 con 0x7f5224105380 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.799+0000 7f5221ffb640 1 -- 172.21.15.3:0/2539378539 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f5210031b10 con 0x7f5224105380 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.799+0000 7f5221ffb640 1 -- 172.21.15.3:0/2539378539 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210035030 con 0x7f5224105380 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.800+0000 7f522ac65640 1 -- 172.21.15.3:0/2539378539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52241a6d60 con 0x7f5224105380 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.801+0000 7f522ac65640 1 -- 172.21.15.3:0/2539378539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f522406bed0 con 0x7f5224105380 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.803+0000 7f5221ffb640 1 -- 172.21.15.3:0/2539378539 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f5210042070 con 0x7f5224105380 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.804+0000 7f5221ffb640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f52140b64d0 0x7f52140b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.804+0000 7f5228a01640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f52140b64d0 0x7f52140b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.804+0000 7f5221ffb640 1 -- 172.21.15.3:0/2539378539 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5210106300 con 0x7f5224105380 2024-09-21T06:06:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.805+0000 7f5228a01640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f52140b64d0 0x7f52140b89e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5218005d00 tx=0x7f521800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.807+0000 7f5221ffb640 1 -- 172.21.15.3:0/2539378539 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f52100ce6f0 con 0x7f5224105380 2024-09-21T06:06:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:46.999+0000 7f522ac65640 1 -- 172.21.15.3:0/2539378539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f5224108f40 con 0x7f5224105380 2024-09-21T06:06:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:47.003+0000 7f5221ffb640 1 -- 172.21.15.3:0/2539378539 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f52100d2ad0 con 0x7f5224105380 2024-09-21T06:06:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:47.006+0000 7f522ac65640 1 -- 172.21.15.3:0/2539378539 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f52140b64d0 msgr2=0x7f52140b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:47.006+0000 7f522ac65640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f52140b64d0 0x7f52140b89e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5218005d00 tx=0x7f521800b410 comp rx=0 tx=0).stop 2024-09-21T06:06:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:47.006+0000 7f522ac65640 1 -- 172.21.15.3:0/2539378539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5224105380 msgr2=0x7f522419a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:47.006+0000 7f522ac65640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5224105380 0x7f522419a940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5210005ab0 tx=0x7f5210003b90 comp rx=0 tx=0).stop 2024-09-21T06:06:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:47.006+0000 7f522ac65640 1 -- 172.21.15.3:0/2539378539 shutdown_connections 2024-09-21T06:06:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:47.006+0000 7f522ac65640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f52140b64d0 0x7f52140b89e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:47.006+0000 7f522ac65640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f522410de60 0x7f522419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:47.006+0000 7f522ac65640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5224105380 0x7f522419a940 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:47.006+0000 7f522ac65640 1 --2- 172.21.15.3:0/2539378539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5224104a30 0x7f522419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:47.006+0000 7f522ac65640 1 -- 172.21.15.3:0/2539378539 >> 172.21.15.3:0/2539378539 conn(0x7f52240797b0 msgr2=0x7f52240ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:47.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:47.006+0000 7f522ac65640 1 -- 172.21.15.3:0/2539378539 shutdown_connections 2024-09-21T06:06:47.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:47.006+0000 7f522ac65640 1 -- 172.21.15.3:0/2539378539 wait complete. 2024-09-21T06:06:48.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:47 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2007294728' entity='client.admin' 2024-09-21T06:06:48.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:47 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2539378539' entity='client.admin' 2024-09-21T06:06:48.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:47 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2007294728' entity='client.admin' 2024-09-21T06:06:48.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:47 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2539378539' entity='client.admin' 2024-09-21T06:06:48.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:47 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2007294728' entity='client.admin' 2024-09-21T06:06:48.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:47 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2539378539' entity='client.admin' 2024-09-21T06:06:48.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.480+0000 7f95ab1e6640 1 -- 172.21.15.3:0/39367150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a41083a0 msgr2=0x7f95a41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:48.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.480+0000 7f95ab1e6640 1 --2- 172.21.15.3:0/39367150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a41083a0 0x7f95a41087b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9590009900 tx=0x7f959002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.480+0000 7f95ab1e6640 1 -- 172.21.15.3:0/39367150 shutdown_connections 2024-09-21T06:06:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.480+0000 7f95ab1e6640 1 --2- 172.21.15.3:0/39367150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95a4108cf0 0x7f95a410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.480+0000 7f95ab1e6640 1 --2- 172.21.15.3:0/39367150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a41083a0 0x7f95a41087b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.480+0000 7f95ab1e6640 1 --2- 172.21.15.3:0/39367150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4102a60 0x7f95a4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.480+0000 7f95ab1e6640 1 -- 172.21.15.3:0/39367150 >> 172.21.15.3:0/39367150 conn(0x7f95a4079730 msgr2=0x7f95a4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.480+0000 7f95ab1e6640 1 -- 172.21.15.3:0/39367150 shutdown_connections 2024-09-21T06:06:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.480+0000 7f95ab1e6640 1 -- 172.21.15.3:0/39367150 wait complete. 2024-09-21T06:06:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.481+0000 7f95ab1e6640 1 Processor -- start 2024-09-21T06:06:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.481+0000 7f95ab1e6640 1 -- start start 2024-09-21T06:06:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.481+0000 7f95ab1e6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95a4102a60 0x7f95a419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.481+0000 7f95ab1e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a41083a0 0x7f95a419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.481+0000 7f95ab1e6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4108cf0 0x7f95a419fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.481+0000 7f95ab1e6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4112720 con 0x7f95a41083a0 2024-09-21T06:06:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.481+0000 7f95ab1e6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41125a0 con 0x7f95a4102a60 2024-09-21T06:06:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.481+0000 7f95ab1e6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41128a0 con 0x7f95a4108cf0 2024-09-21T06:06:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.481+0000 7f95a3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a41083a0 0x7f95a419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.481+0000 7f95a3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a41083a0 0x7f95a419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38794/0 (socket says 172.21.15.3:38794) 2024-09-21T06:06:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.482+0000 7f95a3fff640 1 -- 172.21.15.3:0/1177764985 learned_addr learned my addr 172.21.15.3:0/1177764985 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.482+0000 7f95a8f82640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95a4102a60 0x7f95a419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.482+0000 7f95a9783640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4108cf0 0x7f95a419fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.482+0000 7f95a3fff640 1 -- 172.21.15.3:0/1177764985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4108cf0 msgr2=0x7f95a419fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.482+0000 7f95a3fff640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4108cf0 0x7f95a419fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.482+0000 7f95a3fff640 1 -- 172.21.15.3:0/1177764985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95a4102a60 msgr2=0x7f95a419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.482+0000 7f95a3fff640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95a4102a60 0x7f95a419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.482+0000 7f95a3fff640 1 -- 172.21.15.3:0/1177764985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a41a00f0 con 0x7f95a41083a0 2024-09-21T06:06:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.482+0000 7f95a3fff640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a41083a0 0x7f95a419a7e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9590003c20 tx=0x7f9590003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.482+0000 7f95a1ffb640 1 -- 172.21.15.3:0/1177764985 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959003f070 con 0x7f95a41083a0 2024-09-21T06:06:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.482+0000 7f95a1ffb640 1 -- 172.21.15.3:0/1177764985 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f9590031d20 con 0x7f95a41083a0 2024-09-21T06:06:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.482+0000 7f95ab1e6640 1 -- 172.21.15.3:0/1177764985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a41a0380 con 0x7f95a41083a0 2024-09-21T06:06:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.482+0000 7f95a1ffb640 1 -- 172.21.15.3:0/1177764985 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590035030 con 0x7f95a41083a0 2024-09-21T06:06:48.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.483+0000 7f95ab1e6640 1 -- 172.21.15.3:0/1177764985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a41a08b0 con 0x7f95a41083a0 2024-09-21T06:06:48.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.484+0000 7f95ab1e6640 1 -- 172.21.15.3:0/1177764985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a4102e70 con 0x7f95a41083a0 2024-09-21T06:06:48.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.486+0000 7f95a1ffb640 1 -- 172.21.15.3:0/1177764985 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f9590042070 con 0x7f95a41083a0 2024-09-21T06:06:48.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.487+0000 7f95a1ffb640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f95940b6580 0x7f95940b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:48.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.487+0000 7f95a8f82640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f95940b6580 0x7f95940b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:48.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.487+0000 7f95a1ffb640 1 -- 172.21.15.3:0/1177764985 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9590106400 con 0x7f95a41083a0 2024-09-21T06:06:48.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.488+0000 7f95a8f82640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f95940b6580 0x7f95940b8a90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f959800abe0 tx=0x7f959800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:48.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.490+0000 7f95a1ffb640 1 -- 172.21.15.3:0/1177764985 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f95a41a08b0 con 0x7f95a41083a0 2024-09-21T06:06:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.684+0000 7f95ab1e6640 1 -- 172.21.15.3:0/1177764985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f95a410baf0 con 0x7f95a41083a0 2024-09-21T06:06:48.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.685+0000 7f95a1ffb640 1 -- 172.21.15.3:0/1177764985 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v31)=0 v31) ==== 106+0+0 (secure 0 0 0) 0x7f95900ce7f0 con 0x7f95a41083a0 2024-09-21T06:06:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.689+0000 7f95ab1e6640 1 -- 172.21.15.3:0/1177764985 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f95940b6580 msgr2=0x7f95940b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.689+0000 7f95ab1e6640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f95940b6580 0x7f95940b8a90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f959800abe0 tx=0x7f959800a2d0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.689+0000 7f95ab1e6640 1 -- 172.21.15.3:0/1177764985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a41083a0 msgr2=0x7f95a419a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.689+0000 7f95ab1e6640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a41083a0 0x7f95a419a7e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9590003c20 tx=0x7f9590003c50 comp rx=0 tx=0).stop 2024-09-21T06:06:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.689+0000 7f95ab1e6640 1 -- 172.21.15.3:0/1177764985 shutdown_connections 2024-09-21T06:06:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.689+0000 7f95ab1e6640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f95940b6580 0x7f95940b8a90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.689+0000 7f95ab1e6640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4108cf0 0x7f95a419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.689+0000 7f95ab1e6640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a41083a0 0x7f95a419a7e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.689+0000 7f95ab1e6640 1 --2- 172.21.15.3:0/1177764985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95a4102a60 0x7f95a419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.689+0000 7f95ab1e6640 1 -- 172.21.15.3:0/1177764985 >> 172.21.15.3:0/1177764985 conn(0x7f95a4079730 msgr2=0x7f95a406b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.689+0000 7f95ab1e6640 1 -- 172.21.15.3:0/1177764985 shutdown_connections 2024-09-21T06:06:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.689+0000 7f95ab1e6640 1 -- 172.21.15.3:0/1177764985 wait complete. 2024-09-21T06:06:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.832+0000 7fe2d398b640 1 -- 172.21.15.3:0/1751722852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2cc10de30 msgr2=0x7fe2cc110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.832+0000 7fe2d398b640 1 --2- 172.21.15.3:0/1751722852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2cc10de30 0x7fe2cc110300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009ca0 tx=0x7fe2bc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.832+0000 7fe2d398b640 1 -- 172.21.15.3:0/1751722852 shutdown_connections 2024-09-21T06:06:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.832+0000 7fe2d398b640 1 --2- 172.21.15.3:0/1751722852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2cc1052b0 0x7fe2cc10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.832+0000 7fe2d398b640 1 --2- 172.21.15.3:0/1751722852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2cc10de30 0x7fe2cc110300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.832+0000 7fe2d398b640 1 --2- 172.21.15.3:0/1751722852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2cc104880 0x7fe2cc104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.832+0000 7fe2d398b640 1 -- 172.21.15.3:0/1751722852 >> 172.21.15.3:0/1751722852 conn(0x7fe2cc079730 msgr2=0x7fe2cc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.833+0000 7fe2d398b640 1 -- 172.21.15.3:0/1751722852 shutdown_connections 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.833+0000 7fe2d398b640 1 -- 172.21.15.3:0/1751722852 wait complete. 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.833+0000 7fe2d398b640 1 Processor -- start 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.833+0000 7fe2d398b640 1 -- start start 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d398b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2cc104880 0x7fe2cc19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d398b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2cc1052b0 0x7fe2cc19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d398b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2cc10de30 0x7fe2cc19fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d398b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc1137e0 con 0x7fe2cc1052b0 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d398b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc113660 con 0x7fe2cc10de30 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d398b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc113960 con 0x7fe2cc104880 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d0f26640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2cc1052b0 0x7fe2cc19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d0f26640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2cc1052b0 0x7fe2cc19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36208/0 (socket says 172.21.15.3:36208) 2024-09-21T06:06:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d0f26640 1 -- 172.21.15.3:0/3634885047 learned_addr learned my addr 172.21.15.3:0/3634885047 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d1727640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2cc104880 0x7fe2cc19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d1f28640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2cc10de30 0x7fe2cc19fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d0f26640 1 -- 172.21.15.3:0/3634885047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2cc104880 msgr2=0x7fe2cc19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d0f26640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2cc104880 0x7fe2cc19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d0f26640 1 -- 172.21.15.3:0/3634885047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2cc10de30 msgr2=0x7fe2cc19fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d0f26640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2cc10de30 0x7fe2cc19fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d0f26640 1 -- 172.21.15.3:0/3634885047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc1a0200 con 0x7fe2cc1052b0 2024-09-21T06:06:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2d0f26640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2cc1052b0 0x7fe2cc19a830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000d770 tx=0x7fe2c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.834+0000 7fe2ca7fc640 1 -- 172.21.15.3:0/3634885047 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0017070 con 0x7fe2cc1052b0 2024-09-21T06:06:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.835+0000 7fe2ca7fc640 1 -- 172.21.15.3:0/3634885047 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fe2c0009c30 con 0x7fe2cc1052b0 2024-09-21T06:06:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.835+0000 7fe2d398b640 1 -- 172.21.15.3:0/3634885047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2cc1a04f0 con 0x7fe2cc1052b0 2024-09-21T06:06:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.835+0000 7fe2ca7fc640 1 -- 172.21.15.3:0/3634885047 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000f3c0 con 0x7fe2cc1052b0 2024-09-21T06:06:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.835+0000 7fe2d398b640 1 -- 172.21.15.3:0/3634885047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2cc1a6c50 con 0x7fe2cc1052b0 2024-09-21T06:06:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.836+0000 7fe2d398b640 1 -- 172.21.15.3:0/3634885047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2cc104c90 con 0x7fe2cc1052b0 2024-09-21T06:06:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.837+0000 7fe2ca7fc640 1 -- 172.21.15.3:0/3634885047 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7fe2c001a070 con 0x7fe2cc1052b0 2024-09-21T06:06:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.838+0000 7fe2ca7fc640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2b40b64d0 0x7fe2b40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.838+0000 7fe2d1727640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2b40b64d0 0x7fe2b40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.838+0000 7fe2ca7fc640 1 -- 172.21.15.3:0/3634885047 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe2c00e5170 con 0x7fe2cc1052b0 2024-09-21T06:06:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.839+0000 7fe2d1727640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2b40b64d0 0x7fe2b40b89e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc06bbc0 tx=0x7fe2c400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:42.846+0000 7fe2ca7fc640 1 -- 172.21.15.3:0/3634885047 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe2c0009de0 con 0x7fe2cc1052b0 2024-09-21T06:06:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.045+0000 7fe2d398b640 1 -- 172.21.15.3:0/3634885047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fe2cc19b3c0 con 0x7fe2cc1052b0 2024-09-21T06:06:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.050+0000 7fe2ca7fc640 1 -- 172.21.15.3:0/3634885047 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v26)=0 v26) ==== 120+0+0 (secure 0 0 0) 0x7fe2cc19b3c0 con 0x7fe2cc1052b0 2024-09-21T06:06:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.053+0000 7fe2d398b640 1 -- 172.21.15.3:0/3634885047 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2b40b64d0 msgr2=0x7fe2b40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.053+0000 7fe2d398b640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2b40b64d0 0x7fe2b40b89e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc06bbc0 tx=0x7fe2c400b390 comp rx=0 tx=0).stop 2024-09-21T06:06:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.053+0000 7fe2d398b640 1 -- 172.21.15.3:0/3634885047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2cc1052b0 msgr2=0x7fe2cc19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.053+0000 7fe2d398b640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2cc1052b0 0x7fe2cc19a830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000d770 tx=0x7fe2c000dc30 comp rx=0 tx=0).stop 2024-09-21T06:06:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.053+0000 7fe2d398b640 1 -- 172.21.15.3:0/3634885047 shutdown_connections 2024-09-21T06:06:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.053+0000 7fe2d398b640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2b40b64d0 0x7fe2b40b89e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.053+0000 7fe2d398b640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2cc10de30 0x7fe2cc19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.053+0000 7fe2d398b640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2cc1052b0 0x7fe2cc19a830 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.053+0000 7fe2d398b640 1 --2- 172.21.15.3:0/3634885047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2cc104880 0x7fe2cc19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.053+0000 7fe2d398b640 1 -- 172.21.15.3:0/3634885047 >> 172.21.15.3:0/3634885047 conn(0x7fe2cc079730 msgr2=0x7fe2cc1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.053+0000 7fe2d398b640 1 -- 172.21.15.3:0/3634885047 shutdown_connections 2024-09-21T06:06:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:43.053+0000 7fe2d398b640 1 -- 172.21.15.3:0/3634885047 wait complete. 2024-09-21T06:06:49.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:49 smithi003 ceph-mon[31227]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:49.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:49 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1540756728' entity='client.admin' 2024-09-21T06:06:49.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:49 smithi137 ceph-mon[34464]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:49.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:49 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1540756728' entity='client.admin' 2024-09-21T06:06:49.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:49 smithi204 ceph-mon[34659]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:49.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:49 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1540756728' entity='client.admin' 2024-09-21T06:06:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.467+0000 7f0fa2455640 1 -- 172.21.15.3:0/3350917545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9c108d10 msgr2=0x7f0f9c10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.467+0000 7f0fa2455640 1 --2- 172.21.15.3:0/3350917545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9c108d10 0x7f0f9c10f540 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009ca0 tx=0x7f0f9402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.467+0000 7f0fa2455640 1 -- 172.21.15.3:0/3350917545 shutdown_connections 2024-09-21T06:06:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.467+0000 7f0fa2455640 1 --2- 172.21.15.3:0/3350917545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9c108d10 0x7f0f9c10f540 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.467+0000 7f0fa2455640 1 --2- 172.21.15.3:0/3350917545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f9c1083c0 0x7f0f9c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.467+0000 7f0fa2455640 1 --2- 172.21.15.3:0/3350917545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f9c102a80 0x7f0f9c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.467+0000 7f0fa2455640 1 -- 172.21.15.3:0/3350917545 >> 172.21.15.3:0/3350917545 conn(0x7f0f9c079740 msgr2=0x7f0f9c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.467+0000 7f0fa2455640 1 -- 172.21.15.3:0/3350917545 shutdown_connections 2024-09-21T06:06:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.467+0000 7f0fa2455640 1 -- 172.21.15.3:0/3350917545 wait complete. 2024-09-21T06:06:49.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.467+0000 7f0fa2455640 1 Processor -- start 2024-09-21T06:06:49.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0fa2455640 1 -- start start 2024-09-21T06:06:49.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0fa2455640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9c102a80 0x7f0f9c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:49.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0fa2455640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f9c1083c0 0x7f0f9c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:49.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0fa2455640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f9c108d10 0x7f0f9c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0fa2455640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c112860 con 0x7f0f9c102a80 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0fa2455640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1126e0 con 0x7f0f9c1083c0 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0fa2455640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1129e0 con 0x7f0f9c108d10 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0f9bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9c102a80 0x7f0f9c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0f9bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9c102a80 0x7f0f9c19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38824/0 (socket says 172.21.15.3:38824) 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0f9bfff640 1 -- 172.21.15.3:0/545696389 learned_addr learned my addr 172.21.15.3:0/545696389 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0fa09f2640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f9c108d10 0x7f0f9c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0f9b7fe640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f9c1083c0 0x7f0f9c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0f9bfff640 1 -- 172.21.15.3:0/545696389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f9c108d10 msgr2=0x7f0f9c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0f9bfff640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f9c108d10 0x7f0f9c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0f9bfff640 1 -- 172.21.15.3:0/545696389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f9c1083c0 msgr2=0x7f0f9c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0f9bfff640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f9c1083c0 0x7f0f9c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.468+0000 7f0f9bfff640 1 -- 172.21.15.3:0/545696389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c1a03b0 con 0x7f0f9c102a80 2024-09-21T06:06:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.469+0000 7f0fa09f2640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f9c108d10 0x7f0f9c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.469+0000 7f0f9bfff640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9c102a80 0x7f0f9c19a440 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00d770 tx=0x7f0f8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.469+0000 7f0f997fa640 1 -- 172.21.15.3:0/545696389 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c017070 con 0x7f0f9c102a80 2024-09-21T06:06:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.469+0000 7f0f997fa640 1 -- 172.21.15.3:0/545696389 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0f8c0054b0 con 0x7f0f9c102a80 2024-09-21T06:06:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.469+0000 7f0f997fa640 1 -- 172.21.15.3:0/545696389 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c004c70 con 0x7f0f9c102a80 2024-09-21T06:06:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.469+0000 7f0fa2455640 1 -- 172.21.15.3:0/545696389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9c1a06a0 con 0x7f0f9c102a80 2024-09-21T06:06:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.469+0000 7f0fa2455640 1 -- 172.21.15.3:0/545696389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9c0ff570 con 0x7f0f9c102a80 2024-09-21T06:06:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.470+0000 7f0fa2455640 1 -- 172.21.15.3:0/545696389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f9c103d00 con 0x7f0f9c102a80 2024-09-21T06:06:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.472+0000 7f0f997fa640 1 -- 172.21.15.3:0/545696389 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f0f8c004e10 con 0x7f0f9c102a80 2024-09-21T06:06:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.472+0000 7f0f997fa640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0f840b6550 0x7f0f840b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.473+0000 7f0f9b7fe640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0f840b6550 0x7f0f840b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.473+0000 7f0f997fa640 1 -- 172.21.15.3:0/545696389 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0f8c0e5210 con 0x7f0f9c102a80 2024-09-21T06:06:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.473+0000 7f0f9b7fe640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0f840b6550 0x7f0f840b8a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f90005e00 tx=0x7f0f90005c50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:49.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.477+0000 7f0f997fa640 1 -- 172.21.15.3:0/545696389 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f8c0ad6a0 con 0x7f0f9c102a80 2024-09-21T06:06:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:49.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.671+0000 7f0fa2455640 1 -- 172.21.15.3:0/545696389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f0f9c10bb40 con 0x7f0f9c102a80 2024-09-21T06:06:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.677+0000 7f0f997fa640 1 -- 172.21.15.3:0/545696389 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v33)=0 v33) ==== 123+0+0 (secure 0 0 0) 0x7f0f8c0b1a80 con 0x7f0f9c102a80 2024-09-21T06:06:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.679+0000 7f0f997fa640 1 -- 172.21.15.3:0/545696389 <== mon.0 v2:172.21.15.3:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f8c0acab0 con 0x7f0f9c102a80 2024-09-21T06:06:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.681+0000 7f0fa2455640 1 -- 172.21.15.3:0/545696389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0f840b6550 msgr2=0x7f0f840b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.681+0000 7f0fa2455640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0f840b6550 0x7f0f840b8a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f90005e00 tx=0x7f0f90005c50 comp rx=0 tx=0).stop 2024-09-21T06:06:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.681+0000 7f0fa2455640 1 -- 172.21.15.3:0/545696389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9c102a80 msgr2=0x7f0f9c19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.681+0000 7f0fa2455640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9c102a80 0x7f0f9c19a440 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00d770 tx=0x7f0f8c00dc30 comp rx=0 tx=0).stop 2024-09-21T06:06:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.681+0000 7f0fa2455640 1 -- 172.21.15.3:0/545696389 shutdown_connections 2024-09-21T06:06:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.681+0000 7f0fa2455640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0f840b6550 0x7f0f840b8a60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.681+0000 7f0fa2455640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f9c108d10 0x7f0f9c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.681+0000 7f0fa2455640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f9c1083c0 0x7f0f9c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.681+0000 7f0fa2455640 1 --2- 172.21.15.3:0/545696389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9c102a80 0x7f0f9c19a440 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.681+0000 7f0fa2455640 1 -- 172.21.15.3:0/545696389 >> 172.21.15.3:0/545696389 conn(0x7f0f9c079740 msgr2=0x7f0f9c06bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.681+0000 7f0fa2455640 1 -- 172.21.15.3:0/545696389 shutdown_connections 2024-09-21T06:06:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:49.681+0000 7f0fa2455640 1 -- 172.21.15.3:0/545696389 wait complete. 2024-09-21T06:06:50.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.149+0000 7fd1af333640 1 -- 172.21.15.3:0/715360706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1a81083a0 msgr2=0x7fd1a81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:50.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.149+0000 7fd1af333640 1 --2- 172.21.15.3:0/715360706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1a81083a0 0x7fd1a81087b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd198009900 tx=0x7fd19802a410 comp rx=0 tx=0).stop 2024-09-21T06:06:50.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.149+0000 7fd1af333640 1 -- 172.21.15.3:0/715360706 shutdown_connections 2024-09-21T06:06:50.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.149+0000 7fd1af333640 1 --2- 172.21.15.3:0/715360706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1a8108cf0 0x7fd1a810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:50.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.149+0000 7fd1af333640 1 --2- 172.21.15.3:0/715360706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1a81083a0 0x7fd1a81087b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:50.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.149+0000 7fd1af333640 1 --2- 172.21.15.3:0/715360706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1a8102a60 0x7fd1a8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:50.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.149+0000 7fd1af333640 1 -- 172.21.15.3:0/715360706 >> 172.21.15.3:0/715360706 conn(0x7fd1a8079730 msgr2=0x7fd1a8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:50.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.149+0000 7fd1af333640 1 -- 172.21.15.3:0/715360706 shutdown_connections 2024-09-21T06:06:50.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.149+0000 7fd1af333640 1 -- 172.21.15.3:0/715360706 wait complete. 2024-09-21T06:06:50.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.150+0000 7fd1af333640 1 Processor -- start 2024-09-21T06:06:50.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.150+0000 7fd1af333640 1 -- start start 2024-09-21T06:06:50.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.150+0000 7fd1af333640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1a8102a60 0x7fd1a8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:50.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.150+0000 7fd1af333640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1a81083a0 0x7fd1a8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:50.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.150+0000 7fd1af333640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1a8108cf0 0x7fd1a819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:50.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.150+0000 7fd1af333640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8112960 con 0x7fd1a8102a60 2024-09-21T06:06:50.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.150+0000 7fd1af333640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a81127e0 con 0x7fd1a8108cf0 2024-09-21T06:06:50.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.150+0000 7fd1af333640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8112ae0 con 0x7fd1a81083a0 2024-09-21T06:06:50.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.150+0000 7fd1ad0cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1a8102a60 0x7fd1a8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:50.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.150+0000 7fd1ad0cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1a8102a60 0x7fd1a8199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38854/0 (socket says 172.21.15.3:38854) 2024-09-21T06:06:50.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.150+0000 7fd1ad0cf640 1 -- 172.21.15.3:0/2598603862 learned_addr learned my addr 172.21.15.3:0/2598603862 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:50.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.150+0000 7fd1ad8d0640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1a8108cf0 0x7fd1a819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:50.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.150+0000 7fd1ac8ce640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1a81083a0 0x7fd1a8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.151+0000 7fd1ad0cf640 1 -- 172.21.15.3:0/2598603862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1a81083a0 msgr2=0x7fd1a8199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.151+0000 7fd1ad0cf640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1a81083a0 0x7fd1a8199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.151+0000 7fd1ad0cf640 1 -- 172.21.15.3:0/2598603862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1a8108cf0 msgr2=0x7fd1a819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.151+0000 7fd1ad0cf640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1a8108cf0 0x7fd1a819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.151+0000 7fd1ad0cf640 1 -- 172.21.15.3:0/2598603862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a819f240 con 0x7fd1a8102a60 2024-09-21T06:06:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.151+0000 7fd1ad0cf640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1a8102a60 0x7fd1a8199390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009970 tx=0x7fd1a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.151+0000 7fd19e7fc640 1 -- 172.21.15.3:0/2598603862 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400d020 con 0x7fd1a8102a60 2024-09-21T06:06:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.151+0000 7fd19e7fc640 1 -- 172.21.15.3:0/2598603862 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a400d1c0 con 0x7fd1a8102a60 2024-09-21T06:06:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.151+0000 7fd1af333640 1 -- 172.21.15.3:0/2598603862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a819f530 con 0x7fd1a8102a60 2024-09-21T06:06:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.151+0000 7fd19e7fc640 1 -- 172.21.15.3:0/2598603862 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400cc50 con 0x7fd1a8102a60 2024-09-21T06:06:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.151+0000 7fd1af333640 1 -- 172.21.15.3:0/2598603862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a806ae60 con 0x7fd1a8102a60 2024-09-21T06:06:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.152+0000 7fd1af333640 1 -- 172.21.15.3:0/2598603862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a8103ce0 con 0x7fd1a8102a60 2024-09-21T06:06:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.154+0000 7fd19e7fc640 1 -- 172.21.15.3:0/2598603862 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7fd1a4007860 con 0x7fd1a8102a60 2024-09-21T06:06:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.154+0000 7fd19e7fc640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd1940b6580 0x7fd1940b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.155+0000 7fd1ac8ce640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd1940b6580 0x7fd1940b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.155+0000 7fd19e7fc640 1 -- 172.21.15.3:0/2598603862 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd1a40e6e80 con 0x7fd1a8102a60 2024-09-21T06:06:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.155+0000 7fd1ac8ce640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd1940b6580 0x7fd1940b8a90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd198000c00 tx=0x7fd198031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.158+0000 7fd19e7fc640 1 -- 172.21.15.3:0/2598603862 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd1a40b36f0 con 0x7fd1a8102a60 2024-09-21T06:06:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.351+0000 7fd1af333640 1 -- 172.21.15.3:0/2598603862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fd1a8102e70 con 0x7fd1a8102a60 2024-09-21T06:06:50.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.355+0000 7fd19e7fc640 1 -- 172.21.15.3:0/2598603862 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v34)=0 v34) ==== 116+0+0 (secure 0 0 0) 0x7fd1a40b8630 con 0x7fd1a8102a60 2024-09-21T06:06:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.357+0000 7fd1af333640 1 -- 172.21.15.3:0/2598603862 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd1940b6580 msgr2=0x7fd1940b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.357+0000 7fd1af333640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd1940b6580 0x7fd1940b8a90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd198000c00 tx=0x7fd198031400 comp rx=0 tx=0).stop 2024-09-21T06:06:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.357+0000 7fd1af333640 1 -- 172.21.15.3:0/2598603862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1a8102a60 msgr2=0x7fd1a8199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.357+0000 7fd1af333640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1a8102a60 0x7fd1a8199390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009970 tx=0x7fd1a4009e30 comp rx=0 tx=0).stop 2024-09-21T06:06:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.358+0000 7fd1af333640 1 -- 172.21.15.3:0/2598603862 shutdown_connections 2024-09-21T06:06:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.358+0000 7fd1af333640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd1940b6580 0x7fd1940b8a90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.358+0000 7fd1af333640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1a81083a0 0x7fd1a8199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.358+0000 7fd1af333640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1a8108cf0 0x7fd1a819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.358+0000 7fd1af333640 1 --2- 172.21.15.3:0/2598603862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1a8102a60 0x7fd1a8199390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.358+0000 7fd1af333640 1 -- 172.21.15.3:0/2598603862 >> 172.21.15.3:0/2598603862 conn(0x7fd1a8079730 msgr2=0x7fd1a8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.358+0000 7fd1af333640 1 -- 172.21.15.3:0/2598603862 shutdown_connections 2024-09-21T06:06:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:50.358+0000 7fd1af333640 1 -- 172.21.15.3:0/2598603862 wait complete. 2024-09-21T06:06:50.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:50 smithi003 ceph-mon[31227]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:50.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:50 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/545696389' entity='client.admin' 2024-09-21T06:06:50.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:50 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2598603862' entity='client.admin' 2024-09-21T06:06:51.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:50 smithi137 ceph-mon[34464]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:51.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:50 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/545696389' entity='client.admin' 2024-09-21T06:06:51.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:50 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2598603862' entity='client.admin' 2024-09-21T06:06:51.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:50 smithi204 ceph-mon[34659]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:51.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:50 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/545696389' entity='client.admin' 2024-09-21T06:06:51.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:50 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2598603862' entity='client.admin' 2024-09-21T06:06:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.654+0000 7f684c7e5640 1 -- 172.21.15.3:0/4276188890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6844102a60 msgr2=0x7f6844102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.654+0000 7f684c7e5640 1 --2- 172.21.15.3:0/4276188890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6844102a60 0x7f6844102e70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6840009900 tx=0x7f684002a410 comp rx=0 tx=0).stop 2024-09-21T06:06:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.654+0000 7f684c7e5640 1 -- 172.21.15.3:0/4276188890 shutdown_connections 2024-09-21T06:06:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.654+0000 7f684c7e5640 1 --2- 172.21.15.3:0/4276188890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68441083a0 0x7f68441087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.654+0000 7f684c7e5640 1 --2- 172.21.15.3:0/4276188890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6844108cf0 0x7f684410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.654+0000 7f684c7e5640 1 --2- 172.21.15.3:0/4276188890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6844102a60 0x7f6844102e70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.654+0000 7f684c7e5640 1 -- 172.21.15.3:0/4276188890 >> 172.21.15.3:0/4276188890 conn(0x7f6844079730 msgr2=0x7f6844079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.655+0000 7f684c7e5640 1 -- 172.21.15.3:0/4276188890 shutdown_connections 2024-09-21T06:06:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.655+0000 7f684c7e5640 1 -- 172.21.15.3:0/4276188890 wait complete. 2024-09-21T06:06:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.656+0000 7f684c7e5640 1 Processor -- start 2024-09-21T06:06:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.656+0000 7f684c7e5640 1 -- start start 2024-09-21T06:06:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.656+0000 7f684c7e5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6844102a60 0x7f684419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.656+0000 7f684c7e5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68441083a0 0x7f684419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.656+0000 7f684c7e5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6844108cf0 0x7f684419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.656+0000 7f684c7e5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6844112720 con 0x7f68441083a0 2024-09-21T06:06:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.656+0000 7f684c7e5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f68441125a0 con 0x7f6844102a60 2024-09-21T06:06:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.656+0000 7f684c7e5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68441128a0 con 0x7f6844108cf0 2024-09-21T06:06:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.656+0000 7f6849d80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68441083a0 0x7f684419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.656+0000 7f6849d80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68441083a0 0x7f684419b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38814/0 (socket says 172.21.15.3:38814) 2024-09-21T06:06:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.656+0000 7f6849d80640 1 -- 172.21.15.3:0/1540756728 learned_addr learned my addr 172.21.15.3:0/1540756728 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f684a581640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6844102a60 0x7f684419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f684ad82640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6844108cf0 0x7f684419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f6849d80640 1 -- 172.21.15.3:0/1540756728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6844108cf0 msgr2=0x7f684419bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f6849d80640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6844108cf0 0x7f684419bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f6849d80640 1 -- 172.21.15.3:0/1540756728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6844102a60 msgr2=0x7f684419b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:52.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f6849d80640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6844102a60 0x7f684419b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:52.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f6849d80640 1 -- 172.21.15.3:0/1540756728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68441a6360 con 0x7f68441083a0 2024-09-21T06:06:52.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f684ad82640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6844108cf0 0x7f684419bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:52.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f6849d80640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68441083a0 0x7f684419b610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6834005fd0 tx=0x7f683400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:52.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f683b7fe640 1 -- 172.21.15.3:0/1540756728 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6834017070 con 0x7f68441083a0 2024-09-21T06:06:52.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f683b7fe640 1 -- 172.21.15.3:0/1540756728 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f6834004490 con 0x7f68441083a0 2024-09-21T06:06:52.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f684c7e5640 1 -- 172.21.15.3:0/1540756728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68441a6650 con 0x7f68441083a0 2024-09-21T06:06:52.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f683b7fe640 1 -- 172.21.15.3:0/1540756728 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6834003be0 con 0x7f68441083a0 2024-09-21T06:06:52.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.657+0000 7f684c7e5640 1 -- 172.21.15.3:0/1540756728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68441a6be0 con 0x7f68441083a0 2024-09-21T06:06:52.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.658+0000 7f684c7e5640 1 -- 172.21.15.3:0/1540756728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6844102e70 con 0x7f68441083a0 2024-09-21T06:06:52.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.660+0000 7f683b7fe640 1 -- 172.21.15.3:0/1540756728 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f6834003d80 con 0x7f68441083a0 2024-09-21T06:06:52.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.661+0000 7f683b7fe640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f68300b6580 0x7f68300b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:52.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.662+0000 7f684a581640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f68300b6580 0x7f68300b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.662+0000 7f683b7fe640 1 -- 172.21.15.3:0/1540756728 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f68340e5bb0 con 0x7f68441083a0 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.662+0000 7f684a581640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f68300b6580 0x7f68300b8a90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6840000c00 tx=0x7f6840031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.664+0000 7f683b7fe640 1 -- 172.21.15.3:0/1540756728 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6834020030 con 0x7f68441083a0 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.872+0000 7f684c7e5640 1 -- 172.21.15.3:0/1540756728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f684406b270 con 0x7f68441083a0 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.876+0000 7f683b7fe640 1 -- 172.21.15.3:0/1540756728 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v32)=0 v32) ==== 123+0+0 (secure 0 0 0) 0x7f684406b270 con 0x7f68441083a0 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.877+0000 7f683b7fe640 1 -- 172.21.15.3:0/1540756728 <== mon.0 v2:172.21.15.3:3300/0 8 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6834009dc0 con 0x7f68441083a0 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.878+0000 7f684c7e5640 1 -- 172.21.15.3:0/1540756728 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f68300b6580 msgr2=0x7f68300b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.878+0000 7f684c7e5640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f68300b6580 0x7f68300b8a90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6840000c00 tx=0x7f6840031400 comp rx=0 tx=0).stop 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.878+0000 7f684c7e5640 1 -- 172.21.15.3:0/1540756728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68441083a0 msgr2=0x7f684419b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.878+0000 7f684c7e5640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68441083a0 0x7f684419b610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6834005fd0 tx=0x7f683400dde0 comp rx=0 tx=0).stop 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.878+0000 7f684c7e5640 1 -- 172.21.15.3:0/1540756728 shutdown_connections 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.878+0000 7f684c7e5640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f68300b6580 0x7f68300b8a90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.878+0000 7f684c7e5640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6844108cf0 0x7f684419bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.878+0000 7f684c7e5640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68441083a0 0x7f684419b610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.878+0000 7f684c7e5640 1 --2- 172.21.15.3:0/1540756728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6844102a60 0x7f684419b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.878+0000 7f684c7e5640 1 -- 172.21.15.3:0/1540756728 >> 172.21.15.3:0/1540756728 conn(0x7f6844079730 msgr2=0x7f68441015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.878+0000 7f684c7e5640 1 -- 172.21.15.3:0/1540756728 shutdown_connections 2024-09-21T06:06:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:48.878+0000 7f684c7e5640 1 -- 172.21.15.3:0/1540756728 wait complete. 2024-09-21T06:06:52.838 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:52 smithi003 ceph-mon[31227]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:53.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.946+0000 7fa7599f2640 1 -- 172.21.15.3:0/2992809244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa754102aa0 msgr2=0x7fa754102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:53.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.946+0000 7fa7599f2640 1 --2- 172.21.15.3:0/2992809244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa754102aa0 0x7fa754102eb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa744009900 tx=0x7fa74402a410 comp rx=0 tx=0).stop 2024-09-21T06:06:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.946+0000 7fa7599f2640 1 -- 172.21.15.3:0/2992809244 shutdown_connections 2024-09-21T06:06:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.946+0000 7fa7599f2640 1 --2- 172.21.15.3:0/2992809244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa754108d30 0x7fa75410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.946+0000 7fa7599f2640 1 --2- 172.21.15.3:0/2992809244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7541083e0 0x7fa7541087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.946+0000 7fa7599f2640 1 --2- 172.21.15.3:0/2992809244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa754102aa0 0x7fa754102eb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.946+0000 7fa7599f2640 1 -- 172.21.15.3:0/2992809244 >> 172.21.15.3:0/2992809244 conn(0x7fa7540797e0 msgr2=0x7fa754079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.946+0000 7fa7599f2640 1 -- 172.21.15.3:0/2992809244 shutdown_connections 2024-09-21T06:06:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.946+0000 7fa7599f2640 1 -- 172.21.15.3:0/2992809244 wait complete. 2024-09-21T06:06:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.947+0000 7fa7599f2640 1 Processor -- start 2024-09-21T06:06:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.947+0000 7fa7599f2640 1 -- start start 2024-09-21T06:06:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.947+0000 7fa7599f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa754102aa0 0x7fa75419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.947+0000 7fa7599f2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541083e0 0x7fa75419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.947+0000 7fa7599f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa754108d30 0x7fa7541a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.947+0000 7fa7599f2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa754113ab0 con 0x7fa754108d30 2024-09-21T06:06:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.947+0000 7fa7599f2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa754113930 con 0x7fa754102aa0 2024-09-21T06:06:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.947+0000 7fa7599f2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa754113c30 con 0x7fa7541083e0 2024-09-21T06:06:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.947+0000 7fa7537fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa754108d30 0x7fa7541a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.947+0000 7fa7537fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa754108d30 0x7fa7541a1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38868/0 (socket says 172.21.15.3:38868) 2024-09-21T06:06:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.947+0000 7fa7537fe640 1 -- 172.21.15.3:0/125568800 learned_addr learned my addr 172.21.15.3:0/125568800 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.947+0000 7fa74bfff640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541083e0 0x7fa75419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.948+0000 7fa752ffd640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa754102aa0 0x7fa75419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.948+0000 7fa7537fe640 1 -- 172.21.15.3:0/125568800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541083e0 msgr2=0x7fa75419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.948+0000 7fa7537fe640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541083e0 0x7fa75419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.948+0000 7fa7537fe640 1 -- 172.21.15.3:0/125568800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa754102aa0 msgr2=0x7fa75419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.948+0000 7fa7537fe640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa754102aa0 0x7fa75419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.948+0000 7fa7537fe640 1 -- 172.21.15.3:0/125568800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7541a25d0 con 0x7fa754108d30 2024-09-21T06:06:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.948+0000 7fa752ffd640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa754102aa0 0x7fa75419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.948+0000 7fa7537fe640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa754108d30 0x7fa7541a1ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00eaa0 tx=0x7fa74c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.948+0000 7fa7589f0640 1 -- 172.21.15.3:0/125568800 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c00cdb0 con 0x7fa754108d30 2024-09-21T06:06:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.948+0000 7fa7589f0640 1 -- 172.21.15.3:0/125568800 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa74c005020 con 0x7fa754108d30 2024-09-21T06:06:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.948+0000 7fa7599f2640 1 -- 172.21.15.3:0/125568800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7541a28c0 con 0x7fa754108d30 2024-09-21T06:06:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.948+0000 7fa7589f0640 1 -- 172.21.15.3:0/125568800 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0116d0 con 0x7fa754108d30 2024-09-21T06:06:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.948+0000 7fa7599f2640 1 -- 172.21.15.3:0/125568800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7541a9020 con 0x7fa754108d30 2024-09-21T06:06:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.950+0000 7fa7599f2640 1 -- 172.21.15.3:0/125568800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa754102eb0 con 0x7fa754108d30 2024-09-21T06:06:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.951+0000 7fa7589f0640 1 -- 172.21.15.3:0/125568800 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7fa74c0051d0 con 0x7fa754108d30 2024-09-21T06:06:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.952+0000 7fa7589f0640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa7400b6550 0x7fa7400b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.952+0000 7fa7589f0640 1 -- 172.21.15.3:0/125568800 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa74c0e5cf0 con 0x7fa754108d30 2024-09-21T06:06:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.952+0000 7fa752ffd640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa7400b6550 0x7fa7400b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.953+0000 7fa752ffd640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa7400b6550 0x7fa7400b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa744031440 tx=0x7fa744009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:52.955+0000 7fa7589f0640 1 -- 172.21.15.3:0/125568800 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa74c016030 con 0x7fa754108d30 2024-09-21T06:06:53.148 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:52 smithi137 ceph-mon[34464]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:53.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:52 smithi204 ceph-mon[34659]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.148+0000 7fa7599f2640 1 -- 172.21.15.3:0/125568800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fa7541a93c0 con 0x7fa754108d30 2024-09-21T06:06:53.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.152+0000 7fa7589f0640 1 -- 172.21.15.3:0/125568800 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v35)=0 v35) ==== 119+0+0 (secure 0 0 0) 0x7fa74c0ae160 con 0x7fa754108d30 2024-09-21T06:06:53.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.154+0000 7fa7599f2640 1 -- 172.21.15.3:0/125568800 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa7400b6550 msgr2=0x7fa7400b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:53.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.154+0000 7fa7599f2640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa7400b6550 0x7fa7400b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa744031440 tx=0x7fa744009680 comp rx=0 tx=0).stop 2024-09-21T06:06:53.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.154+0000 7fa7599f2640 1 -- 172.21.15.3:0/125568800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa754108d30 msgr2=0x7fa7541a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:53.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.154+0000 7fa7599f2640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa754108d30 0x7fa7541a1ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00eaa0 tx=0x7fa74c00ef60 comp rx=0 tx=0).stop 2024-09-21T06:06:53.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.155+0000 7fa7599f2640 1 -- 172.21.15.3:0/125568800 shutdown_connections 2024-09-21T06:06:53.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.155+0000 7fa7599f2640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa7400b6550 0x7fa7400b8a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.155+0000 7fa7599f2640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541083e0 0x7fa75419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.155+0000 7fa7599f2640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa754108d30 0x7fa7541a1ea0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.155+0000 7fa7599f2640 1 --2- 172.21.15.3:0/125568800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa754102aa0 0x7fa75419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:53.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.155+0000 7fa7599f2640 1 -- 172.21.15.3:0/125568800 >> 172.21.15.3:0/125568800 conn(0x7fa7540797e0 msgr2=0x7fa75410d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:53.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.155+0000 7fa7599f2640 1 -- 172.21.15.3:0/125568800 shutdown_connections 2024-09-21T06:06:53.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:53.155+0000 7fa7599f2640 1 -- 172.21.15.3:0/125568800 wait complete. 2024-09-21T06:06:53.665 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi003') 2024-09-21T06:06:54.222 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:54 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/125568800' entity='client.admin' 2024-09-21T06:06:54.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:54 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/125568800' entity='client.admin' 2024-09-21T06:06:54.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:54 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/125568800' entity='client.admin' 2024-09-21T06:06:55.573 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:55 smithi137 ceph-mon[34464]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:55.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:55.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:55.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:55 smithi137 ceph-mon[34464]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:06:55.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:55.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:06:55.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:55.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:55 smithi137 ceph-mon[34464]: Reconfiguring daemon mon.b on smithi137 2024-09-21T06:06:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:55 smithi003 ceph-mon[31227]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:55 smithi003 ceph-mon[31227]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:06:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:06:55.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:55.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:55 smithi003 ceph-mon[31227]: Reconfiguring daemon mon.b on smithi137 2024-09-21T06:06:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:55 smithi204 ceph-mon[34659]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:55 smithi204 ceph-mon[34659]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:06:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:06:55.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:55.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:55 smithi204 ceph-mon[34659]: Reconfiguring daemon mon.b on smithi137 2024-09-21T06:06:56.613 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi003') 2024-09-21T06:06:56.613 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi003') 2024-09-21T06:06:56.614 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi003') 2024-09-21T06:06:56.614 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi003') 2024-09-21T06:06:56.614 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi003') 2024-09-21T06:06:56.614 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi003') 2024-09-21T06:06:56.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:56 smithi003 ceph-mon[31227]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:56.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:56 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:56.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:56 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:56.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:56 smithi003 ceph-mon[31227]: Reconfiguring mon.c (monmap changed)... 2024-09-21T06:06:56.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:56 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:56.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:56 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:06:56.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:56 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:56.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:56 smithi003 ceph-mon[31227]: Reconfiguring daemon mon.c on smithi204 2024-09-21T06:06:56.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:56 smithi137 ceph-mon[34464]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:56.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:56 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:56.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:56 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:56.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:56 smithi137 ceph-mon[34464]: Reconfiguring mon.c (monmap changed)... 2024-09-21T06:06:56.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:56 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:56.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:56 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:06:56.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:56 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:56.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:56 smithi137 ceph-mon[34464]: Reconfiguring daemon mon.c on smithi204 2024-09-21T06:06:56.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:56 smithi204 ceph-mon[34659]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:56.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:56 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:56.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:56 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:56.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:56 smithi204 ceph-mon[34659]: Reconfiguring mon.c (monmap changed)... 2024-09-21T06:06:56.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:56 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:06:56.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:56 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:06:56.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:56 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:56.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:56 smithi204 ceph-mon[34659]: Reconfiguring daemon mon.c on smithi204 2024-09-21T06:06:57.192 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi003') 2024-09-21T06:06:57.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.412+0000 7f53abfc7640 1 -- 172.21.15.3:0/1882078152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a41083a0 msgr2=0x7f53a41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:57.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.412+0000 7f53abfc7640 1 --2- 172.21.15.3:0/1882078152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a41083a0 0x7f53a41087b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5394009900 tx=0x7f539402a410 comp rx=0 tx=0).stop 2024-09-21T06:06:57.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.412+0000 7f53abfc7640 1 -- 172.21.15.3:0/1882078152 shutdown_connections 2024-09-21T06:06:57.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.412+0000 7f53abfc7640 1 --2- 172.21.15.3:0/1882078152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53a4108cf0 0x7f53a410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.412+0000 7f53abfc7640 1 --2- 172.21.15.3:0/1882078152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a41083a0 0x7f53a41087b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.412+0000 7f53abfc7640 1 --2- 172.21.15.3:0/1882078152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a4102a60 0x7f53a4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.412+0000 7f53abfc7640 1 -- 172.21.15.3:0/1882078152 >> 172.21.15.3:0/1882078152 conn(0x7f53a4079730 msgr2=0x7f53a4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:57.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.412+0000 7f53abfc7640 1 -- 172.21.15.3:0/1882078152 shutdown_connections 2024-09-21T06:06:57.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.412+0000 7f53abfc7640 1 -- 172.21.15.3:0/1882078152 wait complete. 2024-09-21T06:06:57.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.413+0000 7f53abfc7640 1 Processor -- start 2024-09-21T06:06:57.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.413+0000 7f53abfc7640 1 -- start start 2024-09-21T06:06:57.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53abfc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53a4102a60 0x7f53a419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:57.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53abfc7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a41083a0 0x7f53a419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:57.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53abfc7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a4108cf0 0x7f53a419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:57.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53abfc7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f53a4112960 con 0x7f53a41083a0 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53abfc7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53a41127e0 con 0x7f53a4102a60 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53abfc7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f53a4112ae0 con 0x7f53a4108cf0 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53a9562640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a41083a0 0x7f53a419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53a9562640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a41083a0 0x7f53a419b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50126/0 (socket says 172.21.15.3:50126) 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53a9562640 1 -- 172.21.15.3:0/3523677170 learned_addr learned my addr 172.21.15.3:0/3523677170 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53aa564640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a4108cf0 0x7f53a419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53a9d63640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53a4102a60 0x7f53a419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53a9562640 1 -- 172.21.15.3:0/3523677170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a4108cf0 msgr2=0x7f53a419bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53a9562640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a4108cf0 0x7f53a419bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53a9562640 1 -- 172.21.15.3:0/3523677170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53a4102a60 msgr2=0x7f53a419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53a9562640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53a4102a60 0x7f53a419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53a9562640 1 -- 172.21.15.3:0/3523677170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a41a6530 con 0x7f53a41083a0 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53a9d63640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53a4102a60 0x7f53a419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:57.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.414+0000 7f53a9562640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a41083a0 0x7f53a419b870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5394032fd0 tx=0x7f5394004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:57.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.415+0000 7f539affd640 1 -- 172.21.15.3:0/3523677170 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539403f030 con 0x7f53a41083a0 2024-09-21T06:06:57.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.415+0000 7f53abfc7640 1 -- 172.21.15.3:0/3523677170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a41a67c0 con 0x7f53a41083a0 2024-09-21T06:06:57.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.415+0000 7f539affd640 1 -- 172.21.15.3:0/3523677170 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5394004490 con 0x7f53a41083a0 2024-09-21T06:06:57.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.415+0000 7f539affd640 1 -- 172.21.15.3:0/3523677170 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394035070 con 0x7f53a41083a0 2024-09-21T06:06:57.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.415+0000 7f53abfc7640 1 -- 172.21.15.3:0/3523677170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a41a6ca0 con 0x7f53a41083a0 2024-09-21T06:06:57.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.416+0000 7f53abfc7640 1 -- 172.21.15.3:0/3523677170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a4102e70 con 0x7f53a41083a0 2024-09-21T06:06:57.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.417+0000 7f539affd640 1 -- 172.21.15.3:0/3523677170 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f5394042030 con 0x7f53a41083a0 2024-09-21T06:06:57.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.418+0000 7f539affd640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f53900b6500 0x7f53900b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:57.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.419+0000 7f53a9d63640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f53900b6500 0x7f53900b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:57.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.419+0000 7f539affd640 1 -- 172.21.15.3:0/3523677170 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5394106020 con 0x7f53a41083a0 2024-09-21T06:06:57.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.419+0000 7f53a9d63640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f53900b6500 0x7f53900b8a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f53a0005fd0 tx=0x7f53a0005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:57.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.422+0000 7f539affd640 1 -- 172.21.15.3:0/3523677170 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f53940ce4b0 con 0x7f53a41083a0 2024-09-21T06:06:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.614+0000 7f53abfc7640 1 -- 172.21.15.3:0/3523677170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f53a406b270 con 0x7f53a41083a0 2024-09-21T06:06:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.618+0000 7f539affd640 1 -- 172.21.15.3:0/3523677170 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v36)=0 v36) ==== 116+0+0 (secure 0 0 0) 0x7f53a406b270 con 0x7f53a41083a0 2024-09-21T06:06:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.620+0000 7f53abfc7640 1 -- 172.21.15.3:0/3523677170 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f53900b6500 msgr2=0x7f53900b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.620+0000 7f53abfc7640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f53900b6500 0x7f53900b8a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f53a0005fd0 tx=0x7f53a0005850 comp rx=0 tx=0).stop 2024-09-21T06:06:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.620+0000 7f53abfc7640 1 -- 172.21.15.3:0/3523677170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a41083a0 msgr2=0x7f53a419b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.620+0000 7f53abfc7640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a41083a0 0x7f53a419b870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5394032fd0 tx=0x7f5394004270 comp rx=0 tx=0).stop 2024-09-21T06:06:57.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.620+0000 7f53abfc7640 1 -- 172.21.15.3:0/3523677170 shutdown_connections 2024-09-21T06:06:57.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.620+0000 7f53abfc7640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f53900b6500 0x7f53900b8a10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.620+0000 7f53abfc7640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a4108cf0 0x7f53a419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.620+0000 7f53abfc7640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53a41083a0 0x7f53a419b870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.620+0000 7f53abfc7640 1 --2- 172.21.15.3:0/3523677170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53a4102a60 0x7f53a419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:57.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.620+0000 7f53abfc7640 1 -- 172.21.15.3:0/3523677170 >> 172.21.15.3:0/3523677170 conn(0x7f53a4079730 msgr2=0x7f53a41044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:57.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.620+0000 7f53abfc7640 1 -- 172.21.15.3:0/3523677170 shutdown_connections 2024-09-21T06:06:57.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:57.620+0000 7f53abfc7640 1 -- 172.21.15.3:0/3523677170 wait complete. 2024-09-21T06:06:58.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:58.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:58.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:58.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:58.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:58.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:58.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:57 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3523677170' entity='client.admin' 2024-09-21T06:06:58.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:58.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:58.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:58.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:58.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:58.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:58.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:57 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3523677170' entity='client.admin' 2024-09-21T06:06:58.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:58.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:58.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:58.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:58.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:58.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:06:58.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:57 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3523677170' entity='client.admin' 2024-09-21T06:06:58.796 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi003') 2024-09-21T06:06:58.796 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi003') 2024-09-21T06:06:58.796 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi003') 2024-09-21T06:06:58.796 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi003') 2024-09-21T06:06:58.797 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi003') 2024-09-21T06:06:58.797 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi003') 2024-09-21T06:06:58.797 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi003') 2024-09-21T06:06:58.797 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi003') 2024-09-21T06:06:58.797 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi003') 2024-09-21T06:06:58.797 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi003') 2024-09-21T06:06:58.797 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi003') 2024-09-21T06:06:58.797 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi003') 2024-09-21T06:06:58.797 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi003') 2024-09-21T06:06:58.798 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi003') 2024-09-21T06:06:59.019 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:06:58 smithi003 ceph-mon[31227]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:59.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.248+0000 7f125f3c6640 1 -- 172.21.15.3:0/3846146398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1258110200 msgr2=0x7f1258112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.248+0000 7f125f3c6640 1 --2- 172.21.15.3:0/3846146398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1258110200 0x7f1258112620 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1250009ca0 tx=0x7f125002a950 comp rx=0 tx=0).stop 2024-09-21T06:06:59.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.248+0000 7f125f3c6640 1 -- 172.21.15.3:0/3846146398 shutdown_connections 2024-09-21T06:06:59.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.248+0000 7f125f3c6640 1 --2- 172.21.15.3:0/3846146398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1258102fa0 0x7f125810fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.248+0000 7f125f3c6640 1 --2- 172.21.15.3:0/3846146398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1258110200 0x7f1258112620 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.248+0000 7f125f3c6640 1 --2- 172.21.15.3:0/3846146398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1258102650 0x7f1258102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.248+0000 7f125f3c6640 1 -- 172.21.15.3:0/3846146398 >> 172.21.15.3:0/3846146398 conn(0x7f1258079760 msgr2=0x7f1258079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:59.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.248+0000 7f125f3c6640 1 -- 172.21.15.3:0/3846146398 shutdown_connections 2024-09-21T06:06:59.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.248+0000 7f125f3c6640 1 -- 172.21.15.3:0/3846146398 wait complete. 2024-09-21T06:06:59.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.248+0000 7f125f3c6640 1 Processor -- start 2024-09-21T06:06:59.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.248+0000 7f125f3c6640 1 -- start start 2024-09-21T06:06:59.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125f3c6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1258102650 0x7f125819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125f3c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1258102fa0 0x7f125819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125f3c6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1258110200 0x7f12581a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125f3c6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1258115ca0 con 0x7f1258102650 2024-09-21T06:06:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125f3c6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1258115b20 con 0x7f1258110200 2024-09-21T06:06:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125f3c6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1258115e20 con 0x7f1258102fa0 2024-09-21T06:06:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125d162640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1258102650 0x7f125819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125d162640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1258102650 0x7f125819c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50148/0 (socket says 172.21.15.3:50148) 2024-09-21T06:06:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125d162640 1 -- 172.21.15.3:0/1581459722 learned_addr learned my addr 172.21.15.3:0/1581459722 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:06:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125d963640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1258110200 0x7f12581a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125c961640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1258102fa0 0x7f125819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125d162640 1 -- 172.21.15.3:0/1581459722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1258102fa0 msgr2=0x7f125819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125d162640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1258102fa0 0x7f125819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125d162640 1 -- 172.21.15.3:0/1581459722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1258110200 msgr2=0x7f12581a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125d162640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1258110200 0x7f12581a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125d162640 1 -- 172.21.15.3:0/1581459722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12581a2580 con 0x7f1258102650 2024-09-21T06:06:59.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.249+0000 7f125c961640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1258102fa0 0x7f125819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:59.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.250+0000 7f125d162640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1258102650 0x7f125819c610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f125400d950 tx=0x7f125400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:59.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.250+0000 7f124e7fc640 1 -- 172.21.15.3:0/1581459722 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254017030 con 0x7f1258102650 2024-09-21T06:06:59.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.250+0000 7f124e7fc640 1 -- 172.21.15.3:0/1581459722 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1254004020 con 0x7f1258102650 2024-09-21T06:06:59.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.250+0000 7f125f3c6640 1 -- 172.21.15.3:0/1581459722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12581a2870 con 0x7f1258102650 2024-09-21T06:06:59.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.250+0000 7f124e7fc640 1 -- 172.21.15.3:0/1581459722 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12540047a0 con 0x7f1258102650 2024-09-21T06:06:59.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.250+0000 7f125f3c6640 1 -- 172.21.15.3:0/1581459722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12581a8fd0 con 0x7f1258102650 2024-09-21T06:06:59.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.251+0000 7f125f3c6640 1 -- 172.21.15.3:0/1581459722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125806c1f0 con 0x7f1258102650 2024-09-21T06:06:59.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.253+0000 7f124e7fc640 1 -- 172.21.15.3:0/1581459722 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f1254009a30 con 0x7f1258102650 2024-09-21T06:06:59.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.254+0000 7f124e7fc640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12440b6550 0x7f12440b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.254+0000 7f125c961640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12440b6550 0x7f12440b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.254+0000 7f124e7fc640 1 -- 172.21.15.3:0/1581459722 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f12540e58b0 con 0x7f1258102650 2024-09-21T06:06:59.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.255+0000 7f125c961640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12440b6550 0x7f12440b8a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1248005fd0 tx=0x7f1248005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:59.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.258+0000 7f124e7fc640 1 -- 172.21.15.3:0/1581459722 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12540add20 con 0x7f1258102650 2024-09-21T06:06:59.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:06:58 smithi137 ceph-mon[34464]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:59.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:06:58 smithi204 ceph-mon[34659]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:06:59.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.449+0000 7f125f3c6640 1 -- 172.21.15.3:0/1581459722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f1258102a60 con 0x7f1258102650 2024-09-21T06:06:59.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.453+0000 7f124e7fc640 1 -- 172.21.15.3:0/1581459722 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v37)=0 v37) ==== 138+0+0 (secure 0 0 0) 0x7f12540b2100 con 0x7f1258102650 2024-09-21T06:06:59.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.455+0000 7f125f3c6640 1 -- 172.21.15.3:0/1581459722 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12440b6550 msgr2=0x7f12440b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.455+0000 7f125f3c6640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12440b6550 0x7f12440b8a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1248005fd0 tx=0x7f1248005850 comp rx=0 tx=0).stop 2024-09-21T06:06:59.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.455+0000 7f125f3c6640 1 -- 172.21.15.3:0/1581459722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1258102650 msgr2=0x7f125819c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.455+0000 7f125f3c6640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1258102650 0x7f125819c610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f125400d950 tx=0x7f125400de10 comp rx=0 tx=0).stop 2024-09-21T06:06:59.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.455+0000 7f125f3c6640 1 -- 172.21.15.3:0/1581459722 shutdown_connections 2024-09-21T06:06:59.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.455+0000 7f125f3c6640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12440b6550 0x7f12440b8a60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.455+0000 7f125f3c6640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1258102fa0 0x7f125819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.455+0000 7f125f3c6640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1258110200 0x7f12581a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.455+0000 7f125f3c6640 1 --2- 172.21.15.3:0/1581459722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1258102650 0x7f125819c610 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.455+0000 7f125f3c6640 1 -- 172.21.15.3:0/1581459722 >> 172.21.15.3:0/1581459722 conn(0x7f1258079760 msgr2=0x7f1258109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:59.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.456+0000 7f125f3c6640 1 -- 172.21.15.3:0/1581459722 shutdown_connections 2024-09-21T06:06:59.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:06:59.456+0000 7f125f3c6640 1 -- 172.21.15.3:0/1581459722 wait complete. 2024-09-21T06:07:00.234 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi003') 2024-09-21T06:07:00.234 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi003') 2024-09-21T06:07:00.235 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi003') 2024-09-21T06:07:00.235 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi003') 2024-09-21T06:07:00.235 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '38d3c75c-77df-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi003') 2024-09-21T06:07:00.235 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T06:07:00.494 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:00.746 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:00 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1581459722' entity='client.admin' 2024-09-21T06:07:00.746 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:00.746 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:00.746 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:00.747 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:00.747 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:00 smithi003 ceph-mon[31227]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:00.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:00 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1581459722' entity='client.admin' 2024-09-21T06:07:00.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:00.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:00.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:00.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:00.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:00 smithi137 ceph-mon[34464]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:00 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1581459722' entity='client.admin' 2024-09-21T06:07:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:00.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:00.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:00.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:00.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:00 smithi204 ceph-mon[34659]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:00.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.970+0000 7fc91299d640 1 -- 172.21.15.3:0/387925511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c103bc0 msgr2=0x7fc90c103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.970+0000 7fc91299d640 1 --2- 172.21.15.3:0/387925511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c103bc0 0x7fc90c103fd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009900 tx=0x7fc8fc02a410 comp rx=0 tx=0).stop 2024-09-21T06:07:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.970+0000 7fc91299d640 1 -- 172.21.15.3:0/387925511 shutdown_connections 2024-09-21T06:07:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.970+0000 7fc91299d640 1 --2- 172.21.15.3:0/387925511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90c109e00 0x7fc90c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.970+0000 7fc91299d640 1 --2- 172.21.15.3:0/387925511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c1094b0 0x7fc90c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.970+0000 7fc91299d640 1 --2- 172.21.15.3:0/387925511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c103bc0 0x7fc90c103fd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.970+0000 7fc91299d640 1 -- 172.21.15.3:0/387925511 >> 172.21.15.3:0/387925511 conn(0x7fc90c07b910 msgr2=0x7fc90c07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.970+0000 7fc91299d640 1 -- 172.21.15.3:0/387925511 shutdown_connections 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.970+0000 7fc91299d640 1 -- 172.21.15.3:0/387925511 wait complete. 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.971+0000 7fc91299d640 1 Processor -- start 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.971+0000 7fc91299d640 1 -- start start 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.971+0000 7fc91299d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90c103bc0 0x7fc90c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.971+0000 7fc91299d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c1094b0 0x7fc90c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.971+0000 7fc91299d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c109e00 0x7fc90c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.971+0000 7fc91299d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c113b20 con 0x7fc90c109e00 2024-09-21T06:07:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.971+0000 7fc91299d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c1139a0 con 0x7fc90c1094b0 2024-09-21T06:07:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.971+0000 7fc91299d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c113ca0 con 0x7fc90c103bc0 2024-09-21T06:07:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.971+0000 7fc910f3a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c109e00 0x7fc90c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.971+0000 7fc910f3a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c109e00 0x7fc90c076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50172/0 (socket says 172.21.15.3:50172) 2024-09-21T06:07:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.971+0000 7fc910f3a640 1 -- 172.21.15.3:0/2085684630 learned_addr learned my addr 172.21.15.3:0/2085684630 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:07:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.971+0000 7fc90bfff640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90c103bc0 0x7fc90c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.972+0000 7fc90b7fe640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c1094b0 0x7fc90c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.972+0000 7fc910f3a640 1 -- 172.21.15.3:0/2085684630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90c103bc0 msgr2=0x7fc90c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.972+0000 7fc910f3a640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90c103bc0 0x7fc90c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.972+0000 7fc910f3a640 1 -- 172.21.15.3:0/2085684630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c1094b0 msgr2=0x7fc90c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.972+0000 7fc910f3a640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c1094b0 0x7fc90c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.972+0000 7fc910f3a640 1 -- 172.21.15.3:0/2085684630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c1a87f0 con 0x7fc90c109e00 2024-09-21T06:07:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.972+0000 7fc910f3a640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c109e00 0x7fc90c076190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc90400b970 tx=0x7fc90400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.972+0000 7fc9097fa640 1 -- 172.21.15.3:0/2085684630 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904019070 con 0x7fc90c109e00 2024-09-21T06:07:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.972+0000 7fc9097fa640 1 -- 172.21.15.3:0/2085684630 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc904014440 con 0x7fc90c109e00 2024-09-21T06:07:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.972+0000 7fc91299d640 1 -- 172.21.15.3:0/2085684630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c1a8ae0 con 0x7fc90c109e00 2024-09-21T06:07:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.972+0000 7fc9097fa640 1 -- 172.21.15.3:0/2085684630 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9040075d0 con 0x7fc90c109e00 2024-09-21T06:07:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.972+0000 7fc91299d640 1 -- 172.21.15.3:0/2085684630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c1a9050 con 0x7fc90c109e00 2024-09-21T06:07:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.973+0000 7fc91299d640 1 -- 172.21.15.3:0/2085684630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90c103fd0 con 0x7fc90c109e00 2024-09-21T06:07:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.975+0000 7fc9097fa640 1 -- 172.21.15.3:0/2085684630 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7fc90401c070 con 0x7fc90c109e00 2024-09-21T06:07:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.976+0000 7fc9097fa640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc8f40b64d0 0x7fc8f40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.976+0000 7fc90bfff640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc8f40b64d0 0x7fc8f40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.976+0000 7fc9097fa640 1 -- 172.21.15.3:0/2085684630 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc9040e5150 con 0x7fc90c109e00 2024-09-21T06:07:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.976+0000 7fc90bfff640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc8f40b64d0 0x7fc8f40b89e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000c00 tx=0x7fc8fc031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:00.979+0000 7fc9097fa640 1 -- 172.21.15.3:0/2085684630 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9040b1920 con 0x7fc90c109e00 2024-09-21T06:07:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.166+0000 7fc91299d640 1 -- 172.21.15.3:0/2085684630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc90c076a10 con 0x7fc90c109e00 2024-09-21T06:07:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.167+0000 7fc9097fa640 1 -- 172.21.15.3:0/2085684630 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+22127 (secure 0 0 0) 0x7fc90c076a10 con 0x7fc90c109e00 2024-09-21T06:07:01.169 INFO:teuthology.orchestra.run.smithi003.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:07:01.169 INFO:teuthology.orchestra.run.smithi003.stdout:global dev auth_debug true 2024-09-21T06:07:01.169 INFO:teuthology.orchestra.run.smithi003.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T06:07:01.169 INFO:teuthology.orchestra.run.smithi003.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T06:07:01.169 INFO:teuthology.orchestra.run.smithi003.stdout:global basic log_to_file true 2024-09-21T06:07:01.169 INFO:teuthology.orchestra.run.smithi003.stdout:global basic log_to_journald false 2024-09-21T06:07:01.169 INFO:teuthology.orchestra.run.smithi003.stdout:global basic log_to_stderr false 2024-09-21T06:07:01.169 INFO:teuthology.orchestra.run.smithi003.stdout:global advanced mon_allow_pool_delete true 2024-09-21T06:07:01.169 INFO:teuthology.orchestra.run.smithi003.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:global dev ms_die_on_bug true 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:global dev ms_die_on_old_message true 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:global dev ms_die_on_skipped_message false 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T06:07:01.170 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced debug_mon 20/20 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced debug_ms 1/1 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced debug_paxos 20/20 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T06:07:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced debug_cephsqlite 20/20 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced debug_client 20/20 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced mgr/snap_schedule/allow_m_granularity true 2024-09-21T06:07:01.172 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced mgr/snap_schedule/dump_on_update true 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced mgr/snap_schedule/log_level debug 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:osd dev bdev_debug_aio true 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced bluestore_compression_mode aggressive 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced debug_bluefs 20/20 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced debug_bluestore 20/20 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced debug_ms 1/1 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced debug_osd 20/20 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced debug_rocksdb 10/10 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:osd dev osd_debug_op_order true 2024-09-21T06:07:01.173 INFO:teuthology.orchestra.run.smithi003.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd dev osd_debug_shutdown true 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_max_omap_entries_per_request 10 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T06:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:mds advanced debug_mds 20/20 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:mds advanced debug_ms 2/2 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:mds advanced mds_bal_split_size 100 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:mds dev mds_debug_frag true 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:mds advanced mds_export_ephemeral_random_max 0.100000 2024-09-21T06:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:07:01.176 INFO:teuthology.orchestra.run.smithi003.stdout:mds dev mds_verify_scatter true 2024-09-21T06:07:01.176 INFO:teuthology.orchestra.run.smithi003.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:07:01.176 INFO:teuthology.orchestra.run.smithi003.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:07:01.176 INFO:teuthology.orchestra.run.smithi003.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:07:01.176 INFO:teuthology.orchestra.run.smithi003.stdout:client advanced client_mount_timeout 600 2024-09-21T06:07:01.176 INFO:teuthology.orchestra.run.smithi003.stdout:client advanced debug_client 20/20 2024-09-21T06:07:01.176 INFO:teuthology.orchestra.run.smithi003.stdout:client advanced debug_ms 1/1 2024-09-21T06:07:01.176 INFO:teuthology.orchestra.run.smithi003.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:07:01.176 INFO:teuthology.orchestra.run.smithi003.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:07:01.176 INFO:teuthology.orchestra.run.smithi003.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T06:07:01.176 INFO:teuthology.orchestra.run.smithi003.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T06:07:01.176 INFO:teuthology.orchestra.run.smithi003.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T06:07:01.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.169+0000 7fc91299d640 1 -- 172.21.15.3:0/2085684630 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc8f40b64d0 msgr2=0x7fc8f40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:01.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.169+0000 7fc91299d640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc8f40b64d0 0x7fc8f40b89e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000c00 tx=0x7fc8fc031400 comp rx=0 tx=0).stop 2024-09-21T06:07:01.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.169+0000 7fc91299d640 1 -- 172.21.15.3:0/2085684630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c109e00 msgr2=0x7fc90c076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:01.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.169+0000 7fc91299d640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c109e00 0x7fc90c076190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc90400b970 tx=0x7fc90400be30 comp rx=0 tx=0).stop 2024-09-21T06:07:01.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.169+0000 7fc91299d640 1 -- 172.21.15.3:0/2085684630 shutdown_connections 2024-09-21T06:07:01.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.169+0000 7fc91299d640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc8f40b64d0 0x7fc8f40b89e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:01.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.169+0000 7fc91299d640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c109e00 0x7fc90c076190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:01.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.169+0000 7fc91299d640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c1094b0 0x7fc90c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:01.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.169+0000 7fc91299d640 1 --2- 172.21.15.3:0/2085684630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90c103bc0 0x7fc90c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:01.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.169+0000 7fc91299d640 1 -- 172.21.15.3:0/2085684630 >> 172.21.15.3:0/2085684630 conn(0x7fc90c07b910 msgr2=0x7fc90c1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:01.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.169+0000 7fc91299d640 1 -- 172.21.15.3:0/2085684630 shutdown_connections 2024-09-21T06:07:01.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:01.169+0000 7fc91299d640 1 -- 172.21.15.3:0/2085684630 wait complete. 2024-09-21T06:07:01.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:01 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2085684630' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T06:07:01.874 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T06:07:01.875 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:07:01.875 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:07:01.892 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:07:01.892 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2024-09-21T06:07:01.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:01 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2085684630' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T06:07:01.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:01 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2085684630' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T06:07:01.949 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:07:01.949 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:07:01.950 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T06:07:01.950 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:07:01.950 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:07:01.950 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-21 06:05:37.145680440 +0000 2024-09-21T06:07:01.950 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-21 06:00:56.088908334 +0000 2024-09-21T06:07:01.950 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-21 06:00:56.088908334 +0000 2024-09-21T06:07:01.950 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-21 06:00:56.088908334 +0000 2024-09-21T06:07:01.950 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:07:02.016 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-21T06:07:02.016 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-21T06:07:02.016 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000271259 s, 1.9 MB/s 2024-09-21T06:07:02.018 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:07:02.075 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2024-09-21T06:07:02.132 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:07:02.132 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:07:02.132 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-21T06:07:02.132 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:07:02.132 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:07:02.132 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-21 06:05:37.178679997 +0000 2024-09-21T06:07:02.132 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-21 06:00:56.430903569 +0000 2024-09-21T06:07:02.133 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-21 06:00:56.430903569 +0000 2024-09-21T06:07:02.133 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-21 06:00:56.430903569 +0000 2024-09-21T06:07:02.133 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:07:02.196 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-21T06:07:02.196 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-21T06:07:02.196 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000215191 s, 2.4 MB/s 2024-09-21T06:07:02.197 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:07:02.254 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2024-09-21T06:07:02.310 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:07:02.310 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:07:02.311 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T06:07:02.311 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:07:02.311 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:07:02.311 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-21 06:05:37.210679567 +0000 2024-09-21T06:07:02.311 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-21 06:00:56.770898831 +0000 2024-09-21T06:07:02.311 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-21 06:00:56.770898831 +0000 2024-09-21T06:07:02.311 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-21 06:00:56.770898831 +0000 2024-09-21T06:07:02.311 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:07:02.375 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-21T06:07:02.375 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-21T06:07:02.375 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000196278 s, 2.6 MB/s 2024-09-21T06:07:02.376 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:07:02.433 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2024-09-21T06:07:02.490 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:07:02.490 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:07:02.490 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-21T06:07:02.490 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:07:02.490 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:07:02.490 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-21 06:05:37.243679124 +0000 2024-09-21T06:07:02.490 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-21 06:00:57.108894121 +0000 2024-09-21T06:07:02.490 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-21 06:00:57.108894121 +0000 2024-09-21T06:07:02.491 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-21 06:00:57.108894121 +0000 2024-09-21T06:07:02.491 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:07:02.554 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-21T06:07:02.554 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-21T06:07:02.554 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000231442 s, 2.2 MB/s 2024-09-21T06:07:02.555 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:07:02.612 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-21T06:07:02.612 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:07:02.632 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:07:02.633 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-09-21T06:07:02.691 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:07:02.691 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:07:02.691 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-21T06:07:02.691 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:07:02.692 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:07:02.692 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-09-21 06:06:32.215943563 +0000 2024-09-21T06:07:02.692 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-09-21 06:01:26.972181055 +0000 2024-09-21T06:07:02.692 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-09-21 06:01:26.972181055 +0000 2024-09-21T06:07:02.692 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-09-21 06:01:26.972181055 +0000 2024-09-21T06:07:02.692 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:07:02.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:02 smithi137 ceph-mon[34464]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:02.762 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-09-21T06:07:02.762 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-09-21T06:07:02.762 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000282466 s, 1.8 MB/s 2024-09-21T06:07:02.764 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:07:02.824 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-09-21T06:07:02.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:02 smithi003 ceph-mon[31227]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:02.882 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:07:02.882 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:07:02.882 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-21T06:07:02.882 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:07:02.882 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:07:02.883 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-09-21 06:06:32.216943546 +0000 2024-09-21T06:07:02.883 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-09-21 06:01:27.302175250 +0000 2024-09-21T06:07:02.883 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-09-21 06:01:27.302175250 +0000 2024-09-21T06:07:02.883 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-09-21 06:01:27.302175250 +0000 2024-09-21T06:07:02.883 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:07:02.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:02 smithi204 ceph-mon[34659]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:02.949 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-09-21T06:07:02.949 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-09-21T06:07:02.949 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000234597 s, 2.2 MB/s 2024-09-21T06:07:02.950 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:07:03.008 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-09-21T06:07:03.065 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:07:03.065 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:07:03.065 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T06:07:03.065 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:07:03.065 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:07:03.065 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-09-21 06:06:32.216943546 +0000 2024-09-21T06:07:03.065 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-09-21 06:01:27.629169498 +0000 2024-09-21T06:07:03.066 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-09-21 06:01:27.629169498 +0000 2024-09-21T06:07:03.066 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-09-21 06:01:27.629169498 +0000 2024-09-21T06:07:03.066 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:07:03.130 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-09-21T06:07:03.130 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-09-21T06:07:03.130 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000201636 s, 2.5 MB/s 2024-09-21T06:07:03.132 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:07:03.188 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-09-21T06:07:03.244 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:07:03.244 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:07:03.244 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-21T06:07:03.244 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:07:03.244 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:07:03.245 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-09-21 06:06:32.216943546 +0000 2024-09-21T06:07:03.245 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-09-21 06:01:27.952163816 +0000 2024-09-21T06:07:03.245 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-09-21 06:01:27.952163816 +0000 2024-09-21T06:07:03.245 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-09-21 06:01:27.952163816 +0000 2024-09-21T06:07:03.245 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:07:03.309 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-09-21T06:07:03.309 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-09-21T06:07:03.309 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000257775 s, 2.0 MB/s 2024-09-21T06:07:03.310 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:07:03.367 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-21T06:07:03.367 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:07:03.387 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:07:03.387 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-09-21T06:07:03.445 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:07:03.445 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:07:03.446 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T06:07:03.446 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:07:03.446 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:07:03.446 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-21 06:06:29.826547352 +0000 2024-09-21T06:07:03.446 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-21 06:01:05.884155442 +0000 2024-09-21T06:07:03.446 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-21 06:01:05.884155442 +0000 2024-09-21T06:07:03.446 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-21 06:01:05.884155442 +0000 2024-09-21T06:07:03.446 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:07:03.517 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-21T06:07:03.517 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-21T06:07:03.517 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000291952 s, 1.8 MB/s 2024-09-21T06:07:03.518 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:07:03.577 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-09-21T06:07:03.635 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:07:03.636 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:07:03.636 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-21T06:07:03.636 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:07:03.636 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:07:03.636 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-21 06:06:29.826547352 +0000 2024-09-21T06:07:03.636 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-21 06:01:06.212151089 +0000 2024-09-21T06:07:03.636 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-21 06:01:06.212151089 +0000 2024-09-21T06:07:03.636 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-21 06:01:06.212151089 +0000 2024-09-21T06:07:03.636 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:07:03.701 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-21T06:07:03.701 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-21T06:07:03.702 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000226386 s, 2.3 MB/s 2024-09-21T06:07:03.703 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:07:03.760 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-09-21T06:07:03.817 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:07:03.817 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:07:03.817 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-21T06:07:03.818 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:07:03.818 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:07:03.818 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-21 06:06:29.826547352 +0000 2024-09-21T06:07:03.818 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-21 06:01:06.530146869 +0000 2024-09-21T06:07:03.818 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-21 06:01:06.530146869 +0000 2024-09-21T06:07:03.818 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-21 06:01:06.530146869 +0000 2024-09-21T06:07:03.818 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:07:03.883 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-21T06:07:03.883 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-21T06:07:03.883 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000227806 s, 2.2 MB/s 2024-09-21T06:07:03.884 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:07:03.941 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-09-21T06:07:03.997 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:07:03.997 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:07:03.997 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T06:07:03.997 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:07:03.997 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:07:03.997 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-21 06:06:29.827547339 +0000 2024-09-21T06:07:03.997 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-21 06:01:06.859142503 +0000 2024-09-21T06:07:03.997 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-21 06:01:06.859142503 +0000 2024-09-21T06:07:03.997 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-21 06:01:06.859142503 +0000 2024-09-21T06:07:03.998 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:07:04.062 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-21T06:07:04.062 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-21T06:07:04.062 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000201723 s, 2.5 MB/s 2024-09-21T06:07:04.063 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:07:04.120 INFO:tasks.cephadm:Deploying osd.0 on smithi003 with /dev/vg_nvme/lv_4... 2024-09-21T06:07:04.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T06:07:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:04.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:04 smithi003 ceph-mon[31227]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:04.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:04 smithi137 ceph-mon[34464]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:04.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:04 smithi204 ceph-mon[34659]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:06.892 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:06 smithi003 ceph-mon[31227]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:06.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:06 smithi137 ceph-mon[34464]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:06.896 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:07:06.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:06 smithi204 ceph-mon[34659]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:06.926 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi003:vg_nvme/lv_4 2024-09-21T06:07:07.182 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:07.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.614+0000 7f1306a3d640 1 -- 172.21.15.3:0/4059438748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1300102a60 msgr2=0x7f1300102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:07.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.614+0000 7f1306a3d640 1 --2- 172.21.15.3:0/4059438748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1300102a60 0x7f1300102e70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009900 tx=0x7f12f002a410 comp rx=0 tx=0).stop 2024-09-21T06:07:07.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.614+0000 7f1306a3d640 1 -- 172.21.15.3:0/4059438748 shutdown_connections 2024-09-21T06:07:07.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.614+0000 7f1306a3d640 1 --2- 172.21.15.3:0/4059438748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1300108cf0 0x7f130010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:07.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.614+0000 7f1306a3d640 1 --2- 172.21.15.3:0/4059438748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13001083a0 0x7f13001087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:07.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.614+0000 7f1306a3d640 1 --2- 172.21.15.3:0/4059438748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1300102a60 0x7f1300102e70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.615+0000 7f1306a3d640 1 -- 172.21.15.3:0/4059438748 >> 172.21.15.3:0/4059438748 conn(0x7f1300079730 msgr2=0x7f1300079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.615+0000 7f1306a3d640 1 -- 172.21.15.3:0/4059438748 shutdown_connections 2024-09-21T06:07:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.615+0000 7f1306a3d640 1 -- 172.21.15.3:0/4059438748 wait complete. 2024-09-21T06:07:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.615+0000 7f1306a3d640 1 Processor -- start 2024-09-21T06:07:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.615+0000 7f1306a3d640 1 -- start start 2024-09-21T06:07:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f1306a3d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1300102a60 0x7f130019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f1306a3d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13001083a0 0x7f130019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f1306a3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1300108cf0 0x7f130019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f1306a3d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f13001129c0 con 0x7f1300102a60 2024-09-21T06:07:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f1306a3d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1300112840 con 0x7f1300108cf0 2024-09-21T06:07:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f1306a3d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1300112b40 con 0x7f13001083a0 2024-09-21T06:07:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f12fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1300102a60 0x7f130019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f12fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1300102a60 0x7f130019a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39670/0 (socket says 172.21.15.3:39670) 2024-09-21T06:07:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f12fffff640 1 -- 172.21.15.3:0/3965814606 learned_addr learned my addr 172.21.15.3:0/3965814606 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:07:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f12f7fff640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13001083a0 0x7f130019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f12fffff640 1 -- 172.21.15.3:0/3965814606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13001083a0 msgr2=0x7f130019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f12fffff640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13001083a0 0x7f130019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f1304fda640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1300108cf0 0x7f130019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f12fffff640 1 -- 172.21.15.3:0/3965814606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1300108cf0 msgr2=0x7f130019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:07.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f12fffff640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1300108cf0 0x7f130019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:07.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f12fffff640 1 -- 172.21.15.3:0/3965814606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13001a0470 con 0x7f1300102a60 2024-09-21T06:07:07.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f12f7fff640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13001083a0 0x7f130019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:07.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f1304fda640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1300108cf0 0x7f130019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:07.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f12fffff640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1300102a60 0x7f130019a500 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005940 tx=0x7f12f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:07.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.616+0000 7f12fdffb640 1 -- 172.21.15.3:0/3965814606 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f003f030 con 0x7f1300102a60 2024-09-21T06:07:07.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.617+0000 7f12fdffb640 1 -- 172.21.15.3:0/3965814606 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f0003cb0 con 0x7f1300102a60 2024-09-21T06:07:07.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.617+0000 7f1306a3d640 1 -- 172.21.15.3:0/3965814606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13001a0700 con 0x7f1300102a60 2024-09-21T06:07:07.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.617+0000 7f12fdffb640 1 -- 172.21.15.3:0/3965814606 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0035070 con 0x7f1300102a60 2024-09-21T06:07:07.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.617+0000 7f1306a3d640 1 -- 172.21.15.3:0/3965814606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13001a6ec0 con 0x7f1300102a60 2024-09-21T06:07:07.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.618+0000 7f1306a3d640 1 -- 172.21.15.3:0/3965814606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1300103ce0 con 0x7f1300102a60 2024-09-21T06:07:07.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.619+0000 7f12fdffb640 1 -- 172.21.15.3:0/3965814606 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f12f0042030 con 0x7f1300102a60 2024-09-21T06:07:07.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.620+0000 7f12fdffb640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12ec0b6550 0x7f12ec0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:07.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.620+0000 7f12f7fff640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12ec0b6550 0x7f12ec0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:07.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.620+0000 7f12fdffb640 1 -- 172.21.15.3:0/3965814606 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f12f0106320 con 0x7f1300102a60 2024-09-21T06:07:07.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.621+0000 7f12f7fff640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12ec0b6550 0x7f12ec0b8a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1300100ee0 tx=0x7f12e8005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:07.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.623+0000 7f12fdffb640 1 -- 172.21.15.3:0/3965814606 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12f00ce710 con 0x7f1300102a60 2024-09-21T06:07:07.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:07.824+0000 7f1306a3d640 1 -- 172.21.15.3:0/3965814606 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f130006aef0 con 0x7f12ec0b6550 2024-09-21T06:07:08.834 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:08 smithi003 ceph-mon[31227]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:08.834 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:08 smithi003 ceph-mon[31227]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:08.835 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:08.835 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:08.835 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:08.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:08 smithi137 ceph-mon[34464]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:08.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:08 smithi137 ceph-mon[34464]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:08.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:08.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:08.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:08.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:08 smithi204 ceph-mon[34659]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:08.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:08 smithi204 ceph-mon[34659]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:08.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:08.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:08.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:10.753 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:10 smithi003 ceph-mon[31227]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:10.753 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:10 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/4168391544' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "51736f81-f410-48ee-a8f3-121289607309"} : dispatch 2024-09-21T06:07:10.753 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:10 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/4168391544' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51736f81-f410-48ee-a8f3-121289607309"}]': finished 2024-09-21T06:07:10.753 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:10 smithi003 ceph-mon[31227]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T06:07:10.753 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:10 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:10.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:10 smithi137 ceph-mon[34464]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:10.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:10 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/4168391544' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "51736f81-f410-48ee-a8f3-121289607309"} : dispatch 2024-09-21T06:07:10.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:10 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/4168391544' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51736f81-f410-48ee-a8f3-121289607309"}]': finished 2024-09-21T06:07:10.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:10 smithi137 ceph-mon[34464]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T06:07:10.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:10 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:10.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:10 smithi204 ceph-mon[34659]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:10.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:10 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/4168391544' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "51736f81-f410-48ee-a8f3-121289607309"} : dispatch 2024-09-21T06:07:10.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:10 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/4168391544' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51736f81-f410-48ee-a8f3-121289607309"}]': finished 2024-09-21T06:07:10.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:10 smithi204 ceph-mon[34659]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T06:07:10.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:10 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:11.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:11 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2465960837' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:07:11.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:11 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2465960837' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:07:12.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:11 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2465960837' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:07:12.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:12 smithi137 ceph-mon[34464]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:12 smithi204 ceph-mon[34659]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:13.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:12 smithi003 ceph-mon[31227]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:14.892 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:14 smithi003 ceph-mon[31227]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:14.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:14 smithi137 ceph-mon[34464]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:14.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:14 smithi204 ceph-mon[34659]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:16.729 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:16 smithi003 ceph-mon[31227]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:16.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:16 smithi137 ceph-mon[34464]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:16.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:16 smithi204 ceph-mon[34659]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:18.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:18 smithi003 ceph-mon[31227]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:18.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:18 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T06:07:18.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:18 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:18 smithi137 ceph-mon[34464]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:18 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T06:07:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:18 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:18.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:18 smithi204 ceph-mon[34659]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:18.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:18 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T06:07:18.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:18 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:19.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:19 smithi137 ceph-mon[34464]: Deploying daemon osd.0 on smithi003 2024-09-21T06:07:19.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:19 smithi204 ceph-mon[34659]: Deploying daemon osd.0 on smithi003 2024-09-21T06:07:19.986 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:19 smithi003 ceph-mon[31227]: Deploying daemon osd.0 on smithi003 2024-09-21T06:07:20.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:20 smithi003 ceph-mon[31227]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:20 smithi137 ceph-mon[34464]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:20 smithi204 ceph-mon[34659]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:22 smithi137 ceph-mon[34464]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:22 smithi204 ceph-mon[34659]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:23.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:22 smithi003 ceph-mon[31227]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:24.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:24 smithi137 ceph-mon[34464]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:24.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:24 smithi204 ceph-mon[34659]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:24.939 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:24 smithi003 ceph-mon[31227]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:25.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:25.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:25.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:27.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:27 smithi003 ceph-mon[31227]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:27.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:27 smithi137 ceph-mon[34464]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:27.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:27 smithi204 ceph-mon[34659]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:28.820 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:28.820 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:28.820 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:28 smithi003 ceph-mon[31227]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:28.820 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:28.820 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:28.820 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:28.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:28.858+0000 7f12fdffb640 1 -- 172.21.15.3:0/3965814606 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f130006aef0 con 0x7f12ec0b6550 2024-09-21T06:07:28.860 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 0 on host 'smithi003' 2024-09-21T06:07:28.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:28.862+0000 7f1306a3d640 1 -- 172.21.15.3:0/3965814606 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12ec0b6550 msgr2=0x7f12ec0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:28.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:28.862+0000 7f1306a3d640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12ec0b6550 0x7f12ec0b8a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1300100ee0 tx=0x7f12e8005e50 comp rx=0 tx=0).stop 2024-09-21T06:07:28.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:28.862+0000 7f1306a3d640 1 -- 172.21.15.3:0/3965814606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1300102a60 msgr2=0x7f130019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:28.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:28.862+0000 7f1306a3d640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1300102a60 0x7f130019a500 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005940 tx=0x7f12f00042f0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:28.862+0000 7f1306a3d640 1 -- 172.21.15.3:0/3965814606 shutdown_connections 2024-09-21T06:07:28.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:28.862+0000 7f1306a3d640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f12ec0b6550 0x7f12ec0b8a60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:28.862+0000 7f1306a3d640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13001083a0 0x7f130019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:28.862+0000 7f1306a3d640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1300108cf0 0x7f130019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:28.862+0000 7f1306a3d640 1 --2- 172.21.15.3:0/3965814606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1300102a60 0x7f130019a500 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:28.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:28.862+0000 7f1306a3d640 1 -- 172.21.15.3:0/3965814606 >> 172.21.15.3:0/3965814606 conn(0x7f1300079730 msgr2=0x7f1300101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:28.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:28.862+0000 7f1306a3d640 1 -- 172.21.15.3:0/3965814606 shutdown_connections 2024-09-21T06:07:28.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:07:28.862+0000 7f1306a3d640 1 -- 172.21.15.3:0/3965814606 wait complete. 2024-09-21T06:07:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:28 smithi137 ceph-mon[34464]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:28 smithi204 ceph-mon[34659]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:29.477 DEBUG:teuthology.orchestra.run.smithi003:osd.0> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T06:07:29.479 INFO:tasks.cephadm:Deploying osd.1 on smithi137 with /dev/vg_nvme/lv_4... 2024-09-21T06:07:29.480 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T06:07:29.685 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:29.685 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:29.685 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:29.718 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:07:29.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:29.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:29.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:30.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:30 smithi003 ceph-mon[31227]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:30.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:30 smithi137 ceph-mon[34464]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:30.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:30 smithi204 ceph-mon[34659]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:31.091 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:07:30 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0[48021]: 2024-09-21T06:07:30.919+0000 7f60d2e27640 -1 osd.0 0 log_to_monitors true 2024-09-21T06:07:31.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:31 smithi003 ceph-mon[31227]: from='osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:07:31.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:31 smithi137 ceph-mon[34464]: from='osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:07:31.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:31 smithi204 ceph-mon[34659]: from='osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:07:32.019 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:07:32.045 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi137:vg_nvme/lv_4 2024-09-21T06:07:32.305 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:07:32.590 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0[48021]: 2024-09-21T06:07:32.527+0000 7f60ceda9640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T06:07:32.590 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0[48021]: 2024-09-21T06:07:32.532+0000 7f60caba7640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:07:32.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: from='osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T06:07:32.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T06:07:32.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: from='osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]} : dispatch 2024-09-21T06:07:32.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:32.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:32.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: Detected new or changed devices on smithi003 2024-09-21T06:07:32.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:32.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:32.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:32.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi003 to 17092M 2024-09-21T06:07:32.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:32.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:32.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:32.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:32.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: from='osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T06:07:32.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T06:07:32.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: from='osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]} : dispatch 2024-09-21T06:07:32.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:32.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:32.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: Detected new or changed devices on smithi003 2024-09-21T06:07:32.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:32.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:32.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:32.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi003 to 17092M 2024-09-21T06:07:32.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:32.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:32.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:32.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:32.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.853+0000 7f1851071640 1 -- 172.21.15.137:0/1150437777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c104840 msgr2=0x7f184c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:32.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.853+0000 7f1851071640 1 --2- 172.21.15.137:0/1150437777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c104840 0x7f184c104c50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f183c009900 tx=0x7f183c02a410 comp rx=0 tx=0).stop 2024-09-21T06:07:32.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.853+0000 7f1851071640 1 -- 172.21.15.137:0/1150437777 shutdown_connections 2024-09-21T06:07:32.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.853+0000 7f1851071640 1 --2- 172.21.15.137:0/1150437777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f184c10ddf0 0x7f184c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.853+0000 7f1851071640 1 --2- 172.21.15.137:0/1150437777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f184c105270 0x7f184c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.853+0000 7f1851071640 1 --2- 172.21.15.137:0/1150437777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c104840 0x7f184c104c50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.853+0000 7f1851071640 1 -- 172.21.15.137:0/1150437777 >> 172.21.15.137:0/1150437777 conn(0x7f184c079730 msgr2=0x7f184c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:32.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.853+0000 7f1851071640 1 -- 172.21.15.137:0/1150437777 shutdown_connections 2024-09-21T06:07:32.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.853+0000 7f1851071640 1 -- 172.21.15.137:0/1150437777 wait complete. 2024-09-21T06:07:32.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.854+0000 7f1851071640 1 Processor -- start 2024-09-21T06:07:32.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.854+0000 7f1851071640 1 -- start start 2024-09-21T06:07:32.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f1851071640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f184c104840 0x7f184c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:32.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f1851071640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c105270 0x7f184c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:32.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f1851071640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f184c10ddf0 0x7f184c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f1851071640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f184c113ac0 con 0x7f184c104840 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f1851071640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f184c113940 con 0x7f184c105270 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f1851071640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f184c113c40 con 0x7f184c10ddf0 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f184a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c105270 0x7f184c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f184a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c105270 0x7f184c19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34122/0 (socket says 172.21.15.137:34122) 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f184a59c640 1 -- 172.21.15.137:0/4007602912 learned_addr learned my addr 172.21.15.137:0/4007602912 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f184ad9d640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f184c104840 0x7f184c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f184b59e640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f184c10ddf0 0x7f184c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f184a59c640 1 -- 172.21.15.137:0/4007602912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f184c10ddf0 msgr2=0x7f184c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f184a59c640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f184c10ddf0 0x7f184c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f184a59c640 1 -- 172.21.15.137:0/4007602912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f184c104840 msgr2=0x7f184c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f184a59c640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f184c104840 0x7f184c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.855+0000 7f184a59c640 1 -- 172.21.15.137:0/4007602912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c1a0420 con 0x7f184c105270 2024-09-21T06:07:32.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.856+0000 7f184a59c640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c105270 0x7f184c19aa50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f184000d740 tx=0x7f184000dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:32.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.856+0000 7f1833fff640 1 -- 172.21.15.137:0/4007602912 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840017070 con 0x7f184c105270 2024-09-21T06:07:32.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.856+0000 7f1851071640 1 -- 172.21.15.137:0/4007602912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c1a0710 con 0x7f184c105270 2024-09-21T06:07:32.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.856+0000 7f1833fff640 1 -- 172.21.15.137:0/4007602912 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f184000de50 con 0x7f184c105270 2024-09-21T06:07:32.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.856+0000 7f1833fff640 1 -- 172.21.15.137:0/4007602912 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184000f620 con 0x7f184c105270 2024-09-21T06:07:32.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.856+0000 7f1851071640 1 -- 172.21.15.137:0/4007602912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1a6e70 con 0x7f184c105270 2024-09-21T06:07:32.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.857+0000 7f1851071640 1 -- 172.21.15.137:0/4007602912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184c104c50 con 0x7f184c105270 2024-09-21T06:07:32.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.858+0000 7f1833fff640 1 -- 172.21.15.137:0/4007602912 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f184001a070 con 0x7f184c105270 2024-09-21T06:07:32.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.859+0000 7f1833fff640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18340b64d0 0x7f18340b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:32.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.859+0000 7f1833fff640 1 -- 172.21.15.137:0/4007602912 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1702+0+0 (secure 0 0 0) 0x7f18400e51b0 con 0x7f184c105270 2024-09-21T06:07:32.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.859+0000 7f184ad9d640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18340b64d0 0x7f18340b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:32.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.860+0000 7f184ad9d640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18340b64d0 0x7f18340b89e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f183c031440 tx=0x7f183c005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:32.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:32.864+0000 7f1833fff640 1 -- 172.21.15.137:0/4007602912 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f18400b18d0 con 0x7f184c105270 2024-09-21T06:07:32.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: from='osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T06:07:32.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T06:07:32.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: from='osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]} : dispatch 2024-09-21T06:07:32.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:32.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:32.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: Detected new or changed devices on smithi003 2024-09-21T06:07:32.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:32.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:32.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:32.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi003 to 17092M 2024-09-21T06:07:32.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:32.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:32.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:32.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:33.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:33.080+0000 7f1851071640 1 -- 172.21.15.137:0/4007602912 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f184c101d50 con 0x7f18340b64d0 2024-09-21T06:07:33.603 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:33 smithi137 ceph-mon[34464]: from='osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-21T06:07:33.603 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:33 smithi137 ceph-mon[34464]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T06:07:33.603 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:33 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:33.603 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:33 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:33.603 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:33 smithi137 ceph-mon[34464]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:33.603 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:33 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:33.603 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:33 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:33.603 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:33 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:33.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:33 smithi003 ceph-mon[31227]: from='osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-21T06:07:33.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:33 smithi003 ceph-mon[31227]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T06:07:33.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:33 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:33.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:33 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:33.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:33 smithi003 ceph-mon[31227]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:33.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:33 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:33.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:33 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:33.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:33 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:33 smithi204 ceph-mon[34659]: from='osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-21T06:07:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:33 smithi204 ceph-mon[34659]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T06:07:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:33 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:33.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:33 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:33.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:33 smithi204 ceph-mon[34659]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:33.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:33 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:33.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:33 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:33.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:33 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:34.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:34 smithi137 ceph-mon[34464]: purged_snaps scrub starts 2024-09-21T06:07:34.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:34 smithi137 ceph-mon[34464]: purged_snaps scrub ok 2024-09-21T06:07:34.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:34.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:34 smithi137 ceph-mon[34464]: osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328] boot 2024-09-21T06:07:34.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:34 smithi137 ceph-mon[34464]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T06:07:34.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:34.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:34 smithi137 ceph-mon[34464]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:34.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:34 smithi003 ceph-mon[31227]: purged_snaps scrub starts 2024-09-21T06:07:34.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:34 smithi003 ceph-mon[31227]: purged_snaps scrub ok 2024-09-21T06:07:34.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:34.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:34 smithi003 ceph-mon[31227]: osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328] boot 2024-09-21T06:07:34.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:34 smithi003 ceph-mon[31227]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T06:07:34.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:34.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:34 smithi003 ceph-mon[31227]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:34.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:34 smithi204 ceph-mon[34659]: purged_snaps scrub starts 2024-09-21T06:07:34.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:34 smithi204 ceph-mon[34659]: purged_snaps scrub ok 2024-09-21T06:07:34.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:34.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:34 smithi204 ceph-mon[34659]: osd.0 [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328] boot 2024-09-21T06:07:34.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:34 smithi204 ceph-mon[34659]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T06:07:34.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:07:34.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:34 smithi204 ceph-mon[34659]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:07:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:35 smithi003 ceph-mon[31227]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T06:07:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:35 smithi003 ceph-mon[31227]: from='client.? 172.21.15.137:0/2813872889' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "55696a37-34c1-482a-8561-7d08f0aa29fa"} : dispatch 2024-09-21T06:07:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:35 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "55696a37-34c1-482a-8561-7d08f0aa29fa"} : dispatch 2024-09-21T06:07:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:35 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55696a37-34c1-482a-8561-7d08f0aa29fa"}]': finished 2024-09-21T06:07:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:35 smithi003 ceph-mon[31227]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T06:07:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:07:35.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:35 smithi137 ceph-mon[34464]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T06:07:35.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:35 smithi137 ceph-mon[34464]: from='client.? 172.21.15.137:0/2813872889' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "55696a37-34c1-482a-8561-7d08f0aa29fa"} : dispatch 2024-09-21T06:07:35.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:35 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "55696a37-34c1-482a-8561-7d08f0aa29fa"} : dispatch 2024-09-21T06:07:35.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:35 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55696a37-34c1-482a-8561-7d08f0aa29fa"}]': finished 2024-09-21T06:07:35.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:35 smithi137 ceph-mon[34464]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T06:07:35.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:07:35.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:35 smithi204 ceph-mon[34659]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T06:07:35.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:35 smithi204 ceph-mon[34659]: from='client.? 172.21.15.137:0/2813872889' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "55696a37-34c1-482a-8561-7d08f0aa29fa"} : dispatch 2024-09-21T06:07:35.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:35 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "55696a37-34c1-482a-8561-7d08f0aa29fa"} : dispatch 2024-09-21T06:07:35.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:35 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55696a37-34c1-482a-8561-7d08f0aa29fa"}]': finished 2024-09-21T06:07:35.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:35 smithi204 ceph-mon[34659]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T06:07:35.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:07:36.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:36 smithi137 ceph-mon[34464]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:36.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:36 smithi137 ceph-mon[34464]: from='client.? 172.21.15.137:0/2247905467' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:07:36.805 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:36 smithi204 ceph-mon[34659]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:36.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:36 smithi204 ceph-mon[34659]: from='client.? 172.21.15.137:0/2247905467' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:07:36.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:36 smithi003 ceph-mon[31227]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:36.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:36 smithi003 ceph-mon[31227]: from='client.? 172.21.15.137:0/2247905467' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:07:38.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:38 smithi003 ceph-mon[31227]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:38.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:38 smithi137 ceph-mon[34464]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:38.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:38 smithi204 ceph-mon[34659]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:40.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:40 smithi003 ceph-mon[31227]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:40.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:40 smithi137 ceph-mon[34464]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:40.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:40 smithi204 ceph-mon[34659]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:42.706 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:42 smithi137 ceph-mon[34464]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:42.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:42 smithi003 ceph-mon[31227]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:42 smithi204 ceph-mon[34659]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:43 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T06:07:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:43 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:43.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:43 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T06:07:43.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:43 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:43.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:43 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T06:07:43.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:43 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:44.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:44 smithi003 ceph-mon[31227]: Deploying daemon osd.1 on smithi137 2024-09-21T06:07:44.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:44 smithi003 ceph-mon[31227]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:44.851 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:44 smithi137 ceph-mon[34464]: Deploying daemon osd.1 on smithi137 2024-09-21T06:07:44.851 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:44 smithi137 ceph-mon[34464]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:44.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:44 smithi204 ceph-mon[34659]: Deploying daemon osd.1 on smithi137 2024-09-21T06:07:44.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:44 smithi204 ceph-mon[34659]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:46.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:46 smithi137 ceph-mon[34464]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:46.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:46 smithi003 ceph-mon[31227]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:46.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:46 smithi204 ceph-mon[34659]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:48.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:48 smithi003 ceph-mon[31227]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:48.853 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:48 smithi137 ceph-mon[34464]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:48.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:48 smithi204 ceph-mon[34659]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:50.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:50 smithi003 ceph-mon[31227]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:50.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:50.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:50.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:50.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:50 smithi137 ceph-mon[34464]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:50.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:50.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:50.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:50.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:50 smithi204 ceph-mon[34659]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:50.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:50.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:50.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:52.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:52 smithi003 ceph-mon[31227]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:52.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:52.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:52.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:52.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:52.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:52.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:52 smithi137 ceph-mon[34464]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:52.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:52.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:52.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:52.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:52.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:52.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:52 smithi204 ceph-mon[34659]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:52.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:52.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:52.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:52.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:52.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:53.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:53.673+0000 7f1833fff640 1 -- 172.21.15.137:0/4007602912 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f184c101d50 con 0x7f18340b64d0 2024-09-21T06:07:53.675 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 1 on host 'smithi137' 2024-09-21T06:07:53.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:53.675+0000 7f1851071640 1 -- 172.21.15.137:0/4007602912 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18340b64d0 msgr2=0x7f18340b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:53.675+0000 7f1851071640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18340b64d0 0x7f18340b89e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f183c031440 tx=0x7f183c005a20 comp rx=0 tx=0).stop 2024-09-21T06:07:53.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:53.676+0000 7f1851071640 1 -- 172.21.15.137:0/4007602912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c105270 msgr2=0x7f184c19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:53.676+0000 7f1851071640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c105270 0x7f184c19aa50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f184000d740 tx=0x7f184000dc00 comp rx=0 tx=0).stop 2024-09-21T06:07:53.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:53.676+0000 7f1851071640 1 -- 172.21.15.137:0/4007602912 shutdown_connections 2024-09-21T06:07:53.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:53.676+0000 7f1851071640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18340b64d0 0x7f18340b89e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:53.676+0000 7f1851071640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f184c10ddf0 0x7f184c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:53.676+0000 7f1851071640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c105270 0x7f184c19aa50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:53.676+0000 7f1851071640 1 --2- 172.21.15.137:0/4007602912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f184c104840 0x7f184c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:53.676+0000 7f1851071640 1 -- 172.21.15.137:0/4007602912 >> 172.21.15.137:0/4007602912 conn(0x7f184c079730 msgr2=0x7f184c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:53.676+0000 7f1851071640 1 -- 172.21.15.137:0/4007602912 shutdown_connections 2024-09-21T06:07:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:07:53.676+0000 7f1851071640 1 -- 172.21.15.137:0/4007602912 wait complete. 2024-09-21T06:07:54.333 DEBUG:teuthology.orchestra.run.smithi137:osd.1> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T06:07:54.335 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2024-09-21T06:07:54.335 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T06:07:54.594 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:07:54.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:54 smithi204 ceph-mon[34659]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:54.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:54 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:54.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:54 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:54.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:54 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:54.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:54 smithi003 ceph-mon[31227]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:54.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:54 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:54.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:54 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:54.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:54 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:54.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:54 smithi137 ceph-mon[34464]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:54.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:54 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:54.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:54 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:54.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:54 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:56.145 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:07:56 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1[39181]: 2024-09-21T06:07:56.043+0000 7f5d5c939640 -1 osd.1 0 log_to_monitors true 2024-09-21T06:07:56.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:56 smithi204 ceph-mon[34659]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:56.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:56 smithi204 ceph-mon[34659]: from='osd.1 [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:07:56.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:56 smithi204 ceph-mon[34659]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:07:56.810 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:56 smithi137 ceph-mon[34464]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:56.810 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:56 smithi137 ceph-mon[34464]: from='osd.1 [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:07:56.810 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:56 smithi137 ceph-mon[34464]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:07:56.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:56 smithi003 ceph-mon[31227]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:56.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:56 smithi003 ceph-mon[31227]: from='osd.1 [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:07:56.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:56 smithi003 ceph-mon[31227]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:07:57.147 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:07:57.177 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2024-09-21T06:07:57.428 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:07:57.781 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T06:07:57.781 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T06:07:57.781 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: from='osd.1 [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:07:57.781 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:07:57.781 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:07:57.781 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: Detected new or changed devices on smithi137 2024-09-21T06:07:57.781 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:57.781 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:57.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:57.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi137 to 17088M 2024-09-21T06:07:57.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:57.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:57.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:57.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:57 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:57.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.875+0000 7fc2d7577640 1 -- 172.21.15.204:0/1572785287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d0105270 msgr2=0x7fc2d010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.875+0000 7fc2d7577640 1 --2- 172.21.15.204:0/1572785287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d0105270 0x7fc2d010d8b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009900 tx=0x7fc2cc02a410 comp rx=0 tx=0).stop 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.875+0000 7fc2d7577640 1 -- 172.21.15.204:0/1572785287 shutdown_connections 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.875+0000 7fc2d7577640 1 --2- 172.21.15.204:0/1572785287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d0105270 0x7fc2d010d8b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.875+0000 7fc2d7577640 1 --2- 172.21.15.204:0/1572785287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2d010ddf0 0x7fc2d0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.875+0000 7fc2d7577640 1 --2- 172.21.15.204:0/1572785287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2d0104840 0x7fc2d0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.875+0000 7fc2d7577640 1 -- 172.21.15.204:0/1572785287 >> 172.21.15.204:0/1572785287 conn(0x7fc2d0079730 msgr2=0x7fc2d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.875+0000 7fc2d7577640 1 -- 172.21.15.204:0/1572785287 shutdown_connections 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.875+0000 7fc2d7577640 1 -- 172.21.15.204:0/1572785287 wait complete. 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.876+0000 7fc2d7577640 1 Processor -- start 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.876+0000 7fc2d7577640 1 -- start start 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.876+0000 7fc2d7577640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2d0104840 0x7fc2d019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.876+0000 7fc2d7577640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2d0105270 0x7fc2d019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.876+0000 7fc2d7577640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d010ddf0 0x7fc2d019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d7577640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d0113a70 con 0x7fc2d0105270 2024-09-21T06:07:57.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d7577640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d01138f0 con 0x7fc2d0104840 2024-09-21T06:07:57.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d7577640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d0113bf0 con 0x7fc2d010ddf0 2024-09-21T06:07:57.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d5b14640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d010ddf0 0x7fc2d019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d5b14640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d010ddf0 0x7fc2d019fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:37738/0 (socket says 172.21.15.204:37738) 2024-09-21T06:07:57.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d5b14640 1 -- 172.21.15.204:0/4233097409 learned_addr learned my addr 172.21.15.204:0/4233097409 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-21T06:07:57.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d5313640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2d0104840 0x7fc2d019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d4b12640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2d0105270 0x7fc2d019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d5b14640 1 -- 172.21.15.204:0/4233097409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2d0104840 msgr2=0x7fc2d019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d5b14640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2d0104840 0x7fc2d019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d5b14640 1 -- 172.21.15.204:0/4233097409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2d0105270 msgr2=0x7fc2d019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d5b14640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2d0105270 0x7fc2d019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d5b14640 1 -- 172.21.15.204:0/4233097409 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d01a0410 con 0x7fc2d010ddf0 2024-09-21T06:07:57.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d5b14640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d010ddf0 0x7fc2d019fce0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800eaa0 tx=0x7fc2c800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:57.883 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2c67fc640 1 -- 172.21.15.204:0/4233097409 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c800cdb0 con 0x7fc2d010ddf0 2024-09-21T06:07:57.883 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2d7577640 1 -- 172.21.15.204:0/4233097409 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d01a0700 con 0x7fc2d010ddf0 2024-09-21T06:07:57.883 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2c67fc640 1 -- 172.21.15.204:0/4233097409 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c8005020 con 0x7fc2d010ddf0 2024-09-21T06:07:57.883 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.877+0000 7fc2c67fc640 1 -- 172.21.15.204:0/4233097409 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80116d0 con 0x7fc2d010ddf0 2024-09-21T06:07:57.883 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.878+0000 7fc2d7577640 1 -- 172.21.15.204:0/4233097409 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d01a6e60 con 0x7fc2d010ddf0 2024-09-21T06:07:57.883 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.879+0000 7fc2d7577640 1 -- 172.21.15.204:0/4233097409 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d0104c50 con 0x7fc2d010ddf0 2024-09-21T06:07:57.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.880+0000 7fc2c67fc640 1 -- 172.21.15.204:0/4233097409 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7fc2c80051d0 con 0x7fc2d010ddf0 2024-09-21T06:07:57.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.881+0000 7fc2c67fc640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc2bc0b64d0 0x7fc2bc0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.882+0000 7fc2c67fc640 1 -- 172.21.15.204:0/4233097409 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2244+0+0 (secure 0 0 0) 0x7fc2c80e5fa0 con 0x7fc2d010ddf0 2024-09-21T06:07:57.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.882+0000 7fc2d5313640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc2bc0b64d0 0x7fc2bc0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.882+0000 7fc2d5313640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc2bc0b64d0 0x7fc2bc0b89e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00058c0 tx=0x7fc2c000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:57.887 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:57.885+0000 7fc2c67fc640 1 -- 172.21.15.204:0/4233097409 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc2c8016030 con 0x7fc2d010ddf0 2024-09-21T06:07:57.895 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1[39181]: 2024-09-21T06:07:57.600+0000 7f5d588bb640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T06:07:57.895 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1[39181]: 2024-09-21T06:07:57.604+0000 7f5d546b9640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:07:57.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T06:07:57.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T06:07:57.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: from='osd.1 [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:07:57.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:07:57.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:07:57.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: Detected new or changed devices on smithi137 2024-09-21T06:07:57.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:57.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:57.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:57.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi137 to 17088M 2024-09-21T06:07:57.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:57.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:57.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:57.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:57 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:58.079 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:07:58.077+0000 7fc2d7577640 1 -- 172.21.15.204:0/4233097409 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc2d0063800 con 0x7fc2bc0b64d0 2024-09-21T06:07:58.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T06:07:58.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T06:07:58.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: from='osd.1 [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:07:58.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:07:58.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:07:58.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: Detected new or changed devices on smithi137 2024-09-21T06:07:58.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:58.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:58.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:58.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi137 to 17088M 2024-09-21T06:07:58.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:58.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:58.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:58.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:57 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:07:58.698 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:58 smithi204 ceph-mon[34659]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:58.698 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:58 smithi204 ceph-mon[34659]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-21T06:07:58.698 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:58 smithi204 ceph-mon[34659]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T06:07:58.698 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:58 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:07:58.698 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:58 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:07:58.698 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:58 smithi204 ceph-mon[34659]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:58.698 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:58 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:58.698 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:58 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:58.698 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:58 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:58.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:58 smithi137 ceph-mon[34464]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:58.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:58 smithi137 ceph-mon[34464]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-21T06:07:58.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:58 smithi137 ceph-mon[34464]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T06:07:58.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:58 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:07:58.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:58 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:07:58.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:58 smithi137 ceph-mon[34464]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:58.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:58 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:58.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:58 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:58.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:58 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:59.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:58 smithi003 ceph-mon[31227]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:07:59.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:58 smithi003 ceph-mon[31227]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-21T06:07:59.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:58 smithi003 ceph-mon[31227]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T06:07:59.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:58 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:07:59.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:58 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:07:59.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:58 smithi003 ceph-mon[31227]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:59.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:58 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:59.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:58 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:59.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:58 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:59.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:59 smithi204 ceph-mon[34659]: purged_snaps scrub starts 2024-09-21T06:07:59.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:59 smithi204 ceph-mon[34659]: purged_snaps scrub ok 2024-09-21T06:07:59.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:59 smithi204 ceph-mon[34659]: osd.1 [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115] boot 2024-09-21T06:07:59.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:59 smithi204 ceph-mon[34659]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T06:07:59.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:07:59 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:07:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:59 smithi137 ceph-mon[34464]: purged_snaps scrub starts 2024-09-21T06:07:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:59 smithi137 ceph-mon[34464]: purged_snaps scrub ok 2024-09-21T06:07:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:59 smithi137 ceph-mon[34464]: osd.1 [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115] boot 2024-09-21T06:07:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:59 smithi137 ceph-mon[34464]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T06:07:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:07:59 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:08:00.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:59 smithi003 ceph-mon[31227]: purged_snaps scrub starts 2024-09-21T06:08:00.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:59 smithi003 ceph-mon[31227]: purged_snaps scrub ok 2024-09-21T06:08:00.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:59 smithi003 ceph-mon[31227]: osd.1 [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115] boot 2024-09-21T06:08:00.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:59 smithi003 ceph-mon[31227]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T06:08:00.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:07:59 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:08:00.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:00 smithi137 ceph-mon[34464]: pgmap v50: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:08:00.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:00 smithi137 ceph-mon[34464]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T06:08:00.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:00 smithi137 ceph-mon[34464]: from='client.? 172.21.15.204:0/1086868736' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8a81d34b-464b-46a2-8ef2-5223b5947004"} : dispatch 2024-09-21T06:08:00.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:00 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8a81d34b-464b-46a2-8ef2-5223b5947004"} : dispatch 2024-09-21T06:08:00.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:00 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a81d34b-464b-46a2-8ef2-5223b5947004"}]': finished 2024-09-21T06:08:00.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:00 smithi137 ceph-mon[34464]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T06:08:00.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:00 smithi204 ceph-mon[34659]: pgmap v50: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:08:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:00 smithi204 ceph-mon[34659]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T06:08:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:00 smithi204 ceph-mon[34659]: from='client.? 172.21.15.204:0/1086868736' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8a81d34b-464b-46a2-8ef2-5223b5947004"} : dispatch 2024-09-21T06:08:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:00 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8a81d34b-464b-46a2-8ef2-5223b5947004"} : dispatch 2024-09-21T06:08:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:00 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a81d34b-464b-46a2-8ef2-5223b5947004"}]': finished 2024-09-21T06:08:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:00 smithi204 ceph-mon[34659]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T06:08:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:01.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:00 smithi003 ceph-mon[31227]: pgmap v50: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:08:01.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:00 smithi003 ceph-mon[31227]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T06:08:01.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:00 smithi003 ceph-mon[31227]: from='client.? 172.21.15.204:0/1086868736' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8a81d34b-464b-46a2-8ef2-5223b5947004"} : dispatch 2024-09-21T06:08:01.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:00 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8a81d34b-464b-46a2-8ef2-5223b5947004"} : dispatch 2024-09-21T06:08:01.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:00 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a81d34b-464b-46a2-8ef2-5223b5947004"}]': finished 2024-09-21T06:08:01.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:00 smithi003 ceph-mon[31227]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T06:08:01.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:01.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:01 smithi137 ceph-mon[34464]: from='client.? 172.21.15.204:0/1801963663' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:01.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:01 smithi204 ceph-mon[34659]: from='client.? 172.21.15.204:0/1801963663' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:02.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:01 smithi003 ceph-mon[31227]: from='client.? 172.21.15.204:0/1801963663' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:02.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:02 smithi137 ceph-mon[34464]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:02 smithi204 ceph-mon[34659]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:03.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:02 smithi003 ceph-mon[31227]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:04.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:04 smithi137 ceph-mon[34464]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:04.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:04 smithi204 ceph-mon[34659]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:05.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:04 smithi003 ceph-mon[31227]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:06.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:06 smithi137 ceph-mon[34464]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:06.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:06 smithi204 ceph-mon[34659]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:07.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:06 smithi003 ceph-mon[31227]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:08.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:08 smithi137 ceph-mon[34464]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:08.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:08 smithi204 ceph-mon[34659]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:09.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:08 smithi003 ceph-mon[31227]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:09 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T06:08:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:09 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:09 smithi204 ceph-mon[34659]: Deploying daemon osd.2 on smithi204 2024-09-21T06:08:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:09 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T06:08:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:09 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:09 smithi137 ceph-mon[34464]: Deploying daemon osd.2 on smithi204 2024-09-21T06:08:10.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:09 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T06:08:10.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:09 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:10.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:09 smithi003 ceph-mon[31227]: Deploying daemon osd.2 on smithi204 2024-09-21T06:08:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:10 smithi137 ceph-mon[34464]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:10.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:10 smithi204 ceph-mon[34659]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:11.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:10 smithi003 ceph-mon[31227]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:12.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:12 smithi137 ceph-mon[34464]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:12 smithi204 ceph-mon[34659]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:13.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:12 smithi003 ceph-mon[31227]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:14.892 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:14 smithi003 ceph-mon[31227]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:14.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:14 smithi137 ceph-mon[34464]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:14.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:14 smithi204 ceph-mon[34659]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:15.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:15 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:15.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:15 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:15.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:15 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:15.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:15 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:15.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:15 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:15.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:15 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:16.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:15 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:16.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:15 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:16.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:15 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:16.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:16 smithi204 ceph-mon[34659]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:17.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:16 smithi003 ceph-mon[31227]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:17.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:16 smithi137 ceph-mon[34464]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:18 smithi137 ceph-mon[34464]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:18 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:18 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:18 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:18 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:18 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:18.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:18 smithi204 ceph-mon[34659]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:18.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:18 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:18.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:18 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:18.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:18 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:18.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:18 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:18.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:18 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:18 smithi003 ceph-mon[31227]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:18 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:18 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:18 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:18 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:18 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:19.845 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:08:19.842+0000 7fc2c67fc640 1 -- 172.21.15.204:0/4233097409 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc2d0063800 con 0x7fc2bc0b64d0 2024-09-21T06:08:19.845 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2024-09-21T06:08:19.847 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:08:19.845+0000 7fc2d7577640 1 -- 172.21.15.204:0/4233097409 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc2bc0b64d0 msgr2=0x7fc2bc0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:19.847 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:08:19.845+0000 7fc2d7577640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc2bc0b64d0 0x7fc2bc0b89e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00058c0 tx=0x7fc2c000a2d0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.847 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:08:19.845+0000 7fc2d7577640 1 -- 172.21.15.204:0/4233097409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d010ddf0 msgr2=0x7fc2d019fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:19.847 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:08:19.845+0000 7fc2d7577640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d010ddf0 0x7fc2d019fce0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800eaa0 tx=0x7fc2c800ef60 comp rx=0 tx=0).stop 2024-09-21T06:08:19.847 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:08:19.845+0000 7fc2d7577640 1 -- 172.21.15.204:0/4233097409 shutdown_connections 2024-09-21T06:08:19.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:08:19.845+0000 7fc2d7577640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc2bc0b64d0 0x7fc2bc0b89e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:08:19.845+0000 7fc2d7577640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d010ddf0 0x7fc2d019fce0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:08:19.845+0000 7fc2d7577640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2d0105270 0x7fc2d019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:08:19.845+0000 7fc2d7577640 1 --2- 172.21.15.204:0/4233097409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2d0104840 0x7fc2d019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:19.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:08:19.845+0000 7fc2d7577640 1 -- 172.21.15.204:0/4233097409 >> 172.21.15.204:0/4233097409 conn(0x7fc2d0079730 msgr2=0x7fc2d0100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:19.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:08:19.845+0000 7fc2d7577640 1 -- 172.21.15.204:0/4233097409 shutdown_connections 2024-09-21T06:08:19.849 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:08:19.845+0000 7fc2d7577640 1 -- 172.21.15.204:0/4233097409 wait complete. 2024-09-21T06:08:20.524 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T06:08:20.527 INFO:tasks.cephadm:Deploying osd.3 on smithi003 with /dev/vg_nvme/lv_3... 2024-09-21T06:08:20.527 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T06:08:20.798 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:20.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:20 smithi003 ceph-mon[31227]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:20.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:20 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:20.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:20 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:20.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:20 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:20.855 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:20 smithi204 ceph-mon[34659]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:20.855 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:20 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:20.855 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:20 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:20.855 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:20 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:20 smithi137 ceph-mon[34464]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:20 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:20 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:20 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:21.646 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:08:21 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2[39755]: 2024-09-21T06:08:21.315+0000 7f1746b64640 -1 osd.2 0 log_to_monitors true 2024-09-21T06:08:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:21 smithi204 ceph-mon[34659]: from='osd.2 [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:08:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:21 smithi204 ceph-mon[34659]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:08:21.955 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:21 smithi003 ceph-mon[31227]: from='osd.2 [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:08:21.955 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:21 smithi003 ceph-mon[31227]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:08:22.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:21 smithi137 ceph-mon[34464]: from='osd.2 [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:08:22.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:21 smithi137 ceph-mon[34464]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:08:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:22 smithi003 ceph-mon[31227]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:22 smithi003 ceph-mon[31227]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T06:08:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:22 smithi003 ceph-mon[31227]: from='osd.2 [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:08:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:22 smithi003 ceph-mon[31227]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T06:08:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:22 smithi003 ceph-mon[31227]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:08:23.078 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:22 smithi204 ceph-mon[34659]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:23.078 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:22 smithi204 ceph-mon[34659]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T06:08:23.079 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:22 smithi204 ceph-mon[34659]: from='osd.2 [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:08:23.079 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:22 smithi204 ceph-mon[34659]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T06:08:23.079 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:23.079 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:22 smithi204 ceph-mon[34659]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:08:23.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:22 smithi137 ceph-mon[34464]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:23.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:22 smithi137 ceph-mon[34464]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T06:08:23.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:22 smithi137 ceph-mon[34464]: from='osd.2 [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:08:23.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:22 smithi137 ceph-mon[34464]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T06:08:23.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:23.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:22 smithi137 ceph-mon[34464]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:08:23.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2[39755]: 2024-09-21T06:08:23.084+0000 7f1742ae6640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T06:08:23.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2[39755]: 2024-09-21T06:08:23.091+0000 7f173a8dc640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:08:23.420 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:08:23.453 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi003:vg_nvme/lv_3 2024-09-21T06:08:23.717 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:23.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:23 smithi003 ceph-mon[31227]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-21T06:08:23.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:23 smithi003 ceph-mon[31227]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T06:08:23.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:23.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:23.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:23 smithi003 ceph-mon[31227]: from='osd.2 ' entity='osd.2' 2024-09-21T06:08:23.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:23.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:23.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:23.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:23.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:23.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-mon[34659]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-21T06:08:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-mon[34659]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T06:08:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-mon[34659]: from='osd.2 ' entity='osd.2' 2024-09-21T06:08:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:24.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:23 smithi137 ceph-mon[34464]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-21T06:08:24.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:23 smithi137 ceph-mon[34464]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T06:08:24.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:24.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:24.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:23 smithi137 ceph-mon[34464]: from='osd.2 ' entity='osd.2' 2024-09-21T06:08:24.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:24.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:24.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:24.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:24.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:24.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:24.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:24.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.189+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3555148041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41094a0 msgr2=0x7f2ae41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.189+0000 7f2ae91f3640 1 --2- 172.21.15.3:0/3555148041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41094a0 0x7f2ae41098b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009900 tx=0x7f2acc02a410 comp rx=0 tx=0).stop 2024-09-21T06:08:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.189+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3555148041 shutdown_connections 2024-09-21T06:08:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.189+0000 7f2ae91f3640 1 --2- 172.21.15.3:0/3555148041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4109df0 0x7f2ae4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.189+0000 7f2ae91f3640 1 --2- 172.21.15.3:0/3555148041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41094a0 0x7f2ae41098b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.189+0000 7f2ae91f3640 1 --2- 172.21.15.3:0/3555148041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae4103b60 0x7f2ae4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.189+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3555148041 >> 172.21.15.3:0/3555148041 conn(0x7f2ae407b850 msgr2=0x7f2ae407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.189+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3555148041 shutdown_connections 2024-09-21T06:08:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.189+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3555148041 wait complete. 2024-09-21T06:08:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.190+0000 7f2ae91f3640 1 Processor -- start 2024-09-21T06:08:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.190+0000 7f2ae91f3640 1 -- start start 2024-09-21T06:08:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.190+0000 7f2ae91f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4103b60 0x7f2ae4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.190+0000 7f2ae91f3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41094a0 0x7f2ae4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.190+0000 7f2ae91f3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae4109df0 0x7f2ae4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.190+0000 7f2ae91f3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae4113ab0 con 0x7f2ae41094a0 2024-09-21T06:08:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.190+0000 7f2ae91f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae4113930 con 0x7f2ae4103b60 2024-09-21T06:08:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.190+0000 7f2ae91f3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae4113c30 con 0x7f2ae4109df0 2024-09-21T06:08:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.190+0000 7f2ae259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41094a0 0x7f2ae4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.190+0000 7f2ae259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41094a0 0x7f2ae4075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42760/0 (socket says 172.21.15.3:42760) 2024-09-21T06:08:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.190+0000 7f2ae259c640 1 -- 172.21.15.3:0/3563078361 learned_addr learned my addr 172.21.15.3:0/3563078361 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:08:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2ae2d9d640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4103b60 0x7f2ae4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2ae359e640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae4109df0 0x7f2ae4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2ae259c640 1 -- 172.21.15.3:0/3563078361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae4109df0 msgr2=0x7f2ae4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2ae259c640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae4109df0 0x7f2ae4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2ae259c640 1 -- 172.21.15.3:0/3563078361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4103b60 msgr2=0x7f2ae4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2ae259c640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4103b60 0x7f2ae4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2ae259c640 1 -- 172.21.15.3:0/3563078361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae41a8840 con 0x7f2ae41094a0 2024-09-21T06:08:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2ae359e640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae4109df0 0x7f2ae4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2ae259c640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41094a0 0x7f2ae4075c50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2acc006010 tx=0x7f2acc004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2adbfff640 1 -- 172.21.15.3:0/3563078361 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc03f030 con 0x7f2ae41094a0 2024-09-21T06:08:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3563078361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae41a8ad0 con 0x7f2ae41094a0 2024-09-21T06:08:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2adbfff640 1 -- 172.21.15.3:0/3563078361 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2acc003aa0 con 0x7f2ae41094a0 2024-09-21T06:08:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2adbfff640 1 -- 172.21.15.3:0/3563078361 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc035070 con 0x7f2ae41094a0 2024-09-21T06:08:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.191+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3563078361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae41a8fb0 con 0x7f2ae41094a0 2024-09-21T06:08:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.192+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3563078361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae4103f70 con 0x7f2ae41094a0 2024-09-21T06:08:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.194+0000 7f2adbfff640 1 -- 172.21.15.3:0/3563078361 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 154472+0+0 (secure 0 0 0) 0x7f2acc042030 con 0x7f2ae41094a0 2024-09-21T06:08:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.195+0000 7f2adbfff640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2ad00b64d0 0x7f2ad00b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.195+0000 7f2ae2d9d640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2ad00b64d0 0x7f2ad00b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.195+0000 7f2adbfff640 1 -- 172.21.15.3:0/3563078361 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3066+0+0 (secure 0 0 0) 0x7f2acc1066b0 con 0x7f2ae41094a0 2024-09-21T06:08:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.196+0000 7f2ae2d9d640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2ad00b64d0 0x7f2ad00b89e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4077160 tx=0x7f2adc009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.198+0000 7f2adbfff640 1 -- 172.21.15.3:0/3563078361 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2acc0ce4a0 con 0x7f2ae41094a0 2024-09-21T06:08:24.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:24.400+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3563078361 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2ae4063800 con 0x7f2ad00b64d0 2024-09-21T06:08:25.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:25 smithi137 ceph-mon[34464]: purged_snaps scrub starts 2024-09-21T06:08:25.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:25 smithi137 ceph-mon[34464]: purged_snaps scrub ok 2024-09-21T06:08:25.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:25 smithi137 ceph-mon[34464]: Detected new or changed devices on smithi204 2024-09-21T06:08:25.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:25 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi204 to 17088M 2024-09-21T06:08:25.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:25 smithi137 ceph-mon[34464]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:25.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:25 smithi137 ceph-mon[34464]: osd.2 [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077] boot 2024-09-21T06:08:25.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:25 smithi137 ceph-mon[34464]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T06:08:25.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:25.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:08:25.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:08:25.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:25.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:25 smithi204 ceph-mon[34659]: purged_snaps scrub starts 2024-09-21T06:08:25.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:25 smithi204 ceph-mon[34659]: purged_snaps scrub ok 2024-09-21T06:08:25.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:25 smithi204 ceph-mon[34659]: Detected new or changed devices on smithi204 2024-09-21T06:08:25.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:25 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi204 to 17088M 2024-09-21T06:08:25.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:25 smithi204 ceph-mon[34659]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:25.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:25 smithi204 ceph-mon[34659]: osd.2 [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077] boot 2024-09-21T06:08:25.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:25 smithi204 ceph-mon[34659]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T06:08:25.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:25.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:08:25.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:08:25.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:25.476 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:25 smithi003 ceph-mon[31227]: purged_snaps scrub starts 2024-09-21T06:08:25.476 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:25 smithi003 ceph-mon[31227]: purged_snaps scrub ok 2024-09-21T06:08:25.476 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:25 smithi003 ceph-mon[31227]: Detected new or changed devices on smithi204 2024-09-21T06:08:25.476 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:25 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi204 to 17088M 2024-09-21T06:08:25.476 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:25 smithi003 ceph-mon[31227]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:08:25.476 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:25 smithi003 ceph-mon[31227]: osd.2 [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077] boot 2024-09-21T06:08:25.476 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:25 smithi003 ceph-mon[31227]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T06:08:25.476 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:08:25.476 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:08:25.476 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:08:25.476 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:26.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:26 smithi003 ceph-mon[31227]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:26.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:26 smithi003 ceph-mon[31227]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T06:08:26.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:08:26.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:26 smithi137 ceph-mon[34464]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:26.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:26 smithi137 ceph-mon[34464]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T06:08:26.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:08:26.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:26 smithi204 ceph-mon[34659]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:26.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:26 smithi204 ceph-mon[34659]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T06:08:26.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:08:27.253 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42710]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:08:27.253 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42710]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:08:27.253 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42710]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:08:27.253 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42710]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:08:27.254 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42713]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:08:27.254 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42713]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:08:27.254 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42713]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:08:27.254 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42713]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:08:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 ceph-mon[34464]: pgmap v69: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T06:08:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 ceph-mon[34464]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T06:08:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:08:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3709512817' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94ab3938-7a70-44cb-99bf-9d2dedc4eb87"} : dispatch 2024-09-21T06:08:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:08:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3709512817' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94ab3938-7a70-44cb-99bf-9d2dedc4eb87"}]': finished 2024-09-21T06:08:27.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 ceph-mon[34464]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T06:08:27.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:27.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:27.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:27.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42716]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:08:27.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42716]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:08:27.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42716]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:08:27.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42716]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:08:27.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42719]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:08:27.340 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:08:26 smithi003 sudo[52607]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:08:27.340 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:08:26 smithi003 sudo[52607]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:08:27.340 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:08:26 smithi003 sudo[52607]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:08:27.340 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:08:26 smithi003 sudo[52607]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:08:27.341 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:08:26 smithi003 sudo[52610]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:08:27.341 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:08:26 smithi003 sudo[52610]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:08:27.341 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:08:26 smithi003 sudo[52610]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:08:27.341 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:08:26 smithi003 sudo[52610]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:08:27.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 ceph-mon[31227]: pgmap v69: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:27.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T06:08:27.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 ceph-mon[31227]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T06:08:27.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:08:27.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3709512817' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94ab3938-7a70-44cb-99bf-9d2dedc4eb87"} : dispatch 2024-09-21T06:08:27.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:08:27.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3709512817' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94ab3938-7a70-44cb-99bf-9d2dedc4eb87"}]': finished 2024-09-21T06:08:27.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 ceph-mon[31227]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T06:08:27.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:27.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:27.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:27.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 sudo[52619]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:08:27.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 sudo[52619]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:08:27.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 sudo[52619]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:08:27.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 sudo[52619]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:08:27.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 sudo[52641]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:08:27.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 sudo[52641]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:08:27.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 sudo[52641]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:08:27.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:27 smithi003 sudo[52641]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:08:27.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43253]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:08:27.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43253]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:08:27.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43253]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:08:27.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43253]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:08:27.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43256]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:08:27.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43256]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:08:27.405 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43256]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:08:27.405 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43256]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:08:27.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 ceph-mon[34659]: pgmap v69: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:27.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T06:08:27.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 ceph-mon[34659]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T06:08:27.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:08:27.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3709512817' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94ab3938-7a70-44cb-99bf-9d2dedc4eb87"} : dispatch 2024-09-21T06:08:27.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:08:27.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3709512817' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94ab3938-7a70-44cb-99bf-9d2dedc4eb87"}]': finished 2024-09-21T06:08:27.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 ceph-mon[34659]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T06:08:27.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:27.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43259]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:08:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43259]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:08:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43259]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:08:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43259]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:08:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43262]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:08:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43262]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:08:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43262]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:08:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:27 smithi204 sudo[43262]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:08:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42719]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:08:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42719]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:08:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:27 smithi137 sudo[42719]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:08:28.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:28.147+0000 7f2adbfff640 1 -- 172.21.15.3:0/3563078361 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f2acc0ca010 con 0x7f2ae41094a0 2024-09-21T06:08:28.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:28.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:08:28.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:08:28.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:08:28.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:08:28.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:08:28.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:08:28.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:08:28.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:08:28.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:08:28.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:08:28.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:08:28.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:08:28.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:08:28.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:08:28.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:08:28.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2661840821' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:28.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T06:08:28.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:28.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:28.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:08:28.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:08:28.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:08:28.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:08:28.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:08:28.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:08:28.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:08:28.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:08:28.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:08:28.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:08:28.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:08:28.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:08:28.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:08:28.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:08:28.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:08:28.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2661840821' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:28.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T06:08:28.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:28.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:28.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:08:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:08:28.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2661840821' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:28.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T06:08:28.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:29.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:29 smithi137 ceph-mon[34464]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:29.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:29 smithi137 ceph-mon[34464]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T06:08:29.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:29 smithi204 ceph-mon[34659]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:29.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:29 smithi204 ceph-mon[34659]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T06:08:29.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:29 smithi003 ceph-mon[31227]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:29.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:29 smithi003 ceph-mon[31227]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T06:08:31.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:31 smithi137 ceph-mon[34464]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:31.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:31 smithi204 ceph-mon[34659]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:31 smithi003 ceph-mon[31227]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:33.401 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:33 smithi003 ceph-mon[31227]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:33.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:33 smithi204 ceph-mon[34659]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:33.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:33 smithi137 ceph-mon[34464]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:35.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:35 smithi204 ceph-mon[34659]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:35.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:35 smithi003 ceph-mon[31227]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:35.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:35 smithi137 ceph-mon[34464]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:36.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:36 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T06:08:36.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:36 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:36.547 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:36 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T06:08:36.547 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:36 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:36.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:36 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T06:08:36.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:36 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:37.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:37 smithi003 ceph-mon[31227]: Deploying daemon osd.3 on smithi003 2024-09-21T06:08:37.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:37 smithi003 ceph-mon[31227]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:37.535 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:37 smithi204 ceph-mon[34659]: Deploying daemon osd.3 on smithi003 2024-09-21T06:08:37.535 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:37 smithi204 ceph-mon[34659]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:37.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:37 smithi137 ceph-mon[34464]: Deploying daemon osd.3 on smithi003 2024-09-21T06:08:37.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:37 smithi137 ceph-mon[34464]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:39.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:39 smithi003 ceph-mon[31227]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:39.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:39 smithi137 ceph-mon[34464]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:39.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:39 smithi204 ceph-mon[34659]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:41.314 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:41 smithi003 ceph-mon[31227]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:41.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:41 smithi137 ceph-mon[34464]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:41 smithi204 ceph-mon[34659]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:42.500 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:42.500 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:42.500 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:42.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:42.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:42.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:42.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:42.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:42.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:43.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:43 smithi003 ceph-mon[31227]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:43 smithi137 ceph-mon[34464]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:43.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:43 smithi204 ceph-mon[34659]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:45.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:45 smithi003 ceph-mon[31227]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:45.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:45 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:45.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:45 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:45.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:45 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:45.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:45 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:45.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:45 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:45 smithi137 ceph-mon[34464]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:45 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:45 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:45 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:45 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:45 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:45 smithi204 ceph-mon[34659]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:45 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:45 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:45 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:45 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:45 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:46.114 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 3 on host 'smithi003' 2024-09-21T06:08:46.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:46.111+0000 7f2adbfff640 1 -- 172.21.15.3:0/3563078361 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2ae4063800 con 0x7f2ad00b64d0 2024-09-21T06:08:46.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:46.114+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3563078361 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2ad00b64d0 msgr2=0x7f2ad00b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:46.114+0000 7f2ae91f3640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2ad00b64d0 0x7f2ad00b89e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4077160 tx=0x7f2adc009500 comp rx=0 tx=0).stop 2024-09-21T06:08:46.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:46.114+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3563078361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41094a0 msgr2=0x7f2ae4075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:46.114+0000 7f2ae91f3640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41094a0 0x7f2ae4075c50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2acc006010 tx=0x7f2acc004060 comp rx=0 tx=0).stop 2024-09-21T06:08:46.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:46.114+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3563078361 shutdown_connections 2024-09-21T06:08:46.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:46.114+0000 7f2ae91f3640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2ad00b64d0 0x7f2ad00b89e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:46.114+0000 7f2ae91f3640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae4109df0 0x7f2ae4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:46.114+0000 7f2ae91f3640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41094a0 0x7f2ae4075c50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:46.114+0000 7f2ae91f3640 1 --2- 172.21.15.3:0/3563078361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4103b60 0x7f2ae4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:46.114+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3563078361 >> 172.21.15.3:0/3563078361 conn(0x7f2ae407b850 msgr2=0x7f2ae4105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:46.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:46.114+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3563078361 shutdown_connections 2024-09-21T06:08:46.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:08:46.114+0000 7f2ae91f3640 1 -- 172.21.15.3:0/3563078361 wait complete. 2024-09-21T06:08:46.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:46 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:46.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:46 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:46.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:46 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:46.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:46 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:46.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:46 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:46.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:46 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:46.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:46 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:46.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:46 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:46.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:46 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:46.710 DEBUG:teuthology.orchestra.run.smithi003:osd.3> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T06:08:46.713 INFO:tasks.cephadm:Deploying osd.4 on smithi137 with /dev/vg_nvme/lv_3... 2024-09-21T06:08:46.713 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T06:08:46.967 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:08:47.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:47 smithi003 ceph-mon[31227]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:47.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:47 smithi137 ceph-mon[34464]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:47.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:47 smithi204 ceph-mon[34659]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:47.831 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:08:47 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3[54593]: 2024-09-21T06:08:47.549+0000 7f6eb7cdd640 -1 osd.3 0 log_to_monitors true 2024-09-21T06:08:48.441 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:48 smithi137 ceph-mon[34464]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T06:08:48.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:48 smithi003 ceph-mon[31227]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T06:08:48.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:48 smithi204 ceph-mon[34659]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T06:08:49.349 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:08:49.375 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:49 smithi137 ceph-mon[34464]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:49.375 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:49 smithi137 ceph-mon[34464]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T06:08:49.375 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:49 smithi137 ceph-mon[34464]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T06:08:49.375 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:49 smithi137 ceph-mon[34464]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]} : dispatch 2024-09-21T06:08:49.375 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:49 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:49.377 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi137:vg_nvme/lv_3 2024-09-21T06:08:49.539 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:49 smithi003 ceph-mon[31227]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:49.539 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:49 smithi003 ceph-mon[31227]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T06:08:49.540 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:49 smithi003 ceph-mon[31227]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T06:08:49.540 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:49 smithi003 ceph-mon[31227]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]} : dispatch 2024-09-21T06:08:49.540 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:49 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:49.627 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:08:49.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:49 smithi204 ceph-mon[34659]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:49.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:49 smithi204 ceph-mon[34659]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T06:08:49.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:49 smithi204 ceph-mon[34659]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T06:08:49.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:49 smithi204 ceph-mon[34659]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]} : dispatch 2024-09-21T06:08:49.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:49 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:49.840 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:08:49 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3[54593]: 2024-09-21T06:08:49.639+0000 7f6eb4474640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T06:08:49.840 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:08:49 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3[54593]: 2024-09-21T06:08:49.645+0000 7f6eaba55640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:08:50.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.176+0000 7f60c2bea640 1 -- 172.21.15.137:0/3186403126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc109570 msgr2=0x7f60bc109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:50.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.176+0000 7f60c2bea640 1 --2- 172.21.15.137:0/3186403126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc109570 0x7f60bc109980 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009900 tx=0x7f60a802a410 comp rx=0 tx=0).stop 2024-09-21T06:08:50.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.176+0000 7f60c2bea640 1 -- 172.21.15.137:0/3186403126 shutdown_connections 2024-09-21T06:08:50.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.176+0000 7f60c2bea640 1 --2- 172.21.15.137:0/3186403126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60bc109ec0 0x7f60bc1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:50.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.176+0000 7f60c2bea640 1 --2- 172.21.15.137:0/3186403126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc109570 0x7f60bc109980 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:50.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.176+0000 7f60c2bea640 1 --2- 172.21.15.137:0/3186403126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60bc103c30 0x7f60bc104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:50.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.176+0000 7f60c2bea640 1 -- 172.21.15.137:0/3186403126 >> 172.21.15.137:0/3186403126 conn(0x7f60bc07b8d0 msgr2=0x7f60bc07bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:50.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.176+0000 7f60c2bea640 1 -- 172.21.15.137:0/3186403126 shutdown_connections 2024-09-21T06:08:50.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.176+0000 7f60c2bea640 1 -- 172.21.15.137:0/3186403126 wait complete. 2024-09-21T06:08:50.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60c2bea640 1 Processor -- start 2024-09-21T06:08:50.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60c2bea640 1 -- start start 2024-09-21T06:08:50.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60c2bea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60bc103c30 0x7f60bc19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:50.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60c2bea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc109570 0x7f60bc19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60c2bea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60bc109ec0 0x7f60bc19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60c2bea640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc113ae0 con 0x7f60bc103c30 2024-09-21T06:08:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60c2bea640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc113960 con 0x7f60bc109570 2024-09-21T06:08:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60c2bea640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc113c60 con 0x7f60bc109ec0 2024-09-21T06:08:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60bbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc109570 0x7f60bc19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60bbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc109570 0x7f60bc19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34286/0 (socket says 172.21.15.137:34286) 2024-09-21T06:08:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60bbfff640 1 -- 172.21.15.137:0/3478168634 learned_addr learned my addr 172.21.15.137:0/3478168634 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-21T06:08:50.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60c0986640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60bc103c30 0x7f60bc19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:50.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60c1187640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60bc109ec0 0x7f60bc19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:50.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60bbfff640 1 -- 172.21.15.137:0/3478168634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60bc109ec0 msgr2=0x7f60bc19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:50.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60bbfff640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60bc109ec0 0x7f60bc19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:50.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60bbfff640 1 -- 172.21.15.137:0/3478168634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60bc103c30 msgr2=0x7f60bc19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:50.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60bbfff640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60bc103c30 0x7f60bc19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:50.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.177+0000 7f60bbfff640 1 -- 172.21.15.137:0/3478168634 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc1a0440 con 0x7f60bc109570 2024-09-21T06:08:50.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.178+0000 7f60bbfff640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc109570 0x7f60bc19aa70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f60a802a930 tx=0x7f60a8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:50.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.178+0000 7f60b9ffb640 1 -- 172.21.15.137:0/3478168634 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a803f030 con 0x7f60bc109570 2024-09-21T06:08:50.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.178+0000 7f60c2bea640 1 -- 172.21.15.137:0/3478168634 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60bc1a06d0 con 0x7f60bc109570 2024-09-21T06:08:50.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.178+0000 7f60b9ffb640 1 -- 172.21.15.137:0/3478168634 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a8003aa0 con 0x7f60bc109570 2024-09-21T06:08:50.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.178+0000 7f60b9ffb640 1 -- 172.21.15.137:0/3478168634 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8035070 con 0x7f60bc109570 2024-09-21T06:08:50.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.178+0000 7f60c2bea640 1 -- 172.21.15.137:0/3478168634 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60bc1a6e90 con 0x7f60bc109570 2024-09-21T06:08:50.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.179+0000 7f60c2bea640 1 -- 172.21.15.137:0/3478168634 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60bc104040 con 0x7f60bc109570 2024-09-21T06:08:50.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.181+0000 7f60b9ffb640 1 -- 172.21.15.137:0/3478168634 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f60a8042030 con 0x7f60bc109570 2024-09-21T06:08:50.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.182+0000 7f60b9ffb640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f60ac0b6560 0x7f60ac0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:50.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.182+0000 7f60b9ffb640 1 -- 172.21.15.137:0/3478168634 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3604+0+0 (secure 0 0 0) 0x7f60a8106b90 con 0x7f60bc109570 2024-09-21T06:08:50.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.182+0000 7f60c0986640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f60ac0b6560 0x7f60ac0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:50.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.183+0000 7f60c0986640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f60ac0b6560 0x7f60ac0b8a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60b0005fd0 tx=0x7f60b000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:50.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.190+0000 7f60b9ffb640 1 -- 172.21.15.137:0/3478168634 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f60a80ce680 con 0x7f60bc109570 2024-09-21T06:08:50.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:08:50.386+0000 7f60c2bea640 1 -- 172.21.15.137:0/3478168634 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f60bc063800 con 0x7f60ac0b6560 2024-09-21T06:08:50.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-21T06:08:50.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T06:08:50.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:50.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:50.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:50.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:50.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:50.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:50.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:50.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:50.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:50.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:50.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' 2024-09-21T06:08:50.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635] boot 2024-09-21T06:08:50.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T06:08:50.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:50.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-21T06:08:50.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T06:08:50.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:50.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:50.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:50.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:50.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:50.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:50.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:50.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:50.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:50.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:50.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' 2024-09-21T06:08:50.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635] boot 2024-09-21T06:08:50.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T06:08:50.593 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:50.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-21T06:08:50.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T06:08:50.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:50.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:50.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:50.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:50.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:50.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:50.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:50.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:50.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:50.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:08:50.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: from='osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635]' entity='osd.3' 2024-09-21T06:08:50.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: osd.3 [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635] boot 2024-09-21T06:08:50.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T06:08:50.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:08:51.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:51 smithi137 ceph-mon[34464]: purged_snaps scrub starts 2024-09-21T06:08:51.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:51 smithi137 ceph-mon[34464]: purged_snaps scrub ok 2024-09-21T06:08:51.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:51 smithi137 ceph-mon[34464]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:51.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:51 smithi137 ceph-mon[34464]: Detected new or changed devices on smithi003 2024-09-21T06:08:51.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:51 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi003 to 8546M 2024-09-21T06:08:51.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:51 smithi137 ceph-mon[34464]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:51.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:08:51.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:08:51.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:51.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:51 smithi137 ceph-mon[34464]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T06:08:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:51 smithi003 ceph-mon[31227]: purged_snaps scrub starts 2024-09-21T06:08:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:51 smithi003 ceph-mon[31227]: purged_snaps scrub ok 2024-09-21T06:08:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:51 smithi003 ceph-mon[31227]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:51 smithi003 ceph-mon[31227]: Detected new or changed devices on smithi003 2024-09-21T06:08:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:51 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi003 to 8546M 2024-09-21T06:08:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:51 smithi003 ceph-mon[31227]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:08:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:08:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:51 smithi003 ceph-mon[31227]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T06:08:51.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:51 smithi204 ceph-mon[34659]: purged_snaps scrub starts 2024-09-21T06:08:51.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:51 smithi204 ceph-mon[34659]: purged_snaps scrub ok 2024-09-21T06:08:51.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:51 smithi204 ceph-mon[34659]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:08:51.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:51 smithi204 ceph-mon[34659]: Detected new or changed devices on smithi003 2024-09-21T06:08:51.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:51 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi003 to 8546M 2024-09-21T06:08:51.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:51 smithi204 ceph-mon[34659]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:51.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:08:51.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:08:51.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:51.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:51 smithi204 ceph-mon[34659]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T06:08:53.332 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:53 smithi137 ceph-mon[34464]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:08:53.332 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:53 smithi137 ceph-mon[34464]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T06:08:53.332 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:53 smithi137 ceph-mon[34464]: from='client.? 172.21.15.137:0/2009415151' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "38551535-d692-401c-aad0-bee38f3026d4"} : dispatch 2024-09-21T06:08:53.332 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:53 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "38551535-d692-401c-aad0-bee38f3026d4"} : dispatch 2024-09-21T06:08:53.332 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:53 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38551535-d692-401c-aad0-bee38f3026d4"}]': finished 2024-09-21T06:08:53.332 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:53 smithi137 ceph-mon[34464]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T06:08:53.332 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:53 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:08:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:53 smithi003 ceph-mon[31227]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:08:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:53 smithi003 ceph-mon[31227]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T06:08:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:53 smithi003 ceph-mon[31227]: from='client.? 172.21.15.137:0/2009415151' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "38551535-d692-401c-aad0-bee38f3026d4"} : dispatch 2024-09-21T06:08:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:53 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "38551535-d692-401c-aad0-bee38f3026d4"} : dispatch 2024-09-21T06:08:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:53 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38551535-d692-401c-aad0-bee38f3026d4"}]': finished 2024-09-21T06:08:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:53 smithi003 ceph-mon[31227]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T06:08:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:53 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:08:53.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:53 smithi204 ceph-mon[34659]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:08:53.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:53 smithi204 ceph-mon[34659]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T06:08:53.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:53 smithi204 ceph-mon[34659]: from='client.? 172.21.15.137:0/2009415151' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "38551535-d692-401c-aad0-bee38f3026d4"} : dispatch 2024-09-21T06:08:53.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:53 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "38551535-d692-401c-aad0-bee38f3026d4"} : dispatch 2024-09-21T06:08:53.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:53 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38551535-d692-401c-aad0-bee38f3026d4"}]': finished 2024-09-21T06:08:53.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:53 smithi204 ceph-mon[34659]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T06:08:53.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:53 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:08:54.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:54 smithi003 ceph-mon[31227]: from='client.? 172.21.15.137:0/432250356' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:54.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:54 smithi137 ceph-mon[34464]: from='client.? 172.21.15.137:0/432250356' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:54.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:54 smithi204 ceph-mon[34659]: from='client.? 172.21.15.137:0/432250356' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:55 smithi003 ceph-mon[31227]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:08:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:55 smithi137 ceph-mon[34464]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:08:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:55 smithi204 ceph-mon[34659]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:08:57.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:57 smithi003 ceph-mon[31227]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-21T06:08:57.614 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:57 smithi137 ceph-mon[34464]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-21T06:08:57.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:57 smithi204 ceph-mon[34659]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-21T06:08:59.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:08:59 smithi137 ceph-mon[34464]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-21T06:08:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:08:59 smithi003 ceph-mon[31227]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-21T06:08:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:08:59 smithi204 ceph-mon[34659]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-21T06:09:01.510 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:01 smithi137 ceph-mon[34464]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:09:01.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:01 smithi003 ceph-mon[31227]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:09:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:01 smithi204 ceph-mon[34659]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:09:02.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:02 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T06:09:02.601 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:02 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:02.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:02 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T06:09:02.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:02 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:02.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:02 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T06:09:02.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:02 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:03.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:03 smithi137 ceph-mon[34464]: Deploying daemon osd.4 on smithi137 2024-09-21T06:09:03.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:03 smithi137 ceph-mon[34464]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T06:09:03.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:03 smithi003 ceph-mon[31227]: Deploying daemon osd.4 on smithi137 2024-09-21T06:09:03.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:03 smithi003 ceph-mon[31227]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T06:09:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:03 smithi204 ceph-mon[34659]: Deploying daemon osd.4 on smithi137 2024-09-21T06:09:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:03 smithi204 ceph-mon[34659]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T06:09:05.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:05 smithi003 ceph-mon[31227]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T06:09:05.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:05 smithi137 ceph-mon[34464]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T06:09:05.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:05 smithi204 ceph-mon[34659]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T06:09:07.491 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:07 smithi137 ceph-mon[34464]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:09:07.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:07 smithi003 ceph-mon[31227]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:09:07.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:07 smithi204 ceph-mon[34659]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:09:08.589 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:08.589 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:08.589 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:08.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:08.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:08.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:08.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:08.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:08.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:09.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:09 smithi137 ceph-mon[34464]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:09.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:09 smithi003 ceph-mon[31227]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:09 smithi204 ceph-mon[34659]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:11 smithi003 ceph-mon[31227]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:11 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:11 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:11 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:11 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:11.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:11 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:11 smithi137 ceph-mon[34464]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:11 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:11 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:11 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:11 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:11 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:11 smithi204 ceph-mon[34659]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:11 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:11 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:11 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:11 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:11 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:12.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:09:12.161+0000 7f60b9ffb640 1 -- 172.21.15.137:0/3478168634 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f60bc063800 con 0x7f60ac0b6560 2024-09-21T06:09:12.163 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 4 on host 'smithi137' 2024-09-21T06:09:12.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:09:12.164+0000 7f60c2bea640 1 -- 172.21.15.137:0/3478168634 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f60ac0b6560 msgr2=0x7f60ac0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:12.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:09:12.164+0000 7f60c2bea640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f60ac0b6560 0x7f60ac0b8a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60b0005fd0 tx=0x7f60b000b390 comp rx=0 tx=0).stop 2024-09-21T06:09:12.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:09:12.164+0000 7f60c2bea640 1 -- 172.21.15.137:0/3478168634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc109570 msgr2=0x7f60bc19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:12.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:09:12.164+0000 7f60c2bea640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc109570 0x7f60bc19aa70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f60a802a930 tx=0x7f60a8004270 comp rx=0 tx=0).stop 2024-09-21T06:09:12.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:09:12.164+0000 7f60c2bea640 1 -- 172.21.15.137:0/3478168634 shutdown_connections 2024-09-21T06:09:12.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:09:12.164+0000 7f60c2bea640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f60ac0b6560 0x7f60ac0b8a70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:09:12.164+0000 7f60c2bea640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60bc109ec0 0x7f60bc19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:09:12.164+0000 7f60c2bea640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc109570 0x7f60bc19aa70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:09:12.164+0000 7f60c2bea640 1 --2- 172.21.15.137:0/3478168634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60bc103c30 0x7f60bc19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:09:12.164+0000 7f60c2bea640 1 -- 172.21.15.137:0/3478168634 >> 172.21.15.137:0/3478168634 conn(0x7f60bc07b8d0 msgr2=0x7f60bc1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:12.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:09:12.164+0000 7f60c2bea640 1 -- 172.21.15.137:0/3478168634 shutdown_connections 2024-09-21T06:09:12.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:09:12.164+0000 7f60c2bea640 1 -- 172.21.15.137:0/3478168634 wait complete. 2024-09-21T06:09:12.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:12 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:12.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:12 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:12.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:12 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:12.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:12 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:12.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:12 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:12.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:12 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:12.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:12 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:12.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:12 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:12.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:12 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:12.906 DEBUG:teuthology.orchestra.run.smithi137:osd.4> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T06:09:12.908 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2024-09-21T06:09:12.908 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T06:09:13.164 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:09:13.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:13 smithi137 ceph-mon[34464]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:13.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:13 smithi204 ceph-mon[34659]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:13.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:13 smithi003 ceph-mon[31227]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:13.895 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:09:13 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4[45318]: 2024-09-21T06:09:13.656+0000 7fef04c45640 -1 osd.4 0 log_to_monitors true 2024-09-21T06:09:14.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:14 smithi137 ceph-mon[34464]: from='osd.4 [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:09:14.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:14 smithi137 ceph-mon[34464]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:09:14.511 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:14 smithi003 ceph-mon[31227]: from='osd.4 [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:09:14.511 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:14 smithi003 ceph-mon[31227]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:09:14.531 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:14 smithi204 ceph-mon[34659]: from='osd.4 [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:09:14.531 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:14 smithi204 ceph-mon[34659]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:09:15.486 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:15 smithi204 ceph-mon[34659]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:15.486 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:15 smithi204 ceph-mon[34659]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T06:09:15.486 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:15 smithi204 ceph-mon[34659]: from='osd.4 [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:09:15.486 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:15 smithi204 ceph-mon[34659]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T06:09:15.486 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:15 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:09:15.486 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:15 smithi204 ceph-mon[34659]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:09:15.563 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:09:15 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4[45318]: 2024-09-21T06:09:15.269+0000 7fef00bc7640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T06:09:15.564 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:09:15 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4[45318]: 2024-09-21T06:09:15.277+0000 7feefc9c5640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:09:15.564 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:15 smithi137 ceph-mon[34464]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:15.564 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:15 smithi137 ceph-mon[34464]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T06:09:15.564 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:15 smithi137 ceph-mon[34464]: from='osd.4 [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:09:15.564 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:15 smithi137 ceph-mon[34464]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T06:09:15.564 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:15 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:09:15.564 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:15 smithi137 ceph-mon[34464]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:09:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:15 smithi003 ceph-mon[31227]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:15 smithi003 ceph-mon[31227]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T06:09:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:15 smithi003 ceph-mon[31227]: from='osd.4 [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:09:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:15 smithi003 ceph-mon[31227]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T06:09:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:15 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:09:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:15 smithi003 ceph-mon[31227]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:09:15.879 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:09:15.905 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2024-09-21T06:09:16.161 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:09:16.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:16 smithi003 ceph-mon[31227]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-21T06:09:16.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:16 smithi003 ceph-mon[31227]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T06:09:16.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:16 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:09:16.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:16 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:09:16.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:16 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:16.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:16 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:16.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:16 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:09:16.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:16 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:09:16.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:16 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:16.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:16 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:16.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:16 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:16.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:16 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:16.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:16 smithi204 ceph-mon[34659]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-21T06:09:16.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:16 smithi204 ceph-mon[34659]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T06:09:16.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:16 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:09:16.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:16 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:09:16.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:16 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:16.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:16 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:16.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:16 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:09:16.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:16 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:09:16.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:16 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:16.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:16 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:16.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:16 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:16.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:16 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:16 smithi137 ceph-mon[34464]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-21T06:09:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:16 smithi137 ceph-mon[34464]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T06:09:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:16 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:09:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:16 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:09:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:16 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:16 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:16 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:09:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:16 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:09:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:16 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:16 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:16 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:16.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:16 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:16.754 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.722+0000 7fd834415640 1 -- 172.21.15.204:0/2710670348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd82c108da0 msgr2=0x7fd82c10f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.754 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.722+0000 7fd834415640 1 --2- 172.21.15.204:0/2710670348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd82c108da0 0x7fd82c10f5d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd824009ca0 tx=0x7fd82402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.754 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.722+0000 7fd834415640 1 -- 172.21.15.204:0/2710670348 shutdown_connections 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.722+0000 7fd834415640 1 --2- 172.21.15.204:0/2710670348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd82c108da0 0x7fd82c10f5d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.722+0000 7fd834415640 1 --2- 172.21.15.204:0/2710670348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd82c108450 0x7fd82c108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.722+0000 7fd834415640 1 --2- 172.21.15.204:0/2710670348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd82c102b10 0x7fd82c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.722+0000 7fd834415640 1 -- 172.21.15.204:0/2710670348 >> 172.21.15.204:0/2710670348 conn(0x7fd82c0796d0 msgr2=0x7fd82c079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.722+0000 7fd834415640 1 -- 172.21.15.204:0/2710670348 shutdown_connections 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.722+0000 7fd834415640 1 -- 172.21.15.204:0/2710670348 wait complete. 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.723+0000 7fd834415640 1 Processor -- start 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.723+0000 7fd834415640 1 -- start start 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.723+0000 7fd834415640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd82c102b10 0x7fd82c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.723+0000 7fd834415640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd82c108450 0x7fd82c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.723+0000 7fd834415640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd82c108da0 0x7fd82c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.723+0000 7fd834415640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c112a60 con 0x7fd82c108450 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.723+0000 7fd834415640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c1128e0 con 0x7fd82c108da0 2024-09-21T06:09:16.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.723+0000 7fd834415640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c112be0 con 0x7fd82c102b10 2024-09-21T06:09:16.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.723+0000 7fd8321b1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd82c102b10 0x7fd82c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.723+0000 7fd8321b1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd82c102b10 0x7fd82c1994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:56970/0 (socket says 172.21.15.204:56970) 2024-09-21T06:09:16.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.723+0000 7fd8321b1640 1 -- 172.21.15.204:0/234096381 learned_addr learned my addr 172.21.15.204:0/234096381 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-21T06:09:16.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.723+0000 7fd8329b2640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd82c108da0 0x7fd82c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.723+0000 7fd8319b0640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd82c108450 0x7fd82c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.724+0000 7fd8321b1640 1 -- 172.21.15.204:0/234096381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd82c108da0 msgr2=0x7fd82c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.724+0000 7fd8321b1640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd82c108da0 0x7fd82c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.724+0000 7fd8321b1640 1 -- 172.21.15.204:0/234096381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd82c108450 msgr2=0x7fd82c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.724+0000 7fd8321b1640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd82c108450 0x7fd82c199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.724+0000 7fd8321b1640 1 -- 172.21.15.204:0/234096381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82c19f420 con 0x7fd82c102b10 2024-09-21T06:09:16.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.724+0000 7fd8319b0640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd82c108450 0x7fd82c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:16.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.724+0000 7fd8321b1640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd82c102b10 0x7fd82c1994b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c970 tx=0x7fd82800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:16.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.724+0000 7fd8237fe640 1 -- 172.21.15.204:0/234096381 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828007bb0 con 0x7fd82c102b10 2024-09-21T06:09:16.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.724+0000 7fd8237fe640 1 -- 172.21.15.204:0/234096381 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd828007d50 con 0x7fd82c102b10 2024-09-21T06:09:16.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.724+0000 7fd8237fe640 1 -- 172.21.15.204:0/234096381 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82800f450 con 0x7fd82c102b10 2024-09-21T06:09:16.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.724+0000 7fd834415640 1 -- 172.21.15.204:0/234096381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82c19f710 con 0x7fd82c102b10 2024-09-21T06:09:16.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.724+0000 7fd834415640 1 -- 172.21.15.204:0/234096381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd82c1a5e70 con 0x7fd82c102b10 2024-09-21T06:09:16.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.725+0000 7fd834415640 1 -- 172.21.15.204:0/234096381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd82c102f20 con 0x7fd82c102b10 2024-09-21T06:09:16.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.727+0000 7fd8237fe640 1 -- 172.21.15.204:0/234096381 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7fd828004f20 con 0x7fd82c102b10 2024-09-21T06:09:16.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.727+0000 7fd8237fe640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd8180b65e0 0x7fd8180b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.727+0000 7fd8237fe640 1 -- 172.21.15.204:0/234096381 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4316+0+0 (secure 0 0 0) 0x7fd8280e6030 con 0x7fd82c102b10 2024-09-21T06:09:16.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.728+0000 7fd8319b0640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd8180b65e0 0x7fd8180b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.728+0000 7fd8319b0640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd8180b65e0 0x7fd8180b8af0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd81c005fd0 tx=0x7fd81c005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:16.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.732+0000 7fd8237fe640 1 -- 172.21.15.204:0/234096381 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd828014030 con 0x7fd82c102b10 2024-09-21T06:09:16.924 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:16.922+0000 7fd834415640 1 -- 172.21.15.204:0/234096381 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd82c063800 con 0x7fd8180b65e0 2024-09-21T06:09:17.557 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:17 smithi204 ceph-mon[34659]: purged_snaps scrub starts 2024-09-21T06:09:17.557 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:17 smithi204 ceph-mon[34659]: purged_snaps scrub ok 2024-09-21T06:09:17.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:17 smithi204 ceph-mon[34659]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:17.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:17 smithi204 ceph-mon[34659]: Detected new or changed devices on smithi137 2024-09-21T06:09:17.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:17 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi137 to 8544M 2024-09-21T06:09:17.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:17 smithi204 ceph-mon[34659]: osd.4 [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785] boot 2024-09-21T06:09:17.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:17 smithi204 ceph-mon[34659]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T06:09:17.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:09:17.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:09:17.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:09:17.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:17.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:17 smithi003 ceph-mon[31227]: purged_snaps scrub starts 2024-09-21T06:09:17.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:17 smithi003 ceph-mon[31227]: purged_snaps scrub ok 2024-09-21T06:09:17.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:17 smithi003 ceph-mon[31227]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:17.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:17 smithi003 ceph-mon[31227]: Detected new or changed devices on smithi137 2024-09-21T06:09:17.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:17 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi137 to 8544M 2024-09-21T06:09:17.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:17 smithi003 ceph-mon[31227]: osd.4 [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785] boot 2024-09-21T06:09:17.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:17 smithi003 ceph-mon[31227]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T06:09:17.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:09:17.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:09:17.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:09:17.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:17.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:17 smithi137 ceph-mon[34464]: purged_snaps scrub starts 2024-09-21T06:09:17.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:17 smithi137 ceph-mon[34464]: purged_snaps scrub ok 2024-09-21T06:09:17.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:17 smithi137 ceph-mon[34464]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:09:17.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:17 smithi137 ceph-mon[34464]: Detected new or changed devices on smithi137 2024-09-21T06:09:17.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:17 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi137 to 8544M 2024-09-21T06:09:17.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:17 smithi137 ceph-mon[34464]: osd.4 [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785] boot 2024-09-21T06:09:17.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:17 smithi137 ceph-mon[34464]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T06:09:17.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:09:17.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:09:17.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:09:17.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:18.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:18 smithi204 ceph-mon[34659]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:18.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:18 smithi204 ceph-mon[34659]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T06:09:18.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:18 smithi003 ceph-mon[31227]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:18.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:18 smithi003 ceph-mon[31227]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T06:09:18.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:18 smithi137 ceph-mon[34464]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:18.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:18 smithi137 ceph-mon[34464]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T06:09:19.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:19 smithi003 ceph-mon[31227]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:19.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:19 smithi137 ceph-mon[34464]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:19 smithi204 ceph-mon[34659]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:20.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:20 smithi204 ceph-mon[34659]: from='client.? 172.21.15.204:0/701861564' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3d4b785b-28d0-4654-a9b1-4c5c905f4bcc"} : dispatch 2024-09-21T06:09:20.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:20 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3d4b785b-28d0-4654-a9b1-4c5c905f4bcc"} : dispatch 2024-09-21T06:09:20.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:20 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d4b785b-28d0-4654-a9b1-4c5c905f4bcc"}]': finished 2024-09-21T06:09:20.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:20 smithi204 ceph-mon[34659]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T06:09:20.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:20 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:20.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:20 smithi204 ceph-mon[34659]: from='client.? 172.21.15.204:0/612246110' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:09:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:20 smithi003 ceph-mon[31227]: from='client.? 172.21.15.204:0/701861564' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3d4b785b-28d0-4654-a9b1-4c5c905f4bcc"} : dispatch 2024-09-21T06:09:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:20 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3d4b785b-28d0-4654-a9b1-4c5c905f4bcc"} : dispatch 2024-09-21T06:09:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:20 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d4b785b-28d0-4654-a9b1-4c5c905f4bcc"}]': finished 2024-09-21T06:09:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:20 smithi003 ceph-mon[31227]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T06:09:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:20 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:20 smithi003 ceph-mon[31227]: from='client.? 172.21.15.204:0/612246110' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:09:20.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:20 smithi137 ceph-mon[34464]: from='client.? 172.21.15.204:0/701861564' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3d4b785b-28d0-4654-a9b1-4c5c905f4bcc"} : dispatch 2024-09-21T06:09:20.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:20 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3d4b785b-28d0-4654-a9b1-4c5c905f4bcc"} : dispatch 2024-09-21T06:09:20.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:20 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d4b785b-28d0-4654-a9b1-4c5c905f4bcc"}]': finished 2024-09-21T06:09:20.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:20 smithi137 ceph-mon[34464]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T06:09:20.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:20 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:20.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:20 smithi137 ceph-mon[34464]: from='client.? 172.21.15.204:0/612246110' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:09:21.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:21 smithi003 ceph-mon[31227]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:21.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:21 smithi137 ceph-mon[34464]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:21 smithi204 ceph-mon[34659]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:23.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:23 smithi003 ceph-mon[31227]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:23.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:23 smithi137 ceph-mon[34464]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:23.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:23 smithi204 ceph-mon[34659]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:25.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:25 smithi003 ceph-mon[31227]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:25.606 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:25 smithi204 ceph-mon[34659]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:25 smithi137 ceph-mon[34464]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:27.551 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:27 smithi204 ceph-mon[34659]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:27.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:27 smithi003 ceph-mon[31227]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:27 smithi137 ceph-mon[34464]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:28.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T06:09:28.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:28.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T06:09:28.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:28.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T06:09:28.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:29.576 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:29 smithi204 ceph-mon[34659]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:29.577 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:29 smithi204 ceph-mon[34659]: Deploying daemon osd.5 on smithi204 2024-09-21T06:09:29.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:29 smithi003 ceph-mon[31227]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:29.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:29 smithi003 ceph-mon[31227]: Deploying daemon osd.5 on smithi204 2024-09-21T06:09:29.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:29 smithi137 ceph-mon[34464]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:29.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:29 smithi137 ceph-mon[34464]: Deploying daemon osd.5 on smithi204 2024-09-21T06:09:31.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:31 smithi204 ceph-mon[34659]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:31 smithi003 ceph-mon[31227]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:31 smithi137 ceph-mon[34464]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:33.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:33 smithi204 ceph-mon[34659]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:33.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:33 smithi003 ceph-mon[31227]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:33.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:33 smithi137 ceph-mon[34464]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:35.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:35 smithi003 ceph-mon[31227]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:35.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:35.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:35.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:35.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:35 smithi137 ceph-mon[34464]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:35.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:35.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:35.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:35 smithi204 ceph-mon[34659]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:37.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:37 smithi003 ceph-mon[31227]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:37.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:37 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:37.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:37 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:37.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:37 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:37.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:37 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:37.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:37 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:37.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:37 smithi137 ceph-mon[34464]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:37.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:37 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:37.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:37 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:37.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:37 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:37.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:37 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:37.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:37 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:37.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:37 smithi204 ceph-mon[34659]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:37.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:37 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:37.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:37 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:37.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:37 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:37.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:37 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:37.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:37 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:38.175 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2024-09-21T06:09:38.175 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:38.173+0000 7fd8237fe640 1 -- 172.21.15.204:0/234096381 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd82c063800 con 0x7fd8180b65e0 2024-09-21T06:09:38.177 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:38.176+0000 7fd834415640 1 -- 172.21.15.204:0/234096381 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd8180b65e0 msgr2=0x7fd8180b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.178 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:38.176+0000 7fd834415640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd8180b65e0 0x7fd8180b8af0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd81c005fd0 tx=0x7fd81c005850 comp rx=0 tx=0).stop 2024-09-21T06:09:38.178 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:38.176+0000 7fd834415640 1 -- 172.21.15.204:0/234096381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd82c102b10 msgr2=0x7fd82c1994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.178 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:38.176+0000 7fd834415640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd82c102b10 0x7fd82c1994b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c970 tx=0x7fd82800ce30 comp rx=0 tx=0).stop 2024-09-21T06:09:38.178 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:38.176+0000 7fd834415640 1 -- 172.21.15.204:0/234096381 shutdown_connections 2024-09-21T06:09:38.178 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:38.176+0000 7fd834415640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd8180b65e0 0x7fd8180b8af0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.178 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:38.176+0000 7fd834415640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd82c108da0 0x7fd82c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.178 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:38.176+0000 7fd834415640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd82c108450 0x7fd82c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.178 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:38.176+0000 7fd834415640 1 --2- 172.21.15.204:0/234096381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd82c102b10 0x7fd82c1994b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.178 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:38.176+0000 7fd834415640 1 -- 172.21.15.204:0/234096381 >> 172.21.15.204:0/234096381 conn(0x7fd82c0796d0 msgr2=0x7fd82c106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:38.178 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:38.176+0000 7fd834415640 1 -- 172.21.15.204:0/234096381 shutdown_connections 2024-09-21T06:09:38.178 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:09:38.176+0000 7fd834415640 1 -- 172.21.15.204:0/234096381 wait complete. 2024-09-21T06:09:38.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:38 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:38.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:38 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:38.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:38 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:38.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:38 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:38.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:38 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:38.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:38 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:38.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:38 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:38.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:38 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:38.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:38 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:38.800 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T06:09:38.803 INFO:tasks.cephadm:Deploying osd.6 on smithi003 with /dev/vg_nvme/lv_2... 2024-09-21T06:09:38.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T06:09:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:39.506 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:39 smithi003 ceph-mon[31227]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:39.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:39 smithi137 ceph-mon[34464]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:39.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:39 smithi204 ceph-mon[34659]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:40.498 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:40 smithi003 ceph-mon[31227]: from='osd.5 [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:09:40.498 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:40 smithi003 ceph-mon[31227]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:09:40.531 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:40 smithi204 ceph-mon[34659]: from='osd.5 [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:09:40.531 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:40 smithi204 ceph-mon[34659]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:09:40.531 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:09:40 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:09:40.248+0000 7f87029e6640 -1 osd.5 0 log_to_monitors true 2024-09-21T06:09:40.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:40 smithi137 ceph-mon[34464]: from='osd.5 [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:09:40.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:40 smithi137 ceph-mon[34464]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:09:41.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:41 smithi003 ceph-mon[31227]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:41.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:41 smithi003 ceph-mon[31227]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T06:09:41.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:41 smithi003 ceph-mon[31227]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T06:09:41.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:41 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:41.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:41 smithi003 ceph-mon[31227]: from='osd.5 [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:09:41.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:41 smithi003 ceph-mon[31227]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:09:41.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:41 smithi137 ceph-mon[34464]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:41.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:41 smithi137 ceph-mon[34464]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T06:09:41.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:41 smithi137 ceph-mon[34464]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T06:09:41.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:41 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:41.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:41 smithi137 ceph-mon[34464]: from='osd.5 [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:09:41.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:41 smithi137 ceph-mon[34464]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:09:41.654 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:09:41 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:09:41.352+0000 7f86fe968640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T06:09:41.654 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:09:41 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:09:41.359+0000 7f86fa766640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:09:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:41 smithi204 ceph-mon[34659]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:41.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:41 smithi204 ceph-mon[34659]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T06:09:41.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:41 smithi204 ceph-mon[34659]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T06:09:41.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:41 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:41.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:41 smithi204 ceph-mon[34659]: from='osd.5 [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:09:41.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:41 smithi204 ceph-mon[34659]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:09:41.689 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:09:41.717 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi003:vg_nvme/lv_2 2024-09-21T06:09:41.994 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:42.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:42 smithi003 ceph-mon[31227]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-21T06:09:42.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:42 smithi003 ceph-mon[31227]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T06:09:42.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:42.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:42.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:42.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:42.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:09:42.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:09:42.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:42.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:42.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:42.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.509+0000 7f87ff59c640 1 -- 172.21.15.3:0/3310975317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87f81049b0 msgr2=0x7f87f8104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.509+0000 7f87ff59c640 1 --2- 172.21.15.3:0/3310975317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87f81049b0 0x7f87f8104dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009900 tx=0x7f87f402a410 comp rx=0 tx=0).stop 2024-09-21T06:09:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.509+0000 7f87ff59c640 1 -- 172.21.15.3:0/3310975317 shutdown_connections 2024-09-21T06:09:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.509+0000 7f87ff59c640 1 --2- 172.21.15.3:0/3310975317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87f810de10 0x7f87f8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.509+0000 7f87ff59c640 1 --2- 172.21.15.3:0/3310975317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f8105300 0x7f87f810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.509+0000 7f87ff59c640 1 --2- 172.21.15.3:0/3310975317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87f81049b0 0x7f87f8104dc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.509+0000 7f87ff59c640 1 -- 172.21.15.3:0/3310975317 >> 172.21.15.3:0/3310975317 conn(0x7f87f8079730 msgr2=0x7f87f8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.510+0000 7f87ff59c640 1 -- 172.21.15.3:0/3310975317 shutdown_connections 2024-09-21T06:09:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.510+0000 7f87ff59c640 1 -- 172.21.15.3:0/3310975317 wait complete. 2024-09-21T06:09:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.510+0000 7f87ff59c640 1 Processor -- start 2024-09-21T06:09:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.510+0000 7f87ff59c640 1 -- start start 2024-09-21T06:09:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87ff59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f81049b0 0x7f87f81026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87ff59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87f8105300 0x7f87f8102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87ff59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87f810de10 0x7f87f8103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87ff59c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f87f8113a90 con 0x7f87f810de10 2024-09-21T06:09:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87ff59c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f87f8113910 con 0x7f87f81049b0 2024-09-21T06:09:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87ff59c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87f8113c10 con 0x7f87f8105300 2024-09-21T06:09:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87fdb39640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87f810de10 0x7f87f8103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87fdb39640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87f810de10 0x7f87f8103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53772/0 (socket says 172.21.15.3:53772) 2024-09-21T06:09:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87fdb39640 1 -- 172.21.15.3:0/197358305 learned_addr learned my addr 172.21.15.3:0/197358305 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:09:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87fcb37640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87f8105300 0x7f87f8102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87fd338640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f81049b0 0x7f87f81026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87fdb39640 1 -- 172.21.15.3:0/197358305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87f8105300 msgr2=0x7f87f8102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87fdb39640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87f8105300 0x7f87f8102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87fdb39640 1 -- 172.21.15.3:0/197358305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f81049b0 msgr2=0x7f87f81026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87fdb39640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f81049b0 0x7f87f81026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87fdb39640 1 -- 172.21.15.3:0/197358305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f81a8880 con 0x7f87f810de10 2024-09-21T06:09:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.511+0000 7f87fdb39640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87f810de10 0x7f87f8103170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f87f000eaa0 tx=0x7f87f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.512+0000 7f87ee7fc640 1 -- 172.21.15.3:0/197358305 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f000cdb0 con 0x7f87f810de10 2024-09-21T06:09:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.512+0000 7f87ff59c640 1 -- 172.21.15.3:0/197358305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f81a8b70 con 0x7f87f810de10 2024-09-21T06:09:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.512+0000 7f87ee7fc640 1 -- 172.21.15.3:0/197358305 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f0005020 con 0x7f87f810de10 2024-09-21T06:09:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.512+0000 7f87ee7fc640 1 -- 172.21.15.3:0/197358305 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00116d0 con 0x7f87f810de10 2024-09-21T06:09:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.512+0000 7f87ff59c640 1 -- 172.21.15.3:0/197358305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f81a90b0 con 0x7f87f810de10 2024-09-21T06:09:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.513+0000 7f87ff59c640 1 -- 172.21.15.3:0/197358305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f8104dc0 con 0x7f87f810de10 2024-09-21T06:09:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.514+0000 7f87ee7fc640 1 -- 172.21.15.3:0/197358305 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f87f00054a0 con 0x7f87f810de10 2024-09-21T06:09:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.515+0000 7f87ee7fc640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f87e40b6560 0x7f87e40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.515+0000 7f87fd338640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f87e40b6560 0x7f87e40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.515+0000 7f87ee7fc640 1 -- 172.21.15.3:0/197358305 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4781+0+0 (secure 0 0 0) 0x7f87f00e6a80 con 0x7f87f810de10 2024-09-21T06:09:42.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.516+0000 7f87fd338640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f87e40b6560 0x7f87e40b8a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009230 tx=0x7f87f4031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:42.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.519+0000 7f87ee7fc640 1 -- 172.21.15.3:0/197358305 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f87f0016030 con 0x7f87f810de10 2024-09-21T06:09:42.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:42 smithi137 ceph-mon[34464]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-21T06:09:42.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:42 smithi137 ceph-mon[34464]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T06:09:42.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:42.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:42.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:42.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:42.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:09:42.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:09:42.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:42.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:42.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:42.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:42.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:42 smithi204 ceph-mon[34659]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-21T06:09:42.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:42 smithi204 ceph-mon[34659]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T06:09:42.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:42.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:42.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:42.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:42.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:09:42.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:09:42.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:42.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:42.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:42.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:09:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:09:42.715+0000 7f87ff59c640 1 -- 172.21.15.3:0/197358305 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f87f8063800 con 0x7f87e40b6560 2024-09-21T06:09:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:43 smithi137 ceph-mon[34464]: purged_snaps scrub starts 2024-09-21T06:09:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:43 smithi137 ceph-mon[34464]: purged_snaps scrub ok 2024-09-21T06:09:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:43 smithi137 ceph-mon[34464]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:43 smithi137 ceph-mon[34464]: Detected new or changed devices on smithi204 2024-09-21T06:09:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:43 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi204 to 8544M 2024-09-21T06:09:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:43 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:43 smithi137 ceph-mon[34464]: osd.5 [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044] boot 2024-09-21T06:09:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:43 smithi137 ceph-mon[34464]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T06:09:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:43 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:43.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:43 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:09:43.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:43 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:09:43.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:43 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:43.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:43 smithi204 ceph-mon[34659]: purged_snaps scrub starts 2024-09-21T06:09:43.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:43 smithi204 ceph-mon[34659]: purged_snaps scrub ok 2024-09-21T06:09:43.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:43 smithi204 ceph-mon[34659]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:43.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:43 smithi204 ceph-mon[34659]: Detected new or changed devices on smithi204 2024-09-21T06:09:43.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:43 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi204 to 8544M 2024-09-21T06:09:43.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:43 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:43.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:43 smithi204 ceph-mon[34659]: osd.5 [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044] boot 2024-09-21T06:09:43.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:43 smithi204 ceph-mon[34659]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T06:09:43.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:43 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:43.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:43 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:09:43.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:43 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:09:43.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:43 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:43.773 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:43 smithi003 ceph-mon[31227]: purged_snaps scrub starts 2024-09-21T06:09:43.773 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:43 smithi003 ceph-mon[31227]: purged_snaps scrub ok 2024-09-21T06:09:43.773 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:43 smithi003 ceph-mon[31227]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:09:43.773 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:43 smithi003 ceph-mon[31227]: Detected new or changed devices on smithi204 2024-09-21T06:09:43.773 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:43 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi204 to 8544M 2024-09-21T06:09:43.773 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:43 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:43.773 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:43 smithi003 ceph-mon[31227]: osd.5 [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044] boot 2024-09-21T06:09:43.773 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:43 smithi003 ceph-mon[31227]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T06:09:43.774 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:43 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:09:43.774 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:43 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:09:43.774 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:43 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:09:43.774 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:43 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:44.606 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:44 smithi003 ceph-mon[31227]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:44.606 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:44 smithi003 ceph-mon[31227]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T06:09:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:44 smithi137 ceph-mon[34464]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:44 smithi137 ceph-mon[34464]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T06:09:44.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:44 smithi204 ceph-mon[34659]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:44.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:44 smithi204 ceph-mon[34659]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T06:09:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:45 smithi137 ceph-mon[34464]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:09:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:45 smithi137 ceph-mon[34464]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T06:09:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:45 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2641197764' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ba21dbe9-ab43-4b7e-9446-2929ee874c43"} : dispatch 2024-09-21T06:09:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:45 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2641197764' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba21dbe9-ab43-4b7e-9446-2929ee874c43"}]': finished 2024-09-21T06:09:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:45 smithi137 ceph-mon[34464]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T06:09:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:45 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:09:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:45 smithi204 ceph-mon[34659]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:09:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:45 smithi204 ceph-mon[34659]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T06:09:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:45 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2641197764' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ba21dbe9-ab43-4b7e-9446-2929ee874c43"} : dispatch 2024-09-21T06:09:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:45 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2641197764' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba21dbe9-ab43-4b7e-9446-2929ee874c43"}]': finished 2024-09-21T06:09:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:45 smithi204 ceph-mon[34659]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T06:09:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:45 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:09:45.744 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:45 smithi003 ceph-mon[31227]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:09:45.744 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:45 smithi003 ceph-mon[31227]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T06:09:45.744 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:45 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2641197764' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ba21dbe9-ab43-4b7e-9446-2929ee874c43"} : dispatch 2024-09-21T06:09:45.744 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:45 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2641197764' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba21dbe9-ab43-4b7e-9446-2929ee874c43"}]': finished 2024-09-21T06:09:45.745 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:45 smithi003 ceph-mon[31227]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T06:09:45.745 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:45 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:09:46.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:46 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/4061962637' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:09:46.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:46 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/4061962637' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:09:46.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:46 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/4061962637' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:09:47.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:47 smithi137 ceph-mon[34464]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T06:09:47.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:47 smithi204 ceph-mon[34659]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T06:09:47.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:47 smithi003 ceph-mon[31227]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T06:09:49.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:49 smithi137 ceph-mon[34464]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-21T06:09:49.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:49 smithi204 ceph-mon[34659]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-21T06:09:49.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:49 smithi003 ceph-mon[31227]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-21T06:09:51.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:51 smithi137 ceph-mon[34464]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-21T06:09:51.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:51 smithi204 ceph-mon[34659]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-21T06:09:51.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:51 smithi003 ceph-mon[31227]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-21T06:09:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:53 smithi003 ceph-mon[31227]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:09:53.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:53 smithi137 ceph-mon[34464]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:09:53.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:53 smithi204 ceph-mon[34659]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:09:54.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:54 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T06:09:54.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:54 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:54.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:54 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T06:09:54.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:54 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:54 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T06:09:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:54 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:55 smithi204 ceph-mon[34659]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T06:09:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:55 smithi204 ceph-mon[34659]: Deploying daemon osd.6 on smithi003 2024-09-21T06:09:55.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:55 smithi003 ceph-mon[31227]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T06:09:55.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:55 smithi003 ceph-mon[31227]: Deploying daemon osd.6 on smithi003 2024-09-21T06:09:55.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:55 smithi137 ceph-mon[34464]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T06:09:55.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:55 smithi137 ceph-mon[34464]: Deploying daemon osd.6 on smithi003 2024-09-21T06:09:57.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:57 smithi204 ceph-mon[34659]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:09:57.753 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:57 smithi003 ceph-mon[31227]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:09:57.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:57 smithi137 ceph-mon[34464]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:09:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:09:59 smithi204 ceph-mon[34659]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:09:59.741 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:09:59 smithi003 ceph-mon[31227]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:09:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:09:59 smithi137 ceph-mon[34464]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:00.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:00 smithi003 ceph-mon[31227]: overall HEALTH_OK 2024-09-21T06:10:00.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:00 smithi137 ceph-mon[34464]: overall HEALTH_OK 2024-09-21T06:10:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:00 smithi204 ceph-mon[34659]: overall HEALTH_OK 2024-09-21T06:10:01.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:01 smithi003 ceph-mon[31227]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:01.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:01.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:01.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:01.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:01 smithi137 ceph-mon[34464]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:01.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:01.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:01.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:01.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:01 smithi204 ceph-mon[34659]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:01.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:01.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:01.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:03.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:03 smithi003 ceph-mon[31227]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:03.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:03 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:03.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:03 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:03.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:03 smithi137 ceph-mon[34464]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:03.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:03 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:03.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:03 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:03.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:03 smithi204 ceph-mon[34659]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:03.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:03 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:03.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:03 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:04.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:10:04.348+0000 7f87ee7fc640 1 -- 172.21.15.3:0/197358305 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f87f8063800 con 0x7f87e40b6560 2024-09-21T06:10:04.351 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 6 on host 'smithi003' 2024-09-21T06:10:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:10:04.351+0000 7f87ff59c640 1 -- 172.21.15.3:0/197358305 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f87e40b6560 msgr2=0x7f87e40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:10:04.351+0000 7f87ff59c640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f87e40b6560 0x7f87e40b8a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009230 tx=0x7f87f4031400 comp rx=0 tx=0).stop 2024-09-21T06:10:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:10:04.351+0000 7f87ff59c640 1 -- 172.21.15.3:0/197358305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87f810de10 msgr2=0x7f87f8103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:10:04.351+0000 7f87ff59c640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87f810de10 0x7f87f8103170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f87f000eaa0 tx=0x7f87f000ef60 comp rx=0 tx=0).stop 2024-09-21T06:10:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:10:04.351+0000 7f87ff59c640 1 -- 172.21.15.3:0/197358305 shutdown_connections 2024-09-21T06:10:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:10:04.351+0000 7f87ff59c640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f87e40b6560 0x7f87e40b8a70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:10:04.351+0000 7f87ff59c640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87f8105300 0x7f87f8102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:10:04.351+0000 7f87ff59c640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87f810de10 0x7f87f8103170 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:10:04.351+0000 7f87ff59c640 1 --2- 172.21.15.3:0/197358305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f81049b0 0x7f87f81026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:10:04.351+0000 7f87ff59c640 1 -- 172.21.15.3:0/197358305 >> 172.21.15.3:0/197358305 conn(0x7f87f8079730 msgr2=0x7f87f80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:10:04.351+0000 7f87ff59c640 1 -- 172.21.15.3:0/197358305 shutdown_connections 2024-09-21T06:10:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:10:04.351+0000 7f87ff59c640 1 -- 172.21.15.3:0/197358305 wait complete. 2024-09-21T06:10:04.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:04.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:04.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:04.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:04.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:04.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:04.839 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:04.839 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:04.839 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:04.839 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:04.839 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:04.839 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:04.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:04.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:04.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:04.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:04.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:04.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:04.985 DEBUG:teuthology.orchestra.run.smithi003:osd.6> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T06:10:04.988 INFO:tasks.cephadm:Deploying osd.7 on smithi137 with /dev/vg_nvme/lv_2... 2024-09-21T06:10:04.988 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T06:10:05.244 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:10:05.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:05 smithi003 ceph-mon[31227]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:05.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:05 smithi137 ceph-mon[34464]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:05.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:05 smithi204 ceph-mon[34659]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:06.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:06 smithi003 ceph-mon[31227]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T06:10:06.430 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:10:06 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:10:06.328+0000 7fd38de6c640 -1 osd.6 0 log_to_monitors true 2024-09-21T06:10:06.728 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:06 smithi137 ceph-mon[34464]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T06:10:06.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:06 smithi204 ceph-mon[34659]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T06:10:07.600 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:07 smithi137 ceph-mon[34464]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:07.600 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:07 smithi137 ceph-mon[34464]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T06:10:07.600 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:07 smithi137 ceph-mon[34464]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T06:10:07.600 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:07 smithi137 ceph-mon[34464]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]} : dispatch 2024-09-21T06:10:07.600 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:07 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:07.805 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:07 smithi204 ceph-mon[34659]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:07.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:07 smithi204 ceph-mon[34659]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T06:10:07.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:07 smithi204 ceph-mon[34659]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T06:10:07.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:07 smithi204 ceph-mon[34659]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]} : dispatch 2024-09-21T06:10:07.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:07 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:07.825 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:07 smithi003 ceph-mon[31227]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:07.825 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:07 smithi003 ceph-mon[31227]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T06:10:07.825 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:07 smithi003 ceph-mon[31227]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T06:10:07.825 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:07 smithi003 ceph-mon[31227]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]} : dispatch 2024-09-21T06:10:07.825 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:07 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:07.920 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:10:07.947 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi137:vg_nvme/lv_2 2024-09-21T06:10:08.090 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:10:07 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:10:07.882+0000 7fd38a603640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T06:10:08.090 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:10:07 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:10:07.893+0000 7fd381be4640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:10:08.205 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:10:08.620 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-21T06:10:08.620 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: Detected new or changed devices on smithi003 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi003 to 5697M 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:08.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:08.622 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:08 smithi137 ceph-mon[34464]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' 2024-09-21T06:10:08.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.711+0000 7f5b51bfe640 1 -- 172.21.15.137:0/1166144988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b4c108d20 msgr2=0x7f5b4c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:08.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.711+0000 7f5b51bfe640 1 --2- 172.21.15.137:0/1166144988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b4c108d20 0x7f5b4c10f550 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009ca0 tx=0x7f5b4402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:08.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.711+0000 7f5b51bfe640 1 -- 172.21.15.137:0/1166144988 shutdown_connections 2024-09-21T06:10:08.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.711+0000 7f5b51bfe640 1 --2- 172.21.15.137:0/1166144988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b4c1083d0 0x7f5b4c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:08.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.711+0000 7f5b51bfe640 1 --2- 172.21.15.137:0/1166144988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b4c108d20 0x7f5b4c10f550 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:08.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.711+0000 7f5b51bfe640 1 --2- 172.21.15.137:0/1166144988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b4c102a90 0x7f5b4c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:08.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.711+0000 7f5b51bfe640 1 -- 172.21.15.137:0/1166144988 >> 172.21.15.137:0/1166144988 conn(0x7f5b4c079750 msgr2=0x7f5b4c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:08.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.711+0000 7f5b51bfe640 1 -- 172.21.15.137:0/1166144988 shutdown_connections 2024-09-21T06:10:08.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.711+0000 7f5b51bfe640 1 -- 172.21.15.137:0/1166144988 wait complete. 2024-09-21T06:10:08.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.712+0000 7f5b51bfe640 1 Processor -- start 2024-09-21T06:10:08.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.712+0000 7f5b51bfe640 1 -- start start 2024-09-21T06:10:08.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.712+0000 7f5b51bfe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b4c102a90 0x7f5b4c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:08.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b51bfe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b4c1083d0 0x7f5b4c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:08.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b51bfe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b4c108d20 0x7f5b4c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:08.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b51bfe640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c112990 con 0x7f5b4c102a90 2024-09-21T06:10:08.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b51bfe640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c112810 con 0x7f5b4c108d20 2024-09-21T06:10:08.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b51bfe640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c112b10 con 0x7f5b4c1083d0 2024-09-21T06:10:08.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b4bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b4c108d20 0x7f5b4c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:08.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b4affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b4c1083d0 0x7f5b4c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b4affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b4c1083d0 0x7f5b4c1996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.137:52250/0 (socket says 172.21.15.137:52250) 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b4bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b4c108d20 0x7f5b4c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36768/0 (socket says 172.21.15.137:36768) 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b4affd640 1 -- 172.21.15.137:0/3496657075 learned_addr learned my addr 172.21.15.137:0/3496657075 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b4b7fe640 1 --2- 172.21.15.137:0/3496657075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b4c102a90 0x7f5b4c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b4affd640 1 -- 172.21.15.137:0/3496657075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b4c108d20 msgr2=0x7f5b4c199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b4affd640 1 --2- 172.21.15.137:0/3496657075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b4c108d20 0x7f5b4c199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b4affd640 1 -- 172.21.15.137:0/3496657075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b4c102a90 msgr2=0x7f5b4c1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b4affd640 1 --2- 172.21.15.137:0/3496657075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b4c102a90 0x7f5b4c1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b4affd640 1 -- 172.21.15.137:0/3496657075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c1a8830 con 0x7f5b4c1083d0 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b4affd640 1 --2- 172.21.15.137:0/3496657075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b4c1083d0 0x7f5b4c1996f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000e9f0 tx=0x7f5b4000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b48ff9640 1 -- 172.21.15.137:0/3496657075 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000ccc0 con 0x7f5b4c1083d0 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b51bfe640 1 -- 172.21.15.137:0/3496657075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4c1a8ac0 con 0x7f5b4c1083d0 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b48ff9640 1 -- 172.21.15.137:0/3496657075 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b400054d0 con 0x7f5b4c1083d0 2024-09-21T06:10:08.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.713+0000 7f5b48ff9640 1 -- 172.21.15.137:0/3496657075 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40010430 con 0x7f5b4c1083d0 2024-09-21T06:10:08.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.714+0000 7f5b51bfe640 1 -- 172.21.15.137:0/3496657075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4c1a8fb0 con 0x7f5b4c1083d0 2024-09-21T06:10:08.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.714+0000 7f5b51bfe640 1 -- 172.21.15.137:0/3496657075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b4c103d10 con 0x7f5b4c1083d0 2024-09-21T06:10:08.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.716+0000 7f5b48ff9640 1 -- 172.21.15.137:0/3496657075 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f5b400046c0 con 0x7f5b4c1083d0 2024-09-21T06:10:08.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.717+0000 7f5b48ff9640 1 --2- 172.21.15.137:0/3496657075 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5b340b6560 0x7f5b340b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:08.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.717+0000 7f5b48ff9640 1 -- 172.21.15.137:0/3496657075 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4900+0+0 (secure 0 0 0) 0x7f5b400e6b50 con 0x7f5b4c1083d0 2024-09-21T06:10:08.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.717+0000 7f5b4b7fe640 1 --2- 172.21.15.137:0/3496657075 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5b340b6560 0x7f5b340b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:08.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.717+0000 7f5b4b7fe640 1 --2- 172.21.15.137:0/3496657075 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5b340b6560 0x7f5b340b8a70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c079c10 tx=0x7f5b3c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:08.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.720+0000 7f5b48ff9640 1 -- 172.21.15.137:0/3496657075 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5b40056710 con 0x7f5b4c1083d0 2024-09-21T06:10:08.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-21T06:10:08.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T06:10:08.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:08.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:08.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:08.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: Detected new or changed devices on smithi003 2024-09-21T06:10:08.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:08.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:08.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:08.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:08.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:08.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi003 to 5697M 2024-09-21T06:10:08.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:08.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:08.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:08.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:08.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:08 smithi003 ceph-mon[31227]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' 2024-09-21T06:10:08.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-21T06:10:08.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T06:10:08.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:08.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:08.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:10:08.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: Detected new or changed devices on smithi003 2024-09-21T06:10:08.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:08.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:08.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:08.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:08.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:08.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi003 to 5697M 2024-09-21T06:10:08.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:08.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:08.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:08.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:08.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:08 smithi204 ceph-mon[34659]: from='osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594]' entity='osd.6' 2024-09-21T06:10:08.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:08.911+0000 7f5b51bfe640 1 -- 172.21.15.137:0/3496657075 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5b4c063800 con 0x7f5b340b6560 2024-09-21T06:10:09.557 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:09 smithi137 ceph-mon[34464]: purged_snaps scrub starts 2024-09-21T06:10:09.557 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:09 smithi137 ceph-mon[34464]: purged_snaps scrub ok 2024-09-21T06:10:09.557 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:09 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:09.557 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:09 smithi137 ceph-mon[34464]: osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594] boot 2024-09-21T06:10:09.557 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:09 smithi137 ceph-mon[34464]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T06:10:09.558 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:09 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:09.558 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:09 smithi137 ceph-mon[34464]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:09.558 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:09 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:10:09.558 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:09 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:10:09.558 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:09 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:09 smithi003 ceph-mon[31227]: purged_snaps scrub starts 2024-09-21T06:10:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:09 smithi003 ceph-mon[31227]: purged_snaps scrub ok 2024-09-21T06:10:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:09 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:09 smithi003 ceph-mon[31227]: osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594] boot 2024-09-21T06:10:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:09 smithi003 ceph-mon[31227]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T06:10:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:09 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:09.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:09 smithi003 ceph-mon[31227]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:09.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:09 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:10:09.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:09 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:10:09.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:09 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:09 smithi204 ceph-mon[34659]: purged_snaps scrub starts 2024-09-21T06:10:09.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:09 smithi204 ceph-mon[34659]: purged_snaps scrub ok 2024-09-21T06:10:09.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:09 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:09.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:09 smithi204 ceph-mon[34659]: osd.6 [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594] boot 2024-09-21T06:10:09.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:09 smithi204 ceph-mon[34659]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T06:10:09.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:09 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:10:09.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:09 smithi204 ceph-mon[34659]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:09 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:10:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:09 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:10:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:09 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:11.044 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:10 smithi137 ceph-mon[34464]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:11.045 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:10 smithi137 ceph-mon[34464]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T06:10:11.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:10 smithi204 ceph-mon[34659]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:11.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:10 smithi204 ceph-mon[34659]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T06:10:11.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:10 smithi003 ceph-mon[31227]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:11.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:10 smithi003 ceph-mon[31227]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T06:10:12.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:11 smithi204 ceph-mon[34659]: from='client.? 172.21.15.137:0/4235413378' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9610fe5c-93df-49f9-b8bd-5cb480e0871b"} : dispatch 2024-09-21T06:10:12.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:11 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9610fe5c-93df-49f9-b8bd-5cb480e0871b"} : dispatch 2024-09-21T06:10:12.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:11 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9610fe5c-93df-49f9-b8bd-5cb480e0871b"}]': finished 2024-09-21T06:10:12.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:11 smithi204 ceph-mon[34659]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T06:10:12.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:11 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:12.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:11 smithi003 ceph-mon[31227]: from='client.? 172.21.15.137:0/4235413378' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9610fe5c-93df-49f9-b8bd-5cb480e0871b"} : dispatch 2024-09-21T06:10:12.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:11 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9610fe5c-93df-49f9-b8bd-5cb480e0871b"} : dispatch 2024-09-21T06:10:12.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:11 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9610fe5c-93df-49f9-b8bd-5cb480e0871b"}]': finished 2024-09-21T06:10:12.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:11 smithi003 ceph-mon[31227]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T06:10:12.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:11 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:12.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:11 smithi137 ceph-mon[34464]: from='client.? 172.21.15.137:0/4235413378' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9610fe5c-93df-49f9-b8bd-5cb480e0871b"} : dispatch 2024-09-21T06:10:12.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:11 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9610fe5c-93df-49f9-b8bd-5cb480e0871b"} : dispatch 2024-09-21T06:10:12.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:11 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9610fe5c-93df-49f9-b8bd-5cb480e0871b"}]': finished 2024-09-21T06:10:12.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:11 smithi137 ceph-mon[34464]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T06:10:12.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:11 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:13.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:12 smithi204 ceph-mon[34659]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:13.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:12 smithi204 ceph-mon[34659]: from='client.? 172.21.15.137:0/2556573008' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:10:13.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:12 smithi003 ceph-mon[31227]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:13.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:12 smithi003 ceph-mon[31227]: from='client.? 172.21.15.137:0/2556573008' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:10:13.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:12 smithi137 ceph-mon[34464]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:13.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:12 smithi137 ceph-mon[34464]: from='client.? 172.21.15.137:0/2556573008' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:10:15.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:14 smithi204 ceph-mon[34659]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:15.156 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:14 smithi003 ceph-mon[31227]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:15.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:14 smithi137 ceph-mon[34464]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:17.046 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:16 smithi137 ceph-mon[34464]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:17.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:16 smithi204 ceph-mon[34659]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:17.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:16 smithi003 ceph-mon[31227]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:19.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:18 smithi137 ceph-mon[34464]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:19.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:18 smithi003 ceph-mon[31227]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:19.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:18 smithi204 ceph-mon[34659]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:20.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:19 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T06:10:20.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:19 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:20.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:19 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T06:10:20.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:19 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:20.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:19 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T06:10:20.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:19 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:21.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:20 smithi137 ceph-mon[34464]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:21.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:20 smithi137 ceph-mon[34464]: Deploying daemon osd.7 on smithi137 2024-09-21T06:10:21.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:20 smithi003 ceph-mon[31227]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:21.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:20 smithi003 ceph-mon[31227]: Deploying daemon osd.7 on smithi137 2024-09-21T06:10:21.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:20 smithi204 ceph-mon[34659]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:21.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:20 smithi204 ceph-mon[34659]: Deploying daemon osd.7 on smithi137 2024-09-21T06:10:23.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:22 smithi137 ceph-mon[34464]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:23.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:22 smithi003 ceph-mon[31227]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:23.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:22 smithi204 ceph-mon[34659]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:25.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:24 smithi137 ceph-mon[34464]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:25.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:24 smithi003 ceph-mon[31227]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:25.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:24 smithi204 ceph-mon[34659]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:27.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:26 smithi137 ceph-mon[34464]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:27.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:27.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:27.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:27.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:26 smithi003 ceph-mon[31227]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:27.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:27.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:27.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:27.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:26 smithi204 ceph-mon[34659]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:27.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:27.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:27.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:29.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:28 smithi137 ceph-mon[34464]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:29.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:28 smithi003 ceph-mon[31227]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:29.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:28 smithi204 ceph-mon[34659]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:30.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:30.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:30.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:30.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:30.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:30.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:30.231+0000 7f5b48ff9640 1 -- 172.21.15.137:0/3496657075 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5b4c063800 con 0x7f5b340b6560 2024-09-21T06:10:30.233 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 7 on host 'smithi137' 2024-09-21T06:10:30.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:30.233+0000 7f5b51bfe640 1 -- 172.21.15.137:0/3496657075 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5b340b6560 msgr2=0x7f5b340b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:30.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:30.233+0000 7f5b51bfe640 1 --2- 172.21.15.137:0/3496657075 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5b340b6560 0x7f5b340b8a70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c079c10 tx=0x7f5b3c00a2d0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:30.234+0000 7f5b51bfe640 1 -- 172.21.15.137:0/3496657075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b4c1083d0 msgr2=0x7f5b4c1996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:30.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:30.234+0000 7f5b51bfe640 1 --2- 172.21.15.137:0/3496657075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b4c1083d0 0x7f5b4c1996f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000e9f0 tx=0x7f5b4000edf0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:30.234+0000 7f5b51bfe640 1 -- 172.21.15.137:0/3496657075 shutdown_connections 2024-09-21T06:10:30.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:30.234+0000 7f5b51bfe640 1 --2- 172.21.15.137:0/3496657075 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5b340b6560 0x7f5b340b8a70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:30.234+0000 7f5b51bfe640 1 --2- 172.21.15.137:0/3496657075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b4c1083d0 0x7f5b4c1996f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:30.234+0000 7f5b51bfe640 1 --2- 172.21.15.137:0/3496657075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b4c108d20 0x7f5b4c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:30.234+0000 7f5b51bfe640 1 --2- 172.21.15.137:0/3496657075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b4c102a90 0x7f5b4c1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:30.234+0000 7f5b51bfe640 1 -- 172.21.15.137:0/3496657075 >> 172.21.15.137:0/3496657075 conn(0x7f5b4c079750 msgr2=0x7f5b4c1044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:30.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:30.234+0000 7f5b51bfe640 1 -- 172.21.15.137:0/3496657075 shutdown_connections 2024-09-21T06:10:30.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:10:30.234+0000 7f5b51bfe640 1 -- 172.21.15.137:0/3496657075 wait complete. 2024-09-21T06:10:30.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:30.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:30.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:30.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:30.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:30.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:30.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:30.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:30.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:30.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:30.945 DEBUG:teuthology.orchestra.run.smithi137:osd.7> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T06:10:30.947 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2024-09-21T06:10:30.948 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T06:10:31.183 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:31 smithi137 ceph-mon[34464]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:31.183 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:31.183 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:31.184 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:31.209 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:10:31.330 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:31 smithi204 ceph-mon[34659]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:31.330 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:31.330 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:31.330 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:31.339 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:31 smithi003 ceph-mon[31227]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:31.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:31.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:31.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:32.236 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:32 smithi137 ceph-mon[34464]: from='osd.7 [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:10:32.236 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:32 smithi137 ceph-mon[34464]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:10:32.237 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:10:31 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:10:31.954+0000 7f278e4d7640 -1 osd.7 0 log_to_monitors true 2024-09-21T06:10:32.291 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:32 smithi204 ceph-mon[34659]: from='osd.7 [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:10:32.291 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:32 smithi204 ceph-mon[34659]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:10:32.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:32 smithi003 ceph-mon[31227]: from='osd.7 [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:10:32.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:32 smithi003 ceph-mon[31227]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:10:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:33 smithi003 ceph-mon[31227]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:33 smithi003 ceph-mon[31227]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T06:10:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:33 smithi003 ceph-mon[31227]: from='osd.7 [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:10:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:33 smithi003 ceph-mon[31227]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T06:10:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:33 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:33 smithi003 ceph-mon[31227]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:10:33.368 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:33 smithi204 ceph-mon[34659]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:33.368 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:33 smithi204 ceph-mon[34659]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T06:10:33.368 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:33 smithi204 ceph-mon[34659]: from='osd.7 [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:10:33.368 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:33 smithi204 ceph-mon[34659]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T06:10:33.369 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:33 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:33.369 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:33 smithi204 ceph-mon[34659]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:10:33.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:33 smithi137 ceph-mon[34464]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:33.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:33 smithi137 ceph-mon[34464]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T06:10:33.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:33 smithi137 ceph-mon[34464]: from='osd.7 [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:10:33.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:33 smithi137 ceph-mon[34464]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T06:10:33.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:33 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:33.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:33 smithi137 ceph-mon[34464]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:10:33.788 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:10:33.818 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2024-09-21T06:10:33.878 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:10:33 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:10:33.475+0000 7f278ac6e640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T06:10:33.878 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:10:33 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:10:33.485+0000 7f278224f640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:10:34.079 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:10:34.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-21T06:10:34.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T06:10:34.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:34.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:34.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: from='osd.7 ' entity='osd.7' 2024-09-21T06:10:34.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:34.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:34.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:34.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:34.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:34.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:34.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:34.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:34.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:34 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:34.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-21T06:10:34.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T06:10:34.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:34.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:34.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: from='osd.7 ' entity='osd.7' 2024-09-21T06:10:34.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:34.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:34.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:34.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:34.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:34.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:34.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:34.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:34.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:34 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:34.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-21T06:10:34.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T06:10:34.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:34.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:34.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: from='osd.7 ' entity='osd.7' 2024-09-21T06:10:34.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:34.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:34.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:34.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:34.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:10:34.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:34.346 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:34.346 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:34.346 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:34 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:34.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.588+0000 7fe5ceea6640 1 -- 172.21.15.204:0/1404537692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c81094a0 msgr2=0x7fe5c81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:34.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.588+0000 7fe5ceea6640 1 --2- 172.21.15.204:0/1404537692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c81094a0 0x7fe5c81098b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009900 tx=0x7fe5b402a410 comp rx=0 tx=0).stop 2024-09-21T06:10:34.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.588+0000 7fe5ceea6640 1 -- 172.21.15.204:0/1404537692 shutdown_connections 2024-09-21T06:10:34.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.588+0000 7fe5ceea6640 1 --2- 172.21.15.204:0/1404537692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c81094a0 0x7fe5c81098b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.588+0000 7fe5ceea6640 1 --2- 172.21.15.204:0/1404537692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c8109df0 0x7fe5c8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.588+0000 7fe5ceea6640 1 --2- 172.21.15.204:0/1404537692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c8103b60 0x7fe5c8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.588+0000 7fe5ceea6640 1 -- 172.21.15.204:0/1404537692 >> 172.21.15.204:0/1404537692 conn(0x7fe5c807b850 msgr2=0x7fe5c807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:34.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.588+0000 7fe5ceea6640 1 -- 172.21.15.204:0/1404537692 shutdown_connections 2024-09-21T06:10:34.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.588+0000 7fe5ceea6640 1 -- 172.21.15.204:0/1404537692 wait complete. 2024-09-21T06:10:34.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.589+0000 7fe5ceea6640 1 Processor -- start 2024-09-21T06:10:34.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.589+0000 7fe5ceea6640 1 -- start start 2024-09-21T06:10:34.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.589+0000 7fe5ceea6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c8103b60 0x7fe5c819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:34.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.589+0000 7fe5ceea6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c81094a0 0x7fe5c819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:34.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.589+0000 7fe5ceea6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c8109df0 0x7fe5c819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:34.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.589+0000 7fe5ceea6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c8113ab0 con 0x7fe5c8103b60 2024-09-21T06:10:34.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.589+0000 7fe5ceea6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c8113930 con 0x7fe5c81094a0 2024-09-21T06:10:34.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.589+0000 7fe5ceea6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c8113c30 con 0x7fe5c8109df0 2024-09-21T06:10:34.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.589+0000 7fe5cd443640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c8109df0 0x7fe5c819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:34.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5cd443640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c8109df0 0x7fe5c819fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36562/0 (socket says 172.21.15.204:36562) 2024-09-21T06:10:34.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5cd443640 1 -- 172.21.15.204:0/3742366326 learned_addr learned my addr 172.21.15.204:0/3742366326 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-21T06:10:34.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5c7fff640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c81094a0 0x7fe5c819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:34.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5ccc42640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c8103b60 0x7fe5c819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:34.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5cd443640 1 -- 172.21.15.204:0/3742366326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c81094a0 msgr2=0x7fe5c819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:34.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5cd443640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c81094a0 0x7fe5c819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5cd443640 1 -- 172.21.15.204:0/3742366326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c8103b60 msgr2=0x7fe5c819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:34.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5cd443640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c8103b60 0x7fe5c819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5cd443640 1 -- 172.21.15.204:0/3742366326 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c81a04f0 con 0x7fe5c8109df0 2024-09-21T06:10:34.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5cd443640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c8109df0 0x7fe5c819fdc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000eaa0 tx=0x7fe5c000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:34.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5c5ffb640 1 -- 172.21.15.204:0/3742366326 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c000cdb0 con 0x7fe5c8109df0 2024-09-21T06:10:34.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5ceea6640 1 -- 172.21.15.204:0/3742366326 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c81a07e0 con 0x7fe5c8109df0 2024-09-21T06:10:34.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5c5ffb640 1 -- 172.21.15.204:0/3742366326 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5c0005020 con 0x7fe5c8109df0 2024-09-21T06:10:34.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5c5ffb640 1 -- 172.21.15.204:0/3742366326 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00116d0 con 0x7fe5c8109df0 2024-09-21T06:10:34.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.590+0000 7fe5ceea6640 1 -- 172.21.15.204:0/3742366326 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c81a6f40 con 0x7fe5c8109df0 2024-09-21T06:10:34.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.592+0000 7fe5ceea6640 1 -- 172.21.15.204:0/3742366326 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c8103f70 con 0x7fe5c8109df0 2024-09-21T06:10:34.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.593+0000 7fe5c5ffb640 1 -- 172.21.15.204:0/3742366326 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7fe5c00051d0 con 0x7fe5c8109df0 2024-09-21T06:10:34.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.594+0000 7fe5c5ffb640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe5b80b6560 0x7fe5b80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:34.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.594+0000 7fe5ccc42640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe5b80b6560 0x7fe5b80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:34.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.594+0000 7fe5c5ffb640 1 -- 172.21.15.204:0/3742366326 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5612+0+0 (secure 0 0 0) 0x7fe5c00e6db0 con 0x7fe5c8109df0 2024-09-21T06:10:34.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.595+0000 7fe5ccc42640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe5b80b6560 0x7fe5b80b8a70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005d00 tx=0x7fe5bc00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:34.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.597+0000 7fe5c5ffb640 1 -- 172.21.15.204:0/3742366326 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe5c0016030 con 0x7fe5c8109df0 2024-09-21T06:10:34.788 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:34.786+0000 7fe5ceea6640 1 -- 172.21.15.204:0/3742366326 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe5c8063800 con 0x7fe5b80b6560 2024-09-21T06:10:35.294 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:35 smithi204 ceph-mon[34659]: purged_snaps scrub starts 2024-09-21T06:10:35.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:35 smithi204 ceph-mon[34659]: purged_snaps scrub ok 2024-09-21T06:10:35.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:35 smithi204 ceph-mon[34659]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:35.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:35 smithi204 ceph-mon[34659]: Detected new or changed devices on smithi137 2024-09-21T06:10:35.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:35 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi137 to 5696M 2024-09-21T06:10:35.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:35 smithi204 ceph-mon[34659]: osd.7 [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496] boot 2024-09-21T06:10:35.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:35 smithi204 ceph-mon[34659]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T06:10:35.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:35.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:35 smithi204 ceph-mon[34659]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T06:10:35.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:10:35.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:10:35.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:35.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:35 smithi003 ceph-mon[31227]: purged_snaps scrub starts 2024-09-21T06:10:35.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:35 smithi003 ceph-mon[31227]: purged_snaps scrub ok 2024-09-21T06:10:35.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:35 smithi003 ceph-mon[31227]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:35.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:35 smithi003 ceph-mon[31227]: Detected new or changed devices on smithi137 2024-09-21T06:10:35.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:35 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi137 to 5696M 2024-09-21T06:10:35.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:35 smithi003 ceph-mon[31227]: osd.7 [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496] boot 2024-09-21T06:10:35.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:35 smithi003 ceph-mon[31227]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T06:10:35.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:35.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:35 smithi003 ceph-mon[31227]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T06:10:35.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:10:35.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:10:35.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:35 smithi137 ceph-mon[34464]: purged_snaps scrub starts 2024-09-21T06:10:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:35 smithi137 ceph-mon[34464]: purged_snaps scrub ok 2024-09-21T06:10:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:35 smithi137 ceph-mon[34464]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:10:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:35 smithi137 ceph-mon[34464]: Detected new or changed devices on smithi137 2024-09-21T06:10:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:35 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi137 to 5696M 2024-09-21T06:10:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:35 smithi137 ceph-mon[34464]: osd.7 [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496] boot 2024-09-21T06:10:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:35 smithi137 ceph-mon[34464]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T06:10:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:10:35.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:35 smithi137 ceph-mon[34464]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T06:10:35.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:10:35.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:10:35.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:36.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:36 smithi204 ceph-mon[34659]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:36.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:36 smithi204 ceph-mon[34659]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T06:10:36.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:36 smithi003 ceph-mon[31227]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:36.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:36 smithi003 ceph-mon[31227]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T06:10:36.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:36 smithi137 ceph-mon[34464]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:36.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:36 smithi137 ceph-mon[34464]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T06:10:37.296 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:37 smithi137 ceph-mon[34464]: pgmap v164: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:37.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:37 smithi003 ceph-mon[31227]: pgmap v164: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:37.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:37 smithi204 ceph-mon[34659]: pgmap v164: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:38.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:38 smithi003 ceph-mon[31227]: from='client.? 172.21.15.204:0/748454331' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "49f6e9b4-6d14-434e-8b42-8b7ffeb2a072"} : dispatch 2024-09-21T06:10:38.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:38 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "49f6e9b4-6d14-434e-8b42-8b7ffeb2a072"} : dispatch 2024-09-21T06:10:38.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:38 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49f6e9b4-6d14-434e-8b42-8b7ffeb2a072"}]': finished 2024-09-21T06:10:38.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:38 smithi003 ceph-mon[31227]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T06:10:38.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:38 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:10:38.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:38 smithi003 ceph-mon[31227]: from='client.? 172.21.15.204:0/1219696811' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:10:38.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:38 smithi137 ceph-mon[34464]: from='client.? 172.21.15.204:0/748454331' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "49f6e9b4-6d14-434e-8b42-8b7ffeb2a072"} : dispatch 2024-09-21T06:10:38.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:38 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "49f6e9b4-6d14-434e-8b42-8b7ffeb2a072"} : dispatch 2024-09-21T06:10:38.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:38 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49f6e9b4-6d14-434e-8b42-8b7ffeb2a072"}]': finished 2024-09-21T06:10:38.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:38 smithi137 ceph-mon[34464]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T06:10:38.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:38 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:10:38.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:38 smithi137 ceph-mon[34464]: from='client.? 172.21.15.204:0/1219696811' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:10:38.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:38 smithi204 ceph-mon[34659]: from='client.? 172.21.15.204:0/748454331' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "49f6e9b4-6d14-434e-8b42-8b7ffeb2a072"} : dispatch 2024-09-21T06:10:38.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:38 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "49f6e9b4-6d14-434e-8b42-8b7ffeb2a072"} : dispatch 2024-09-21T06:10:38.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:38 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49f6e9b4-6d14-434e-8b42-8b7ffeb2a072"}]': finished 2024-09-21T06:10:38.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:38 smithi204 ceph-mon[34659]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T06:10:38.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:38 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:10:38.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:38 smithi204 ceph-mon[34659]: from='client.? 172.21.15.204:0/1219696811' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:10:39.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:39 smithi003 ceph-mon[31227]: pgmap v166: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:39.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:39 smithi137 ceph-mon[34464]: pgmap v166: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:39.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:39 smithi204 ceph-mon[34659]: pgmap v166: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:41.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:41 smithi003 ceph-mon[31227]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 208 KiB/s, 0 objects/s recovering 2024-09-21T06:10:41.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:41 smithi137 ceph-mon[34464]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 208 KiB/s, 0 objects/s recovering 2024-09-21T06:10:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:41 smithi204 ceph-mon[34659]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 208 KiB/s, 0 objects/s recovering 2024-09-21T06:10:43.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:43 smithi204 ceph-mon[34659]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 156 KiB/s, 0 objects/s recovering 2024-09-21T06:10:43.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:43 smithi003 ceph-mon[31227]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 156 KiB/s, 0 objects/s recovering 2024-09-21T06:10:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:43 smithi137 ceph-mon[34464]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 156 KiB/s, 0 objects/s recovering 2024-09-21T06:10:45.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:45 smithi003 ceph-mon[31227]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-21T06:10:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:45 smithi137 ceph-mon[34464]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-21T06:10:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:45 smithi204 ceph-mon[34659]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-21T06:10:46.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:46 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T06:10:46.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:46 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:46.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:46 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T06:10:46.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:46 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:46.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:46 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T06:10:46.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:46 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:47.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:47 smithi003 ceph-mon[31227]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T06:10:47.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:47 smithi003 ceph-mon[31227]: Deploying daemon osd.8 on smithi204 2024-09-21T06:10:47.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:47 smithi137 ceph-mon[34464]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T06:10:47.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:47 smithi137 ceph-mon[34464]: Deploying daemon osd.8 on smithi204 2024-09-21T06:10:47.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:47 smithi204 ceph-mon[34659]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T06:10:47.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:47 smithi204 ceph-mon[34659]: Deploying daemon osd.8 on smithi204 2024-09-21T06:10:49.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:49 smithi003 ceph-mon[31227]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-09-21T06:10:49.626 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:49 smithi204 ceph-mon[34659]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-09-21T06:10:49.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:49 smithi137 ceph-mon[34464]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-09-21T06:10:51.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:51 smithi204 ceph-mon[34659]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:10:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:51 smithi003 ceph-mon[31227]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:10:51.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:51 smithi137 ceph-mon[34464]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:10:52.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:52.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:52.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:52.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:52.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:52.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:53 smithi003 ceph-mon[31227]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:53.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:53 smithi137 ceph-mon[34464]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:53.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:53 smithi204 ceph-mon[34659]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:55 smithi003 ceph-mon[31227]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:55 smithi137 ceph-mon[34464]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:55.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:55 smithi204 ceph-mon[34659]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:55.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:55.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:55.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:55.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:55.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:56.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:56.088+0000 7fe5c5ffb640 1 -- 172.21.15.204:0/3742366326 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe5c8063800 con 0x7fe5b80b6560 2024-09-21T06:10:56.091 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2024-09-21T06:10:56.093 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:56.091+0000 7fe5ceea6640 1 -- 172.21.15.204:0/3742366326 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe5b80b6560 msgr2=0x7fe5b80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:56.093 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:56.091+0000 7fe5ceea6640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe5b80b6560 0x7fe5b80b8a70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005d00 tx=0x7fe5bc00b410 comp rx=0 tx=0).stop 2024-09-21T06:10:56.093 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:56.091+0000 7fe5ceea6640 1 -- 172.21.15.204:0/3742366326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c8109df0 msgr2=0x7fe5c819fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:56.093 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:56.091+0000 7fe5ceea6640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c8109df0 0x7fe5c819fdc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000eaa0 tx=0x7fe5c000ef60 comp rx=0 tx=0).stop 2024-09-21T06:10:56.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:56.091+0000 7fe5ceea6640 1 -- 172.21.15.204:0/3742366326 shutdown_connections 2024-09-21T06:10:56.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:56.091+0000 7fe5ceea6640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe5b80b6560 0x7fe5b80b8a70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:56.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:56.091+0000 7fe5ceea6640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c8109df0 0x7fe5c819fdc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:56.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:56.091+0000 7fe5ceea6640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c81094a0 0x7fe5c819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:56.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:56.091+0000 7fe5ceea6640 1 --2- 172.21.15.204:0/3742366326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c8103b60 0x7fe5c819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:56.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:56.091+0000 7fe5ceea6640 1 -- 172.21.15.204:0/3742366326 >> 172.21.15.204:0/3742366326 conn(0x7fe5c807b850 msgr2=0x7fe5c8107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:56.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:56.091+0000 7fe5ceea6640 1 -- 172.21.15.204:0/3742366326 shutdown_connections 2024-09-21T06:10:56.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:10:56.092+0000 7fe5ceea6640 1 -- 172.21.15.204:0/3742366326 wait complete. 2024-09-21T06:10:56.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:56 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:56.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:56 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:56.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:56 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:56.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:56 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:56.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:56 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:56.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:56 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:56.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:56 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:56.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:56 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:56.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:56 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:10:56.878 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T06:10:56.880 INFO:tasks.cephadm:Deploying osd.9 on smithi003 with /dev/vg_nvme/lv_1... 2024-09-21T06:10:56.881 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T06:10:57.151 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:57.421 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:57 smithi204 ceph-mon[34659]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:57.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:57 smithi003 ceph-mon[31227]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:57.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:57 smithi137 ceph-mon[34464]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:58.345 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:10:58 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:10:58.222+0000 7fdc1f28b640 -1 osd.8 0 log_to_monitors true 2024-09-21T06:10:58.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:58 smithi003 ceph-mon[31227]: from='osd.8 [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:10:58.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:58 smithi003 ceph-mon[31227]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:10:58.603 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:58 smithi204 ceph-mon[34659]: from='osd.8 [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:10:58.603 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:58 smithi204 ceph-mon[34659]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:10:58.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:58 smithi137 ceph-mon[34464]: from='osd.8 [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:10:58.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:58 smithi137 ceph-mon[34464]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:10:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:59 smithi003 ceph-mon[31227]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:59 smithi003 ceph-mon[31227]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T06:10:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:59 smithi003 ceph-mon[31227]: from='osd.8 [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:10:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:59 smithi003 ceph-mon[31227]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T06:10:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:59 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:10:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:10:59 smithi003 ceph-mon[31227]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:10:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:59 smithi137 ceph-mon[34464]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:59 smithi137 ceph-mon[34464]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T06:10:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:59 smithi137 ceph-mon[34464]: from='osd.8 [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:10:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:59 smithi137 ceph-mon[34464]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T06:10:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:59 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:10:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:10:59 smithi137 ceph-mon[34464]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:10:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:59 smithi204 ceph-mon[34659]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:10:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:59 smithi204 ceph-mon[34659]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T06:10:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:59 smithi204 ceph-mon[34659]: from='osd.8 [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:10:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:59 smithi204 ceph-mon[34659]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T06:10:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:59 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:10:59.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:10:59 smithi204 ceph-mon[34659]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:10:59.775 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:10:59.802 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi003:vg_nvme/lv_1 2024-09-21T06:11:00.059 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:00.113 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:10:59 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:10:59.771+0000 7fdc1b20d640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T06:11:00.113 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:10:59 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:10:59.783+0000 7fdc13003640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:11:00.483 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-21T06:11:00.483 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T06:11:00.483 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:11:00.483 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:11:00.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: from='osd.8 ' entity='osd.8' 2024-09-21T06:11:00.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:00.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:00.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:00.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:00.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:00.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:00.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:00.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:00.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:00 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:00.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.577+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2099515349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d41083a0 msgr2=0x7f65d41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.577+0000 7f65d8ec9640 1 --2- 172.21.15.3:0/2099515349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d41083a0 0x7f65d41087b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009ca0 tx=0x7f65c802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.577+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2099515349 shutdown_connections 2024-09-21T06:11:00.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.577+0000 7f65d8ec9640 1 --2- 172.21.15.3:0/2099515349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d4108cf0 0x7f65d410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.577+0000 7f65d8ec9640 1 --2- 172.21.15.3:0/2099515349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d41083a0 0x7f65d41087b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.577+0000 7f65d8ec9640 1 --2- 172.21.15.3:0/2099515349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d4102a60 0x7f65d4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.577+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2099515349 >> 172.21.15.3:0/2099515349 conn(0x7f65d4079730 msgr2=0x7f65d4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.577+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2099515349 shutdown_connections 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.577+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2099515349 wait complete. 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.578+0000 7f65d8ec9640 1 Processor -- start 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.578+0000 7f65d8ec9640 1 -- start start 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d8ec9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d4102a60 0x7f65d410ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d8ec9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d41083a0 0x7f65d41104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d8ec9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d4108cf0 0x7f65d4110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d8ec9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4113a80 con 0x7f65d4102a60 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d8ec9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4113900 con 0x7f65d4108cf0 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d8ec9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4113c00 con 0x7f65d41083a0 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d4102a60 0x7f65d410ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d4102a60 0x7f65d410ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38344/0 (socket says 172.21.15.3:38344) 2024-09-21T06:11:00.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d259c640 1 -- 172.21.15.3:0/2530829690 learned_addr learned my addr 172.21.15.3:0/2530829690 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:11:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d2d9d640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d4108cf0 0x7f65d4110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d1d9b640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d41083a0 0x7f65d41104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d259c640 1 -- 172.21.15.3:0/2530829690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d41083a0 msgr2=0x7f65d41104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d259c640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d41083a0 0x7f65d41104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d259c640 1 -- 172.21.15.3:0/2530829690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d4108cf0 msgr2=0x7f65d4110a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d259c640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d4108cf0 0x7f65d4110a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d259c640 1 -- 172.21.15.3:0/2530829690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d41a87d0 con 0x7f65d4102a60 2024-09-21T06:11:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d2d9d640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d4108cf0 0x7f65d4110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d259c640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d4102a60 0x7f65d410ffb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f65c400d950 tx=0x7f65c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65c37fe640 1 -- 172.21.15.3:0/2530829690 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4017030 con 0x7f65d4102a60 2024-09-21T06:11:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.579+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2530829690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d41a8ac0 con 0x7f65d4102a60 2024-09-21T06:11:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.580+0000 7f65c37fe640 1 -- 172.21.15.3:0/2530829690 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c4004020 con 0x7f65d4102a60 2024-09-21T06:11:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.580+0000 7f65c37fe640 1 -- 172.21.15.3:0/2530829690 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c40047a0 con 0x7f65d4102a60 2024-09-21T06:11:00.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.580+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2530829690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d41a9050 con 0x7f65d4102a60 2024-09-21T06:11:00.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.581+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2530829690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d4102e70 con 0x7f65d4102a60 2024-09-21T06:11:00.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.582+0000 7f65c37fe640 1 -- 172.21.15.3:0/2530829690 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f65c4009a30 con 0x7f65d4102a60 2024-09-21T06:11:00.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.583+0000 7f65c37fe640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f65bc0b6560 0x7f65bc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.583+0000 7f65d1d9b640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f65bc0b6560 0x7f65bc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.583+0000 7f65c37fe640 1 -- 172.21.15.3:0/2530829690 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6077+0+0 (secure 0 0 0) 0x7f65c40e6c80 con 0x7f65d4102a60 2024-09-21T06:11:00.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.584+0000 7f65d1d9b640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f65bc0b6560 0x7f65bc0b8a70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f65d4107b40 tx=0x7f65c8009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:00.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.586+0000 7f65c37fe640 1 -- 172.21.15.3:0/2530829690 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f65c40add30 con 0x7f65d4102a60 2024-09-21T06:11:00.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-21T06:11:00.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T06:11:00.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:11:00.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:11:00.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: from='osd.8 ' entity='osd.8' 2024-09-21T06:11:00.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:00.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:00.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:00.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:00.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:00.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:00.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:00.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:00.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:00 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:00.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-21T06:11:00.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T06:11:00.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:11:00.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:11:00.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: from='osd.8 ' entity='osd.8' 2024-09-21T06:11:00.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:00.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:00.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:00.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:00.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:00.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:00.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:00.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:00.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:00 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:00.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:00.783+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2530829690 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f65d4063800 con 0x7f65bc0b6560 2024-09-21T06:11:01.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:01 smithi003 ceph-mon[31227]: purged_snaps scrub starts 2024-09-21T06:11:01.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:01 smithi003 ceph-mon[31227]: purged_snaps scrub ok 2024-09-21T06:11:01.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:01 smithi003 ceph-mon[31227]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:11:01.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:01 smithi003 ceph-mon[31227]: Detected new or changed devices on smithi204 2024-09-21T06:11:01.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:01 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi204 to 5696M 2024-09-21T06:11:01.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:11:01.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:01 smithi003 ceph-mon[31227]: osd.8 [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798] boot 2024-09-21T06:11:01.424 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:01 smithi003 ceph-mon[31227]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T06:11:01.424 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:11:01.424 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:11:01.424 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:11:01.424 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:01 smithi137 ceph-mon[34464]: purged_snaps scrub starts 2024-09-21T06:11:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:01 smithi137 ceph-mon[34464]: purged_snaps scrub ok 2024-09-21T06:11:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:01 smithi137 ceph-mon[34464]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:11:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:01 smithi137 ceph-mon[34464]: Detected new or changed devices on smithi204 2024-09-21T06:11:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:01 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi204 to 5696M 2024-09-21T06:11:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:11:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:01 smithi137 ceph-mon[34464]: osd.8 [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798] boot 2024-09-21T06:11:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:01 smithi137 ceph-mon[34464]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T06:11:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:11:01.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:11:01.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:11:01.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:01 smithi204 ceph-mon[34659]: purged_snaps scrub starts 2024-09-21T06:11:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:01 smithi204 ceph-mon[34659]: purged_snaps scrub ok 2024-09-21T06:11:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:01 smithi204 ceph-mon[34659]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:11:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:01 smithi204 ceph-mon[34659]: Detected new or changed devices on smithi204 2024-09-21T06:11:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:01 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi204 to 5696M 2024-09-21T06:11:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:11:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:01 smithi204 ceph-mon[34659]: osd.8 [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798] boot 2024-09-21T06:11:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:01 smithi204 ceph-mon[34659]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T06:11:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:11:01.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:11:01.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:11:01.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:02.586 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:02 smithi003 ceph-mon[31227]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:02.586 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:02 smithi003 ceph-mon[31227]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T06:11:02.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:02 smithi137 ceph-mon[34464]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:02.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:02 smithi137 ceph-mon[34464]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T06:11:02.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:02 smithi204 ceph-mon[34659]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:02.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:02 smithi204 ceph-mon[34659]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T06:11:03.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:03 smithi137 ceph-mon[34464]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:03.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:03 smithi137 ceph-mon[34464]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T06:11:03.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:03 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1848766045' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "633f67a7-b759-4ea6-82ed-3c42b3fb70ff"} : dispatch 2024-09-21T06:11:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:03 smithi204 ceph-mon[34659]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:03 smithi204 ceph-mon[34659]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T06:11:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:03 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1848766045' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "633f67a7-b759-4ea6-82ed-3c42b3fb70ff"} : dispatch 2024-09-21T06:11:03.700 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:03 smithi003 ceph-mon[31227]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:03.700 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:03 smithi003 ceph-mon[31227]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T06:11:03.700 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:03 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1848766045' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "633f67a7-b759-4ea6-82ed-3c42b3fb70ff"} : dispatch 2024-09-21T06:11:04.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:04 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1848766045' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "633f67a7-b759-4ea6-82ed-3c42b3fb70ff"}]': finished 2024-09-21T06:11:04.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:04 smithi204 ceph-mon[34659]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T06:11:04.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:04.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:04 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3351177954' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:11:04.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:04 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1848766045' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "633f67a7-b759-4ea6-82ed-3c42b3fb70ff"}]': finished 2024-09-21T06:11:04.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:04 smithi003 ceph-mon[31227]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T06:11:04.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:04.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:04 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3351177954' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:11:04.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:04 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1848766045' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "633f67a7-b759-4ea6-82ed-3c42b3fb70ff"}]': finished 2024-09-21T06:11:04.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:04 smithi137 ceph-mon[34464]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T06:11:04.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:04.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:04 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3351177954' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:11:05.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:05 smithi003 ceph-mon[31227]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:05.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:05 smithi137 ceph-mon[34464]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:05.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:05 smithi204 ceph-mon[34659]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:07.697 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:07 smithi137 ceph-mon[34464]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-21T06:11:07.706 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:07 smithi204 ceph-mon[34659]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-21T06:11:07.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:07 smithi003 ceph-mon[31227]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-21T06:11:09.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:09 smithi003 ceph-mon[31227]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-21T06:11:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:09 smithi137 ceph-mon[34464]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-21T06:11:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:09 smithi204 ceph-mon[34659]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-21T06:11:11.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:11 smithi003 ceph-mon[31227]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:11:11.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:11 smithi137 ceph-mon[34464]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:11:11.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:11 smithi204 ceph-mon[34659]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:11:12.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:12 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T06:11:12.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:12 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:12.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:12 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T06:11:12.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:12 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:12 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T06:11:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:12 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:13.688 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:13 smithi003 ceph-mon[31227]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T06:11:13.688 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:13 smithi003 ceph-mon[31227]: Deploying daemon osd.9 on smithi003 2024-09-21T06:11:13.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:13 smithi137 ceph-mon[34464]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T06:11:13.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:13 smithi137 ceph-mon[34464]: Deploying daemon osd.9 on smithi003 2024-09-21T06:11:13.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:13 smithi204 ceph-mon[34659]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T06:11:13.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:13 smithi204 ceph-mon[34659]: Deploying daemon osd.9 on smithi003 2024-09-21T06:11:15.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:15 smithi003 ceph-mon[31227]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T06:11:15.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:15 smithi137 ceph-mon[34464]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T06:11:15.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:15 smithi204 ceph-mon[34659]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T06:11:16.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:16 smithi003 ceph-mon[31227]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:11:16.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:16 smithi137 ceph-mon[34464]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:11:16.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:16 smithi204 ceph-mon[34659]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:11:18.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:18 smithi204 ceph-mon[34659]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:18.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:18 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:18.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:18 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:18.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:18 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:18.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:18 smithi003 ceph-mon[31227]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:18.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:18 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:18.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:18 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:18.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:18 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:19.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:18 smithi137 ceph-mon[34464]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:19.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:18 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:19.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:18 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:19.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:18 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:21.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:21 smithi003 ceph-mon[31227]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:21.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:21 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:21.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:21 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:21.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:21 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:21.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:21 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:21.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:21 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:21.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:21.472+0000 7f65c37fe640 1 -- 172.21.15.3:0/2530829690 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f65d4063800 con 0x7f65bc0b6560 2024-09-21T06:11:21.474 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 9 on host 'smithi003' 2024-09-21T06:11:21.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:21.474+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2530829690 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f65bc0b6560 msgr2=0x7f65bc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:21.474+0000 7f65d8ec9640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f65bc0b6560 0x7f65bc0b8a70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f65d4107b40 tx=0x7f65c8009360 comp rx=0 tx=0).stop 2024-09-21T06:11:21.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:21.474+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2530829690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d4102a60 msgr2=0x7f65d410ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:21.474+0000 7f65d8ec9640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d4102a60 0x7f65d410ffb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f65c400d950 tx=0x7f65c400de10 comp rx=0 tx=0).stop 2024-09-21T06:11:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:21.474+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2530829690 shutdown_connections 2024-09-21T06:11:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:21.474+0000 7f65d8ec9640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f65bc0b6560 0x7f65bc0b8a70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:21.474+0000 7f65d8ec9640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d41083a0 0x7f65d41104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:21.474+0000 7f65d8ec9640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d4108cf0 0x7f65d4110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:21.474+0000 7f65d8ec9640 1 --2- 172.21.15.3:0/2530829690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d4102a60 0x7f65d410ffb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:21.474+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2530829690 >> 172.21.15.3:0/2530829690 conn(0x7f65d4079730 msgr2=0x7f65d41052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:21.474+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2530829690 shutdown_connections 2024-09-21T06:11:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:11:21.474+0000 7f65d8ec9640 1 -- 172.21.15.3:0/2530829690 wait complete. 2024-09-21T06:11:21.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:21 smithi137 ceph-mon[34464]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:21.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:21 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:21.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:21 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:21.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:21 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:21.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:21 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:21.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:21 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:21 smithi204 ceph-mon[34659]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:21 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:21 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:21 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:21 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:21 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:22.190 DEBUG:teuthology.orchestra.run.smithi003:osd.9> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T06:11:22.192 INFO:tasks.cephadm:Deploying osd.10 on smithi137 with /dev/vg_nvme/lv_1... 2024-09-21T06:11:22.193 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T06:11:22.451 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:11:22.466 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:22.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:22.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:22.571 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:22.572 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:22.572 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:22.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:22.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:22.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:23.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:23 smithi003 ceph-mon[31227]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:23.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:23 smithi003 ceph-mon[31227]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T06:11:23.341 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:11:23 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:11:23.270+0000 7f781cfc4640 -1 osd.9 0 log_to_monitors true 2024-09-21T06:11:23.604 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:23 smithi137 ceph-mon[34464]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:23.604 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:23 smithi137 ceph-mon[34464]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T06:11:23.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:23 smithi204 ceph-mon[34659]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:23.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:23 smithi204 ceph-mon[34659]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T06:11:24.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:24 smithi003 ceph-mon[31227]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T06:11:24.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:24 smithi003 ceph-mon[31227]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T06:11:24.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:24 smithi003 ceph-mon[31227]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]} : dispatch 2024-09-21T06:11:24.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:24.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:24 smithi003 ceph-mon[31227]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:24.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:24 smithi137 ceph-mon[34464]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T06:11:24.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:24 smithi137 ceph-mon[34464]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T06:11:24.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:24 smithi137 ceph-mon[34464]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]} : dispatch 2024-09-21T06:11:24.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:24.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:24 smithi137 ceph-mon[34464]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:24.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:24 smithi204 ceph-mon[34659]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T06:11:24.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:24 smithi204 ceph-mon[34659]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T06:11:24.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:24 smithi204 ceph-mon[34659]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]} : dispatch 2024-09-21T06:11:24.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:24.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:24 smithi204 ceph-mon[34659]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:25.078 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:11:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:11:24.969+0000 7f781975b640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T06:11:25.078 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:11:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:11:24.981+0000 7f7810d3c640 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:11:25.271 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-21T06:11:25.297 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi137:vg_nvme/lv_1 2024-09-21T06:11:25.556 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:11:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-21T06:11:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T06:11:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' 2024-09-21T06:11:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: Detected new or changed devices on smithi003 2024-09-21T06:11:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:25.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:25.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi003 to 4273M 2024-09-21T06:11:25.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:25.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:25.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:25.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-21T06:11:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T06:11:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' 2024-09-21T06:11:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: Detected new or changed devices on smithi003 2024-09-21T06:11:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:25.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:25.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:25.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:25.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:25.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi003 to 4273M 2024-09-21T06:11:25.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:25.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:25.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:25.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-21T06:11:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T06:11:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549]' entity='osd.9' 2024-09-21T06:11:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: Detected new or changed devices on smithi003 2024-09-21T06:11:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi003 to 4273M 2024-09-21T06:11:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:26.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.048+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/4217741759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b441049b0 msgr2=0x7f4b44104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:26.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.048+0000 7f4b4bfeb640 1 --2- 172.21.15.137:0/4217741759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b441049b0 0x7f4b44104dc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009ca0 tx=0x7f4b3402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:26.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.048+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/4217741759 shutdown_connections 2024-09-21T06:11:26.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.048+0000 7f4b4bfeb640 1 --2- 172.21.15.137:0/4217741759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b44105300 0x7f4b4410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:26.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.048+0000 7f4b4bfeb640 1 --2- 172.21.15.137:0/4217741759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b4410de10 0x7f4b44110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:26.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.048+0000 7f4b4bfeb640 1 --2- 172.21.15.137:0/4217741759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b441049b0 0x7f4b44104dc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:26.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.048+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/4217741759 >> 172.21.15.137:0/4217741759 conn(0x7f4b44079730 msgr2=0x7f4b44079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:26.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.048+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/4217741759 shutdown_connections 2024-09-21T06:11:26.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.048+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/4217741759 wait complete. 2024-09-21T06:11:26.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.049+0000 7f4b4bfeb640 1 Processor -- start 2024-09-21T06:11:26.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.049+0000 7f4b4bfeb640 1 -- start start 2024-09-21T06:11:26.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.049+0000 7f4b4bfeb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b441049b0 0x7f4b4419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:26.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.049+0000 7f4b4bfeb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b44105300 0x7f4b4419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:26.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.049+0000 7f4b4bfeb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4410de10 0x7f4b4419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:26.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.049+0000 7f4b4bfeb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44113a70 con 0x7f4b441049b0 2024-09-21T06:11:26.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.049+0000 7f4b4bfeb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4b441138f0 con 0x7f4b44105300 2024-09-21T06:11:26.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.049+0000 7f4b4bfeb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44113bf0 con 0x7f4b4410de10 2024-09-21T06:11:26.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.049+0000 7f4b49586640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b44105300 0x7f4b4419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:26.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.049+0000 7f4b49586640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b44105300 0x7f4b4419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59534/0 (socket says 172.21.15.137:59534) 2024-09-21T06:11:26.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.049+0000 7f4b49586640 1 -- 172.21.15.137:0/3125456141 learned_addr learned my addr 172.21.15.137:0/3125456141 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-21T06:11:26.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.049+0000 7f4b49d87640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b441049b0 0x7f4b4419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:26.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.050+0000 7f4b49586640 1 -- 172.21.15.137:0/3125456141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4410de10 msgr2=0x7f4b4419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:26.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.050+0000 7f4b4a588640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4410de10 0x7f4b4419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:26.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.050+0000 7f4b49586640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4410de10 0x7f4b4419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:26.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.050+0000 7f4b49586640 1 -- 172.21.15.137:0/3125456141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b441049b0 msgr2=0x7f4b4419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:26.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.050+0000 7f4b49586640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b441049b0 0x7f4b4419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:26.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.050+0000 7f4b49586640 1 -- 172.21.15.137:0/3125456141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b441a0470 con 0x7f4b44105300 2024-09-21T06:11:26.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.050+0000 7f4b4a588640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4410de10 0x7f4b4419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:26.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.050+0000 7f4b49586640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b44105300 0x7f4b4419aaa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800d950 tx=0x7f4b3800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:26.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.050+0000 7f4b42ffd640 1 -- 172.21.15.137:0/3125456141 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38017070 con 0x7f4b44105300 2024-09-21T06:11:26.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.050+0000 7f4b42ffd640 1 -- 172.21.15.137:0/3125456141 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b38009c30 con 0x7f4b44105300 2024-09-21T06:11:26.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.050+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/3125456141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b441a0760 con 0x7f4b44105300 2024-09-21T06:11:26.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.050+0000 7f4b42ffd640 1 -- 172.21.15.137:0/3125456141 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b380047a0 con 0x7f4b44105300 2024-09-21T06:11:26.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.050+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/3125456141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b441a6ec0 con 0x7f4b44105300 2024-09-21T06:11:26.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.051+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/3125456141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b4406c2a0 con 0x7f4b44105300 2024-09-21T06:11:26.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.053+0000 7f4b42ffd640 1 -- 172.21.15.137:0/3125456141 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f4b38004020 con 0x7f4b44105300 2024-09-21T06:11:26.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.053+0000 7f4b42ffd640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4b2c0b65e0 0x7f4b2c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:26.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.053+0000 7f4b42ffd640 1 -- 172.21.15.137:0/3125456141 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6476+0+0 (secure 0 0 0) 0x7f4b380e6c90 con 0x7f4b44105300 2024-09-21T06:11:26.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.054+0000 7f4b49d87640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4b2c0b65e0 0x7f4b2c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:26.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.054+0000 7f4b49d87640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4b2c0b65e0 0x7f4b2c0b8af0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005e30 tx=0x7f4b34009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:26.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.057+0000 7f4b42ffd640 1 -- 172.21.15.137:0/3125456141 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4b38020020 con 0x7f4b44105300 2024-09-21T06:11:26.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:26.245+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/3125456141 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4b44063800 con 0x7f4b2c0b65e0 2024-09-21T06:11:26.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:26 smithi137 ceph-mon[34464]: purged_snaps scrub starts 2024-09-21T06:11:26.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:26 smithi137 ceph-mon[34464]: purged_snaps scrub ok 2024-09-21T06:11:26.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:26.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:26 smithi137 ceph-mon[34464]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:26.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:26 smithi137 ceph-mon[34464]: osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549] boot 2024-09-21T06:11:26.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:26 smithi137 ceph-mon[34464]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T06:11:26.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:26.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:26 smithi137 ceph-mon[34464]: from='client.24542 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:26.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:11:26.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:11:26.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:26.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:26 smithi003 ceph-mon[31227]: purged_snaps scrub starts 2024-09-21T06:11:26.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:26 smithi003 ceph-mon[31227]: purged_snaps scrub ok 2024-09-21T06:11:26.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:26.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:26 smithi003 ceph-mon[31227]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:26 smithi003 ceph-mon[31227]: osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549] boot 2024-09-21T06:11:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:26 smithi003 ceph-mon[31227]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T06:11:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:26 smithi003 ceph-mon[31227]: from='client.24542 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:11:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:11:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:26 smithi204 ceph-mon[34659]: purged_snaps scrub starts 2024-09-21T06:11:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:26 smithi204 ceph-mon[34659]: purged_snaps scrub ok 2024-09-21T06:11:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:26 smithi204 ceph-mon[34659]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:11:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:26 smithi204 ceph-mon[34659]: osd.9 [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549] boot 2024-09-21T06:11:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:26 smithi204 ceph-mon[34659]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T06:11:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:11:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:26 smithi204 ceph-mon[34659]: from='client.24542 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:26.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:11:26.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:11:26.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:28.216 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:27 smithi137 ceph-mon[34464]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T06:11:28.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:27 smithi003 ceph-mon[31227]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T06:11:28.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:27 smithi204 ceph-mon[34659]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T06:11:29.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:28 smithi137 ceph-mon[34464]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:29.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:28 smithi137 ceph-mon[34464]: from='client.? 172.21.15.137:0/1174391458' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94b65a48-ee87-4e59-9296-a72a22787f03"} : dispatch 2024-09-21T06:11:29.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:28 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94b65a48-ee87-4e59-9296-a72a22787f03"} : dispatch 2024-09-21T06:11:29.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:28 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94b65a48-ee87-4e59-9296-a72a22787f03"}]': finished 2024-09-21T06:11:29.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:28 smithi137 ceph-mon[34464]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T06:11:29.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:29.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:28 smithi003 ceph-mon[31227]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:29.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:28 smithi003 ceph-mon[31227]: from='client.? 172.21.15.137:0/1174391458' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94b65a48-ee87-4e59-9296-a72a22787f03"} : dispatch 2024-09-21T06:11:29.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:28 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94b65a48-ee87-4e59-9296-a72a22787f03"} : dispatch 2024-09-21T06:11:29.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:28 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94b65a48-ee87-4e59-9296-a72a22787f03"}]': finished 2024-09-21T06:11:29.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:28 smithi003 ceph-mon[31227]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T06:11:29.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:29.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:28 smithi204 ceph-mon[34659]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:29.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:28 smithi204 ceph-mon[34659]: from='client.? 172.21.15.137:0/1174391458' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94b65a48-ee87-4e59-9296-a72a22787f03"} : dispatch 2024-09-21T06:11:29.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:28 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94b65a48-ee87-4e59-9296-a72a22787f03"} : dispatch 2024-09-21T06:11:29.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:28 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94b65a48-ee87-4e59-9296-a72a22787f03"}]': finished 2024-09-21T06:11:29.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:28 smithi204 ceph-mon[34659]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T06:11:29.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:30.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:30 smithi003 ceph-mon[31227]: from='client.? 172.21.15.137:0/2120557676' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:11:30.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:30 smithi137 ceph-mon[34464]: from='client.? 172.21.15.137:0/2120557676' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:11:30.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:30 smithi204 ceph-mon[34659]: from='client.? 172.21.15.137:0/2120557676' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:11:31.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:31 smithi003 ceph-mon[31227]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:31.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:31 smithi137 ceph-mon[34464]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:31.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:31 smithi204 ceph-mon[34659]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:33 smithi003 ceph-mon[31227]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:33.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:33 smithi137 ceph-mon[34464]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:33.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:33 smithi204 ceph-mon[34659]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:35.306 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:35 smithi137 ceph-mon[34464]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:35.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:35 smithi003 ceph-mon[31227]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:35.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:35 smithi204 ceph-mon[34659]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:37.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:37 smithi003 ceph-mon[31227]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:37.375 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:37 smithi137 ceph-mon[34464]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:37.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:37 smithi204 ceph-mon[34659]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:38.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:38 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T06:11:38.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:38 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:38.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:38 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T06:11:38.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:38 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:38.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:38 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T06:11:38.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:38 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:39.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:39 smithi137 ceph-mon[34464]: Deploying daemon osd.10 on smithi137 2024-09-21T06:11:39.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:39 smithi137 ceph-mon[34464]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:39.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:39 smithi003 ceph-mon[31227]: Deploying daemon osd.10 on smithi137 2024-09-21T06:11:39.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:39 smithi003 ceph-mon[31227]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:39.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:39 smithi204 ceph-mon[34659]: Deploying daemon osd.10 on smithi137 2024-09-21T06:11:39.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:39 smithi204 ceph-mon[34659]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:41.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:41 smithi003 ceph-mon[31227]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:41.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:41 smithi137 ceph-mon[34464]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:41.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:41 smithi204 ceph-mon[34659]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:43.290 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:43 smithi137 ceph-mon[34464]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:43.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:43 smithi003 ceph-mon[31227]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:43.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:43 smithi204 ceph-mon[34659]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:44.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:44 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:44.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:44 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:44.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:44 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:44.363 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:44 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:44.363 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:44 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:44.363 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:44 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:44.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:44 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:44.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:44 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:44.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:44 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:45.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:45 smithi137 ceph-mon[34464]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:45.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:45 smithi003 ceph-mon[31227]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:45.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:45 smithi204 ceph-mon[34659]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:47.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:47 smithi137 ceph-mon[34464]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:47.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:47 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:47.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:47 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:47.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:47 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:47.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:47 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:47.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:47 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:47.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:47.315+0000 7f4b42ffd640 1 -- 172.21.15.137:0/3125456141 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4b44063800 con 0x7f4b2c0b65e0 2024-09-21T06:11:47.317 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 10 on host 'smithi137' 2024-09-21T06:11:47.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:47.318+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/3125456141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4b2c0b65e0 msgr2=0x7f4b2c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:47.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:47.318+0000 7f4b4bfeb640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4b2c0b65e0 0x7f4b2c0b8af0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005e30 tx=0x7f4b34009360 comp rx=0 tx=0).stop 2024-09-21T06:11:47.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:47.318+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/3125456141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b44105300 msgr2=0x7f4b4419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:47.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:47.318+0000 7f4b4bfeb640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b44105300 0x7f4b4419aaa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800d950 tx=0x7f4b3800de10 comp rx=0 tx=0).stop 2024-09-21T06:11:47.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:47.318+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/3125456141 shutdown_connections 2024-09-21T06:11:47.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:47.318+0000 7f4b4bfeb640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4b2c0b65e0 0x7f4b2c0b8af0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:47.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:47.318+0000 7f4b4bfeb640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4410de10 0x7f4b4419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:47.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:47.318+0000 7f4b4bfeb640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b44105300 0x7f4b4419aaa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:47.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:47.318+0000 7f4b4bfeb640 1 --2- 172.21.15.137:0/3125456141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b441049b0 0x7f4b4419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:47.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:47.318+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/3125456141 >> 172.21.15.137:0/3125456141 conn(0x7f4b44079730 msgr2=0x7f4b441008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:47.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:47.318+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/3125456141 shutdown_connections 2024-09-21T06:11:47.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-21T06:11:47.318+0000 7f4b4bfeb640 1 -- 172.21.15.137:0/3125456141 wait complete. 2024-09-21T06:11:47.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:47 smithi003 ceph-mon[31227]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:47.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:47 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:47.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:47 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:47.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:47 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:47.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:47 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:47.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:47 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:47.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:47 smithi204 ceph-mon[34659]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:47.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:47 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:47.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:47 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:47.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:47 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:47.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:47 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:47.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:47 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:47.971 DEBUG:teuthology.orchestra.run.smithi137:osd.10> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T06:11:47.973 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2024-09-21T06:11:47.973 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T06:11:48.219 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:48 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:48.219 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:48 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:48.219 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:48 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:48.235 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:11:48.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:48 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:48.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:48 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:48.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:48 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:48.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:48 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:48.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:48 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:48.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:48 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:49.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:49 smithi003 ceph-mon[31227]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:49.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:49 smithi003 ceph-mon[31227]: from='osd.10 [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:11:49.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:49 smithi003 ceph-mon[31227]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:11:49.593 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:49 smithi137 ceph-mon[34464]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:49.593 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:49 smithi137 ceph-mon[34464]: from='osd.10 [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:11:49.593 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:49 smithi137 ceph-mon[34464]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:11:49.593 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:11:49 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:11:49.293+0000 7fccc2ab8640 -1 osd.10 0 log_to_monitors true 2024-09-21T06:11:49.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:49 smithi204 ceph-mon[34659]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:49.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:49 smithi204 ceph-mon[34659]: from='osd.10 [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:11:49.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:49 smithi204 ceph-mon[34659]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:11:50.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:50 smithi003 ceph-mon[31227]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T06:11:50.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:50 smithi003 ceph-mon[31227]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T06:11:50.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:50 smithi003 ceph-mon[31227]: from='osd.10 [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:11:50.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:50.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:50 smithi003 ceph-mon[31227]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:11:50.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:50 smithi003 ceph-mon[31227]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-21T06:11:50.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:50 smithi003 ceph-mon[31227]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T06:11:50.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:50 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:50.619 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:50 smithi204 ceph-mon[34659]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T06:11:50.619 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:50 smithi204 ceph-mon[34659]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T06:11:50.619 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:50 smithi204 ceph-mon[34659]: from='osd.10 [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:11:50.619 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:50.619 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:50 smithi204 ceph-mon[34659]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:11:50.619 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:50 smithi204 ceph-mon[34659]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-21T06:11:50.619 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:50 smithi204 ceph-mon[34659]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T06:11:50.619 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:50 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:50.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:11:50 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:11:50.326+0000 7fccbea3a640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T06:11:50.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:11:50 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:11:50.338+0000 7fccba838640 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:11:50.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:50 smithi137 ceph-mon[34464]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T06:11:50.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:50 smithi137 ceph-mon[34464]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T06:11:50.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:50 smithi137 ceph-mon[34464]: from='osd.10 [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:11:50.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:50.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:50 smithi137 ceph-mon[34464]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]} : dispatch 2024-09-21T06:11:50.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:50 smithi137 ceph-mon[34464]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-21T06:11:50.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:50 smithi137 ceph-mon[34464]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T06:11:50.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:50 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-21T06:11:51.103 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2024-09-21T06:11:51.359 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:11:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: osd.10 [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683] boot 2024-09-21T06:11:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T06:11:51.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:51.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:51.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:51.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:51.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:51.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:51.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:51.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:51.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:51.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:51.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:51.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:51.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:51.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: osd.10 [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683] boot 2024-09-21T06:11:51.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T06:11:51.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:51.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:11:51.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:51.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:51.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:51.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:51.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:51.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:51.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:11:51.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:51.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:51.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:51.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:11:51.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: osd.10 [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683] boot 2024-09-21T06:11:51.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T06:11:51.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:11:51.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.875+0000 7fc0232fb640 1 -- 172.21.15.204:0/2517772914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc01c102a60 msgr2=0x7fc01c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:51.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.875+0000 7fc0232fb640 1 --2- 172.21.15.204:0/2517772914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc01c102a60 0x7fc01c102e70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc018009900 tx=0x7fc01802a410 comp rx=0 tx=0).stop 2024-09-21T06:11:51.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.875+0000 7fc0232fb640 1 -- 172.21.15.204:0/2517772914 shutdown_connections 2024-09-21T06:11:51.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.875+0000 7fc0232fb640 1 --2- 172.21.15.204:0/2517772914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc01c108cf0 0x7fc01c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:51.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.875+0000 7fc0232fb640 1 --2- 172.21.15.204:0/2517772914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc01c1083a0 0x7fc01c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:51.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.875+0000 7fc0232fb640 1 --2- 172.21.15.204:0/2517772914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc01c102a60 0x7fc01c102e70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:51.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.875+0000 7fc0232fb640 1 -- 172.21.15.204:0/2517772914 >> 172.21.15.204:0/2517772914 conn(0x7fc01c079730 msgr2=0x7fc01c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:51.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.875+0000 7fc0232fb640 1 -- 172.21.15.204:0/2517772914 shutdown_connections 2024-09-21T06:11:51.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.875+0000 7fc0232fb640 1 -- 172.21.15.204:0/2517772914 wait complete. 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.876+0000 7fc0232fb640 1 Processor -- start 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.876+0000 7fc0232fb640 1 -- start start 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.876+0000 7fc0232fb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc01c102a60 0x7fc01c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.876+0000 7fc0232fb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc01c1083a0 0x7fc01c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.876+0000 7fc0232fb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc01c108cf0 0x7fc01c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.876+0000 7fc0232fb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c112960 con 0x7fc01c108cf0 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.876+0000 7fc0232fb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1127e0 con 0x7fc01c1083a0 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.876+0000 7fc0232fb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c112ae0 con 0x7fc01c102a60 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.876+0000 7fc021097640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc01c102a60 0x7fc01c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc021097640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc01c102a60 0x7fc01c19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47678/0 (socket says 172.21.15.204:47678) 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc021097640 1 -- 172.21.15.204:0/2879441571 learned_addr learned my addr 172.21.15.204:0/2879441571 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc021898640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc01c108cf0 0x7fc01c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc020896640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc01c1083a0 0x7fc01c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:51.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc021097640 1 -- 172.21.15.204:0/2879441571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc01c1083a0 msgr2=0x7fc01c19b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc021097640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc01c1083a0 0x7fc01c19b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc021097640 1 -- 172.21.15.204:0/2879441571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc01c108cf0 msgr2=0x7fc01c19bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc021097640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc01c108cf0 0x7fc01c19bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc021097640 1 -- 172.21.15.204:0/2879441571 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c1a65c0 con 0x7fc01c102a60 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc021898640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc01c108cf0 0x7fc01c19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc021097640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc01c102a60 0x7fc01c19b330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc018005940 tx=0x7fc0180042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc00e7fc640 1 -- 172.21.15.204:0/2879441571 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01803f030 con 0x7fc01c102a60 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc0232fb640 1 -- 172.21.15.204:0/2879441571 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c199120 con 0x7fc01c102a60 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc00e7fc640 1 -- 172.21.15.204:0/2879441571 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc018003cb0 con 0x7fc01c102a60 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc00e7fc640 1 -- 172.21.15.204:0/2879441571 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018035070 con 0x7fc01c102a60 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.877+0000 7fc0232fb640 1 -- 172.21.15.204:0/2879441571 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c199600 con 0x7fc01c102a60 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.878+0000 7fc0232fb640 1 -- 172.21.15.204:0/2879441571 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c102e70 con 0x7fc01c102a60 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.880+0000 7fc00e7fc640 1 -- 172.21.15.204:0/2879441571 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7fc018042030 con 0x7fc01c102a60 2024-09-21T06:11:51.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.881+0000 7fc00e7fc640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc0080b6590 0x7fc0080b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:51.887 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.881+0000 7fc020896640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc0080b6590 0x7fc0080b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:51.887 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.881+0000 7fc00e7fc640 1 -- 172.21.15.204:0/2879441571 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6942+0+0 (secure 0 0 0) 0x7fc018107750 con 0x7fc01c102a60 2024-09-21T06:11:51.887 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.882+0000 7fc020896640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc0080b6590 0x7fc0080b8aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc010005fd0 tx=0x7fc010005e20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:51.887 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:51.884+0000 7fc00e7fc640 1 -- 172.21.15.204:0/2879441571 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc0180ce4a0 con 0x7fc01c102a60 2024-09-21T06:11:52.071 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:11:52.069+0000 7fc0232fb640 1 -- 172.21.15.204:0/2879441571 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc01c063800 con 0x7fc0080b6590 2024-09-21T06:11:52.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:52 smithi003 ceph-mon[31227]: purged_snaps scrub starts 2024-09-21T06:11:52.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:52 smithi003 ceph-mon[31227]: purged_snaps scrub ok 2024-09-21T06:11:52.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:52 smithi003 ceph-mon[31227]: Detected new or changed devices on smithi137 2024-09-21T06:11:52.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:52 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi137 to 4272M 2024-09-21T06:11:52.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:11:52.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:11:52.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:52.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:52 smithi003 ceph-mon[31227]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T06:11:52.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:52 smithi204 ceph-mon[34659]: purged_snaps scrub starts 2024-09-21T06:11:52.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:52 smithi204 ceph-mon[34659]: purged_snaps scrub ok 2024-09-21T06:11:52.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:52 smithi204 ceph-mon[34659]: Detected new or changed devices on smithi137 2024-09-21T06:11:52.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:52 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi137 to 4272M 2024-09-21T06:11:52.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:11:52.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:11:52.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:52.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:52 smithi204 ceph-mon[34659]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T06:11:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:52 smithi137 ceph-mon[34464]: purged_snaps scrub starts 2024-09-21T06:11:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:52 smithi137 ceph-mon[34464]: purged_snaps scrub ok 2024-09-21T06:11:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:52 smithi137 ceph-mon[34464]: Detected new or changed devices on smithi137 2024-09-21T06:11:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:52 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi137 to 4272M 2024-09-21T06:11:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:11:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:11:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:52 smithi137 ceph-mon[34464]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T06:11:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:53 smithi003 ceph-mon[31227]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:11:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:53 smithi003 ceph-mon[31227]: from='client.24568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:53 smithi003 ceph-mon[31227]: osdmap e67: 11 total, 11 up, 11 in 2024-09-21T06:11:53.595 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:53 smithi204 ceph-mon[34659]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:11:53.595 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:53 smithi204 ceph-mon[34659]: from='client.24568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:53.595 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:53 smithi204 ceph-mon[34659]: osdmap e67: 11 total, 11 up, 11 in 2024-09-21T06:11:53.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:53 smithi137 ceph-mon[34464]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:11:53.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:53 smithi137 ceph-mon[34464]: from='client.24568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:53.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:53 smithi137 ceph-mon[34464]: osdmap e67: 11 total, 11 up, 11 in 2024-09-21T06:11:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:55 smithi137 ceph-mon[34464]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:11:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:55 smithi137 ceph-mon[34464]: from='client.? 172.21.15.204:0/3853125634' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5cc2850-9062-4b8e-beb3-bef0db30a410"} : dispatch 2024-09-21T06:11:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:55 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5cc2850-9062-4b8e-beb3-bef0db30a410"} : dispatch 2024-09-21T06:11:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:55 smithi137 ceph-mon[34464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5cc2850-9062-4b8e-beb3-bef0db30a410"}]': finished 2024-09-21T06:11:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:55 smithi137 ceph-mon[34464]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T06:11:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:11:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:55 smithi137 ceph-mon[34464]: from='client.? 172.21.15.204:0/1240312535' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:11:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:55 smithi204 ceph-mon[34659]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:11:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:55 smithi204 ceph-mon[34659]: from='client.? 172.21.15.204:0/3853125634' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5cc2850-9062-4b8e-beb3-bef0db30a410"} : dispatch 2024-09-21T06:11:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:55 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5cc2850-9062-4b8e-beb3-bef0db30a410"} : dispatch 2024-09-21T06:11:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:55 smithi204 ceph-mon[34659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5cc2850-9062-4b8e-beb3-bef0db30a410"}]': finished 2024-09-21T06:11:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:55 smithi204 ceph-mon[34659]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T06:11:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:11:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:55 smithi204 ceph-mon[34659]: from='client.? 172.21.15.204:0/1240312535' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:11:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:55 smithi003 ceph-mon[31227]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:11:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:55 smithi003 ceph-mon[31227]: from='client.? 172.21.15.204:0/3853125634' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5cc2850-9062-4b8e-beb3-bef0db30a410"} : dispatch 2024-09-21T06:11:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:55 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5cc2850-9062-4b8e-beb3-bef0db30a410"} : dispatch 2024-09-21T06:11:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:55 smithi003 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5cc2850-9062-4b8e-beb3-bef0db30a410"}]': finished 2024-09-21T06:11:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:55 smithi003 ceph-mon[31227]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T06:11:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:11:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:55 smithi003 ceph-mon[31227]: from='client.? 172.21.15.204:0/1240312535' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:11:57.614 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:57 smithi003 ceph-mon[31227]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-21T06:11:57.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:57 smithi137 ceph-mon[34464]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-21T06:11:57.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:57 smithi204 ceph-mon[34659]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-21T06:11:59.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:11:59 smithi204 ceph-mon[34659]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:11:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:11:59 smithi137 ceph-mon[34464]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:11:59.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:11:59 smithi003 ceph-mon[31227]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:12:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:01 smithi137 ceph-mon[34464]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T06:12:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:01 smithi204 ceph-mon[34659]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T06:12:01.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:01 smithi003 ceph-mon[31227]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T06:12:03.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:03 smithi137 ceph-mon[34464]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-21T06:12:03.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:03 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T06:12:03.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:03 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:03 smithi204 ceph-mon[34659]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-21T06:12:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:03 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T06:12:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:03 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:03.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:03 smithi003 ceph-mon[31227]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-21T06:12:03.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:03 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T06:12:03.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:03 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:04.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:04 smithi204 ceph-mon[34659]: Deploying daemon osd.11 on smithi204 2024-09-21T06:12:04.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:04 smithi137 ceph-mon[34464]: Deploying daemon osd.11 on smithi204 2024-09-21T06:12:04.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:04 smithi003 ceph-mon[31227]: Deploying daemon osd.11 on smithi204 2024-09-21T06:12:05.618 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:05 smithi204 ceph-mon[34659]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:12:05.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:05 smithi137 ceph-mon[34464]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:12:05.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:05 smithi003 ceph-mon[31227]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:12:07.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:07 smithi137 ceph-mon[34464]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:07.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:07 smithi204 ceph-mon[34659]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:07.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:07 smithi003 ceph-mon[31227]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:09.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:09 smithi137 ceph-mon[34464]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:09.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:09 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:12:09.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:09 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:09.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:09 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:09 smithi204 ceph-mon[34659]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:09 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:12:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:09 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:09 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:09 smithi003 ceph-mon[31227]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:09 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:12:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:09 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:09 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:11 smithi137 ceph-mon[34464]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:11 smithi204 ceph-mon[34659]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:11.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:11 smithi003 ceph-mon[31227]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:12.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:12.987+0000 7fc00e7fc640 1 -- 172.21.15.204:0/2879441571 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc01c063800 con 0x7fc0080b6590 2024-09-21T06:12:12.989 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2024-09-21T06:12:12.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:12.989+0000 7fc0232fb640 1 -- 172.21.15.204:0/2879441571 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc0080b6590 msgr2=0x7fc0080b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:12.989+0000 7fc0232fb640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc0080b6590 0x7fc0080b8aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc010005fd0 tx=0x7fc010005e20 comp rx=0 tx=0).stop 2024-09-21T06:12:12.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:12.989+0000 7fc0232fb640 1 -- 172.21.15.204:0/2879441571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc01c102a60 msgr2=0x7fc01c19b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:12.989+0000 7fc0232fb640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc01c102a60 0x7fc01c19b330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc018005940 tx=0x7fc0180042f0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:12.989+0000 7fc0232fb640 1 -- 172.21.15.204:0/2879441571 shutdown_connections 2024-09-21T06:12:12.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:12.989+0000 7fc0232fb640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc0080b6590 0x7fc0080b8aa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:12.989+0000 7fc0232fb640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc01c108cf0 0x7fc01c19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:12.989+0000 7fc0232fb640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc01c1083a0 0x7fc01c19b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:12.989+0000 7fc0232fb640 1 --2- 172.21.15.204:0/2879441571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc01c102a60 0x7fc01c19b330 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:12.989+0000 7fc0232fb640 1 -- 172.21.15.204:0/2879441571 >> 172.21.15.204:0/2879441571 conn(0x7fc01c079730 msgr2=0x7fc01c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:12.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:12.989+0000 7fc0232fb640 1 -- 172.21.15.204:0/2879441571 shutdown_connections 2024-09-21T06:12:12.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:12.989+0000 7fc0232fb640 1 -- 172.21.15.204:0/2879441571 wait complete. 2024-09-21T06:12:13.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:13 smithi137 ceph-mon[34464]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:13.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:13 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:13.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:13 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:13.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:13 smithi204 ceph-mon[34659]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:13.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:13 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:13.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:13 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:13.704 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T06:12:13.706 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T06:12:13.706 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T06:12:13.736 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:13 smithi003 ceph-mon[31227]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:13.736 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:13 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:13.736 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:13 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:13.983 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.497+0000 7f17114ca640 1 -- 172.21.15.3:0/48766528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f170c10de90 msgr2=0x7f170c110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.497+0000 7f17114ca640 1 --2- 172.21.15.3:0/48766528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f170c10de90 0x7f170c110380 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1704009ca0 tx=0x7f170402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.497+0000 7f17114ca640 1 -- 172.21.15.3:0/48766528 shutdown_connections 2024-09-21T06:12:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.497+0000 7f17114ca640 1 --2- 172.21.15.3:0/48766528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f170c105380 0x7f170c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.497+0000 7f17114ca640 1 --2- 172.21.15.3:0/48766528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f170c10de90 0x7f170c110380 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.497+0000 7f17114ca640 1 --2- 172.21.15.3:0/48766528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f170c104a30 0x7f170c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.497+0000 7f17114ca640 1 -- 172.21.15.3:0/48766528 >> 172.21.15.3:0/48766528 conn(0x7f170c079770 msgr2=0x7f170c079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.497+0000 7f17114ca640 1 -- 172.21.15.3:0/48766528 shutdown_connections 2024-09-21T06:12:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.498+0000 7f17114ca640 1 -- 172.21.15.3:0/48766528 wait complete. 2024-09-21T06:12:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.498+0000 7f17114ca640 1 Processor -- start 2024-09-21T06:12:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.498+0000 7f17114ca640 1 -- start start 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f17114ca640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f170c104a30 0x7f170c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f17114ca640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f170c105380 0x7f170c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f17114ca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f170c10de90 0x7f170c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f17114ca640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f170c113b10 con 0x7f170c104a30 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f17114ca640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f170c113990 con 0x7f170c10de90 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f17114ca640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f170c113c90 con 0x7f170c105380 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f170affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f170c104a30 0x7f170c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f170affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f170c104a30 0x7f170c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43368/0 (socket says 172.21.15.3:43368) 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f170a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f170c105380 0x7f170c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f170affd640 1 -- 172.21.15.3:0/3319408193 learned_addr learned my addr 172.21.15.3:0/3319408193 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f170b7fe640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f170c10de90 0x7f170c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f170affd640 1 -- 172.21.15.3:0/3319408193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f170c105380 msgr2=0x7f170c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f170affd640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f170c105380 0x7f170c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f170affd640 1 -- 172.21.15.3:0/3319408193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f170c10de90 msgr2=0x7f170c19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f170affd640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f170c10de90 0x7f170c19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f170affd640 1 -- 172.21.15.3:0/3319408193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c1a0350 con 0x7f170c104a30 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f170b7fe640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f170c10de90 0x7f170c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.499+0000 7f170affd640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f170c104a30 0x7f170c19a4a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00c920 tx=0x7f16fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.500+0000 7f16f3fff640 1 -- 172.21.15.3:0/3319408193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc007c70 con 0x7f170c104a30 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.500+0000 7f17114ca640 1 -- 172.21.15.3:0/3319408193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170c1a0640 con 0x7f170c104a30 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.500+0000 7f16f3fff640 1 -- 172.21.15.3:0/3319408193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16fc007e10 con 0x7f170c104a30 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.500+0000 7f16f3fff640 1 -- 172.21.15.3:0/3319408193 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc018820 con 0x7f170c104a30 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.500+0000 7f17114ca640 1 -- 172.21.15.3:0/3319408193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f170c1a6e60 con 0x7f170c104a30 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.501+0000 7f17114ca640 1 -- 172.21.15.3:0/3319408193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f170c06c260 con 0x7f170c104a30 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.504+0000 7f16f3fff640 1 -- 172.21.15.3:0/3319408193 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f16fc00f450 con 0x7f170c104a30 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.504+0000 7f16f3fff640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f16f40b6590 0x7f16f40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.504+0000 7f170a7fc640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f16f40b6590 0x7f16f40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.505+0000 7f16f3fff640 1 -- 172.21.15.3:0/3319408193 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7024+0+0 (secure 0 0 0) 0x7f16fc0e7210 con 0x7f170c104a30 2024-09-21T06:12:14.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.505+0000 7f170a7fc640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f16f40b6590 0x7f16f40b8aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f170c10f680 tx=0x7f170000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:14.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.508+0000 7f16f3fff640 1 -- 172.21.15.3:0/3319408193 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f16fc014030 con 0x7f170c104a30 2024-09-21T06:12:14.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.699+0000 7f17114ca640 1 -- 172.21.15.3:0/3319408193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f170c104e40 con 0x7f170c104a30 2024-09-21T06:12:14.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.699+0000 7f16f3fff640 1 -- 172.21.15.3:0/3319408193 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f16fc0ae040 con 0x7f170c104a30 2024-09-21T06:12:14.702 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:12:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.702+0000 7f17114ca640 1 -- 172.21.15.3:0/3319408193 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f16f40b6590 msgr2=0x7f16f40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.702+0000 7f17114ca640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f16f40b6590 0x7f16f40b8aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f170c10f680 tx=0x7f170000b390 comp rx=0 tx=0).stop 2024-09-21T06:12:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.702+0000 7f17114ca640 1 -- 172.21.15.3:0/3319408193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f170c104a30 msgr2=0x7f170c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.702+0000 7f17114ca640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f170c104a30 0x7f170c19a4a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00c920 tx=0x7f16fc00cde0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.702+0000 7f17114ca640 1 -- 172.21.15.3:0/3319408193 shutdown_connections 2024-09-21T06:12:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.702+0000 7f17114ca640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f16f40b6590 0x7f16f40b8aa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.702+0000 7f17114ca640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f170c105380 0x7f170c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.702+0000 7f17114ca640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f170c10de90 0x7f170c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.702+0000 7f17114ca640 1 --2- 172.21.15.3:0/3319408193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f170c104a30 0x7f170c19a4a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.702+0000 7f17114ca640 1 -- 172.21.15.3:0/3319408193 >> 172.21.15.3:0/3319408193 conn(0x7f170c079770 msgr2=0x7f170c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.702+0000 7f17114ca640 1 -- 172.21.15.3:0/3319408193 shutdown_connections 2024-09-21T06:12:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:14.702+0000 7f17114ca640 1 -- 172.21.15.3:0/3319408193 wait complete. 2024-09-21T06:12:15.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:14 smithi003 ceph-mon[31227]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:15.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:14 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:15.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:14 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:15.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:14 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3319408193' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:12:15.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:14 smithi137 ceph-mon[34464]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:15.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:14 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:15.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:14 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:15.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:14 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3319408193' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:12:15.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:14 smithi204 ceph-mon[34659]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:15.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:14 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:15.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:14 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:15.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:14 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3319408193' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:12:15.492 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:12:15 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:12:15.413+0000 7f38cf3b5640 -1 osd.11 0 log_to_monitors true 2024-09-21T06:12:15.603 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726899111,"num_in_osds":12,"osd_in_since":1726899114,"num_remapped_pgs":0} 2024-09-21T06:12:15.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:15 smithi003 ceph-mon[31227]: from='osd.11 [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:12:15.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:15 smithi003 ceph-mon[31227]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:12:16.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:15 smithi137 ceph-mon[34464]: from='osd.11 [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:12:16.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:15 smithi137 ceph-mon[34464]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:12:16.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:15 smithi204 ceph-mon[34659]: from='osd.11 [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:12:16.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:15 smithi204 ceph-mon[34659]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:12:16.604 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T06:12:16.880 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:16.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:16 smithi204 ceph-mon[34659]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:16.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:16 smithi204 ceph-mon[34659]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T06:12:16.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:16 smithi204 ceph-mon[34659]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T06:12:16.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:16 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:12:16.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:16 smithi204 ceph-mon[34659]: from='osd.11 [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:12:16.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:16 smithi204 ceph-mon[34659]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:12:16.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:12:16 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:12:16.782+0000 7f38cb337640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T06:12:16.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:12:16 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:12:16.796+0000 7f38c7135640 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:12:17.001 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:16 smithi003 ceph-mon[31227]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:17.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:16 smithi003 ceph-mon[31227]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T06:12:17.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:16 smithi003 ceph-mon[31227]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T06:12:17.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:16 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:12:17.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:16 smithi003 ceph-mon[31227]: from='osd.11 [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:12:17.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:16 smithi003 ceph-mon[31227]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:12:17.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:16 smithi137 ceph-mon[34464]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:17.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:16 smithi137 ceph-mon[34464]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T06:12:17.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:16 smithi137 ceph-mon[34464]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T06:12:17.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:16 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:12:17.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:16 smithi137 ceph-mon[34464]: from='osd.11 [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:12:17.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:16 smithi137 ceph-mon[34464]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]} : dispatch 2024-09-21T06:12:17.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.363+0000 7f15a225c640 1 -- 172.21.15.3:0/3193158323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c1094a0 msgr2=0x7f159c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.363+0000 7f15a225c640 1 --2- 172.21.15.3:0/3193158323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c1094a0 0x7f159c1098b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1590009900 tx=0x7f159002a410 comp rx=0 tx=0).stop 2024-09-21T06:12:17.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.363+0000 7f15a225c640 1 -- 172.21.15.3:0/3193158323 shutdown_connections 2024-09-21T06:12:17.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.363+0000 7f15a225c640 1 --2- 172.21.15.3:0/3193158323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f159c109df0 0x7f159c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.363+0000 7f15a225c640 1 --2- 172.21.15.3:0/3193158323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c1094a0 0x7f159c1098b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.363+0000 7f15a225c640 1 --2- 172.21.15.3:0/3193158323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f159c103b60 0x7f159c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.363+0000 7f15a225c640 1 -- 172.21.15.3:0/3193158323 >> 172.21.15.3:0/3193158323 conn(0x7f159c07b850 msgr2=0x7f159c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.363+0000 7f15a225c640 1 -- 172.21.15.3:0/3193158323 shutdown_connections 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.363+0000 7f15a225c640 1 -- 172.21.15.3:0/3193158323 wait complete. 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f15a225c640 1 Processor -- start 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f15a225c640 1 -- start start 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f15a225c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f159c103b60 0x7f159c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f15a225c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c1094a0 0x7f159c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f15a225c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f159c109df0 0x7f159c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f15a225c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f159c113ab0 con 0x7f159c1094a0 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f15a225c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f159c113930 con 0x7f159c109df0 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f15a225c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f159c113c30 con 0x7f159c103b60 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f159affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c1094a0 0x7f159c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f159affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c1094a0 0x7f159c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50926/0 (socket says 172.21.15.3:50926) 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f159affd640 1 -- 172.21.15.3:0/4110527994 learned_addr learned my addr 172.21.15.3:0/4110527994 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f159b7fe640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f159c103b60 0x7f159c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f159bfff640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f159c109df0 0x7f159c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f159affd640 1 -- 172.21.15.3:0/4110527994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f159c103b60 msgr2=0x7f159c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.364+0000 7f159affd640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f159c103b60 0x7f159c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.365+0000 7f159affd640 1 -- 172.21.15.3:0/4110527994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f159c109df0 msgr2=0x7f159c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.365+0000 7f159affd640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f159c109df0 0x7f159c19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.365+0000 7f159affd640 1 -- 172.21.15.3:0/4110527994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159c1a03f0 con 0x7f159c1094a0 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.365+0000 7f159affd640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c1094a0 0x7f159c19aa20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1590006010 tx=0x7f1590004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.365+0000 7f1598ff9640 1 -- 172.21.15.3:0/4110527994 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159003f030 con 0x7f159c1094a0 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.365+0000 7f1598ff9640 1 -- 172.21.15.3:0/4110527994 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1590003aa0 con 0x7f159c1094a0 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.365+0000 7f1598ff9640 1 -- 172.21.15.3:0/4110527994 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590035070 con 0x7f159c1094a0 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.365+0000 7f15a225c640 1 -- 172.21.15.3:0/4110527994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159c1a0680 con 0x7f159c1094a0 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.365+0000 7f15a225c640 1 -- 172.21.15.3:0/4110527994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f159c1a6e40 con 0x7f159c1094a0 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.366+0000 7f15a225c640 1 -- 172.21.15.3:0/4110527994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f159c103f70 con 0x7f159c1094a0 2024-09-21T06:12:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.368+0000 7f1598ff9640 1 -- 172.21.15.3:0/4110527994 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f1590042030 con 0x7f159c1094a0 2024-09-21T06:12:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.369+0000 7f1598ff9640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f15840b65e0 0x7f15840b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.369+0000 7f159b7fe640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f15840b65e0 0x7f15840b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.369+0000 7f1598ff9640 1 -- 172.21.15.3:0/4110527994 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7062+0+0 (secure 0 0 0) 0x7f1590107760 con 0x7f159c1094a0 2024-09-21T06:12:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.369+0000 7f159b7fe640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f15840b65e0 0x7f15840b8af0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f158c005d00 tx=0x7f158c00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.372+0000 7f1598ff9640 1 -- 172.21.15.3:0/4110527994 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15900ce470 con 0x7f159c1094a0 2024-09-21T06:12:17.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.562+0000 7f15a225c640 1 -- 172.21.15.3:0/4110527994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f159c19b5b0 con 0x7f159c1094a0 2024-09-21T06:12:17.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.562+0000 7f1598ff9640 1 -- 172.21.15.3:0/4110527994 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f15900d2850 con 0x7f159c1094a0 2024-09-21T06:12:17.564 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:12:17.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.565+0000 7f15a225c640 1 -- 172.21.15.3:0/4110527994 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f15840b65e0 msgr2=0x7f15840b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.565+0000 7f15a225c640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f15840b65e0 0x7f15840b8af0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f158c005d00 tx=0x7f158c00b410 comp rx=0 tx=0).stop 2024-09-21T06:12:17.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.565+0000 7f15a225c640 1 -- 172.21.15.3:0/4110527994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c1094a0 msgr2=0x7f159c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.565+0000 7f15a225c640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c1094a0 0x7f159c19aa20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1590006010 tx=0x7f1590004060 comp rx=0 tx=0).stop 2024-09-21T06:12:17.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.565+0000 7f15a225c640 1 -- 172.21.15.3:0/4110527994 shutdown_connections 2024-09-21T06:12:17.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.565+0000 7f15a225c640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f15840b65e0 0x7f15840b8af0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.565+0000 7f15a225c640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f159c109df0 0x7f159c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.565+0000 7f15a225c640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c1094a0 0x7f159c19aa20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.565+0000 7f15a225c640 1 --2- 172.21.15.3:0/4110527994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f159c103b60 0x7f159c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.565+0000 7f15a225c640 1 -- 172.21.15.3:0/4110527994 >> 172.21.15.3:0/4110527994 conn(0x7f159c07b850 msgr2=0x7f159c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:17.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.565+0000 7f15a225c640 1 -- 172.21.15.3:0/4110527994 shutdown_connections 2024-09-21T06:12:17.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:17.565+0000 7f15a225c640 1 -- 172.21.15.3:0/4110527994 wait complete. 2024-09-21T06:12:17.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-21T06:12:17.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: osdmap e70: 12 total, 11 up, 12 in 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: Detected new or changed devices on smithi204 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi204 to 4272M 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:12:17.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:17.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:12:17.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:17.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:12:17.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:17.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:17 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/4110527994' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:12:18.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-21T06:12:18.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: osdmap e70: 12 total, 11 up, 12 in 2024-09-21T06:12:18.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:12:18.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:12:18.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: Detected new or changed devices on smithi204 2024-09-21T06:12:18.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:18.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:18.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:12:18.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:12:18.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:12:18.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:12:18.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi204 to 4272M 2024-09-21T06:12:18.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:18.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:18.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:12:18.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:18.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:12:18.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:18.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:12:18.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:18.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:17 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/4110527994' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:12:18.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-21T06:12:18.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: osdmap e70: 12 total, 11 up, 12 in 2024-09-21T06:12:18.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:12:18.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:12:18.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: Detected new or changed devices on smithi204 2024-09-21T06:12:18.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi204 to 4272M 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:18.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:17 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/4110527994' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:12:18.473 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1726899111,"num_in_osds":12,"osd_in_since":1726899114,"num_remapped_pgs":0} 2024-09-21T06:12:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:18 smithi003 ceph-mon[31227]: purged_snaps scrub starts 2024-09-21T06:12:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:18 smithi003 ceph-mon[31227]: purged_snaps scrub ok 2024-09-21T06:12:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:18 smithi003 ceph-mon[31227]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:18 smithi003 ceph-mon[31227]: osd.11 [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610] boot 2024-09-21T06:12:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:18 smithi003 ceph-mon[31227]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T06:12:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:18 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:12:19.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:18 smithi137 ceph-mon[34464]: purged_snaps scrub starts 2024-09-21T06:12:19.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:18 smithi137 ceph-mon[34464]: purged_snaps scrub ok 2024-09-21T06:12:19.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:18 smithi137 ceph-mon[34464]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:19.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:18 smithi137 ceph-mon[34464]: osd.11 [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610] boot 2024-09-21T06:12:19.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:18 smithi137 ceph-mon[34464]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T06:12:19.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:18 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:12:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:18 smithi204 ceph-mon[34659]: purged_snaps scrub starts 2024-09-21T06:12:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:18 smithi204 ceph-mon[34659]: purged_snaps scrub ok 2024-09-21T06:12:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:18 smithi204 ceph-mon[34659]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:12:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:18 smithi204 ceph-mon[34659]: osd.11 [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610] boot 2024-09-21T06:12:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:18 smithi204 ceph-mon[34659]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T06:12:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:18 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:12:19.474 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T06:12:19.745 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:20.120 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:20 smithi003 ceph-mon[31227]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T06:12:20.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.206+0000 7f56e2449640 1 -- 172.21.15.3:0/3666768053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56dc108d20 msgr2=0x7f56dc10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.206+0000 7f56e2449640 1 --2- 172.21.15.3:0/3666768053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56dc108d20 0x7f56dc10f550 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009ca0 tx=0x7f56d402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.206+0000 7f56e2449640 1 -- 172.21.15.3:0/3666768053 shutdown_connections 2024-09-21T06:12:20.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.206+0000 7f56e2449640 1 --2- 172.21.15.3:0/3666768053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56dc1083d0 0x7f56dc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.206+0000 7f56e2449640 1 --2- 172.21.15.3:0/3666768053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56dc108d20 0x7f56dc10f550 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.206+0000 7f56e2449640 1 --2- 172.21.15.3:0/3666768053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56dc102ae0 0x7f56dc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.206+0000 7f56e2449640 1 -- 172.21.15.3:0/3666768053 >> 172.21.15.3:0/3666768053 conn(0x7f56dc079760 msgr2=0x7f56dc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:20.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.206+0000 7f56e2449640 1 -- 172.21.15.3:0/3666768053 shutdown_connections 2024-09-21T06:12:20.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.206+0000 7f56e2449640 1 -- 172.21.15.3:0/3666768053 wait complete. 2024-09-21T06:12:20.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e2449640 1 Processor -- start 2024-09-21T06:12:20.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e2449640 1 -- start start 2024-09-21T06:12:20.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e2449640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56dc102ae0 0x7f56dc1a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e2449640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56dc1083d0 0x7f56dc1a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e2449640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56dc108d20 0x7f56dc1a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e2449640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc11a0b0 con 0x7f56dc108d20 2024-09-21T06:12:20.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e2449640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc119f30 con 0x7f56dc102ae0 2024-09-21T06:12:20.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e2449640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc11a230 con 0x7f56dc1083d0 2024-09-21T06:12:20.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e09e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56dc108d20 0x7f56dc1a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e09e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56dc108d20 0x7f56dc1a6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50950/0 (socket says 172.21.15.3:50950) 2024-09-21T06:12:20.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e09e6640 1 -- 172.21.15.3:0/2655123297 learned_addr learned my addr 172.21.15.3:0/2655123297 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56dbfff640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56dc102ae0 0x7f56dc1a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56db7fe640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56dc1083d0 0x7f56dc1a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e09e6640 1 -- 172.21.15.3:0/2655123297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56dc1083d0 msgr2=0x7f56dc1a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e09e6640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56dc1083d0 0x7f56dc1a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e09e6640 1 -- 172.21.15.3:0/2655123297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56dc102ae0 msgr2=0x7f56dc1a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e09e6640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56dc102ae0 0x7f56dc1a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.207+0000 7f56e09e6640 1 -- 172.21.15.3:0/2655123297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc1a68d0 con 0x7f56dc108d20 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.208+0000 7f56db7fe640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56dc1083d0 0x7f56dc1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.208+0000 7f56e09e6640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56dc108d20 0x7f56dc1a6260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f56d4005cc0 tx=0x7f56d402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.208+0000 7f56d97fa640 1 -- 172.21.15.3:0/2655123297 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4003cf0 con 0x7f56dc108d20 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.208+0000 7f56d97fa640 1 -- 172.21.15.3:0/2655123297 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56d40044f0 con 0x7f56dc108d20 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.208+0000 7f56d97fa640 1 -- 172.21.15.3:0/2655123297 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d402fc30 con 0x7f56dc108d20 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.208+0000 7f56e2449640 1 -- 172.21.15.3:0/2655123297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56dc1a6b60 con 0x7f56dc108d20 2024-09-21T06:12:20.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.208+0000 7f56e2449640 1 -- 172.21.15.3:0/2655123297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56dc1ad3e0 con 0x7f56dc108d20 2024-09-21T06:12:20.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.209+0000 7f56e2449640 1 -- 172.21.15.3:0/2655123297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56dc103d10 con 0x7f56dc108d20 2024-09-21T06:12:20.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.211+0000 7f56d97fa640 1 -- 172.21.15.3:0/2655123297 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f56d40046a0 con 0x7f56dc108d20 2024-09-21T06:12:20.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.212+0000 7f56d97fa640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f56c40b6600 0x7f56c40b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.212+0000 7f56dbfff640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f56c40b6600 0x7f56c40b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.212+0000 7f56d97fa640 1 -- 172.21.15.3:0/2655123297 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f56d4108560 con 0x7f56dc108d20 2024-09-21T06:12:20.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.212+0000 7f56dbfff640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f56c40b6600 0x7f56c40b8b10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00abe0 tx=0x7f56cc00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:20.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.215+0000 7f56d97fa640 1 -- 172.21.15.3:0/2655123297 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f56d40cf1c0 con 0x7f56dc108d20 2024-09-21T06:12:20.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:20 smithi137 ceph-mon[34464]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T06:12:20.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.402+0000 7f56e2449640 1 -- 172.21.15.3:0/2655123297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f56dc10ab40 con 0x7f56dc108d20 2024-09-21T06:12:20.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:20 smithi204 ceph-mon[34659]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T06:12:20.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.402+0000 7f56d97fa640 1 -- 172.21.15.3:0/2655123297 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f56d40d35a0 con 0x7f56dc108d20 2024-09-21T06:12:20.405 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:12:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.405+0000 7f56e2449640 1 -- 172.21.15.3:0/2655123297 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f56c40b6600 msgr2=0x7f56c40b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.405+0000 7f56e2449640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f56c40b6600 0x7f56c40b8b10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00abe0 tx=0x7f56cc00a2d0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.405+0000 7f56e2449640 1 -- 172.21.15.3:0/2655123297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56dc108d20 msgr2=0x7f56dc1a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.405+0000 7f56e2449640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56dc108d20 0x7f56dc1a6260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f56d4005cc0 tx=0x7f56d402f470 comp rx=0 tx=0).stop 2024-09-21T06:12:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.405+0000 7f56e2449640 1 -- 172.21.15.3:0/2655123297 shutdown_connections 2024-09-21T06:12:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.405+0000 7f56e2449640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f56c40b6600 0x7f56c40b8b10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.405+0000 7f56e2449640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56dc1083d0 0x7f56dc1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.405+0000 7f56e2449640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56dc108d20 0x7f56dc1a6260 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.405+0000 7f56e2449640 1 --2- 172.21.15.3:0/2655123297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56dc102ae0 0x7f56dc1a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.405+0000 7f56e2449640 1 -- 172.21.15.3:0/2655123297 >> 172.21.15.3:0/2655123297 conn(0x7f56dc079760 msgr2=0x7f56dc1043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.405+0000 7f56e2449640 1 -- 172.21.15.3:0/2655123297 shutdown_connections 2024-09-21T06:12:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:20.405+0000 7f56e2449640 1 -- 172.21.15.3:0/2655123297 wait complete. 2024-09-21T06:12:21.322 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:21 smithi003 ceph-mon[31227]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:21.323 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:21 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2655123297' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:12:21.355 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1726899137,"num_in_osds":12,"osd_in_since":1726899114,"num_remapped_pgs":0} 2024-09-21T06:12:21.356 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:12:21.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:21 smithi137 ceph-mon[34464]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:21.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:21 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2655123297' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:12:21.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:21 smithi204 ceph-mon[34659]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:21.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:21 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2655123297' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:12:21.614 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:22.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.090+0000 7f39d0a8e640 1 -- 172.21.15.3:0/12097778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39cc110140 msgr2=0x7f39cc112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.090+0000 7f39d0a8e640 1 --2- 172.21.15.3:0/12097778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39cc110140 0x7f39cc112560 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009900 tx=0x7f39c402a410 comp rx=0 tx=0).stop 2024-09-21T06:12:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.090+0000 7f39d0a8e640 1 -- 172.21.15.3:0/12097778 shutdown_connections 2024-09-21T06:12:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.090+0000 7f39d0a8e640 1 --2- 172.21.15.3:0/12097778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39cc076080 0x7f39cc10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.090+0000 7f39d0a8e640 1 --2- 172.21.15.3:0/12097778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39cc110140 0x7f39cc112560 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.090+0000 7f39d0a8e640 1 --2- 172.21.15.3:0/12097778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc075730 0x7f39cc075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.090+0000 7f39d0a8e640 1 -- 172.21.15.3:0/12097778 >> 172.21.15.3:0/12097778 conn(0x7f39cc07b850 msgr2=0x7f39cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.090+0000 7f39d0a8e640 1 -- 172.21.15.3:0/12097778 shutdown_connections 2024-09-21T06:12:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.090+0000 7f39d0a8e640 1 -- 172.21.15.3:0/12097778 wait complete. 2024-09-21T06:12:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39d0a8e640 1 Processor -- start 2024-09-21T06:12:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39d0a8e640 1 -- start start 2024-09-21T06:12:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39d0a8e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39cc075730 0x7f39cc19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39d0a8e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc076080 0x7f39cc19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39d0a8e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39cc110140 0x7f39cc1a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39d0a8e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc115ca0 con 0x7f39cc075730 2024-09-21T06:12:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39d0a8e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc115b20 con 0x7f39cc076080 2024-09-21T06:12:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39d0a8e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc115e20 con 0x7f39cc110140 2024-09-21T06:12:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39ca59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39cc075730 0x7f39cc19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39ca59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39cc075730 0x7f39cc19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50966/0 (socket says 172.21.15.3:50966) 2024-09-21T06:12:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39ca59c640 1 -- 172.21.15.3:0/2639190456 learned_addr learned my addr 172.21.15.3:0/2639190456 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39c9d9b640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc076080 0x7f39cc19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39cad9d640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39cc110140 0x7f39cc1a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39ca59c640 1 -- 172.21.15.3:0/2639190456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39cc110140 msgr2=0x7f39cc1a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39ca59c640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39cc110140 0x7f39cc1a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39ca59c640 1 -- 172.21.15.3:0/2639190456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc076080 msgr2=0x7f39cc19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39ca59c640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc076080 0x7f39cc19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.091+0000 7f39ca59c640 1 -- 172.21.15.3:0/2639190456 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc1a25c0 con 0x7f39cc075730 2024-09-21T06:12:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.092+0000 7f39ca59c640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39cc075730 0x7f39cc19c650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00eab0 tx=0x7f39bc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.092+0000 7f39c37fe640 1 -- 172.21.15.3:0/2639190456 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc00ce40 con 0x7f39cc075730 2024-09-21T06:12:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.092+0000 7f39c37fe640 1 -- 172.21.15.3:0/2639190456 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39bc011920 con 0x7f39cc075730 2024-09-21T06:12:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.092+0000 7f39d0a8e640 1 -- 172.21.15.3:0/2639190456 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39cc1a28b0 con 0x7f39cc075730 2024-09-21T06:12:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.092+0000 7f39c37fe640 1 -- 172.21.15.3:0/2639190456 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc010a30 con 0x7f39cc075730 2024-09-21T06:12:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.092+0000 7f39d0a8e640 1 -- 172.21.15.3:0/2639190456 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39cc1a9010 con 0x7f39cc075730 2024-09-21T06:12:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.093+0000 7f39d0a8e640 1 -- 172.21.15.3:0/2639190456 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39cc075b40 con 0x7f39cc075730 2024-09-21T06:12:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.094+0000 7f39c37fe640 1 -- 172.21.15.3:0/2639190456 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f39bc010bd0 con 0x7f39cc075730 2024-09-21T06:12:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.095+0000 7f39c37fe640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f39b80b6560 0x7f39b80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.096+0000 7f39c9d9b640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f39b80b6560 0x7f39b80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.096+0000 7f39c37fe640 1 -- 172.21.15.3:0/2639190456 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f39bc0e7510 con 0x7f39cc075730 2024-09-21T06:12:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.096+0000 7f39c9d9b640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f39b80b6560 0x7f39b80b8a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f39cc111d80 tx=0x7f39b400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.100+0000 7f39c37fe640 1 -- 172.21.15.3:0/2639190456 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39bc016030 con 0x7f39cc075730 2024-09-21T06:12:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.287+0000 7f39d0a8e640 1 -- 172.21.15.3:0/2639190456 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39cc1a92a0 con 0x7f39cc075730 2024-09-21T06:12:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.288+0000 7f39c37fe640 1 -- 172.21.15.3:0/2639190456 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18741 (secure 0 0 0) 0x7f39bc0ae170 con 0x7f39cc075730 2024-09-21T06:12:22.290 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:12:22.290 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":72,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","created":"2024-09-21T06:04:24.293786+0000","modified":"2024-09-21T06:12:19.038483+0000","last_up_change":"2024-09-21T06:12:17.781243+0000","last_in_change":"2024-09-21T06:11:54.727435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:08:26.015803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51736f81-f410-48ee-a8f3-121289607309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6803","nonce":615312328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6805","nonce":615312328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6809","nonce":615312328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6807","nonce":615312328}]},"public_addr":"172.21.15.3:6803/615312328","cluster_addr":"172.21.15.3:6805/615312328","heartbeat_back_addr":"172.21.15.3:6809/615312328","heartbeat_front_addr":"172.21.15.3:6807/615312328","state":["exists","up"]},{"osd":1,"uuid":"55696a37-34c1-482a-8561-7d08f0aa29fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6801","nonce":4150747115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6803","nonce":4150747115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6807","nonce":4150747115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6805","nonce":4150747115}]},"public_addr":"172.21.15.137:6801/4150747115","cluster_addr":"172.21.15.137:6803/4150747115","heartbeat_back_addr":"172.21.15.137:6807/4150747115","heartbeat_front_addr":"172.21.15.137:6805/4150747115","state":["exists","up"]},{"osd":2,"uuid":"8a81d34b-464b-46a2-8ef2-5223b5947004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6801","nonce":3392577077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6803","nonce":3392577077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6807","nonce":3392577077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6805","nonce":3392577077}]},"public_addr":"172.21.15.204:6801/3392577077","cluster_addr":"172.21.15.204:6803/3392577077","heartbeat_back_addr":"172.21.15.204:6807/3392577077","heartbeat_front_addr":"172.21.15.204:6805/3392577077","state":["exists","up"]},{"osd":3,"uuid":"94ab3938-7a70-44cb-99bf-9d2dedc4eb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6811","nonce":4293741635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6813","nonce":4293741635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6817","nonce":4293741635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6815","nonce":4293741635}]},"public_addr":"172.21.15.3:6811/4293741635","cluster_addr":"172.21.15.3:6813/4293741635","heartbeat_back_addr":"172.21.15.3:6817/4293741635","heartbeat_front_addr":"172.21.15.3:6815/4293741635","state":["exists","up"]},{"osd":4,"uuid":"38551535-d692-401c-aad0-bee38f3026d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6809","nonce":2608988785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6811","nonce":2608988785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6815","nonce":2608988785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6813","nonce":2608988785}]},"public_addr":"172.21.15.137:6809/2608988785","cluster_addr":"172.21.15.137:6811/2608988785","heartbeat_back_addr":"172.21.15.137:6815/2608988785","heartbeat_front_addr":"172.21.15.137:6813/2608988785","state":["exists","up"]},{"osd":5,"uuid":"3d4b785b-28d0-4654-a9b1-4c5c905f4bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6809","nonce":3709074044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6811","nonce":3709074044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6815","nonce":3709074044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6813","nonce":3709074044}]},"public_addr":"172.21.15.204:6809/3709074044","cluster_addr":"172.21.15.204:6811/3709074044","heartbeat_back_addr":"172.21.15.204:6815/3709074044","heartbeat_front_addr":"172.21.15.204:6813/3709074044","state":["exists","up"]},{"osd":6,"uuid":"ba21dbe9-ab43-4b7e-9446-2929ee874c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6819","nonce":3862646594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6821","nonce":3862646594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6825","nonce":3862646594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6823","nonce":3862646594}]},"public_addr":"172.21.15.3:6819/3862646594","cluster_addr":"172.21.15.3:6821/3862646594","heartbeat_back_addr":"172.21.15.3:6825/3862646594","heartbeat_front_addr":"172.21.15.3:6823/3862646594","state":["exists","up"]},{"osd":7,"uuid":"9610fe5c-93df-49f9-b8bd-5cb480e0871b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6817","nonce":2086822496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6819","nonce":2086822496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6823","nonce":2086822496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6821","nonce":2086822496}]},"public_addr":"172.21.15.137:6817/2086822496","cluster_addr":"172.21.15.137:6819/2086822496","heartbeat_back_addr":"172.21.15.137:6823/2086822496","heartbeat_front_addr":"172.21.15.137:6821/2086822496","state":["exists","up"]},{"osd":8,"uuid":"49f6e9b4-6d14-434e-8b42-8b7ffeb2a072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6817","nonce":153476798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6819","nonce":153476798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6823","nonce":153476798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6821","nonce":153476798}]},"public_addr":"172.21.15.204:6817/153476798","cluster_addr":"172.21.15.204:6819/153476798","heartbeat_back_addr":"172.21.15.204:6823/153476798","heartbeat_front_addr":"172.21.15.204:6821/153476798","state":["exists","up"]},{"osd":9,"uuid":"633f67a7-b759-4ea6-82ed-3c42b3fb70ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6827","nonce":1238491549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6829","nonce":1238491549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6833","nonce":1238491549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6831","nonce":1238491549}]},"public_addr":"172.21.15.3:6827/1238491549","cluster_addr":"172.21.15.3:6829/1238491549","heartbeat_back_addr":"172.21.15.3:6833/1238491549","heartbeat_front_addr":"172.21.15.3:6831/1238491549","state":["exists","up"]},{"osd":10,"uuid":"94b65a48-ee87-4e59-9296-a72a22787f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6825","nonce":3486552683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6827","nonce":3486552683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6831","nonce":3486552683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6829","nonce":3486552683}]},"public_addr":"172.21.15.137:6825/3486552683","cluster_addr":"172.21.15.137:6827/3486552683","heartbeat_back_addr":"172.21.15.137:6831/3486552683","heartbeat_front_addr":"172.21.15.137:6829/3486552683","state":["exists","up"]},{"osd":11,"uuid":"c5cc2850-9062-4b8e-beb3-bef0db30a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6825","nonce":1084308610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6827","nonce":1084308610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6831","nonce":1084308610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6829","nonce":1084308610}]},"public_addr":"172.21.15.204:6825/1084308610","cluster_addr":"172.21.15.204:6827/1084308610","heartbeat_back_addr":"172.21.15.204:6831/1084308610","heartbeat_front_addr":"172.21.15.204:6829/1084308610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:31.898745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.089965+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:22.367464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:48.543248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:14.687331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:41.243579+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:07.290227+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:32.930046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:59.197638+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:24.315219+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:50.333639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:12:16.453640+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1573374138":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1595702358":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/547897695":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/3673347975":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/141972779":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6800/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:6801/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/2363985440":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/2138752581":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1820734047":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/2929116087":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/45453677":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/2929116087":"2024-09-22T06:05:21.689191+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:12:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.290+0000 7f39d0a8e640 1 -- 172.21.15.3:0/2639190456 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f39b80b6560 msgr2=0x7f39b80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.290+0000 7f39d0a8e640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f39b80b6560 0x7f39b80b8a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f39cc111d80 tx=0x7f39b400b390 comp rx=0 tx=0).stop 2024-09-21T06:12:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.290+0000 7f39d0a8e640 1 -- 172.21.15.3:0/2639190456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39cc075730 msgr2=0x7f39cc19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.290+0000 7f39d0a8e640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39cc075730 0x7f39cc19c650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00eab0 tx=0x7f39bc00ef70 comp rx=0 tx=0).stop 2024-09-21T06:12:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.290+0000 7f39d0a8e640 1 -- 172.21.15.3:0/2639190456 shutdown_connections 2024-09-21T06:12:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.290+0000 7f39d0a8e640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f39b80b6560 0x7f39b80b8a70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.290+0000 7f39d0a8e640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39cc110140 0x7f39cc1a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.290+0000 7f39d0a8e640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc076080 0x7f39cc19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.290+0000 7f39d0a8e640 1 --2- 172.21.15.3:0/2639190456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39cc075730 0x7f39cc19c650 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.290+0000 7f39d0a8e640 1 -- 172.21.15.3:0/2639190456 >> 172.21.15.3:0/2639190456 conn(0x7f39cc07b850 msgr2=0x7f39cc109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.291+0000 7f39d0a8e640 1 -- 172.21.15.3:0/2639190456 shutdown_connections 2024-09-21T06:12:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:22.291+0000 7f39d0a8e640 1 -- 172.21.15.3:0/2639190456 wait complete. 2024-09-21T06:12:23.011 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:08:26.015803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:12:23.011 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T06:12:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:23.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:23 smithi003 ceph-mon[31227]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:23.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:23 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2639190456' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:12:23.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:23 smithi137 ceph-mon[34464]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:23.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:23 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2639190456' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:12:23.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:23 smithi204 ceph-mon[34659]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:23.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:23 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2639190456' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:12:23.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.779+0000 7f6143d6c640 1 -- 172.21.15.3:0/3009534856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f613c108d20 msgr2=0x7f613c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:23.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.779+0000 7f6143d6c640 1 --2- 172.21.15.3:0/3009534856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f613c108d20 0x7f613c10f550 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6134009ca0 tx=0x7f613402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.779+0000 7f6143d6c640 1 -- 172.21.15.3:0/3009534856 shutdown_connections 2024-09-21T06:12:23.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.779+0000 7f6143d6c640 1 --2- 172.21.15.3:0/3009534856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f613c108d20 0x7f613c10f550 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.779+0000 7f6143d6c640 1 --2- 172.21.15.3:0/3009534856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f613c1083d0 0x7f613c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.779+0000 7f6143d6c640 1 --2- 172.21.15.3:0/3009534856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f613c102ae0 0x7f613c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.779+0000 7f6143d6c640 1 -- 172.21.15.3:0/3009534856 >> 172.21.15.3:0/3009534856 conn(0x7f613c079760 msgr2=0x7f613c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:23.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.779+0000 7f6143d6c640 1 -- 172.21.15.3:0/3009534856 shutdown_connections 2024-09-21T06:12:23.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.779+0000 7f6143d6c640 1 -- 172.21.15.3:0/3009534856 wait complete. 2024-09-21T06:12:23.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6143d6c640 1 Processor -- start 2024-09-21T06:12:23.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6143d6c640 1 -- start start 2024-09-21T06:12:23.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6143d6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f613c102ae0 0x7f613c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:23.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6143d6c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f613c1083d0 0x7f613c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:23.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6143d6c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f613c108d20 0x7f613c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:23.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6143d6c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f613c113aa0 con 0x7f613c108d20 2024-09-21T06:12:23.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6143d6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f613c113920 con 0x7f613c102ae0 2024-09-21T06:12:23.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6143d6c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f613c113c20 con 0x7f613c1083d0 2024-09-21T06:12:23.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6142309640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f613c108d20 0x7f613c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:23.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6142309640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f613c108d20 0x7f613c19fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50982/0 (socket says 172.21.15.3:50982) 2024-09-21T06:12:23.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6142309640 1 -- 172.21.15.3:0/3178748998 learned_addr learned my addr 172.21.15.3:0/3178748998 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:23.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6141307640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f613c1083d0 0x7f613c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:23.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6141b08640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f613c102ae0 0x7f613c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:23.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6142309640 1 -- 172.21.15.3:0/3178748998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f613c1083d0 msgr2=0x7f613c19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6142309640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f613c1083d0 0x7f613c19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6142309640 1 -- 172.21.15.3:0/3178748998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f613c102ae0 msgr2=0x7f613c19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6142309640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f613c102ae0 0x7f613c19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.780+0000 7f6142309640 1 -- 172.21.15.3:0/3178748998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c1a0340 con 0x7f613c108d20 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.781+0000 7f6142309640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f613c108d20 0x7f613c19fc10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6134005cc0 tx=0x7f613402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.781+0000 7f6132ffd640 1 -- 172.21.15.3:0/3178748998 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134003bf0 con 0x7f613c108d20 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.781+0000 7f6132ffd640 1 -- 172.21.15.3:0/3178748998 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6134003d90 con 0x7f613c108d20 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.781+0000 7f6143d6c640 1 -- 172.21.15.3:0/3178748998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613c1a05d0 con 0x7f613c108d20 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.781+0000 7f6132ffd640 1 -- 172.21.15.3:0/3178748998 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613402fbc0 con 0x7f613c108d20 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.781+0000 7f6143d6c640 1 -- 172.21.15.3:0/3178748998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613c1a6d90 con 0x7f613c108d20 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.782+0000 7f6143d6c640 1 -- 172.21.15.3:0/3178748998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f613c103d10 con 0x7f613c108d20 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.784+0000 7f6132ffd640 1 -- 172.21.15.3:0/3178748998 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f613402fd60 con 0x7f613c108d20 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.785+0000 7f6132ffd640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f61280b6560 0x7f61280b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.785+0000 7f6141b08640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f61280b6560 0x7f61280b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:23.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.785+0000 7f6132ffd640 1 -- 172.21.15.3:0/3178748998 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f6134108710 con 0x7f613c108d20 2024-09-21T06:12:23.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.786+0000 7f6141b08640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f61280b6560 0x7f61280b8a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6138005fd0 tx=0x7f6138005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:23.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.788+0000 7f6132ffd640 1 -- 172.21.15.3:0/3178748998 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f61340cf2d0 con 0x7f613c108d20 2024-09-21T06:12:23.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.974+0000 7f6143d6c640 1 -- 172.21.15.3:0/3178748998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f613c10ab20 con 0x7f613c108d20 2024-09-21T06:12:23.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.974+0000 7f6132ffd640 1 -- 172.21.15.3:0/3178748998 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f61340d36b0 con 0x7f613c108d20 2024-09-21T06:12:23.977 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-21T06:12:23.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.977+0000 7f6143d6c640 1 -- 172.21.15.3:0/3178748998 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f61280b6560 msgr2=0x7f61280b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:23.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.977+0000 7f6143d6c640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f61280b6560 0x7f61280b8a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6138005fd0 tx=0x7f6138005850 comp rx=0 tx=0).stop 2024-09-21T06:12:23.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.977+0000 7f6143d6c640 1 -- 172.21.15.3:0/3178748998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f613c108d20 msgr2=0x7f613c19fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:23.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.977+0000 7f6143d6c640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f613c108d20 0x7f613c19fc10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6134005cc0 tx=0x7f613402f380 comp rx=0 tx=0).stop 2024-09-21T06:12:23.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.977+0000 7f6143d6c640 1 -- 172.21.15.3:0/3178748998 shutdown_connections 2024-09-21T06:12:23.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.977+0000 7f6143d6c640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f61280b6560 0x7f61280b8a70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.977+0000 7f6143d6c640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f613c1083d0 0x7f613c19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.977+0000 7f6143d6c640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f613c108d20 0x7f613c19fc10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.977+0000 7f6143d6c640 1 --2- 172.21.15.3:0/3178748998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f613c102ae0 0x7f613c19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:23.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.977+0000 7f6143d6c640 1 -- 172.21.15.3:0/3178748998 >> 172.21.15.3:0/3178748998 conn(0x7f613c079760 msgr2=0x7f613c1043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:23.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.977+0000 7f6143d6c640 1 -- 172.21.15.3:0/3178748998 shutdown_connections 2024-09-21T06:12:23.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:23.977+0000 7f6143d6c640 1 -- 172.21.15.3:0/3178748998 wait complete. 2024-09-21T06:12:24.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:24 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3178748998' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:12:24.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:24 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3178748998' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:12:24.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:24 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3178748998' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:12:24.772 INFO:tasks.cephadm:Adding mds.a on smithi003 2024-09-21T06:12:24.773 INFO:tasks.cephadm:Adding mds.d on smithi003 2024-09-21T06:12:24.773 INFO:tasks.cephadm:Adding mds.g on smithi003 2024-09-21T06:12:24.773 INFO:tasks.cephadm:Adding mds.j on smithi003 2024-09-21T06:12:24.773 INFO:tasks.cephadm:Adding mds.b on smithi137 2024-09-21T06:12:24.773 INFO:tasks.cephadm:Adding mds.e on smithi137 2024-09-21T06:12:24.773 INFO:tasks.cephadm:Adding mds.h on smithi137 2024-09-21T06:12:24.773 INFO:tasks.cephadm:Adding mds.k on smithi137 2024-09-21T06:12:24.773 INFO:tasks.cephadm:Adding mds.c on smithi204 2024-09-21T06:12:24.773 INFO:tasks.cephadm:Adding mds.f on smithi204 2024-09-21T06:12:24.773 INFO:tasks.cephadm:Adding mds.i on smithi204 2024-09-21T06:12:24.773 INFO:tasks.cephadm:Adding mds.l on smithi204 2024-09-21T06:12:24.773 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2024-09-21T06:12:25.036 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:12:25.314 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:25 smithi204 ceph-mon[34659]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:25.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:25 smithi003 ceph-mon[31227]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:25.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:25 smithi137 ceph-mon[34464]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:25.542 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.532+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3383139092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc0810de30 msgr2=0x7fdc08110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:25.542 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.532+0000 7fdc0d8e2640 1 --2- 172.21.15.204:0/3383139092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc0810de30 0x7fdc08110300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009900 tx=0x7fdc0002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.542 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.532+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3383139092 shutdown_connections 2024-09-21T06:12:25.542 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.532+0000 7fdc0d8e2640 1 --2- 172.21.15.204:0/3383139092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc0810de30 0x7fdc08110300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.542 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.532+0000 7fdc0d8e2640 1 --2- 172.21.15.204:0/3383139092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc081052b0 0x7fdc0810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.542 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.532+0000 7fdc0d8e2640 1 --2- 172.21.15.204:0/3383139092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc08104880 0x7fdc08104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.532+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3383139092 >> 172.21.15.204:0/3383139092 conn(0x7fdc08079730 msgr2=0x7fdc08079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:25.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.532+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3383139092 shutdown_connections 2024-09-21T06:12:25.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.532+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3383139092 wait complete. 2024-09-21T06:12:25.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.533+0000 7fdc0d8e2640 1 Processor -- start 2024-09-21T06:12:25.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.533+0000 7fdc0d8e2640 1 -- start start 2024-09-21T06:12:25.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.533+0000 7fdc0d8e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc08104880 0x7fdc08102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:25.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.533+0000 7fdc0d8e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc081052b0 0x7fdc08102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:25.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.533+0000 7fdc0d8e2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc0810de30 0x7fdc08103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:25.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.533+0000 7fdc0d8e2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08113a70 con 0x7fdc081052b0 2024-09-21T06:12:25.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.533+0000 7fdc0d8e2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081138f0 con 0x7fdc08104880 2024-09-21T06:12:25.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.533+0000 7fdc0d8e2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08113bf0 con 0x7fdc0810de30 2024-09-21T06:12:25.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.533+0000 7fdc077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc0810de30 0x7fdc08103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:25.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.533+0000 7fdc077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc0810de30 0x7fdc08103210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41274/0 (socket says 172.21.15.204:41274) 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.533+0000 7fdc077fe640 1 -- 172.21.15.204:0/3380157075 learned_addr learned my addr 172.21.15.204:0/3380157075 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc067fc640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc081052b0 0x7fdc08102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc06ffd640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc08104880 0x7fdc08102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc077fe640 1 -- 172.21.15.204:0/3380157075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc08104880 msgr2=0x7fdc08102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc077fe640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc08104880 0x7fdc08102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc077fe640 1 -- 172.21.15.204:0/3380157075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc081052b0 msgr2=0x7fdc08102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc077fe640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc081052b0 0x7fdc08102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc077fe640 1 -- 172.21.15.204:0/3380157075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc081a86b0 con 0x7fdc0810de30 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc06ffd640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc08104880 0x7fdc08102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc077fe640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc0810de30 0x7fdc08103210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009a20 tx=0x7fdc00004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc0c8e0640 1 -- 172.21.15.204:0/3380157075 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0003f030 con 0x7fdc0810de30 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc0c8e0640 1 -- 172.21.15.204:0/3380157075 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc00003aa0 con 0x7fdc0810de30 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3380157075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc081a8940 con 0x7fdc0810de30 2024-09-21T06:12:25.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc0c8e0640 1 -- 172.21.15.204:0/3380157075 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00035070 con 0x7fdc0810de30 2024-09-21T06:12:25.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.534+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3380157075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc081a8e20 con 0x7fdc0810de30 2024-09-21T06:12:25.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.535+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3380157075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc08104c90 con 0x7fdc0810de30 2024-09-21T06:12:25.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.537+0000 7fdc0c8e0640 1 -- 172.21.15.204:0/3380157075 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7fdc00042030 con 0x7fdc0810de30 2024-09-21T06:12:25.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.537+0000 7fdc0c8e0640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdbf40b6560 0x7fdbf40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:25.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.538+0000 7fdc0c8e0640 1 -- 172.21.15.204:0/3380157075 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fdc00107ac0 con 0x7fdc0810de30 2024-09-21T06:12:25.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.538+0000 7fdc06ffd640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdbf40b6560 0x7fdbf40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:25.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.538+0000 7fdc06ffd640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdbf40b6560 0x7fdbf40b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8005fd0 tx=0x7fdbf8008330 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:25.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.541+0000 7fdc0c8e0640 1 -- 172.21.15.204:0/3380157075 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdc000ce720 con 0x7fdc0810de30 2024-09-21T06:12:25.735 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.732+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3380157075 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) -- 0x7fdc08101eb0 con 0x7fdbf40b6560 2024-09-21T06:12:25.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.743+0000 7fdc0c8e0640 1 -- 172.21.15.204:0/3380157075 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fdc08101eb0 con 0x7fdbf40b6560 2024-09-21T06:12:25.746 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2024-09-21T06:12:25.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.745+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3380157075 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdbf40b6560 msgr2=0x7fdbf40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:25.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.745+0000 7fdc0d8e2640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdbf40b6560 0x7fdbf40b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8005fd0 tx=0x7fdbf8008330 comp rx=0 tx=0).stop 2024-09-21T06:12:25.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.745+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3380157075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc0810de30 msgr2=0x7fdc08103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:25.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.745+0000 7fdc0d8e2640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc0810de30 0x7fdc08103210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009a20 tx=0x7fdc00004270 comp rx=0 tx=0).stop 2024-09-21T06:12:25.749 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.745+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3380157075 shutdown_connections 2024-09-21T06:12:25.749 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.745+0000 7fdc0d8e2640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdbf40b6560 0x7fdbf40b8a70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.749 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.745+0000 7fdc0d8e2640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc0810de30 0x7fdc08103210 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.749 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.745+0000 7fdc0d8e2640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc081052b0 0x7fdc08102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.749 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.745+0000 7fdc0d8e2640 1 --2- 172.21.15.204:0/3380157075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc08104880 0x7fdc08102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:25.749 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.745+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3380157075 >> 172.21.15.204:0/3380157075 conn(0x7fdc08079730 msgr2=0x7fdc08100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:25.749 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.745+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3380157075 shutdown_connections 2024-09-21T06:12:25.749 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-21T06:12:25.746+0000 7fdc0d8e2640 1 -- 172.21.15.204:0/3380157075 wait complete. 2024-09-21T06:12:26.584 DEBUG:teuthology.orchestra.run.smithi003:mds.a> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T06:12:26.586 DEBUG:teuthology.orchestra.run.smithi003:mds.d> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T06:12:26.630 DEBUG:teuthology.orchestra.run.smithi003:mds.g> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T06:12:26.673 DEBUG:teuthology.orchestra.run.smithi003:mds.j> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T06:12:26.715 DEBUG:teuthology.orchestra.run.smithi137:mds.b> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T06:12:26.718 DEBUG:teuthology.orchestra.run.smithi137:mds.e> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T06:12:26.762 DEBUG:teuthology.orchestra.run.smithi137:mds.h> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T06:12:26.804 DEBUG:teuthology.orchestra.run.smithi137:mds.k> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T06:12:26.847 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T06:12:26.850 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T06:12:26.869 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:26 smithi204 ceph-mon[34659]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:26.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:26 smithi204 ceph-mon[34659]: from='client.24619 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:12:26.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:26 smithi204 ceph-mon[34659]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-21T06:12:26.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:26.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:12:26.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:26.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:12:26.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:26.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:26.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:26.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:26.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:26 smithi204 ceph-mon[34659]: Deploying daemon mds.l on smithi204 2024-09-21T06:12:26.871 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T06:12:26.914 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T06:12:26.958 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T06:12:26.959 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:12:26.982 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:26 smithi003 ceph-mon[31227]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:26.982 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:26 smithi003 ceph-mon[31227]: from='client.24619 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:12:26.982 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:26 smithi003 ceph-mon[31227]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-21T06:12:26.982 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:26.982 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:12:26.982 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:26.983 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:12:26.983 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:26.983 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:26.983 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:26.983 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:26.983 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:26 smithi003 ceph-mon[31227]: Deploying daemon mds.l on smithi204 2024-09-21T06:12:27.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:26 smithi137 ceph-mon[34464]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:27.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:26 smithi137 ceph-mon[34464]: from='client.24619 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:12:27.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:26 smithi137 ceph-mon[34464]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-21T06:12:27.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:27.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:12:27.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:27.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:12:27.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:27.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:27.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:27.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:27.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:26 smithi137 ceph-mon[34464]: Deploying daemon mds.l on smithi204 2024-09-21T06:12:27.211 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.709+0000 7f6d88165640 1 -- 172.21.15.3:0/1245974318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80107bd0 msgr2=0x7f6d80107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.709+0000 7f6d88165640 1 --2- 172.21.15.3:0/1245974318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80107bd0 0x7f6d80107fe0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009900 tx=0x7f6d7002a410 comp rx=0 tx=0).stop 2024-09-21T06:12:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.709+0000 7f6d88165640 1 -- 172.21.15.3:0/1245974318 shutdown_connections 2024-09-21T06:12:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.709+0000 7f6d88165640 1 --2- 172.21.15.3:0/1245974318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d80108520 0x7f6d8010ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.709+0000 7f6d88165640 1 --2- 172.21.15.3:0/1245974318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80107bd0 0x7f6d80107fe0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.709+0000 7f6d88165640 1 --2- 172.21.15.3:0/1245974318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d80102290 0x7f6d801026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.709+0000 7f6d88165640 1 -- 172.21.15.3:0/1245974318 >> 172.21.15.3:0/1245974318 conn(0x7f6d80079ea0 msgr2=0x7f6d8007a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.709+0000 7f6d88165640 1 -- 172.21.15.3:0/1245974318 shutdown_connections 2024-09-21T06:12:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.709+0000 7f6d88165640 1 -- 172.21.15.3:0/1245974318 wait complete. 2024-09-21T06:12:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.710+0000 7f6d88165640 1 Processor -- start 2024-09-21T06:12:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.710+0000 7f6d88165640 1 -- start start 2024-09-21T06:12:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.710+0000 7f6d88165640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80102290 0x7f6d80110640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.710+0000 7f6d88165640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d80107bd0 0x7f6d80110b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.710+0000 7f6d88165640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d80108520 0x7f6d801110c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.710+0000 7f6d88165640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6d80113b50 con 0x7f6d80102290 2024-09-21T06:12:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.710+0000 7f6d88165640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6d801139d0 con 0x7f6d80108520 2024-09-21T06:12:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.710+0000 7f6d88165640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d80113cd0 con 0x7f6d80107bd0 2024-09-21T06:12:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.710+0000 7f6d85f01640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80102290 0x7f6d80110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.710+0000 7f6d85f01640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80102290 0x7f6d80110640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39790/0 (socket says 172.21.15.3:39790) 2024-09-21T06:12:27.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.710+0000 7f6d85f01640 1 -- 172.21.15.3:0/1524476169 learned_addr learned my addr 172.21.15.3:0/1524476169 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:27.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.710+0000 7f6d85700640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d80107bd0 0x7f6d80110b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:27.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.710+0000 7f6d86702640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d80108520 0x7f6d801110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:27.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.711+0000 7f6d85f01640 1 -- 172.21.15.3:0/1524476169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d80107bd0 msgr2=0x7f6d80110b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:27.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.711+0000 7f6d85f01640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d80107bd0 0x7f6d80110b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.711+0000 7f6d85f01640 1 -- 172.21.15.3:0/1524476169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d80108520 msgr2=0x7f6d801110c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:27.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.711+0000 7f6d85f01640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d80108520 0x7f6d801110c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.711+0000 7f6d85f01640 1 -- 172.21.15.3:0/1524476169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d801a8800 con 0x7f6d80102290 2024-09-21T06:12:27.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.711+0000 7f6d85f01640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80102290 0x7f6d80110640 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00e9f0 tx=0x7f6d7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:27.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.711+0000 7f6d76ffd640 1 -- 172.21.15.3:0/1524476169 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00cd80 con 0x7f6d80102290 2024-09-21T06:12:27.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.711+0000 7f6d88165640 1 -- 172.21.15.3:0/1524476169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d801a8af0 con 0x7f6d80102290 2024-09-21T06:12:27.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.711+0000 7f6d76ffd640 1 -- 172.21.15.3:0/1524476169 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d7c0108e0 con 0x7f6d80102290 2024-09-21T06:12:27.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.711+0000 7f6d76ffd640 1 -- 172.21.15.3:0/1524476169 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c018980 con 0x7f6d80102290 2024-09-21T06:12:27.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.711+0000 7f6d88165640 1 -- 172.21.15.3:0/1524476169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d801a9050 con 0x7f6d80102290 2024-09-21T06:12:27.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.712+0000 7f6d88165640 1 -- 172.21.15.3:0/1524476169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d801026a0 con 0x7f6d80102290 2024-09-21T06:12:27.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.714+0000 7f6d76ffd640 1 -- 172.21.15.3:0/1524476169 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f6d7c010430 con 0x7f6d80102290 2024-09-21T06:12:27.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.715+0000 7f6d76ffd640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6d6c0b6560 0x7f6d6c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:27.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.715+0000 7f6d85700640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6d6c0b6560 0x7f6d6c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:27.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.715+0000 7f6d76ffd640 1 -- 172.21.15.3:0/1524476169 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f6d7c0e7880 con 0x7f6d80102290 2024-09-21T06:12:27.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.715+0000 7f6d85700640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6d6c0b6560 0x7f6d6c0b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009230 tx=0x7f6d70031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:27.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.718+0000 7f6d76ffd640 1 -- 172.21.15.3:0/1524476169 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6d7c0ae440 con 0x7f6d80102290 2024-09-21T06:12:27.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.944+0000 7f6d88165640 1 -- 172.21.15.3:0/1524476169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d8002e640 con 0x7f6d80102290 2024-09-21T06:12:27.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.944+0000 7f6d76ffd640 1 -- 172.21.15.3:0/1524476169 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f6d7c0b2820 con 0x7f6d80102290 2024-09-21T06:12:27.946 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:12:27.947 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":1,"btime":"2024-09-21T06:04:24:293546+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T06:12:27.947 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 1 2024-09-21T06:12:27.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.947+0000 7f6d88165640 1 -- 172.21.15.3:0/1524476169 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6d6c0b6560 msgr2=0x7f6d6c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:27.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.947+0000 7f6d88165640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6d6c0b6560 0x7f6d6c0b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009230 tx=0x7f6d70031400 comp rx=0 tx=0).stop 2024-09-21T06:12:27.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.947+0000 7f6d88165640 1 -- 172.21.15.3:0/1524476169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80102290 msgr2=0x7f6d80110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:27.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.947+0000 7f6d88165640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80102290 0x7f6d80110640 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00e9f0 tx=0x7f6d7c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.948+0000 7f6d88165640 1 -- 172.21.15.3:0/1524476169 shutdown_connections 2024-09-21T06:12:27.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.948+0000 7f6d88165640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6d6c0b6560 0x7f6d6c0b8a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.948+0000 7f6d88165640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d80107bd0 0x7f6d80110b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.948+0000 7f6d88165640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d80108520 0x7f6d801110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.948+0000 7f6d88165640 1 --2- 172.21.15.3:0/1524476169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80102290 0x7f6d80110640 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:27.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.948+0000 7f6d88165640 1 -- 172.21.15.3:0/1524476169 >> 172.21.15.3:0/1524476169 conn(0x7f6d80079ea0 msgr2=0x7f6d80103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:27.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.948+0000 7f6d88165640 1 -- 172.21.15.3:0/1524476169 shutdown_connections 2024-09-21T06:12:27.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:27.948+0000 7f6d88165640 1 -- 172.21.15.3:0/1524476169 wait complete. 2024-09-21T06:12:28.600 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:12:28 smithi204 systemd[1]: Starting Ceph mds.l for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:12:28.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:28 smithi204 ceph-mon[34659]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:28.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:28 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1524476169' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:28.863 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:12:28 smithi204 podman[62728]: 2024-09-21 06:12:28.501435428 +0000 UTC m=+0.016221851 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:12:28.863 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:12:28 smithi204 podman[62728]: 2024-09-21 06:12:28.616816776 +0000 UTC m=+0.131603192 container create 3debaa0031e23eab5005b195055950b467971ba305501733450d872dc4235568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-l, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T06:12:28.863 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:12:28 smithi204 podman[62728]: 2024-09-21 06:12:28.861858255 +0000 UTC m=+0.376644681 container init 3debaa0031e23eab5005b195055950b467971ba305501733450d872dc4235568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-l, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T06:12:29.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:28 smithi003 ceph-mon[31227]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:29.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:28 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1524476169' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:29.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:28 smithi137 ceph-mon[34464]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:29.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:28 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1524476169' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:29.154 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:12:28 smithi204 podman[62728]: 2024-09-21 06:12:28.864971056 +0000 UTC m=+0.379757472 container start 3debaa0031e23eab5005b195055950b467971ba305501733450d872dc4235568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-l, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:12:29.154 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:12:28 smithi204 ceph-mds[62747]: -- 172.21.15.204:0/1348118175 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd16183c20 con 0x55bd1a32b600 2024-09-21T06:12:29.154 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:12:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-l[62743]: starting mds.l at 2024-09-21T06:12:29.154 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:12:28 smithi204 bash[62728]: 3debaa0031e23eab5005b195055950b467971ba305501733450d872dc4235568 2024-09-21T06:12:29.154 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:12:28 smithi204 systemd[1]: Started Ceph mds.l for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:12:30.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:29 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.204:6832/2480313554,v1:172.21.15.204:6833/2480313554] up:boot 2024-09-21T06:12:30.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:29 smithi137 ceph-mon[34464]: fsmap 1 up:standby 2024-09-21T06:12:30.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:12:30.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:30.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:30.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:30.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:30.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:30.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:30.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:29 smithi137 ceph-mon[34464]: Deploying daemon mds.k on smithi137 2024-09-21T06:12:30.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:29 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.204:6832/2480313554,v1:172.21.15.204:6833/2480313554] up:boot 2024-09-21T06:12:30.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:29 smithi003 ceph-mon[31227]: fsmap 1 up:standby 2024-09-21T06:12:30.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:12:30.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:30.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:30.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:30.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:30.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:30.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:30.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:29 smithi003 ceph-mon[31227]: Deploying daemon mds.k on smithi137 2024-09-21T06:12:30.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:29 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.204:6832/2480313554,v1:172.21.15.204:6833/2480313554] up:boot 2024-09-21T06:12:30.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:29 smithi204 ceph-mon[34659]: fsmap 1 up:standby 2024-09-21T06:12:30.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:12:30.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:30.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:30.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:30.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:30.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:30.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:30.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:29 smithi204 ceph-mon[34659]: Deploying daemon mds.k on smithi137 2024-09-21T06:12:30.725 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:12:30.995 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:31.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:31 smithi003 ceph-mon[31227]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:31.343 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:31 smithi137 ceph-mon[34464]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:31.343 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:12:31 smithi137 systemd[1]: Starting Ceph mds.k for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:12:31.344 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:12:31 smithi137 podman[62224]: 2024-09-21 06:12:31.244461099 +0000 UTC m=+0.016313472 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:12:31.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:31 smithi204 ceph-mon[34659]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:31.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.493+0000 7f556b35c640 1 -- 172.21.15.3:0/240801186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5564102a90 msgr2=0x7f5564102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.493+0000 7f556b35c640 1 --2- 172.21.15.3:0/240801186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5564102a90 0x7f5564102ea0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5560009900 tx=0x7f556002a410 comp rx=0 tx=0).stop 2024-09-21T06:12:31.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.493+0000 7f556b35c640 1 -- 172.21.15.3:0/240801186 shutdown_connections 2024-09-21T06:12:31.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.493+0000 7f556b35c640 1 --2- 172.21.15.3:0/240801186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55641083d0 0x7f55641087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.493+0000 7f556b35c640 1 --2- 172.21.15.3:0/240801186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5564108d20 0x7f556410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.493+0000 7f556b35c640 1 --2- 172.21.15.3:0/240801186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5564102a90 0x7f5564102ea0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.493+0000 7f556b35c640 1 -- 172.21.15.3:0/240801186 >> 172.21.15.3:0/240801186 conn(0x7f5564079750 msgr2=0x7f5564079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.493+0000 7f556b35c640 1 -- 172.21.15.3:0/240801186 shutdown_connections 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.493+0000 7f556b35c640 1 -- 172.21.15.3:0/240801186 wait complete. 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.494+0000 7f556b35c640 1 Processor -- start 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.494+0000 7f556b35c640 1 -- start start 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.494+0000 7f556b35c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5564102a90 0x7f556419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.494+0000 7f556b35c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55641083d0 0x7f556419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.494+0000 7f556b35c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5564108d20 0x7f556419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.494+0000 7f556b35c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5564112990 con 0x7f5564108d20 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.494+0000 7f556b35c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5564112810 con 0x7f5564102a90 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.494+0000 7f556b35c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5564112b10 con 0x7f55641083d0 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f55690f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5564102a90 0x7f556419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f55698f9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5564108d20 0x7f556419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f55690f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5564102a90 0x7f556419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:48300/0 (socket says 172.21.15.3:48300) 2024-09-21T06:12:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f55690f8640 1 -- 172.21.15.3:0/427014654 learned_addr learned my addr 172.21.15.3:0/427014654 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f55688f7640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55641083d0 0x7f556419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f55698f9640 1 -- 172.21.15.3:0/427014654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55641083d0 msgr2=0x7f556419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f55698f9640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55641083d0 0x7f556419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f55698f9640 1 -- 172.21.15.3:0/427014654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5564102a90 msgr2=0x7f556419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f55698f9640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5564102a90 0x7f556419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f55698f9640 1 -- 172.21.15.3:0/427014654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55641a03c0 con 0x7f5564108d20 2024-09-21T06:12:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f55690f8640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5564102a90 0x7f556419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f55698f9640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5564108d20 0x7f556419fc90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f555c00da10 tx=0x7f555c00ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f555a7fc640 1 -- 172.21.15.3:0/427014654 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c00bd70 con 0x7f5564108d20 2024-09-21T06:12:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f555a7fc640 1 -- 172.21.15.3:0/427014654 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f555c003e20 con 0x7f5564108d20 2024-09-21T06:12:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f556b35c640 1 -- 172.21.15.3:0/427014654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55641a06b0 con 0x7f5564108d20 2024-09-21T06:12:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.495+0000 7f555a7fc640 1 -- 172.21.15.3:0/427014654 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c0106c0 con 0x7f5564108d20 2024-09-21T06:12:31.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.496+0000 7f556b35c640 1 -- 172.21.15.3:0/427014654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55641a6e10 con 0x7f5564108d20 2024-09-21T06:12:31.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.497+0000 7f556b35c640 1 -- 172.21.15.3:0/427014654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5564103d10 con 0x7f5564108d20 2024-09-21T06:12:31.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.498+0000 7f555a7fc640 1 -- 172.21.15.3:0/427014654 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f555c004270 con 0x7f5564108d20 2024-09-21T06:12:31.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.499+0000 7f555a7fc640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55500b65e0 0x7f55500b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.499+0000 7f55690f8640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55500b65e0 0x7f55500b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.499+0000 7f555a7fc640 1 -- 172.21.15.3:0/427014654 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f555c0e7350 con 0x7f5564108d20 2024-09-21T06:12:31.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.499+0000 7f55690f8640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55500b65e0 0x7f55500b8af0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5560031440 tx=0x7f55600058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.503+0000 7f555a7fc640 1 -- 172.21.15.3:0/427014654 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f555c0adf10 con 0x7f5564108d20 2024-09-21T06:12:31.596 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:12:31 smithi137 podman[62224]: 2024-09-21 06:12:31.359885665 +0000 UTC m=+0.131738038 container create b1d6d9ac62cbf7c71259b402ab70c3f584fdad135bc81acc412433afbb9fe6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-k, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T06:12:31.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.730+0000 7f556b35c640 1 -- 172.21.15.3:0/427014654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55641a70c0 con 0x7f5564108d20 2024-09-21T06:12:31.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.730+0000 7f555a7fc640 1 -- 172.21.15.3:0/427014654 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f555c0b22f0 con 0x7f5564108d20 2024-09-21T06:12:31.733 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:12:31.733 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":2,"btime":"2024-09-21T06:12:28:903700+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24631,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T06:12:31.733 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 2 2024-09-21T06:12:31.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.732+0000 7f556b35c640 1 -- 172.21.15.3:0/427014654 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55500b65e0 msgr2=0x7f55500b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.732+0000 7f556b35c640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55500b65e0 0x7f55500b8af0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5560031440 tx=0x7f55600058d0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.732+0000 7f556b35c640 1 -- 172.21.15.3:0/427014654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5564108d20 msgr2=0x7f556419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.732+0000 7f556b35c640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5564108d20 0x7f556419fc90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f555c00da10 tx=0x7f555c00ded0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.732+0000 7f556b35c640 1 -- 172.21.15.3:0/427014654 shutdown_connections 2024-09-21T06:12:31.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.732+0000 7f556b35c640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55500b65e0 0x7f55500b8af0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.732+0000 7f556b35c640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55641083d0 0x7f556419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.732+0000 7f556b35c640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5564108d20 0x7f556419fc90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.732+0000 7f556b35c640 1 --2- 172.21.15.3:0/427014654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5564102a90 0x7f556419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.733+0000 7f556b35c640 1 -- 172.21.15.3:0/427014654 >> 172.21.15.3:0/427014654 conn(0x7f5564079750 msgr2=0x7f5564101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.733+0000 7f556b35c640 1 -- 172.21.15.3:0/427014654 shutdown_connections 2024-09-21T06:12:31.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:31.733+0000 7f556b35c640 1 -- 172.21.15.3:0/427014654 wait complete. 2024-09-21T06:12:31.895 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:12:31 smithi137 podman[62224]: 2024-09-21 06:12:31.595940917 +0000 UTC m=+0.367793290 container init b1d6d9ac62cbf7c71259b402ab70c3f584fdad135bc81acc412433afbb9fe6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-k, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T06:12:31.895 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:12:31 smithi137 podman[62224]: 2024-09-21 06:12:31.59928386 +0000 UTC m=+0.371136242 container start b1d6d9ac62cbf7c71259b402ab70c3f584fdad135bc81acc412433afbb9fe6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T06:12:31.895 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:12:31 smithi137 ceph-mds[62243]: -- 172.21.15.137:0/3451744735 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e517513860 con 0x55e51b6bb180 2024-09-21T06:12:31.895 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:12:31 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-k[62239]: starting mds.k at 2024-09-21T06:12:31.895 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:12:31 smithi137 bash[62224]: b1d6d9ac62cbf7c71259b402ab70c3f584fdad135bc81acc412433afbb9fe6f7 2024-09-21T06:12:31.895 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:12:31 smithi137 systemd[1]: Started Ceph mds.k for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:12:32.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:32 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/427014654' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:32.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:32.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:32.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:32.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:32.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:32.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:32 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:32.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:32 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/427014654' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:32.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:32.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:32.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:32.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:32.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:32.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:32 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:32.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:32 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/427014654' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:32.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:32.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:32.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:32.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:32.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:32.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:32 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:33 smithi003 ceph-mon[31227]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:33 smithi003 ceph-mon[31227]: Deploying daemon mds.j on smithi003 2024-09-21T06:12:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:33 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.137:6832/4270441897,v1:172.21.15.137:6833/4270441897] up:boot 2024-09-21T06:12:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:33 smithi003 ceph-mon[31227]: fsmap 2 up:standby 2024-09-21T06:12:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:33 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:12:33.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:33 smithi137 ceph-mon[34464]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:33.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:33 smithi137 ceph-mon[34464]: Deploying daemon mds.j on smithi003 2024-09-21T06:12:33.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:33 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.137:6832/4270441897,v1:172.21.15.137:6833/4270441897] up:boot 2024-09-21T06:12:33.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:33 smithi137 ceph-mon[34464]: fsmap 2 up:standby 2024-09-21T06:12:33.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:33 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:12:33.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:33 smithi204 ceph-mon[34659]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:33.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:33 smithi204 ceph-mon[34659]: Deploying daemon mds.j on smithi003 2024-09-21T06:12:33.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:33 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.137:6832/4270441897,v1:172.21.15.137:6833/4270441897] up:boot 2024-09-21T06:12:33.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:33 smithi204 ceph-mon[34659]: fsmap 2 up:standby 2024-09-21T06:12:33.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:33 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:12:34.340 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:12:34 smithi003 systemd[1]: Starting Ceph mds.j for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:12:34.470 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:12:34.722 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:12:34 smithi003 podman[72627]: 2024-09-21 06:12:34.366378105 +0000 UTC m=+0.016566512 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:12:34.722 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:12:34 smithi003 podman[72627]: 2024-09-21 06:12:34.484576181 +0000 UTC m=+0.134764573 container create 1b18b84989ff4d2faa9af7f1d7acc6ca484e5344455722becaede758af73192c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-j, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:12:34.722 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:12:34 smithi003 podman[72627]: 2024-09-21 06:12:34.721001351 +0000 UTC m=+0.371189744 container init 1b18b84989ff4d2faa9af7f1d7acc6ca484e5344455722becaede758af73192c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-j, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T06:12:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:35.000 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:12:34 smithi003 podman[72627]: 2024-09-21 06:12:34.72452838 +0000 UTC m=+0.374716781 container start 1b18b84989ff4d2faa9af7f1d7acc6ca484e5344455722becaede758af73192c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T06:12:35.000 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:12:34 smithi003 ceph-mds[72676]: -- 172.21.15.3:0/4009507274 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565d872b860 con 0x5565dc8d3600 2024-09-21T06:12:35.001 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:12:34 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-j[72660]: starting mds.j at 2024-09-21T06:12:35.001 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:12:34 smithi003 bash[72627]: 1b18b84989ff4d2faa9af7f1d7acc6ca484e5344455722becaede758af73192c 2024-09-21T06:12:35.001 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:12:34 smithi003 systemd[1]: Started Ceph mds.j for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:12:35.326 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:35 smithi003 ceph-mon[31227]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:35.326 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:35.327 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:35.327 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:35.327 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:35.327 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:35.327 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:35 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:35 smithi137 ceph-mon[34464]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:35.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:35 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:35.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.418+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2255228761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09d4104860 msgr2=0x7f09d4104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.418+0000 7f09d9f0a640 1 --2- 172.21.15.3:0/2255228761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09d4104860 0x7f09d4104c70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009ca0 tx=0x7f09c402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.418+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2255228761 shutdown_connections 2024-09-21T06:12:35.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.418+0000 7f09d9f0a640 1 --2- 172.21.15.3:0/2255228761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d410de10 0x7f09d4110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.418+0000 7f09d9f0a640 1 --2- 172.21.15.3:0/2255228761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09d4105290 0x7f09d410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.418+0000 7f09d9f0a640 1 --2- 172.21.15.3:0/2255228761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09d4104860 0x7f09d4104c70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.418+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2255228761 >> 172.21.15.3:0/2255228761 conn(0x7f09d4079740 msgr2=0x7f09d4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.418+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2255228761 shutdown_connections 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.418+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2255228761 wait complete. 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.419+0000 7f09d9f0a640 1 Processor -- start 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.419+0000 7f09d9f0a640 1 -- start start 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d9f0a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09d4104860 0x7f09d419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d9f0a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09d4105290 0x7f09d419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d9f0a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d410de10 0x7f09d419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d9f0a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4113ab0 con 0x7f09d4105290 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d9f0a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4113930 con 0x7f09d4104860 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d9f0a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4113c30 con 0x7f09d410de10 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09d4105290 0x7f09d419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09d4105290 0x7f09d419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60216/0 (socket says 172.21.15.3:60216) 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d2ffd640 1 -- 172.21.15.3:0/2721403051 learned_addr learned my addr 172.21.15.3:0/2721403051 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:35.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d3fff640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d410de10 0x7f09d419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d37fe640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09d4104860 0x7f09d419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d2ffd640 1 -- 172.21.15.3:0/2721403051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d410de10 msgr2=0x7f09d419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d2ffd640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d410de10 0x7f09d419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d2ffd640 1 -- 172.21.15.3:0/2721403051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09d4104860 msgr2=0x7f09d419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d2ffd640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09d4104860 0x7f09d419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d2ffd640 1 -- 172.21.15.3:0/2721403051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d41a0470 con 0x7f09d4105290 2024-09-21T06:12:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d2ffd640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09d4105290 0x7f09d419aaa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00d740 tx=0x7f09bc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.420+0000 7f09d0ff9640 1 -- 172.21.15.3:0/2721403051 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc017070 con 0x7f09d4105290 2024-09-21T06:12:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.421+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2721403051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d41a0760 con 0x7f09d4105290 2024-09-21T06:12:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.421+0000 7f09d0ff9640 1 -- 172.21.15.3:0/2721403051 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09bc00de50 con 0x7f09d4105290 2024-09-21T06:12:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.421+0000 7f09d0ff9640 1 -- 172.21.15.3:0/2721403051 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00f620 con 0x7f09d4105290 2024-09-21T06:12:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.421+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2721403051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d41a6ec0 con 0x7f09d4105290 2024-09-21T06:12:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.422+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2721403051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d406c1e0 con 0x7f09d4105290 2024-09-21T06:12:35.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.423+0000 7f09d0ff9640 1 -- 172.21.15.3:0/2721403051 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f09bc01a070 con 0x7f09d4105290 2024-09-21T06:12:35.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.425+0000 7f09d0ff9640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09c00b6560 0x7f09c00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:35.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.425+0000 7f09d37fe640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09c00b6560 0x7f09c00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:35.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.425+0000 7f09d0ff9640 1 -- 172.21.15.3:0/2721403051 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f09bc0e6940 con 0x7f09d4105290 2024-09-21T06:12:35.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.425+0000 7f09d37fe640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09c00b6560 0x7f09c00b8a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005e30 tx=0x7f09c4009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:35.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.428+0000 7f09d0ff9640 1 -- 172.21.15.3:0/2721403051 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f09bc0b18e0 con 0x7f09d4105290 2024-09-21T06:12:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:35 smithi204 ceph-mon[34659]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:35.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:35.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:35 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.664+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2721403051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09d419b630 con 0x7f09d4105290 2024-09-21T06:12:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.665+0000 7f09d0ff9640 1 -- 172.21.15.3:0/2721403051 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7f09bc0b6820 con 0x7f09d4105290 2024-09-21T06:12:35.667 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:12:35.667 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":4,"btime":"2024-09-21T06:12:35:209047+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24631,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T06:12:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 4 2024-09-21T06:12:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.667+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2721403051 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09c00b6560 msgr2=0x7f09c00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.667+0000 7f09d9f0a640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09c00b6560 0x7f09c00b8a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005e30 tx=0x7f09c4009360 comp rx=0 tx=0).stop 2024-09-21T06:12:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.667+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2721403051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09d4105290 msgr2=0x7f09d419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.667+0000 7f09d9f0a640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09d4105290 0x7f09d419aaa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00d740 tx=0x7f09bc00dc00 comp rx=0 tx=0).stop 2024-09-21T06:12:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.667+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2721403051 shutdown_connections 2024-09-21T06:12:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.667+0000 7f09d9f0a640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09c00b6560 0x7f09c00b8a70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.667+0000 7f09d9f0a640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d410de10 0x7f09d419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.667+0000 7f09d9f0a640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09d4105290 0x7f09d419aaa0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.667+0000 7f09d9f0a640 1 --2- 172.21.15.3:0/2721403051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09d4104860 0x7f09d419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.667+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2721403051 >> 172.21.15.3:0/2721403051 conn(0x7f09d4079740 msgr2=0x7f09d41006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.667+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2721403051 shutdown_connections 2024-09-21T06:12:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:35.667+0000 7f09d9f0a640 1 -- 172.21.15.3:0/2721403051 wait complete. 2024-09-21T06:12:36.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:36 smithi003 ceph-mon[31227]: Deploying daemon mds.d on smithi003 2024-09-21T06:12:36.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:36 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.3:6834/601337393,v1:172.21.15.3:6835/601337393] up:boot 2024-09-21T06:12:36.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:36 smithi003 ceph-mon[31227]: fsmap 3 up:standby 2024-09-21T06:12:36.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:36 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T06:12:36.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:36 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2721403051' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:36.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:36 smithi137 ceph-mon[34464]: Deploying daemon mds.d on smithi003 2024-09-21T06:12:36.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:36 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.3:6834/601337393,v1:172.21.15.3:6835/601337393] up:boot 2024-09-21T06:12:36.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:36 smithi137 ceph-mon[34464]: fsmap 3 up:standby 2024-09-21T06:12:36.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:36 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T06:12:36.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:36 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2721403051' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:36 smithi204 ceph-mon[34659]: Deploying daemon mds.d on smithi003 2024-09-21T06:12:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:36 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.3:6834/601337393,v1:172.21.15.3:6835/601337393] up:boot 2024-09-21T06:12:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:36 smithi204 ceph-mon[34659]: fsmap 3 up:standby 2024-09-21T06:12:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:36 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T06:12:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:36 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2721403051' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:37.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:37 smithi003 ceph-mon[31227]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:37.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:37 smithi204 ceph-mon[34659]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:37.546 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:37 smithi137 ceph-mon[34464]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:38.090 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 21 06:12:37 smithi003 systemd[1]: Started Ceph mds.d for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:12:39.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:38 smithi003 ceph-mon[31227]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:39.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:38 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:39.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:38 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:39.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:38 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:39.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:38 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:39.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:38 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:39.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:38 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:39.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:38 smithi003 ceph-mon[31227]: Deploying daemon mds.e on smithi137 2024-09-21T06:12:39.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:38 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.3:6836/2476398034,v1:172.21.15.3:6837/2476398034] up:boot 2024-09-21T06:12:39.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:38 smithi003 ceph-mon[31227]: fsmap 4 up:standby 2024-09-21T06:12:39.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:38 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T06:12:39.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:12:39.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:38 smithi137 ceph-mon[34464]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:39.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:38 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:39.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:38 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:39.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:38 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:39.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:38 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:39.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:38 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:39.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:38 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:39.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:38 smithi137 ceph-mon[34464]: Deploying daemon mds.e on smithi137 2024-09-21T06:12:39.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:38 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.3:6836/2476398034,v1:172.21.15.3:6837/2476398034] up:boot 2024-09-21T06:12:39.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:38 smithi137 ceph-mon[34464]: fsmap 4 up:standby 2024-09-21T06:12:39.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:38 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T06:12:39.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:38 smithi204 ceph-mon[34659]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:39.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:38 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:39.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:38 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:39.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:38 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:39.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:38 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:39.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:38 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:39.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:38 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:39.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:38 smithi204 ceph-mon[34659]: Deploying daemon mds.e on smithi137 2024-09-21T06:12:39.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:38 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.3:6836/2476398034,v1:172.21.15.3:6837/2476398034] up:boot 2024-09-21T06:12:39.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:38 smithi204 ceph-mon[34659]: fsmap 4 up:standby 2024-09-21T06:12:39.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:38 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T06:12:39.659 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:40.145 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:12:39 smithi137 systemd[1]: Starting Ceph mds.e for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:12:40.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.176+0000 7fc4eba67640 1 -- 172.21.15.3:0/3989754923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e4108d20 msgr2=0x7fc4e410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:40.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.176+0000 7fc4eba67640 1 --2- 172.21.15.3:0/3989754923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e4108d20 0x7fc4e410f550 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009900 tx=0x7fc4dc02a410 comp rx=0 tx=0).stop 2024-09-21T06:12:40.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.176+0000 7fc4eba67640 1 -- 172.21.15.3:0/3989754923 shutdown_connections 2024-09-21T06:12:40.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.176+0000 7fc4eba67640 1 --2- 172.21.15.3:0/3989754923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e41083d0 0x7fc4e41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:40.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.176+0000 7fc4eba67640 1 --2- 172.21.15.3:0/3989754923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e4108d20 0x7fc4e410f550 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:40.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.176+0000 7fc4eba67640 1 --2- 172.21.15.3:0/3989754923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e4102ae0 0x7fc4e4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:40.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.176+0000 7fc4eba67640 1 -- 172.21.15.3:0/3989754923 >> 172.21.15.3:0/3989754923 conn(0x7fc4e4079760 msgr2=0x7fc4e4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:40.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.176+0000 7fc4eba67640 1 -- 172.21.15.3:0/3989754923 shutdown_connections 2024-09-21T06:12:40.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.176+0000 7fc4eba67640 1 -- 172.21.15.3:0/3989754923 wait complete. 2024-09-21T06:12:40.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.177+0000 7fc4eba67640 1 Processor -- start 2024-09-21T06:12:40.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.177+0000 7fc4eba67640 1 -- start start 2024-09-21T06:12:40.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4eba67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e4102ae0 0x7fc4e419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4eba67640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e41083d0 0x7fc4e419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4eba67640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e4108d20 0x7fc4e419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4eba67640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e41129e0 con 0x7fc4e4102ae0 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4eba67640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4112860 con 0x7fc4e41083d0 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4eba67640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4112b60 con 0x7fc4e4108d20 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4e9803640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e4102ae0 0x7fc4e419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4e9803640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e4102ae0 0x7fc4e419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60266/0 (socket says 172.21.15.3:60266) 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4e9803640 1 -- 172.21.15.3:0/1061201537 learned_addr learned my addr 172.21.15.3:0/1061201537 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4ea004640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e4108d20 0x7fc4e419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4e9002640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e41083d0 0x7fc4e419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4e9803640 1 -- 172.21.15.3:0/1061201537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e4108d20 msgr2=0x7fc4e419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4e9803640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e4108d20 0x7fc4e419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4e9803640 1 -- 172.21.15.3:0/1061201537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e41083d0 msgr2=0x7fc4e419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:40.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4e9803640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e41083d0 0x7fc4e419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:40.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4e9803640 1 -- 172.21.15.3:0/1061201537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e41a0370 con 0x7fc4e4102ae0 2024-09-21T06:12:40.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4ea004640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e4108d20 0x7fc4e419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:40.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.178+0000 7fc4e9803640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e4102ae0 0x7fc4e419a400 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400ebc0 tx=0x7fc4d400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:40.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.179+0000 7fc4e2ffd640 1 -- 172.21.15.3:0/1061201537 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4010630 con 0x7fc4e4102ae0 2024-09-21T06:12:40.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.179+0000 7fc4eba67640 1 -- 172.21.15.3:0/1061201537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e41a0660 con 0x7fc4e4102ae0 2024-09-21T06:12:40.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.179+0000 7fc4e2ffd640 1 -- 172.21.15.3:0/1061201537 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4d4010c80 con 0x7fc4e4102ae0 2024-09-21T06:12:40.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.179+0000 7fc4e2ffd640 1 -- 172.21.15.3:0/1061201537 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4018b60 con 0x7fc4e4102ae0 2024-09-21T06:12:40.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.179+0000 7fc4eba67640 1 -- 172.21.15.3:0/1061201537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e41a6dc0 con 0x7fc4e4102ae0 2024-09-21T06:12:40.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.180+0000 7fc4eba67640 1 -- 172.21.15.3:0/1061201537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e4103d10 con 0x7fc4e4102ae0 2024-09-21T06:12:40.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.181+0000 7fc4e2ffd640 1 -- 172.21.15.3:0/1061201537 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7fc4d4018d00 con 0x7fc4e4102ae0 2024-09-21T06:12:40.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.182+0000 7fc4e2ffd640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc4cc0b6560 0x7fc4cc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:40.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.182+0000 7fc4e9002640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc4cc0b6560 0x7fc4cc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:40.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.182+0000 7fc4e2ffd640 1 -- 172.21.15.3:0/1061201537 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fc4d40e7b30 con 0x7fc4e4102ae0 2024-09-21T06:12:40.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.183+0000 7fc4e9002640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc4cc0b6560 0x7fc4cc0b8a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4102700 tx=0x7fc4d800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:40.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.185+0000 7fc4e2ffd640 1 -- 172.21.15.3:0/1061201537 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc4d40ae6f0 con 0x7fc4e4102ae0 2024-09-21T06:12:40.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.410+0000 7fc4eba67640 1 -- 172.21.15.3:0/1061201537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4e4102ef0 con 0x7fc4e4102ae0 2024-09-21T06:12:40.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.411+0000 7fc4e2ffd640 1 -- 172.21.15.3:0/1061201537 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7fc4d40b2ad0 con 0x7fc4e4102ae0 2024-09-21T06:12:40.414 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:12:40.414 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"btime":"2024-09-21T06:12:38:222351+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24631,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T06:12:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 5 2024-09-21T06:12:40.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.414+0000 7fc4eba67640 1 -- 172.21.15.3:0/1061201537 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc4cc0b6560 msgr2=0x7fc4cc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:40.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.414+0000 7fc4eba67640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc4cc0b6560 0x7fc4cc0b8a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4102700 tx=0x7fc4d800b410 comp rx=0 tx=0).stop 2024-09-21T06:12:40.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.414+0000 7fc4eba67640 1 -- 172.21.15.3:0/1061201537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e4102ae0 msgr2=0x7fc4e419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:40.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.414+0000 7fc4eba67640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e4102ae0 0x7fc4e419a400 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400ebc0 tx=0x7fc4d400c6a0 comp rx=0 tx=0).stop 2024-09-21T06:12:40.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.414+0000 7fc4eba67640 1 -- 172.21.15.3:0/1061201537 shutdown_connections 2024-09-21T06:12:40.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.414+0000 7fc4eba67640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc4cc0b6560 0x7fc4cc0b8a70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:40.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.414+0000 7fc4eba67640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e4108d20 0x7fc4e419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.414+0000 7fc4eba67640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e41083d0 0x7fc4e419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.414+0000 7fc4eba67640 1 --2- 172.21.15.3:0/1061201537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e4102ae0 0x7fc4e419a400 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.414+0000 7fc4eba67640 1 -- 172.21.15.3:0/1061201537 >> 172.21.15.3:0/1061201537 conn(0x7fc4e4079760 msgr2=0x7fc4e406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.414+0000 7fc4eba67640 1 -- 172.21.15.3:0/1061201537 shutdown_connections 2024-09-21T06:12:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:40.414+0000 7fc4eba67640 1 -- 172.21.15.3:0/1061201537 wait complete. 2024-09-21T06:12:40.460 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:12:40 smithi137 podman[62506]: 2024-09-21 06:12:40.085194456 +0000 UTC m=+0.015702887 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:12:40.460 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:12:40 smithi137 podman[62506]: 2024-09-21 06:12:40.206866178 +0000 UTC m=+0.137374643 container create 367b67e561674325194d629fc3fe477d637e6b2e2bf7d123370f77254b501ccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T06:12:40.460 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:12:40 smithi137 podman[62506]: 2024-09-21 06:12:40.459556746 +0000 UTC m=+0.390065170 container init 367b67e561674325194d629fc3fe477d637e6b2e2bf7d123370f77254b501ccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T06:12:40.895 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:12:40 smithi137 podman[62506]: 2024-09-21 06:12:40.462419611 +0000 UTC m=+0.392928038 container start 367b67e561674325194d629fc3fe477d637e6b2e2bf7d123370f77254b501ccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T06:12:40.895 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:12:40 smithi137 ceph-mds[62526]: -- 172.21.15.137:0/284097668 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca7e193860 con 0x55ca8233ba80 2024-09-21T06:12:40.895 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:12:40 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e[62522]: starting mds.e at 2024-09-21T06:12:40.896 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:12:40 smithi137 bash[62506]: 367b67e561674325194d629fc3fe477d637e6b2e2bf7d123370f77254b501ccd 2024-09-21T06:12:40.896 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:12:40 smithi137 systemd[1]: Started Ceph mds.e for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:12:41.079 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:41 smithi003 ceph-mon[31227]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:41.079 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:41 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1061201537' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:41.079 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:41 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:41.079 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:41 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:41.079 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:41 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:41.079 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:41 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:41.079 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:41 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:41.079 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:41 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:41.300 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:41 smithi204 ceph-mon[34659]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:41.300 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:41 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1061201537' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:41.300 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:41 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:41.300 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:41 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:41.300 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:41 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:41.300 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:41 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:41.301 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:41 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:41.301 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:41 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:41.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:41 smithi137 ceph-mon[34464]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:41.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:41 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1061201537' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:41.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:41 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:41.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:41 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:41.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:41 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:41.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:41 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:41.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:41 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:41.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:41 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:42.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:42 smithi003 ceph-mon[31227]: Deploying daemon mds.f on smithi204 2024-09-21T06:12:42.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:42 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.137:6834/399369618,v1:172.21.15.137:6835/399369618] up:boot 2024-09-21T06:12:42.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:42 smithi003 ceph-mon[31227]: fsmap 5 up:standby 2024-09-21T06:12:42.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:42 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:12:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:42 smithi137 ceph-mon[34464]: Deploying daemon mds.f on smithi204 2024-09-21T06:12:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:42 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.137:6834/399369618,v1:172.21.15.137:6835/399369618] up:boot 2024-09-21T06:12:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:42 smithi137 ceph-mon[34464]: fsmap 5 up:standby 2024-09-21T06:12:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:42 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:12:42.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:42 smithi204 ceph-mon[34659]: Deploying daemon mds.f on smithi204 2024-09-21T06:12:42.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:42 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.137:6834/399369618,v1:172.21.15.137:6835/399369618] up:boot 2024-09-21T06:12:42.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:42 smithi204 ceph-mon[34659]: fsmap 5 up:standby 2024-09-21T06:12:42.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:42 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:12:42.844 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:12:42 smithi204 systemd[1]: Starting Ceph mds.f for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:12:43.108 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:12:43.118 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:43 smithi204 ceph-mon[34659]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:43.119 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:12:42 smithi204 podman[63010]: 2024-09-21 06:12:42.74584064 +0000 UTC m=+0.016865312 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:12:43.119 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:12:42 smithi204 podman[63010]: 2024-09-21 06:12:42.864897371 +0000 UTC m=+0.135922037 container create 682f556a8944104417fbb329ae0a187f43e3a285033d727dfddbf6cb5336f89a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-f, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:12:43.119 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:12:43 smithi204 podman[63010]: 2024-09-21 06:12:43.118193326 +0000 UTC m=+0.389217989 container init 682f556a8944104417fbb329ae0a187f43e3a285033d727dfddbf6cb5336f89a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-f, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T06:12:43.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:43 smithi003 ceph-mon[31227]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:43 smithi137 ceph-mon[34464]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:43.404 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:12:43 smithi204 podman[63010]: 2024-09-21 06:12:43.121228452 +0000 UTC m=+0.392253131 container start 682f556a8944104417fbb329ae0a187f43e3a285033d727dfddbf6cb5336f89a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-f, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T06:12:43.404 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:12:43 smithi204 ceph-mds[63029]: -- 172.21.15.204:0/1092477270 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557ec862d860 con 0x557ecc7d5180 2024-09-21T06:12:43.404 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:12:43 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-f[63025]: starting mds.f at 2024-09-21T06:12:43.404 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:12:43 smithi204 bash[63010]: 682f556a8944104417fbb329ae0a187f43e3a285033d727dfddbf6cb5336f89a 2024-09-21T06:12:43.404 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:12:43 smithi204 systemd[1]: Started Ceph mds.f for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:12:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.871+0000 7ff7e25a4640 1 -- 172.21.15.3:0/792440033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7dc108d20 msgr2=0x7ff7dc10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.871+0000 7ff7e25a4640 1 --2- 172.21.15.3:0/792440033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7dc108d20 0x7ff7dc10f550 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009ca0 tx=0x7ff7d402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:12:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.871+0000 7ff7e25a4640 1 -- 172.21.15.3:0/792440033 shutdown_connections 2024-09-21T06:12:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.871+0000 7ff7e25a4640 1 --2- 172.21.15.3:0/792440033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7dc1083d0 0x7ff7dc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.871+0000 7ff7e25a4640 1 --2- 172.21.15.3:0/792440033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7dc108d20 0x7ff7dc10f550 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.871+0000 7ff7e25a4640 1 --2- 172.21.15.3:0/792440033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7dc102ae0 0x7ff7dc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.871+0000 7ff7e25a4640 1 -- 172.21.15.3:0/792440033 >> 172.21.15.3:0/792440033 conn(0x7ff7dc079760 msgr2=0x7ff7dc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.872+0000 7ff7e25a4640 1 -- 172.21.15.3:0/792440033 shutdown_connections 2024-09-21T06:12:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.872+0000 7ff7e25a4640 1 -- 172.21.15.3:0/792440033 wait complete. 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.872+0000 7ff7e25a4640 1 Processor -- start 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.872+0000 7ff7e25a4640 1 -- start start 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e25a4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7dc102ae0 0x7ff7dc199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e25a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7dc1083d0 0x7ff7dc1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e25a4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7dc108d20 0x7ff7dc19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e25a4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc112990 con 0x7ff7dc108d20 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e25a4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc112810 con 0x7ff7dc1083d0 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e25a4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc112b10 con 0x7ff7dc102ae0 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e0b41640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7dc108d20 0x7ff7dc19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e0b41640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7dc108d20 0x7ff7dc19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60274/0 (socket says 172.21.15.3:60274) 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e0b41640 1 -- 172.21.15.3:0/1979069027 learned_addr learned my addr 172.21.15.3:0/1979069027 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7dbfff640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7dc102ae0 0x7ff7dc199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7db7fe640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7dc1083d0 0x7ff7dc1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e0b41640 1 -- 172.21.15.3:0/1979069027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7dc102ae0 msgr2=0x7ff7dc199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e0b41640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7dc102ae0 0x7ff7dc199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e0b41640 1 -- 172.21.15.3:0/1979069027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7dc1083d0 msgr2=0x7ff7dc1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e0b41640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7dc1083d0 0x7ff7dc1998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e0b41640 1 -- 172.21.15.3:0/1979069027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7dc19f270 con 0x7ff7dc108d20 2024-09-21T06:12:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7db7fe640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7dc1083d0 0x7ff7dc1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.873+0000 7ff7e0b41640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7dc108d20 0x7ff7dc19eb40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4005cc0 tx=0x7ff7d402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.874+0000 7ff7d97fa640 1 -- 172.21.15.3:0/1979069027 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4003bf0 con 0x7ff7dc108d20 2024-09-21T06:12:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.874+0000 7ff7d97fa640 1 -- 172.21.15.3:0/1979069027 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d4003d90 con 0x7ff7dc108d20 2024-09-21T06:12:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.874+0000 7ff7e25a4640 1 -- 172.21.15.3:0/1979069027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7dc19f500 con 0x7ff7dc108d20 2024-09-21T06:12:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.874+0000 7ff7d97fa640 1 -- 172.21.15.3:0/1979069027 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d402fbc0 con 0x7ff7dc108d20 2024-09-21T06:12:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.874+0000 7ff7e25a4640 1 -- 172.21.15.3:0/1979069027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7dc1a5cc0 con 0x7ff7dc108d20 2024-09-21T06:12:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.875+0000 7ff7e25a4640 1 -- 172.21.15.3:0/1979069027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7dc103d10 con 0x7ff7dc108d20 2024-09-21T06:12:43.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.876+0000 7ff7d97fa640 1 -- 172.21.15.3:0/1979069027 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7ff7d402fd60 con 0x7ff7dc108d20 2024-09-21T06:12:43.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.877+0000 7ff7d97fa640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7c40b6560 0x7ff7c40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:43.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.877+0000 7ff7dbfff640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7c40b6560 0x7ff7c40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:43.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.877+0000 7ff7d97fa640 1 -- 172.21.15.3:0/1979069027 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7ff7d41085d0 con 0x7ff7dc108d20 2024-09-21T06:12:43.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.878+0000 7ff7dbfff640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7c40b6560 0x7ff7c40b8a70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc005d00 tx=0x7ff7cc00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:43.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:43.882+0000 7ff7d97fa640 1 -- 172.21.15.3:0/1979069027 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff7d40cf190 con 0x7ff7dc108d20 2024-09-21T06:12:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.109+0000 7ff7e25a4640 1 -- 172.21.15.3:0/1979069027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7dc10ab70 con 0x7ff7dc108d20 2024-09-21T06:12:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.110+0000 7ff7d97fa640 1 -- 172.21.15.3:0/1979069027 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5557 (secure 0 0 0) 0x7ff7d40098b0 con 0x7ff7dc108d20 2024-09-21T06:12:44.112 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:12:44.112 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-21T06:12:44:013511+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24631,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24644,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-21T06:12:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-21T06:12:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.112+0000 7ff7e25a4640 1 -- 172.21.15.3:0/1979069027 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7c40b6560 msgr2=0x7ff7c40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.112+0000 7ff7e25a4640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7c40b6560 0x7ff7c40b8a70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc005d00 tx=0x7ff7cc00b410 comp rx=0 tx=0).stop 2024-09-21T06:12:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.112+0000 7ff7e25a4640 1 -- 172.21.15.3:0/1979069027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7dc108d20 msgr2=0x7ff7dc19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.112+0000 7ff7e25a4640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7dc108d20 0x7ff7dc19eb40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4005cc0 tx=0x7ff7d402f380 comp rx=0 tx=0).stop 2024-09-21T06:12:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.112+0000 7ff7e25a4640 1 -- 172.21.15.3:0/1979069027 shutdown_connections 2024-09-21T06:12:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.112+0000 7ff7e25a4640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7c40b6560 0x7ff7c40b8a70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.112+0000 7ff7e25a4640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7dc108d20 0x7ff7dc19eb40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.112+0000 7ff7e25a4640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7dc1083d0 0x7ff7dc1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.112+0000 7ff7e25a4640 1 --2- 172.21.15.3:0/1979069027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7dc102ae0 0x7ff7dc199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.112+0000 7ff7e25a4640 1 -- 172.21.15.3:0/1979069027 >> 172.21.15.3:0/1979069027 conn(0x7ff7dc079760 msgr2=0x7ff7dc1043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.112+0000 7ff7e25a4640 1 -- 172.21.15.3:0/1979069027 shutdown_connections 2024-09-21T06:12:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:44.112+0000 7ff7e25a4640 1 -- 172.21.15.3:0/1979069027 wait complete. 2024-09-21T06:12:44.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:44 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:44.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:44 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:44.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:44 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:44.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:44 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:44.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:44 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:44.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:44 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:44.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:44 smithi003 ceph-mon[31227]: Deploying daemon mds.c on smithi204 2024-09-21T06:12:44.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:44 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] up:boot 2024-09-21T06:12:44.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:44 smithi003 ceph-mon[31227]: fsmap 6 up:standby 2024-09-21T06:12:44.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:44 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:12:44.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:44 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1979069027' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:44 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:44 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:44 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:44 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:44 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:44 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:44 smithi137 ceph-mon[34464]: Deploying daemon mds.c on smithi204 2024-09-21T06:12:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:44 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] up:boot 2024-09-21T06:12:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:44 smithi137 ceph-mon[34464]: fsmap 6 up:standby 2024-09-21T06:12:44.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:44 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:12:44.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:44 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1979069027' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:44.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:44 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:44.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:44 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:44.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:44 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:44.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:44 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:44.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:44 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:44.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:44 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:44.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:44 smithi204 ceph-mon[34659]: Deploying daemon mds.c on smithi204 2024-09-21T06:12:44.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:44 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] up:boot 2024-09-21T06:12:44.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:44 smithi204 ceph-mon[34659]: fsmap 6 up:standby 2024-09-21T06:12:44.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:44 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:12:44.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:44 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1979069027' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:45.301 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:12:45 smithi204 systemd[1]: Starting Ceph mds.c for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:12:45.552 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:45 smithi204 ceph-mon[34659]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:45.553 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:12:45 smithi204 podman[63279]: 2024-09-21 06:12:45.308837513 +0000 UTC m=+0.016616387 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:12:45.553 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:12:45 smithi204 podman[63279]: 2024-09-21 06:12:45.429583223 +0000 UTC m=+0.137362097 container create a9f07c06da34ceacca1d1f0560814156d44136883a1cdaaa4674836e93ac1d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-21T06:12:45.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:45 smithi003 ceph-mon[31227]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:45.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:45 smithi137 ceph-mon[34464]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:46.154 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:12:45 smithi204 podman[63279]: 2024-09-21 06:12:45.682766716 +0000 UTC m=+0.390545597 container init a9f07c06da34ceacca1d1f0560814156d44136883a1cdaaa4674836e93ac1d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T06:12:46.154 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:12:45 smithi204 podman[63279]: 2024-09-21 06:12:45.686003684 +0000 UTC m=+0.393782561 container start a9f07c06da34ceacca1d1f0560814156d44136883a1cdaaa4674836e93ac1d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-c, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T06:12:46.154 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:12:45 smithi204 ceph-mds[63298]: -- 172.21.15.204:0/1285424375 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620a2bd3860 con 0x5620a6d7b600 2024-09-21T06:12:46.155 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:12:45 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-c[63294]: starting mds.c at 2024-09-21T06:12:46.155 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:12:45 smithi204 bash[63279]: a9f07c06da34ceacca1d1f0560814156d44136883a1cdaaa4674836e93ac1d27 2024-09-21T06:12:46.155 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:12:45 smithi204 systemd[1]: Started Ceph mds.c for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:12:46.827 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:12:47.088 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:47.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:46 smithi137 ceph-mon[34464]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:47.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:46 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:47.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:46 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:47.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:46 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:47.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:46 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:47.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:46 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:47.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:46 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:47.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:46 smithi137 ceph-mon[34464]: Deploying daemon mds.h on smithi137 2024-09-21T06:12:47.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:46 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] up:boot 2024-09-21T06:12:47.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:46 smithi137 ceph-mon[34464]: fsmap 7 up:standby 2024-09-21T06:12:47.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:46 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:12:47.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:46 smithi204 ceph-mon[34659]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:47.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:46 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:47.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:46 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:47.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:46 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:47.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:46 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:47.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:46 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:47.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:46 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:47.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:46 smithi204 ceph-mon[34659]: Deploying daemon mds.h on smithi137 2024-09-21T06:12:47.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:46 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] up:boot 2024-09-21T06:12:47.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:46 smithi204 ceph-mon[34659]: fsmap 7 up:standby 2024-09-21T06:12:47.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:46 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:12:47.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:46 smithi003 ceph-mon[31227]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:47.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:46 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:47.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:46 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:47.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:46 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:47.212 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:46 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:47.212 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:46 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:47.212 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:46 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:47.212 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:46 smithi003 ceph-mon[31227]: Deploying daemon mds.h on smithi137 2024-09-21T06:12:47.212 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:46 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] up:boot 2024-09-21T06:12:47.212 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:46 smithi003 ceph-mon[31227]: fsmap 7 up:standby 2024-09-21T06:12:47.212 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:46 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:12:47.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.640+0000 7f6f63ce6640 1 -- 172.21.15.3:0/584308238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f5c10de10 msgr2=0x7f6f5c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:47.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.640+0000 7f6f63ce6640 1 --2- 172.21.15.3:0/584308238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f5c10de10 0x7f6f5c110300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009ca0 tx=0x7f6f5402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:12:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.640+0000 7f6f63ce6640 1 -- 172.21.15.3:0/584308238 shutdown_connections 2024-09-21T06:12:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.640+0000 7f6f63ce6640 1 --2- 172.21.15.3:0/584308238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c105300 0x7f6f5c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.640+0000 7f6f63ce6640 1 --2- 172.21.15.3:0/584308238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f5c10de10 0x7f6f5c110300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.640+0000 7f6f63ce6640 1 --2- 172.21.15.3:0/584308238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c1049b0 0x7f6f5c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.640+0000 7f6f63ce6640 1 -- 172.21.15.3:0/584308238 >> 172.21.15.3:0/584308238 conn(0x7f6f5c079730 msgr2=0x7f6f5c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.640+0000 7f6f63ce6640 1 -- 172.21.15.3:0/584308238 shutdown_connections 2024-09-21T06:12:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.640+0000 7f6f63ce6640 1 -- 172.21.15.3:0/584308238 wait complete. 2024-09-21T06:12:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.641+0000 7f6f63ce6640 1 Processor -- start 2024-09-21T06:12:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.641+0000 7f6f63ce6640 1 -- start start 2024-09-21T06:12:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.641+0000 7f6f63ce6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f5c1049b0 0x7f6f5c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.641+0000 7f6f63ce6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c105300 0x7f6f5c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.641+0000 7f6f63ce6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c10de10 0x7f6f5c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.641+0000 7f6f63ce6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c113a70 con 0x7f6f5c1049b0 2024-09-21T06:12:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.641+0000 7f6f63ce6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c1138f0 con 0x7f6f5c105300 2024-09-21T06:12:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.641+0000 7f6f63ce6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c113bf0 con 0x7f6f5c10de10 2024-09-21T06:12:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f61a82640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f5c1049b0 0x7f6f5c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f61a82640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f5c1049b0 0x7f6f5c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39994/0 (socket says 172.21.15.3:39994) 2024-09-21T06:12:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f61a82640 1 -- 172.21.15.3:0/1459423096 learned_addr learned my addr 172.21.15.3:0/1459423096 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f61281640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c105300 0x7f6f5c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f62283640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c10de10 0x7f6f5c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f61a82640 1 -- 172.21.15.3:0/1459423096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c10de10 msgr2=0x7f6f5c19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f61a82640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c10de10 0x7f6f5c19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f61a82640 1 -- 172.21.15.3:0/1459423096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c105300 msgr2=0x7f6f5c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f61a82640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c105300 0x7f6f5c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f61a82640 1 -- 172.21.15.3:0/1459423096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5c1a0350 con 0x7f6f5c1049b0 2024-09-21T06:12:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f61a82640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f5c1049b0 0x7f6f5c19a4a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800e9c0 tx=0x7f6f5800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f52ffd640 1 -- 172.21.15.3:0/1459423096 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800ccc0 con 0x7f6f5c1049b0 2024-09-21T06:12:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f52ffd640 1 -- 172.21.15.3:0/1459423096 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f580035f0 con 0x7f6f5c1049b0 2024-09-21T06:12:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f63ce6640 1 -- 172.21.15.3:0/1459423096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f5c1a0640 con 0x7f6f5c1049b0 2024-09-21T06:12:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f52ffd640 1 -- 172.21.15.3:0/1459423096 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800ce60 con 0x7f6f5c1049b0 2024-09-21T06:12:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.642+0000 7f6f63ce6640 1 -- 172.21.15.3:0/1459423096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f5c1a6e60 con 0x7f6f5c1049b0 2024-09-21T06:12:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.643+0000 7f6f63ce6640 1 -- 172.21.15.3:0/1459423096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f5c104dc0 con 0x7f6f5c1049b0 2024-09-21T06:12:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.645+0000 7f6f52ffd640 1 -- 172.21.15.3:0/1459423096 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f6f580037a0 con 0x7f6f5c1049b0 2024-09-21T06:12:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.646+0000 7f6f52ffd640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6f480b6590 0x7f6f480b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.646+0000 7f6f61281640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6f480b6590 0x7f6f480b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.646+0000 7f6f52ffd640 1 -- 172.21.15.3:0/1459423096 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f6f580e74c0 con 0x7f6f5c1049b0 2024-09-21T06:12:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.647+0000 7f6f61281640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6f480b6590 0x7f6f480b8aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c07a030 tx=0x7f6f4c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.649+0000 7f6f52ffd640 1 -- 172.21.15.3:0/1459423096 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6f580ae110 con 0x7f6f5c1049b0 2024-09-21T06:12:47.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.875+0000 7f6f63ce6640 1 -- 172.21.15.3:0/1459423096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f5c19b5d0 con 0x7f6f5c1049b0 2024-09-21T06:12:47.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.876+0000 7f6f52ffd640 1 -- 172.21.15.3:0/1459423096 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6385 (secure 0 0 0) 0x7f6f580b24f0 con 0x7f6f5c1049b0 2024-09-21T06:12:47.878 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:12:47.878 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"btime":"2024-09-21T06:12:46:299443+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24631,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24644,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T06:12:47.878 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-09-21T06:12:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.878+0000 7f6f63ce6640 1 -- 172.21.15.3:0/1459423096 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6f480b6590 msgr2=0x7f6f480b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.878+0000 7f6f63ce6640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6f480b6590 0x7f6f480b8aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c07a030 tx=0x7f6f4c00a2d0 comp rx=0 tx=0).stop 2024-09-21T06:12:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.878+0000 7f6f63ce6640 1 -- 172.21.15.3:0/1459423096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f5c1049b0 msgr2=0x7f6f5c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.878+0000 7f6f63ce6640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f5c1049b0 0x7f6f5c19a4a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800e9c0 tx=0x7f6f5800edc0 comp rx=0 tx=0).stop 2024-09-21T06:12:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.878+0000 7f6f63ce6640 1 -- 172.21.15.3:0/1459423096 shutdown_connections 2024-09-21T06:12:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.878+0000 7f6f63ce6640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6f480b6590 0x7f6f480b8aa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.878+0000 7f6f63ce6640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c10de10 0x7f6f5c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.878+0000 7f6f63ce6640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c105300 0x7f6f5c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.878+0000 7f6f63ce6640 1 --2- 172.21.15.3:0/1459423096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f5c1049b0 0x7f6f5c19a4a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.878+0000 7f6f63ce6640 1 -- 172.21.15.3:0/1459423096 >> 172.21.15.3:0/1459423096 conn(0x7f6f5c079730 msgr2=0x7f6f5c1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.878+0000 7f6f63ce6640 1 -- 172.21.15.3:0/1459423096 shutdown_connections 2024-09-21T06:12:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:47.878+0000 7f6f63ce6640 1 -- 172.21.15.3:0/1459423096 wait complete. 2024-09-21T06:12:48.091 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:12:47 smithi137 systemd[1]: Starting Ceph mds.h for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:12:48.395 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:12:48 smithi137 podman[62776]: 2024-09-21 06:12:47.992269209 +0000 UTC m=+0.015905602 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:12:48.395 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:12:48 smithi137 podman[62776]: 2024-09-21 06:12:48.120518743 +0000 UTC m=+0.144155117 container create 155436ebcdc74ac44d60e50a03293990a322f0217e8ac5ff26976550933f8235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-h, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-21T06:12:48.849 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:12:48 smithi137 podman[62776]: 2024-09-21 06:12:48.398480995 +0000 UTC m=+0.422117368 container init 155436ebcdc74ac44d60e50a03293990a322f0217e8ac5ff26976550933f8235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2) 2024-09-21T06:12:48.849 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:12:48 smithi137 podman[62776]: 2024-09-21 06:12:48.401435631 +0000 UTC m=+0.425072005 container start 155436ebcdc74ac44d60e50a03293990a322f0217e8ac5ff26976550933f8235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T06:12:48.849 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:12:48 smithi137 ceph-mds[62795]: -- 172.21.15.137:0/832661730 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562669bf9c20 con 0x56266dda1a80 2024-09-21T06:12:48.849 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:12:48 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-h[62791]: starting mds.h at 2024-09-21T06:12:48.849 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:12:48 smithi137 bash[62776]: 155436ebcdc74ac44d60e50a03293990a322f0217e8ac5ff26976550933f8235 2024-09-21T06:12:48.849 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:12:48 smithi137 systemd[1]: Started Ceph mds.h for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:12:48.966 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:48 smithi204 ceph-mon[34659]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:48.967 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:48 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1459423096' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:48.967 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:48 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:48.967 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:48 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:48.967 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:48 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:48.967 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:48 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:48.967 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:48 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:48.967 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:48 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:49.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:48 smithi137 ceph-mon[34464]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:49.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:48 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1459423096' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:49.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:48 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:49.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:48 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:49.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:48 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:49.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:48 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:49.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:48 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:49.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:48 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:49.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:48 smithi003 ceph-mon[31227]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:49.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:48 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1459423096' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:49.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:48 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:49.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:48 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:49.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:48 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:49.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:48 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:49.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:48 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:49.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:48 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:50.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:49 smithi137 ceph-mon[34464]: Deploying daemon mds.i on smithi204 2024-09-21T06:12:50.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:49 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] up:boot 2024-09-21T06:12:50.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:49 smithi137 ceph-mon[34464]: fsmap 8 up:standby 2024-09-21T06:12:50.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:49 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:12:50.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:49 smithi204 ceph-mon[34659]: Deploying daemon mds.i on smithi204 2024-09-21T06:12:50.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:49 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] up:boot 2024-09-21T06:12:50.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:49 smithi204 ceph-mon[34659]: fsmap 8 up:standby 2024-09-21T06:12:50.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:49 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:12:50.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:49 smithi003 ceph-mon[31227]: Deploying daemon mds.i on smithi204 2024-09-21T06:12:50.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:49 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] up:boot 2024-09-21T06:12:50.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:49 smithi003 ceph-mon[31227]: fsmap 8 up:standby 2024-09-21T06:12:50.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:49 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:12:50.542 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:12:50 smithi204 systemd[1]: Starting Ceph mds.i for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:12:50.574 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:12:50.847 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:50.860 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:12:50 smithi204 podman[63548]: 2024-09-21 06:12:50.656416863 +0000 UTC m=+0.016202310 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:12:50.861 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:12:50 smithi204 podman[63548]: 2024-09-21 06:12:50.777654162 +0000 UTC m=+0.137439596 container create 648de31a88de9dfb31caa3cc319191790ca3bc0951f4b670d9427359df6af911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-i, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:12:51.142 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:50 smithi003 ceph-mon[31227]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:51.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:50 smithi137 ceph-mon[34464]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:51.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:50 smithi204 ceph-mon[34659]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:51.148 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:12:51 smithi204 podman[63548]: 2024-09-21 06:12:51.038916332 +0000 UTC m=+0.398701781 container init 648de31a88de9dfb31caa3cc319191790ca3bc0951f4b670d9427359df6af911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T06:12:51.148 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:12:51 smithi204 podman[63548]: 2024-09-21 06:12:51.041808171 +0000 UTC m=+0.401593620 container start 648de31a88de9dfb31caa3cc319191790ca3bc0951f4b670d9427359df6af911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-i, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T06:12:51.148 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:12:51 smithi204 ceph-mds[63567]: -- 172.21.15.204:0/2291942636 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55623c87dc20 con 0x556240a25a80 2024-09-21T06:12:51.148 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:12:51 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-i[63563]: starting mds.i at 2024-09-21T06:12:51.149 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:12:51 smithi204 bash[63548]: 648de31a88de9dfb31caa3cc319191790ca3bc0951f4b670d9427359df6af911 2024-09-21T06:12:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.375+0000 7f5573a5c640 1 -- 172.21.15.3:0/668106290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c075dd0 msgr2=0x7f556c0761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.375+0000 7f5573a5c640 1 --2- 172.21.15.3:0/668106290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c075dd0 0x7f556c0761e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f555c009900 tx=0x7f555c02a410 comp rx=0 tx=0).stop 2024-09-21T06:12:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.375+0000 7f5573a5c640 1 -- 172.21.15.3:0/668106290 shutdown_connections 2024-09-21T06:12:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.375+0000 7f5573a5c640 1 --2- 172.21.15.3:0/668106290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f556c114460 0x7f556c116930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.375+0000 7f5573a5c640 1 --2- 172.21.15.3:0/668106290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c075dd0 0x7f556c0761e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.375+0000 7f5573a5c640 1 --2- 172.21.15.3:0/668106290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f556c077380 0x7f556c0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.375+0000 7f5573a5c640 1 -- 172.21.15.3:0/668106290 >> 172.21.15.3:0/668106290 conn(0x7f556c0ff600 msgr2=0x7f556c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.375+0000 7f5573a5c640 1 -- 172.21.15.3:0/668106290 shutdown_connections 2024-09-21T06:12:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.375+0000 7f5573a5c640 1 -- 172.21.15.3:0/668106290 wait complete. 2024-09-21T06:12:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.376+0000 7f5573a5c640 1 Processor -- start 2024-09-21T06:12:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.376+0000 7f5573a5c640 1 -- start start 2024-09-21T06:12:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.376+0000 7f5573a5c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c075dd0 0x7f556c1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.376+0000 7f5573a5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f556c077380 0x7f556c1a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.376+0000 7f5573a5c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f556c114460 0x7f556c1a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.376+0000 7f5573a5c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f556c11a070 con 0x7f556c075dd0 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.376+0000 7f5573a5c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f556c119ef0 con 0x7f556c077380 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.376+0000 7f5573a5c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f556c11a1f0 con 0x7f556c114460 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.376+0000 7f55717f8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c075dd0 0x7f556c1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f55717f8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c075dd0 0x7f556c1a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40014/0 (socket says 172.21.15.3:40014) 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f55717f8640 1 -- 172.21.15.3:0/438773348 learned_addr learned my addr 172.21.15.3:0/438773348 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f5570ff7640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f556c077380 0x7f556c1a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f5571ff9640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f556c114460 0x7f556c1a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f55717f8640 1 -- 172.21.15.3:0/438773348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f556c114460 msgr2=0x7f556c1a6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f55717f8640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f556c114460 0x7f556c1a6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f55717f8640 1 -- 172.21.15.3:0/438773348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f556c077380 msgr2=0x7f556c1a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f55717f8640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f556c077380 0x7f556c1a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f55717f8640 1 -- 172.21.15.3:0/438773348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c1a68f0 con 0x7f556c075dd0 2024-09-21T06:12:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f5571ff9640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f556c114460 0x7f556c1a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f55717f8640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c075dd0 0x7f556c1a0a40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f556800c920 tx=0x7f556800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f55627fc640 1 -- 172.21.15.3:0/438773348 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568007c70 con 0x7f556c075dd0 2024-09-21T06:12:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f5573a5c640 1 -- 172.21.15.3:0/438773348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556c1a6be0 con 0x7f556c075dd0 2024-09-21T06:12:51.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f55627fc640 1 -- 172.21.15.3:0/438773348 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5568007e10 con 0x7f556c075dd0 2024-09-21T06:12:51.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.377+0000 7f5573a5c640 1 -- 172.21.15.3:0/438773348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556c1ad400 con 0x7f556c075dd0 2024-09-21T06:12:51.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.378+0000 7f55627fc640 1 -- 172.21.15.3:0/438773348 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568018820 con 0x7f556c075dd0 2024-09-21T06:12:51.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.379+0000 7f5573a5c640 1 -- 172.21.15.3:0/438773348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55500052c0 con 0x7f556c075dd0 2024-09-21T06:12:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.381+0000 7f55627fc640 1 -- 172.21.15.3:0/438773348 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f55680189c0 con 0x7f556c075dd0 2024-09-21T06:12:51.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.382+0000 7f55627fc640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55580b6590 0x7f55580b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:51.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.382+0000 7f5570ff7640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55580b6590 0x7f55580b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:51.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.382+0000 7f55627fc640 1 -- 172.21.15.3:0/438773348 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f55680e7040 con 0x7f556c075dd0 2024-09-21T06:12:51.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.383+0000 7f5570ff7640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55580b6590 0x7f55580b8aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f555c000c00 tx=0x7f555c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.386+0000 7f55627fc640 1 -- 172.21.15.3:0/438773348 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5568014030 con 0x7f556c075dd0 2024-09-21T06:12:51.404 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:12:51 smithi204 systemd[1]: Started Ceph mds.i for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:12:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.636+0000 7f5573a5c640 1 -- 172.21.15.3:0/438773348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55500055b0 con 0x7f556c075dd0 2024-09-21T06:12:51.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.637+0000 7f55627fc640 1 -- 172.21.15.3:0/438773348 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7207 (secure 0 0 0) 0x7f55680adc00 con 0x7f556c075dd0 2024-09-21T06:12:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:12:51.639 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"btime":"2024-09-21T06:12:48:844601+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24631,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24644,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24674,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T06:12:51.639 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2024-09-21T06:12:51.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.639+0000 7f5573a5c640 1 -- 172.21.15.3:0/438773348 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55580b6590 msgr2=0x7f55580b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:51.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.639+0000 7f5573a5c640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55580b6590 0x7f55580b8aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f555c000c00 tx=0x7f555c031400 comp rx=0 tx=0).stop 2024-09-21T06:12:51.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.639+0000 7f5573a5c640 1 -- 172.21.15.3:0/438773348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c075dd0 msgr2=0x7f556c1a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:51.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.639+0000 7f5573a5c640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c075dd0 0x7f556c1a0a40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f556800c920 tx=0x7f556800cde0 comp rx=0 tx=0).stop 2024-09-21T06:12:51.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.639+0000 7f5573a5c640 1 -- 172.21.15.3:0/438773348 shutdown_connections 2024-09-21T06:12:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.639+0000 7f5573a5c640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55580b6590 0x7f55580b8aa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.639+0000 7f5573a5c640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f556c114460 0x7f556c1a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.639+0000 7f5573a5c640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f556c077380 0x7f556c1a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.639+0000 7f5573a5c640 1 --2- 172.21.15.3:0/438773348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c075dd0 0x7f556c1a0a40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.639+0000 7f5573a5c640 1 -- 172.21.15.3:0/438773348 >> 172.21.15.3:0/438773348 conn(0x7f556c0ff600 msgr2=0x7f556c110360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.639+0000 7f5573a5c640 1 -- 172.21.15.3:0/438773348 shutdown_connections 2024-09-21T06:12:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:51.639+0000 7f5573a5c640 1 -- 172.21.15.3:0/438773348 wait complete. 2024-09-21T06:12:52.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:52.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:52.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:52.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:52.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:52.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:52.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:52 smithi003 ceph-mon[31227]: Deploying daemon mds.g on smithi003 2024-09-21T06:12:52.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:52 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/438773348' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:52.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:52 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] up:boot 2024-09-21T06:12:52.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:52 smithi003 ceph-mon[31227]: fsmap 9 up:standby 2024-09-21T06:12:52.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:52 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:12:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:52.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:52.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:52 smithi137 ceph-mon[34464]: Deploying daemon mds.g on smithi003 2024-09-21T06:12:52.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:52 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/438773348' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:52.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:52 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] up:boot 2024-09-21T06:12:52.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:52 smithi137 ceph-mon[34464]: fsmap 9 up:standby 2024-09-21T06:12:52.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:52 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:12:52.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:52.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:52.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:52.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:52.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:52.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:52.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:52 smithi204 ceph-mon[34659]: Deploying daemon mds.g on smithi003 2024-09-21T06:12:52.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:52 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/438773348' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:52.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:52 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] up:boot 2024-09-21T06:12:52.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:52 smithi204 ceph-mon[34659]: fsmap 9 up:standby 2024-09-21T06:12:52.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:52 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:12:53.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:53 smithi003 ceph-mon[31227]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:53.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:53 smithi137 ceph-mon[34464]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:53.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:53 smithi204 ceph-mon[34659]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:54.040 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:12:53 smithi003 systemd[1]: Starting Ceph mds.g for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:12:54.321 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:12:54 smithi003 podman[73748]: 2024-09-21 06:12:53.941450036 +0000 UTC m=+0.017798008 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:12:54.321 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:12:54 smithi003 podman[73748]: 2024-09-21 06:12:54.074848559 +0000 UTC m=+0.151196519 container create 9257d07f9b96a8f80eef8d70f1c78fd4253797782cf9e9262094cf456ff11c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:12:54.321 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:12:54 smithi003 podman[73748]: 2024-09-21 06:12:54.319617485 +0000 UTC m=+0.395965446 container init 9257d07f9b96a8f80eef8d70f1c78fd4253797782cf9e9262094cf456ff11c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-g, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:12:54.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:12:54.590 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:12:54 smithi003 podman[73748]: 2024-09-21 06:12:54.322959314 +0000 UTC m=+0.399307277 container start 9257d07f9b96a8f80eef8d70f1c78fd4253797782cf9e9262094cf456ff11c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T06:12:54.590 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:12:54 smithi003 ceph-mds[73767]: -- 172.21.15.3:0/3425828085 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5635b29cf860 con 0x5635b6b77180 2024-09-21T06:12:54.590 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:12:54 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-g[73763]: starting mds.g at 2024-09-21T06:12:54.590 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:12:54 smithi003 bash[73748]: 9257d07f9b96a8f80eef8d70f1c78fd4253797782cf9e9262094cf456ff11c18 2024-09-21T06:12:54.591 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:12:54 smithi003 systemd[1]: Started Ceph mds.g for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:12:54.691 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:55.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.208+0000 7f639b322640 1 -- 172.21.15.3:0/752152496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63941094a0 msgr2=0x7f63941098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.208+0000 7f639b322640 1 --2- 172.21.15.3:0/752152496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63941094a0 0x7f63941098b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6384009900 tx=0x7f638402a410 comp rx=0 tx=0).stop 2024-09-21T06:12:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.208+0000 7f639b322640 1 -- 172.21.15.3:0/752152496 shutdown_connections 2024-09-21T06:12:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.208+0000 7f639b322640 1 --2- 172.21.15.3:0/752152496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6394109df0 0x7f6394110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.208+0000 7f639b322640 1 --2- 172.21.15.3:0/752152496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63941094a0 0x7f63941098b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.208+0000 7f639b322640 1 --2- 172.21.15.3:0/752152496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6394103b60 0x7f6394103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.208+0000 7f639b322640 1 -- 172.21.15.3:0/752152496 >> 172.21.15.3:0/752152496 conn(0x7f639407b850 msgr2=0x7f639407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.208+0000 7f639b322640 1 -- 172.21.15.3:0/752152496 shutdown_connections 2024-09-21T06:12:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.208+0000 7f639b322640 1 -- 172.21.15.3:0/752152496 wait complete. 2024-09-21T06:12:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f639b322640 1 Processor -- start 2024-09-21T06:12:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f639b322640 1 -- start start 2024-09-21T06:12:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f639b322640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6394103b60 0x7f639419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f639b322640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63941094a0 0x7f639419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f639b322640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6394109df0 0x7f639419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f639b322640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6394113ab0 con 0x7f6394109df0 2024-09-21T06:12:55.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f639b322640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6394113930 con 0x7f6394103b60 2024-09-21T06:12:55.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f639b322640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6394113c30 con 0x7f63941094a0 2024-09-21T06:12:55.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f63998bf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6394109df0 0x7f639419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f63998bf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6394109df0 0x7f639419fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40054/0 (socket says 172.21.15.3:40054) 2024-09-21T06:12:55.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f63998bf640 1 -- 172.21.15.3:0/3320539096 learned_addr learned my addr 172.21.15.3:0/3320539096 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:55.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f63990be640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6394103b60 0x7f639419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f63988bd640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63941094a0 0x7f639419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f63998bf640 1 -- 172.21.15.3:0/3320539096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63941094a0 msgr2=0x7f639419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f63998bf640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63941094a0 0x7f639419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f63998bf640 1 -- 172.21.15.3:0/3320539096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6394103b60 msgr2=0x7f639419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f63998bf640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6394103b60 0x7f639419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f63998bf640 1 -- 172.21.15.3:0/3320539096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63941a04f0 con 0x7f6394109df0 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.209+0000 7f63988bd640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63941094a0 0x7f639419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.210+0000 7f63998bf640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6394109df0 0x7f639419fdc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f638c009fd0 tx=0x7f638c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.210+0000 7f638a7fc640 1 -- 172.21.15.3:0/3320539096 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c017070 con 0x7f6394109df0 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.210+0000 7f638a7fc640 1 -- 172.21.15.3:0/3320539096 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f638c003e20 con 0x7f6394109df0 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.210+0000 7f639b322640 1 -- 172.21.15.3:0/3320539096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63941a07e0 con 0x7f6394109df0 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.210+0000 7f638a7fc640 1 -- 172.21.15.3:0/3320539096 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c010460 con 0x7f6394109df0 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.210+0000 7f639b322640 1 -- 172.21.15.3:0/3320539096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63941a6f40 con 0x7f6394109df0 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.211+0000 7f639b322640 1 -- 172.21.15.3:0/3320539096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6394103f70 con 0x7f6394109df0 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.212+0000 7f638a7fc640 1 -- 172.21.15.3:0/3320539096 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f638c003970 con 0x7f6394109df0 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.213+0000 7f638a7fc640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f63800b65e0 0x7f63800b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.213+0000 7f63990be640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f63800b65e0 0x7f63800b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.213+0000 7f638a7fc640 1 -- 172.21.15.3:0/3320539096 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f638c0e6d20 con 0x7f6394109df0 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.214+0000 7f63990be640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f63800b65e0 0x7f63800b8af0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6390005d90 tx=0x7f6390005d20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.218+0000 7f638a7fc640 1 -- 172.21.15.3:0/3320539096 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f638c0ad8e0 con 0x7f6394109df0 2024-09-21T06:12:55.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:55 smithi003 ceph-mon[31227]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:55.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:55.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:55.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:55.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:55.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:55.368 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:55 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:55.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.450+0000 7f639b322640 1 -- 172.21.15.3:0/3320539096 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f639402e640 con 0x7f6394109df0 2024-09-21T06:12:55.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.450+0000 7f638a7fc640 1 -- 172.21.15.3:0/3320539096 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8860 (secure 0 0 0) 0x7f638c0b1cc0 con 0x7f6394109df0 2024-09-21T06:12:55.453 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:12:55.453 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":11,"btime":"2024-09-21T06:12:55:232299+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24631,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24644,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24674,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24691,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T06:12:55.453 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 11 2024-09-21T06:12:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.452+0000 7f639b322640 1 -- 172.21.15.3:0/3320539096 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f63800b65e0 msgr2=0x7f63800b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.452+0000 7f639b322640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f63800b65e0 0x7f63800b8af0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6390005d90 tx=0x7f6390005d20 comp rx=0 tx=0).stop 2024-09-21T06:12:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.452+0000 7f639b322640 1 -- 172.21.15.3:0/3320539096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6394109df0 msgr2=0x7f639419fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.452+0000 7f639b322640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6394109df0 0x7f639419fdc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f638c009fd0 tx=0x7f638c00b260 comp rx=0 tx=0).stop 2024-09-21T06:12:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.453+0000 7f639b322640 1 -- 172.21.15.3:0/3320539096 shutdown_connections 2024-09-21T06:12:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.453+0000 7f639b322640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f63800b65e0 0x7f63800b8af0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.453+0000 7f639b322640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63941094a0 0x7f639419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.453+0000 7f639b322640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6394109df0 0x7f639419fdc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.453+0000 7f639b322640 1 --2- 172.21.15.3:0/3320539096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6394103b60 0x7f639419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.453+0000 7f639b322640 1 -- 172.21.15.3:0/3320539096 >> 172.21.15.3:0/3320539096 conn(0x7f639407b850 msgr2=0x7f6394105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.453+0000 7f639b322640 1 -- 172.21.15.3:0/3320539096 shutdown_connections 2024-09-21T06:12:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:55.453+0000 7f639b322640 1 -- 172.21.15.3:0/3320539096 wait complete. 2024-09-21T06:12:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:55 smithi137 ceph-mon[34464]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:55.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:55 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:55 smithi204 ceph-mon[34659]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:55 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:56.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:56 smithi003 ceph-mon[31227]: Deploying daemon mds.a on smithi003 2024-09-21T06:12:56.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:56 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.3:6838/1325295587,v1:172.21.15.3:6839/1325295587] up:boot 2024-09-21T06:12:56.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:56 smithi003 ceph-mon[31227]: fsmap 10 up:standby 2024-09-21T06:12:56.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:56 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:12:56.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:56 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3320539096' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:56.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:56 smithi137 ceph-mon[34464]: Deploying daemon mds.a on smithi003 2024-09-21T06:12:56.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:56 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.3:6838/1325295587,v1:172.21.15.3:6839/1325295587] up:boot 2024-09-21T06:12:56.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:56 smithi137 ceph-mon[34464]: fsmap 10 up:standby 2024-09-21T06:12:56.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:56 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:12:56.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:56 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3320539096' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:56.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:56 smithi204 ceph-mon[34659]: Deploying daemon mds.a on smithi003 2024-09-21T06:12:56.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:56 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.3:6838/1325295587,v1:172.21.15.3:6839/1325295587] up:boot 2024-09-21T06:12:56.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:56 smithi204 ceph-mon[34659]: fsmap 10 up:standby 2024-09-21T06:12:56.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:56 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:12:56.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:56 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3320539096' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:57.442 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:57 smithi003 ceph-mon[31227]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:57.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:57 smithi137 ceph-mon[34464]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:57.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:57 smithi204 ceph-mon[34659]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:58.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:12:58.528 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:59.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.016+0000 7fe5ea095640 1 -- 172.21.15.3:0/766951047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5e41083d0 msgr2=0x7fe5e41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.016+0000 7fe5ea095640 1 --2- 172.21.15.3:0/766951047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5e41083d0 0x7fe5e41087e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009900 tx=0x7fe5cc02a410 comp rx=0 tx=0).stop 2024-09-21T06:12:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.016+0000 7fe5ea095640 1 -- 172.21.15.3:0/766951047 shutdown_connections 2024-09-21T06:12:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.016+0000 7fe5ea095640 1 --2- 172.21.15.3:0/766951047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5e4108d20 0x7fe5e410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.016+0000 7fe5ea095640 1 --2- 172.21.15.3:0/766951047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5e41083d0 0x7fe5e41087e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.016+0000 7fe5ea095640 1 --2- 172.21.15.3:0/766951047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e4102ae0 0x7fe5e4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.016+0000 7fe5ea095640 1 -- 172.21.15.3:0/766951047 >> 172.21.15.3:0/766951047 conn(0x7fe5e40797b0 msgr2=0x7fe5e4079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.016+0000 7fe5ea095640 1 -- 172.21.15.3:0/766951047 shutdown_connections 2024-09-21T06:12:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.016+0000 7fe5ea095640 1 -- 172.21.15.3:0/766951047 wait complete. 2024-09-21T06:12:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.017+0000 7fe5ea095640 1 Processor -- start 2024-09-21T06:12:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.017+0000 7fe5ea095640 1 -- start start 2024-09-21T06:12:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5ea095640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5e4102ae0 0x7fe5e419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5ea095640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5e41083d0 0x7fe5e419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5ea095640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e4108d20 0x7fe5e419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5ea095640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4113b00 con 0x7fe5e41083d0 2024-09-21T06:12:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5ea095640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4113980 con 0x7fe5e4108d20 2024-09-21T06:12:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5ea095640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4113c80 con 0x7fe5e4102ae0 2024-09-21T06:12:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5e2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5e41083d0 0x7fe5e419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5e2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5e41083d0 0x7fe5e419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57864/0 (socket says 172.21.15.3:57864) 2024-09-21T06:12:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5e2ffd640 1 -- 172.21.15.3:0/902372315 learned_addr learned my addr 172.21.15.3:0/902372315 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:12:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5e37fe640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5e4102ae0 0x7fe5e419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5e3fff640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e4108d20 0x7fe5e419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5e2ffd640 1 -- 172.21.15.3:0/902372315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5e4102ae0 msgr2=0x7fe5e419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5e2ffd640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5e4102ae0 0x7fe5e419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5e2ffd640 1 -- 172.21.15.3:0/902372315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e4108d20 msgr2=0x7fe5e419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5e2ffd640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e4108d20 0x7fe5e419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5e2ffd640 1 -- 172.21.15.3:0/902372315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e41a03a0 con 0x7fe5e41083d0 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.018+0000 7fe5e2ffd640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5e41083d0 0x7fe5e419aa90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc02a930 tx=0x7fe5cc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.019+0000 7fe5e0ff9640 1 -- 172.21.15.3:0/902372315 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc03f030 con 0x7fe5e41083d0 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.019+0000 7fe5e0ff9640 1 -- 172.21.15.3:0/902372315 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5cc003aa0 con 0x7fe5e41083d0 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.019+0000 7fe5ea095640 1 -- 172.21.15.3:0/902372315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e41a0630 con 0x7fe5e41083d0 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.019+0000 7fe5e0ff9640 1 -- 172.21.15.3:0/902372315 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc035070 con 0x7fe5e41083d0 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.019+0000 7fe5ea095640 1 -- 172.21.15.3:0/902372315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e41a0b10 con 0x7fe5e41083d0 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.020+0000 7fe5ea095640 1 -- 172.21.15.3:0/902372315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e4102ef0 con 0x7fe5e41083d0 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.021+0000 7fe5e0ff9640 1 -- 172.21.15.3:0/902372315 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7fe5cc042030 con 0x7fe5e41083d0 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.022+0000 7fe5e0ff9640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe5d00b6590 0x7fe5d00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.022+0000 7fe5e37fe640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe5d00b6590 0x7fe5d00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.022+0000 7fe5e0ff9640 1 -- 172.21.15.3:0/902372315 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fe5cc107a40 con 0x7fe5e41083d0 2024-09-21T06:12:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.023+0000 7fe5e37fe640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe5d00b6590 0x7fe5d00b8aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400bbe0 tx=0x7fe5d400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.025+0000 7fe5e0ff9640 1 -- 172.21.15.3:0/902372315 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe5cc0d29e0 con 0x7fe5e41083d0 2024-09-21T06:12:59.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.254+0000 7fe5ea095640 1 -- 172.21.15.3:0/902372315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5e419b620 con 0x7fe5e41083d0 2024-09-21T06:12:59.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.259+0000 7fe5e0ff9640 1 -- 172.21.15.3:0/902372315 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9683 (secure 0 0 0) 0x7fe5e419b620 con 0x7fe5e41083d0 2024-09-21T06:12:59.261 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:12:59.261 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":12,"btime":"2024-09-21T06:12:58:248207+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24631,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24644,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24674,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24691,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T06:12:59.261 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 12 2024-09-21T06:12:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.261+0000 7fe5ea095640 1 -- 172.21.15.3:0/902372315 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe5d00b6590 msgr2=0x7fe5d00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.261+0000 7fe5ea095640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe5d00b6590 0x7fe5d00b8aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400bbe0 tx=0x7fe5d400b390 comp rx=0 tx=0).stop 2024-09-21T06:12:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.261+0000 7fe5ea095640 1 -- 172.21.15.3:0/902372315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5e41083d0 msgr2=0x7fe5e419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.261+0000 7fe5ea095640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5e41083d0 0x7fe5e419aa90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc02a930 tx=0x7fe5cc004270 comp rx=0 tx=0).stop 2024-09-21T06:12:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.261+0000 7fe5ea095640 1 -- 172.21.15.3:0/902372315 shutdown_connections 2024-09-21T06:12:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.261+0000 7fe5ea095640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe5d00b6590 0x7fe5d00b8aa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.261+0000 7fe5ea095640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e4108d20 0x7fe5e419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:59.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.261+0000 7fe5ea095640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5e41083d0 0x7fe5e419aa90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:59.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.261+0000 7fe5ea095640 1 --2- 172.21.15.3:0/902372315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5e4102ae0 0x7fe5e419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:59.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.261+0000 7fe5ea095640 1 -- 172.21.15.3:0/902372315 >> 172.21.15.3:0/902372315 conn(0x7fe5e40797b0 msgr2=0x7fe5e41044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:59.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.261+0000 7fe5ea095640 1 -- 172.21.15.3:0/902372315 shutdown_connections 2024-09-21T06:12:59.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:12:59.261+0000 7fe5ea095640 1 -- 172.21.15.3:0/902372315 wait complete. 2024-09-21T06:12:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:59 smithi003 ceph-mon[31227]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:59 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.3:6840/2403205583,v1:172.21.15.3:6841/2403205583] up:boot 2024-09-21T06:12:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:59 smithi003 ceph-mon[31227]: fsmap 11 up:standby 2024-09-21T06:12:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:59 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:12:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:59 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:59 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:59 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:59 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:59 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:59 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:59.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:12:59 smithi003 ceph-mon[31227]: Deploying daemon mds.b on smithi137 2024-09-21T06:12:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:59 smithi137 ceph-mon[34464]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:59 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.3:6840/2403205583,v1:172.21.15.3:6841/2403205583] up:boot 2024-09-21T06:12:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:59 smithi137 ceph-mon[34464]: fsmap 11 up:standby 2024-09-21T06:12:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:59 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:12:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:59 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:59 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:59 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:59 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:59 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:59 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:59.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:12:59 smithi137 ceph-mon[34464]: Deploying daemon mds.b on smithi137 2024-09-21T06:12:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:59 smithi204 ceph-mon[34659]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:12:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:59 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.3:6840/2403205583,v1:172.21.15.3:6841/2403205583] up:boot 2024-09-21T06:12:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:59 smithi204 ceph-mon[34659]: fsmap 11 up:standby 2024-09-21T06:12:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:59 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:12:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:59 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:59.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:59 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:59.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:59 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:12:59.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:59 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:12:59.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:59 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:12:59.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:59 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:59.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:12:59 smithi204 ceph-mon[34659]: Deploying daemon mds.b on smithi137 2024-09-21T06:13:00.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:00 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/902372315' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:00.593 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:00 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/902372315' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:00.593 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:13:00 smithi137 systemd[1]: Starting Ceph mds.b for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:13:00.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:00 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/902372315' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:00.864 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:13:00 smithi137 podman[63042]: 2024-09-21 06:13:00.494495839 +0000 UTC m=+0.015959388 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:13:00.864 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:13:00 smithi137 podman[63042]: 2024-09-21 06:13:00.626321562 +0000 UTC m=+0.147785106 container create 35f23dd78deddf49f51a526bf343fd2d1dd644a9ab4d1d711cd959c8ee06e847 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:13:00.864 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:13:00 smithi137 podman[63042]: 2024-09-21 06:13:00.86292525 +0000 UTC m=+0.384388805 container init 35f23dd78deddf49f51a526bf343fd2d1dd644a9ab4d1d711cd959c8ee06e847 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T06:13:01.145 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:13:00 smithi137 podman[63042]: 2024-09-21 06:13:00.865829702 +0000 UTC m=+0.387293257 container start 35f23dd78deddf49f51a526bf343fd2d1dd644a9ab4d1d711cd959c8ee06e847 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:13:01.145 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:13:00 smithi137 ceph-mds[63061]: -- 172.21.15.137:0/3118579890 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c8de5f9860 con 0x55c8e27a1600 2024-09-21T06:13:01.145 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:13:00 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b[63057]: starting mds.b at 2024-09-21T06:13:01.145 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:13:00 smithi137 bash[63042]: 35f23dd78deddf49f51a526bf343fd2d1dd644a9ab4d1d711cd959c8ee06e847 2024-09-21T06:13:01.145 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:13:00 smithi137 systemd[1]: Started Ceph mds.b for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:13:01.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:01 smithi003 ceph-mon[31227]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:01.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:01 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:13:01.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:01 smithi137 ceph-mon[34464]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:01 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:13:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:01 smithi204 ceph-mon[34659]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:01 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:13:01.949 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:13:02.217 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:02.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:02 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] up:boot 2024-09-21T06:13:02.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:02 smithi003 ceph-mon[31227]: fsmap 12 up:standby 2024-09-21T06:13:02.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:02 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:13:02.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:02 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] up:boot 2024-09-21T06:13:02.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:02 smithi137 ceph-mon[34464]: fsmap 12 up:standby 2024-09-21T06:13:02.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:02 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:13:02.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:02 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] up:boot 2024-09-21T06:13:02.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:02 smithi204 ceph-mon[34659]: fsmap 12 up:standby 2024-09-21T06:13:02.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:02 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:13:02.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.719+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/2849683914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf8102a60 msgr2=0x7f6bf8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:02.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.719+0000 7f6bfe7cd640 1 --2- 172.21.15.3:0/2849683914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf8102a60 0x7f6bf8102e70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009900 tx=0x7f6be802a410 comp rx=0 tx=0).stop 2024-09-21T06:13:02.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.719+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/2849683914 shutdown_connections 2024-09-21T06:13:02.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.719+0000 7f6bfe7cd640 1 --2- 172.21.15.3:0/2849683914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf8108cf0 0x7f6bf810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:02.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.719+0000 7f6bfe7cd640 1 --2- 172.21.15.3:0/2849683914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bf81083a0 0x7f6bf81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:02.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.719+0000 7f6bfe7cd640 1 --2- 172.21.15.3:0/2849683914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf8102a60 0x7f6bf8102e70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:02.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.719+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/2849683914 >> 172.21.15.3:0/2849683914 conn(0x7f6bf8079730 msgr2=0x7f6bf8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:02.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.719+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/2849683914 shutdown_connections 2024-09-21T06:13:02.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.719+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/2849683914 wait complete. 2024-09-21T06:13:02.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.720+0000 7f6bfe7cd640 1 Processor -- start 2024-09-21T06:13:02.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.720+0000 7f6bfe7cd640 1 -- start start 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.720+0000 7f6bfe7cd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf8102a60 0x7f6bf819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.720+0000 7f6bfe7cd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bf81083a0 0x7f6bf819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfe7cd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf8108cf0 0x7f6bf819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfe7cd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf8112960 con 0x7f6bf8108cf0 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfe7cd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81127e0 con 0x7f6bf81083a0 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfe7cd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf8112ae0 con 0x7f6bf8102a60 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfcd6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf8108cf0 0x7f6bf819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bf7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf8102a60 0x7f6bf819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfcd6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf8108cf0 0x7f6bf819bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57882/0 (socket says 172.21.15.3:57882) 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bf7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf8102a60 0x7f6bf819b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:55822/0 (socket says 172.21.15.3:55822) 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfcd6a640 1 -- 172.21.15.3:0/68730735 learned_addr learned my addr 172.21.15.3:0/68730735 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bf77fe640 1 --2- 172.21.15.3:0/68730735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bf81083a0 0x7f6bf819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfcd6a640 1 -- 172.21.15.3:0/68730735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf8102a60 msgr2=0x7f6bf819b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:02.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfcd6a640 1 --2- 172.21.15.3:0/68730735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf8102a60 0x7f6bf819b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfcd6a640 1 -- 172.21.15.3:0/68730735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bf81083a0 msgr2=0x7f6bf819b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfcd6a640 1 --2- 172.21.15.3:0/68730735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bf81083a0 0x7f6bf819b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfcd6a640 1 -- 172.21.15.3:0/68730735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf81a64c0 con 0x7f6bf8108cf0 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfcd6a640 1 --2- 172.21.15.3:0/68730735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf8108cf0 0x7f6bf819bcb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000eb10 tx=0x7f6bf000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bf57fa640 1 -- 172.21.15.3:0/68730735 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0004b20 con 0x7f6bf8108cf0 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bf57fa640 1 -- 172.21.15.3:0/68730735 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bf0004cc0 con 0x7f6bf8108cf0 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/68730735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf81a67b0 con 0x7f6bf8108cf0 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.721+0000 7f6bf57fa640 1 -- 172.21.15.3:0/68730735 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0010670 con 0x7f6bf8108cf0 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.722+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/68730735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf81a6d40 con 0x7f6bf8108cf0 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.723+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/68730735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf8102e70 con 0x7f6bf8108cf0 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.724+0000 7f6bf57fa640 1 -- 172.21.15.3:0/68730735 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f6bf0004e70 con 0x7f6bf8108cf0 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.725+0000 7f6bf57fa640 1 --2- 172.21.15.3:0/68730735 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6be00b6610 0x7f6be00b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.725+0000 7f6bf7fff640 1 --2- 172.21.15.3:0/68730735 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6be00b6610 0x7f6be00b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:02.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.725+0000 7f6bf57fa640 1 -- 172.21.15.3:0/68730735 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f6bf00e7650 con 0x7f6bf8108cf0 2024-09-21T06:13:02.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.726+0000 7f6bf7fff640 1 --2- 172.21.15.3:0/68730735 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6be00b6610 0x7f6be00b8b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6be8000c00 tx=0x7f6be8031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:02.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:02.729+0000 7f6bf57fa640 1 -- 172.21.15.3:0/68730735 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6bf0056710 con 0x7f6bf8108cf0 2024-09-21T06:13:03.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.016+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/68730735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bf806b270 con 0x7f6bf8108cf0 2024-09-21T06:13:03.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.017+0000 7f6bf57fa640 1 -- 172.21.15.3:0/68730735 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10512 (secure 0 0 0) 0x7f6bf00ae250 con 0x7f6bf8108cf0 2024-09-21T06:13:03.019 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:13:03.019 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":13,"btime":"2024-09-21T06:13:01:261760+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24631,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24644,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24674,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24691,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24704,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T06:13:03.020 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 13 2024-09-21T06:13:03.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.019+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/68730735 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6be00b6610 msgr2=0x7f6be00b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:03.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.019+0000 7f6bfe7cd640 1 --2- 172.21.15.3:0/68730735 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6be00b6610 0x7f6be00b8b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6be8000c00 tx=0x7f6be8031400 comp rx=0 tx=0).stop 2024-09-21T06:13:03.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.019+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/68730735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf8108cf0 msgr2=0x7f6bf819bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:03.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.019+0000 7f6bfe7cd640 1 --2- 172.21.15.3:0/68730735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf8108cf0 0x7f6bf819bcb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000eb10 tx=0x7f6bf000c2d0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.020+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/68730735 shutdown_connections 2024-09-21T06:13:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.020+0000 7f6bfe7cd640 1 --2- 172.21.15.3:0/68730735 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6be00b6610 0x7f6be00b8b20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.020+0000 7f6bfe7cd640 1 --2- 172.21.15.3:0/68730735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf8108cf0 0x7f6bf819bcb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.020+0000 7f6bfe7cd640 1 --2- 172.21.15.3:0/68730735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bf81083a0 0x7f6bf819b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.020+0000 7f6bfe7cd640 1 --2- 172.21.15.3:0/68730735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf8102a60 0x7f6bf819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.020+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/68730735 >> 172.21.15.3:0/68730735 conn(0x7f6bf8079730 msgr2=0x7f6bf8101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.020+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/68730735 shutdown_connections 2024-09-21T06:13:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:03.020+0000 7f6bfe7cd640 1 -- 172.21.15.3:0/68730735 wait complete. 2024-09-21T06:13:03.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:03 smithi003 ceph-mon[31227]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:03.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:03 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:03.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:03 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:03.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:03 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:03.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:03 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:03.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:03 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/68730735' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:03.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:03 smithi137 ceph-mon[34464]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:03.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:03 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:03.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:03 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:03.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:03 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:03.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:03 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:03.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:03 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/68730735' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:03 smithi204 ceph-mon[34659]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:03 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:03 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:03 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:03 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:03 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/68730735' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:03.722 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T06:13:04.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:04.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:04.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:13:04.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:13:04.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:04.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:13:04.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:04 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:04.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:04.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:04.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:13:04.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:13:04.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:04.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:13:04.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:04 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:04.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:04.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:04.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:13:04.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:13:04.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:04.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:13:04.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:04 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:05.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.167+0000 7f3163504640 1 -- 172.21.15.3:0/149635539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f315c102600 msgr2=0x7f315c102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.167+0000 7f3163504640 1 --2- 172.21.15.3:0/149635539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f315c102600 0x7f315c102a10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3158009900 tx=0x7f315802a480 comp rx=0 tx=0).stop 2024-09-21T06:13:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.167+0000 7f3163504640 1 -- 172.21.15.3:0/149635539 shutdown_connections 2024-09-21T06:13:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.167+0000 7f3163504640 1 --2- 172.21.15.3:0/149635539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f315c102f50 0x7f315c10fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.167+0000 7f3163504640 1 --2- 172.21.15.3:0/149635539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f315c1101b0 0x7f315c1125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.167+0000 7f3163504640 1 --2- 172.21.15.3:0/149635539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f315c102600 0x7f315c102a10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.167+0000 7f3163504640 1 -- 172.21.15.3:0/149635539 >> 172.21.15.3:0/149635539 conn(0x7f315c079730 msgr2=0x7f315c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.167+0000 7f3163504640 1 -- 172.21.15.3:0/149635539 shutdown_connections 2024-09-21T06:13:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.167+0000 7f3163504640 1 -- 172.21.15.3:0/149635539 wait complete. 2024-09-21T06:13:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3163504640 1 Processor -- start 2024-09-21T06:13:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3163504640 1 -- start start 2024-09-21T06:13:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3163504640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f315c102600 0x7f315c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3163504640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f315c102f50 0x7f315c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3163504640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f315c1101b0 0x7f315c1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3163504640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f315c115ca0 con 0x7f315c1101b0 2024-09-21T06:13:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3163504640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f315c115b20 con 0x7f315c102f50 2024-09-21T06:13:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3163504640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f315c115e20 con 0x7f315c102600 2024-09-21T06:13:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3161aa1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f315c1101b0 0x7f315c1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3161aa1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f315c1101b0 0x7f315c1a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57896/0 (socket says 172.21.15.3:57896) 2024-09-21T06:13:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3161aa1640 1 -- 172.21.15.3:0/2414395523 learned_addr learned my addr 172.21.15.3:0/2414395523 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f31612a0640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f315c102600 0x7f315c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3160a9f640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f315c102f50 0x7f315c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3161aa1640 1 -- 172.21.15.3:0/2414395523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f315c102600 msgr2=0x7f315c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3161aa1640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f315c102600 0x7f315c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3161aa1640 1 -- 172.21.15.3:0/2414395523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f315c102f50 msgr2=0x7f315c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3161aa1640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f315c102f50 0x7f315c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.168+0000 7f3161aa1640 1 -- 172.21.15.3:0/2414395523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315c1a2640 con 0x7f315c1101b0 2024-09-21T06:13:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.169+0000 7f3161aa1640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f315c1101b0 0x7f315c1a1f10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f315400eaa0 tx=0x7f315400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.169+0000 7f31527fc640 1 -- 172.21.15.3:0/2414395523 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315400cdb0 con 0x7f315c1101b0 2024-09-21T06:13:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.169+0000 7f31527fc640 1 -- 172.21.15.3:0/2414395523 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3154005020 con 0x7f315c1101b0 2024-09-21T06:13:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.169+0000 7f3163504640 1 -- 172.21.15.3:0/2414395523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315c1a2930 con 0x7f315c1101b0 2024-09-21T06:13:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.169+0000 7f31527fc640 1 -- 172.21.15.3:0/2414395523 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31540116d0 con 0x7f315c1101b0 2024-09-21T06:13:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.169+0000 7f3163504640 1 -- 172.21.15.3:0/2414395523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f315c1a9090 con 0x7f315c1101b0 2024-09-21T06:13:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.170+0000 7f3163504640 1 -- 172.21.15.3:0/2414395523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f315c06c280 con 0x7f315c1101b0 2024-09-21T06:13:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.171+0000 7f31527fc640 1 -- 172.21.15.3:0/2414395523 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f31540051d0 con 0x7f315c1101b0 2024-09-21T06:13:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.172+0000 7f31527fc640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f31480b65e0 0x7f31480b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.173+0000 7f31612a0640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f31480b65e0 0x7f31480b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:05.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.173+0000 7f31527fc640 1 -- 172.21.15.3:0/2414395523 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f31540e73e0 con 0x7f315c1101b0 2024-09-21T06:13:05.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.173+0000 7f31612a0640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f31480b65e0 0x7f31480b8af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3158031440 tx=0x7f3158009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:05.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.176+0000 7f31527fc640 1 -- 172.21.15.3:0/2414395523 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3154016030 con 0x7f315c1101b0 2024-09-21T06:13:05.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.391+0000 7f3163504640 1 -- 172.21.15.3:0/2414395523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f315c102a10 con 0x7f315c1101b0 2024-09-21T06:13:05.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.391+0000 7f31527fc640 1 -- 172.21.15.3:0/2414395523 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f315c102a10 con 0x7f315c1101b0 2024-09-21T06:13:05.393 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:13:05.394 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-21T06:13:05.394 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:13:05.394 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-21T06:13:05.394 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-21T06:13:05.394 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:13:05.394 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-21T06:13:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.394+0000 7f3163504640 1 -- 172.21.15.3:0/2414395523 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f31480b65e0 msgr2=0x7f31480b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.394+0000 7f3163504640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f31480b65e0 0x7f31480b8af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3158031440 tx=0x7f3158009560 comp rx=0 tx=0).stop 2024-09-21T06:13:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.394+0000 7f3163504640 1 -- 172.21.15.3:0/2414395523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f315c1101b0 msgr2=0x7f315c1a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.394+0000 7f3163504640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f315c1101b0 0x7f315c1a1f10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f315400eaa0 tx=0x7f315400ef60 comp rx=0 tx=0).stop 2024-09-21T06:13:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.394+0000 7f3163504640 1 -- 172.21.15.3:0/2414395523 shutdown_connections 2024-09-21T06:13:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.394+0000 7f3163504640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f31480b65e0 0x7f31480b8af0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.394+0000 7f3163504640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f315c1101b0 0x7f315c1a1f10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.394+0000 7f3163504640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f315c102f50 0x7f315c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.394+0000 7f3163504640 1 --2- 172.21.15.3:0/2414395523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f315c102600 0x7f315c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.394+0000 7f3163504640 1 -- 172.21.15.3:0/2414395523 >> 172.21.15.3:0/2414395523 conn(0x7f315c079730 msgr2=0x7f315c108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.394+0000 7f3163504640 1 -- 172.21.15.3:0/2414395523 shutdown_connections 2024-09-21T06:13:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:05.394+0000 7f3163504640 1 -- 172.21.15.3:0/2414395523 wait complete. 2024-09-21T06:13:05.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:05 smithi003 ceph-mon[31227]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:05.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:05 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:05.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:05 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:05.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:05 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:05.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:05 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:05.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:05 smithi137 ceph-mon[34464]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:05.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:05 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:05.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:05 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:05.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:05 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:05.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:05 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:05.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:05 smithi204 ceph-mon[34659]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:05.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:05 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:05.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:05 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:05.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:05 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:05.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:05 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:06.115 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:13:06.115 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:13:06.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:06 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2414395523' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T06:13:06.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:06 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:06.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:06 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:06.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:06 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:13:06.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:06 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:13:06.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:06 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:06.378 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:06.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:06 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2414395523' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T06:13:06.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:06 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:06.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:06 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:06.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:06 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:13:06.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:06 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:13:06.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:06 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:06.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:06 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2414395523' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T06:13:06.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:06 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:06.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:06 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:06.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:06 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:13:06.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:06 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:13:06.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:06 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:13:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.880+0000 7f785add0640 1 -- 172.21.15.3:0/3543489603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7854102ae0 msgr2=0x7f7854102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.880+0000 7f785add0640 1 --2- 172.21.15.3:0/3543489603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7854102ae0 0x7f7854102ef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7848009900 tx=0x7f784802a410 comp rx=0 tx=0).stop 2024-09-21T06:13:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.881+0000 7f785add0640 1 -- 172.21.15.3:0/3543489603 shutdown_connections 2024-09-21T06:13:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.881+0000 7f785add0640 1 --2- 172.21.15.3:0/3543489603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7854108420 0x7f7854108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.881+0000 7f785add0640 1 --2- 172.21.15.3:0/3543489603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7854108d70 0x7f785410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.881+0000 7f785add0640 1 --2- 172.21.15.3:0/3543489603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7854102ae0 0x7f7854102ef0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.881+0000 7f785add0640 1 -- 172.21.15.3:0/3543489603 >> 172.21.15.3:0/3543489603 conn(0x7f7854079770 msgr2=0x7f7854079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.881+0000 7f785add0640 1 -- 172.21.15.3:0/3543489603 shutdown_connections 2024-09-21T06:13:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.881+0000 7f785add0640 1 -- 172.21.15.3:0/3543489603 wait complete. 2024-09-21T06:13:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.881+0000 7f785add0640 1 Processor -- start 2024-09-21T06:13:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785add0640 1 -- start start 2024-09-21T06:13:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785add0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7854102ae0 0x7f785419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785add0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7854108420 0x7f785419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785add0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7854108d70 0x7f785419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785add0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7854112a30 con 0x7f7854108d70 2024-09-21T06:13:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785add0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f78541128b0 con 0x7f7854108420 2024-09-21T06:13:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785add0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7854112bb0 con 0x7f7854102ae0 2024-09-21T06:13:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785936d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7854108d70 0x7f785419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785936d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7854108d70 0x7f785419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41232/0 (socket says 172.21.15.3:41232) 2024-09-21T06:13:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785936d640 1 -- 172.21.15.3:0/1890490252 learned_addr learned my addr 172.21.15.3:0/1890490252 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f7858b6c640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7854102ae0 0x7f785419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f7853fff640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7854108420 0x7f785419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785936d640 1 -- 172.21.15.3:0/1890490252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7854102ae0 msgr2=0x7f785419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785936d640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7854102ae0 0x7f785419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785936d640 1 -- 172.21.15.3:0/1890490252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7854108420 msgr2=0x7f785419b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785936d640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7854108420 0x7f785419b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f785936d640 1 -- 172.21.15.3:0/1890490252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78541a6680 con 0x7f7854108d70 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.882+0000 7f7853fff640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7854108420 0x7f785419b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.883+0000 7f785936d640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7854108d70 0x7f785419be40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f784c00eaa0 tx=0x7f784c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.883+0000 7f7851ffb640 1 -- 172.21.15.3:0/1890490252 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c00cdb0 con 0x7f7854108d70 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.883+0000 7f7851ffb640 1 -- 172.21.15.3:0/1890490252 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f784c005020 con 0x7f7854108d70 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.883+0000 7f785add0640 1 -- 172.21.15.3:0/1890490252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78541a6970 con 0x7f7854108d70 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.883+0000 7f7851ffb640 1 -- 172.21.15.3:0/1890490252 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c0116d0 con 0x7f7854108d70 2024-09-21T06:13:06.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.883+0000 7f785add0640 1 -- 172.21.15.3:0/1890490252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78541a6eb0 con 0x7f7854108d70 2024-09-21T06:13:06.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.884+0000 7f785add0640 1 -- 172.21.15.3:0/1890490252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7854103d60 con 0x7f7854108d70 2024-09-21T06:13:06.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.886+0000 7f7851ffb640 1 -- 172.21.15.3:0/1890490252 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f784c0051d0 con 0x7f7854108d70 2024-09-21T06:13:06.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.887+0000 7f7851ffb640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f78440b65e0 0x7f78440b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:06.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.887+0000 7f7858b6c640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f78440b65e0 0x7f78440b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:06.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.887+0000 7f7851ffb640 1 -- 172.21.15.3:0/1890490252 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f784c0e70f0 con 0x7f7854108d70 2024-09-21T06:13:06.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.887+0000 7f7858b6c640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f78440b65e0 0x7f78440b8af0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7848000c00 tx=0x7f7848031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:06.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:06.891+0000 7f7851ffb640 1 -- 172.21.15.3:0/1890490252 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f784c016030 con 0x7f7854108d70 2024-09-21T06:13:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.079+0000 7f785add0640 1 -- 172.21.15.3:0/1890490252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f785402e640 con 0x7f7854108d70 2024-09-21T06:13:07.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.465+0000 7f7851ffb640 1 -- 172.21.15.3:0/1890490252 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f784c0add50 con 0x7f7854108d70 2024-09-21T06:13:07.467 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-21T06:13:07.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.468+0000 7f785add0640 1 -- 172.21.15.3:0/1890490252 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f78440b65e0 msgr2=0x7f78440b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.468+0000 7f785add0640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f78440b65e0 0x7f78440b8af0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7848000c00 tx=0x7f7848031400 comp rx=0 tx=0).stop 2024-09-21T06:13:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.468+0000 7f785add0640 1 -- 172.21.15.3:0/1890490252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7854108d70 msgr2=0x7f785419be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.468+0000 7f785add0640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7854108d70 0x7f785419be40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f784c00eaa0 tx=0x7f784c00ef60 comp rx=0 tx=0).stop 2024-09-21T06:13:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.468+0000 7f785add0640 1 -- 172.21.15.3:0/1890490252 shutdown_connections 2024-09-21T06:13:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.469+0000 7f785add0640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f78440b65e0 0x7f78440b8af0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.469+0000 7f785add0640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7854108d70 0x7f785419be40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.469+0000 7f785add0640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7854108420 0x7f785419b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.469+0000 7f785add0640 1 --2- 172.21.15.3:0/1890490252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7854102ae0 0x7f785419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.469+0000 7f785add0640 1 -- 172.21.15.3:0/1890490252 >> 172.21.15.3:0/1890490252 conn(0x7f7854079770 msgr2=0x7f7854106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.469+0000 7f785add0640 1 -- 172.21.15.3:0/1890490252 shutdown_connections 2024-09-21T06:13:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:07.469+0000 7f785add0640 1 -- 172.21.15.3:0/1890490252 wait complete. 2024-09-21T06:13:07.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:07 smithi003 ceph-mon[31227]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:07.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:07 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1890490252' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:13:07.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:07 smithi137 ceph-mon[34464]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:07.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:07 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1890490252' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:13:07.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:07 smithi204 ceph-mon[34659]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:07.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:07 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1890490252' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:13:08.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:13:08.436 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:08.743 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:08 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1890490252' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T06:13:08.743 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:08 smithi003 ceph-mon[31227]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T06:13:08.743 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:08 smithi003 ceph-mon[31227]: pgmap v263: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:08.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:08 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1890490252' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T06:13:08.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:08 smithi137 ceph-mon[34464]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T06:13:08.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:08 smithi137 ceph-mon[34464]: pgmap v263: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:08.816 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:08 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1890490252' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T06:13:08.816 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:08 smithi204 ceph-mon[34659]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T06:13:08.816 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:08 smithi204 ceph-mon[34659]: pgmap v263: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:08.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.942+0000 7fd271945640 1 -- 172.21.15.3:0/676662986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd26c1048c0 msgr2=0x7fd26c104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:08.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.942+0000 7fd271945640 1 --2- 172.21.15.3:0/676662986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd26c1048c0 0x7fd26c104cd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009900 tx=0x7fd25c02a410 comp rx=0 tx=0).stop 2024-09-21T06:13:08.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.942+0000 7fd271945640 1 -- 172.21.15.3:0/676662986 shutdown_connections 2024-09-21T06:13:08.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.942+0000 7fd271945640 1 --2- 172.21.15.3:0/676662986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd26c10de70 0x7fd26c110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.942+0000 7fd271945640 1 --2- 172.21.15.3:0/676662986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd26c1052f0 0x7fd26c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.942+0000 7fd271945640 1 --2- 172.21.15.3:0/676662986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd26c1048c0 0x7fd26c104cd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.942+0000 7fd271945640 1 -- 172.21.15.3:0/676662986 >> 172.21.15.3:0/676662986 conn(0x7fd26c079770 msgr2=0x7fd26c079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:08.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.942+0000 7fd271945640 1 -- 172.21.15.3:0/676662986 shutdown_connections 2024-09-21T06:13:08.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.943+0000 7fd271945640 1 -- 172.21.15.3:0/676662986 wait complete. 2024-09-21T06:13:08.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.943+0000 7fd271945640 1 Processor -- start 2024-09-21T06:13:08.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.943+0000 7fd271945640 1 -- start start 2024-09-21T06:13:08.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd271945640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd26c1048c0 0x7fd26c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:08.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd271945640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd26c1052f0 0x7fd26c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:08.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd271945640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd26c10de70 0x7fd26c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:08.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd271945640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c113b10 con 0x7fd26c1048c0 2024-09-21T06:13:08.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd271945640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c113990 con 0x7fd26c10de70 2024-09-21T06:13:08.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd271945640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c113c90 con 0x7fd26c1052f0 2024-09-21T06:13:08.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd26affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd26c1048c0 0x7fd26c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:08.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd26affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd26c1048c0 0x7fd26c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41256/0 (socket says 172.21.15.3:41256) 2024-09-21T06:13:08.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd26affd640 1 -- 172.21.15.3:0/3956352379 learned_addr learned my addr 172.21.15.3:0/3956352379 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:08.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd26a7fc640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd26c1052f0 0x7fd26c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:08.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd26b7fe640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd26c10de70 0x7fd26c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:08.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd26affd640 1 -- 172.21.15.3:0/3956352379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd26c1052f0 msgr2=0x7fd26c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:08.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd26affd640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd26c1052f0 0x7fd26c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd26affd640 1 -- 172.21.15.3:0/3956352379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd26c10de70 msgr2=0x7fd26c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:08.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd26affd640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd26c10de70 0x7fd26c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:08.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd26affd640 1 -- 172.21.15.3:0/3956352379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26c1a03b0 con 0x7fd26c1048c0 2024-09-21T06:13:08.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.944+0000 7fd26affd640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd26c1048c0 0x7fd26c19a500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd25c005940 tx=0x7fd25c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:08.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.945+0000 7fd270943640 1 -- 172.21.15.3:0/3956352379 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c03f030 con 0x7fd26c1048c0 2024-09-21T06:13:08.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.945+0000 7fd270943640 1 -- 172.21.15.3:0/3956352379 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd25c003aa0 con 0x7fd26c1048c0 2024-09-21T06:13:08.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.945+0000 7fd271945640 1 -- 172.21.15.3:0/3956352379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26c1a0640 con 0x7fd26c1048c0 2024-09-21T06:13:08.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.945+0000 7fd270943640 1 -- 172.21.15.3:0/3956352379 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c035070 con 0x7fd26c1048c0 2024-09-21T06:13:08.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.945+0000 7fd271945640 1 -- 172.21.15.3:0/3956352379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26c1a0b20 con 0x7fd26c1048c0 2024-09-21T06:13:08.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.946+0000 7fd271945640 1 -- 172.21.15.3:0/3956352379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26c06c200 con 0x7fd26c1048c0 2024-09-21T06:13:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.948+0000 7fd270943640 1 -- 172.21.15.3:0/3956352379 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7fd25c042030 con 0x7fd26c1048c0 2024-09-21T06:13:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.948+0000 7fd270943640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd2540b6590 0x7fd2540b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.949+0000 7fd26a7fc640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd2540b6590 0x7fd2540b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.949+0000 7fd270943640 1 -- 172.21.15.3:0/3956352379 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7712+0+0 (secure 0 0 0) 0x7fd25c1079e0 con 0x7fd26c1048c0 2024-09-21T06:13:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.949+0000 7fd26a7fc640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd2540b6590 0x7fd2540b8aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd26c100100 tx=0x7fd26000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:08.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:08.952+0000 7fd270943640 1 -- 172.21.15.3:0/3956352379 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd25c0ce4d0 con 0x7fd26c1048c0 2024-09-21T06:13:09.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.145+0000 7fd271945640 1 -- 172.21.15.3:0/3956352379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd26c1a6f80 con 0x7fd26c1048c0 2024-09-21T06:13:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.478+0000 7fd270943640 1 -- 172.21.15.3:0/3956352379 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7fd25c0d28b0 con 0x7fd26c1048c0 2024-09-21T06:13:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-21T06:13:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.481+0000 7fd271945640 1 -- 172.21.15.3:0/3956352379 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd2540b6590 msgr2=0x7fd2540b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.481+0000 7fd271945640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd2540b6590 0x7fd2540b8aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd26c100100 tx=0x7fd26000a2d0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.481+0000 7fd271945640 1 -- 172.21.15.3:0/3956352379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd26c1048c0 msgr2=0x7fd26c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.481+0000 7fd271945640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd26c1048c0 0x7fd26c19a500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd25c005940 tx=0x7fd25c004270 comp rx=0 tx=0).stop 2024-09-21T06:13:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.482+0000 7fd271945640 1 -- 172.21.15.3:0/3956352379 shutdown_connections 2024-09-21T06:13:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.482+0000 7fd271945640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd2540b6590 0x7fd2540b8aa0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.482+0000 7fd271945640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd26c1052f0 0x7fd26c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.482+0000 7fd271945640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd26c10de70 0x7fd26c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.482+0000 7fd271945640 1 --2- 172.21.15.3:0/3956352379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd26c1048c0 0x7fd26c19a500 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.482+0000 7fd271945640 1 -- 172.21.15.3:0/3956352379 >> 172.21.15.3:0/3956352379 conn(0x7fd26c079770 msgr2=0x7fd26c1079f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.482+0000 7fd271945640 1 -- 172.21.15.3:0/3956352379 shutdown_connections 2024-09-21T06:13:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:09.482+0000 7fd271945640 1 -- 172.21.15.3:0/3956352379 wait complete. 2024-09-21T06:13:09.778 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:09 smithi204 ceph-mon[34659]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T06:13:09.779 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:09 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3956352379' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:13:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:09 smithi003 ceph-mon[31227]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T06:13:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:09 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3956352379' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:13:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:09 smithi137 ceph-mon[34464]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T06:13:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:09 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3956352379' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:13:10.211 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:13:10.477 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:10.490 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:10 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3956352379' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T06:13:10.490 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:10 smithi003 ceph-mon[31227]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T06:13:10.743 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:10 smithi003 ceph-mon[31227]: pgmap v266: 97 pgs: 7 active+clean, 13 creating+peering, 77 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:10.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:10 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3956352379' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T06:13:10.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:10 smithi137 ceph-mon[34464]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T06:13:10.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:10 smithi137 ceph-mon[34464]: pgmap v266: 97 pgs: 7 active+clean, 13 creating+peering, 77 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:10.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:10 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3956352379' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T06:13:10.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:10 smithi204 ceph-mon[34659]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T06:13:10.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:10 smithi204 ceph-mon[34659]: pgmap v266: 97 pgs: 7 active+clean, 13 creating+peering, 77 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:10.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.965+0000 7fc27d941640 1 -- 172.21.15.3:0/75337822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27810ddf0 msgr2=0x7fc278110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:10.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.965+0000 7fc27d941640 1 --2- 172.21.15.3:0/75337822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27810ddf0 0x7fc278110300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc270009900 tx=0x7fc27002a410 comp rx=0 tx=0).stop 2024-09-21T06:13:10.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.965+0000 7fc27d941640 1 -- 172.21.15.3:0/75337822 shutdown_connections 2024-09-21T06:13:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.965+0000 7fc27d941640 1 --2- 172.21.15.3:0/75337822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc278105270 0x7fc27810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.965+0000 7fc27d941640 1 --2- 172.21.15.3:0/75337822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27810ddf0 0x7fc278110300 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.965+0000 7fc27d941640 1 --2- 172.21.15.3:0/75337822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc278104840 0x7fc278104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.965+0000 7fc27d941640 1 -- 172.21.15.3:0/75337822 >> 172.21.15.3:0/75337822 conn(0x7fc278079730 msgr2=0x7fc278079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.965+0000 7fc27d941640 1 -- 172.21.15.3:0/75337822 shutdown_connections 2024-09-21T06:13:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.965+0000 7fc27d941640 1 -- 172.21.15.3:0/75337822 wait complete. 2024-09-21T06:13:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.966+0000 7fc27d941640 1 Processor -- start 2024-09-21T06:13:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.966+0000 7fc27d941640 1 -- start start 2024-09-21T06:13:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.966+0000 7fc27d941640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278104840 0x7fc27819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.966+0000 7fc27d941640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc278105270 0x7fc27819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.966+0000 7fc27d941640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc27810ddf0 0x7fc27819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.966+0000 7fc27d941640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc278113a90 con 0x7fc278104840 2024-09-21T06:13:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.966+0000 7fc27d941640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc278113910 con 0x7fc27810ddf0 2024-09-21T06:13:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.966+0000 7fc27d941640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc278113c10 con 0x7fc278105270 2024-09-21T06:13:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.966+0000 7fc276ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278104840 0x7fc27819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.966+0000 7fc276ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278104840 0x7fc27819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41278/0 (socket says 172.21.15.3:41278) 2024-09-21T06:13:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.966+0000 7fc276ffd640 1 -- 172.21.15.3:0/3243079857 learned_addr learned my addr 172.21.15.3:0/3243079857 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.966+0000 7fc2777fe640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc27810ddf0 0x7fc27819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.966+0000 7fc2767fc640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc278105270 0x7fc27819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.967+0000 7fc276ffd640 1 -- 172.21.15.3:0/3243079857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc278105270 msgr2=0x7fc27819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.967+0000 7fc276ffd640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc278105270 0x7fc27819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.967+0000 7fc276ffd640 1 -- 172.21.15.3:0/3243079857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc27810ddf0 msgr2=0x7fc27819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.967+0000 7fc276ffd640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc27810ddf0 0x7fc27819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.967+0000 7fc276ffd640 1 -- 172.21.15.3:0/3243079857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2781a0430 con 0x7fc278104840 2024-09-21T06:13:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.967+0000 7fc276ffd640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278104840 0x7fc27819a4c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc26800ccb0 tx=0x7fc26800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.967+0000 7fc27c93f640 1 -- 172.21.15.3:0/3243079857 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268003990 con 0x7fc278104840 2024-09-21T06:13:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.967+0000 7fc27c93f640 1 -- 172.21.15.3:0/3243079857 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc268004020 con 0x7fc278104840 2024-09-21T06:13:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.967+0000 7fc27d941640 1 -- 172.21.15.3:0/3243079857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2781a0720 con 0x7fc278104840 2024-09-21T06:13:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.967+0000 7fc27c93f640 1 -- 172.21.15.3:0/3243079857 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26800f6b0 con 0x7fc278104840 2024-09-21T06:13:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.967+0000 7fc27d941640 1 -- 172.21.15.3:0/3243079857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2781a6e80 con 0x7fc278104840 2024-09-21T06:13:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.968+0000 7fc27d941640 1 -- 172.21.15.3:0/3243079857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc278104c50 con 0x7fc278104840 2024-09-21T06:13:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.970+0000 7fc27c93f640 1 -- 172.21.15.3:0/3243079857 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7fc2680044b0 con 0x7fc278104840 2024-09-21T06:13:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.971+0000 7fc27c93f640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc2640b6560 0x7fc2640b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.971+0000 7fc2767fc640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc2640b6560 0x7fc2640b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.971+0000 7fc27c93f640 1 -- 172.21.15.3:0/3243079857 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8094+0+0 (secure 0 0 0) 0x7fc2680e7690 con 0x7fc278104840 2024-09-21T06:13:10.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.971+0000 7fc2767fc640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc2640b6560 0x7fc2640b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc260005fd0 tx=0x7fc260005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:10.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:10.974+0000 7fc27c93f640 1 -- 172.21.15.3:0/3243079857 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc268014030 con 0x7fc278104840 2024-09-21T06:13:11.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.203+0000 7fc27d941640 1 -- 172.21.15.3:0/3243079857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc27819b5f0 con 0x7fc278104840 2024-09-21T06:13:11.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.487+0000 7fc27c93f640 1 -- 172.21.15.3:0/3243079857 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc2680adf60 con 0x7fc278104840 2024-09-21T06:13:11.489 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T06:13:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.490+0000 7fc27d941640 1 -- 172.21.15.3:0/3243079857 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc2640b6560 msgr2=0x7fc2640b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.490+0000 7fc27d941640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc2640b6560 0x7fc2640b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc260005fd0 tx=0x7fc260005850 comp rx=0 tx=0).stop 2024-09-21T06:13:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.490+0000 7fc27d941640 1 -- 172.21.15.3:0/3243079857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278104840 msgr2=0x7fc27819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.490+0000 7fc27d941640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278104840 0x7fc27819a4c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc26800ccb0 tx=0x7fc26800a620 comp rx=0 tx=0).stop 2024-09-21T06:13:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.491+0000 7fc27d941640 1 -- 172.21.15.3:0/3243079857 shutdown_connections 2024-09-21T06:13:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.491+0000 7fc27d941640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc2640b6560 0x7fc2640b8a70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.491+0000 7fc27d941640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc278105270 0x7fc27819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.491+0000 7fc27d941640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc27810ddf0 0x7fc27819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.491+0000 7fc27d941640 1 --2- 172.21.15.3:0/3243079857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278104840 0x7fc27819a4c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.491+0000 7fc27d941640 1 -- 172.21.15.3:0/3243079857 >> 172.21.15.3:0/3243079857 conn(0x7fc278079730 msgr2=0x7fc278107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.491+0000 7fc27d941640 1 -- 172.21.15.3:0/3243079857 shutdown_connections 2024-09-21T06:13:11.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:11.491+0000 7fc27d941640 1 -- 172.21.15.3:0/3243079857 wait complete. 2024-09-21T06:13:11.817 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:11 smithi003 ceph-mon[31227]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T06:13:11.817 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:11 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3243079857' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:13:11.817 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:11 smithi003 ceph-mon[31227]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T06:13:11.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:11 smithi137 ceph-mon[34464]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T06:13:11.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:11 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3243079857' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:13:11.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:11 smithi137 ceph-mon[34464]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T06:13:11.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:11 smithi204 ceph-mon[34659]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T06:13:11.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:11 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3243079857' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:13:11.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:11 smithi204 ceph-mon[34659]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T06:13:12.202 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:13:12.464 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:12.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:12 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3243079857' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T06:13:12.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:12 smithi003 ceph-mon[31227]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:13:12.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:12 smithi003 ceph-mon[31227]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T06:13:12.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:12 smithi003 ceph-mon[31227]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T06:13:12.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:12 smithi003 ceph-mon[31227]: pgmap v269: 97 pgs: 35 active+clean, 31 creating+peering, 31 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:12.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:12 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3243079857' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T06:13:12.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:12 smithi137 ceph-mon[34464]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:13:12.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:12 smithi137 ceph-mon[34464]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T06:13:12.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:12 smithi137 ceph-mon[34464]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T06:13:12.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:12 smithi137 ceph-mon[34464]: pgmap v269: 97 pgs: 35 active+clean, 31 creating+peering, 31 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:12 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3243079857' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T06:13:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:12 smithi204 ceph-mon[34659]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:13:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:12 smithi204 ceph-mon[34659]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T06:13:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:12 smithi204 ceph-mon[34659]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T06:13:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:12 smithi204 ceph-mon[34659]: pgmap v269: 97 pgs: 35 active+clean, 31 creating+peering, 31 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.945+0000 7f3e5f02f640 1 -- 172.21.15.3:0/3913882874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e581094c0 msgr2=0x7f3e581098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.945+0000 7f3e5f02f640 1 --2- 172.21.15.3:0/3913882874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e581094c0 0x7f3e581098d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009900 tx=0x7f3e4c02a410 comp rx=0 tx=0).stop 2024-09-21T06:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.945+0000 7f3e5f02f640 1 -- 172.21.15.3:0/3913882874 shutdown_connections 2024-09-21T06:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.945+0000 7f3e5f02f640 1 --2- 172.21.15.3:0/3913882874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e58109e10 0x7f3e58110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.945+0000 7f3e5f02f640 1 --2- 172.21.15.3:0/3913882874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e581094c0 0x7f3e581098d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.945+0000 7f3e5f02f640 1 --2- 172.21.15.3:0/3913882874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e58103b80 0x7f3e58103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.945+0000 7f3e5f02f640 1 -- 172.21.15.3:0/3913882874 >> 172.21.15.3:0/3913882874 conn(0x7f3e580ff5d0 msgr2=0x7f3e58101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.945+0000 7f3e5f02f640 1 -- 172.21.15.3:0/3913882874 shutdown_connections 2024-09-21T06:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.945+0000 7f3e5f02f640 1 -- 172.21.15.3:0/3913882874 wait complete. 2024-09-21T06:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.946+0000 7f3e5f02f640 1 Processor -- start 2024-09-21T06:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.946+0000 7f3e5f02f640 1 -- start start 2024-09-21T06:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.946+0000 7f3e5f02f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e58103b80 0x7f3e5819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.946+0000 7f3e5f02f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e581094c0 0x7f3e5819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.946+0000 7f3e5f02f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58109e10 0x7f3e5819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.946+0000 7f3e5f02f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58113ad0 con 0x7f3e58109e10 2024-09-21T06:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.946+0000 7f3e5f02f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58113950 con 0x7f3e581094c0 2024-09-21T06:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.946+0000 7f3e5f02f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58113c50 con 0x7f3e58103b80 2024-09-21T06:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.946+0000 7f3e5d5cc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58109e10 0x7f3e5819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.946+0000 7f3e5d5cc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58109e10 0x7f3e5819fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41296/0 (socket says 172.21.15.3:41296) 2024-09-21T06:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.946+0000 7f3e5d5cc640 1 -- 172.21.15.3:0/2886803497 learned_addr learned my addr 172.21.15.3:0/2886803497 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.946+0000 7f3e5cdcb640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e58103b80 0x7f3e5819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.946+0000 7f3e57fff640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e581094c0 0x7f3e5819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.947+0000 7f3e5d5cc640 1 -- 172.21.15.3:0/2886803497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e58103b80 msgr2=0x7f3e5819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.947+0000 7f3e5d5cc640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e58103b80 0x7f3e5819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.947+0000 7f3e5d5cc640 1 -- 172.21.15.3:0/2886803497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e581094c0 msgr2=0x7f3e5819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.947+0000 7f3e5d5cc640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e581094c0 0x7f3e5819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.947+0000 7f3e5d5cc640 1 -- 172.21.15.3:0/2886803497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e581a0490 con 0x7f3e58109e10 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.947+0000 7f3e5cdcb640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e58103b80 0x7f3e5819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.947+0000 7f3e5d5cc640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58109e10 0x7f3e5819fd60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009fd0 tx=0x7f3e5000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.947+0000 7f3e55ffb640 1 -- 172.21.15.3:0/2886803497 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50017070 con 0x7f3e58109e10 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.947+0000 7f3e55ffb640 1 -- 172.21.15.3:0/2886803497 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e50003e20 con 0x7f3e58109e10 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.947+0000 7f3e5f02f640 1 -- 172.21.15.3:0/2886803497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e581a0780 con 0x7f3e58109e10 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.947+0000 7f3e55ffb640 1 -- 172.21.15.3:0/2886803497 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50010460 con 0x7f3e58109e10 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.947+0000 7f3e5f02f640 1 -- 172.21.15.3:0/2886803497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e581a6ee0 con 0x7f3e58109e10 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.948+0000 7f3e5f02f640 1 -- 172.21.15.3:0/2886803497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e58103f90 con 0x7f3e58109e10 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.950+0000 7f3e55ffb640 1 -- 172.21.15.3:0/2886803497 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f3e50003970 con 0x7f3e58109e10 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.951+0000 7f3e55ffb640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3e400b65e0 0x7f3e400b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.951+0000 7f3e5cdcb640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3e400b65e0 0x7f3e400b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.951+0000 7f3e55ffb640 1 -- 172.21.15.3:0/2886803497 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8194+0+0 (secure 0 0 0) 0x7f3e500e6f90 con 0x7f3e58109e10 2024-09-21T06:13:12.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.951+0000 7f3e5cdcb640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3e400b65e0 0x7f3e400b8af0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3e480058c0 tx=0x7f3e48005ec0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:12.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:12.955+0000 7f3e55ffb640 1 -- 172.21.15.3:0/2886803497 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3e500ad8e0 con 0x7f3e58109e10 2024-09-21T06:13:13.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.143+0000 7f3e5f02f640 1 -- 172.21.15.3:0/2886803497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e5810bc40 con 0x7f3e58109e10 2024-09-21T06:13:13.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.145+0000 7f3e55ffb640 1 -- 172.21.15.3:0/2886803497 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22552 (secure 0 0 0) 0x7f3e500b1cc0 con 0x7f3e58109e10 2024-09-21T06:13:13.147 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:13:13.147 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":77,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","created":"2024-09-21T06:04:24.293786+0000","modified":"2024-09-21T06:13:11.205276+0000","last_up_change":"2024-09-21T06:12:17.781243+0000","last_in_change":"2024-09-21T06:11:54.727435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:08:26.015803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:13:07.084406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:13:09.149891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51736f81-f410-48ee-a8f3-121289607309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6803","nonce":615312328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6805","nonce":615312328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6809","nonce":615312328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6807","nonce":615312328}]},"public_addr":"172.21.15.3:6803/615312328","cluster_addr":"172.21.15.3:6805/615312328","heartbeat_back_addr":"172.21.15.3:6809/615312328","heartbeat_front_addr":"172.21.15.3:6807/615312328","state":["exists","up"]},{"osd":1,"uuid":"55696a37-34c1-482a-8561-7d08f0aa29fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6801","nonce":4150747115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6803","nonce":4150747115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6807","nonce":4150747115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6805","nonce":4150747115}]},"public_addr":"172.21.15.137:6801/4150747115","cluster_addr":"172.21.15.137:6803/4150747115","heartbeat_back_addr":"172.21.15.137:6807/4150747115","heartbeat_front_addr":"172.21.15.137:6805/4150747115","state":["exists","up"]},{"osd":2,"uuid":"8a81d34b-464b-46a2-8ef2-5223b5947004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6801","nonce":3392577077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6803","nonce":3392577077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6807","nonce":3392577077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6805","nonce":3392577077}]},"public_addr":"172.21.15.204:6801/3392577077","cluster_addr":"172.21.15.204:6803/3392577077","heartbeat_back_addr":"172.21.15.204:6807/3392577077","heartbeat_front_addr":"172.21.15.204:6805/3392577077","state":["exists","up"]},{"osd":3,"uuid":"94ab3938-7a70-44cb-99bf-9d2dedc4eb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6811","nonce":4293741635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6813","nonce":4293741635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6817","nonce":4293741635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6815","nonce":4293741635}]},"public_addr":"172.21.15.3:6811/4293741635","cluster_addr":"172.21.15.3:6813/4293741635","heartbeat_back_addr":"172.21.15.3:6817/4293741635","heartbeat_front_addr":"172.21.15.3:6815/4293741635","state":["exists","up"]},{"osd":4,"uuid":"38551535-d692-401c-aad0-bee38f3026d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6809","nonce":2608988785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6811","nonce":2608988785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6815","nonce":2608988785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6813","nonce":2608988785}]},"public_addr":"172.21.15.137:6809/2608988785","cluster_addr":"172.21.15.137:6811/2608988785","heartbeat_back_addr":"172.21.15.137:6815/2608988785","heartbeat_front_addr":"172.21.15.137:6813/2608988785","state":["exists","up"]},{"osd":5,"uuid":"3d4b785b-28d0-4654-a9b1-4c5c905f4bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6809","nonce":3709074044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6811","nonce":3709074044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6815","nonce":3709074044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6813","nonce":3709074044}]},"public_addr":"172.21.15.204:6809/3709074044","cluster_addr":"172.21.15.204:6811/3709074044","heartbeat_back_addr":"172.21.15.204:6815/3709074044","heartbeat_front_addr":"172.21.15.204:6813/3709074044","state":["exists","up"]},{"osd":6,"uuid":"ba21dbe9-ab43-4b7e-9446-2929ee874c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6819","nonce":3862646594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6821","nonce":3862646594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6825","nonce":3862646594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6823","nonce":3862646594}]},"public_addr":"172.21.15.3:6819/3862646594","cluster_addr":"172.21.15.3:6821/3862646594","heartbeat_back_addr":"172.21.15.3:6825/3862646594","heartbeat_front_addr":"172.21.15.3:6823/3862646594","state":["exists","up"]},{"osd":7,"uuid":"9610fe5c-93df-49f9-b8bd-5cb480e0871b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6817","nonce":2086822496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6819","nonce":2086822496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6823","nonce":2086822496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6821","nonce":2086822496}]},"public_addr":"172.21.15.137:6817/2086822496","cluster_addr":"172.21.15.137:6819/2086822496","heartbeat_back_addr":"172.21.15.137:6823/2086822496","heartbeat_front_addr":"172.21.15.137:6821/2086822496","state":["exists","up"]},{"osd":8,"uuid":"49f6e9b4-6d14-434e-8b42-8b7ffeb2a072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6817","nonce":153476798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6819","nonce":153476798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6823","nonce":153476798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6821","nonce":153476798}]},"public_addr":"172.21.15.204:6817/153476798","cluster_addr":"172.21.15.204:6819/153476798","heartbeat_back_addr":"172.21.15.204:6823/153476798","heartbeat_front_addr":"172.21.15.204:6821/153476798","state":["exists","up"]},{"osd":9,"uuid":"633f67a7-b759-4ea6-82ed-3c42b3fb70ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6827","nonce":1238491549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6829","nonce":1238491549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6833","nonce":1238491549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6831","nonce":1238491549}]},"public_addr":"172.21.15.3:6827/1238491549","cluster_addr":"172.21.15.3:6829/1238491549","heartbeat_back_addr":"172.21.15.3:6833/1238491549","heartbeat_front_addr":"172.21.15.3:6831/1238491549","state":["exists","up"]},{"osd":10,"uuid":"94b65a48-ee87-4e59-9296-a72a22787f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6825","nonce":3486552683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6827","nonce":3486552683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6831","nonce":3486552683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6829","nonce":3486552683}]},"public_addr":"172.21.15.137:6825/3486552683","cluster_addr":"172.21.15.137:6827/3486552683","heartbeat_back_addr":"172.21.15.137:6831/3486552683","heartbeat_front_addr":"172.21.15.137:6829/3486552683","state":["exists","up"]},{"osd":11,"uuid":"c5cc2850-9062-4b8e-beb3-bef0db30a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6825","nonce":1084308610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6827","nonce":1084308610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6831","nonce":1084308610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6829","nonce":1084308610}]},"public_addr":"172.21.15.204:6825/1084308610","cluster_addr":"172.21.15.204:6827/1084308610","heartbeat_back_addr":"172.21.15.204:6831/1084308610","heartbeat_front_addr":"172.21.15.204:6829/1084308610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:31.898745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.089965+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:22.367464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:48.543248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:14.687331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:41.243579+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:07.290227+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:32.930046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:59.197638+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:24.315219+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:50.333639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:12:16.453640+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1573374138":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1595702358":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/547897695":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/3673347975":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/141972779":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6800/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:6801/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/2363985440":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/2138752581":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1820734047":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/2929116087":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/45453677":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/2929116087":"2024-09-22T06:05:21.689191+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:13:13.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.147+0000 7f3e5f02f640 1 -- 172.21.15.3:0/2886803497 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3e400b65e0 msgr2=0x7f3e400b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:13.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.147+0000 7f3e5f02f640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3e400b65e0 0x7f3e400b8af0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3e480058c0 tx=0x7f3e48005ec0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.147+0000 7f3e5f02f640 1 -- 172.21.15.3:0/2886803497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58109e10 msgr2=0x7f3e5819fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:13.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.147+0000 7f3e5f02f640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58109e10 0x7f3e5819fd60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009fd0 tx=0x7f3e5000b260 comp rx=0 tx=0).stop 2024-09-21T06:13:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.147+0000 7f3e5f02f640 1 -- 172.21.15.3:0/2886803497 shutdown_connections 2024-09-21T06:13:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.147+0000 7f3e5f02f640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3e400b65e0 0x7f3e400b8af0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.147+0000 7f3e5f02f640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58109e10 0x7f3e5819fd60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.147+0000 7f3e5f02f640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e581094c0 0x7f3e5819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.147+0000 7f3e5f02f640 1 --2- 172.21.15.3:0/2886803497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e58103b80 0x7f3e5819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.147+0000 7f3e5f02f640 1 -- 172.21.15.3:0/2886803497 >> 172.21.15.3:0/2886803497 conn(0x7f3e580ff5d0 msgr2=0x7f3e581019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.147+0000 7f3e5f02f640 1 -- 172.21.15.3:0/2886803497 shutdown_connections 2024-09-21T06:13:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:13.147+0000 7f3e5f02f640 1 -- 172.21.15.3:0/2886803497 wait complete. 2024-09-21T06:13:13.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:13 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] up:active 2024-09-21T06:13:13.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:13 smithi003 ceph-mon[31227]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:13:13.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:13 smithi003 ceph-mon[31227]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:13:13.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:13 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2886803497' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:13.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:13:13.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:13 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] up:active 2024-09-21T06:13:13.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:13 smithi137 ceph-mon[34464]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:13:13.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:13 smithi137 ceph-mon[34464]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:13:13.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:13 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2886803497' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:13.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:13 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] up:active 2024-09-21T06:13:13.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:13 smithi204 ceph-mon[34659]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:13:13.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:13 smithi204 ceph-mon[34659]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:13:13.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:13 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2886803497' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:14.143 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:14.565 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:14 smithi003 ceph-mon[31227]: pgmap v270: 97 pgs: 50 active+clean, 26 creating+peering, 21 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 398 B/s wr, 1 op/s 2024-09-21T06:13:14.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.653+0000 7f3c4770f640 1 -- 172.21.15.3:0/3575612713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c40108d20 msgr2=0x7f3c4010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:14.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.654+0000 7f3c4770f640 1 --2- 172.21.15.3:0/3575612713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c40108d20 0x7f3c4010f550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009ca0 tx=0x7f3c3802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.654+0000 7f3c4770f640 1 -- 172.21.15.3:0/3575612713 shutdown_connections 2024-09-21T06:13:14.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.654+0000 7f3c4770f640 1 --2- 172.21.15.3:0/3575612713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c401083d0 0x7f3c401087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.654+0000 7f3c4770f640 1 --2- 172.21.15.3:0/3575612713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c40108d20 0x7f3c4010f550 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.654+0000 7f3c4770f640 1 --2- 172.21.15.3:0/3575612713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c40102a90 0x7f3c40102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.654+0000 7f3c4770f640 1 -- 172.21.15.3:0/3575612713 >> 172.21.15.3:0/3575612713 conn(0x7f3c40079750 msgr2=0x7f3c40079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:14.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.654+0000 7f3c4770f640 1 -- 172.21.15.3:0/3575612713 shutdown_connections 2024-09-21T06:13:14.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.654+0000 7f3c4770f640 1 -- 172.21.15.3:0/3575612713 wait complete. 2024-09-21T06:13:14.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.654+0000 7f3c4770f640 1 Processor -- start 2024-09-21T06:13:14.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.654+0000 7f3c4770f640 1 -- start start 2024-09-21T06:13:14.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c4770f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c40102a90 0x7f3c4019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:14.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c4770f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c401083d0 0x7f3c4019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:14.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c4770f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c40108d20 0x7f3c4019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:14.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c4770f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40112990 con 0x7f3c40108d20 2024-09-21T06:13:14.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c4770f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40112810 con 0x7f3c401083d0 2024-09-21T06:13:14.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c4770f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40112b10 con 0x7f3c40102a90 2024-09-21T06:13:14.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c45cac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c40108d20 0x7f3c4019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:14.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c45cac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c40108d20 0x7f3c4019fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41320/0 (socket says 172.21.15.3:41320) 2024-09-21T06:13:14.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c45cac640 1 -- 172.21.15.3:0/4211938535 learned_addr learned my addr 172.21.15.3:0/4211938535 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:14.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c44caa640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c401083d0 0x7f3c4019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:14.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c454ab640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c40102a90 0x7f3c4019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:14.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c45cac640 1 -- 172.21.15.3:0/4211938535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c40102a90 msgr2=0x7f3c4019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:14.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c45cac640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c40102a90 0x7f3c4019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c45cac640 1 -- 172.21.15.3:0/4211938535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c401083d0 msgr2=0x7f3c4019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:14.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c45cac640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c401083d0 0x7f3c4019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c45cac640 1 -- 172.21.15.3:0/4211938535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c401a02e0 con 0x7f3c40108d20 2024-09-21T06:13:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.655+0000 7f3c45cac640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c40108d20 0x7f3c4019fc70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3c38005cc0 tx=0x7f3c3802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.656+0000 7f3c367fc640 1 -- 172.21.15.3:0/4211938535 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38003bf0 con 0x7f3c40108d20 2024-09-21T06:13:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.656+0000 7f3c367fc640 1 -- 172.21.15.3:0/4211938535 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c38003d90 con 0x7f3c40108d20 2024-09-21T06:13:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.656+0000 7f3c4770f640 1 -- 172.21.15.3:0/4211938535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c401a0570 con 0x7f3c40108d20 2024-09-21T06:13:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.656+0000 7f3c367fc640 1 -- 172.21.15.3:0/4211938535 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3802fbc0 con 0x7f3c40108d20 2024-09-21T06:13:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.656+0000 7f3c4770f640 1 -- 172.21.15.3:0/4211938535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c401a0ab0 con 0x7f3c40108d20 2024-09-21T06:13:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.657+0000 7f3c4770f640 1 -- 172.21.15.3:0/4211938535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c40103d10 con 0x7f3c40108d20 2024-09-21T06:13:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.658+0000 7f3c367fc640 1 -- 172.21.15.3:0/4211938535 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f3c3802fd60 con 0x7f3c40108d20 2024-09-21T06:13:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.659+0000 7f3c367fc640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3c2c0b6590 0x7f3c2c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.659+0000 7f3c454ab640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3c2c0b6590 0x7f3c2c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.659+0000 7f3c367fc640 1 -- 172.21.15.3:0/4211938535 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8194+0+0 (secure 0 0 0) 0x7f3c381088c0 con 0x7f3c40108d20 2024-09-21T06:13:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.660+0000 7f3c454ab640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3c2c0b6590 0x7f3c2c0b8aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c005fd0 tx=0x7f3c3c005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:14.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.663+0000 7f3c367fc640 1 -- 172.21.15.3:0/4211938535 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3c380cf170 con 0x7f3c40108d20 2024-09-21T06:13:14.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.851+0000 7f3c4770f640 1 -- 172.21.15.3:0/4211938535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c4019b560 con 0x7f3c40108d20 2024-09-21T06:13:14.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.853+0000 7f3c367fc640 1 -- 172.21.15.3:0/4211938535 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22552 (secure 0 0 0) 0x7f3c380d3550 con 0x7f3c40108d20 2024-09-21T06:13:14.855 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:13:14.855 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":77,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","created":"2024-09-21T06:04:24.293786+0000","modified":"2024-09-21T06:13:11.205276+0000","last_up_change":"2024-09-21T06:12:17.781243+0000","last_in_change":"2024-09-21T06:11:54.727435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:08:26.015803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:13:07.084406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:13:09.149891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51736f81-f410-48ee-a8f3-121289607309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6803","nonce":615312328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6805","nonce":615312328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6809","nonce":615312328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6807","nonce":615312328}]},"public_addr":"172.21.15.3:6803/615312328","cluster_addr":"172.21.15.3:6805/615312328","heartbeat_back_addr":"172.21.15.3:6809/615312328","heartbeat_front_addr":"172.21.15.3:6807/615312328","state":["exists","up"]},{"osd":1,"uuid":"55696a37-34c1-482a-8561-7d08f0aa29fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6801","nonce":4150747115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6803","nonce":4150747115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6807","nonce":4150747115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6805","nonce":4150747115}]},"public_addr":"172.21.15.137:6801/4150747115","cluster_addr":"172.21.15.137:6803/4150747115","heartbeat_back_addr":"172.21.15.137:6807/4150747115","heartbeat_front_addr":"172.21.15.137:6805/4150747115","state":["exists","up"]},{"osd":2,"uuid":"8a81d34b-464b-46a2-8ef2-5223b5947004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6801","nonce":3392577077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6803","nonce":3392577077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6807","nonce":3392577077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6805","nonce":3392577077}]},"public_addr":"172.21.15.204:6801/3392577077","cluster_addr":"172.21.15.204:6803/3392577077","heartbeat_back_addr":"172.21.15.204:6807/3392577077","heartbeat_front_addr":"172.21.15.204:6805/3392577077","state":["exists","up"]},{"osd":3,"uuid":"94ab3938-7a70-44cb-99bf-9d2dedc4eb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6811","nonce":4293741635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6813","nonce":4293741635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6817","nonce":4293741635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6815","nonce":4293741635}]},"public_addr":"172.21.15.3:6811/4293741635","cluster_addr":"172.21.15.3:6813/4293741635","heartbeat_back_addr":"172.21.15.3:6817/4293741635","heartbeat_front_addr":"172.21.15.3:6815/4293741635","state":["exists","up"]},{"osd":4,"uuid":"38551535-d692-401c-aad0-bee38f3026d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6809","nonce":2608988785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6811","nonce":2608988785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6815","nonce":2608988785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6813","nonce":2608988785}]},"public_addr":"172.21.15.137:6809/2608988785","cluster_addr":"172.21.15.137:6811/2608988785","heartbeat_back_addr":"172.21.15.137:6815/2608988785","heartbeat_front_addr":"172.21.15.137:6813/2608988785","state":["exists","up"]},{"osd":5,"uuid":"3d4b785b-28d0-4654-a9b1-4c5c905f4bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6809","nonce":3709074044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6811","nonce":3709074044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6815","nonce":3709074044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6813","nonce":3709074044}]},"public_addr":"172.21.15.204:6809/3709074044","cluster_addr":"172.21.15.204:6811/3709074044","heartbeat_back_addr":"172.21.15.204:6815/3709074044","heartbeat_front_addr":"172.21.15.204:6813/3709074044","state":["exists","up"]},{"osd":6,"uuid":"ba21dbe9-ab43-4b7e-9446-2929ee874c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6819","nonce":3862646594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6821","nonce":3862646594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6825","nonce":3862646594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6823","nonce":3862646594}]},"public_addr":"172.21.15.3:6819/3862646594","cluster_addr":"172.21.15.3:6821/3862646594","heartbeat_back_addr":"172.21.15.3:6825/3862646594","heartbeat_front_addr":"172.21.15.3:6823/3862646594","state":["exists","up"]},{"osd":7,"uuid":"9610fe5c-93df-49f9-b8bd-5cb480e0871b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6817","nonce":2086822496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6819","nonce":2086822496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6823","nonce":2086822496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6821","nonce":2086822496}]},"public_addr":"172.21.15.137:6817/2086822496","cluster_addr":"172.21.15.137:6819/2086822496","heartbeat_back_addr":"172.21.15.137:6823/2086822496","heartbeat_front_addr":"172.21.15.137:6821/2086822496","state":["exists","up"]},{"osd":8,"uuid":"49f6e9b4-6d14-434e-8b42-8b7ffeb2a072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6817","nonce":153476798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6819","nonce":153476798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6823","nonce":153476798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6821","nonce":153476798}]},"public_addr":"172.21.15.204:6817/153476798","cluster_addr":"172.21.15.204:6819/153476798","heartbeat_back_addr":"172.21.15.204:6823/153476798","heartbeat_front_addr":"172.21.15.204:6821/153476798","state":["exists","up"]},{"osd":9,"uuid":"633f67a7-b759-4ea6-82ed-3c42b3fb70ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6827","nonce":1238491549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6829","nonce":1238491549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6833","nonce":1238491549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6831","nonce":1238491549}]},"public_addr":"172.21.15.3:6827/1238491549","cluster_addr":"172.21.15.3:6829/1238491549","heartbeat_back_addr":"172.21.15.3:6833/1238491549","heartbeat_front_addr":"172.21.15.3:6831/1238491549","state":["exists","up"]},{"osd":10,"uuid":"94b65a48-ee87-4e59-9296-a72a22787f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6825","nonce":3486552683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6827","nonce":3486552683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6831","nonce":3486552683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6829","nonce":3486552683}]},"public_addr":"172.21.15.137:6825/3486552683","cluster_addr":"172.21.15.137:6827/3486552683","heartbeat_back_addr":"172.21.15.137:6831/3486552683","heartbeat_front_addr":"172.21.15.137:6829/3486552683","state":["exists","up"]},{"osd":11,"uuid":"c5cc2850-9062-4b8e-beb3-bef0db30a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6825","nonce":1084308610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6827","nonce":1084308610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6831","nonce":1084308610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6829","nonce":1084308610}]},"public_addr":"172.21.15.204:6825/1084308610","cluster_addr":"172.21.15.204:6827/1084308610","heartbeat_back_addr":"172.21.15.204:6831/1084308610","heartbeat_front_addr":"172.21.15.204:6829/1084308610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:31.898745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.089965+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:22.367464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:48.543248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:14.687331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:41.243579+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:07.290227+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:32.930046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:59.197638+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:24.315219+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:50.333639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:12:16.453640+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1573374138":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1595702358":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/547897695":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/3673347975":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/141972779":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6800/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:6801/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/2363985440":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/2138752581":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1820734047":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/2929116087":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/45453677":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/2929116087":"2024-09-22T06:05:21.689191+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:13:14.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.855+0000 7f3c4770f640 1 -- 172.21.15.3:0/4211938535 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3c2c0b6590 msgr2=0x7f3c2c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:14.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.855+0000 7f3c4770f640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3c2c0b6590 0x7f3c2c0b8aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c005fd0 tx=0x7f3c3c005850 comp rx=0 tx=0).stop 2024-09-21T06:13:14.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.855+0000 7f3c4770f640 1 -- 172.21.15.3:0/4211938535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c40108d20 msgr2=0x7f3c4019fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:14.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.855+0000 7f3c4770f640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c40108d20 0x7f3c4019fc70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3c38005cc0 tx=0x7f3c3802f380 comp rx=0 tx=0).stop 2024-09-21T06:13:14.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.855+0000 7f3c4770f640 1 -- 172.21.15.3:0/4211938535 shutdown_connections 2024-09-21T06:13:14.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.855+0000 7f3c4770f640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3c2c0b6590 0x7f3c2c0b8aa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.855+0000 7f3c4770f640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c40108d20 0x7f3c4019fc70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.855+0000 7f3c4770f640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c401083d0 0x7f3c4019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.855+0000 7f3c4770f640 1 --2- 172.21.15.3:0/4211938535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c40102a90 0x7f3c4019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.855+0000 7f3c4770f640 1 -- 172.21.15.3:0/4211938535 >> 172.21.15.3:0/4211938535 conn(0x7f3c40079750 msgr2=0x7f3c4006bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:14.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.855+0000 7f3c4770f640 1 -- 172.21.15.3:0/4211938535 shutdown_connections 2024-09-21T06:13:14.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:14.855+0000 7f3c4770f640 1 -- 172.21.15.3:0/4211938535 wait complete. 2024-09-21T06:13:14.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:14 smithi137 ceph-mon[34464]: pgmap v270: 97 pgs: 50 active+clean, 26 creating+peering, 21 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 398 B/s wr, 1 op/s 2024-09-21T06:13:14.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:14 smithi204 ceph-mon[34659]: pgmap v270: 97 pgs: 50 active+clean, 26 creating+peering, 21 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 398 B/s wr, 1 op/s 2024-09-21T06:13:15.601 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:13:15.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:15 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/4211938535' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:15.860 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:15.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:15 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/4211938535' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:15.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:15 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/4211938535' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.362+0000 7f17c970e640 1 -- 172.21.15.3:0/3377227715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c41083b0 msgr2=0x7f17c41087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.362+0000 7f17c970e640 1 --2- 172.21.15.3:0/3377227715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c41083b0 0x7f17c41087c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009900 tx=0x7f17ac02a410 comp rx=0 tx=0).stop 2024-09-21T06:13:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.362+0000 7f17c970e640 1 -- 172.21.15.3:0/3377227715 shutdown_connections 2024-09-21T06:13:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.362+0000 7f17c970e640 1 --2- 172.21.15.3:0/3377227715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17c4108d00 0x7f17c410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.362+0000 7f17c970e640 1 --2- 172.21.15.3:0/3377227715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c41083b0 0x7f17c41087c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.362+0000 7f17c970e640 1 --2- 172.21.15.3:0/3377227715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c4102a70 0x7f17c4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.362+0000 7f17c970e640 1 -- 172.21.15.3:0/3377227715 >> 172.21.15.3:0/3377227715 conn(0x7f17c4079730 msgr2=0x7f17c4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.362+0000 7f17c970e640 1 -- 172.21.15.3:0/3377227715 shutdown_connections 2024-09-21T06:13:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.362+0000 7f17c970e640 1 -- 172.21.15.3:0/3377227715 wait complete. 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.363+0000 7f17c970e640 1 Processor -- start 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.363+0000 7f17c970e640 1 -- start start 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.363+0000 7f17c970e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c4102a70 0x7f17c419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.363+0000 7f17c970e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c41083b0 0x7f17c419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.363+0000 7f17c970e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17c4108d00 0x7f17c419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.363+0000 7f17c970e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4112970 con 0x7f17c4102a70 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.363+0000 7f17c970e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f17c41127f0 con 0x7f17c4108d00 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.363+0000 7f17c970e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4112af0 con 0x7f17c41083b0 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.363+0000 7f17c2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c4102a70 0x7f17c419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.363+0000 7f17c2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c4102a70 0x7f17c419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50804/0 (socket says 172.21.15.3:50804) 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.363+0000 7f17c2ffd640 1 -- 172.21.15.3:0/825688182 learned_addr learned my addr 172.21.15.3:0/825688182 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.363+0000 7f17bbfff640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c41083b0 0x7f17c419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.364+0000 7f17c37fe640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17c4108d00 0x7f17c419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.364+0000 7f17c2ffd640 1 -- 172.21.15.3:0/825688182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c41083b0 msgr2=0x7f17c419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.364+0000 7f17c2ffd640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c41083b0 0x7f17c419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.364+0000 7f17c2ffd640 1 -- 172.21.15.3:0/825688182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17c4108d00 msgr2=0x7f17c419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.364+0000 7f17c2ffd640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17c4108d00 0x7f17c419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.364+0000 7f17c2ffd640 1 -- 172.21.15.3:0/825688182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c41a0440 con 0x7f17c4102a70 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.364+0000 7f17c2ffd640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c4102a70 0x7f17c419a4d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f17b400c970 tx=0x7f17b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.364+0000 7f17c0ff9640 1 -- 172.21.15.3:0/825688182 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4007bb0 con 0x7f17c4102a70 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.364+0000 7f17c970e640 1 -- 172.21.15.3:0/825688182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c41a0730 con 0x7f17c4102a70 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.364+0000 7f17c970e640 1 -- 172.21.15.3:0/825688182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c41a6e90 con 0x7f17c4102a70 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.364+0000 7f17c0ff9640 1 -- 172.21.15.3:0/825688182 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17b4007d50 con 0x7f17c4102a70 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.364+0000 7f17c0ff9640 1 -- 172.21.15.3:0/825688182 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b400f450 con 0x7f17c4102a70 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.365+0000 7f17c970e640 1 -- 172.21.15.3:0/825688182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a40052c0 con 0x7f17c4102a70 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.368+0000 7f17c0ff9640 1 -- 172.21.15.3:0/825688182 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f17b40046c0 con 0x7f17c4102a70 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.369+0000 7f17c0ff9640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f17b00b6560 0x7f17b00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.369+0000 7f17bbfff640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f17b00b6560 0x7f17b00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.369+0000 7f17c0ff9640 1 -- 172.21.15.3:0/825688182 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8194+0+0 (secure 0 0 0) 0x7f17b40e72e0 con 0x7f17c4102a70 2024-09-21T06:13:16.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.370+0000 7f17bbfff640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f17b00b6560 0x7f17b00b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f17ac000c00 tx=0x7f17ac031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:16.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.371+0000 7f17c0ff9640 1 -- 172.21.15.3:0/825688182 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f17b4014030 con 0x7f17c4102a70 2024-09-21T06:13:16.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:16.606+0000 7f17c970e640 1 -- 172.21.15.3:0/825688182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f17a40055b0 con 0x7f17c4102a70 2024-09-21T06:13:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:16 smithi137 ceph-mon[34464]: pgmap v271: 97 pgs: 97 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2024-09-21T06:13:16.645 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:13:16 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b[63057]: 2024-09-21T06:13:16.273+0000 7f365ef26640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:13:16.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:16 smithi003 ceph-mon[31227]: pgmap v271: 97 pgs: 97 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2024-09-21T06:13:16.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:16 smithi204 ceph-mon[34659]: pgmap v271: 97 pgs: 97 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2024-09-21T06:13:17.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:17.528+0000 7f17c0ff9640 1 -- 172.21.15.3:0/825688182 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f17b40adb30 con 0x7f17c4102a70 2024-09-21T06:13:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:17.531+0000 7f17c970e640 1 -- 172.21.15.3:0/825688182 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f17b00b6560 msgr2=0x7f17b00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:17.531+0000 7f17c970e640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f17b00b6560 0x7f17b00b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f17ac000c00 tx=0x7f17ac031400 comp rx=0 tx=0).stop 2024-09-21T06:13:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:17.531+0000 7f17c970e640 1 -- 172.21.15.3:0/825688182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c4102a70 msgr2=0x7f17c419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:17.531+0000 7f17c970e640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c4102a70 0x7f17c419a4d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f17b400c970 tx=0x7f17b400ce30 comp rx=0 tx=0).stop 2024-09-21T06:13:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:17.531+0000 7f17c970e640 1 -- 172.21.15.3:0/825688182 shutdown_connections 2024-09-21T06:13:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:17.531+0000 7f17c970e640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f17b00b6560 0x7f17b00b8a70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:17.531+0000 7f17c970e640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c41083b0 0x7f17c419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:17.531+0000 7f17c970e640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17c4108d00 0x7f17c419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:17.531+0000 7f17c970e640 1 --2- 172.21.15.3:0/825688182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c4102a70 0x7f17c419a4d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:17.531+0000 7f17c970e640 1 -- 172.21.15.3:0/825688182 >> 172.21.15.3:0/825688182 conn(0x7f17c4079730 msgr2=0x7f17c4101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:17.531+0000 7f17c970e640 1 -- 172.21.15.3:0/825688182 shutdown_connections 2024-09-21T06:13:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:17.531+0000 7f17c970e640 1 -- 172.21.15.3:0/825688182 wait complete. 2024-09-21T06:13:17.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:17 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/825688182' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:13:17.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:17 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/825688182' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:13:17.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:17 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/825688182' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:13:18.236 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-21T06:13:18.237 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 5 2024-09-21T06:13:18.501 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:18.822 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:18 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/825688182' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T06:13:18.822 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:18 smithi003 ceph-mon[31227]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:13:18.822 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:18 smithi003 ceph-mon[31227]: pgmap v272: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T06:13:18.834 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:18 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/825688182' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T06:13:18.834 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:18 smithi137 ceph-mon[34464]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:13:18.834 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:18 smithi137 ceph-mon[34464]: pgmap v272: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T06:13:18.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:18 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/825688182' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T06:13:18.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:18 smithi204 ceph-mon[34659]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:13:18.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:18 smithi204 ceph-mon[34659]: pgmap v272: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T06:13:19.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.044+0000 7fc3a25dc640 1 -- 172.21.15.3:0/206245753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc39c103b60 msgr2=0x7fc39c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.044+0000 7fc3a25dc640 1 --2- 172.21.15.3:0/206245753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc39c103b60 0x7fc39c103f70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009900 tx=0x7fc38c02a410 comp rx=0 tx=0).stop 2024-09-21T06:13:19.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.044+0000 7fc3a25dc640 1 -- 172.21.15.3:0/206245753 shutdown_connections 2024-09-21T06:13:19.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.044+0000 7fc3a25dc640 1 --2- 172.21.15.3:0/206245753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c109df0 0x7fc39c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.044+0000 7fc3a25dc640 1 --2- 172.21.15.3:0/206245753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc39c1094a0 0x7fc39c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.044+0000 7fc3a25dc640 1 --2- 172.21.15.3:0/206245753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc39c103b60 0x7fc39c103f70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.044+0000 7fc3a25dc640 1 -- 172.21.15.3:0/206245753 >> 172.21.15.3:0/206245753 conn(0x7fc39c07b850 msgr2=0x7fc39c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:19.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.044+0000 7fc3a25dc640 1 -- 172.21.15.3:0/206245753 shutdown_connections 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.044+0000 7fc3a25dc640 1 -- 172.21.15.3:0/206245753 wait complete. 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.044+0000 7fc3a25dc640 1 Processor -- start 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.044+0000 7fc3a25dc640 1 -- start start 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc3a25dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc39c103b60 0x7fc39c077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc3a25dc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc39c1094a0 0x7fc39c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc3a25dc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c109df0 0x7fc39c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc3a25dc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c113af0 con 0x7fc39c1094a0 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc3a25dc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c113970 con 0x7fc39c103b60 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc3a25dc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c113c70 con 0x7fc39c109df0 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc39b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc39c1094a0 0x7fc39c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc39b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc39c1094a0 0x7fc39c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50816/0 (socket says 172.21.15.3:50816) 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc39b7fe640 1 -- 172.21.15.3:0/2294474035 learned_addr learned my addr 172.21.15.3:0/2294474035 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc3a0b79640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c109df0 0x7fc39c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc39bfff640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc39c103b60 0x7fc39c077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc39b7fe640 1 -- 172.21.15.3:0/2294474035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c109df0 msgr2=0x7fc39c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc39b7fe640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c109df0 0x7fc39c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc39b7fe640 1 -- 172.21.15.3:0/2294474035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc39c103b60 msgr2=0x7fc39c077120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc39b7fe640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc39c103b60 0x7fc39c077120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc39b7fe640 1 -- 172.21.15.3:0/2294474035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39c0763e0 con 0x7fc39c1094a0 2024-09-21T06:13:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.045+0000 7fc39bfff640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc39c103b60 0x7fc39c077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.046+0000 7fc39b7fe640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc39c1094a0 0x7fc39c075710 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc390006fd0 tx=0x7fc39000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.046+0000 7fc3997fa640 1 -- 172.21.15.3:0/2294474035 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390011830 con 0x7fc39c1094a0 2024-09-21T06:13:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.046+0000 7fc3997fa640 1 -- 172.21.15.3:0/2294474035 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc390016070 con 0x7fc39c1094a0 2024-09-21T06:13:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.046+0000 7fc3a25dc640 1 -- 172.21.15.3:0/2294474035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39c1ace60 con 0x7fc39c1094a0 2024-09-21T06:13:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.046+0000 7fc3997fa640 1 -- 172.21.15.3:0/2294474035 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390010e00 con 0x7fc39c1094a0 2024-09-21T06:13:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.046+0000 7fc3a25dc640 1 -- 172.21.15.3:0/2294474035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39c1ad340 con 0x7fc39c1094a0 2024-09-21T06:13:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.047+0000 7fc3a25dc640 1 -- 172.21.15.3:0/2294474035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc39c103f70 con 0x7fc39c1094a0 2024-09-21T06:13:19.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.049+0000 7fc3997fa640 1 -- 172.21.15.3:0/2294474035 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7fc39001b460 con 0x7fc39c1094a0 2024-09-21T06:13:19.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.049+0000 7fc3997fa640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc3840b6560 0x7fc3840b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.050+0000 7fc39bfff640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc3840b6560 0x7fc3840b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.050+0000 7fc3997fa640 1 -- 172.21.15.3:0/2294474035 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8194+0+0 (secure 0 0 0) 0x7fc3900e6f80 con 0x7fc39c1094a0 2024-09-21T06:13:19.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.050+0000 7fc39bfff640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc3840b6560 0x7fc3840b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc38c006010 tx=0x7fc38c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.053+0000 7fc3997fa640 1 -- 172.21.15.3:0/2294474035 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3900ad830 con 0x7fc39c1094a0 2024-09-21T06:13:19.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.280+0000 7fc3a25dc640 1 -- 172.21.15.3:0/2294474035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fc39c10bbc0 con 0x7fc39c1094a0 2024-09-21T06:13:19.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.541+0000 7fc3997fa640 1 -- 172.21.15.3:0/2294474035 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fc3900b1c10 con 0x7fc39c1094a0 2024-09-21T06:13:19.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.543+0000 7fc3a25dc640 1 -- 172.21.15.3:0/2294474035 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc3840b6560 msgr2=0x7fc3840b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.543+0000 7fc3a25dc640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc3840b6560 0x7fc3840b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc38c006010 tx=0x7fc38c031400 comp rx=0 tx=0).stop 2024-09-21T06:13:19.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.543+0000 7fc3a25dc640 1 -- 172.21.15.3:0/2294474035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc39c1094a0 msgr2=0x7fc39c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.543+0000 7fc3a25dc640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc39c1094a0 0x7fc39c075710 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc390006fd0 tx=0x7fc39000b8e0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.543+0000 7fc3a25dc640 1 -- 172.21.15.3:0/2294474035 shutdown_connections 2024-09-21T06:13:19.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.543+0000 7fc3a25dc640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc3840b6560 0x7fc3840b8a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.543+0000 7fc3a25dc640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c109df0 0x7fc39c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.543+0000 7fc3a25dc640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc39c1094a0 0x7fc39c075710 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.543+0000 7fc3a25dc640 1 --2- 172.21.15.3:0/2294474035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc39c103b60 0x7fc39c077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.543+0000 7fc3a25dc640 1 -- 172.21.15.3:0/2294474035 >> 172.21.15.3:0/2294474035 conn(0x7fc39c07b850 msgr2=0x7fc39c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:19.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.543+0000 7fc3a25dc640 1 -- 172.21.15.3:0/2294474035 shutdown_connections 2024-09-21T06:13:19.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:19.543+0000 7fc3a25dc640 1 -- 172.21.15.3:0/2294474035 wait complete. 2024-09-21T06:13:19.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:19 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2294474035' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T06:13:19.850 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:19 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2294474035' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T06:13:19.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:19 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2294474035' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T06:13:20.288 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T06:13:20.552 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:20.820 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:20 smithi003 ceph-mon[31227]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T06:13:20.820 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:20 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2294474035' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T06:13:20.820 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:20 smithi003 ceph-mon[31227]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T06:13:20.820 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:20 smithi003 ceph-mon[31227]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:13:20.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:20 smithi003 ceph-mon[31227]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T06:13:20.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:20 smithi003 ceph-mon[31227]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T06:13:20.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:20 smithi003 ceph-mon[31227]: pgmap v273: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-21T06:13:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:20 smithi137 ceph-mon[34464]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T06:13:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:20 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2294474035' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T06:13:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:20 smithi137 ceph-mon[34464]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T06:13:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:20 smithi137 ceph-mon[34464]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:13:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:20 smithi137 ceph-mon[34464]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T06:13:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:20 smithi137 ceph-mon[34464]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T06:13:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:20 smithi137 ceph-mon[34464]: pgmap v273: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-21T06:13:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:20 smithi204 ceph-mon[34659]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T06:13:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:20 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2294474035' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T06:13:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:20 smithi204 ceph-mon[34659]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T06:13:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:20 smithi204 ceph-mon[34659]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:13:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:20 smithi204 ceph-mon[34659]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T06:13:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:20 smithi204 ceph-mon[34659]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T06:13:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:20 smithi204 ceph-mon[34659]: pgmap v273: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-21T06:13:21.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.053+0000 7fb7adcea640 1 -- 172.21.15.3:0/3953688162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a8109e00 msgr2=0x7fb7a8110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.053+0000 7fb7adcea640 1 --2- 172.21.15.3:0/3953688162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a8109e00 0x7fb7a8110630 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0009ca0 tx=0x7fb7a002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.053+0000 7fb7adcea640 1 -- 172.21.15.3:0/3953688162 shutdown_connections 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.053+0000 7fb7adcea640 1 --2- 172.21.15.3:0/3953688162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a81094b0 0x7fb7a81098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.053+0000 7fb7adcea640 1 --2- 172.21.15.3:0/3953688162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a8109e00 0x7fb7a8110630 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.053+0000 7fb7adcea640 1 --2- 172.21.15.3:0/3953688162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a8103bc0 0x7fb7a8103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.053+0000 7fb7adcea640 1 -- 172.21.15.3:0/3953688162 >> 172.21.15.3:0/3953688162 conn(0x7fb7a807b910 msgr2=0x7fb7a807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.053+0000 7fb7adcea640 1 -- 172.21.15.3:0/3953688162 shutdown_connections 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.053+0000 7fb7adcea640 1 -- 172.21.15.3:0/3953688162 wait complete. 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.054+0000 7fb7adcea640 1 Processor -- start 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.054+0000 7fb7adcea640 1 -- start start 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.054+0000 7fb7adcea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a8103bc0 0x7fb7a819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.054+0000 7fb7adcea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a81094b0 0x7fb7a819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.054+0000 7fb7adcea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a8109e00 0x7fb7a819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.054+0000 7fb7adcea640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8113ac0 con 0x7fb7a8103bc0 2024-09-21T06:13:21.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.054+0000 7fb7adcea640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8113940 con 0x7fb7a81094b0 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.054+0000 7fb7adcea640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8113c40 con 0x7fb7a8109e00 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a8103bc0 0x7fb7a819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a8103bc0 0x7fb7a819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50844/0 (socket says 172.21.15.3:50844) 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a77fe640 1 -- 172.21.15.3:0/943679179 learned_addr learned my addr 172.21.15.3:0/943679179 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a6ffd640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a81094b0 0x7fb7a819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a7fff640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a8109e00 0x7fb7a819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a77fe640 1 -- 172.21.15.3:0/943679179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a8109e00 msgr2=0x7fb7a819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a77fe640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a8109e00 0x7fb7a819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a77fe640 1 -- 172.21.15.3:0/943679179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a81094b0 msgr2=0x7fb7a819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a77fe640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a81094b0 0x7fb7a819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a77fe640 1 -- 172.21.15.3:0/943679179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a81a0380 con 0x7fb7a8103bc0 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a77fe640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a8103bc0 0x7fb7a819a4d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb79800c920 tx=0x7fb79800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a4ff9640 1 -- 172.21.15.3:0/943679179 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb798007c70 con 0x7fb7a8103bc0 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a4ff9640 1 -- 172.21.15.3:0/943679179 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb798007e10 con 0x7fb7a8103bc0 2024-09-21T06:13:21.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7adcea640 1 -- 172.21.15.3:0/943679179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a81a0670 con 0x7fb7a8103bc0 2024-09-21T06:13:21.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7a4ff9640 1 -- 172.21.15.3:0/943679179 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb798018820 con 0x7fb7a8103bc0 2024-09-21T06:13:21.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.055+0000 7fb7adcea640 1 -- 172.21.15.3:0/943679179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a81a6e90 con 0x7fb7a8103bc0 2024-09-21T06:13:21.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.057+0000 7fb7adcea640 1 -- 172.21.15.3:0/943679179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a8103fd0 con 0x7fb7a8103bc0 2024-09-21T06:13:21.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.058+0000 7fb7a4ff9640 1 -- 172.21.15.3:0/943679179 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7fb79800f450 con 0x7fb7a8103bc0 2024-09-21T06:13:21.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.059+0000 7fb7a4ff9640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb7900b6590 0x7fb7900b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.059+0000 7fb7a6ffd640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb7900b6590 0x7fb7900b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.059+0000 7fb7a4ff9640 1 -- 172.21.15.3:0/943679179 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8194+0+0 (secure 0 0 0) 0x7fb7980e7470 con 0x7fb7a8103bc0 2024-09-21T06:13:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.059+0000 7fb7a6ffd640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb7900b6590 0x7fb7900b8aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8108fd0 tx=0x7fb79c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:21.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.063+0000 7fb7a4ff9640 1 -- 172.21.15.3:0/943679179 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb798014030 con 0x7fb7a8103bc0 2024-09-21T06:13:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.296+0000 7fb7adcea640 1 -- 172.21.15.3:0/943679179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fb7a802e640 con 0x7fb7a8103bc0 2024-09-21T06:13:21.558 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:13:21 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b[63057]: 2024-09-21T06:13:21.274+0000 7f365ef26640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T06:13:21.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.563+0000 7fb7a4ff9640 1 -- 172.21.15.3:0/943679179 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fb7980add20 con 0x7fb7a8103bc0 2024-09-21T06:13:21.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.565+0000 7fb7adcea640 1 -- 172.21.15.3:0/943679179 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb7900b6590 msgr2=0x7fb7900b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.565+0000 7fb7adcea640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb7900b6590 0x7fb7900b8aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8108fd0 tx=0x7fb79c00a2d0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.565+0000 7fb7adcea640 1 -- 172.21.15.3:0/943679179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a8103bc0 msgr2=0x7fb7a819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.565+0000 7fb7adcea640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a8103bc0 0x7fb7a819a4d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb79800c920 tx=0x7fb79800cde0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.565+0000 7fb7adcea640 1 -- 172.21.15.3:0/943679179 shutdown_connections 2024-09-21T06:13:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.565+0000 7fb7adcea640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb7900b6590 0x7fb7900b8aa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.565+0000 7fb7adcea640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7a8109e00 0x7fb7a819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.565+0000 7fb7adcea640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a81094b0 0x7fb7a819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.565+0000 7fb7adcea640 1 --2- 172.21.15.3:0/943679179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a8103bc0 0x7fb7a819a4d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.565+0000 7fb7adcea640 1 -- 172.21.15.3:0/943679179 >> 172.21.15.3:0/943679179 conn(0x7fb7a807b910 msgr2=0x7fb7a8107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.566+0000 7fb7adcea640 1 -- 172.21.15.3:0/943679179 shutdown_connections 2024-09-21T06:13:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:21.566+0000 7fb7adcea640 1 -- 172.21.15.3:0/943679179 wait complete. 2024-09-21T06:13:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:21 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] up:active 2024-09-21T06:13:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:21 smithi003 ceph-mon[31227]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T06:13:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:21 smithi003 ceph-mon[31227]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T06:13:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:21 smithi003 ceph-mon[31227]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-21T06:13:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:21 smithi003 ceph-mon[31227]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-21T06:13:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:21 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/943679179' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T06:13:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:21 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] up:active 2024-09-21T06:13:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:21 smithi137 ceph-mon[34464]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T06:13:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:21 smithi137 ceph-mon[34464]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T06:13:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:21 smithi137 ceph-mon[34464]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-21T06:13:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:21 smithi137 ceph-mon[34464]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-21T06:13:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:21 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/943679179' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T06:13:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:21 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] up:active 2024-09-21T06:13:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:21 smithi204 ceph-mon[34659]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T06:13:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:21 smithi204 ceph-mon[34659]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T06:13:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:21 smithi204 ceph-mon[34659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-21T06:13:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:21 smithi204 ceph-mon[34659]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-21T06:13:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:21 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/943679179' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T06:13:22.318 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T06:13:22.585 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:22 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/943679179' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T06:13:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:22 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] up:active 2024-09-21T06:13:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:22 smithi003 ceph-mon[31227]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T06:13:22.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:22 smithi003 ceph-mon[31227]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-21T06:13:22.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:22 smithi003 ceph-mon[31227]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-21T06:13:22.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:22 smithi003 ceph-mon[31227]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-21T06:13:22.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:22 smithi003 ceph-mon[31227]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-09-21T06:13:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:22 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/943679179' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T06:13:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:22 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] up:active 2024-09-21T06:13:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:22 smithi137 ceph-mon[34464]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T06:13:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:22 smithi137 ceph-mon[34464]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-21T06:13:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:22 smithi137 ceph-mon[34464]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-21T06:13:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:22 smithi137 ceph-mon[34464]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-21T06:13:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:22 smithi137 ceph-mon[34464]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-09-21T06:13:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:22 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/943679179' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T06:13:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:22 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] up:active 2024-09-21T06:13:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:22 smithi204 ceph-mon[34659]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T06:13:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:22 smithi204 ceph-mon[34659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-21T06:13:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:22 smithi204 ceph-mon[34659]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-21T06:13:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:22 smithi204 ceph-mon[34659]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-21T06:13:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:22 smithi204 ceph-mon[34659]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-09-21T06:13:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.087+0000 7fa143ffa640 1 -- 172.21.15.3:0/1655684450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c103b60 msgr2=0x7fa13c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.087+0000 7fa143ffa640 1 --2- 172.21.15.3:0/1655684450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c103b60 0x7fa13c103f70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa138009900 tx=0x7fa13802a410 comp rx=0 tx=0).stop 2024-09-21T06:13:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.087+0000 7fa143ffa640 1 -- 172.21.15.3:0/1655684450 shutdown_connections 2024-09-21T06:13:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.087+0000 7fa143ffa640 1 --2- 172.21.15.3:0/1655684450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa13c1094a0 0x7fa13c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.087+0000 7fa143ffa640 1 --2- 172.21.15.3:0/1655684450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa13c109df0 0x7fa13c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.087+0000 7fa143ffa640 1 --2- 172.21.15.3:0/1655684450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c103b60 0x7fa13c103f70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.087+0000 7fa143ffa640 1 -- 172.21.15.3:0/1655684450 >> 172.21.15.3:0/1655684450 conn(0x7fa13c07b850 msgr2=0x7fa13c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.087+0000 7fa143ffa640 1 -- 172.21.15.3:0/1655684450 shutdown_connections 2024-09-21T06:13:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.087+0000 7fa143ffa640 1 -- 172.21.15.3:0/1655684450 wait complete. 2024-09-21T06:13:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa143ffa640 1 Processor -- start 2024-09-21T06:13:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa143ffa640 1 -- start start 2024-09-21T06:13:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa143ffa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa13c103b60 0x7fa13c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa143ffa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa13c1094a0 0x7fa13c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa143ffa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c109df0 0x7fa13c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa143ffa640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c113ab0 con 0x7fa13c109df0 2024-09-21T06:13:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa143ffa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c113930 con 0x7fa13c103b60 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa143ffa640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c113c30 con 0x7fa13c1094a0 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa142597640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c109df0 0x7fa13c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa142597640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c109df0 0x7fa13c19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50864/0 (socket says 172.21.15.3:50864) 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa142597640 1 -- 172.21.15.3:0/13570222 learned_addr learned my addr 172.21.15.3:0/13570222 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa141d96640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa13c103b60 0x7fa13c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa141595640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa13c1094a0 0x7fa13c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa142597640 1 -- 172.21.15.3:0/13570222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa13c1094a0 msgr2=0x7fa13c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa142597640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa13c1094a0 0x7fa13c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa142597640 1 -- 172.21.15.3:0/13570222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa13c103b60 msgr2=0x7fa13c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa142597640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa13c103b60 0x7fa13c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.088+0000 7fa142597640 1 -- 172.21.15.3:0/13570222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c1a0410 con 0x7fa13c109df0 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.089+0000 7fa142597640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c109df0 0x7fa13c19fce0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa13400eaa0 tx=0x7fa13400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.089+0000 7fa132ffd640 1 -- 172.21.15.3:0/13570222 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13400cdb0 con 0x7fa13c109df0 2024-09-21T06:13:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.089+0000 7fa143ffa640 1 -- 172.21.15.3:0/13570222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13c1a0700 con 0x7fa13c109df0 2024-09-21T06:13:23.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.089+0000 7fa132ffd640 1 -- 172.21.15.3:0/13570222 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa134005020 con 0x7fa13c109df0 2024-09-21T06:13:23.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.089+0000 7fa132ffd640 1 -- 172.21.15.3:0/13570222 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1340116d0 con 0x7fa13c109df0 2024-09-21T06:13:23.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.089+0000 7fa143ffa640 1 -- 172.21.15.3:0/13570222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13c1a6e60 con 0x7fa13c109df0 2024-09-21T06:13:23.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.090+0000 7fa143ffa640 1 -- 172.21.15.3:0/13570222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13c103f70 con 0x7fa13c109df0 2024-09-21T06:13:23.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.092+0000 7fa132ffd640 1 -- 172.21.15.3:0/13570222 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7fa1340051d0 con 0x7fa13c109df0 2024-09-21T06:13:23.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.093+0000 7fa132ffd640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa1280b6560 0x7fa1280b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:23.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.093+0000 7fa141d96640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa1280b6560 0x7fa1280b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:23.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.093+0000 7fa132ffd640 1 -- 172.21.15.3:0/13570222 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8194+0+0 (secure 0 0 0) 0x7fa1340e78c0 con 0x7fa13c109df0 2024-09-21T06:13:23.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.093+0000 7fa141d96640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa1280b6560 0x7fa1280b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa138031440 tx=0x7fa138009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:23.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.096+0000 7fa132ffd640 1 -- 172.21.15.3:0/13570222 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa134016030 con 0x7fa13c109df0 2024-09-21T06:13:23.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.327+0000 7fa143ffa640 1 -- 172.21.15.3:0/13570222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa13c10bbc0 con 0x7fa13c109df0 2024-09-21T06:13:23.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.593+0000 7fa132ffd640 1 -- 172.21.15.3:0/13570222 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fa1340ae170 con 0x7fa13c109df0 2024-09-21T06:13:23.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.597+0000 7fa143ffa640 1 -- 172.21.15.3:0/13570222 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa1280b6560 msgr2=0x7fa1280b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:23.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.597+0000 7fa143ffa640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa1280b6560 0x7fa1280b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa138031440 tx=0x7fa138009560 comp rx=0 tx=0).stop 2024-09-21T06:13:23.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.597+0000 7fa143ffa640 1 -- 172.21.15.3:0/13570222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c109df0 msgr2=0x7fa13c19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:23.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.597+0000 7fa143ffa640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c109df0 0x7fa13c19fce0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa13400eaa0 tx=0x7fa13400ef60 comp rx=0 tx=0).stop 2024-09-21T06:13:23.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.597+0000 7fa143ffa640 1 -- 172.21.15.3:0/13570222 shutdown_connections 2024-09-21T06:13:23.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.597+0000 7fa143ffa640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa1280b6560 0x7fa1280b8a70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:23.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.597+0000 7fa143ffa640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa13c1094a0 0x7fa13c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:23.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.597+0000 7fa143ffa640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c109df0 0x7fa13c19fce0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:23.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.597+0000 7fa143ffa640 1 --2- 172.21.15.3:0/13570222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa13c103b60 0x7fa13c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:23.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.597+0000 7fa143ffa640 1 -- 172.21.15.3:0/13570222 >> 172.21.15.3:0/13570222 conn(0x7fa13c07b850 msgr2=0x7fa13c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:23.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.597+0000 7fa143ffa640 1 -- 172.21.15.3:0/13570222 shutdown_connections 2024-09-21T06:13:23.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:23.597+0000 7fa143ffa640 1 -- 172.21.15.3:0/13570222 wait complete. 2024-09-21T06:13:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] up:active 2024-09-21T06:13:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T06:13:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: Cluster is now healthy 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/13570222' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]} : dispatch 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]} : dispatch 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]} : dispatch 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]} : dispatch 2024-09-21T06:13:23.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]} : dispatch 2024-09-21T06:13:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] up:active 2024-09-21T06:13:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T06:13:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T06:13:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: Cluster is now healthy 2024-09-21T06:13:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-09-21T06:13:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T06:13:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-21T06:13:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/13570222' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:13:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-21T06:13:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-09-21T06:13:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-09-21T06:13:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-21T06:13:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]} : dispatch 2024-09-21T06:13:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]} : dispatch 2024-09-21T06:13:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]} : dispatch 2024-09-21T06:13:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]} : dispatch 2024-09-21T06:13:23.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]} : dispatch 2024-09-21T06:13:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] up:active 2024-09-21T06:13:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T06:13:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T06:13:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: Cluster is now healthy 2024-09-21T06:13:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-09-21T06:13:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T06:13:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-21T06:13:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/13570222' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:13:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-21T06:13:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-09-21T06:13:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-09-21T06:13:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-21T06:13:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]} : dispatch 2024-09-21T06:13:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]} : dispatch 2024-09-21T06:13:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]} : dispatch 2024-09-21T06:13:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]} : dispatch 2024-09-21T06:13:23.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]} : dispatch 2024-09-21T06:13:24.513 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-21T06:13:24.514 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T06:13:24.786 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:24.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-21T06:13:24.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-21T06:13:24.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-21T06:13:24.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-21T06:13:24.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]': finished 2024-09-21T06:13:24.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-21T06:13:24.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-09-21T06:13:24.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]': finished 2024-09-21T06:13:24.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-21T06:13:24.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:13:24.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/13570222' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T06:13:24.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] up:active 2024-09-21T06:13:24.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T06:13:24.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:13:24.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:24 smithi003 ceph-mon[31227]: pgmap v276: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s wr, 10 op/s 2024-09-21T06:13:24.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-21T06:13:24.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-21T06:13:24.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-21T06:13:24.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-21T06:13:24.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]': finished 2024-09-21T06:13:24.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-21T06:13:24.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-09-21T06:13:24.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]': finished 2024-09-21T06:13:24.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-21T06:13:24.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:13:24.913 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/13570222' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T06:13:24.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] up:active 2024-09-21T06:13:24.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T06:13:24.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:13:24.914 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:24 smithi137 ceph-mon[34464]: pgmap v276: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s wr, 10 op/s 2024-09-21T06:13:24.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-21T06:13:24.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-21T06:13:24.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-21T06:13:24.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-21T06:13:24.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]': finished 2024-09-21T06:13:24.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-21T06:13:24.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-09-21T06:13:24.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]': finished 2024-09-21T06:13:24.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-21T06:13:24.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:13:24.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/13570222' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T06:13:24.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] up:active 2024-09-21T06:13:24.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T06:13:24.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:13:24.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:24 smithi204 ceph-mon[34659]: pgmap v276: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s wr, 10 op/s 2024-09-21T06:13:25.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.289+0000 7f981157d640 1 -- 172.21.15.3:0/3636199413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f980c103b60 msgr2=0x7f980c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.289+0000 7f981157d640 1 --2- 172.21.15.3:0/3636199413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f980c103b60 0x7f980c103f70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009900 tx=0x7f97fc02a410 comp rx=0 tx=0).stop 2024-09-21T06:13:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.289+0000 7f981157d640 1 -- 172.21.15.3:0/3636199413 shutdown_connections 2024-09-21T06:13:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.289+0000 7f981157d640 1 --2- 172.21.15.3:0/3636199413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f980c1094a0 0x7f980c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.289+0000 7f981157d640 1 --2- 172.21.15.3:0/3636199413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f980c109df0 0x7f980c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.289+0000 7f981157d640 1 --2- 172.21.15.3:0/3636199413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f980c103b60 0x7f980c103f70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.289+0000 7f981157d640 1 -- 172.21.15.3:0/3636199413 >> 172.21.15.3:0/3636199413 conn(0x7f980c07b850 msgr2=0x7f980c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.289+0000 7f981157d640 1 -- 172.21.15.3:0/3636199413 shutdown_connections 2024-09-21T06:13:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.289+0000 7f981157d640 1 -- 172.21.15.3:0/3636199413 wait complete. 2024-09-21T06:13:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.289+0000 7f981157d640 1 Processor -- start 2024-09-21T06:13:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f981157d640 1 -- start start 2024-09-21T06:13:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f981157d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f980c103b60 0x7f980c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f981157d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f980c1094a0 0x7f980c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f981157d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f980c109df0 0x7f980c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f981157d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f980c113ab0 con 0x7f980c1094a0 2024-09-21T06:13:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f981157d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f980c113930 con 0x7f980c109df0 2024-09-21T06:13:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f981157d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f980c113c30 con 0x7f980c103b60 2024-09-21T06:13:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f980a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f980c1094a0 0x7f980c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f980a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f980c1094a0 0x7f980c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44946/0 (socket says 172.21.15.3:44946) 2024-09-21T06:13:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f980a7fc640 1 -- 172.21.15.3:0/4033461531 learned_addr learned my addr 172.21.15.3:0/4033461531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f980affd640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f980c103b60 0x7f980c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f980b7fe640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f980c109df0 0x7f980c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f980a7fc640 1 -- 172.21.15.3:0/4033461531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f980c103b60 msgr2=0x7f980c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f980a7fc640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f980c103b60 0x7f980c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f980a7fc640 1 -- 172.21.15.3:0/4033461531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f980c109df0 msgr2=0x7f980c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f980a7fc640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f980c109df0 0x7f980c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f980a7fc640 1 -- 172.21.15.3:0/4033461531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c1a87e0 con 0x7f980c1094a0 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.290+0000 7f980b7fe640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f980c109df0 0x7f980c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.291+0000 7f980a7fc640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f980c1094a0 0x7f980c075c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f980000ea20 tx=0x7f980000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.291+0000 7f97f3fff640 1 -- 172.21.15.3:0/4033461531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980000cde0 con 0x7f980c1094a0 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.291+0000 7f981157d640 1 -- 172.21.15.3:0/4033461531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980c1a8ad0 con 0x7f980c1094a0 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.291+0000 7f97f3fff640 1 -- 172.21.15.3:0/4033461531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9800005020 con 0x7f980c1094a0 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.291+0000 7f97f3fff640 1 -- 172.21.15.3:0/4033461531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98000116d0 con 0x7f980c1094a0 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.291+0000 7f981157d640 1 -- 172.21.15.3:0/4033461531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f980c1a8fc0 con 0x7f980c1094a0 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.292+0000 7f981157d640 1 -- 172.21.15.3:0/4033461531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f980c103f70 con 0x7f980c1094a0 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.293+0000 7f97f3fff640 1 -- 172.21.15.3:0/4033461531 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 21) ==== 154528+0+0 (secure 0 0 0) 0x7f98000051d0 con 0x7f980c1094a0 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.294+0000 7f97f3fff640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f97f40b6560 0x7f97f40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.294+0000 7f980affd640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f97f40b6560 0x7f97f40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.294+0000 7f97f3fff640 1 -- 172.21.15.3:0/4033461531 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f98000e7b00 con 0x7f980c1094a0 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.295+0000 7f980affd640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f97f40b6560 0x7f97f40b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f97fc000c00 tx=0x7f97fc031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:25.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.297+0000 7f97f3fff640 1 -- 172.21.15.3:0/4033461531 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9800016030 con 0x7f980c1094a0 2024-09-21T06:13:25.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.509+0000 7f981157d640 1 -- 172.21.15.3:0/4033461531 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f980c063800 con 0x7f97f40b6560 2024-09-21T06:13:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.544+0000 7f97f3fff640 1 -- 172.21.15.3:0/4033461531 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f980c063800 con 0x7f97f40b6560 2024-09-21T06:13:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.546+0000 7f981157d640 1 -- 172.21.15.3:0/4033461531 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f97f40b6560 msgr2=0x7f97f40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.546+0000 7f981157d640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f97f40b6560 0x7f97f40b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f97fc000c00 tx=0x7f97fc031400 comp rx=0 tx=0).stop 2024-09-21T06:13:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.546+0000 7f981157d640 1 -- 172.21.15.3:0/4033461531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f980c1094a0 msgr2=0x7f980c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.546+0000 7f981157d640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f980c1094a0 0x7f980c075c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f980000ea20 tx=0x7f980000eee0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.546+0000 7f981157d640 1 -- 172.21.15.3:0/4033461531 shutdown_connections 2024-09-21T06:13:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.546+0000 7f981157d640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f97f40b6560 0x7f97f40b8a70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.546+0000 7f981157d640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f980c109df0 0x7f980c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.546+0000 7f981157d640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f980c1094a0 0x7f980c075c50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.546+0000 7f981157d640 1 --2- 172.21.15.3:0/4033461531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f980c103b60 0x7f980c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.546+0000 7f981157d640 1 -- 172.21.15.3:0/4033461531 >> 172.21.15.3:0/4033461531 conn(0x7f980c07b850 msgr2=0x7f980c1075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.546+0000 7f981157d640 1 -- 172.21.15.3:0/4033461531 shutdown_connections 2024-09-21T06:13:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:25.546+0000 7f981157d640 1 -- 172.21.15.3:0/4033461531 wait complete. 2024-09-21T06:13:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:25 smithi003 ceph-mon[31227]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:13:25.840 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:13:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:13:25.515+0000 7fc4decf8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:25.840 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:13:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:13:25.515+0000 7fc4decf8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:25.840 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:13:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:13:25.515+0000 7fc4decf8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:25.840 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:13:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:13:25.515+0000 7fc4decf8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:25.840 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:13:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:13:25.515+0000 7fc4decf8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:13:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:25 smithi137 ceph-mon[34464]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:13:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:25 smithi204 ceph-mon[34659]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:13:26.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T06:13:26.607 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:13:26 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b[63057]: 2024-09-21T06:13:26.274+0000 7f365ef26640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-21T06:13:26.607 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:13:26 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b[63057]: 2024-09-21T06:13:26.274+0000 7f365ef26640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-21T06:13:26.760 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:26.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:26 smithi003 ceph-mon[31227]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:26.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:26 smithi003 ceph-mon[31227]: pgmap v278: 97 pgs: 3 peering, 94 active+clean; 584 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 5.4 KiB/s wr, 16 op/s 2024-09-21T06:13:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:26 smithi137 ceph-mon[34464]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:26 smithi137 ceph-mon[34464]: pgmap v278: 97 pgs: 3 peering, 94 active+clean; 584 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 5.4 KiB/s wr, 16 op/s 2024-09-21T06:13:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:26 smithi204 ceph-mon[34659]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:26 smithi204 ceph-mon[34659]: pgmap v278: 97 pgs: 3 peering, 94 active+clean; 584 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 5.4 KiB/s wr, 16 op/s 2024-09-21T06:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.250+0000 7f90e63ef640 1 -- 172.21.15.3:0/3091971170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e01049b0 msgr2=0x7f90e0104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.250+0000 7f90e63ef640 1 --2- 172.21.15.3:0/3091971170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e01049b0 0x7f90e0104dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009900 tx=0x7f90d002a410 comp rx=0 tx=0).stop 2024-09-21T06:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.250+0000 7f90e63ef640 1 -- 172.21.15.3:0/3091971170 shutdown_connections 2024-09-21T06:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.250+0000 7f90e63ef640 1 --2- 172.21.15.3:0/3091971170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0105300 0x7f90e010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.250+0000 7f90e63ef640 1 --2- 172.21.15.3:0/3091971170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e010de10 0x7f90e0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.250+0000 7f90e63ef640 1 --2- 172.21.15.3:0/3091971170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e01049b0 0x7f90e0104dc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.250+0000 7f90e63ef640 1 -- 172.21.15.3:0/3091971170 >> 172.21.15.3:0/3091971170 conn(0x7f90e0079730 msgr2=0x7f90e0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.250+0000 7f90e63ef640 1 -- 172.21.15.3:0/3091971170 shutdown_connections 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.250+0000 7f90e63ef640 1 -- 172.21.15.3:0/3091971170 wait complete. 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.251+0000 7f90e63ef640 1 Processor -- start 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.251+0000 7f90e63ef640 1 -- start start 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.251+0000 7f90e63ef640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e01049b0 0x7f90e01026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.251+0000 7f90e63ef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e0105300 0x7f90e0102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.251+0000 7f90e63ef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e010de10 0x7f90e0103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.251+0000 7f90e63ef640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0113a90 con 0x7f90e01049b0 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.251+0000 7f90e63ef640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0113910 con 0x7f90e0105300 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.251+0000 7f90e63ef640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0113c10 con 0x7f90e010de10 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.251+0000 7f90dffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e01049b0 0x7f90e01026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.251+0000 7f90dffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e01049b0 0x7f90e01026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44990/0 (socket says 172.21.15.3:44990) 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.251+0000 7f90dffff640 1 -- 172.21.15.3:0/1699784898 learned_addr learned my addr 172.21.15.3:0/1699784898 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90d7fff640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e0105300 0x7f90e0102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90e498c640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e010de10 0x7f90e0103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90dffff640 1 -- 172.21.15.3:0/1699784898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e010de10 msgr2=0x7f90e0103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90dffff640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e010de10 0x7f90e0103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90dffff640 1 -- 172.21.15.3:0/1699784898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e0105300 msgr2=0x7f90e0102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90dffff640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e0105300 0x7f90e0102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90dffff640 1 -- 172.21.15.3:0/1699784898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e01a8820 con 0x7f90e01049b0 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90e498c640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e010de10 0x7f90e0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90dffff640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e01049b0 0x7f90e01026f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f90d0032fd0 tx=0x7f90d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90ddffb640 1 -- 172.21.15.3:0/1699784898 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d003f030 con 0x7f90e01049b0 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90ddffb640 1 -- 172.21.15.3:0/1699784898 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d0004490 con 0x7f90e01049b0 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90e63ef640 1 -- 172.21.15.3:0/1699784898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e01a8ab0 con 0x7f90e01049b0 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90ddffb640 1 -- 172.21.15.3:0/1699784898 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0035070 con 0x7f90e01049b0 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.252+0000 7f90e63ef640 1 -- 172.21.15.3:0/1699784898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e01a8f90 con 0x7f90e01049b0 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.253+0000 7f90e63ef640 1 -- 172.21.15.3:0/1699784898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e0104dc0 con 0x7f90e01049b0 2024-09-21T06:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.255+0000 7f90ddffb640 1 -- 172.21.15.3:0/1699784898 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f90d0042030 con 0x7f90e01049b0 2024-09-21T06:13:27.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.256+0000 7f90ddffb640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f90cc0b65f0 0x7f90cc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:27.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.256+0000 7f90ddffb640 1 -- 172.21.15.3:0/1699784898 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f90d0107ed0 con 0x7f90e01049b0 2024-09-21T06:13:27.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.256+0000 7f90d7fff640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f90cc0b65f0 0x7f90cc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:27.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.256+0000 7f90d7fff640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f90cc0b65f0 0x7f90cc0b8b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f90e007a000 tx=0x7f90c800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:27.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.259+0000 7f90ddffb640 1 -- 172.21.15.3:0/1699784898 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f90d00ce520 con 0x7f90e01049b0 2024-09-21T06:13:27.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.481+0000 7f90e63ef640 1 -- 172.21.15.3:0/1699784898 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f90e0063800 con 0x7f90cc0b65f0 2024-09-21T06:13:27.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.510+0000 7f90ddffb640 1 -- 172.21.15.3:0/1699784898 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f90e0063800 con 0x7f90cc0b65f0 2024-09-21T06:13:27.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.512+0000 7f90e63ef640 1 -- 172.21.15.3:0/1699784898 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f90cc0b65f0 msgr2=0x7f90cc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:27.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.512+0000 7f90e63ef640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f90cc0b65f0 0x7f90cc0b8b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f90e007a000 tx=0x7f90c800b390 comp rx=0 tx=0).stop 2024-09-21T06:13:27.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.512+0000 7f90e63ef640 1 -- 172.21.15.3:0/1699784898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e01049b0 msgr2=0x7f90e01026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:27.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.512+0000 7f90e63ef640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e01049b0 0x7f90e01026f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f90d0032fd0 tx=0x7f90d0004270 comp rx=0 tx=0).stop 2024-09-21T06:13:27.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.512+0000 7f90e63ef640 1 -- 172.21.15.3:0/1699784898 shutdown_connections 2024-09-21T06:13:27.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.512+0000 7f90e63ef640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f90cc0b65f0 0x7f90cc0b8b00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:27.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.512+0000 7f90e63ef640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e010de10 0x7f90e0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:27.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.512+0000 7f90e63ef640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e0105300 0x7f90e0102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:27.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.512+0000 7f90e63ef640 1 --2- 172.21.15.3:0/1699784898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e01049b0 0x7f90e01026f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:27.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.512+0000 7f90e63ef640 1 -- 172.21.15.3:0/1699784898 >> 172.21.15.3:0/1699784898 conn(0x7f90e0079730 msgr2=0x7f90e00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:27.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.512+0000 7f90e63ef640 1 -- 172.21.15.3:0/1699784898 shutdown_connections 2024-09-21T06:13:27.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:27.512+0000 7f90e63ef640 1 -- 172.21.15.3:0/1699784898 wait complete. 2024-09-21T06:13:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:27 smithi003 ceph-mon[31227]: mgrmap e22: x(active, since 7m), standbys: z, y 2024-09-21T06:13:27.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:27 smithi137 ceph-mon[34464]: mgrmap e22: x(active, since 7m), standbys: z, y 2024-09-21T06:13:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:27 smithi204 ceph-mon[34659]: mgrmap e22: x(active, since 7m), standbys: z, y 2024-09-21T06:13:28.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T06:13:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:28 smithi137 ceph-mon[34464]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:28 smithi137 ceph-mon[34464]: pgmap v279: 97 pgs: 3 peering, 94 active+clean; 591 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 7.1 KiB/s wr, 25 op/s; 215 B/s, 1 objects/s recovering 2024-09-21T06:13:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:28 smithi204 ceph-mon[34659]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:28 smithi204 ceph-mon[34659]: pgmap v279: 97 pgs: 3 peering, 94 active+clean; 591 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 7.1 KiB/s wr, 25 op/s; 215 B/s, 1 objects/s recovering 2024-09-21T06:13:28.940 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:28 smithi003 ceph-mon[31227]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:28.940 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:28 smithi003 ceph-mon[31227]: pgmap v279: 97 pgs: 3 peering, 94 active+clean; 591 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 7.1 KiB/s wr, 25 op/s; 215 B/s, 1 objects/s recovering 2024-09-21T06:13:29.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.034+0000 7f34394ee640 1 -- 172.21.15.3:0/977330794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34341049d0 msgr2=0x7f3434104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.034+0000 7f34394ee640 1 --2- 172.21.15.3:0/977330794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34341049d0 0x7f3434104de0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3424009900 tx=0x7f342402a410 comp rx=0 tx=0).stop 2024-09-21T06:13:29.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.034+0000 7f34394ee640 1 -- 172.21.15.3:0/977330794 shutdown_connections 2024-09-21T06:13:29.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.034+0000 7f34394ee640 1 --2- 172.21.15.3:0/977330794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3434105320 0x7f343410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.034+0000 7f34394ee640 1 --2- 172.21.15.3:0/977330794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f343410de30 0x7f3434110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.034+0000 7f34394ee640 1 --2- 172.21.15.3:0/977330794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34341049d0 0x7f3434104de0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.034+0000 7f34394ee640 1 -- 172.21.15.3:0/977330794 >> 172.21.15.3:0/977330794 conn(0x7f3434079740 msgr2=0x7f3434079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.035+0000 7f34394ee640 1 -- 172.21.15.3:0/977330794 shutdown_connections 2024-09-21T06:13:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.035+0000 7f34394ee640 1 -- 172.21.15.3:0/977330794 wait complete. 2024-09-21T06:13:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.035+0000 7f34394ee640 1 Processor -- start 2024-09-21T06:13:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.035+0000 7f34394ee640 1 -- start start 2024-09-21T06:13:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f34394ee640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34341049d0 0x7f3434102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f34394ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3434105320 0x7f3434102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f34394ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f343410de30 0x7f3434103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f34394ee640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3434113a90 con 0x7f3434105320 2024-09-21T06:13:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f34394ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3434113910 con 0x7f343410de30 2024-09-21T06:13:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f34394ee640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3434113c10 con 0x7f34341049d0 2024-09-21T06:13:29.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f342bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3434105320 0x7f3434102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f342bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3434105320 0x7f3434102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45006/0 (socket says 172.21.15.3:45006) 2024-09-21T06:13:29.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f342bfff640 1 -- 172.21.15.3:0/1781443947 learned_addr learned my addr 172.21.15.3:0/1781443947 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:29.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f3432ffd640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34341049d0 0x7f3434102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f34337fe640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f343410de30 0x7f3434103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f342bfff640 1 -- 172.21.15.3:0/1781443947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34341049d0 msgr2=0x7f3434102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f342bfff640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34341049d0 0x7f3434102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f342bfff640 1 -- 172.21.15.3:0/1781443947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f343410de30 msgr2=0x7f3434103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f342bfff640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f343410de30 0x7f3434103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f342bfff640 1 -- 172.21.15.3:0/1781443947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34341a87c0 con 0x7f3434105320 2024-09-21T06:13:29.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f342bfff640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3434105320 0x7f3434102c50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f341c00c970 tx=0x7f341c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:29.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f3430ff9640 1 -- 172.21.15.3:0/1781443947 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c007bb0 con 0x7f3434105320 2024-09-21T06:13:29.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f3430ff9640 1 -- 172.21.15.3:0/1781443947 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f341c007d50 con 0x7f3434105320 2024-09-21T06:13:29.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f34394ee640 1 -- 172.21.15.3:0/1781443947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34341a8ab0 con 0x7f3434105320 2024-09-21T06:13:29.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f3430ff9640 1 -- 172.21.15.3:0/1781443947 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c00f450 con 0x7f3434105320 2024-09-21T06:13:29.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.036+0000 7f34394ee640 1 -- 172.21.15.3:0/1781443947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34341a8ff0 con 0x7f3434105320 2024-09-21T06:13:29.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.038+0000 7f34394ee640 1 -- 172.21.15.3:0/1781443947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3434104de0 con 0x7f3434105320 2024-09-21T06:13:29.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.039+0000 7f3430ff9640 1 -- 172.21.15.3:0/1781443947 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f341c004eb0 con 0x7f3434105320 2024-09-21T06:13:29.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.040+0000 7f3430ff9640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f34200b65f0 0x7f34200b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:29.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.040+0000 7f3432ffd640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f34200b65f0 0x7f34200b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:29.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.040+0000 7f3430ff9640 1 -- 172.21.15.3:0/1781443947 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f341c0e73e0 con 0x7f3434105320 2024-09-21T06:13:29.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.041+0000 7f3432ffd640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f34200b65f0 0x7f34200b8b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3424009230 tx=0x7f3424031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:29.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.044+0000 7f3430ff9640 1 -- 172.21.15.3:0/1781443947 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f341c014030 con 0x7f3434105320 2024-09-21T06:13:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.264+0000 7f34394ee640 1 -- 172.21.15.3:0/1781443947 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f3434108f90 con 0x7f34200b65f0 2024-09-21T06:13:29.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.271+0000 7f3430ff9640 1 -- 172.21.15.3:0/1781443947 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3434108f90 con 0x7f34200b65f0 2024-09-21T06:13:29.273 INFO:teuthology.orchestra.run.smithi003.stdout:/volumes/qa/sv_0/8a2de1e1-1e6f-48a0-b56f-892c6caa39ae 2024-09-21T06:13:29.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.273+0000 7f34394ee640 1 -- 172.21.15.3:0/1781443947 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f34200b65f0 msgr2=0x7f34200b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.273+0000 7f34394ee640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f34200b65f0 0x7f34200b8b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3424009230 tx=0x7f3424031400 comp rx=0 tx=0).stop 2024-09-21T06:13:29.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.273+0000 7f34394ee640 1 -- 172.21.15.3:0/1781443947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3434105320 msgr2=0x7f3434102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:29.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.273+0000 7f34394ee640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3434105320 0x7f3434102c50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f341c00c970 tx=0x7f341c00ce30 comp rx=0 tx=0).stop 2024-09-21T06:13:29.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.273+0000 7f34394ee640 1 -- 172.21.15.3:0/1781443947 shutdown_connections 2024-09-21T06:13:29.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.273+0000 7f34394ee640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f34200b65f0 0x7f34200b8b00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.273+0000 7f34394ee640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f343410de30 0x7f3434103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.273+0000 7f34394ee640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3434105320 0x7f3434102c50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.273+0000 7f34394ee640 1 --2- 172.21.15.3:0/1781443947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34341049d0 0x7f3434102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:29.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.274+0000 7f34394ee640 1 -- 172.21.15.3:0/1781443947 >> 172.21.15.3:0/1781443947 conn(0x7f3434079740 msgr2=0x7f3434100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:29.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.274+0000 7f34394ee640 1 -- 172.21.15.3:0/1781443947 shutdown_connections 2024-09-21T06:13:29.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:29.274+0000 7f34394ee640 1 -- 172.21.15.3:0/1781443947 wait complete. 2024-09-21T06:13:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:29 smithi137 ceph-mon[34464]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:29 smithi204 ceph-mon[34659]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:29.954 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:29 smithi003 ceph-mon[31227]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:29.987 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T06:13:30.249 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.729+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/1021874583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf74102ae0 msgr2=0x7fcf74102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.729+0000 7fcf7c6b9640 1 --2- 172.21.15.3:0/1021874583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf74102ae0 0x7fcf74102ef0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009900 tx=0x7fcf7002a410 comp rx=0 tx=0).stop 2024-09-21T06:13:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.729+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/1021874583 shutdown_connections 2024-09-21T06:13:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.729+0000 7fcf7c6b9640 1 --2- 172.21.15.3:0/1021874583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf741083d0 0x7fcf741087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.729+0000 7fcf7c6b9640 1 --2- 172.21.15.3:0/1021874583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf74108d20 0x7fcf7410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.729+0000 7fcf7c6b9640 1 --2- 172.21.15.3:0/1021874583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf74102ae0 0x7fcf74102ef0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.729+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/1021874583 >> 172.21.15.3:0/1021874583 conn(0x7fcf74079760 msgr2=0x7fcf74079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.729+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/1021874583 shutdown_connections 2024-09-21T06:13:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.729+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/1021874583 wait complete. 2024-09-21T06:13:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.730+0000 7fcf7c6b9640 1 Processor -- start 2024-09-21T06:13:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.730+0000 7fcf7c6b9640 1 -- start start 2024-09-21T06:13:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.730+0000 7fcf7c6b9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf74102ae0 0x7fcf7419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.730+0000 7fcf7c6b9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf741083d0 0x7fcf7419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.730+0000 7fcf7c6b9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf74108d20 0x7fcf7419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.730+0000 7fcf7c6b9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74112990 con 0x7fcf74102ae0 2024-09-21T06:13:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.730+0000 7fcf7c6b9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74112810 con 0x7fcf74108d20 2024-09-21T06:13:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.730+0000 7fcf7c6b9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74112b10 con 0x7fcf741083d0 2024-09-21T06:13:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.730+0000 7fcf7a455640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf74102ae0 0x7fcf7419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.730+0000 7fcf7a455640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf74102ae0 0x7fcf7419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45028/0 (socket says 172.21.15.3:45028) 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.730+0000 7fcf7a455640 1 -- 172.21.15.3:0/4240371832 learned_addr learned my addr 172.21.15.3:0/4240371832 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.730+0000 7fcf79c54640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf741083d0 0x7fcf7419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.730+0000 7fcf7ac56640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf74108d20 0x7fcf7419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.731+0000 7fcf7a455640 1 -- 172.21.15.3:0/4240371832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf741083d0 msgr2=0x7fcf7419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.731+0000 7fcf7a455640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf741083d0 0x7fcf7419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.731+0000 7fcf7a455640 1 -- 172.21.15.3:0/4240371832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf74108d20 msgr2=0x7fcf7419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.731+0000 7fcf7a455640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf74108d20 0x7fcf7419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.731+0000 7fcf7a455640 1 -- 172.21.15.3:0/4240371832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf741a0400 con 0x7fcf74102ae0 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.731+0000 7fcf79c54640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf741083d0 0x7fcf7419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.731+0000 7fcf7ac56640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf74108d20 0x7fcf7419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.731+0000 7fcf7a455640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf74102ae0 0x7fcf7419a490 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcf70006010 tx=0x7fcf700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.731+0000 7fcf6b7fe640 1 -- 172.21.15.3:0/4240371832 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7003f030 con 0x7fcf74102ae0 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.731+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/4240371832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf741a0690 con 0x7fcf74102ae0 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.731+0000 7fcf6b7fe640 1 -- 172.21.15.3:0/4240371832 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf70003800 con 0x7fcf74102ae0 2024-09-21T06:13:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.731+0000 7fcf6b7fe640 1 -- 172.21.15.3:0/4240371832 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70035070 con 0x7fcf74102ae0 2024-09-21T06:13:30.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.731+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/4240371832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf74104160 con 0x7fcf74102ae0 2024-09-21T06:13:30.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.732+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/4240371832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf74103d10 con 0x7fcf74102ae0 2024-09-21T06:13:30.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.734+0000 7fcf6b7fe640 1 -- 172.21.15.3:0/4240371832 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fcf70003cb0 con 0x7fcf74102ae0 2024-09-21T06:13:30.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.734+0000 7fcf6b7fe640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcf600b65f0 0x7fcf600b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:30.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.734+0000 7fcf79c54640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcf600b65f0 0x7fcf600b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:30.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.735+0000 7fcf6b7fe640 1 -- 172.21.15.3:0/4240371832 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcf701082b0 con 0x7fcf74102ae0 2024-09-21T06:13:30.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.735+0000 7fcf79c54640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcf600b65f0 0x7fcf600b8b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcf74079ca0 tx=0x7fcf64005ec0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:30.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.738+0000 7fcf6b7fe640 1 -- 172.21.15.3:0/4240371832 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcf700ce900 con 0x7fcf74102ae0 2024-09-21T06:13:30.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:30 smithi137 ceph-mon[34464]: pgmap v280: 97 pgs: 97 active+clean; 592 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.9 KiB/s wr, 23 op/s; 215 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T06:13:30.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:30 smithi204 ceph-mon[34659]: pgmap v280: 97 pgs: 97 active+clean; 592 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.9 KiB/s wr, 23 op/s; 215 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T06:13:30.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.954+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/4240371832 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcf74063800 con 0x7fcf600b65f0 2024-09-21T06:13:30.956 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:30 smithi003 ceph-mon[31227]: pgmap v280: 97 pgs: 97 active+clean; 592 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.9 KiB/s wr, 23 op/s; 215 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T06:13:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.981+0000 7fcf6b7fe640 1 -- 172.21.15.3:0/4240371832 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcf74063800 con 0x7fcf600b65f0 2024-09-21T06:13:30.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.983+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/4240371832 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcf600b65f0 msgr2=0x7fcf600b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.983+0000 7fcf7c6b9640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcf600b65f0 0x7fcf600b8b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcf74079ca0 tx=0x7fcf64005ec0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.983+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/4240371832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf74102ae0 msgr2=0x7fcf7419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:30.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.983+0000 7fcf7c6b9640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf74102ae0 0x7fcf7419a490 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcf70006010 tx=0x7fcf700042f0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.983+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/4240371832 shutdown_connections 2024-09-21T06:13:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.983+0000 7fcf7c6b9640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcf600b65f0 0x7fcf600b8b00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.983+0000 7fcf7c6b9640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf741083d0 0x7fcf7419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.983+0000 7fcf7c6b9640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf74108d20 0x7fcf7419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.983+0000 7fcf7c6b9640 1 --2- 172.21.15.3:0/4240371832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf74102ae0 0x7fcf7419a490 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.983+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/4240371832 >> 172.21.15.3:0/4240371832 conn(0x7fcf74079760 msgr2=0x7fcf74101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.983+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/4240371832 shutdown_connections 2024-09-21T06:13:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:30.983+0000 7fcf7c6b9640 1 -- 172.21.15.3:0/4240371832 wait complete. 2024-09-21T06:13:31.695 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T06:13:31.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:31 smithi137 ceph-mon[34464]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:31.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:31 smithi204 ceph-mon[34659]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:31.956 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:32.077 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:31 smithi003 ceph-mon[31227]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.455+0000 7f8682577640 1 -- 172.21.15.3:0/1606496516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c103b60 msgr2=0x7f867c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.455+0000 7f8682577640 1 --2- 172.21.15.3:0/1606496516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c103b60 0x7f867c103f70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f866c009900 tx=0x7f866c02a410 comp rx=0 tx=0).stop 2024-09-21T06:13:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.455+0000 7f8682577640 1 -- 172.21.15.3:0/1606496516 shutdown_connections 2024-09-21T06:13:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.455+0000 7f8682577640 1 --2- 172.21.15.3:0/1606496516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f867c109df0 0x7f867c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.455+0000 7f8682577640 1 --2- 172.21.15.3:0/1606496516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f867c1094a0 0x7f867c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.455+0000 7f8682577640 1 --2- 172.21.15.3:0/1606496516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c103b60 0x7f867c103f70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.455+0000 7f8682577640 1 -- 172.21.15.3:0/1606496516 >> 172.21.15.3:0/1606496516 conn(0x7f867c07b850 msgr2=0x7f867c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.455+0000 7f8682577640 1 -- 172.21.15.3:0/1606496516 shutdown_connections 2024-09-21T06:13:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.455+0000 7f8682577640 1 -- 172.21.15.3:0/1606496516 wait complete. 2024-09-21T06:13:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f8682577640 1 Processor -- start 2024-09-21T06:13:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f8682577640 1 -- start start 2024-09-21T06:13:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f8682577640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c103b60 0x7f867c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f8682577640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f867c1094a0 0x7f867c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f8682577640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f867c109df0 0x7f867c19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f8682577640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f867c113b70 con 0x7f867c103b60 2024-09-21T06:13:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f8682577640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f867c1139f0 con 0x7f867c109df0 2024-09-21T06:13:32.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f8682577640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f867c113cf0 con 0x7f867c1094a0 2024-09-21T06:13:32.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f867bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c103b60 0x7f867c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:32.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f867bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c103b60 0x7f867c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45044/0 (socket says 172.21.15.3:45044) 2024-09-21T06:13:32.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f867bfff640 1 -- 172.21.15.3:0/3317843529 learned_addr learned my addr 172.21.15.3:0/3317843529 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:32.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f867bfff640 1 -- 172.21.15.3:0/3317843529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f867c1094a0 msgr2=0x7f867c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:32.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f867bfff640 1 --2- 172.21.15.3:0/3317843529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f867c1094a0 0x7f867c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f867bfff640 1 -- 172.21.15.3:0/3317843529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f867c109df0 msgr2=0x7f867c19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f867bfff640 1 --2- 172.21.15.3:0/3317843529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f867c109df0 0x7f867c19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.456+0000 7f867bfff640 1 -- 172.21.15.3:0/3317843529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c1a0530 con 0x7f867c103b60 2024-09-21T06:13:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.457+0000 7f867bfff640 1 --2- 172.21.15.3:0/3317843529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c103b60 0x7f867c19a5c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f866c006010 tx=0x7f866c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.457+0000 7f8679ffb640 1 -- 172.21.15.3:0/3317843529 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c03f030 con 0x7f867c103b60 2024-09-21T06:13:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.457+0000 7f8679ffb640 1 -- 172.21.15.3:0/3317843529 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f866c003800 con 0x7f867c103b60 2024-09-21T06:13:32.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.457+0000 7f8682577640 1 -- 172.21.15.3:0/3317843529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867c1a07c0 con 0x7f867c103b60 2024-09-21T06:13:32.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.457+0000 7f8679ffb640 1 -- 172.21.15.3:0/3317843529 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c035070 con 0x7f867c103b60 2024-09-21T06:13:32.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.457+0000 7f8682577640 1 -- 172.21.15.3:0/3317843529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c1a6f80 con 0x7f867c103b60 2024-09-21T06:13:32.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.458+0000 7f8682577640 1 -- 172.21.15.3:0/3317843529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867c103ff0 con 0x7f867c103b60 2024-09-21T06:13:32.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.459+0000 7f8679ffb640 1 -- 172.21.15.3:0/3317843529 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f866c0039b0 con 0x7f867c103b60 2024-09-21T06:13:32.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.460+0000 7f8679ffb640 1 --2- 172.21.15.3:0/3317843529 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86680b65f0 0x7f86680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:32.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.460+0000 7f8673fff640 1 --2- 172.21.15.3:0/3317843529 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86680b65f0 0x7f86680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:32.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.460+0000 7f8679ffb640 1 -- 172.21.15.3:0/3317843529 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f866c108000 con 0x7f867c103b60 2024-09-21T06:13:32.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.461+0000 7f8673fff640 1 --2- 172.21.15.3:0/3317843529 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86680b65f0 0x7f86680b8b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f867c106450 tx=0x7f8664006c90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:32.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.465+0000 7f8679ffb640 1 -- 172.21.15.3:0/3317843529 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f866c0ce650 con 0x7f867c103b60 2024-09-21T06:13:32.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.689+0000 7f8682577640 1 -- 172.21.15.3:0/3317843529 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f867c063800 con 0x7f86680b65f0 2024-09-21T06:13:32.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.694+0000 7f8679ffb640 1 -- 172.21.15.3:0/3317843529 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f867c063800 con 0x7f86680b65f0 2024-09-21T06:13:32.696 INFO:teuthology.orchestra.run.smithi003.stdout:/volumes/qa/sv_1/dacd083d-59f5-4b54-9712-1121e8b11822 2024-09-21T06:13:32.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.696+0000 7f8682577640 1 -- 172.21.15.3:0/3317843529 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86680b65f0 msgr2=0x7f86680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:32.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.696+0000 7f8682577640 1 --2- 172.21.15.3:0/3317843529 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86680b65f0 0x7f86680b8b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f867c106450 tx=0x7f8664006c90 comp rx=0 tx=0).stop 2024-09-21T06:13:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.696+0000 7f8682577640 1 -- 172.21.15.3:0/3317843529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c103b60 msgr2=0x7f867c19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.696+0000 7f8682577640 1 --2- 172.21.15.3:0/3317843529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c103b60 0x7f867c19a5c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f866c006010 tx=0x7f866c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:13:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.696+0000 7f8682577640 1 -- 172.21.15.3:0/3317843529 shutdown_connections 2024-09-21T06:13:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.696+0000 7f8682577640 1 --2- 172.21.15.3:0/3317843529 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86680b65f0 0x7f86680b8b00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.696+0000 7f8682577640 1 --2- 172.21.15.3:0/3317843529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f867c1094a0 0x7f867c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.696+0000 7f8682577640 1 --2- 172.21.15.3:0/3317843529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f867c109df0 0x7f867c19fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.696+0000 7f8682577640 1 --2- 172.21.15.3:0/3317843529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f867c103b60 0x7f867c19a5c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.696+0000 7f8682577640 1 -- 172.21.15.3:0/3317843529 >> 172.21.15.3:0/3317843529 conn(0x7f867c07b850 msgr2=0x7f867c1075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.697+0000 7f8682577640 1 -- 172.21.15.3:0/3317843529 shutdown_connections 2024-09-21T06:13:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:32.697+0000 7f8682577640 1 -- 172.21.15.3:0/3317843529 wait complete. 2024-09-21T06:13:32.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:32 smithi137 ceph-mon[34464]: pgmap v281: 97 pgs: 97 active+clean; 629 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 26 op/s; 214 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T06:13:32.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:32 smithi204 ceph-mon[34659]: pgmap v281: 97 pgs: 97 active+clean; 629 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 26 op/s; 214 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T06:13:33.086 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:32 smithi003 ceph-mon[31227]: pgmap v281: 97 pgs: 97 active+clean; 629 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 26 op/s; 214 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T06:13:33.505 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:13:33.771 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:33.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:33 smithi003 ceph-mon[31227]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:33.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:33 smithi137 ceph-mon[34464]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:33 smithi204 ceph-mon[34659]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:13:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.267+0000 7fd2aa74f640 1 -- 172.21.15.3:0/4042675883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a41083a0 msgr2=0x7fd2a41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.267+0000 7fd2aa74f640 1 --2- 172.21.15.3:0/4042675883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a41083a0 0x7fd2a41087b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd298009900 tx=0x7fd29802a410 comp rx=0 tx=0).stop 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.267+0000 7fd2aa74f640 1 -- 172.21.15.3:0/4042675883 shutdown_connections 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.267+0000 7fd2aa74f640 1 --2- 172.21.15.3:0/4042675883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4108cf0 0x7fd2a410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.267+0000 7fd2aa74f640 1 --2- 172.21.15.3:0/4042675883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a41083a0 0x7fd2a41087b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.267+0000 7fd2aa74f640 1 --2- 172.21.15.3:0/4042675883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a4102a60 0x7fd2a4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.267+0000 7fd2aa74f640 1 -- 172.21.15.3:0/4042675883 >> 172.21.15.3:0/4042675883 conn(0x7fd2a4079730 msgr2=0x7fd2a4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.267+0000 7fd2aa74f640 1 -- 172.21.15.3:0/4042675883 shutdown_connections 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.267+0000 7fd2aa74f640 1 -- 172.21.15.3:0/4042675883 wait complete. 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.268+0000 7fd2aa74f640 1 Processor -- start 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.268+0000 7fd2aa74f640 1 -- start start 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.268+0000 7fd2aa74f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a4102a60 0x7fd2a4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.268+0000 7fd2aa74f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a41083a0 0x7fd2a41999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.268+0000 7fd2aa74f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4108cf0 0x7fd2a419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.268+0000 7fd2aa74f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a4112960 con 0x7fd2a4102a60 2024-09-21T06:13:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.268+0000 7fd2aa74f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a41127e0 con 0x7fd2a4108cf0 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.268+0000 7fd2aa74f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a4112ae0 con 0x7fd2a41083a0 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.268+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a4102a60 0x7fd2a4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.268+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a4102a60 0x7fd2a4199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45066/0 (socket says 172.21.15.3:45066) 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2a3fff640 1 -- 172.21.15.3:0/175844899 learned_addr learned my addr 172.21.15.3:0/175844899 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2a37fe640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a41083a0 0x7fd2a41999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2a8cec640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4108cf0 0x7fd2a419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2a3fff640 1 -- 172.21.15.3:0/175844899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a41083a0 msgr2=0x7fd2a41999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2a3fff640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a41083a0 0x7fd2a41999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2a3fff640 1 -- 172.21.15.3:0/175844899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4108cf0 msgr2=0x7fd2a419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2a3fff640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4108cf0 0x7fd2a419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2a3fff640 1 -- 172.21.15.3:0/175844899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a419f380 con 0x7fd2a4102a60 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2a3fff640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a4102a60 0x7fd2a4199410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd29400d770 tx=0x7fd29400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2a17fa640 1 -- 172.21.15.3:0/175844899 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294017070 con 0x7fd2a4102a60 2024-09-21T06:13:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2a17fa640 1 -- 172.21.15.3:0/175844899 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2940054d0 con 0x7fd2a4102a60 2024-09-21T06:13:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2aa74f640 1 -- 172.21.15.3:0/175844899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a419f670 con 0x7fd2a4102a60 2024-09-21T06:13:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2a17fa640 1 -- 172.21.15.3:0/175844899 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294004c30 con 0x7fd2a4102a60 2024-09-21T06:13:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.269+0000 7fd2aa74f640 1 -- 172.21.15.3:0/175844899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a406ae60 con 0x7fd2a4102a60 2024-09-21T06:13:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.270+0000 7fd2aa74f640 1 -- 172.21.15.3:0/175844899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a4103ce0 con 0x7fd2a4102a60 2024-09-21T06:13:34.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.272+0000 7fd2a17fa640 1 -- 172.21.15.3:0/175844899 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fd294004dd0 con 0x7fd2a4102a60 2024-09-21T06:13:34.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.273+0000 7fd2a17fa640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd28c0b6670 0x7fd28c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:34.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.273+0000 7fd2a37fe640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd28c0b6670 0x7fd28c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:34.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.273+0000 7fd2a17fa640 1 -- 172.21.15.3:0/175844899 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fd2940e73e0 con 0x7fd2a4102a60 2024-09-21T06:13:34.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.273+0000 7fd2a37fe640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd28c0b6670 0x7fd28c0b8b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd298000c00 tx=0x7fd298031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.276+0000 7fd2a17fa640 1 -- 172.21.15.3:0/175844899 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd2940b1e30 con 0x7fd2a4102a60 2024-09-21T06:13:34.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.501+0000 7fd2aa74f640 1 -- 172.21.15.3:0/175844899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2a4102e70 con 0x7fd2a4102a60 2024-09-21T06:13:34.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.502+0000 7fd2a17fa640 1 -- 172.21.15.3:0/175844899 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11962 (secure 0 0 0) 0x7fd2a4102e70 con 0x7fd2a4102a60 2024-09-21T06:13:34.504 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:13:34.504 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-09-21T06:13:23:600046+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:13:23.600022+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:13:34.504 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-09-21T06:13:34.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.504+0000 7fd2aa74f640 1 -- 172.21.15.3:0/175844899 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd28c0b6670 msgr2=0x7fd28c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:34.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.504+0000 7fd2aa74f640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd28c0b6670 0x7fd28c0b8b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd298000c00 tx=0x7fd298031400 comp rx=0 tx=0).stop 2024-09-21T06:13:34.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.504+0000 7fd2aa74f640 1 -- 172.21.15.3:0/175844899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a4102a60 msgr2=0x7fd2a4199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:34.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.504+0000 7fd2aa74f640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a4102a60 0x7fd2a4199410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd29400d770 tx=0x7fd29400dc30 comp rx=0 tx=0).stop 2024-09-21T06:13:34.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.504+0000 7fd2aa74f640 1 -- 172.21.15.3:0/175844899 shutdown_connections 2024-09-21T06:13:34.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.504+0000 7fd2aa74f640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd28c0b6670 0x7fd28c0b8b80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:34.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.504+0000 7fd2aa74f640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a41083a0 0x7fd2a41999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:34.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.504+0000 7fd2aa74f640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4108cf0 0x7fd2a419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:34.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.504+0000 7fd2aa74f640 1 --2- 172.21.15.3:0/175844899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a4102a60 0x7fd2a4199410 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:34.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.504+0000 7fd2aa74f640 1 -- 172.21.15.3:0/175844899 >> 172.21.15.3:0/175844899 conn(0x7fd2a4079730 msgr2=0x7fd2a41044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:34.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.504+0000 7fd2aa74f640 1 -- 172.21.15.3:0/175844899 shutdown_connections 2024-09-21T06:13:34.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:34.505+0000 7fd2aa74f640 1 -- 172.21.15.3:0/175844899 wait complete. 2024-09-21T06:13:34.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:34 smithi003 ceph-mon[31227]: pgmap v282: 97 pgs: 97 active+clean; 629 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.4 KiB/s wr, 22 op/s; 172 B/s, 0 keys/s, 2 objects/s recovering 2024-09-21T06:13:34.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:34 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/175844899' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:34.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:34 smithi137 ceph-mon[34464]: pgmap v282: 97 pgs: 97 active+clean; 629 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.4 KiB/s wr, 22 op/s; 172 B/s, 0 keys/s, 2 objects/s recovering 2024-09-21T06:13:34.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:34 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/175844899' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:34.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:34 smithi204 ceph-mon[34659]: pgmap v282: 97 pgs: 97 active+clean; 629 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.4 KiB/s wr, 22 op/s; 172 B/s, 0 keys/s, 2 objects/s recovering 2024-09-21T06:13:34.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:34 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/175844899' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:35.334 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:13:35.597 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:36.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.077+0000 7f6b1792a640 1 -- 172.21.15.3:0/1184899254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b10103b60 msgr2=0x7f6b10103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.077+0000 7f6b1792a640 1 --2- 172.21.15.3:0/1184899254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b10103b60 0x7f6b10103f70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009900 tx=0x7f6b0c02a410 comp rx=0 tx=0).stop 2024-09-21T06:13:36.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.078+0000 7f6b1792a640 1 -- 172.21.15.3:0/1184899254 shutdown_connections 2024-09-21T06:13:36.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.078+0000 7f6b1792a640 1 --2- 172.21.15.3:0/1184899254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b101094a0 0x7f6b101098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.078+0000 7f6b1792a640 1 --2- 172.21.15.3:0/1184899254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b10109df0 0x7f6b10110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.078+0000 7f6b1792a640 1 --2- 172.21.15.3:0/1184899254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b10103b60 0x7f6b10103f70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.078+0000 7f6b1792a640 1 -- 172.21.15.3:0/1184899254 >> 172.21.15.3:0/1184899254 conn(0x7f6b1007b850 msgr2=0x7f6b1007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:36.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.078+0000 7f6b1792a640 1 -- 172.21.15.3:0/1184899254 shutdown_connections 2024-09-21T06:13:36.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.078+0000 7f6b1792a640 1 -- 172.21.15.3:0/1184899254 wait complete. 2024-09-21T06:13:36.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.078+0000 7f6b1792a640 1 Processor -- start 2024-09-21T06:13:36.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.078+0000 7f6b1792a640 1 -- start start 2024-09-21T06:13:36.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b1792a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b10103b60 0x7f6b1019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:36.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b1792a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b101094a0 0x7f6b1019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:36.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b1792a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b10109df0 0x7f6b1019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:36.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b1792a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10113ab0 con 0x7f6b10103b60 2024-09-21T06:13:36.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b1792a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10113930 con 0x7f6b101094a0 2024-09-21T06:13:36.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b1792a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10113c30 con 0x7f6b10109df0 2024-09-21T06:13:36.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b156c6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b10103b60 0x7f6b1019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:36.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b156c6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b10103b60 0x7f6b1019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44068/0 (socket says 172.21.15.3:44068) 2024-09-21T06:13:36.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b156c6640 1 -- 172.21.15.3:0/214288219 learned_addr learned my addr 172.21.15.3:0/214288219 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:36.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b15ec7640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b10109df0 0x7f6b1019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:36.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b14ec5640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b101094a0 0x7f6b1019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:36.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b156c6640 1 -- 172.21.15.3:0/214288219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b10109df0 msgr2=0x7f6b1019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b156c6640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b10109df0 0x7f6b1019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b156c6640 1 -- 172.21.15.3:0/214288219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b101094a0 msgr2=0x7f6b1019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b156c6640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b101094a0 0x7f6b1019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b156c6640 1 -- 172.21.15.3:0/214288219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b101a0410 con 0x7f6b10103b60 2024-09-21T06:13:36.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.079+0000 7f6b156c6640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b10103b60 0x7f6b1019a4a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c005940 tx=0x7f6b0c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:36.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.080+0000 7f6b067fc640 1 -- 172.21.15.3:0/214288219 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c03f030 con 0x7f6b10103b60 2024-09-21T06:13:36.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.080+0000 7f6b1792a640 1 -- 172.21.15.3:0/214288219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b101a06a0 con 0x7f6b10103b60 2024-09-21T06:13:36.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.080+0000 7f6b067fc640 1 -- 172.21.15.3:0/214288219 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b0c003cb0 con 0x7f6b10103b60 2024-09-21T06:13:36.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.080+0000 7f6b067fc640 1 -- 172.21.15.3:0/214288219 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c035070 con 0x7f6b10103b60 2024-09-21T06:13:36.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.080+0000 7f6b1792a640 1 -- 172.21.15.3:0/214288219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b101a6e60 con 0x7f6b10103b60 2024-09-21T06:13:36.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.081+0000 7f6b1792a640 1 -- 172.21.15.3:0/214288219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b10104de0 con 0x7f6b10103b60 2024-09-21T06:13:36.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.083+0000 7f6b067fc640 1 -- 172.21.15.3:0/214288219 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f6b0c042030 con 0x7f6b10103b60 2024-09-21T06:13:36.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.084+0000 7f6b067fc640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6afc0b65f0 0x7f6afc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:36.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.084+0000 7f6b14ec5640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6afc0b65f0 0x7f6afc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:36.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.084+0000 7f6b067fc640 1 -- 172.21.15.3:0/214288219 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f6b0c108200 con 0x7f6b10103b60 2024-09-21T06:13:36.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.085+0000 7f6b14ec5640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6afc0b65f0 0x7f6afc0b8b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6b00005fd0 tx=0x7f6b00005e20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:36.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.088+0000 7f6b067fc640 1 -- 172.21.15.3:0/214288219 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6b0c0ce7f0 con 0x7f6b10103b60 2024-09-21T06:13:36.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.275+0000 7f6b1792a640 1 -- 172.21.15.3:0/214288219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b10103f70 con 0x7f6b10103b60 2024-09-21T06:13:36.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.277+0000 7f6b067fc640 1 -- 172.21.15.3:0/214288219 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23048 (secure 0 0 0) 0x7f6b10103f70 con 0x7f6b10103b60 2024-09-21T06:13:36.279 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:13:36.279 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","created":"2024-09-21T06:04:24.293786+0000","modified":"2024-09-21T06:13:24.594225+0000","last_up_change":"2024-09-21T06:12:17.781243+0000","last_in_change":"2024-09-21T06:11:54.727435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:08:26.015803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:13:07.084406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:13:09.149891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51736f81-f410-48ee-a8f3-121289607309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6803","nonce":615312328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6805","nonce":615312328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6809","nonce":615312328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6807","nonce":615312328}]},"public_addr":"172.21.15.3:6803/615312328","cluster_addr":"172.21.15.3:6805/615312328","heartbeat_back_addr":"172.21.15.3:6809/615312328","heartbeat_front_addr":"172.21.15.3:6807/615312328","state":["exists","up"]},{"osd":1,"uuid":"55696a37-34c1-482a-8561-7d08f0aa29fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6801","nonce":4150747115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6803","nonce":4150747115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6807","nonce":4150747115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6805","nonce":4150747115}]},"public_addr":"172.21.15.137:6801/4150747115","cluster_addr":"172.21.15.137:6803/4150747115","heartbeat_back_addr":"172.21.15.137:6807/4150747115","heartbeat_front_addr":"172.21.15.137:6805/4150747115","state":["exists","up"]},{"osd":2,"uuid":"8a81d34b-464b-46a2-8ef2-5223b5947004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6801","nonce":3392577077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6803","nonce":3392577077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6807","nonce":3392577077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6805","nonce":3392577077}]},"public_addr":"172.21.15.204:6801/3392577077","cluster_addr":"172.21.15.204:6803/3392577077","heartbeat_back_addr":"172.21.15.204:6807/3392577077","heartbeat_front_addr":"172.21.15.204:6805/3392577077","state":["exists","up"]},{"osd":3,"uuid":"94ab3938-7a70-44cb-99bf-9d2dedc4eb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6811","nonce":4293741635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6813","nonce":4293741635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6817","nonce":4293741635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6815","nonce":4293741635}]},"public_addr":"172.21.15.3:6811/4293741635","cluster_addr":"172.21.15.3:6813/4293741635","heartbeat_back_addr":"172.21.15.3:6817/4293741635","heartbeat_front_addr":"172.21.15.3:6815/4293741635","state":["exists","up"]},{"osd":4,"uuid":"38551535-d692-401c-aad0-bee38f3026d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6809","nonce":2608988785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6811","nonce":2608988785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6815","nonce":2608988785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6813","nonce":2608988785}]},"public_addr":"172.21.15.137:6809/2608988785","cluster_addr":"172.21.15.137:6811/2608988785","heartbeat_back_addr":"172.21.15.137:6815/2608988785","heartbeat_front_addr":"172.21.15.137:6813/2608988785","state":["exists","up"]},{"osd":5,"uuid":"3d4b785b-28d0-4654-a9b1-4c5c905f4bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6809","nonce":3709074044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6811","nonce":3709074044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6815","nonce":3709074044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6813","nonce":3709074044}]},"public_addr":"172.21.15.204:6809/3709074044","cluster_addr":"172.21.15.204:6811/3709074044","heartbeat_back_addr":"172.21.15.204:6815/3709074044","heartbeat_front_addr":"172.21.15.204:6813/3709074044","state":["exists","up"]},{"osd":6,"uuid":"ba21dbe9-ab43-4b7e-9446-2929ee874c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6819","nonce":3862646594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6821","nonce":3862646594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6825","nonce":3862646594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6823","nonce":3862646594}]},"public_addr":"172.21.15.3:6819/3862646594","cluster_addr":"172.21.15.3:6821/3862646594","heartbeat_back_addr":"172.21.15.3:6825/3862646594","heartbeat_front_addr":"172.21.15.3:6823/3862646594","state":["exists","up"]},{"osd":7,"uuid":"9610fe5c-93df-49f9-b8bd-5cb480e0871b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6817","nonce":2086822496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6819","nonce":2086822496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6823","nonce":2086822496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6821","nonce":2086822496}]},"public_addr":"172.21.15.137:6817/2086822496","cluster_addr":"172.21.15.137:6819/2086822496","heartbeat_back_addr":"172.21.15.137:6823/2086822496","heartbeat_front_addr":"172.21.15.137:6821/2086822496","state":["exists","up"]},{"osd":8,"uuid":"49f6e9b4-6d14-434e-8b42-8b7ffeb2a072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6817","nonce":153476798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6819","nonce":153476798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6823","nonce":153476798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6821","nonce":153476798}]},"public_addr":"172.21.15.204:6817/153476798","cluster_addr":"172.21.15.204:6819/153476798","heartbeat_back_addr":"172.21.15.204:6823/153476798","heartbeat_front_addr":"172.21.15.204:6821/153476798","state":["exists","up"]},{"osd":9,"uuid":"633f67a7-b759-4ea6-82ed-3c42b3fb70ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6827","nonce":1238491549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6829","nonce":1238491549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6833","nonce":1238491549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6831","nonce":1238491549}]},"public_addr":"172.21.15.3:6827/1238491549","cluster_addr":"172.21.15.3:6829/1238491549","heartbeat_back_addr":"172.21.15.3:6833/1238491549","heartbeat_front_addr":"172.21.15.3:6831/1238491549","state":["exists","up"]},{"osd":10,"uuid":"94b65a48-ee87-4e59-9296-a72a22787f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6825","nonce":3486552683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6827","nonce":3486552683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6831","nonce":3486552683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6829","nonce":3486552683}]},"public_addr":"172.21.15.137:6825/3486552683","cluster_addr":"172.21.15.137:6827/3486552683","heartbeat_back_addr":"172.21.15.137:6831/3486552683","heartbeat_front_addr":"172.21.15.137:6829/3486552683","state":["exists","up"]},{"osd":11,"uuid":"c5cc2850-9062-4b8e-beb3-bef0db30a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6825","nonce":1084308610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6827","nonce":1084308610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6831","nonce":1084308610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6829","nonce":1084308610}]},"public_addr":"172.21.15.204:6825/1084308610","cluster_addr":"172.21.15.204:6827/1084308610","heartbeat_back_addr":"172.21.15.204:6831/1084308610","heartbeat_front_addr":"172.21.15.204:6829/1084308610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:31.898745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.089965+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:22.367464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:48.543248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:14.687331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:41.243579+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:07.290227+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:32.930046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:59.197638+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:24.315219+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:50.333639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:12:16.453640+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1573374138":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1595702358":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/547897695":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/3673347975":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/141972779":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6800/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:6801/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/2363985440":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/2138752581":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1820734047":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/2929116087":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/45453677":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/2929116087":"2024-09-22T06:05:21.689191+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:13:36.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.280+0000 7f6b1792a640 1 -- 172.21.15.3:0/214288219 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6afc0b65f0 msgr2=0x7f6afc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.280+0000 7f6b1792a640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6afc0b65f0 0x7f6afc0b8b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6b00005fd0 tx=0x7f6b00005e20 comp rx=0 tx=0).stop 2024-09-21T06:13:36.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.280+0000 7f6b1792a640 1 -- 172.21.15.3:0/214288219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b10103b60 msgr2=0x7f6b1019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:36.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.280+0000 7f6b1792a640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b10103b60 0x7f6b1019a4a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c005940 tx=0x7f6b0c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.280+0000 7f6b1792a640 1 -- 172.21.15.3:0/214288219 shutdown_connections 2024-09-21T06:13:36.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.280+0000 7f6b1792a640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6afc0b65f0 0x7f6afc0b8b00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.280+0000 7f6b1792a640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b10109df0 0x7f6b1019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.280+0000 7f6b1792a640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b101094a0 0x7f6b1019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.280+0000 7f6b1792a640 1 --2- 172.21.15.3:0/214288219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b10103b60 0x7f6b1019a4a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:36.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.280+0000 7f6b1792a640 1 -- 172.21.15.3:0/214288219 >> 172.21.15.3:0/214288219 conn(0x7f6b1007b850 msgr2=0x7f6b10107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:36.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.280+0000 7f6b1792a640 1 -- 172.21.15.3:0/214288219 shutdown_connections 2024-09-21T06:13:36.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:36.280+0000 7f6b1792a640 1 -- 172.21.15.3:0/214288219 wait complete. 2024-09-21T06:13:36.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:36 smithi137 ceph-mon[34464]: pgmap v283: 97 pgs: 97 active+clean; 629 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.0 KiB/s wr, 17 op/s; 156 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T06:13:36.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:36 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/214288219' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:36.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:36 smithi204 ceph-mon[34659]: pgmap v283: 97 pgs: 97 active+clean; 629 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.0 KiB/s wr, 17 op/s; 156 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T06:13:36.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:36 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/214288219' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:37.076 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:36 smithi003 ceph-mon[31227]: pgmap v283: 97 pgs: 97 active+clean; 629 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.0 KiB/s wr, 17 op/s; 156 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T06:13:37.076 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:36 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/214288219' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:37.112 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T06:13:37.112 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:13:37.377 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:37.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.864+0000 7f7675a19640 1 -- 172.21.15.3:0/75643310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7670108d20 msgr2=0x7f767010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.864+0000 7f7675a19640 1 --2- 172.21.15.3:0/75643310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7670108d20 0x7f767010f550 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7668009ca0 tx=0x7f766802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.864+0000 7f7675a19640 1 -- 172.21.15.3:0/75643310 shutdown_connections 2024-09-21T06:13:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.864+0000 7f7675a19640 1 --2- 172.21.15.3:0/75643310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7670108d20 0x7f767010f550 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.864+0000 7f7675a19640 1 --2- 172.21.15.3:0/75643310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76701083d0 0x7f76701087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.864+0000 7f7675a19640 1 --2- 172.21.15.3:0/75643310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7670102ae0 0x7f7670102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.864+0000 7f7675a19640 1 -- 172.21.15.3:0/75643310 >> 172.21.15.3:0/75643310 conn(0x7f76700797b0 msgr2=0x7f7670079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.864+0000 7f7675a19640 1 -- 172.21.15.3:0/75643310 shutdown_connections 2024-09-21T06:13:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.864+0000 7f7675a19640 1 -- 172.21.15.3:0/75643310 wait complete. 2024-09-21T06:13:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.864+0000 7f7675a19640 1 Processor -- start 2024-09-21T06:13:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f7675a19640 1 -- start start 2024-09-21T06:13:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f7675a19640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7670102ae0 0x7f767019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f7675a19640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76701083d0 0x7f767019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f7675a19640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7670108d20 0x7f767019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f7675a19640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f76701129e0 con 0x7f76701083d0 2024-09-21T06:13:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f7675a19640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7670112860 con 0x7f7670102ae0 2024-09-21T06:13:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f7675a19640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7670112b60 con 0x7f7670108d20 2024-09-21T06:13:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f766e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76701083d0 0x7f767019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f766e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76701083d0 0x7f767019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44082/0 (socket says 172.21.15.3:44082) 2024-09-21T06:13:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f766e7fc640 1 -- 172.21.15.3:0/2390660696 learned_addr learned my addr 172.21.15.3:0/2390660696 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f766effd640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7670102ae0 0x7f767019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f766f7fe640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7670108d20 0x7f767019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f766e7fc640 1 -- 172.21.15.3:0/2390660696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7670108d20 msgr2=0x7f767019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f766e7fc640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7670108d20 0x7f767019fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f766e7fc640 1 -- 172.21.15.3:0/2390660696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7670102ae0 msgr2=0x7f767019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f766e7fc640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7670102ae0 0x7f767019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f766e7fc640 1 -- 172.21.15.3:0/2390660696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76701a0410 con 0x7f76701083d0 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.865+0000 7f766effd640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7670102ae0 0x7f767019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.866+0000 7f766e7fc640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76701083d0 0x7f767019aa40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f765800d740 tx=0x7f765800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.866+0000 7f7674a17640 1 -- 172.21.15.3:0/2390660696 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658017070 con 0x7f76701083d0 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.866+0000 7f7674a17640 1 -- 172.21.15.3:0/2390660696 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f765800de50 con 0x7f76701083d0 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.866+0000 7f7675a19640 1 -- 172.21.15.3:0/2390660696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76701a0700 con 0x7f76701083d0 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.866+0000 7f7674a17640 1 -- 172.21.15.3:0/2390660696 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765800f620 con 0x7f76701083d0 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.866+0000 7f7675a19640 1 -- 172.21.15.3:0/2390660696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76701a6e60 con 0x7f76701083d0 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.867+0000 7f7675a19640 1 -- 172.21.15.3:0/2390660696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7670102ef0 con 0x7f76701083d0 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.869+0000 7f7674a17640 1 -- 172.21.15.3:0/2390660696 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f765801a070 con 0x7f76701083d0 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.869+0000 7f7674a17640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f765c0b65f0 0x7f765c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.870+0000 7f766effd640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f765c0b65f0 0x7f765c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.870+0000 7f7674a17640 1 -- 172.21.15.3:0/2390660696 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f76580e7070 con 0x7f76701083d0 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.870+0000 7f766effd640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f765c0b65f0 0x7f765c0b8b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7660009c20 tx=0x7f7660005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:37.872+0000 7f7674a17640 1 -- 172.21.15.3:0/2390660696 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f76580ad6c0 con 0x7f76701083d0 2024-09-21T06:13:38.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.059+0000 7f7675a19640 1 -- 172.21.15.3:0/2390660696 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f767010bb80 con 0x7f765c0b65f0 2024-09-21T06:13:38.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.064+0000 7f7674a17640 1 -- 172.21.15.3:0/2390660696 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259031 (secure 0 0 0) 0x7f767010bb80 con 0x7f765c0b65f0 2024-09-21T06:13:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:13:38.072 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-21T06:13:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.068+0000 7f7675a19640 1 -- 172.21.15.3:0/2390660696 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f765c0b65f0 msgr2=0x7f765c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.068+0000 7f7675a19640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f765c0b65f0 0x7f765c0b8b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7660009c20 tx=0x7f7660005f90 comp rx=0 tx=0).stop 2024-09-21T06:13:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.068+0000 7f7675a19640 1 -- 172.21.15.3:0/2390660696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76701083d0 msgr2=0x7f767019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.068+0000 7f7675a19640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76701083d0 0x7f767019aa40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f765800d740 tx=0x7f765800dc00 comp rx=0 tx=0).stop 2024-09-21T06:13:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.068+0000 7f7675a19640 1 -- 172.21.15.3:0/2390660696 shutdown_connections 2024-09-21T06:13:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.068+0000 7f7675a19640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f765c0b65f0 0x7f765c0b8b00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.068+0000 7f7675a19640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7670108d20 0x7f767019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.068+0000 7f7675a19640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76701083d0 0x7f767019aa40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.068+0000 7f7675a19640 1 --2- 172.21.15.3:0/2390660696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7670102ae0 0x7f767019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.068+0000 7f7675a19640 1 -- 172.21.15.3:0/2390660696 >> 172.21.15.3:0/2390660696 conn(0x7f76700797b0 msgr2=0x7f7670101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.068+0000 7f7675a19640 1 -- 172.21.15.3:0/2390660696 shutdown_connections 2024-09-21T06:13:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:38.068+0000 7f7675a19640 1 -- 172.21.15.3:0/2390660696 wait complete. 2024-09-21T06:13:38.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:38 smithi137 ceph-mon[34464]: pgmap v284: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 17 op/s; 143 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T06:13:38.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:38 smithi137 ceph-mon[34464]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:13:38.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:38 smithi204 ceph-mon[34659]: pgmap v284: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 17 op/s; 143 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T06:13:38.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:38 smithi204 ceph-mon[34659]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:13:38.917 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:38 smithi003 ceph-mon[31227]: pgmap v284: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 17 op/s; 143 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T06:13:38.918 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:38 smithi003 ceph-mon[31227]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:13:38.918 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-21T06:13:37.623103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656780,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":236,"num_read_kb":209,"num_write":362,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2953564,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359940,"kb_used_data":10680,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581884,"statfs":{"total":1151940427776,"available":1151571849216,"internally_reserved":0,"allocated":10936320,"data_stored":7057136,"data_compressed":177687,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":59264,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":-8,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":147,"num_read_kb":134,"num_write":68,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1724,"num_keys_recovered":10,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003068"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607638+0000","last_change":"2024-09-21T06:13:10.515115+0000","last_active":"2024-09-21T06:13:24.607638+0000","last_peered":"2024-09-21T06:13:24.607638+0000","last_clean":"2024-09-21T06:13:24.607638+0000","last_became_active":"2024-09-21T06:13:10.514553+0000","last_became_peered":"2024-09-21T06:13:10.514553+0000","last_unstale":"2024-09-21T06:13:24.607638+0000","last_undegraded":"2024-09-21T06:13:24.607638+0000","last_fullsized":"2024-09-21T06:13:24.607638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:33.268926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609526+0000","last_change":"2024-09-21T06:13:10.511469+0000","last_active":"2024-09-21T06:13:24.609526+0000","last_peered":"2024-09-21T06:13:24.609526+0000","last_clean":"2024-09-21T06:13:24.609526+0000","last_became_active":"2024-09-21T06:13:10.511102+0000","last_became_peered":"2024-09-21T06:13:10.511102+0000","last_unstale":"2024-09-21T06:13:24.609526+0000","last_undegraded":"2024-09-21T06:13:24.609526+0000","last_fullsized":"2024-09-21T06:13:24.609526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:12.483821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609288+0000","last_change":"2024-09-21T06:13:10.520090+0000","last_active":"2024-09-21T06:13:24.609288+0000","last_peered":"2024-09-21T06:13:24.609288+0000","last_clean":"2024-09-21T06:13:24.609288+0000","last_became_active":"2024-09-21T06:13:10.519758+0000","last_became_peered":"2024-09-21T06:13:10.519758+0000","last_unstale":"2024-09-21T06:13:24.609288+0000","last_undegraded":"2024-09-21T06:13:24.609288+0000","last_fullsized":"2024-09-21T06:13:24.609288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:20.836847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.617812+0000","last_change":"2024-09-21T06:13:10.511732+0000","last_active":"2024-09-21T06:13:24.617812+0000","last_peered":"2024-09-21T06:13:24.617812+0000","last_clean":"2024-09-21T06:13:24.617812+0000","last_became_active":"2024-09-21T06:13:10.511377+0000","last_became_peered":"2024-09-21T06:13:10.511377+0000","last_unstale":"2024-09-21T06:13:24.617812+0000","last_undegraded":"2024-09-21T06:13:24.617812+0000","last_fullsized":"2024-09-21T06:13:24.617812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:18.500739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607690+0000","last_change":"2024-09-21T06:13:10.510464+0000","last_active":"2024-09-21T06:13:24.607690+0000","last_peered":"2024-09-21T06:13:24.607690+0000","last_clean":"2024-09-21T06:13:24.607690+0000","last_became_active":"2024-09-21T06:13:10.509979+0000","last_became_peered":"2024-09-21T06:13:10.509979+0000","last_unstale":"2024-09-21T06:13:24.607690+0000","last_undegraded":"2024-09-21T06:13:24.607690+0000","last_fullsized":"2024-09-21T06:13:24.607690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:56.871043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608937+0000","last_change":"2024-09-21T06:13:24.608832+0000","last_active":"2024-09-21T06:13:24.608937+0000","last_peered":"2024-09-21T06:13:24.608937+0000","last_clean":"2024-09-21T06:13:24.608937+0000","last_became_active":"2024-09-21T06:13:24.608439+0000","last_became_peered":"2024-09-21T06:13:24.608439+0000","last_unstale":"2024-09-21T06:13:24.608937+0000","last_undegraded":"2024-09-21T06:13:24.608937+0000","last_fullsized":"2024-09-21T06:13:24.608937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:41.515894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609585+0000","last_change":"2024-09-21T06:13:10.511391+0000","last_active":"2024-09-21T06:13:24.609585+0000","last_peered":"2024-09-21T06:13:24.609585+0000","last_clean":"2024-09-21T06:13:24.609585+0000","last_became_active":"2024-09-21T06:13:10.510733+0000","last_became_peered":"2024-09-21T06:13:10.510733+0000","last_unstale":"2024-09-21T06:13:24.609585+0000","last_undegraded":"2024-09-21T06:13:24.609585+0000","last_fullsized":"2024-09-21T06:13:24.609585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:24.071860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609673+0000","last_change":"2024-09-21T06:13:10.510886+0000","last_active":"2024-09-21T06:13:24.609673+0000","last_peered":"2024-09-21T06:13:24.609673+0000","last_clean":"2024-09-21T06:13:24.609673+0000","last_became_active":"2024-09-21T06:13:10.510159+0000","last_became_peered":"2024-09-21T06:13:10.510159+0000","last_unstale":"2024-09-21T06:13:24.609673+0000","last_undegraded":"2024-09-21T06:13:24.609673+0000","last_fullsized":"2024-09-21T06:13:24.609673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:37.124848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598718+0000","last_change":"2024-09-21T06:13:10.510466+0000","last_active":"2024-09-21T06:13:24.598718+0000","last_peered":"2024-09-21T06:13:24.598718+0000","last_clean":"2024-09-21T06:13:24.598718+0000","last_became_active":"2024-09-21T06:13:10.509928+0000","last_became_peered":"2024-09-21T06:13:10.509928+0000","last_unstale":"2024-09-21T06:13:24.598718+0000","last_undegraded":"2024-09-21T06:13:24.598718+0000","last_fullsized":"2024-09-21T06:13:24.598718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:06.156505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600080+0000","last_change":"2024-09-21T06:13:10.510622+0000","last_active":"2024-09-21T06:13:24.600080+0000","last_peered":"2024-09-21T06:13:24.600080+0000","last_clean":"2024-09-21T06:13:24.600080+0000","last_became_active":"2024-09-21T06:13:10.510206+0000","last_became_peered":"2024-09-21T06:13:10.510206+0000","last_unstale":"2024-09-21T06:13:24.600080+0000","last_undegraded":"2024-09-21T06:13:24.600080+0000","last_fullsized":"2024-09-21T06:13:24.600080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:03.223451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610031+0000","last_change":"2024-09-21T06:13:10.517525+0000","last_active":"2024-09-21T06:13:24.610031+0000","last_peered":"2024-09-21T06:13:24.610031+0000","last_clean":"2024-09-21T06:13:24.610031+0000","last_became_active":"2024-09-21T06:13:10.517144+0000","last_became_peered":"2024-09-21T06:13:10.517144+0000","last_unstale":"2024-09-21T06:13:24.610031+0000","last_undegraded":"2024-09-21T06:13:24.610031+0000","last_fullsized":"2024-09-21T06:13:24.610031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:46.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.617837+0000","last_change":"2024-09-21T06:13:10.515537+0000","last_active":"2024-09-21T06:13:24.617837+0000","last_peered":"2024-09-21T06:13:24.617837+0000","last_clean":"2024-09-21T06:13:24.617837+0000","last_became_active":"2024-09-21T06:13:10.515063+0000","last_became_peered":"2024-09-21T06:13:10.515063+0000","last_unstale":"2024-09-21T06:13:24.617837+0000","last_undegraded":"2024-09-21T06:13:24.617837+0000","last_fullsized":"2024-09-21T06:13:24.617837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:33.718471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597513+0000","last_change":"2024-09-21T06:13:10.509368+0000","last_active":"2024-09-21T06:13:24.597513+0000","last_peered":"2024-09-21T06:13:24.597513+0000","last_clean":"2024-09-21T06:13:24.597513+0000","last_became_active":"2024-09-21T06:13:10.508982+0000","last_became_peered":"2024-09-21T06:13:10.508982+0000","last_unstale":"2024-09-21T06:13:24.597513+0000","last_undegraded":"2024-09-21T06:13:24.597513+0000","last_fullsized":"2024-09-21T06:13:24.597513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:54.430889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608602+0000","last_change":"2024-09-21T06:13:10.506559+0000","last_active":"2024-09-21T06:13:24.608602+0000","last_peered":"2024-09-21T06:13:24.608602+0000","last_clean":"2024-09-21T06:13:24.608602+0000","last_became_active":"2024-09-21T06:13:10.506161+0000","last_became_peered":"2024-09-21T06:13:10.506161+0000","last_unstale":"2024-09-21T06:13:24.608602+0000","last_undegraded":"2024-09-21T06:13:24.608602+0000","last_fullsized":"2024-09-21T06:13:24.608602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:19.870292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608304+0000","last_change":"2024-09-21T06:13:10.516000+0000","last_active":"2024-09-21T06:13:24.608304+0000","last_peered":"2024-09-21T06:13:24.608304+0000","last_clean":"2024-09-21T06:13:24.608304+0000","last_became_active":"2024-09-21T06:13:10.515543+0000","last_became_peered":"2024-09-21T06:13:10.515543+0000","last_unstale":"2024-09-21T06:13:24.608304+0000","last_undegraded":"2024-09-21T06:13:24.608304+0000","last_fullsized":"2024-09-21T06:13:24.608304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:48.968489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598302+0000","last_change":"2024-09-21T06:13:10.513753+0000","last_active":"2024-09-21T06:13:24.598302+0000","last_peered":"2024-09-21T06:13:24.598302+0000","last_clean":"2024-09-21T06:13:24.598302+0000","last_became_active":"2024-09-21T06:13:10.511079+0000","last_became_peered":"2024-09-21T06:13:10.511079+0000","last_unstale":"2024-09-21T06:13:24.598302+0000","last_undegraded":"2024-09-21T06:13:24.598302+0000","last_fullsized":"2024-09-21T06:13:24.598302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:25.320041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607768+0000","last_change":"2024-09-21T06:13:10.515222+0000","last_active":"2024-09-21T06:13:24.607768+0000","last_peered":"2024-09-21T06:13:24.607768+0000","last_clean":"2024-09-21T06:13:24.607768+0000","last_became_active":"2024-09-21T06:13:10.514604+0000","last_became_peered":"2024-09-21T06:13:10.514604+0000","last_unstale":"2024-09-21T06:13:24.607768+0000","last_undegraded":"2024-09-21T06:13:24.607768+0000","last_fullsized":"2024-09-21T06:13:24.607768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:03.165695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600779+0000","last_change":"2024-09-21T06:13:10.518334+0000","last_active":"2024-09-21T06:13:24.600779+0000","last_peered":"2024-09-21T06:13:24.600779+0000","last_clean":"2024-09-21T06:13:24.600779+0000","last_became_active":"2024-09-21T06:13:10.517956+0000","last_became_peered":"2024-09-21T06:13:10.517956+0000","last_unstale":"2024-09-21T06:13:24.600779+0000","last_undegraded":"2024-09-21T06:13:24.600779+0000","last_fullsized":"2024-09-21T06:13:24.600779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:05.094454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604993+0000","last_change":"2024-09-21T06:13:10.508763+0000","last_active":"2024-09-21T06:13:24.604993+0000","last_peered":"2024-09-21T06:13:24.604993+0000","last_clean":"2024-09-21T06:13:24.604993+0000","last_became_active":"2024-09-21T06:13:10.508371+0000","last_became_peered":"2024-09-21T06:13:10.508371+0000","last_unstale":"2024-09-21T06:13:24.604993+0000","last_undegraded":"2024-09-21T06:13:24.604993+0000","last_fullsized":"2024-09-21T06:13:24.604993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:50.602692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607019+0000","last_change":"2024-09-21T06:13:10.503465+0000","last_active":"2024-09-21T06:13:24.607019+0000","last_peered":"2024-09-21T06:13:24.607019+0000","last_clean":"2024-09-21T06:13:24.607019+0000","last_became_active":"2024-09-21T06:13:10.503072+0000","last_became_peered":"2024-09-21T06:13:10.503072+0000","last_unstale":"2024-09-21T06:13:24.607019+0000","last_undegraded":"2024-09-21T06:13:24.607019+0000","last_fullsized":"2024-09-21T06:13:24.607019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:36.368987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597579+0000","last_change":"2024-09-21T06:13:10.512841+0000","last_active":"2024-09-21T06:13:24.597579+0000","last_peered":"2024-09-21T06:13:24.597579+0000","last_clean":"2024-09-21T06:13:24.597579+0000","last_became_active":"2024-09-21T06:13:10.512407+0000","last_became_peered":"2024-09-21T06:13:10.512407+0000","last_unstale":"2024-09-21T06:13:24.597579+0000","last_undegraded":"2024-09-21T06:13:24.597579+0000","last_fullsized":"2024-09-21T06:13:24.597579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:09.855608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598794+0000","last_change":"2024-09-21T06:13:10.510088+0000","last_active":"2024-09-21T06:13:24.598794+0000","last_peered":"2024-09-21T06:13:24.598794+0000","last_clean":"2024-09-21T06:13:24.598794+0000","last_became_active":"2024-09-21T06:13:10.507904+0000","last_became_peered":"2024-09-21T06:13:10.507904+0000","last_unstale":"2024-09-21T06:13:24.598794+0000","last_undegraded":"2024-09-21T06:13:24.598794+0000","last_fullsized":"2024-09-21T06:13:24.598794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:49.404402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.611267+0000","last_change":"2024-09-21T06:13:10.517720+0000","last_active":"2024-09-21T06:13:24.611267+0000","last_peered":"2024-09-21T06:13:24.611267+0000","last_clean":"2024-09-21T06:13:24.611267+0000","last_became_active":"2024-09-21T06:13:10.514641+0000","last_became_peered":"2024-09-21T06:13:10.514641+0000","last_unstale":"2024-09-21T06:13:24.611267+0000","last_undegraded":"2024-09-21T06:13:24.611267+0000","last_fullsized":"2024-09-21T06:13:24.611267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:40.639665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602839+0000","last_change":"2024-09-21T06:13:10.512234+0000","last_active":"2024-09-21T06:13:24.602839+0000","last_peered":"2024-09-21T06:13:24.602839+0000","last_clean":"2024-09-21T06:13:24.602839+0000","last_became_active":"2024-09-21T06:13:10.511594+0000","last_became_peered":"2024-09-21T06:13:10.511594+0000","last_unstale":"2024-09-21T06:13:24.602839+0000","last_undegraded":"2024-09-21T06:13:24.602839+0000","last_fullsized":"2024-09-21T06:13:24.602839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:04.410342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610455+0000","last_change":"2024-09-21T06:13:10.513216+0000","last_active":"2024-09-21T06:13:24.610455+0000","last_peered":"2024-09-21T06:13:24.610455+0000","last_clean":"2024-09-21T06:13:24.610455+0000","last_became_active":"2024-09-21T06:13:10.512830+0000","last_became_peered":"2024-09-21T06:13:10.512830+0000","last_unstale":"2024-09-21T06:13:24.610455+0000","last_undegraded":"2024-09-21T06:13:24.610455+0000","last_fullsized":"2024-09-21T06:13:24.610455+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:46.418161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.605196+0000","last_change":"2024-09-21T06:13:10.518863+0000","last_active":"2024-09-21T06:13:24.605196+0000","last_peered":"2024-09-21T06:13:24.605196+0000","last_clean":"2024-09-21T06:13:24.605196+0000","last_became_active":"2024-09-21T06:13:10.518492+0000","last_became_peered":"2024-09-21T06:13:10.518492+0000","last_unstale":"2024-09-21T06:13:24.605196+0000","last_undegraded":"2024-09-21T06:13:24.605196+0000","last_fullsized":"2024-09-21T06:13:24.605196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:23.770763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608235+0000","last_change":"2024-09-21T06:13:10.508538+0000","last_active":"2024-09-21T06:13:24.608235+0000","last_peered":"2024-09-21T06:13:24.608235+0000","last_clean":"2024-09-21T06:13:24.608235+0000","last_became_active":"2024-09-21T06:13:10.508017+0000","last_became_peered":"2024-09-21T06:13:10.508017+0000","last_unstale":"2024-09-21T06:13:24.608235+0000","last_undegraded":"2024-09-21T06:13:24.608235+0000","last_fullsized":"2024-09-21T06:13:24.608235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:00.168741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597567+0000","last_change":"2024-09-21T06:13:10.516734+0000","last_active":"2024-09-21T06:13:24.597567+0000","last_peered":"2024-09-21T06:13:24.597567+0000","last_clean":"2024-09-21T06:13:24.597567+0000","last_became_active":"2024-09-21T06:13:10.516362+0000","last_became_peered":"2024-09-21T06:13:10.516362+0000","last_unstale":"2024-09-21T06:13:24.597567+0000","last_undegraded":"2024-09-21T06:13:24.597567+0000","last_fullsized":"2024-09-21T06:13:24.597567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:49.792011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608841+0000","last_change":"2024-09-21T06:13:10.513143+0000","last_active":"2024-09-21T06:13:24.608841+0000","last_peered":"2024-09-21T06:13:24.608841+0000","last_clean":"2024-09-21T06:13:24.608841+0000","last_became_active":"2024-09-21T06:13:10.512338+0000","last_became_peered":"2024-09-21T06:13:10.512338+0000","last_unstale":"2024-09-21T06:13:24.608841+0000","last_undegraded":"2024-09-21T06:13:24.608841+0000","last_fullsized":"2024-09-21T06:13:24.608841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:02.963860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602980+0000","last_change":"2024-09-21T06:13:10.513429+0000","last_active":"2024-09-21T06:13:24.602980+0000","last_peered":"2024-09-21T06:13:24.602980+0000","last_clean":"2024-09-21T06:13:24.602980+0000","last_became_active":"2024-09-21T06:13:10.512995+0000","last_became_peered":"2024-09-21T06:13:10.512995+0000","last_unstale":"2024-09-21T06:13:24.602980+0000","last_undegraded":"2024-09-21T06:13:24.602980+0000","last_fullsized":"2024-09-21T06:13:24.602980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:39.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603628+0000","last_change":"2024-09-21T06:13:10.509580+0000","last_active":"2024-09-21T06:13:24.603628+0000","last_peered":"2024-09-21T06:13:24.603628+0000","last_clean":"2024-09-21T06:13:24.603628+0000","last_became_active":"2024-09-21T06:13:10.509179+0000","last_became_peered":"2024-09-21T06:13:10.509179+0000","last_unstale":"2024-09-21T06:13:24.603628+0000","last_undegraded":"2024-09-21T06:13:24.603628+0000","last_fullsized":"2024-09-21T06:13:24.603628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:39.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.606565+0000","last_change":"2024-09-21T06:13:10.508358+0000","last_active":"2024-09-21T06:13:24.606565+0000","last_peered":"2024-09-21T06:13:24.606565+0000","last_clean":"2024-09-21T06:13:24.606565+0000","last_became_active":"2024-09-21T06:13:10.507952+0000","last_became_peered":"2024-09-21T06:13:10.507952+0000","last_unstale":"2024-09-21T06:13:24.606565+0000","last_undegraded":"2024-09-21T06:13:24.606565+0000","last_fullsized":"2024-09-21T06:13:24.606565+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:55.565203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607615+0000","last_change":"2024-09-21T06:13:10.510580+0000","last_active":"2024-09-21T06:13:24.607615+0000","last_peered":"2024-09-21T06:13:24.607615+0000","last_clean":"2024-09-21T06:13:24.607615+0000","last_became_active":"2024-09-21T06:13:10.510175+0000","last_became_peered":"2024-09-21T06:13:10.510175+0000","last_unstale":"2024-09-21T06:13:24.607615+0000","last_undegraded":"2024-09-21T06:13:24.607615+0000","last_fullsized":"2024-09-21T06:13:24.607615+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:49.467057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":42,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:31.624778+0000","last_change":"2024-09-21T06:13:08.485527+0000","last_active":"2024-09-21T06:13:31.624778+0000","last_peered":"2024-09-21T06:13:31.624778+0000","last_clean":"2024-09-21T06:13:31.624778+0000","last_became_active":"2024-09-21T06:13:08.485094+0000","last_became_peered":"2024-09-21T06:13:08.485094+0000","last_unstale":"2024-09-21T06:13:31.624778+0000","last_undegraded":"2024-09-21T06:13:31.624778+0000","last_fullsized":"2024-09-21T06:13:31.624778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:48.285638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603582+0000","last_change":"2024-09-21T06:13:10.507977+0000","last_active":"2024-09-21T06:13:24.603582+0000","last_peered":"2024-09-21T06:13:24.603582+0000","last_clean":"2024-09-21T06:13:24.603582+0000","last_became_active":"2024-09-21T06:13:10.507594+0000","last_became_peered":"2024-09-21T06:13:10.507594+0000","last_unstale":"2024-09-21T06:13:24.603582+0000","last_undegraded":"2024-09-21T06:13:24.603582+0000","last_fullsized":"2024-09-21T06:13:24.603582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:57.318240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.599690+0000","last_change":"2024-09-21T06:13:08.491237+0000","last_active":"2024-09-21T06:13:24.599690+0000","last_peered":"2024-09-21T06:13:24.599690+0000","last_clean":"2024-09-21T06:13:24.599690+0000","last_became_active":"2024-09-21T06:13:08.490908+0000","last_became_peered":"2024-09-21T06:13:08.490908+0000","last_unstale":"2024-09-21T06:13:24.599690+0000","last_undegraded":"2024-09-21T06:13:24.599690+0000","last_fullsized":"2024-09-21T06:13:24.599690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:53.185029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597228+0000","last_change":"2024-09-21T06:13:08.488721+0000","last_active":"2024-09-21T06:13:24.597228+0000","last_peered":"2024-09-21T06:13:24.597228+0000","last_clean":"2024-09-21T06:13:24.597228+0000","last_became_active":"2024-09-21T06:13:08.488347+0000","last_became_peered":"2024-09-21T06:13:08.488347+0000","last_unstale":"2024-09-21T06:13:24.597228+0000","last_undegraded":"2024-09-21T06:13:24.597228+0000","last_fullsized":"2024-09-21T06:13:24.597228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:27.008479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609911+0000","last_change":"2024-09-21T06:13:10.506527+0000","last_active":"2024-09-21T06:13:24.609911+0000","last_peered":"2024-09-21T06:13:24.609911+0000","last_clean":"2024-09-21T06:13:24.609911+0000","last_became_active":"2024-09-21T06:13:10.506163+0000","last_became_peered":"2024-09-21T06:13:10.506163+0000","last_unstale":"2024-09-21T06:13:24.609911+0000","last_undegraded":"2024-09-21T06:13:24.609911+0000","last_fullsized":"2024-09-21T06:13:24.609911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.244097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604150+0000","last_change":"2024-09-21T06:13:08.489292+0000","last_active":"2024-09-21T06:13:24.604150+0000","last_peered":"2024-09-21T06:13:24.604150+0000","last_clean":"2024-09-21T06:13:24.604150+0000","last_became_active":"2024-09-21T06:13:08.488918+0000","last_became_peered":"2024-09-21T06:13:08.488918+0000","last_unstale":"2024-09-21T06:13:24.604150+0000","last_undegraded":"2024-09-21T06:13:24.604150+0000","last_fullsized":"2024-09-21T06:13:24.604150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:37.992260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600648+0000","last_change":"2024-09-21T06:13:10.505027+0000","last_active":"2024-09-21T06:13:24.600648+0000","last_peered":"2024-09-21T06:13:24.600648+0000","last_clean":"2024-09-21T06:13:24.600648+0000","last_became_active":"2024-09-21T06:13:10.504669+0000","last_became_peered":"2024-09-21T06:13:10.504669+0000","last_unstale":"2024-09-21T06:13:24.600648+0000","last_undegraded":"2024-09-21T06:13:24.600648+0000","last_fullsized":"2024-09-21T06:13:24.600648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:43.927229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":52,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:35.629799+0000","last_change":"2024-09-21T06:13:08.491101+0000","last_active":"2024-09-21T06:13:35.629799+0000","last_peered":"2024-09-21T06:13:35.629799+0000","last_clean":"2024-09-21T06:13:35.629799+0000","last_became_active":"2024-09-21T06:13:08.490607+0000","last_became_peered":"2024-09-21T06:13:08.490607+0000","last_unstale":"2024-09-21T06:13:35.629799+0000","last_undegraded":"2024-09-21T06:13:35.629799+0000","last_fullsized":"2024-09-21T06:13:35.629799+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:16.758463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598830+0000","last_change":"2024-09-21T06:13:10.510453+0000","last_active":"2024-09-21T06:13:24.598830+0000","last_peered":"2024-09-21T06:13:24.598830+0000","last_clean":"2024-09-21T06:13:24.598830+0000","last_became_active":"2024-09-21T06:13:10.508513+0000","last_became_peered":"2024-09-21T06:13:10.508513+0000","last_unstale":"2024-09-21T06:13:24.598830+0000","last_undegraded":"2024-09-21T06:13:24.598830+0000","last_fullsized":"2024-09-21T06:13:24.598830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:48.388140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609878+0000","last_change":"2024-09-21T06:13:08.488714+0000","last_active":"2024-09-21T06:13:24.609878+0000","last_peered":"2024-09-21T06:13:24.609878+0000","last_clean":"2024-09-21T06:13:24.609878+0000","last_became_active":"2024-09-21T06:13:08.488363+0000","last_became_peered":"2024-09-21T06:13:08.488363+0000","last_unstale":"2024-09-21T06:13:24.609878+0000","last_undegraded":"2024-09-21T06:13:24.609878+0000","last_fullsized":"2024-09-21T06:13:24.609878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:33.626165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609602+0000","last_change":"2024-09-21T06:13:10.508046+0000","last_active":"2024-09-21T06:13:24.609602+0000","last_peered":"2024-09-21T06:13:24.609602+0000","last_clean":"2024-09-21T06:13:24.609602+0000","last_became_active":"2024-09-21T06:13:10.507660+0000","last_became_peered":"2024-09-21T06:13:10.507660+0000","last_unstale":"2024-09-21T06:13:24.609602+0000","last_undegraded":"2024-09-21T06:13:24.609602+0000","last_fullsized":"2024-09-21T06:13:24.609602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:35.466965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603272+0000","last_change":"2024-09-21T06:13:08.562863+0000","last_active":"2024-09-21T06:13:24.603272+0000","last_peered":"2024-09-21T06:13:24.603272+0000","last_clean":"2024-09-21T06:13:24.603272+0000","last_became_active":"2024-09-21T06:13:08.562424+0000","last_became_peered":"2024-09-21T06:13:08.562424+0000","last_unstale":"2024-09-21T06:13:24.603272+0000","last_undegraded":"2024-09-21T06:13:24.603272+0000","last_fullsized":"2024-09-21T06:13:24.603272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:43.251932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598595+0000","last_change":"2024-09-21T06:13:10.509818+0000","last_active":"2024-09-21T06:13:24.598595+0000","last_peered":"2024-09-21T06:13:24.598595+0000","last_clean":"2024-09-21T06:13:24.598595+0000","last_became_active":"2024-09-21T06:13:10.509468+0000","last_became_peered":"2024-09-21T06:13:10.509468+0000","last_unstale":"2024-09-21T06:13:24.598595+0000","last_undegraded":"2024-09-21T06:13:24.598595+0000","last_fullsized":"2024-09-21T06:13:24.598595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:25.130868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'27","reported_seq":100,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:35.908304+0000","last_change":"2024-09-21T06:13:24.633807+0000","last_active":"2024-09-21T06:13:35.908304+0000","last_peered":"2024-09-21T06:13:35.908304+0000","last_clean":"2024-09-21T06:13:35.908304+0000","last_became_active":"2024-09-21T06:13:24.618599+0000","last_became_peered":"2024-09-21T06:13:24.618599+0000","last_unstale":"2024-09-21T06:13:35.908304+0000","last_undegraded":"2024-09-21T06:13:35.908304+0000","last_fullsized":"2024-09-21T06:13:35.908304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:07.449253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58583,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":59,"num_write":29,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608537+0000","last_change":"2024-09-21T06:13:10.511360+0000","last_active":"2024-09-21T06:13:24.608537+0000","last_peered":"2024-09-21T06:13:24.608537+0000","last_clean":"2024-09-21T06:13:24.608537+0000","last_became_active":"2024-09-21T06:13:10.510980+0000","last_became_peered":"2024-09-21T06:13:10.510980+0000","last_unstale":"2024-09-21T06:13:24.608537+0000","last_undegraded":"2024-09-21T06:13:24.608537+0000","last_fullsized":"2024-09-21T06:13:24.608537+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:43.859758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604427+0000","last_change":"2024-09-21T06:13:08.486262+0000","last_active":"2024-09-21T06:13:24.604427+0000","last_peered":"2024-09-21T06:13:24.604427+0000","last_clean":"2024-09-21T06:13:24.604427+0000","last_became_active":"2024-09-21T06:13:08.485509+0000","last_became_peered":"2024-09-21T06:13:08.485509+0000","last_unstale":"2024-09-21T06:13:24.604427+0000","last_undegraded":"2024-09-21T06:13:24.604427+0000","last_fullsized":"2024-09-21T06:13:24.604427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:31.726746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597324+0000","last_change":"2024-09-21T06:13:10.506894+0000","last_active":"2024-09-21T06:13:24.597324+0000","last_peered":"2024-09-21T06:13:24.597324+0000","last_clean":"2024-09-21T06:13:24.597324+0000","last_became_active":"2024-09-21T06:13:10.506489+0000","last_became_peered":"2024-09-21T06:13:10.506489+0000","last_unstale":"2024-09-21T06:13:24.597324+0000","last_undegraded":"2024-09-21T06:13:24.597324+0000","last_fullsized":"2024-09-21T06:13:24.597324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:58.755601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603141+0000","last_change":"2024-09-21T06:13:08.483429+0000","last_active":"2024-09-21T06:13:24.603141+0000","last_peered":"2024-09-21T06:13:24.603141+0000","last_clean":"2024-09-21T06:13:24.603141+0000","last_became_active":"2024-09-21T06:13:08.483028+0000","last_became_peered":"2024-09-21T06:13:08.483028+0000","last_unstale":"2024-09-21T06:13:24.603141+0000","last_undegraded":"2024-09-21T06:13:24.603141+0000","last_fullsized":"2024-09-21T06:13:24.603141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:33.258154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609758+0000","last_change":"2024-09-21T06:13:10.513398+0000","last_active":"2024-09-21T06:13:24.609758+0000","last_peered":"2024-09-21T06:13:24.609758+0000","last_clean":"2024-09-21T06:13:24.609758+0000","last_became_active":"2024-09-21T06:13:10.512713+0000","last_became_peered":"2024-09-21T06:13:10.512713+0000","last_unstale":"2024-09-21T06:13:24.609758+0000","last_undegraded":"2024-09-21T06:13:24.609758+0000","last_fullsized":"2024-09-21T06:13:24.609758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:21.026984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":45,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:35.638107+0000","last_change":"2024-09-21T06:13:08.493771+0000","last_active":"2024-09-21T06:13:35.638107+0000","last_peered":"2024-09-21T06:13:35.638107+0000","last_clean":"2024-09-21T06:13:35.638107+0000","last_became_active":"2024-09-21T06:13:08.493380+0000","last_became_peered":"2024-09-21T06:13:08.493380+0000","last_unstale":"2024-09-21T06:13:35.638107+0000","last_undegraded":"2024-09-21T06:13:35.638107+0000","last_fullsized":"2024-09-21T06:13:35.638107+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:12.552137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609811+0000","last_change":"2024-09-21T06:13:10.511679+0000","last_active":"2024-09-21T06:13:24.609811+0000","last_peered":"2024-09-21T06:13:24.609811+0000","last_clean":"2024-09-21T06:13:24.609811+0000","last_became_active":"2024-09-21T06:13:10.511270+0000","last_became_peered":"2024-09-21T06:13:10.511270+0000","last_unstale":"2024-09-21T06:13:24.609811+0000","last_undegraded":"2024-09-21T06:13:24.609811+0000","last_fullsized":"2024-09-21T06:13:24.609811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:10.521853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.627560+0000","last_change":"2024-09-21T06:13:24.627467+0000","last_active":"2024-09-21T06:13:24.627560+0000","last_peered":"2024-09-21T06:13:24.627560+0000","last_clean":"2024-09-21T06:13:24.627560+0000","last_became_active":"2024-09-21T06:13:24.610263+0000","last_became_peered":"2024-09-21T06:13:24.610263+0000","last_unstale":"2024-09-21T06:13:24.627560+0000","last_undegraded":"2024-09-21T06:13:24.627560+0000","last_fullsized":"2024-09-21T06:13:24.627560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:26.472147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602912+0000","last_change":"2024-09-21T06:13:10.517214+0000","last_active":"2024-09-21T06:13:24.602912+0000","last_peered":"2024-09-21T06:13:24.602912+0000","last_clean":"2024-09-21T06:13:24.602912+0000","last_became_active":"2024-09-21T06:13:10.516828+0000","last_became_peered":"2024-09-21T06:13:10.516828+0000","last_unstale":"2024-09-21T06:13:24.602912+0000","last_undegraded":"2024-09-21T06:13:24.602912+0000","last_fullsized":"2024-09-21T06:13:24.602912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:58.077670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608703+0000","last_change":"2024-09-21T06:13:08.489153+0000","last_active":"2024-09-21T06:13:24.608703+0000","last_peered":"2024-09-21T06:13:24.608703+0000","last_clean":"2024-09-21T06:13:24.608703+0000","last_became_active":"2024-09-21T06:13:08.488732+0000","last_became_peered":"2024-09-21T06:13:08.488732+0000","last_unstale":"2024-09-21T06:13:24.608703+0000","last_undegraded":"2024-09-21T06:13:24.608703+0000","last_fullsized":"2024-09-21T06:13:24.608703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:22.848136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610561+0000","last_change":"2024-09-21T06:13:24.610441+0000","last_active":"2024-09-21T06:13:24.610561+0000","last_peered":"2024-09-21T06:13:24.610561+0000","last_clean":"2024-09-21T06:13:24.610561+0000","last_became_active":"2024-09-21T06:13:24.609992+0000","last_became_peered":"2024-09-21T06:13:24.609992+0000","last_unstale":"2024-09-21T06:13:24.610561+0000","last_undegraded":"2024-09-21T06:13:24.610561+0000","last_fullsized":"2024-09-21T06:13:24.610561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:59.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597725+0000","last_change":"2024-09-21T06:13:08.484514+0000","last_active":"2024-09-21T06:13:24.597725+0000","last_peered":"2024-09-21T06:13:24.597725+0000","last_clean":"2024-09-21T06:13:24.597725+0000","last_became_active":"2024-09-21T06:13:08.484134+0000","last_became_peered":"2024-09-21T06:13:08.484134+0000","last_unstale":"2024-09-21T06:13:24.597725+0000","last_undegraded":"2024-09-21T06:13:24.597725+0000","last_fullsized":"2024-09-21T06:13:24.597725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:45.124734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610027+0000","last_change":"2024-09-21T06:13:10.503798+0000","last_active":"2024-09-21T06:13:24.610027+0000","last_peered":"2024-09-21T06:13:24.610027+0000","last_clean":"2024-09-21T06:13:24.610027+0000","last_became_active":"2024-09-21T06:13:10.502418+0000","last_became_peered":"2024-09-21T06:13:10.502418+0000","last_unstale":"2024-09-21T06:13:24.610027+0000","last_undegraded":"2024-09-21T06:13:24.610027+0000","last_fullsized":"2024-09-21T06:13:24.610027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:57.575569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603439+0000","last_change":"2024-09-21T06:13:08.487191+0000","last_active":"2024-09-21T06:13:24.603439+0000","last_peered":"2024-09-21T06:13:24.603439+0000","last_clean":"2024-09-21T06:13:24.603439+0000","last_became_active":"2024-09-21T06:13:08.486812+0000","last_became_peered":"2024-09-21T06:13:08.486812+0000","last_unstale":"2024-09-21T06:13:24.603439+0000","last_undegraded":"2024-09-21T06:13:24.603439+0000","last_fullsized":"2024-09-21T06:13:24.603439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:51.989708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610179+0000","last_change":"2024-09-21T06:13:10.511678+0000","last_active":"2024-09-21T06:13:24.610179+0000","last_peered":"2024-09-21T06:13:24.610179+0000","last_clean":"2024-09-21T06:13:24.610179+0000","last_became_active":"2024-09-21T06:13:10.510408+0000","last_became_peered":"2024-09-21T06:13:10.510408+0000","last_unstale":"2024-09-21T06:13:24.610179+0000","last_undegraded":"2024-09-21T06:13:24.610179+0000","last_fullsized":"2024-09-21T06:13:24.610179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:58.711190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'6","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:33.630194+0000","last_change":"2024-09-21T06:13:08.560894+0000","last_active":"2024-09-21T06:13:33.630194+0000","last_peered":"2024-09-21T06:13:33.630194+0000","last_clean":"2024-09-21T06:13:33.630194+0000","last_became_active":"2024-09-21T06:13:08.560423+0000","last_became_peered":"2024-09-21T06:13:08.560423+0000","last_unstale":"2024-09-21T06:13:33.630194+0000","last_undegraded":"2024-09-21T06:13:33.630194+0000","last_fullsized":"2024-09-21T06:13:33.630194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:53.015711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610081+0000","last_change":"2024-09-21T06:13:10.516009+0000","last_active":"2024-09-21T06:13:24.610081+0000","last_peered":"2024-09-21T06:13:24.610081+0000","last_clean":"2024-09-21T06:13:24.610081+0000","last_became_active":"2024-09-21T06:13:10.515669+0000","last_became_peered":"2024-09-21T06:13:10.515669+0000","last_unstale":"2024-09-21T06:13:24.610081+0000","last_undegraded":"2024-09-21T06:13:24.610081+0000","last_fullsized":"2024-09-21T06:13:24.610081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:54.134564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":99,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610317+0000","last_change":"2024-09-21T06:11:53.408030+0000","last_active":"2024-09-21T06:13:24.610317+0000","last_peered":"2024-09-21T06:13:24.610317+0000","last_clean":"2024-09-21T06:13:24.610317+0000","last_became_active":"2024-09-21T06:11:53.402296+0000","last_became_peered":"2024-09-21T06:11:53.402296+0000","last_unstale":"2024-09-21T06:13:24.610317+0000","last_undegraded":"2024-09-21T06:13:24.610317+0000","last_fullsized":"2024-09-21T06:13:24.610317+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_clean_scrub_stamp":"2024-09-21T06:08:26.098183+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:08.239872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616959+0000","last_change":"2024-09-21T06:13:08.489209+0000","last_active":"2024-09-21T06:13:24.616959+0000","last_peered":"2024-09-21T06:13:24.616959+0000","last_clean":"2024-09-21T06:13:24.616959+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.616959+0000","last_undegraded":"2024-09-21T06:13:24.616959+0000","last_fullsized":"2024-09-21T06:13:24.616959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:38.798499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607003+0000","last_change":"2024-09-21T06:13:24.606487+0000","last_active":"2024-09-21T06:13:24.607003+0000","last_peered":"2024-09-21T06:13:24.607003+0000","last_clean":"2024-09-21T06:13:24.607003+0000","last_became_active":"2024-09-21T06:13:24.606141+0000","last_became_peered":"2024-09-21T06:13:24.606141+0000","last_unstale":"2024-09-21T06:13:24.607003+0000","last_undegraded":"2024-09-21T06:13:24.607003+0000","last_fullsized":"2024-09-21T06:13:24.607003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:24.040770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609976+0000","last_change":"2024-09-21T06:13:08.489206+0000","last_active":"2024-09-21T06:13:24.609976+0000","last_peered":"2024-09-21T06:13:24.609976+0000","last_clean":"2024-09-21T06:13:24.609976+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.609976+0000","last_undegraded":"2024-09-21T06:13:24.609976+0000","last_fullsized":"2024-09-21T06:13:24.609976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.627305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597214+0000","last_change":"2024-09-21T06:13:10.513999+0000","last_active":"2024-09-21T06:13:24.597214+0000","last_peered":"2024-09-21T06:13:24.597214+0000","last_clean":"2024-09-21T06:13:24.597214+0000","last_became_active":"2024-09-21T06:13:10.513657+0000","last_became_peered":"2024-09-21T06:13:10.513657+0000","last_unstale":"2024-09-21T06:13:24.597214+0000","last_undegraded":"2024-09-21T06:13:24.597214+0000","last_fullsized":"2024-09-21T06:13:24.597214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:05.536921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'5","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:32.633741+0000","last_change":"2024-09-21T06:13:08.560811+0000","last_active":"2024-09-21T06:13:32.633741+0000","last_peered":"2024-09-21T06:13:32.633741+0000","last_clean":"2024-09-21T06:13:32.633741+0000","last_became_active":"2024-09-21T06:13:08.560145+0000","last_became_peered":"2024-09-21T06:13:08.560145+0000","last_unstale":"2024-09-21T06:13:32.633741+0000","last_undegraded":"2024-09-21T06:13:32.633741+0000","last_fullsized":"2024-09-21T06:13:32.633741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:24.550008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.613627+0000","last_change":"2024-09-21T06:13:24.613495+0000","last_active":"2024-09-21T06:13:24.613627+0000","last_peered":"2024-09-21T06:13:24.613627+0000","last_clean":"2024-09-21T06:13:24.613627+0000","last_became_active":"2024-09-21T06:13:24.612634+0000","last_became_peered":"2024-09-21T06:13:24.612634+0000","last_unstale":"2024-09-21T06:13:24.613627+0000","last_undegraded":"2024-09-21T06:13:24.613627+0000","last_fullsized":"2024-09-21T06:13:24.613627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.863141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'5","reported_seq":40,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:32.625052+0000","last_change":"2024-09-21T06:13:08.486973+0000","last_active":"2024-09-21T06:13:32.625052+0000","last_peered":"2024-09-21T06:13:32.625052+0000","last_clean":"2024-09-21T06:13:32.625052+0000","last_became_active":"2024-09-21T06:13:08.486602+0000","last_became_peered":"2024-09-21T06:13:08.486602+0000","last_unstale":"2024-09-21T06:13:32.625052+0000","last_undegraded":"2024-09-21T06:13:32.625052+0000","last_fullsized":"2024-09-21T06:13:32.625052+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:41.804955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.599893+0000","last_change":"2024-09-21T06:13:10.509499+0000","last_active":"2024-09-21T06:13:24.599893+0000","last_peered":"2024-09-21T06:13:24.599893+0000","last_clean":"2024-09-21T06:13:24.599893+0000","last_became_active":"2024-09-21T06:13:10.508566+0000","last_became_peered":"2024-09-21T06:13:10.508566+0000","last_unstale":"2024-09-21T06:13:24.599893+0000","last_undegraded":"2024-09-21T06:13:24.599893+0000","last_fullsized":"2024-09-21T06:13:24.599893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:12.701631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600737+0000","last_change":"2024-09-21T06:13:08.561378+0000","last_active":"2024-09-21T06:13:24.600737+0000","last_peered":"2024-09-21T06:13:24.600737+0000","last_clean":"2024-09-21T06:13:24.600737+0000","last_became_active":"2024-09-21T06:13:08.561002+0000","last_became_peered":"2024-09-21T06:13:08.561002+0000","last_unstale":"2024-09-21T06:13:24.600737+0000","last_undegraded":"2024-09-21T06:13:24.600737+0000","last_fullsized":"2024-09-21T06:13:24.600737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:11.325377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602794+0000","last_change":"2024-09-21T06:13:24.602347+0000","last_active":"2024-09-21T06:13:24.602794+0000","last_peered":"2024-09-21T06:13:24.602794+0000","last_clean":"2024-09-21T06:13:24.602794+0000","last_became_active":"2024-09-21T06:13:24.601978+0000","last_became_peered":"2024-09-21T06:13:24.601978+0000","last_unstale":"2024-09-21T06:13:24.602794+0000","last_undegraded":"2024-09-21T06:13:24.602794+0000","last_fullsized":"2024-09-21T06:13:24.602794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:34.860096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608569+0000","last_change":"2024-09-21T06:13:08.561937+0000","last_active":"2024-09-21T06:13:24.608569+0000","last_peered":"2024-09-21T06:13:24.608569+0000","last_clean":"2024-09-21T06:13:24.608569+0000","last_became_active":"2024-09-21T06:13:08.560547+0000","last_became_peered":"2024-09-21T06:13:08.560547+0000","last_unstale":"2024-09-21T06:13:24.608569+0000","last_undegraded":"2024-09-21T06:13:24.608569+0000","last_fullsized":"2024-09-21T06:13:24.608569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:29.702291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597628+0000","last_change":"2024-09-21T06:13:10.516833+0000","last_active":"2024-09-21T06:13:24.597628+0000","last_peered":"2024-09-21T06:13:24.597628+0000","last_clean":"2024-09-21T06:13:24.597628+0000","last_became_active":"2024-09-21T06:13:10.516486+0000","last_became_peered":"2024-09-21T06:13:10.516486+0000","last_unstale":"2024-09-21T06:13:24.597628+0000","last_undegraded":"2024-09-21T06:13:24.597628+0000","last_fullsized":"2024-09-21T06:13:24.597628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:05.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'13","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:33.499117+0000","last_change":"2024-09-21T06:13:08.483271+0000","last_active":"2024-09-21T06:13:33.499117+0000","last_peered":"2024-09-21T06:13:33.499117+0000","last_clean":"2024-09-21T06:13:33.499117+0000","last_became_active":"2024-09-21T06:13:08.482951+0000","last_became_peered":"2024-09-21T06:13:08.482951+0000","last_unstale":"2024-09-21T06:13:33.499117+0000","last_undegraded":"2024-09-21T06:13:33.499117+0000","last_fullsized":"2024-09-21T06:13:33.499117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:17.537988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610524+0000","last_change":"2024-09-21T06:13:10.509125+0000","last_active":"2024-09-21T06:13:24.610524+0000","last_peered":"2024-09-21T06:13:24.610524+0000","last_clean":"2024-09-21T06:13:24.610524+0000","last_became_active":"2024-09-21T06:13:10.508773+0000","last_became_peered":"2024-09-21T06:13:10.508773+0000","last_unstale":"2024-09-21T06:13:24.610524+0000","last_undegraded":"2024-09-21T06:13:24.610524+0000","last_fullsized":"2024-09-21T06:13:24.610524+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:44.570477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616932+0000","last_change":"2024-09-21T06:13:08.497885+0000","last_active":"2024-09-21T06:13:24.616932+0000","last_peered":"2024-09-21T06:13:24.616932+0000","last_clean":"2024-09-21T06:13:24.616932+0000","last_became_active":"2024-09-21T06:13:08.497593+0000","last_became_peered":"2024-09-21T06:13:08.497593+0000","last_unstale":"2024-09-21T06:13:24.616932+0000","last_undegraded":"2024-09-21T06:13:24.616932+0000","last_fullsized":"2024-09-21T06:13:24.616932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:48.361175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609527+0000","last_change":"2024-09-21T06:13:10.514430+0000","last_active":"2024-09-21T06:13:24.609527+0000","last_peered":"2024-09-21T06:13:24.609527+0000","last_clean":"2024-09-21T06:13:24.609527+0000","last_became_active":"2024-09-21T06:13:10.514063+0000","last_became_peered":"2024-09-21T06:13:10.514063+0000","last_unstale":"2024-09-21T06:13:24.609527+0000","last_undegraded":"2024-09-21T06:13:24.609527+0000","last_fullsized":"2024-09-21T06:13:24.609527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:37.036497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598415+0000","last_change":"2024-09-21T06:13:08.560564+0000","last_active":"2024-09-21T06:13:24.598415+0000","last_peered":"2024-09-21T06:13:24.598415+0000","last_clean":"2024-09-21T06:13:24.598415+0000","last_became_active":"2024-09-21T06:13:08.560339+0000","last_became_peered":"2024-09-21T06:13:08.560339+0000","last_unstale":"2024-09-21T06:13:24.598415+0000","last_undegraded":"2024-09-21T06:13:24.598415+0000","last_fullsized":"2024-09-21T06:13:24.598415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:55.378911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.605861+0000","last_change":"2024-09-21T06:13:10.511354+0000","last_active":"2024-09-21T06:13:24.605861+0000","last_peered":"2024-09-21T06:13:24.605861+0000","last_clean":"2024-09-21T06:13:24.605861+0000","last_became_active":"2024-09-21T06:13:10.510946+0000","last_became_peered":"2024-09-21T06:13:10.510946+0000","last_unstale":"2024-09-21T06:13:24.605861+0000","last_undegraded":"2024-09-21T06:13:24.605861+0000","last_fullsized":"2024-09-21T06:13:24.605861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:11.535461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610883+0000","last_change":"2024-09-21T06:13:08.485578+0000","last_active":"2024-09-21T06:13:24.610883+0000","last_peered":"2024-09-21T06:13:24.610883+0000","last_clean":"2024-09-21T06:13:24.610883+0000","last_became_active":"2024-09-21T06:13:08.485170+0000","last_became_peered":"2024-09-21T06:13:08.485170+0000","last_unstale":"2024-09-21T06:13:24.610883+0000","last_undegraded":"2024-09-21T06:13:24.610883+0000","last_fullsized":"2024-09-21T06:13:24.610883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:39.000114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.606691+0000","last_change":"2024-09-21T06:13:10.510360+0000","last_active":"2024-09-21T06:13:24.606691+0000","last_peered":"2024-09-21T06:13:24.606691+0000","last_clean":"2024-09-21T06:13:24.606691+0000","last_became_active":"2024-09-21T06:13:10.509887+0000","last_became_peered":"2024-09-21T06:13:10.509887+0000","last_unstale":"2024-09-21T06:13:24.606691+0000","last_undegraded":"2024-09-21T06:13:24.606691+0000","last_fullsized":"2024-09-21T06:13:24.606691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:58.135706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.625360+0000","last_change":"2024-09-21T06:13:24.625284+0000","last_active":"2024-09-21T06:13:24.625360+0000","last_peered":"2024-09-21T06:13:24.625360+0000","last_clean":"2024-09-21T06:13:24.625360+0000","last_became_active":"2024-09-21T06:13:24.612704+0000","last_became_peered":"2024-09-21T06:13:24.612704+0000","last_unstale":"2024-09-21T06:13:24.625360+0000","last_undegraded":"2024-09-21T06:13:24.625360+0000","last_fullsized":"2024-09-21T06:13:24.625360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:29.385258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.611377+0000","last_change":"2024-09-21T06:13:10.514128+0000","last_active":"2024-09-21T06:13:24.611377+0000","last_peered":"2024-09-21T06:13:24.611377+0000","last_clean":"2024-09-21T06:13:24.611377+0000","last_became_active":"2024-09-21T06:13:10.513790+0000","last_became_peered":"2024-09-21T06:13:10.513790+0000","last_unstale":"2024-09-21T06:13:24.611377+0000","last_undegraded":"2024-09-21T06:13:24.611377+0000","last_fullsized":"2024-09-21T06:13:24.611377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:33.407189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607762+0000","last_change":"2024-09-21T06:13:08.489105+0000","last_active":"2024-09-21T06:13:24.607762+0000","last_peered":"2024-09-21T06:13:24.607762+0000","last_clean":"2024-09-21T06:13:24.607762+0000","last_became_active":"2024-09-21T06:13:08.487110+0000","last_became_peered":"2024-09-21T06:13:08.487110+0000","last_unstale":"2024-09-21T06:13:24.607762+0000","last_undegraded":"2024-09-21T06:13:24.607762+0000","last_fullsized":"2024-09-21T06:13:24.607762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:24.849365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610194+0000","last_change":"2024-09-21T06:13:10.513751+0000","last_active":"2024-09-21T06:13:24.610194+0000","last_peered":"2024-09-21T06:13:24.610194+0000","last_clean":"2024-09-21T06:13:24.610194+0000","last_became_active":"2024-09-21T06:13:10.513376+0000","last_became_peered":"2024-09-21T06:13:10.513376+0000","last_unstale":"2024-09-21T06:13:24.610194+0000","last_undegraded":"2024-09-21T06:13:24.610194+0000","last_fullsized":"2024-09-21T06:13:24.610194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.364959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603110+0000","last_change":"2024-09-21T06:13:10.519359+0000","last_active":"2024-09-21T06:13:24.603110+0000","last_peered":"2024-09-21T06:13:24.603110+0000","last_clean":"2024-09-21T06:13:24.603110+0000","last_became_active":"2024-09-21T06:13:10.518518+0000","last_became_peered":"2024-09-21T06:13:10.518518+0000","last_unstale":"2024-09-21T06:13:24.603110+0000","last_undegraded":"2024-09-21T06:13:24.603110+0000","last_fullsized":"2024-09-21T06:13:24.603110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:55.554710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608166+0000","last_change":"2024-09-21T06:13:08.488122+0000","last_active":"2024-09-21T06:13:24.608166+0000","last_peered":"2024-09-21T06:13:24.608166+0000","last_clean":"2024-09-21T06:13:24.608166+0000","last_became_active":"2024-09-21T06:13:08.487679+0000","last_became_peered":"2024-09-21T06:13:08.487679+0000","last_unstale":"2024-09-21T06:13:24.608166+0000","last_undegraded":"2024-09-21T06:13:24.608166+0000","last_fullsized":"2024-09-21T06:13:24.608166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:12.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598434+0000","last_change":"2024-09-21T06:13:10.516794+0000","last_active":"2024-09-21T06:13:24.598434+0000","last_peered":"2024-09-21T06:13:24.598434+0000","last_clean":"2024-09-21T06:13:24.598434+0000","last_became_active":"2024-09-21T06:13:10.516297+0000","last_became_peered":"2024-09-21T06:13:10.516297+0000","last_unstale":"2024-09-21T06:13:24.598434+0000","last_undegraded":"2024-09-21T06:13:24.598434+0000","last_fullsized":"2024-09-21T06:13:24.598434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:19.873742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.621488+0000","last_change":"2024-09-21T06:13:24.621358+0000","last_active":"2024-09-21T06:13:24.621488+0000","last_peered":"2024-09-21T06:13:24.621488+0000","last_clean":"2024-09-21T06:13:24.621488+0000","last_became_active":"2024-09-21T06:13:24.611058+0000","last_became_peered":"2024-09-21T06:13:24.611058+0000","last_unstale":"2024-09-21T06:13:24.621488+0000","last_undegraded":"2024-09-21T06:13:24.621488+0000","last_fullsized":"2024-09-21T06:13:24.621488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:28.759029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609712+0000","last_change":"2024-09-21T06:13:10.516339+0000","last_active":"2024-09-21T06:13:24.609712+0000","last_peered":"2024-09-21T06:13:24.609712+0000","last_clean":"2024-09-21T06:13:24.609712+0000","last_became_active":"2024-09-21T06:13:10.515397+0000","last_became_peered":"2024-09-21T06:13:10.515397+0000","last_unstale":"2024-09-21T06:13:24.609712+0000","last_undegraded":"2024-09-21T06:13:24.609712+0000","last_fullsized":"2024-09-21T06:13:24.609712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:56.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610156+0000","last_change":"2024-09-21T06:13:08.560494+0000","last_active":"2024-09-21T06:13:24.610156+0000","last_peered":"2024-09-21T06:13:24.610156+0000","last_clean":"2024-09-21T06:13:24.610156+0000","last_became_active":"2024-09-21T06:13:08.560118+0000","last_became_peered":"2024-09-21T06:13:08.560118+0000","last_unstale":"2024-09-21T06:13:24.610156+0000","last_undegraded":"2024-09-21T06:13:24.610156+0000","last_fullsized":"2024-09-21T06:13:24.610156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:05.597233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610032+0000","last_change":"2024-09-21T06:13:10.515780+0000","last_active":"2024-09-21T06:13:24.610032+0000","last_peered":"2024-09-21T06:13:24.610032+0000","last_clean":"2024-09-21T06:13:24.610032+0000","last_became_active":"2024-09-21T06:13:10.515437+0000","last_became_peered":"2024-09-21T06:13:10.515437+0000","last_unstale":"2024-09-21T06:13:24.610032+0000","last_undegraded":"2024-09-21T06:13:24.610032+0000","last_fullsized":"2024-09-21T06:13:24.610032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:04.848003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'13","reported_seq":50,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:34.626991+0000","last_change":"2024-09-21T06:13:08.483300+0000","last_active":"2024-09-21T06:13:34.626991+0000","last_peered":"2024-09-21T06:13:34.626991+0000","last_clean":"2024-09-21T06:13:34.626991+0000","last_became_active":"2024-09-21T06:13:08.482908+0000","last_became_peered":"2024-09-21T06:13:08.482908+0000","last_unstale":"2024-09-21T06:13:34.626991+0000","last_undegraded":"2024-09-21T06:13:34.626991+0000","last_fullsized":"2024-09-21T06:13:34.626991+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:09.202075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66178,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":155,"num_read_kb":140,"num_write":237,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1724,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":186292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282438,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678033,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":864256,"data_stored":427523,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874263,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1064893,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037789,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":427179,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234018,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":427440,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430246,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":1020349,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593771,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":426779,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790001,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1019636,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":485614,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":426487,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378689,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":428582,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542214,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":474145,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705738,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":892928,"data_stored":428509,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":48205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":59674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:13:38.923 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:13:39.186 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:39.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.647+0000 7f777d631640 1 -- 172.21.15.3:0/3389489941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7778105380 msgr2=0x7f777810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.647+0000 7f777d631640 1 --2- 172.21.15.3:0/3389489941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7778105380 0x7f777810d8d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f776c009900 tx=0x7f776c02a410 comp rx=0 tx=0).stop 2024-09-21T06:13:39.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.647+0000 7f777d631640 1 -- 172.21.15.3:0/3389489941 shutdown_connections 2024-09-21T06:13:39.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.647+0000 7f777d631640 1 --2- 172.21.15.3:0/3389489941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f777810de60 0x7f7778110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.647+0000 7f777d631640 1 --2- 172.21.15.3:0/3389489941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7778105380 0x7f777810d8d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.647+0000 7f777d631640 1 --2- 172.21.15.3:0/3389489941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7778104a30 0x7f7778104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.647+0000 7f777d631640 1 -- 172.21.15.3:0/3389489941 >> 172.21.15.3:0/3389489941 conn(0x7f7778079760 msgr2=0x7f7778079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:39.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.647+0000 7f777d631640 1 -- 172.21.15.3:0/3389489941 shutdown_connections 2024-09-21T06:13:39.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.647+0000 7f777d631640 1 -- 172.21.15.3:0/3389489941 wait complete. 2024-09-21T06:13:39.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.648+0000 7f777d631640 1 Processor -- start 2024-09-21T06:13:39.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.648+0000 7f777d631640 1 -- start start 2024-09-21T06:13:39.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.648+0000 7f777d631640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7778104a30 0x7f777819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.648+0000 7f777d631640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7778105380 0x7f777819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.648+0000 7f777d631640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f777810de60 0x7f777819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.648+0000 7f777d631640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7778113aa0 con 0x7f777810de60 2024-09-21T06:13:39.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.648+0000 7f777d631640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7778113920 con 0x7f7778105380 2024-09-21T06:13:39.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.648+0000 7f777d631640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7778113c20 con 0x7f7778104a30 2024-09-21T06:13:39.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.648+0000 7f77777fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f777810de60 0x7f777819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.648+0000 7f77777fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f777810de60 0x7f777819fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44104/0 (socket says 172.21.15.3:44104) 2024-09-21T06:13:39.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.648+0000 7f77777fe640 1 -- 172.21.15.3:0/4131131214 learned_addr learned my addr 172.21.15.3:0/4131131214 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:39.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.649+0000 7f77767fc640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7778105380 0x7f777819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.649+0000 7f7776ffd640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7778104a30 0x7f777819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.649+0000 7f77777fe640 1 -- 172.21.15.3:0/4131131214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7778104a30 msgr2=0x7f777819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.649+0000 7f77777fe640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7778104a30 0x7f777819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.649+0000 7f77777fe640 1 -- 172.21.15.3:0/4131131214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7778105380 msgr2=0x7f777819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.649+0000 7f77777fe640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7778105380 0x7f777819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.649+0000 7f77777fe640 1 -- 172.21.15.3:0/4131131214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77781a0360 con 0x7f777810de60 2024-09-21T06:13:39.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.649+0000 7f77777fe640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f777810de60 0x7f777819fcf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f777000d950 tx=0x7f777000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:39.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.649+0000 7f775ffff640 1 -- 172.21.15.3:0/4131131214 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770017030 con 0x7f777810de60 2024-09-21T06:13:39.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.649+0000 7f777d631640 1 -- 172.21.15.3:0/4131131214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77781a0650 con 0x7f777810de60 2024-09-21T06:13:39.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.649+0000 7f775ffff640 1 -- 172.21.15.3:0/4131131214 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7770004020 con 0x7f777810de60 2024-09-21T06:13:39.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.649+0000 7f775ffff640 1 -- 172.21.15.3:0/4131131214 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770003b50 con 0x7f777810de60 2024-09-21T06:13:39.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.649+0000 7f777d631640 1 -- 172.21.15.3:0/4131131214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77781a6e70 con 0x7f777810de60 2024-09-21T06:13:39.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.650+0000 7f777d631640 1 -- 172.21.15.3:0/4131131214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f777806c1f0 con 0x7f777810de60 2024-09-21T06:13:39.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.652+0000 7f775ffff640 1 -- 172.21.15.3:0/4131131214 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f777001a070 con 0x7f777810de60 2024-09-21T06:13:39.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.653+0000 7f775ffff640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f77600b6620 0x7f77600b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:39.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.653+0000 7f7776ffd640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f77600b6620 0x7f77600b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:39.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.653+0000 7f775ffff640 1 -- 172.21.15.3:0/4131131214 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f77700e6fc0 con 0x7f777810de60 2024-09-21T06:13:39.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.654+0000 7f7776ffd640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f77600b6620 0x7f77600b8b30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f777806c8f0 tx=0x7f7768007600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:39.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.656+0000 7f775ffff640 1 -- 172.21.15.3:0/4131131214 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7770021070 con 0x7f777810de60 2024-09-21T06:13:39.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.841+0000 7f777d631640 1 -- 172.21.15.3:0/4131131214 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7778063800 con 0x7f77600b6620 2024-09-21T06:13:39.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.845+0000 7f775ffff640 1 -- 172.21.15.3:0/4131131214 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259023 (secure 0 0 0) 0x7f7778063800 con 0x7f77600b6620 2024-09-21T06:13:39.849 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:13:39.850 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-21T06:13:39.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.848+0000 7f777d631640 1 -- 172.21.15.3:0/4131131214 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f77600b6620 msgr2=0x7f77600b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.848+0000 7f777d631640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f77600b6620 0x7f77600b8b30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f777806c8f0 tx=0x7f7768007600 comp rx=0 tx=0).stop 2024-09-21T06:13:39.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.848+0000 7f777d631640 1 -- 172.21.15.3:0/4131131214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f777810de60 msgr2=0x7f777819fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:39.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.848+0000 7f777d631640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f777810de60 0x7f777819fcf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f777000d950 tx=0x7f777000de10 comp rx=0 tx=0).stop 2024-09-21T06:13:39.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.848+0000 7f777d631640 1 -- 172.21.15.3:0/4131131214 shutdown_connections 2024-09-21T06:13:39.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.848+0000 7f777d631640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f77600b6620 0x7f77600b8b30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.848+0000 7f777d631640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f777810de60 0x7f777819fcf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.848+0000 7f777d631640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7778105380 0x7f777819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.848+0000 7f777d631640 1 --2- 172.21.15.3:0/4131131214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7778104a30 0x7f777819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:39.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.848+0000 7f777d631640 1 -- 172.21.15.3:0/4131131214 >> 172.21.15.3:0/4131131214 conn(0x7f7778079760 msgr2=0x7f77781009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:39.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.849+0000 7f777d631640 1 -- 172.21.15.3:0/4131131214 shutdown_connections 2024-09-21T06:13:39.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:39.849+0000 7f777d631640 1 -- 172.21.15.3:0/4131131214 wait complete. 2024-09-21T06:13:40.666 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-21T06:13:39.623742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656780,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":251,"num_read_kb":214,"num_write":362,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2953564,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359940,"kb_used_data":10680,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581884,"statfs":{"total":1151940427776,"available":1151571849216,"internally_reserved":0,"allocated":10936320,"data_stored":7057136,"data_compressed":177687,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51387,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":-8,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":104,"num_write":31,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003258"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607638+0000","last_change":"2024-09-21T06:13:10.515115+0000","last_active":"2024-09-21T06:13:24.607638+0000","last_peered":"2024-09-21T06:13:24.607638+0000","last_clean":"2024-09-21T06:13:24.607638+0000","last_became_active":"2024-09-21T06:13:10.514553+0000","last_became_peered":"2024-09-21T06:13:10.514553+0000","last_unstale":"2024-09-21T06:13:24.607638+0000","last_undegraded":"2024-09-21T06:13:24.607638+0000","last_fullsized":"2024-09-21T06:13:24.607638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:33.268926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609526+0000","last_change":"2024-09-21T06:13:10.511469+0000","last_active":"2024-09-21T06:13:24.609526+0000","last_peered":"2024-09-21T06:13:24.609526+0000","last_clean":"2024-09-21T06:13:24.609526+0000","last_became_active":"2024-09-21T06:13:10.511102+0000","last_became_peered":"2024-09-21T06:13:10.511102+0000","last_unstale":"2024-09-21T06:13:24.609526+0000","last_undegraded":"2024-09-21T06:13:24.609526+0000","last_fullsized":"2024-09-21T06:13:24.609526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:12.483821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609288+0000","last_change":"2024-09-21T06:13:10.520090+0000","last_active":"2024-09-21T06:13:24.609288+0000","last_peered":"2024-09-21T06:13:24.609288+0000","last_clean":"2024-09-21T06:13:24.609288+0000","last_became_active":"2024-09-21T06:13:10.519758+0000","last_became_peered":"2024-09-21T06:13:10.519758+0000","last_unstale":"2024-09-21T06:13:24.609288+0000","last_undegraded":"2024-09-21T06:13:24.609288+0000","last_fullsized":"2024-09-21T06:13:24.609288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:20.836847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.617812+0000","last_change":"2024-09-21T06:13:10.511732+0000","last_active":"2024-09-21T06:13:24.617812+0000","last_peered":"2024-09-21T06:13:24.617812+0000","last_clean":"2024-09-21T06:13:24.617812+0000","last_became_active":"2024-09-21T06:13:10.511377+0000","last_became_peered":"2024-09-21T06:13:10.511377+0000","last_unstale":"2024-09-21T06:13:24.617812+0000","last_undegraded":"2024-09-21T06:13:24.617812+0000","last_fullsized":"2024-09-21T06:13:24.617812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:18.500739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607690+0000","last_change":"2024-09-21T06:13:10.510464+0000","last_active":"2024-09-21T06:13:24.607690+0000","last_peered":"2024-09-21T06:13:24.607690+0000","last_clean":"2024-09-21T06:13:24.607690+0000","last_became_active":"2024-09-21T06:13:10.509979+0000","last_became_peered":"2024-09-21T06:13:10.509979+0000","last_unstale":"2024-09-21T06:13:24.607690+0000","last_undegraded":"2024-09-21T06:13:24.607690+0000","last_fullsized":"2024-09-21T06:13:24.607690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:56.871043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608937+0000","last_change":"2024-09-21T06:13:24.608832+0000","last_active":"2024-09-21T06:13:24.608937+0000","last_peered":"2024-09-21T06:13:24.608937+0000","last_clean":"2024-09-21T06:13:24.608937+0000","last_became_active":"2024-09-21T06:13:24.608439+0000","last_became_peered":"2024-09-21T06:13:24.608439+0000","last_unstale":"2024-09-21T06:13:24.608937+0000","last_undegraded":"2024-09-21T06:13:24.608937+0000","last_fullsized":"2024-09-21T06:13:24.608937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:41.515894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609585+0000","last_change":"2024-09-21T06:13:10.511391+0000","last_active":"2024-09-21T06:13:24.609585+0000","last_peered":"2024-09-21T06:13:24.609585+0000","last_clean":"2024-09-21T06:13:24.609585+0000","last_became_active":"2024-09-21T06:13:10.510733+0000","last_became_peered":"2024-09-21T06:13:10.510733+0000","last_unstale":"2024-09-21T06:13:24.609585+0000","last_undegraded":"2024-09-21T06:13:24.609585+0000","last_fullsized":"2024-09-21T06:13:24.609585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:24.071860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609673+0000","last_change":"2024-09-21T06:13:10.510886+0000","last_active":"2024-09-21T06:13:24.609673+0000","last_peered":"2024-09-21T06:13:24.609673+0000","last_clean":"2024-09-21T06:13:24.609673+0000","last_became_active":"2024-09-21T06:13:10.510159+0000","last_became_peered":"2024-09-21T06:13:10.510159+0000","last_unstale":"2024-09-21T06:13:24.609673+0000","last_undegraded":"2024-09-21T06:13:24.609673+0000","last_fullsized":"2024-09-21T06:13:24.609673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:37.124848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598718+0000","last_change":"2024-09-21T06:13:10.510466+0000","last_active":"2024-09-21T06:13:24.598718+0000","last_peered":"2024-09-21T06:13:24.598718+0000","last_clean":"2024-09-21T06:13:24.598718+0000","last_became_active":"2024-09-21T06:13:10.509928+0000","last_became_peered":"2024-09-21T06:13:10.509928+0000","last_unstale":"2024-09-21T06:13:24.598718+0000","last_undegraded":"2024-09-21T06:13:24.598718+0000","last_fullsized":"2024-09-21T06:13:24.598718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:06.156505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600080+0000","last_change":"2024-09-21T06:13:10.510622+0000","last_active":"2024-09-21T06:13:24.600080+0000","last_peered":"2024-09-21T06:13:24.600080+0000","last_clean":"2024-09-21T06:13:24.600080+0000","last_became_active":"2024-09-21T06:13:10.510206+0000","last_became_peered":"2024-09-21T06:13:10.510206+0000","last_unstale":"2024-09-21T06:13:24.600080+0000","last_undegraded":"2024-09-21T06:13:24.600080+0000","last_fullsized":"2024-09-21T06:13:24.600080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:03.223451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610031+0000","last_change":"2024-09-21T06:13:10.517525+0000","last_active":"2024-09-21T06:13:24.610031+0000","last_peered":"2024-09-21T06:13:24.610031+0000","last_clean":"2024-09-21T06:13:24.610031+0000","last_became_active":"2024-09-21T06:13:10.517144+0000","last_became_peered":"2024-09-21T06:13:10.517144+0000","last_unstale":"2024-09-21T06:13:24.610031+0000","last_undegraded":"2024-09-21T06:13:24.610031+0000","last_fullsized":"2024-09-21T06:13:24.610031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:46.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.617837+0000","last_change":"2024-09-21T06:13:10.515537+0000","last_active":"2024-09-21T06:13:24.617837+0000","last_peered":"2024-09-21T06:13:24.617837+0000","last_clean":"2024-09-21T06:13:24.617837+0000","last_became_active":"2024-09-21T06:13:10.515063+0000","last_became_peered":"2024-09-21T06:13:10.515063+0000","last_unstale":"2024-09-21T06:13:24.617837+0000","last_undegraded":"2024-09-21T06:13:24.617837+0000","last_fullsized":"2024-09-21T06:13:24.617837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:33.718471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597513+0000","last_change":"2024-09-21T06:13:10.509368+0000","last_active":"2024-09-21T06:13:24.597513+0000","last_peered":"2024-09-21T06:13:24.597513+0000","last_clean":"2024-09-21T06:13:24.597513+0000","last_became_active":"2024-09-21T06:13:10.508982+0000","last_became_peered":"2024-09-21T06:13:10.508982+0000","last_unstale":"2024-09-21T06:13:24.597513+0000","last_undegraded":"2024-09-21T06:13:24.597513+0000","last_fullsized":"2024-09-21T06:13:24.597513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:54.430889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608602+0000","last_change":"2024-09-21T06:13:10.506559+0000","last_active":"2024-09-21T06:13:24.608602+0000","last_peered":"2024-09-21T06:13:24.608602+0000","last_clean":"2024-09-21T06:13:24.608602+0000","last_became_active":"2024-09-21T06:13:10.506161+0000","last_became_peered":"2024-09-21T06:13:10.506161+0000","last_unstale":"2024-09-21T06:13:24.608602+0000","last_undegraded":"2024-09-21T06:13:24.608602+0000","last_fullsized":"2024-09-21T06:13:24.608602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:19.870292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608304+0000","last_change":"2024-09-21T06:13:10.516000+0000","last_active":"2024-09-21T06:13:24.608304+0000","last_peered":"2024-09-21T06:13:24.608304+0000","last_clean":"2024-09-21T06:13:24.608304+0000","last_became_active":"2024-09-21T06:13:10.515543+0000","last_became_peered":"2024-09-21T06:13:10.515543+0000","last_unstale":"2024-09-21T06:13:24.608304+0000","last_undegraded":"2024-09-21T06:13:24.608304+0000","last_fullsized":"2024-09-21T06:13:24.608304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:48.968489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598302+0000","last_change":"2024-09-21T06:13:10.513753+0000","last_active":"2024-09-21T06:13:24.598302+0000","last_peered":"2024-09-21T06:13:24.598302+0000","last_clean":"2024-09-21T06:13:24.598302+0000","last_became_active":"2024-09-21T06:13:10.511079+0000","last_became_peered":"2024-09-21T06:13:10.511079+0000","last_unstale":"2024-09-21T06:13:24.598302+0000","last_undegraded":"2024-09-21T06:13:24.598302+0000","last_fullsized":"2024-09-21T06:13:24.598302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:25.320041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607768+0000","last_change":"2024-09-21T06:13:10.515222+0000","last_active":"2024-09-21T06:13:24.607768+0000","last_peered":"2024-09-21T06:13:24.607768+0000","last_clean":"2024-09-21T06:13:24.607768+0000","last_became_active":"2024-09-21T06:13:10.514604+0000","last_became_peered":"2024-09-21T06:13:10.514604+0000","last_unstale":"2024-09-21T06:13:24.607768+0000","last_undegraded":"2024-09-21T06:13:24.607768+0000","last_fullsized":"2024-09-21T06:13:24.607768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:03.165695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600779+0000","last_change":"2024-09-21T06:13:10.518334+0000","last_active":"2024-09-21T06:13:24.600779+0000","last_peered":"2024-09-21T06:13:24.600779+0000","last_clean":"2024-09-21T06:13:24.600779+0000","last_became_active":"2024-09-21T06:13:10.517956+0000","last_became_peered":"2024-09-21T06:13:10.517956+0000","last_unstale":"2024-09-21T06:13:24.600779+0000","last_undegraded":"2024-09-21T06:13:24.600779+0000","last_fullsized":"2024-09-21T06:13:24.600779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:05.094454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604993+0000","last_change":"2024-09-21T06:13:10.508763+0000","last_active":"2024-09-21T06:13:24.604993+0000","last_peered":"2024-09-21T06:13:24.604993+0000","last_clean":"2024-09-21T06:13:24.604993+0000","last_became_active":"2024-09-21T06:13:10.508371+0000","last_became_peered":"2024-09-21T06:13:10.508371+0000","last_unstale":"2024-09-21T06:13:24.604993+0000","last_undegraded":"2024-09-21T06:13:24.604993+0000","last_fullsized":"2024-09-21T06:13:24.604993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:50.602692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607019+0000","last_change":"2024-09-21T06:13:10.503465+0000","last_active":"2024-09-21T06:13:24.607019+0000","last_peered":"2024-09-21T06:13:24.607019+0000","last_clean":"2024-09-21T06:13:24.607019+0000","last_became_active":"2024-09-21T06:13:10.503072+0000","last_became_peered":"2024-09-21T06:13:10.503072+0000","last_unstale":"2024-09-21T06:13:24.607019+0000","last_undegraded":"2024-09-21T06:13:24.607019+0000","last_fullsized":"2024-09-21T06:13:24.607019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:36.368987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597579+0000","last_change":"2024-09-21T06:13:10.512841+0000","last_active":"2024-09-21T06:13:24.597579+0000","last_peered":"2024-09-21T06:13:24.597579+0000","last_clean":"2024-09-21T06:13:24.597579+0000","last_became_active":"2024-09-21T06:13:10.512407+0000","last_became_peered":"2024-09-21T06:13:10.512407+0000","last_unstale":"2024-09-21T06:13:24.597579+0000","last_undegraded":"2024-09-21T06:13:24.597579+0000","last_fullsized":"2024-09-21T06:13:24.597579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:09.855608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598794+0000","last_change":"2024-09-21T06:13:10.510088+0000","last_active":"2024-09-21T06:13:24.598794+0000","last_peered":"2024-09-21T06:13:24.598794+0000","last_clean":"2024-09-21T06:13:24.598794+0000","last_became_active":"2024-09-21T06:13:10.507904+0000","last_became_peered":"2024-09-21T06:13:10.507904+0000","last_unstale":"2024-09-21T06:13:24.598794+0000","last_undegraded":"2024-09-21T06:13:24.598794+0000","last_fullsized":"2024-09-21T06:13:24.598794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:49.404402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.611267+0000","last_change":"2024-09-21T06:13:10.517720+0000","last_active":"2024-09-21T06:13:24.611267+0000","last_peered":"2024-09-21T06:13:24.611267+0000","last_clean":"2024-09-21T06:13:24.611267+0000","last_became_active":"2024-09-21T06:13:10.514641+0000","last_became_peered":"2024-09-21T06:13:10.514641+0000","last_unstale":"2024-09-21T06:13:24.611267+0000","last_undegraded":"2024-09-21T06:13:24.611267+0000","last_fullsized":"2024-09-21T06:13:24.611267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:40.639665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602839+0000","last_change":"2024-09-21T06:13:10.512234+0000","last_active":"2024-09-21T06:13:24.602839+0000","last_peered":"2024-09-21T06:13:24.602839+0000","last_clean":"2024-09-21T06:13:24.602839+0000","last_became_active":"2024-09-21T06:13:10.511594+0000","last_became_peered":"2024-09-21T06:13:10.511594+0000","last_unstale":"2024-09-21T06:13:24.602839+0000","last_undegraded":"2024-09-21T06:13:24.602839+0000","last_fullsized":"2024-09-21T06:13:24.602839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:04.410342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610455+0000","last_change":"2024-09-21T06:13:10.513216+0000","last_active":"2024-09-21T06:13:24.610455+0000","last_peered":"2024-09-21T06:13:24.610455+0000","last_clean":"2024-09-21T06:13:24.610455+0000","last_became_active":"2024-09-21T06:13:10.512830+0000","last_became_peered":"2024-09-21T06:13:10.512830+0000","last_unstale":"2024-09-21T06:13:24.610455+0000","last_undegraded":"2024-09-21T06:13:24.610455+0000","last_fullsized":"2024-09-21T06:13:24.610455+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:46.418161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.605196+0000","last_change":"2024-09-21T06:13:10.518863+0000","last_active":"2024-09-21T06:13:24.605196+0000","last_peered":"2024-09-21T06:13:24.605196+0000","last_clean":"2024-09-21T06:13:24.605196+0000","last_became_active":"2024-09-21T06:13:10.518492+0000","last_became_peered":"2024-09-21T06:13:10.518492+0000","last_unstale":"2024-09-21T06:13:24.605196+0000","last_undegraded":"2024-09-21T06:13:24.605196+0000","last_fullsized":"2024-09-21T06:13:24.605196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:23.770763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608235+0000","last_change":"2024-09-21T06:13:10.508538+0000","last_active":"2024-09-21T06:13:24.608235+0000","last_peered":"2024-09-21T06:13:24.608235+0000","last_clean":"2024-09-21T06:13:24.608235+0000","last_became_active":"2024-09-21T06:13:10.508017+0000","last_became_peered":"2024-09-21T06:13:10.508017+0000","last_unstale":"2024-09-21T06:13:24.608235+0000","last_undegraded":"2024-09-21T06:13:24.608235+0000","last_fullsized":"2024-09-21T06:13:24.608235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:00.168741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597567+0000","last_change":"2024-09-21T06:13:10.516734+0000","last_active":"2024-09-21T06:13:24.597567+0000","last_peered":"2024-09-21T06:13:24.597567+0000","last_clean":"2024-09-21T06:13:24.597567+0000","last_became_active":"2024-09-21T06:13:10.516362+0000","last_became_peered":"2024-09-21T06:13:10.516362+0000","last_unstale":"2024-09-21T06:13:24.597567+0000","last_undegraded":"2024-09-21T06:13:24.597567+0000","last_fullsized":"2024-09-21T06:13:24.597567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:49.792011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608841+0000","last_change":"2024-09-21T06:13:10.513143+0000","last_active":"2024-09-21T06:13:24.608841+0000","last_peered":"2024-09-21T06:13:24.608841+0000","last_clean":"2024-09-21T06:13:24.608841+0000","last_became_active":"2024-09-21T06:13:10.512338+0000","last_became_peered":"2024-09-21T06:13:10.512338+0000","last_unstale":"2024-09-21T06:13:24.608841+0000","last_undegraded":"2024-09-21T06:13:24.608841+0000","last_fullsized":"2024-09-21T06:13:24.608841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:02.963860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602980+0000","last_change":"2024-09-21T06:13:10.513429+0000","last_active":"2024-09-21T06:13:24.602980+0000","last_peered":"2024-09-21T06:13:24.602980+0000","last_clean":"2024-09-21T06:13:24.602980+0000","last_became_active":"2024-09-21T06:13:10.512995+0000","last_became_peered":"2024-09-21T06:13:10.512995+0000","last_unstale":"2024-09-21T06:13:24.602980+0000","last_undegraded":"2024-09-21T06:13:24.602980+0000","last_fullsized":"2024-09-21T06:13:24.602980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:39.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603628+0000","last_change":"2024-09-21T06:13:10.509580+0000","last_active":"2024-09-21T06:13:24.603628+0000","last_peered":"2024-09-21T06:13:24.603628+0000","last_clean":"2024-09-21T06:13:24.603628+0000","last_became_active":"2024-09-21T06:13:10.509179+0000","last_became_peered":"2024-09-21T06:13:10.509179+0000","last_unstale":"2024-09-21T06:13:24.603628+0000","last_undegraded":"2024-09-21T06:13:24.603628+0000","last_fullsized":"2024-09-21T06:13:24.603628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:39.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.606565+0000","last_change":"2024-09-21T06:13:10.508358+0000","last_active":"2024-09-21T06:13:24.606565+0000","last_peered":"2024-09-21T06:13:24.606565+0000","last_clean":"2024-09-21T06:13:24.606565+0000","last_became_active":"2024-09-21T06:13:10.507952+0000","last_became_peered":"2024-09-21T06:13:10.507952+0000","last_unstale":"2024-09-21T06:13:24.606565+0000","last_undegraded":"2024-09-21T06:13:24.606565+0000","last_fullsized":"2024-09-21T06:13:24.606565+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:55.565203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607615+0000","last_change":"2024-09-21T06:13:10.510580+0000","last_active":"2024-09-21T06:13:24.607615+0000","last_peered":"2024-09-21T06:13:24.607615+0000","last_clean":"2024-09-21T06:13:24.607615+0000","last_became_active":"2024-09-21T06:13:10.510175+0000","last_became_peered":"2024-09-21T06:13:10.510175+0000","last_unstale":"2024-09-21T06:13:24.607615+0000","last_undegraded":"2024-09-21T06:13:24.607615+0000","last_fullsized":"2024-09-21T06:13:24.607615+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:49.467057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":47,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:36.632107+0000","last_change":"2024-09-21T06:13:08.485527+0000","last_active":"2024-09-21T06:13:36.632107+0000","last_peered":"2024-09-21T06:13:36.632107+0000","last_clean":"2024-09-21T06:13:36.632107+0000","last_became_active":"2024-09-21T06:13:08.485094+0000","last_became_peered":"2024-09-21T06:13:08.485094+0000","last_unstale":"2024-09-21T06:13:36.632107+0000","last_undegraded":"2024-09-21T06:13:36.632107+0000","last_fullsized":"2024-09-21T06:13:36.632107+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:48.285638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603582+0000","last_change":"2024-09-21T06:13:10.507977+0000","last_active":"2024-09-21T06:13:24.603582+0000","last_peered":"2024-09-21T06:13:24.603582+0000","last_clean":"2024-09-21T06:13:24.603582+0000","last_became_active":"2024-09-21T06:13:10.507594+0000","last_became_peered":"2024-09-21T06:13:10.507594+0000","last_unstale":"2024-09-21T06:13:24.603582+0000","last_undegraded":"2024-09-21T06:13:24.603582+0000","last_fullsized":"2024-09-21T06:13:24.603582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:57.318240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.599690+0000","last_change":"2024-09-21T06:13:08.491237+0000","last_active":"2024-09-21T06:13:24.599690+0000","last_peered":"2024-09-21T06:13:24.599690+0000","last_clean":"2024-09-21T06:13:24.599690+0000","last_became_active":"2024-09-21T06:13:08.490908+0000","last_became_peered":"2024-09-21T06:13:08.490908+0000","last_unstale":"2024-09-21T06:13:24.599690+0000","last_undegraded":"2024-09-21T06:13:24.599690+0000","last_fullsized":"2024-09-21T06:13:24.599690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:53.185029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597228+0000","last_change":"2024-09-21T06:13:08.488721+0000","last_active":"2024-09-21T06:13:24.597228+0000","last_peered":"2024-09-21T06:13:24.597228+0000","last_clean":"2024-09-21T06:13:24.597228+0000","last_became_active":"2024-09-21T06:13:08.488347+0000","last_became_peered":"2024-09-21T06:13:08.488347+0000","last_unstale":"2024-09-21T06:13:24.597228+0000","last_undegraded":"2024-09-21T06:13:24.597228+0000","last_fullsized":"2024-09-21T06:13:24.597228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:27.008479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609911+0000","last_change":"2024-09-21T06:13:10.506527+0000","last_active":"2024-09-21T06:13:24.609911+0000","last_peered":"2024-09-21T06:13:24.609911+0000","last_clean":"2024-09-21T06:13:24.609911+0000","last_became_active":"2024-09-21T06:13:10.506163+0000","last_became_peered":"2024-09-21T06:13:10.506163+0000","last_unstale":"2024-09-21T06:13:24.609911+0000","last_undegraded":"2024-09-21T06:13:24.609911+0000","last_fullsized":"2024-09-21T06:13:24.609911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.244097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604150+0000","last_change":"2024-09-21T06:13:08.489292+0000","last_active":"2024-09-21T06:13:24.604150+0000","last_peered":"2024-09-21T06:13:24.604150+0000","last_clean":"2024-09-21T06:13:24.604150+0000","last_became_active":"2024-09-21T06:13:08.488918+0000","last_became_peered":"2024-09-21T06:13:08.488918+0000","last_unstale":"2024-09-21T06:13:24.604150+0000","last_undegraded":"2024-09-21T06:13:24.604150+0000","last_fullsized":"2024-09-21T06:13:24.604150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:37.992260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600648+0000","last_change":"2024-09-21T06:13:10.505027+0000","last_active":"2024-09-21T06:13:24.600648+0000","last_peered":"2024-09-21T06:13:24.600648+0000","last_clean":"2024-09-21T06:13:24.600648+0000","last_became_active":"2024-09-21T06:13:10.504669+0000","last_became_peered":"2024-09-21T06:13:10.504669+0000","last_unstale":"2024-09-21T06:13:24.600648+0000","last_undegraded":"2024-09-21T06:13:24.600648+0000","last_fullsized":"2024-09-21T06:13:24.600648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:43.927229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":52,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:35.629799+0000","last_change":"2024-09-21T06:13:08.491101+0000","last_active":"2024-09-21T06:13:35.629799+0000","last_peered":"2024-09-21T06:13:35.629799+0000","last_clean":"2024-09-21T06:13:35.629799+0000","last_became_active":"2024-09-21T06:13:08.490607+0000","last_became_peered":"2024-09-21T06:13:08.490607+0000","last_unstale":"2024-09-21T06:13:35.629799+0000","last_undegraded":"2024-09-21T06:13:35.629799+0000","last_fullsized":"2024-09-21T06:13:35.629799+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:16.758463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598830+0000","last_change":"2024-09-21T06:13:10.510453+0000","last_active":"2024-09-21T06:13:24.598830+0000","last_peered":"2024-09-21T06:13:24.598830+0000","last_clean":"2024-09-21T06:13:24.598830+0000","last_became_active":"2024-09-21T06:13:10.508513+0000","last_became_peered":"2024-09-21T06:13:10.508513+0000","last_unstale":"2024-09-21T06:13:24.598830+0000","last_undegraded":"2024-09-21T06:13:24.598830+0000","last_fullsized":"2024-09-21T06:13:24.598830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:48.388140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609878+0000","last_change":"2024-09-21T06:13:08.488714+0000","last_active":"2024-09-21T06:13:24.609878+0000","last_peered":"2024-09-21T06:13:24.609878+0000","last_clean":"2024-09-21T06:13:24.609878+0000","last_became_active":"2024-09-21T06:13:08.488363+0000","last_became_peered":"2024-09-21T06:13:08.488363+0000","last_unstale":"2024-09-21T06:13:24.609878+0000","last_undegraded":"2024-09-21T06:13:24.609878+0000","last_fullsized":"2024-09-21T06:13:24.609878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:33.626165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609602+0000","last_change":"2024-09-21T06:13:10.508046+0000","last_active":"2024-09-21T06:13:24.609602+0000","last_peered":"2024-09-21T06:13:24.609602+0000","last_clean":"2024-09-21T06:13:24.609602+0000","last_became_active":"2024-09-21T06:13:10.507660+0000","last_became_peered":"2024-09-21T06:13:10.507660+0000","last_unstale":"2024-09-21T06:13:24.609602+0000","last_undegraded":"2024-09-21T06:13:24.609602+0000","last_fullsized":"2024-09-21T06:13:24.609602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:35.466965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603272+0000","last_change":"2024-09-21T06:13:08.562863+0000","last_active":"2024-09-21T06:13:24.603272+0000","last_peered":"2024-09-21T06:13:24.603272+0000","last_clean":"2024-09-21T06:13:24.603272+0000","last_became_active":"2024-09-21T06:13:08.562424+0000","last_became_peered":"2024-09-21T06:13:08.562424+0000","last_unstale":"2024-09-21T06:13:24.603272+0000","last_undegraded":"2024-09-21T06:13:24.603272+0000","last_fullsized":"2024-09-21T06:13:24.603272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:43.251932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598595+0000","last_change":"2024-09-21T06:13:10.509818+0000","last_active":"2024-09-21T06:13:24.598595+0000","last_peered":"2024-09-21T06:13:24.598595+0000","last_clean":"2024-09-21T06:13:24.598595+0000","last_became_active":"2024-09-21T06:13:10.509468+0000","last_became_peered":"2024-09-21T06:13:10.509468+0000","last_unstale":"2024-09-21T06:13:24.598595+0000","last_undegraded":"2024-09-21T06:13:24.598595+0000","last_fullsized":"2024-09-21T06:13:24.598595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:25.130868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'27","reported_seq":100,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:35.908304+0000","last_change":"2024-09-21T06:13:24.633807+0000","last_active":"2024-09-21T06:13:35.908304+0000","last_peered":"2024-09-21T06:13:35.908304+0000","last_clean":"2024-09-21T06:13:35.908304+0000","last_became_active":"2024-09-21T06:13:24.618599+0000","last_became_peered":"2024-09-21T06:13:24.618599+0000","last_unstale":"2024-09-21T06:13:35.908304+0000","last_undegraded":"2024-09-21T06:13:35.908304+0000","last_fullsized":"2024-09-21T06:13:35.908304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:07.449253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58583,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":59,"num_write":29,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608537+0000","last_change":"2024-09-21T06:13:10.511360+0000","last_active":"2024-09-21T06:13:24.608537+0000","last_peered":"2024-09-21T06:13:24.608537+0000","last_clean":"2024-09-21T06:13:24.608537+0000","last_became_active":"2024-09-21T06:13:10.510980+0000","last_became_peered":"2024-09-21T06:13:10.510980+0000","last_unstale":"2024-09-21T06:13:24.608537+0000","last_undegraded":"2024-09-21T06:13:24.608537+0000","last_fullsized":"2024-09-21T06:13:24.608537+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:43.859758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604427+0000","last_change":"2024-09-21T06:13:08.486262+0000","last_active":"2024-09-21T06:13:24.604427+0000","last_peered":"2024-09-21T06:13:24.604427+0000","last_clean":"2024-09-21T06:13:24.604427+0000","last_became_active":"2024-09-21T06:13:08.485509+0000","last_became_peered":"2024-09-21T06:13:08.485509+0000","last_unstale":"2024-09-21T06:13:24.604427+0000","last_undegraded":"2024-09-21T06:13:24.604427+0000","last_fullsized":"2024-09-21T06:13:24.604427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:31.726746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597324+0000","last_change":"2024-09-21T06:13:10.506894+0000","last_active":"2024-09-21T06:13:24.597324+0000","last_peered":"2024-09-21T06:13:24.597324+0000","last_clean":"2024-09-21T06:13:24.597324+0000","last_became_active":"2024-09-21T06:13:10.506489+0000","last_became_peered":"2024-09-21T06:13:10.506489+0000","last_unstale":"2024-09-21T06:13:24.597324+0000","last_undegraded":"2024-09-21T06:13:24.597324+0000","last_fullsized":"2024-09-21T06:13:24.597324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:58.755601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603141+0000","last_change":"2024-09-21T06:13:08.483429+0000","last_active":"2024-09-21T06:13:24.603141+0000","last_peered":"2024-09-21T06:13:24.603141+0000","last_clean":"2024-09-21T06:13:24.603141+0000","last_became_active":"2024-09-21T06:13:08.483028+0000","last_became_peered":"2024-09-21T06:13:08.483028+0000","last_unstale":"2024-09-21T06:13:24.603141+0000","last_undegraded":"2024-09-21T06:13:24.603141+0000","last_fullsized":"2024-09-21T06:13:24.603141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:33.258154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609758+0000","last_change":"2024-09-21T06:13:10.513398+0000","last_active":"2024-09-21T06:13:24.609758+0000","last_peered":"2024-09-21T06:13:24.609758+0000","last_clean":"2024-09-21T06:13:24.609758+0000","last_became_active":"2024-09-21T06:13:10.512713+0000","last_became_peered":"2024-09-21T06:13:10.512713+0000","last_unstale":"2024-09-21T06:13:24.609758+0000","last_undegraded":"2024-09-21T06:13:24.609758+0000","last_fullsized":"2024-09-21T06:13:24.609758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:21.026984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":45,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:35.638107+0000","last_change":"2024-09-21T06:13:08.493771+0000","last_active":"2024-09-21T06:13:35.638107+0000","last_peered":"2024-09-21T06:13:35.638107+0000","last_clean":"2024-09-21T06:13:35.638107+0000","last_became_active":"2024-09-21T06:13:08.493380+0000","last_became_peered":"2024-09-21T06:13:08.493380+0000","last_unstale":"2024-09-21T06:13:35.638107+0000","last_undegraded":"2024-09-21T06:13:35.638107+0000","last_fullsized":"2024-09-21T06:13:35.638107+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:12.552137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609811+0000","last_change":"2024-09-21T06:13:10.511679+0000","last_active":"2024-09-21T06:13:24.609811+0000","last_peered":"2024-09-21T06:13:24.609811+0000","last_clean":"2024-09-21T06:13:24.609811+0000","last_became_active":"2024-09-21T06:13:10.511270+0000","last_became_peered":"2024-09-21T06:13:10.511270+0000","last_unstale":"2024-09-21T06:13:24.609811+0000","last_undegraded":"2024-09-21T06:13:24.609811+0000","last_fullsized":"2024-09-21T06:13:24.609811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:10.521853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.627560+0000","last_change":"2024-09-21T06:13:24.627467+0000","last_active":"2024-09-21T06:13:24.627560+0000","last_peered":"2024-09-21T06:13:24.627560+0000","last_clean":"2024-09-21T06:13:24.627560+0000","last_became_active":"2024-09-21T06:13:24.610263+0000","last_became_peered":"2024-09-21T06:13:24.610263+0000","last_unstale":"2024-09-21T06:13:24.627560+0000","last_undegraded":"2024-09-21T06:13:24.627560+0000","last_fullsized":"2024-09-21T06:13:24.627560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:26.472147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602912+0000","last_change":"2024-09-21T06:13:10.517214+0000","last_active":"2024-09-21T06:13:24.602912+0000","last_peered":"2024-09-21T06:13:24.602912+0000","last_clean":"2024-09-21T06:13:24.602912+0000","last_became_active":"2024-09-21T06:13:10.516828+0000","last_became_peered":"2024-09-21T06:13:10.516828+0000","last_unstale":"2024-09-21T06:13:24.602912+0000","last_undegraded":"2024-09-21T06:13:24.602912+0000","last_fullsized":"2024-09-21T06:13:24.602912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:58.077670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608703+0000","last_change":"2024-09-21T06:13:08.489153+0000","last_active":"2024-09-21T06:13:24.608703+0000","last_peered":"2024-09-21T06:13:24.608703+0000","last_clean":"2024-09-21T06:13:24.608703+0000","last_became_active":"2024-09-21T06:13:08.488732+0000","last_became_peered":"2024-09-21T06:13:08.488732+0000","last_unstale":"2024-09-21T06:13:24.608703+0000","last_undegraded":"2024-09-21T06:13:24.608703+0000","last_fullsized":"2024-09-21T06:13:24.608703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:22.848136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610561+0000","last_change":"2024-09-21T06:13:24.610441+0000","last_active":"2024-09-21T06:13:24.610561+0000","last_peered":"2024-09-21T06:13:24.610561+0000","last_clean":"2024-09-21T06:13:24.610561+0000","last_became_active":"2024-09-21T06:13:24.609992+0000","last_became_peered":"2024-09-21T06:13:24.609992+0000","last_unstale":"2024-09-21T06:13:24.610561+0000","last_undegraded":"2024-09-21T06:13:24.610561+0000","last_fullsized":"2024-09-21T06:13:24.610561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:59.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597725+0000","last_change":"2024-09-21T06:13:08.484514+0000","last_active":"2024-09-21T06:13:24.597725+0000","last_peered":"2024-09-21T06:13:24.597725+0000","last_clean":"2024-09-21T06:13:24.597725+0000","last_became_active":"2024-09-21T06:13:08.484134+0000","last_became_peered":"2024-09-21T06:13:08.484134+0000","last_unstale":"2024-09-21T06:13:24.597725+0000","last_undegraded":"2024-09-21T06:13:24.597725+0000","last_fullsized":"2024-09-21T06:13:24.597725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:45.124734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610027+0000","last_change":"2024-09-21T06:13:10.503798+0000","last_active":"2024-09-21T06:13:24.610027+0000","last_peered":"2024-09-21T06:13:24.610027+0000","last_clean":"2024-09-21T06:13:24.610027+0000","last_became_active":"2024-09-21T06:13:10.502418+0000","last_became_peered":"2024-09-21T06:13:10.502418+0000","last_unstale":"2024-09-21T06:13:24.610027+0000","last_undegraded":"2024-09-21T06:13:24.610027+0000","last_fullsized":"2024-09-21T06:13:24.610027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:57.575569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603439+0000","last_change":"2024-09-21T06:13:08.487191+0000","last_active":"2024-09-21T06:13:24.603439+0000","last_peered":"2024-09-21T06:13:24.603439+0000","last_clean":"2024-09-21T06:13:24.603439+0000","last_became_active":"2024-09-21T06:13:08.486812+0000","last_became_peered":"2024-09-21T06:13:08.486812+0000","last_unstale":"2024-09-21T06:13:24.603439+0000","last_undegraded":"2024-09-21T06:13:24.603439+0000","last_fullsized":"2024-09-21T06:13:24.603439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:51.989708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610179+0000","last_change":"2024-09-21T06:13:10.511678+0000","last_active":"2024-09-21T06:13:24.610179+0000","last_peered":"2024-09-21T06:13:24.610179+0000","last_clean":"2024-09-21T06:13:24.610179+0000","last_became_active":"2024-09-21T06:13:10.510408+0000","last_became_peered":"2024-09-21T06:13:10.510408+0000","last_unstale":"2024-09-21T06:13:24.610179+0000","last_undegraded":"2024-09-21T06:13:24.610179+0000","last_fullsized":"2024-09-21T06:13:24.610179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:58.711190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'6","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:33.630194+0000","last_change":"2024-09-21T06:13:08.560894+0000","last_active":"2024-09-21T06:13:33.630194+0000","last_peered":"2024-09-21T06:13:33.630194+0000","last_clean":"2024-09-21T06:13:33.630194+0000","last_became_active":"2024-09-21T06:13:08.560423+0000","last_became_peered":"2024-09-21T06:13:08.560423+0000","last_unstale":"2024-09-21T06:13:33.630194+0000","last_undegraded":"2024-09-21T06:13:33.630194+0000","last_fullsized":"2024-09-21T06:13:33.630194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:53.015711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610081+0000","last_change":"2024-09-21T06:13:10.516009+0000","last_active":"2024-09-21T06:13:24.610081+0000","last_peered":"2024-09-21T06:13:24.610081+0000","last_clean":"2024-09-21T06:13:24.610081+0000","last_became_active":"2024-09-21T06:13:10.515669+0000","last_became_peered":"2024-09-21T06:13:10.515669+0000","last_unstale":"2024-09-21T06:13:24.610081+0000","last_undegraded":"2024-09-21T06:13:24.610081+0000","last_fullsized":"2024-09-21T06:13:24.610081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:54.134564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":99,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610317+0000","last_change":"2024-09-21T06:11:53.408030+0000","last_active":"2024-09-21T06:13:24.610317+0000","last_peered":"2024-09-21T06:13:24.610317+0000","last_clean":"2024-09-21T06:13:24.610317+0000","last_became_active":"2024-09-21T06:11:53.402296+0000","last_became_peered":"2024-09-21T06:11:53.402296+0000","last_unstale":"2024-09-21T06:13:24.610317+0000","last_undegraded":"2024-09-21T06:13:24.610317+0000","last_fullsized":"2024-09-21T06:13:24.610317+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_clean_scrub_stamp":"2024-09-21T06:08:26.098183+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:08.239872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616959+0000","last_change":"2024-09-21T06:13:08.489209+0000","last_active":"2024-09-21T06:13:24.616959+0000","last_peered":"2024-09-21T06:13:24.616959+0000","last_clean":"2024-09-21T06:13:24.616959+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.616959+0000","last_undegraded":"2024-09-21T06:13:24.616959+0000","last_fullsized":"2024-09-21T06:13:24.616959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:38.798499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607003+0000","last_change":"2024-09-21T06:13:24.606487+0000","last_active":"2024-09-21T06:13:24.607003+0000","last_peered":"2024-09-21T06:13:24.607003+0000","last_clean":"2024-09-21T06:13:24.607003+0000","last_became_active":"2024-09-21T06:13:24.606141+0000","last_became_peered":"2024-09-21T06:13:24.606141+0000","last_unstale":"2024-09-21T06:13:24.607003+0000","last_undegraded":"2024-09-21T06:13:24.607003+0000","last_fullsized":"2024-09-21T06:13:24.607003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:24.040770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609976+0000","last_change":"2024-09-21T06:13:08.489206+0000","last_active":"2024-09-21T06:13:24.609976+0000","last_peered":"2024-09-21T06:13:24.609976+0000","last_clean":"2024-09-21T06:13:24.609976+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.609976+0000","last_undegraded":"2024-09-21T06:13:24.609976+0000","last_fullsized":"2024-09-21T06:13:24.609976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.627305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597214+0000","last_change":"2024-09-21T06:13:10.513999+0000","last_active":"2024-09-21T06:13:24.597214+0000","last_peered":"2024-09-21T06:13:24.597214+0000","last_clean":"2024-09-21T06:13:24.597214+0000","last_became_active":"2024-09-21T06:13:10.513657+0000","last_became_peered":"2024-09-21T06:13:10.513657+0000","last_unstale":"2024-09-21T06:13:24.597214+0000","last_undegraded":"2024-09-21T06:13:24.597214+0000","last_fullsized":"2024-09-21T06:13:24.597214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:05.536921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'5","reported_seq":44,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:37.640134+0000","last_change":"2024-09-21T06:13:08.560811+0000","last_active":"2024-09-21T06:13:37.640134+0000","last_peered":"2024-09-21T06:13:37.640134+0000","last_clean":"2024-09-21T06:13:37.640134+0000","last_became_active":"2024-09-21T06:13:08.560145+0000","last_became_peered":"2024-09-21T06:13:08.560145+0000","last_unstale":"2024-09-21T06:13:37.640134+0000","last_undegraded":"2024-09-21T06:13:37.640134+0000","last_fullsized":"2024-09-21T06:13:37.640134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:24.550008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.613627+0000","last_change":"2024-09-21T06:13:24.613495+0000","last_active":"2024-09-21T06:13:24.613627+0000","last_peered":"2024-09-21T06:13:24.613627+0000","last_clean":"2024-09-21T06:13:24.613627+0000","last_became_active":"2024-09-21T06:13:24.612634+0000","last_became_peered":"2024-09-21T06:13:24.612634+0000","last_unstale":"2024-09-21T06:13:24.613627+0000","last_undegraded":"2024-09-21T06:13:24.613627+0000","last_fullsized":"2024-09-21T06:13:24.613627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.863141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'5","reported_seq":45,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:37.632182+0000","last_change":"2024-09-21T06:13:08.486973+0000","last_active":"2024-09-21T06:13:37.632182+0000","last_peered":"2024-09-21T06:13:37.632182+0000","last_clean":"2024-09-21T06:13:37.632182+0000","last_became_active":"2024-09-21T06:13:08.486602+0000","last_became_peered":"2024-09-21T06:13:08.486602+0000","last_unstale":"2024-09-21T06:13:37.632182+0000","last_undegraded":"2024-09-21T06:13:37.632182+0000","last_fullsized":"2024-09-21T06:13:37.632182+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:41.804955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.599893+0000","last_change":"2024-09-21T06:13:10.509499+0000","last_active":"2024-09-21T06:13:24.599893+0000","last_peered":"2024-09-21T06:13:24.599893+0000","last_clean":"2024-09-21T06:13:24.599893+0000","last_became_active":"2024-09-21T06:13:10.508566+0000","last_became_peered":"2024-09-21T06:13:10.508566+0000","last_unstale":"2024-09-21T06:13:24.599893+0000","last_undegraded":"2024-09-21T06:13:24.599893+0000","last_fullsized":"2024-09-21T06:13:24.599893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:12.701631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600737+0000","last_change":"2024-09-21T06:13:08.561378+0000","last_active":"2024-09-21T06:13:24.600737+0000","last_peered":"2024-09-21T06:13:24.600737+0000","last_clean":"2024-09-21T06:13:24.600737+0000","last_became_active":"2024-09-21T06:13:08.561002+0000","last_became_peered":"2024-09-21T06:13:08.561002+0000","last_unstale":"2024-09-21T06:13:24.600737+0000","last_undegraded":"2024-09-21T06:13:24.600737+0000","last_fullsized":"2024-09-21T06:13:24.600737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:11.325377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602794+0000","last_change":"2024-09-21T06:13:24.602347+0000","last_active":"2024-09-21T06:13:24.602794+0000","last_peered":"2024-09-21T06:13:24.602794+0000","last_clean":"2024-09-21T06:13:24.602794+0000","last_became_active":"2024-09-21T06:13:24.601978+0000","last_became_peered":"2024-09-21T06:13:24.601978+0000","last_unstale":"2024-09-21T06:13:24.602794+0000","last_undegraded":"2024-09-21T06:13:24.602794+0000","last_fullsized":"2024-09-21T06:13:24.602794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:34.860096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608569+0000","last_change":"2024-09-21T06:13:08.561937+0000","last_active":"2024-09-21T06:13:24.608569+0000","last_peered":"2024-09-21T06:13:24.608569+0000","last_clean":"2024-09-21T06:13:24.608569+0000","last_became_active":"2024-09-21T06:13:08.560547+0000","last_became_peered":"2024-09-21T06:13:08.560547+0000","last_unstale":"2024-09-21T06:13:24.608569+0000","last_undegraded":"2024-09-21T06:13:24.608569+0000","last_fullsized":"2024-09-21T06:13:24.608569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:29.702291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597628+0000","last_change":"2024-09-21T06:13:10.516833+0000","last_active":"2024-09-21T06:13:24.597628+0000","last_peered":"2024-09-21T06:13:24.597628+0000","last_clean":"2024-09-21T06:13:24.597628+0000","last_became_active":"2024-09-21T06:13:10.516486+0000","last_became_peered":"2024-09-21T06:13:10.516486+0000","last_unstale":"2024-09-21T06:13:24.597628+0000","last_undegraded":"2024-09-21T06:13:24.597628+0000","last_fullsized":"2024-09-21T06:13:24.597628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:05.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'13","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:33.499117+0000","last_change":"2024-09-21T06:13:08.483271+0000","last_active":"2024-09-21T06:13:33.499117+0000","last_peered":"2024-09-21T06:13:33.499117+0000","last_clean":"2024-09-21T06:13:33.499117+0000","last_became_active":"2024-09-21T06:13:08.482951+0000","last_became_peered":"2024-09-21T06:13:08.482951+0000","last_unstale":"2024-09-21T06:13:33.499117+0000","last_undegraded":"2024-09-21T06:13:33.499117+0000","last_fullsized":"2024-09-21T06:13:33.499117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:17.537988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610524+0000","last_change":"2024-09-21T06:13:10.509125+0000","last_active":"2024-09-21T06:13:24.610524+0000","last_peered":"2024-09-21T06:13:24.610524+0000","last_clean":"2024-09-21T06:13:24.610524+0000","last_became_active":"2024-09-21T06:13:10.508773+0000","last_became_peered":"2024-09-21T06:13:10.508773+0000","last_unstale":"2024-09-21T06:13:24.610524+0000","last_undegraded":"2024-09-21T06:13:24.610524+0000","last_fullsized":"2024-09-21T06:13:24.610524+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:44.570477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616932+0000","last_change":"2024-09-21T06:13:08.497885+0000","last_active":"2024-09-21T06:13:24.616932+0000","last_peered":"2024-09-21T06:13:24.616932+0000","last_clean":"2024-09-21T06:13:24.616932+0000","last_became_active":"2024-09-21T06:13:08.497593+0000","last_became_peered":"2024-09-21T06:13:08.497593+0000","last_unstale":"2024-09-21T06:13:24.616932+0000","last_undegraded":"2024-09-21T06:13:24.616932+0000","last_fullsized":"2024-09-21T06:13:24.616932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:48.361175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609527+0000","last_change":"2024-09-21T06:13:10.514430+0000","last_active":"2024-09-21T06:13:24.609527+0000","last_peered":"2024-09-21T06:13:24.609527+0000","last_clean":"2024-09-21T06:13:24.609527+0000","last_became_active":"2024-09-21T06:13:10.514063+0000","last_became_peered":"2024-09-21T06:13:10.514063+0000","last_unstale":"2024-09-21T06:13:24.609527+0000","last_undegraded":"2024-09-21T06:13:24.609527+0000","last_fullsized":"2024-09-21T06:13:24.609527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:37.036497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598415+0000","last_change":"2024-09-21T06:13:08.560564+0000","last_active":"2024-09-21T06:13:24.598415+0000","last_peered":"2024-09-21T06:13:24.598415+0000","last_clean":"2024-09-21T06:13:24.598415+0000","last_became_active":"2024-09-21T06:13:08.560339+0000","last_became_peered":"2024-09-21T06:13:08.560339+0000","last_unstale":"2024-09-21T06:13:24.598415+0000","last_undegraded":"2024-09-21T06:13:24.598415+0000","last_fullsized":"2024-09-21T06:13:24.598415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:55.378911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.605861+0000","last_change":"2024-09-21T06:13:10.511354+0000","last_active":"2024-09-21T06:13:24.605861+0000","last_peered":"2024-09-21T06:13:24.605861+0000","last_clean":"2024-09-21T06:13:24.605861+0000","last_became_active":"2024-09-21T06:13:10.510946+0000","last_became_peered":"2024-09-21T06:13:10.510946+0000","last_unstale":"2024-09-21T06:13:24.605861+0000","last_undegraded":"2024-09-21T06:13:24.605861+0000","last_fullsized":"2024-09-21T06:13:24.605861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:11.535461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610883+0000","last_change":"2024-09-21T06:13:08.485578+0000","last_active":"2024-09-21T06:13:24.610883+0000","last_peered":"2024-09-21T06:13:24.610883+0000","last_clean":"2024-09-21T06:13:24.610883+0000","last_became_active":"2024-09-21T06:13:08.485170+0000","last_became_peered":"2024-09-21T06:13:08.485170+0000","last_unstale":"2024-09-21T06:13:24.610883+0000","last_undegraded":"2024-09-21T06:13:24.610883+0000","last_fullsized":"2024-09-21T06:13:24.610883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:39.000114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.606691+0000","last_change":"2024-09-21T06:13:10.510360+0000","last_active":"2024-09-21T06:13:24.606691+0000","last_peered":"2024-09-21T06:13:24.606691+0000","last_clean":"2024-09-21T06:13:24.606691+0000","last_became_active":"2024-09-21T06:13:10.509887+0000","last_became_peered":"2024-09-21T06:13:10.509887+0000","last_unstale":"2024-09-21T06:13:24.606691+0000","last_undegraded":"2024-09-21T06:13:24.606691+0000","last_fullsized":"2024-09-21T06:13:24.606691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:58.135706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.625360+0000","last_change":"2024-09-21T06:13:24.625284+0000","last_active":"2024-09-21T06:13:24.625360+0000","last_peered":"2024-09-21T06:13:24.625360+0000","last_clean":"2024-09-21T06:13:24.625360+0000","last_became_active":"2024-09-21T06:13:24.612704+0000","last_became_peered":"2024-09-21T06:13:24.612704+0000","last_unstale":"2024-09-21T06:13:24.625360+0000","last_undegraded":"2024-09-21T06:13:24.625360+0000","last_fullsized":"2024-09-21T06:13:24.625360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:29.385258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.611377+0000","last_change":"2024-09-21T06:13:10.514128+0000","last_active":"2024-09-21T06:13:24.611377+0000","last_peered":"2024-09-21T06:13:24.611377+0000","last_clean":"2024-09-21T06:13:24.611377+0000","last_became_active":"2024-09-21T06:13:10.513790+0000","last_became_peered":"2024-09-21T06:13:10.513790+0000","last_unstale":"2024-09-21T06:13:24.611377+0000","last_undegraded":"2024-09-21T06:13:24.611377+0000","last_fullsized":"2024-09-21T06:13:24.611377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:33.407189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607762+0000","last_change":"2024-09-21T06:13:08.489105+0000","last_active":"2024-09-21T06:13:24.607762+0000","last_peered":"2024-09-21T06:13:24.607762+0000","last_clean":"2024-09-21T06:13:24.607762+0000","last_became_active":"2024-09-21T06:13:08.487110+0000","last_became_peered":"2024-09-21T06:13:08.487110+0000","last_unstale":"2024-09-21T06:13:24.607762+0000","last_undegraded":"2024-09-21T06:13:24.607762+0000","last_fullsized":"2024-09-21T06:13:24.607762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:24.849365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610194+0000","last_change":"2024-09-21T06:13:10.513751+0000","last_active":"2024-09-21T06:13:24.610194+0000","last_peered":"2024-09-21T06:13:24.610194+0000","last_clean":"2024-09-21T06:13:24.610194+0000","last_became_active":"2024-09-21T06:13:10.513376+0000","last_became_peered":"2024-09-21T06:13:10.513376+0000","last_unstale":"2024-09-21T06:13:24.610194+0000","last_undegraded":"2024-09-21T06:13:24.610194+0000","last_fullsized":"2024-09-21T06:13:24.610194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.364959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603110+0000","last_change":"2024-09-21T06:13:10.519359+0000","last_active":"2024-09-21T06:13:24.603110+0000","last_peered":"2024-09-21T06:13:24.603110+0000","last_clean":"2024-09-21T06:13:24.603110+0000","last_became_active":"2024-09-21T06:13:10.518518+0000","last_became_peered":"2024-09-21T06:13:10.518518+0000","last_unstale":"2024-09-21T06:13:24.603110+0000","last_undegraded":"2024-09-21T06:13:24.603110+0000","last_fullsized":"2024-09-21T06:13:24.603110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:55.554710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608166+0000","last_change":"2024-09-21T06:13:08.488122+0000","last_active":"2024-09-21T06:13:24.608166+0000","last_peered":"2024-09-21T06:13:24.608166+0000","last_clean":"2024-09-21T06:13:24.608166+0000","last_became_active":"2024-09-21T06:13:08.487679+0000","last_became_peered":"2024-09-21T06:13:08.487679+0000","last_unstale":"2024-09-21T06:13:24.608166+0000","last_undegraded":"2024-09-21T06:13:24.608166+0000","last_fullsized":"2024-09-21T06:13:24.608166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:12.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598434+0000","last_change":"2024-09-21T06:13:10.516794+0000","last_active":"2024-09-21T06:13:24.598434+0000","last_peered":"2024-09-21T06:13:24.598434+0000","last_clean":"2024-09-21T06:13:24.598434+0000","last_became_active":"2024-09-21T06:13:10.516297+0000","last_became_peered":"2024-09-21T06:13:10.516297+0000","last_unstale":"2024-09-21T06:13:24.598434+0000","last_undegraded":"2024-09-21T06:13:24.598434+0000","last_fullsized":"2024-09-21T06:13:24.598434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:19.873742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.621488+0000","last_change":"2024-09-21T06:13:24.621358+0000","last_active":"2024-09-21T06:13:24.621488+0000","last_peered":"2024-09-21T06:13:24.621488+0000","last_clean":"2024-09-21T06:13:24.621488+0000","last_became_active":"2024-09-21T06:13:24.611058+0000","last_became_peered":"2024-09-21T06:13:24.611058+0000","last_unstale":"2024-09-21T06:13:24.621488+0000","last_undegraded":"2024-09-21T06:13:24.621488+0000","last_fullsized":"2024-09-21T06:13:24.621488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:28.759029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609712+0000","last_change":"2024-09-21T06:13:10.516339+0000","last_active":"2024-09-21T06:13:24.609712+0000","last_peered":"2024-09-21T06:13:24.609712+0000","last_clean":"2024-09-21T06:13:24.609712+0000","last_became_active":"2024-09-21T06:13:10.515397+0000","last_became_peered":"2024-09-21T06:13:10.515397+0000","last_unstale":"2024-09-21T06:13:24.609712+0000","last_undegraded":"2024-09-21T06:13:24.609712+0000","last_fullsized":"2024-09-21T06:13:24.609712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:56.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610156+0000","last_change":"2024-09-21T06:13:08.560494+0000","last_active":"2024-09-21T06:13:24.610156+0000","last_peered":"2024-09-21T06:13:24.610156+0000","last_clean":"2024-09-21T06:13:24.610156+0000","last_became_active":"2024-09-21T06:13:08.560118+0000","last_became_peered":"2024-09-21T06:13:08.560118+0000","last_unstale":"2024-09-21T06:13:24.610156+0000","last_undegraded":"2024-09-21T06:13:24.610156+0000","last_fullsized":"2024-09-21T06:13:24.610156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:05.597233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610032+0000","last_change":"2024-09-21T06:13:10.515780+0000","last_active":"2024-09-21T06:13:24.610032+0000","last_peered":"2024-09-21T06:13:24.610032+0000","last_clean":"2024-09-21T06:13:24.610032+0000","last_became_active":"2024-09-21T06:13:10.515437+0000","last_became_peered":"2024-09-21T06:13:10.515437+0000","last_unstale":"2024-09-21T06:13:24.610032+0000","last_undegraded":"2024-09-21T06:13:24.610032+0000","last_fullsized":"2024-09-21T06:13:24.610032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:04.848003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'13","reported_seq":50,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:34.626991+0000","last_change":"2024-09-21T06:13:08.483300+0000","last_active":"2024-09-21T06:13:34.626991+0000","last_peered":"2024-09-21T06:13:34.626991+0000","last_clean":"2024-09-21T06:13:34.626991+0000","last_became_active":"2024-09-21T06:13:08.482908+0000","last_became_peered":"2024-09-21T06:13:08.482908+0000","last_unstale":"2024-09-21T06:13:34.626991+0000","last_undegraded":"2024-09-21T06:13:34.626991+0000","last_fullsized":"2024-09-21T06:13:34.626991+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:09.202075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66178,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":170,"num_read_kb":145,"num_write":237,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1724,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":186292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282438,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678034,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":864256,"data_stored":427523,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874263,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1064893,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037789,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":427179,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234018,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":427440,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430247,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":1020349,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593772,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":426779,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790002,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1019636,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":485614,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":426487,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378689,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":428582,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542214,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":474145,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705739,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":892928,"data_stored":428509,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":48205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":59674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:13:40.670 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T06:13:40.670 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T06:13:40.671 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T06:13:40.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:40 smithi137 ceph-mon[34464]: pgmap v285: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 13 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T06:13:40.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:40 smithi137 ceph-mon[34464]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:13:40.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:40 smithi204 ceph-mon[34659]: pgmap v285: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 13 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T06:13:40.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:40 smithi204 ceph-mon[34659]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:13:40.930 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:41.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:40 smithi003 ceph-mon[31227]: pgmap v285: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 13 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T06:13:41.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:40 smithi003 ceph-mon[31227]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:13:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.444+0000 7f01161e9640 1 -- 172.21.15.3:0/3618428576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0110109e20 msgr2=0x7f0110110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.444+0000 7f01161e9640 1 --2- 172.21.15.3:0/3618428576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0110109e20 0x7f0110110650 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0108009ca0 tx=0x7f010802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.444+0000 7f01161e9640 1 -- 172.21.15.3:0/3618428576 shutdown_connections 2024-09-21T06:13:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.444+0000 7f01161e9640 1 --2- 172.21.15.3:0/3618428576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0110109e20 0x7f0110110650 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.444+0000 7f01161e9640 1 --2- 172.21.15.3:0/3618428576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01101094d0 0x7f01101098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.444+0000 7f01161e9640 1 --2- 172.21.15.3:0/3618428576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f011006be50 0x7f011006c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.444+0000 7f01161e9640 1 -- 172.21.15.3:0/3618428576 >> 172.21.15.3:0/3618428576 conn(0x7f0110079820 msgr2=0x7f0110079c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:41.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.444+0000 7f01161e9640 1 -- 172.21.15.3:0/3618428576 shutdown_connections 2024-09-21T06:13:41.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.444+0000 7f01161e9640 1 -- 172.21.15.3:0/3618428576 wait complete. 2024-09-21T06:13:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.445+0000 7f01161e9640 1 Processor -- start 2024-09-21T06:13:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.445+0000 7f01161e9640 1 -- start start 2024-09-21T06:13:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.445+0000 7f01161e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f011006be50 0x7f011019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.445+0000 7f01161e9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01101094d0 0x7f011019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.445+0000 7f01161e9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0110109e20 0x7f011019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.445+0000 7f01161e9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0110113ae0 con 0x7f0110109e20 2024-09-21T06:13:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.445+0000 7f01161e9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0110113960 con 0x7f011006be50 2024-09-21T06:13:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.445+0000 7f01161e9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0110113c60 con 0x7f01101094d0 2024-09-21T06:13:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.445+0000 7f010ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0110109e20 0x7f011019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.445+0000 7f010ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0110109e20 0x7f011019fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44124/0 (socket says 172.21.15.3:44124) 2024-09-21T06:13:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.445+0000 7f010ffff640 1 -- 172.21.15.3:0/1543950599 learned_addr learned my addr 172.21.15.3:0/1543950599 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.445+0000 7f010f7fe640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f011006be50 0x7f011019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.446+0000 7f010effd640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01101094d0 0x7f011019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.446+0000 7f010ffff640 1 -- 172.21.15.3:0/1543950599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01101094d0 msgr2=0x7f011019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.446+0000 7f010ffff640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01101094d0 0x7f011019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.446+0000 7f010ffff640 1 -- 172.21.15.3:0/1543950599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f011006be50 msgr2=0x7f011019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.446+0000 7f010ffff640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f011006be50 0x7f011019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.446+0000 7f010ffff640 1 -- 172.21.15.3:0/1543950599 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01101a04a0 con 0x7f0110109e20 2024-09-21T06:13:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.446+0000 7f010ffff640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0110109e20 0x7f011019fd70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0108005cc0 tx=0x7f010802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.446+0000 7f010cff9640 1 -- 172.21.15.3:0/1543950599 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108003bf0 con 0x7f0110109e20 2024-09-21T06:13:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.446+0000 7f010cff9640 1 -- 172.21.15.3:0/1543950599 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0108003d90 con 0x7f0110109e20 2024-09-21T06:13:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.446+0000 7f01161e9640 1 -- 172.21.15.3:0/1543950599 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01101a0730 con 0x7f0110109e20 2024-09-21T06:13:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.446+0000 7f010cff9640 1 -- 172.21.15.3:0/1543950599 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010802fbc0 con 0x7f0110109e20 2024-09-21T06:13:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.446+0000 7f01161e9640 1 -- 172.21.15.3:0/1543950599 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01101a6ef0 con 0x7f0110109e20 2024-09-21T06:13:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.447+0000 7f01161e9640 1 -- 172.21.15.3:0/1543950599 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011006b2d0 con 0x7f0110109e20 2024-09-21T06:13:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.449+0000 7f010cff9640 1 -- 172.21.15.3:0/1543950599 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f010802fd60 con 0x7f0110109e20 2024-09-21T06:13:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.450+0000 7f010cff9640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f00f80b65f0 0x7f00f80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.450+0000 7f010f7fe640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f00f80b65f0 0x7f00f80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.450+0000 7f010cff9640 1 -- 172.21.15.3:0/1543950599 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f0108108c20 con 0x7f0110109e20 2024-09-21T06:13:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.450+0000 7f010f7fe640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f00f80b65f0 0x7f00f80b8b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f010000bbe0 tx=0x7f010000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:41.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.453+0000 7f010cff9640 1 -- 172.21.15.3:0/1543950599 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01080cf210 con 0x7f0110109e20 2024-09-21T06:13:41.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.713+0000 7f01161e9640 1 -- 172.21.15.3:0/1543950599 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f011006c260 con 0x7f0110109e20 2024-09-21T06:13:41.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.717+0000 7f010cff9640 1 -- 172.21.15.3:0/1543950599 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f01080d35f0 con 0x7f0110109e20 2024-09-21T06:13:41.718 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-21T06:13:41.719 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAVZO5mpzGgKhAAGR5IahIskEbsStzbkcLEDw== 2024-09-21T06:13:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.719+0000 7f01161e9640 1 -- 172.21.15.3:0/1543950599 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f00f80b65f0 msgr2=0x7f00f80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.719+0000 7f01161e9640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f00f80b65f0 0x7f00f80b8b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f010000bbe0 tx=0x7f010000b390 comp rx=0 tx=0).stop 2024-09-21T06:13:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.719+0000 7f01161e9640 1 -- 172.21.15.3:0/1543950599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0110109e20 msgr2=0x7f011019fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.719+0000 7f01161e9640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0110109e20 0x7f011019fd70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0108005cc0 tx=0x7f010802f380 comp rx=0 tx=0).stop 2024-09-21T06:13:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.719+0000 7f01161e9640 1 -- 172.21.15.3:0/1543950599 shutdown_connections 2024-09-21T06:13:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.719+0000 7f01161e9640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f00f80b65f0 0x7f00f80b8b00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.719+0000 7f01161e9640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01101094d0 0x7f011019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.719+0000 7f01161e9640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0110109e20 0x7f011019fd70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.719+0000 7f01161e9640 1 --2- 172.21.15.3:0/1543950599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f011006be50 0x7f011019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.719+0000 7f01161e9640 1 -- 172.21.15.3:0/1543950599 >> 172.21.15.3:0/1543950599 conn(0x7f0110079820 msgr2=0x7f01101054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.719+0000 7f01161e9640 1 -- 172.21.15.3:0/1543950599 shutdown_connections 2024-09-21T06:13:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:41.719+0000 7f01161e9640 1 -- 172.21.15.3:0/1543950599 wait complete. 2024-09-21T06:13:42.546 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:13:42.546 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T06:13:42.547 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T06:13:42.587 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T06:13:42.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:42 smithi003 ceph-mon[31227]: pgmap v286: 97 pgs: 97 active+clean; 647 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 14 op/s 2024-09-21T06:13:42.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:42 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1543950599' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T06:13:42.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:42 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1543950599' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T06:13:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:42.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:42 smithi137 ceph-mon[34464]: pgmap v286: 97 pgs: 97 active+clean; 647 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 14 op/s 2024-09-21T06:13:42.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:42 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1543950599' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T06:13:42.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:42 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1543950599' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T06:13:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:42 smithi204 ceph-mon[34659]: pgmap v286: 97 pgs: 97 active+clean; 647 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 14 op/s 2024-09-21T06:13:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:42 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1543950599' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T06:13:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:42 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1543950599' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T06:13:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.381+0000 7f013710e640 1 -- 172.21.15.3:0/1906268527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0130105300 msgr2=0x7f013010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.381+0000 7f013710e640 1 --2- 172.21.15.3:0/1906268527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0130105300 0x7f013010d850 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0124009900 tx=0x7f012402a410 comp rx=0 tx=0).stop 2024-09-21T06:13:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.381+0000 7f013710e640 1 -- 172.21.15.3:0/1906268527 shutdown_connections 2024-09-21T06:13:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.381+0000 7f013710e640 1 --2- 172.21.15.3:0/1906268527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f013010de10 0x7f0130110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.381+0000 7f013710e640 1 --2- 172.21.15.3:0/1906268527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0130105300 0x7f013010d850 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.381+0000 7f013710e640 1 --2- 172.21.15.3:0/1906268527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01301049b0 0x7f0130104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.381+0000 7f013710e640 1 -- 172.21.15.3:0/1906268527 >> 172.21.15.3:0/1906268527 conn(0x7f0130079730 msgr2=0x7f0130079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.381+0000 7f013710e640 1 -- 172.21.15.3:0/1906268527 shutdown_connections 2024-09-21T06:13:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.381+0000 7f013710e640 1 -- 172.21.15.3:0/1906268527 wait complete. 2024-09-21T06:13:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f013710e640 1 Processor -- start 2024-09-21T06:13:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f013710e640 1 -- start start 2024-09-21T06:13:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f013710e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01301049b0 0x7f013019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f013710e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0130105300 0x7f013019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f013710e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f013010de10 0x7f013019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f013710e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0130113a70 con 0x7f01301049b0 2024-09-21T06:13:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f013710e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01301138f0 con 0x7f013010de10 2024-09-21T06:13:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f013710e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0130113bf0 con 0x7f0130105300 2024-09-21T06:13:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f0134eaa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01301049b0 0x7f013019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f0134eaa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01301049b0 0x7f013019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44156/0 (socket says 172.21.15.3:44156) 2024-09-21T06:13:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f0134eaa640 1 -- 172.21.15.3:0/2148459606 learned_addr learned my addr 172.21.15.3:0/2148459606 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f012ffff640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0130105300 0x7f013019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f01356ab640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f013010de10 0x7f013019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f0134eaa640 1 -- 172.21.15.3:0/2148459606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0130105300 msgr2=0x7f013019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f0134eaa640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0130105300 0x7f013019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f0134eaa640 1 -- 172.21.15.3:0/2148459606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f013010de10 msgr2=0x7f013019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f0134eaa640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f013010de10 0x7f013019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f0134eaa640 1 -- 172.21.15.3:0/2148459606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01301a0410 con 0x7f01301049b0 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.382+0000 7f01356ab640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f013010de10 0x7f013019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.383+0000 7f0134eaa640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01301049b0 0x7f013019a4a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f012000d950 tx=0x7f012000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.383+0000 7f012dffb640 1 -- 172.21.15.3:0/2148459606 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120017070 con 0x7f01301049b0 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.383+0000 7f012dffb640 1 -- 172.21.15.3:0/2148459606 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0120009c30 con 0x7f01301049b0 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.383+0000 7f013710e640 1 -- 172.21.15.3:0/2148459606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01301a0700 con 0x7f01301049b0 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.383+0000 7f012dffb640 1 -- 172.21.15.3:0/2148459606 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012000f620 con 0x7f01301049b0 2024-09-21T06:13:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.383+0000 7f013710e640 1 -- 172.21.15.3:0/2148459606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01301a6e60 con 0x7f01301049b0 2024-09-21T06:13:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.384+0000 7f013710e640 1 -- 172.21.15.3:0/2148459606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0130104dc0 con 0x7f01301049b0 2024-09-21T06:13:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.386+0000 7f012dffb640 1 -- 172.21.15.3:0/2148459606 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f012001a070 con 0x7f01301049b0 2024-09-21T06:13:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.387+0000 7f012dffb640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f01180b6670 0x7f01180b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.387+0000 7f012ffff640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f01180b6670 0x7f01180b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.387+0000 7f012dffb640 1 -- 172.21.15.3:0/2148459606 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f01200e6fb0 con 0x7f01301049b0 2024-09-21T06:13:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.388+0000 7f012ffff640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f01180b6670 0x7f01180b8b80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0124009230 tx=0x7f0124031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.391+0000 7f012dffb640 1 -- 172.21.15.3:0/2148459606 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0120009de0 con 0x7f01301049b0 2024-09-21T06:13:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.580+0000 7f013710e640 1 -- 172.21.15.3:0/2148459606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f0130108f40 con 0x7f01301049b0 2024-09-21T06:13:43.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.581+0000 7f012dffb640 1 -- 172.21.15.3:0/2148459606 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v56) ==== 86+0+133 (secure 0 0 0) 0x7f01200ad620 con 0x7f01301049b0 2024-09-21T06:13:43.583 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:13:43.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.583+0000 7f013710e640 1 -- 172.21.15.3:0/2148459606 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f01180b6670 msgr2=0x7f01180b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:43.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.583+0000 7f013710e640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f01180b6670 0x7f01180b8b80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0124009230 tx=0x7f0124031400 comp rx=0 tx=0).stop 2024-09-21T06:13:43.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.583+0000 7f013710e640 1 -- 172.21.15.3:0/2148459606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01301049b0 msgr2=0x7f013019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:43.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.583+0000 7f013710e640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01301049b0 0x7f013019a4a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f012000d950 tx=0x7f012000de10 comp rx=0 tx=0).stop 2024-09-21T06:13:43.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.583+0000 7f013710e640 1 -- 172.21.15.3:0/2148459606 shutdown_connections 2024-09-21T06:13:43.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.583+0000 7f013710e640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f01180b6670 0x7f01180b8b80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.583+0000 7f013710e640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0130105300 0x7f013019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.583+0000 7f013710e640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f013010de10 0x7f013019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.583+0000 7f013710e640 1 --2- 172.21.15.3:0/2148459606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01301049b0 0x7f013019a4a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:43.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.583+0000 7f013710e640 1 -- 172.21.15.3:0/2148459606 >> 172.21.15.3:0/2148459606 conn(0x7f0130079730 msgr2=0x7f0130100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:43.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.583+0000 7f013710e640 1 -- 172.21.15.3:0/2148459606 shutdown_connections 2024-09-21T06:13:43.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:43.583+0000 7f013710e640 1 -- 172.21.15.3:0/2148459606 wait complete. 2024-09-21T06:13:43.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:43 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2148459606' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T06:13:43.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:43 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2148459606' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T06:13:44.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:43 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2148459606' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T06:13:44.316 INFO:teuthology.orchestra.run.smithi003.stdout:[{"version":56,"timestamp":"2024-09-21T06:13:01.024289+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T06:13:44.316 INFO:tasks.ceph_manager:config epoch is 56 2024-09-21T06:13:44.316 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T06:13:44.316 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T06:13:44.317 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T06:13:44.581 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:44.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:44 smithi204 ceph-mon[34659]: pgmap v287: 97 pgs: 97 active+clean; 647 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-21T06:13:45.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:44 smithi003 ceph-mon[31227]: pgmap v287: 97 pgs: 97 active+clean; 647 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-21T06:13:45.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.101+0000 7f2dfcc53640 1 -- 172.21.15.3:0/3215468014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df81094b0 msgr2=0x7f2df81098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.101+0000 7f2dfcc53640 1 --2- 172.21.15.3:0/3215468014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df81094b0 0x7f2df81098c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009900 tx=0x7f2de002a410 comp rx=0 tx=0).stop 2024-09-21T06:13:45.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.101+0000 7f2dfcc53640 1 -- 172.21.15.3:0/3215468014 shutdown_connections 2024-09-21T06:13:45.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.101+0000 7f2dfcc53640 1 --2- 172.21.15.3:0/3215468014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df8109e00 0x7f2df8110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.101+0000 7f2dfcc53640 1 --2- 172.21.15.3:0/3215468014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df81094b0 0x7f2df81098c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.101+0000 7f2dfcc53640 1 --2- 172.21.15.3:0/3215468014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2df8103bc0 0x7f2df8103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.101+0000 7f2dfcc53640 1 -- 172.21.15.3:0/3215468014 >> 172.21.15.3:0/3215468014 conn(0x7f2df807b910 msgr2=0x7f2df807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:45.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.102+0000 7f2dfcc53640 1 -- 172.21.15.3:0/3215468014 shutdown_connections 2024-09-21T06:13:45.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.102+0000 7f2dfcc53640 1 -- 172.21.15.3:0/3215468014 wait complete. 2024-09-21T06:13:45.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.103+0000 7f2dfcc53640 1 Processor -- start 2024-09-21T06:13:45.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.103+0000 7f2dfcc53640 1 -- start start 2024-09-21T06:13:45.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.103+0000 7f2dfcc53640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df8103bc0 0x7f2df8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.103+0000 7f2dfcc53640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2df81094b0 0x7f2df8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.103+0000 7f2dfcc53640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df8109e00 0x7f2df8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.103+0000 7f2dfcc53640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2df8113ac0 con 0x7f2df8103bc0 2024-09-21T06:13:45.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.103+0000 7f2dfcc53640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2df8113940 con 0x7f2df8109e00 2024-09-21T06:13:45.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.103+0000 7f2dfcc53640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2df8113c40 con 0x7f2df81094b0 2024-09-21T06:13:45.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.103+0000 7f2df659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df8103bc0 0x7f2df8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.103+0000 7f2df659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df8103bc0 0x7f2df8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44180/0 (socket says 172.21.15.3:44180) 2024-09-21T06:13:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.103+0000 7f2df659c640 1 -- 172.21.15.3:0/2609370817 learned_addr learned my addr 172.21.15.3:0/2609370817 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.103+0000 7f2df6d9d640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df8109e00 0x7f2df8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.103+0000 7f2deffff640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2df81094b0 0x7f2df8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.104+0000 7f2df659c640 1 -- 172.21.15.3:0/2609370817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2df81094b0 msgr2=0x7f2df8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.104+0000 7f2df659c640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2df81094b0 0x7f2df8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.104+0000 7f2df659c640 1 -- 172.21.15.3:0/2609370817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df8109e00 msgr2=0x7f2df8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.104+0000 7f2df659c640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df8109e00 0x7f2df8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.104+0000 7f2df659c640 1 -- 172.21.15.3:0/2609370817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df81a8730 con 0x7f2df8103bc0 2024-09-21T06:13:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.104+0000 7f2deffff640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2df81094b0 0x7f2df8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.104+0000 7f2df659c640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df8103bc0 0x7f2df8075710 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2de800d770 tx=0x7f2de800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.104+0000 7f2def7fe640 1 -- 172.21.15.3:0/2609370817 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8017070 con 0x7f2df8103bc0 2024-09-21T06:13:45.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.104+0000 7f2dfcc53640 1 -- 172.21.15.3:0/2609370817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df81a8a20 con 0x7f2df8103bc0 2024-09-21T06:13:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.104+0000 7f2dfcc53640 1 -- 172.21.15.3:0/2609370817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df81a8f60 con 0x7f2df8103bc0 2024-09-21T06:13:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.104+0000 7f2def7fe640 1 -- 172.21.15.3:0/2609370817 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de80054d0 con 0x7f2df8103bc0 2024-09-21T06:13:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.104+0000 7f2def7fe640 1 -- 172.21.15.3:0/2609370817 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8004c30 con 0x7f2df8103bc0 2024-09-21T06:13:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.105+0000 7f2dfcc53640 1 -- 172.21.15.3:0/2609370817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df8103fd0 con 0x7f2df8103bc0 2024-09-21T06:13:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.107+0000 7f2def7fe640 1 -- 172.21.15.3:0/2609370817 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f2de8004dd0 con 0x7f2df8103bc0 2024-09-21T06:13:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.108+0000 7f2def7fe640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2de40b65f0 0x7f2de40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.108+0000 7f2deffff640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2de40b65f0 0x7f2de40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.108+0000 7f2def7fe640 1 -- 172.21.15.3:0/2609370817 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2de80e7510 con 0x7f2df8103bc0 2024-09-21T06:13:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.109+0000 7f2deffff640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2de40b65f0 0x7f2de40b8b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2de0000c00 tx=0x7f2de0031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:45.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.115+0000 7f2def7fe640 1 -- 172.21.15.3:0/2609370817 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2de80adb00 con 0x7f2df8103bc0 2024-09-21T06:13:45.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:44 smithi137 ceph-mon[34464]: pgmap v287: 97 pgs: 97 active+clean; 647 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-21T06:13:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.358+0000 7f2dfcc53640 1 -- 172.21.15.3:0/2609370817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f2df802e640 con 0x7f2df8103bc0 2024-09-21T06:13:45.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.366+0000 7f2def7fe640 1 -- 172.21.15.3:0/2609370817 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v22) ==== 74+0+296443 (secure 0 0 0) 0x7f2de80b1ee0 con 0x7f2df8103bc0 2024-09-21T06:13:45.407 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:13:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.370+0000 7f2dfcc53640 1 -- 172.21.15.3:0/2609370817 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2de40b65f0 msgr2=0x7f2de40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.370+0000 7f2dfcc53640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2de40b65f0 0x7f2de40b8b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2de0000c00 tx=0x7f2de0031400 comp rx=0 tx=0).stop 2024-09-21T06:13:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.370+0000 7f2dfcc53640 1 -- 172.21.15.3:0/2609370817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df8103bc0 msgr2=0x7f2df8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.370+0000 7f2dfcc53640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df8103bc0 0x7f2df8075710 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2de800d770 tx=0x7f2de800dc30 comp rx=0 tx=0).stop 2024-09-21T06:13:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.370+0000 7f2dfcc53640 1 -- 172.21.15.3:0/2609370817 shutdown_connections 2024-09-21T06:13:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.370+0000 7f2dfcc53640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2de40b65f0 0x7f2de40b8b00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.370+0000 7f2dfcc53640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2df81094b0 0x7f2df8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.370+0000 7f2dfcc53640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df8109e00 0x7f2df8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.370+0000 7f2dfcc53640 1 --2- 172.21.15.3:0/2609370817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df8103bc0 0x7f2df8075710 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.370+0000 7f2dfcc53640 1 -- 172.21.15.3:0/2609370817 >> 172.21.15.3:0/2609370817 conn(0x7f2df807b910 msgr2=0x7f2df8105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.370+0000 7f2dfcc53640 1 -- 172.21.15.3:0/2609370817 shutdown_connections 2024-09-21T06:13:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:45.370+0000 7f2dfcc53640 1 -- 172.21.15.3:0/2609370817 wait complete. 2024-09-21T06:13:46.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:45 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2609370817' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T06:13:46.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:45 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2609370817' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T06:13:46.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:45 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2609370817' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T06:13:46.210 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"flags":0,"active_gid":14223,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2227168690},{"type":"v1","addr":"172.21.15.3:6801","nonce":2227168690}]},"active_addr":"172.21.15.3:6801/2227168690","active_change":"2024-09-21T06:06:23.529345+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24142,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24164,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.3:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2266698128}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":4066069382}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2473672193}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":394674212}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":374544884}]}]} 2024-09-21T06:13:46.216 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T06:13:46.217 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T06:13:46.217 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:13:46.490 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:46.889 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:46 smithi003 ceph-mon[31227]: pgmap v288: 97 pgs: 97 active+clean; 647 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-09-21T06:13:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.980+0000 7f3eea094640 1 -- 172.21.15.3:0/2253194230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ee4108cf0 msgr2=0x7f3ee410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.980+0000 7f3eea094640 1 --2- 172.21.15.3:0/2253194230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ee4108cf0 0x7f3ee410f520 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3edc009ca0 tx=0x7f3edc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.980+0000 7f3eea094640 1 -- 172.21.15.3:0/2253194230 shutdown_connections 2024-09-21T06:13:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.980+0000 7f3eea094640 1 --2- 172.21.15.3:0/2253194230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ee4108cf0 0x7f3ee410f520 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.980+0000 7f3eea094640 1 --2- 172.21.15.3:0/2253194230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ee41083a0 0x7f3ee41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.980+0000 7f3eea094640 1 --2- 172.21.15.3:0/2253194230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee4102a60 0x7f3ee4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.980+0000 7f3eea094640 1 -- 172.21.15.3:0/2253194230 >> 172.21.15.3:0/2253194230 conn(0x7f3ee4079730 msgr2=0x7f3ee4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.980+0000 7f3eea094640 1 -- 172.21.15.3:0/2253194230 shutdown_connections 2024-09-21T06:13:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.980+0000 7f3eea094640 1 -- 172.21.15.3:0/2253194230 wait complete. 2024-09-21T06:13:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.981+0000 7f3eea094640 1 Processor -- start 2024-09-21T06:13:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.981+0000 7f3eea094640 1 -- start start 2024-09-21T06:13:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.981+0000 7f3eea094640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee4102a60 0x7f3ee40ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.981+0000 7f3eea094640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ee41083a0 0x7f3ee40ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.981+0000 7f3eea094640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ee4108cf0 0x7f3ee41000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.981+0000 7f3eea094640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee4112960 con 0x7f3ee41083a0 2024-09-21T06:13:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.981+0000 7f3eea094640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee41127e0 con 0x7f3ee4108cf0 2024-09-21T06:13:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.981+0000 7f3eea094640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee4112ae0 con 0x7f3ee4102a60 2024-09-21T06:13:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ee41083a0 0x7f3ee40ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ee41083a0 0x7f3ee40ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53812/0 (socket says 172.21.15.3:53812) 2024-09-21T06:13:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee2ffd640 1 -- 172.21.15.3:0/1847531370 learned_addr learned my addr 172.21.15.3:0/1847531370 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee37fe640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee4102a60 0x7f3ee40ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee3fff640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ee4108cf0 0x7f3ee41000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee2ffd640 1 -- 172.21.15.3:0/1847531370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee4102a60 msgr2=0x7f3ee40ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee2ffd640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee4102a60 0x7f3ee40ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee2ffd640 1 -- 172.21.15.3:0/1847531370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ee4108cf0 msgr2=0x7f3ee41000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee2ffd640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ee4108cf0 0x7f3ee41000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee2ffd640 1 -- 172.21.15.3:0/1847531370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee41a85f0 con 0x7f3ee41083a0 2024-09-21T06:13:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee3fff640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ee4108cf0 0x7f3ee41000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee2ffd640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ee41083a0 0x7f3ee40ffbb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800c970 tx=0x7f3ed800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee0ff9640 1 -- 172.21.15.3:0/1847531370 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8014070 con 0x7f3ee41083a0 2024-09-21T06:13:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3eea094640 1 -- 172.21.15.3:0/1847531370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee41a88e0 con 0x7f3ee41083a0 2024-09-21T06:13:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee0ff9640 1 -- 172.21.15.3:0/1847531370 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ed8004440 con 0x7f3ee41083a0 2024-09-21T06:13:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3ee0ff9640 1 -- 172.21.15.3:0/1847531370 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8003d30 con 0x7f3ee41083a0 2024-09-21T06:13:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.982+0000 7f3eea094640 1 -- 172.21.15.3:0/1847531370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee41a8e20 con 0x7f3ee41083a0 2024-09-21T06:13:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.983+0000 7f3eea094640 1 -- 172.21.15.3:0/1847531370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee4102e70 con 0x7f3ee41083a0 2024-09-21T06:13:47.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.985+0000 7f3ee0ff9640 1 -- 172.21.15.3:0/1847531370 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f3ed8017070 con 0x7f3ee41083a0 2024-09-21T06:13:47.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.986+0000 7f3ee0ff9640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3ecc0b65f0 0x7f3ecc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:47.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.986+0000 7f3ee37fe640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3ecc0b65f0 0x7f3ecc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:47.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.986+0000 7f3ee0ff9640 1 -- 172.21.15.3:0/1847531370 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f3ed80e7860 con 0x7f3ee41083a0 2024-09-21T06:13:47.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.987+0000 7f3ee37fe640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3ecc0b65f0 0x7f3ecc0b8b00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ee41010c0 tx=0x7f3ed400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:47.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:46.989+0000 7f3ee0ff9640 1 -- 172.21.15.3:0/1847531370 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3ed801e020 con 0x7f3ee41083a0 2024-09-21T06:13:47.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:46 smithi137 ceph-mon[34464]: pgmap v288: 97 pgs: 97 active+clean; 647 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-09-21T06:13:47.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:46 smithi204 ceph-mon[34659]: pgmap v288: 97 pgs: 97 active+clean; 647 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-09-21T06:13:47.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.183+0000 7f3eea094640 1 -- 172.21.15.3:0/1847531370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ee410b540 con 0x7f3ee41083a0 2024-09-21T06:13:47.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.184+0000 7f3ee0ff9640 1 -- 172.21.15.3:0/1847531370 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23048 (secure 0 0 0) 0x7f3ed80ade50 con 0x7f3ee41083a0 2024-09-21T06:13:47.187 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:13:47.187 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","created":"2024-09-21T06:04:24.293786+0000","modified":"2024-09-21T06:13:24.594225+0000","last_up_change":"2024-09-21T06:12:17.781243+0000","last_in_change":"2024-09-21T06:11:54.727435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:08:26.015803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:13:07.084406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:13:09.149891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51736f81-f410-48ee-a8f3-121289607309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6803","nonce":615312328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6805","nonce":615312328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6809","nonce":615312328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6807","nonce":615312328}]},"public_addr":"172.21.15.3:6803/615312328","cluster_addr":"172.21.15.3:6805/615312328","heartbeat_back_addr":"172.21.15.3:6809/615312328","heartbeat_front_addr":"172.21.15.3:6807/615312328","state":["exists","up"]},{"osd":1,"uuid":"55696a37-34c1-482a-8561-7d08f0aa29fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6801","nonce":4150747115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6803","nonce":4150747115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6807","nonce":4150747115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6805","nonce":4150747115}]},"public_addr":"172.21.15.137:6801/4150747115","cluster_addr":"172.21.15.137:6803/4150747115","heartbeat_back_addr":"172.21.15.137:6807/4150747115","heartbeat_front_addr":"172.21.15.137:6805/4150747115","state":["exists","up"]},{"osd":2,"uuid":"8a81d34b-464b-46a2-8ef2-5223b5947004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6801","nonce":3392577077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6803","nonce":3392577077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6807","nonce":3392577077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6805","nonce":3392577077}]},"public_addr":"172.21.15.204:6801/3392577077","cluster_addr":"172.21.15.204:6803/3392577077","heartbeat_back_addr":"172.21.15.204:6807/3392577077","heartbeat_front_addr":"172.21.15.204:6805/3392577077","state":["exists","up"]},{"osd":3,"uuid":"94ab3938-7a70-44cb-99bf-9d2dedc4eb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6811","nonce":4293741635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6813","nonce":4293741635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6817","nonce":4293741635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6815","nonce":4293741635}]},"public_addr":"172.21.15.3:6811/4293741635","cluster_addr":"172.21.15.3:6813/4293741635","heartbeat_back_addr":"172.21.15.3:6817/4293741635","heartbeat_front_addr":"172.21.15.3:6815/4293741635","state":["exists","up"]},{"osd":4,"uuid":"38551535-d692-401c-aad0-bee38f3026d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6809","nonce":2608988785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6811","nonce":2608988785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6815","nonce":2608988785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6813","nonce":2608988785}]},"public_addr":"172.21.15.137:6809/2608988785","cluster_addr":"172.21.15.137:6811/2608988785","heartbeat_back_addr":"172.21.15.137:6815/2608988785","heartbeat_front_addr":"172.21.15.137:6813/2608988785","state":["exists","up"]},{"osd":5,"uuid":"3d4b785b-28d0-4654-a9b1-4c5c905f4bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6809","nonce":3709074044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6811","nonce":3709074044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6815","nonce":3709074044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6813","nonce":3709074044}]},"public_addr":"172.21.15.204:6809/3709074044","cluster_addr":"172.21.15.204:6811/3709074044","heartbeat_back_addr":"172.21.15.204:6815/3709074044","heartbeat_front_addr":"172.21.15.204:6813/3709074044","state":["exists","up"]},{"osd":6,"uuid":"ba21dbe9-ab43-4b7e-9446-2929ee874c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6819","nonce":3862646594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6821","nonce":3862646594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6825","nonce":3862646594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6823","nonce":3862646594}]},"public_addr":"172.21.15.3:6819/3862646594","cluster_addr":"172.21.15.3:6821/3862646594","heartbeat_back_addr":"172.21.15.3:6825/3862646594","heartbeat_front_addr":"172.21.15.3:6823/3862646594","state":["exists","up"]},{"osd":7,"uuid":"9610fe5c-93df-49f9-b8bd-5cb480e0871b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6817","nonce":2086822496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6819","nonce":2086822496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6823","nonce":2086822496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6821","nonce":2086822496}]},"public_addr":"172.21.15.137:6817/2086822496","cluster_addr":"172.21.15.137:6819/2086822496","heartbeat_back_addr":"172.21.15.137:6823/2086822496","heartbeat_front_addr":"172.21.15.137:6821/2086822496","state":["exists","up"]},{"osd":8,"uuid":"49f6e9b4-6d14-434e-8b42-8b7ffeb2a072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6817","nonce":153476798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6819","nonce":153476798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6823","nonce":153476798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6821","nonce":153476798}]},"public_addr":"172.21.15.204:6817/153476798","cluster_addr":"172.21.15.204:6819/153476798","heartbeat_back_addr":"172.21.15.204:6823/153476798","heartbeat_front_addr":"172.21.15.204:6821/153476798","state":["exists","up"]},{"osd":9,"uuid":"633f67a7-b759-4ea6-82ed-3c42b3fb70ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6827","nonce":1238491549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6829","nonce":1238491549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6833","nonce":1238491549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6831","nonce":1238491549}]},"public_addr":"172.21.15.3:6827/1238491549","cluster_addr":"172.21.15.3:6829/1238491549","heartbeat_back_addr":"172.21.15.3:6833/1238491549","heartbeat_front_addr":"172.21.15.3:6831/1238491549","state":["exists","up"]},{"osd":10,"uuid":"94b65a48-ee87-4e59-9296-a72a22787f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6825","nonce":3486552683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6827","nonce":3486552683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6831","nonce":3486552683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6829","nonce":3486552683}]},"public_addr":"172.21.15.137:6825/3486552683","cluster_addr":"172.21.15.137:6827/3486552683","heartbeat_back_addr":"172.21.15.137:6831/3486552683","heartbeat_front_addr":"172.21.15.137:6829/3486552683","state":["exists","up"]},{"osd":11,"uuid":"c5cc2850-9062-4b8e-beb3-bef0db30a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6825","nonce":1084308610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6827","nonce":1084308610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6831","nonce":1084308610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6829","nonce":1084308610}]},"public_addr":"172.21.15.204:6825/1084308610","cluster_addr":"172.21.15.204:6827/1084308610","heartbeat_back_addr":"172.21.15.204:6831/1084308610","heartbeat_front_addr":"172.21.15.204:6829/1084308610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:31.898745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.089965+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:22.367464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:48.543248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:14.687331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:41.243579+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:07.290227+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:32.930046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:59.197638+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:24.315219+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:50.333639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:12:16.453640+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1573374138":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1595702358":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/547897695":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/3673347975":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/141972779":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6800/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:6801/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/2363985440":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/2138752581":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1820734047":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/2929116087":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/45453677":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/2929116087":"2024-09-22T06:05:21.689191+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:13:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.187+0000 7f3eea094640 1 -- 172.21.15.3:0/1847531370 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3ecc0b65f0 msgr2=0x7f3ecc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.187+0000 7f3eea094640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3ecc0b65f0 0x7f3ecc0b8b00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ee41010c0 tx=0x7f3ed400a2d0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.187+0000 7f3eea094640 1 -- 172.21.15.3:0/1847531370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ee41083a0 msgr2=0x7f3ee40ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.187+0000 7f3eea094640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ee41083a0 0x7f3ee40ffbb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800c970 tx=0x7f3ed800cd70 comp rx=0 tx=0).stop 2024-09-21T06:13:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.187+0000 7f3eea094640 1 -- 172.21.15.3:0/1847531370 shutdown_connections 2024-09-21T06:13:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.187+0000 7f3eea094640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3ecc0b65f0 0x7f3ecc0b8b00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.187+0000 7f3eea094640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ee4108cf0 0x7f3ee41000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.187+0000 7f3eea094640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ee41083a0 0x7f3ee40ffbb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.187+0000 7f3eea094640 1 --2- 172.21.15.3:0/1847531370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee4102a60 0x7f3ee40ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.187+0000 7f3eea094640 1 -- 172.21.15.3:0/1847531370 >> 172.21.15.3:0/1847531370 conn(0x7f3ee4079730 msgr2=0x7f3ee406b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:47.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.187+0000 7f3eea094640 1 -- 172.21.15.3:0/1847531370 shutdown_connections 2024-09-21T06:13:47.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:47.187+0000 7f3eea094640 1 -- 172.21.15.3:0/1847531370 wait complete. 2024-09-21T06:13:47.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:47 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1847531370' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:48.002 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T06:13:48.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:13:48.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:47 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1847531370' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:48.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:47 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1847531370' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.759+0000 7fb97826d640 1 -- 172.21.15.3:0/557833667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb97010de10 msgr2=0x7fb970110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.759+0000 7fb97826d640 1 --2- 172.21.15.3:0/557833667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb97010de10 0x7fb970110300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb960009ca0 tx=0x7fb96002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.759+0000 7fb97826d640 1 -- 172.21.15.3:0/557833667 shutdown_connections 2024-09-21T06:13:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.759+0000 7fb97826d640 1 --2- 172.21.15.3:0/557833667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb970105300 0x7fb97010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.759+0000 7fb97826d640 1 --2- 172.21.15.3:0/557833667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb97010de10 0x7fb970110300 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.759+0000 7fb97826d640 1 --2- 172.21.15.3:0/557833667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9701049b0 0x7fb970104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.759+0000 7fb97826d640 1 -- 172.21.15.3:0/557833667 >> 172.21.15.3:0/557833667 conn(0x7fb970079730 msgr2=0x7fb970079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.759+0000 7fb97826d640 1 -- 172.21.15.3:0/557833667 shutdown_connections 2024-09-21T06:13:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.759+0000 7fb97826d640 1 -- 172.21.15.3:0/557833667 wait complete. 2024-09-21T06:13:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.760+0000 7fb97826d640 1 Processor -- start 2024-09-21T06:13:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.760+0000 7fb97826d640 1 -- start start 2024-09-21T06:13:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.760+0000 7fb97826d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9701049b0 0x7fb97019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.760+0000 7fb97826d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb970105300 0x7fb97019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.760+0000 7fb97826d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb97010de10 0x7fb97019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.760+0000 7fb97826d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb970113a70 con 0x7fb9701049b0 2024-09-21T06:13:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.760+0000 7fb97826d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb9701138f0 con 0x7fb97010de10 2024-09-21T06:13:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.760+0000 7fb97826d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb970113bf0 con 0x7fb970105300 2024-09-21T06:13:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.760+0000 7fb976009640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9701049b0 0x7fb97019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb976009640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9701049b0 0x7fb97019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53842/0 (socket says 172.21.15.3:53842) 2024-09-21T06:13:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb976009640 1 -- 172.21.15.3:0/524734687 learned_addr learned my addr 172.21.15.3:0/524734687 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb97680a640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb97010de10 0x7fb97019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb975808640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb970105300 0x7fb97019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb976009640 1 -- 172.21.15.3:0/524734687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb970105300 msgr2=0x7fb97019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb976009640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb970105300 0x7fb97019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb976009640 1 -- 172.21.15.3:0/524734687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb97010de10 msgr2=0x7fb97019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb976009640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb97010de10 0x7fb97019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb976009640 1 -- 172.21.15.3:0/524734687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9701a0350 con 0x7fb9701049b0 2024-09-21T06:13:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb97680a640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb97010de10 0x7fb97019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb976009640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9701049b0 0x7fb97019a4a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb96800d770 tx=0x7fb96800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb96f7fe640 1 -- 172.21.15.3:0/524734687 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968017070 con 0x7fb9701049b0 2024-09-21T06:13:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb96f7fe640 1 -- 172.21.15.3:0/524734687 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb968009c30 con 0x7fb9701049b0 2024-09-21T06:13:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb97826d640 1 -- 172.21.15.3:0/524734687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9701a0640 con 0x7fb9701049b0 2024-09-21T06:13:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb96f7fe640 1 -- 172.21.15.3:0/524734687 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968003cc0 con 0x7fb9701049b0 2024-09-21T06:13:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.761+0000 7fb97826d640 1 -- 172.21.15.3:0/524734687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9701a6e60 con 0x7fb9701049b0 2024-09-21T06:13:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.762+0000 7fb97826d640 1 -- 172.21.15.3:0/524734687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb970104dc0 con 0x7fb9701049b0 2024-09-21T06:13:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.764+0000 7fb96f7fe640 1 -- 172.21.15.3:0/524734687 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fb968003e60 con 0x7fb9701049b0 2024-09-21T06:13:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.764+0000 7fb96f7fe640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb9580b66a0 0x7fb9580b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.765+0000 7fb975808640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb9580b66a0 0x7fb9580b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.765+0000 7fb96f7fe640 1 -- 172.21.15.3:0/524734687 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb9680e73c0 con 0x7fb9701049b0 2024-09-21T06:13:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.765+0000 7fb975808640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb9580b66a0 0x7fb9580b8bb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb97007a030 tx=0x7fb96400c4a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.768+0000 7fb96f7fe640 1 -- 172.21.15.3:0/524734687 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb9680ada90 con 0x7fb9701049b0 2024-09-21T06:13:48.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.960+0000 7fb97826d640 1 -- 172.21.15.3:0/524734687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb97019b5d0 con 0x7fb9701049b0 2024-09-21T06:13:48.963 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:48 smithi003 ceph-mon[31227]: pgmap v289: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-21T06:13:48.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.962+0000 7fb96f7fe640 1 -- 172.21.15.3:0/524734687 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23048 (secure 0 0 0) 0x7fb9680b1e70 con 0x7fb9701049b0 2024-09-21T06:13:48.964 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:13:48.964 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","created":"2024-09-21T06:04:24.293786+0000","modified":"2024-09-21T06:13:24.594225+0000","last_up_change":"2024-09-21T06:12:17.781243+0000","last_in_change":"2024-09-21T06:11:54.727435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:08:26.015803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:13:07.084406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:13:09.149891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51736f81-f410-48ee-a8f3-121289607309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6803","nonce":615312328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6805","nonce":615312328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6809","nonce":615312328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6807","nonce":615312328}]},"public_addr":"172.21.15.3:6803/615312328","cluster_addr":"172.21.15.3:6805/615312328","heartbeat_back_addr":"172.21.15.3:6809/615312328","heartbeat_front_addr":"172.21.15.3:6807/615312328","state":["exists","up"]},{"osd":1,"uuid":"55696a37-34c1-482a-8561-7d08f0aa29fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6801","nonce":4150747115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6803","nonce":4150747115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6807","nonce":4150747115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6805","nonce":4150747115}]},"public_addr":"172.21.15.137:6801/4150747115","cluster_addr":"172.21.15.137:6803/4150747115","heartbeat_back_addr":"172.21.15.137:6807/4150747115","heartbeat_front_addr":"172.21.15.137:6805/4150747115","state":["exists","up"]},{"osd":2,"uuid":"8a81d34b-464b-46a2-8ef2-5223b5947004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6801","nonce":3392577077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6803","nonce":3392577077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6807","nonce":3392577077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6805","nonce":3392577077}]},"public_addr":"172.21.15.204:6801/3392577077","cluster_addr":"172.21.15.204:6803/3392577077","heartbeat_back_addr":"172.21.15.204:6807/3392577077","heartbeat_front_addr":"172.21.15.204:6805/3392577077","state":["exists","up"]},{"osd":3,"uuid":"94ab3938-7a70-44cb-99bf-9d2dedc4eb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6811","nonce":4293741635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6813","nonce":4293741635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6817","nonce":4293741635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6815","nonce":4293741635}]},"public_addr":"172.21.15.3:6811/4293741635","cluster_addr":"172.21.15.3:6813/4293741635","heartbeat_back_addr":"172.21.15.3:6817/4293741635","heartbeat_front_addr":"172.21.15.3:6815/4293741635","state":["exists","up"]},{"osd":4,"uuid":"38551535-d692-401c-aad0-bee38f3026d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6809","nonce":2608988785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6811","nonce":2608988785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6815","nonce":2608988785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6813","nonce":2608988785}]},"public_addr":"172.21.15.137:6809/2608988785","cluster_addr":"172.21.15.137:6811/2608988785","heartbeat_back_addr":"172.21.15.137:6815/2608988785","heartbeat_front_addr":"172.21.15.137:6813/2608988785","state":["exists","up"]},{"osd":5,"uuid":"3d4b785b-28d0-4654-a9b1-4c5c905f4bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6809","nonce":3709074044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6811","nonce":3709074044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6815","nonce":3709074044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6813","nonce":3709074044}]},"public_addr":"172.21.15.204:6809/3709074044","cluster_addr":"172.21.15.204:6811/3709074044","heartbeat_back_addr":"172.21.15.204:6815/3709074044","heartbeat_front_addr":"172.21.15.204:6813/3709074044","state":["exists","up"]},{"osd":6,"uuid":"ba21dbe9-ab43-4b7e-9446-2929ee874c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6819","nonce":3862646594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6821","nonce":3862646594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6825","nonce":3862646594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6823","nonce":3862646594}]},"public_addr":"172.21.15.3:6819/3862646594","cluster_addr":"172.21.15.3:6821/3862646594","heartbeat_back_addr":"172.21.15.3:6825/3862646594","heartbeat_front_addr":"172.21.15.3:6823/3862646594","state":["exists","up"]},{"osd":7,"uuid":"9610fe5c-93df-49f9-b8bd-5cb480e0871b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6817","nonce":2086822496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6819","nonce":2086822496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6823","nonce":2086822496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6821","nonce":2086822496}]},"public_addr":"172.21.15.137:6817/2086822496","cluster_addr":"172.21.15.137:6819/2086822496","heartbeat_back_addr":"172.21.15.137:6823/2086822496","heartbeat_front_addr":"172.21.15.137:6821/2086822496","state":["exists","up"]},{"osd":8,"uuid":"49f6e9b4-6d14-434e-8b42-8b7ffeb2a072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6817","nonce":153476798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6819","nonce":153476798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6823","nonce":153476798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6821","nonce":153476798}]},"public_addr":"172.21.15.204:6817/153476798","cluster_addr":"172.21.15.204:6819/153476798","heartbeat_back_addr":"172.21.15.204:6823/153476798","heartbeat_front_addr":"172.21.15.204:6821/153476798","state":["exists","up"]},{"osd":9,"uuid":"633f67a7-b759-4ea6-82ed-3c42b3fb70ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6827","nonce":1238491549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6829","nonce":1238491549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6833","nonce":1238491549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6831","nonce":1238491549}]},"public_addr":"172.21.15.3:6827/1238491549","cluster_addr":"172.21.15.3:6829/1238491549","heartbeat_back_addr":"172.21.15.3:6833/1238491549","heartbeat_front_addr":"172.21.15.3:6831/1238491549","state":["exists","up"]},{"osd":10,"uuid":"94b65a48-ee87-4e59-9296-a72a22787f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6825","nonce":3486552683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6827","nonce":3486552683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6831","nonce":3486552683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6829","nonce":3486552683}]},"public_addr":"172.21.15.137:6825/3486552683","cluster_addr":"172.21.15.137:6827/3486552683","heartbeat_back_addr":"172.21.15.137:6831/3486552683","heartbeat_front_addr":"172.21.15.137:6829/3486552683","state":["exists","up"]},{"osd":11,"uuid":"c5cc2850-9062-4b8e-beb3-bef0db30a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6825","nonce":1084308610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6827","nonce":1084308610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6831","nonce":1084308610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6829","nonce":1084308610}]},"public_addr":"172.21.15.204:6825/1084308610","cluster_addr":"172.21.15.204:6827/1084308610","heartbeat_back_addr":"172.21.15.204:6831/1084308610","heartbeat_front_addr":"172.21.15.204:6829/1084308610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:31.898745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.089965+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:22.367464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:48.543248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:14.687331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:41.243579+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:07.290227+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:32.930046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:59.197638+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:24.315219+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:50.333639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:12:16.453640+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1573374138":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1595702358":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/547897695":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/3673347975":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/141972779":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6800/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:6801/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/2363985440":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/2138752581":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1820734047":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/2929116087":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/45453677":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/2929116087":"2024-09-22T06:05:21.689191+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:13:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.964+0000 7fb97826d640 1 -- 172.21.15.3:0/524734687 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb9580b66a0 msgr2=0x7fb9580b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.964+0000 7fb97826d640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb9580b66a0 0x7fb9580b8bb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb97007a030 tx=0x7fb96400c4a0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.964+0000 7fb97826d640 1 -- 172.21.15.3:0/524734687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9701049b0 msgr2=0x7fb97019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.964+0000 7fb97826d640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9701049b0 0x7fb97019a4a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb96800d770 tx=0x7fb96800dc30 comp rx=0 tx=0).stop 2024-09-21T06:13:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.964+0000 7fb97826d640 1 -- 172.21.15.3:0/524734687 shutdown_connections 2024-09-21T06:13:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.964+0000 7fb97826d640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb9580b66a0 0x7fb9580b8bb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.964+0000 7fb97826d640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb970105300 0x7fb97019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.964+0000 7fb97826d640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb97010de10 0x7fb97019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.964+0000 7fb97826d640 1 --2- 172.21.15.3:0/524734687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9701049b0 0x7fb97019a4a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.964+0000 7fb97826d640 1 -- 172.21.15.3:0/524734687 >> 172.21.15.3:0/524734687 conn(0x7fb970079730 msgr2=0x7fb9701007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.965+0000 7fb97826d640 1 -- 172.21.15.3:0/524734687 shutdown_connections 2024-09-21T06:13:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:48.965+0000 7fb97826d640 1 -- 172.21.15.3:0/524734687 wait complete. 2024-09-21T06:13:49.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:48 smithi137 ceph-mon[34464]: pgmap v289: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-21T06:13:49.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:48 smithi204 ceph-mon[34659]: pgmap v289: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-21T06:13:49.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T06:13:49.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T06:13:49.746 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T06:13:49.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T06:13:49.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T06:13:49.748 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T06:13:49.748 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T06:13:49.748 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T06:13:49.749 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T06:13:49.749 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T06:13:49.750 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T06:13:49.750 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T06:13:50.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:49 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/524734687' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:50.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:49 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/524734687' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:50.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:49 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/524734687' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:13:50.417 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:50.526 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:50.527 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:50.528 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:50.528 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:50.528 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:50.531 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:50.762 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:50.771 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:50.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:50 smithi003 ceph-mon[31227]: pgmap v290: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T06:13:51.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:50 smithi137 ceph-mon[34464]: pgmap v290: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T06:13:51.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:50 smithi204 ceph-mon[34659]: pgmap v290: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T06:13:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.250+0000 7fef9825b640 1 -- 172.21.15.3:0/25616743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef90103b60 msgr2=0x7fef90103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.250+0000 7fef9825b640 1 --2- 172.21.15.3:0/25616743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef90103b60 0x7fef90103f70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009900 tx=0x7fef8c02a410 comp rx=0 tx=0).stop 2024-09-21T06:13:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.250+0000 7fef9825b640 1 -- 172.21.15.3:0/25616743 shutdown_connections 2024-09-21T06:13:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.250+0000 7fef9825b640 1 --2- 172.21.15.3:0/25616743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90109df0 0x7fef90110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.250+0000 7fef9825b640 1 --2- 172.21.15.3:0/25616743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef901094a0 0x7fef901098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.250+0000 7fef9825b640 1 --2- 172.21.15.3:0/25616743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef90103b60 0x7fef90103f70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.250+0000 7fef9825b640 1 -- 172.21.15.3:0/25616743 >> 172.21.15.3:0/25616743 conn(0x7fef9007b850 msgr2=0x7fef9007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.250+0000 7fef9825b640 1 -- 172.21.15.3:0/25616743 shutdown_connections 2024-09-21T06:13:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.250+0000 7fef9825b640 1 -- 172.21.15.3:0/25616743 wait complete. 2024-09-21T06:13:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.251+0000 7fef9825b640 1 Processor -- start 2024-09-21T06:13:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.251+0000 7fef9825b640 1 -- start start 2024-09-21T06:13:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.251+0000 7fef9825b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef90103b60 0x7fef90074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.251+0000 7fef9825b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef901094a0 0x7fef9006e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.251+0000 7fef9825b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90109df0 0x7fef9006e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef9825b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fef90113ab0 con 0x7fef901094a0 2024-09-21T06:13:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef9825b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef90113930 con 0x7fef90103b60 2024-09-21T06:13:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef9825b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fef90113c30 con 0x7fef90109df0 2024-09-21T06:13:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef957f6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef901094a0 0x7fef9006e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef957f6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef901094a0 0x7fef9006e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53854/0 (socket says 172.21.15.3:53854) 2024-09-21T06:13:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef957f6640 1 -- 172.21.15.3:0/1652763236 learned_addr learned my addr 172.21.15.3:0/1652763236 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef95ff7640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef90103b60 0x7fef90074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef967f8640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90109df0 0x7fef9006e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef957f6640 1 -- 172.21.15.3:0/1652763236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90109df0 msgr2=0x7fef9006e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef957f6640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90109df0 0x7fef9006e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef957f6640 1 -- 172.21.15.3:0/1652763236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef90103b60 msgr2=0x7fef90074ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef957f6640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef90103b60 0x7fef90074ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef957f6640 1 -- 172.21.15.3:0/1652763236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9006ef40 con 0x7fef901094a0 2024-09-21T06:13:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef957f6640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef901094a0 0x7fef9006e1b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c8c0 tx=0x7fef8000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef86ffd640 1 -- 172.21.15.3:0/1652763236 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80007bb0 con 0x7fef901094a0 2024-09-21T06:13:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef86ffd640 1 -- 172.21.15.3:0/1652763236 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef80007d50 con 0x7fef901094a0 2024-09-21T06:13:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.252+0000 7fef9825b640 1 -- 172.21.15.3:0/1652763236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef90073c10 con 0x7fef901094a0 2024-09-21T06:13:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.253+0000 7fef86ffd640 1 -- 172.21.15.3:0/1652763236 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800187c0 con 0x7fef901094a0 2024-09-21T06:13:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.253+0000 7fef9825b640 1 -- 172.21.15.3:0/1652763236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef90074100 con 0x7fef901094a0 2024-09-21T06:13:51.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.254+0000 7fef9825b640 1 -- 172.21.15.3:0/1652763236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef90074710 con 0x7fef901094a0 2024-09-21T06:13:51.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.255+0000 7fef86ffd640 1 -- 172.21.15.3:0/1652763236 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fef8000fa20 con 0x7fef901094a0 2024-09-21T06:13:51.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.256+0000 7fef86ffd640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fef7c0b65f0 0x7fef7c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.256+0000 7fef95ff7640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fef7c0b65f0 0x7fef7c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.256+0000 7fef86ffd640 1 -- 172.21.15.3:0/1652763236 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fef800e7ad0 con 0x7fef901094a0 2024-09-21T06:13:51.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.256+0000 7fef86ffd640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594] conn(0x7fef7c0c03e0 0x7fef7c0c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.256+0000 7fef86ffd640 1 -- 172.21.15.3:0/1652763236 --> [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef80010e70 con 0x7fef7c0c03e0 2024-09-21T06:13:51.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.257+0000 7fef86ffd640 1 -- 172.21.15.3:0/1652763236 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fef800e7e80 con 0x7fef901094a0 2024-09-21T06:13:51.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.257+0000 7fef967f8640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594] conn(0x7fef7c0c03e0 0x7fef7c0c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.257+0000 7fef95ff7640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fef7c0b65f0 0x7fef7c0b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fef8c031440 tx=0x7fef8c009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.257+0000 7fef967f8640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594] conn(0x7fef7c0c03e0 0x7fef7c0c28d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.258+0000 7fef86ffd640 1 -- 172.21.15.3:0/1652763236 <== osd.6 v2:172.21.15.3:6818/3862646594 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef80010e70 con 0x7fef7c0c03e0 2024-09-21T06:13:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.271+0000 7fef9825b640 1 -- 172.21.15.3:0/1652763236 --> [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef90104de0 con 0x7fef7c0c03e0 2024-09-21T06:13:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.272+0000 7fef86ffd640 1 -- 172.21.15.3:0/1652763236 <== osd.6 v2:172.21.15.3:6818/3862646594 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fef90104de0 con 0x7fef7c0c03e0 2024-09-21T06:13:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.272+0000 7fef9825b640 1 -- 172.21.15.3:0/1652763236 >> [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594] conn(0x7fef7c0c03e0 msgr2=0x7fef7c0c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.272+0000 7fef9825b640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594] conn(0x7fef7c0c03e0 0x7fef7c0c28d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.272+0000 7fef9825b640 1 -- 172.21.15.3:0/1652763236 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fef7c0b65f0 msgr2=0x7fef7c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.272+0000 7fef9825b640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fef7c0b65f0 0x7fef7c0b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fef8c031440 tx=0x7fef8c009560 comp rx=0 tx=0).stop 2024-09-21T06:13:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.272+0000 7fef9825b640 1 -- 172.21.15.3:0/1652763236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef901094a0 msgr2=0x7fef9006e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.272+0000 7fef9825b640 1 --2- 172.21.15.3:0/1652763236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef901094a0 0x7fef9006e1b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c8c0 tx=0x7fef8000cd80 comp rx=0 tx=0).stop 2024-09-21T06:13:51.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.272+0000 7fef967f8640 1 -- 172.21.15.3:0/1652763236 reap_dead start 2024-09-21T06:13:51.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.272+0000 7fef9825b640 1 -- 172.21.15.3:0/1652763236 shutdown_connections 2024-09-21T06:13:51.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.272+0000 7fef9825b640 1 -- 172.21.15.3:0/1652763236 >> 172.21.15.3:0/1652763236 conn(0x7fef9007b850 msgr2=0x7fef90107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:51.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.272+0000 7fef9825b640 1 -- 172.21.15.3:0/1652763236 shutdown_connections 2024-09-21T06:13:51.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.272+0000 7fef9825b640 1 -- 172.21.15.3:0/1652763236 wait complete. 2024-09-21T06:13:51.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.683+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/777283652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0ec1052a0 msgr2=0x7fb0ec10d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.683+0000 7fb0f1dc7640 1 --2- 172.21.15.3:0/777283652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0ec1052a0 0x7fb0ec10d8e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009900 tx=0x7fb0e002a410 comp rx=0 tx=0).stop 2024-09-21T06:13:51.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.683+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/777283652 shutdown_connections 2024-09-21T06:13:51.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.683+0000 7fb0f1dc7640 1 --2- 172.21.15.3:0/777283652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0ec10de20 0x7fb0ec110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.683+0000 7fb0f1dc7640 1 --2- 172.21.15.3:0/777283652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0ec1052a0 0x7fb0ec10d8e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.683+0000 7fb0f1dc7640 1 --2- 172.21.15.3:0/777283652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0ec104870 0x7fb0ec104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.683+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/777283652 >> 172.21.15.3:0/777283652 conn(0x7fb0ec079750 msgr2=0x7fb0ec079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:51.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.684+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/777283652 shutdown_connections 2024-09-21T06:13:51.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.684+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/777283652 wait complete. 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.684+0000 7fb0f1dc7640 1 Processor -- start 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0f1dc7640 1 -- start start 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0f1dc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0ec104870 0x7fb0ec19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0f1dc7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0ec1052a0 0x7fb0ec19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0f1dc7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0ec10de20 0x7fb0ec19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0f1dc7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec113aa0 con 0x7fb0ec10de20 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0f1dc7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec113920 con 0x7fb0ec104870 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0f1dc7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec113c20 con 0x7fb0ec1052a0 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0ebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0ec10de20 0x7fb0ec19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0ebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0ec10de20 0x7fb0ec19fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53868/0 (socket says 172.21.15.3:53868) 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0ebfff640 1 -- 172.21.15.3:0/780353062 learned_addr learned my addr 172.21.15.3:0/780353062 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0eb7fe640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0ec104870 0x7fb0ec19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0eaffd640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0ec1052a0 0x7fb0ec19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0ebfff640 1 -- 172.21.15.3:0/780353062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0ec1052a0 msgr2=0x7fb0ec19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0ebfff640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0ec1052a0 0x7fb0ec19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0ebfff640 1 -- 172.21.15.3:0/780353062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0ec104870 msgr2=0x7fb0ec19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0ebfff640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0ec104870 0x7fb0ec19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.685+0000 7fb0ebfff640 1 -- 172.21.15.3:0/780353062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ec1a0420 con 0x7fb0ec10de20 2024-09-21T06:13:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.686+0000 7fb0ebfff640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0ec10de20 0x7fb0ec19fcf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400eaa0 tx=0x7fb0e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.686+0000 7fb0e8ff9640 1 -- 172.21.15.3:0/780353062 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400cdb0 con 0x7fb0ec10de20 2024-09-21T06:13:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.686+0000 7fb0e8ff9640 1 -- 172.21.15.3:0/780353062 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e4005020 con 0x7fb0ec10de20 2024-09-21T06:13:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.686+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/780353062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ec1a0710 con 0x7fb0ec10de20 2024-09-21T06:13:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.686+0000 7fb0e8ff9640 1 -- 172.21.15.3:0/780353062 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e40116d0 con 0x7fb0ec10de20 2024-09-21T06:13:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.686+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/780353062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ec1a6e70 con 0x7fb0ec10de20 2024-09-21T06:13:51.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.687+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/780353062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb0ec108420 con 0x7fb0ec10de20 2024-09-21T06:13:51.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.689+0000 7fb0e8ff9640 1 -- 172.21.15.3:0/780353062 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fb0e40051d0 con 0x7fb0ec10de20 2024-09-21T06:13:51.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.689+0000 7fb0e8ff9640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb0d40b65f0 0x7fb0d40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.690+0000 7fb0eb7fe640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb0d40b65f0 0x7fb0d40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.690+0000 7fb0e8ff9640 1 -- 172.21.15.3:0/780353062 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb0e40e7d30 con 0x7fb0ec10de20 2024-09-21T06:13:51.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.690+0000 7fb0e8ff9640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683] conn(0x7fb0d40c03e0 0x7fb0d40c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:51.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.690+0000 7fb0e8ff9640 1 -- 172.21.15.3:0/780353062 --> [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0e4014ec0 con 0x7fb0d40c03e0 2024-09-21T06:13:51.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.690+0000 7fb0e8ff9640 1 -- 172.21.15.3:0/780353062 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fb0e40e8180 con 0x7fb0ec10de20 2024-09-21T06:13:51.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.690+0000 7fb0eb7fe640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb0d40b65f0 0x7fb0d40b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec06c920 tx=0x7fb0dc00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:51.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.690+0000 7fb0eaffd640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683] conn(0x7fb0d40c03e0 0x7fb0d40c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:51.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.691+0000 7fb0eaffd640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683] conn(0x7fb0d40c03e0 0x7fb0d40c28d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:51.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.692+0000 7fb0e8ff9640 1 -- 172.21.15.3:0/780353062 <== osd.10 v2:172.21.15.137:6824/3486552683 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0e4014ec0 con 0x7fb0d40c03e0 2024-09-21T06:13:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.705+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/780353062 --> [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb0ec06b350 con 0x7fb0d40c03e0 2024-09-21T06:13:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.706+0000 7fb0e8ff9640 1 -- 172.21.15.3:0/780353062 <== osd.10 v2:172.21.15.137:6824/3486552683 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb0ec06b350 con 0x7fb0d40c03e0 2024-09-21T06:13:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.706+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/780353062 >> [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683] conn(0x7fb0d40c03e0 msgr2=0x7fb0d40c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.706+0000 7fb0f1dc7640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683] conn(0x7fb0d40c03e0 0x7fb0d40c28d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.706+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/780353062 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb0d40b65f0 msgr2=0x7fb0d40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.706+0000 7fb0f1dc7640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb0d40b65f0 0x7fb0d40b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec06c920 tx=0x7fb0dc00b390 comp rx=0 tx=0).stop 2024-09-21T06:13:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.706+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/780353062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0ec10de20 msgr2=0x7fb0ec19fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.706+0000 7fb0f1dc7640 1 --2- 172.21.15.3:0/780353062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0ec10de20 0x7fb0ec19fcf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400eaa0 tx=0x7fb0e400ef60 comp rx=0 tx=0).stop 2024-09-21T06:13:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.706+0000 7fb0ebfff640 1 -- 172.21.15.3:0/780353062 reap_dead start 2024-09-21T06:13:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.707+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/780353062 shutdown_connections 2024-09-21T06:13:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.707+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/780353062 >> 172.21.15.3:0/780353062 conn(0x7fb0ec079750 msgr2=0x7fb0ec1079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.707+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/780353062 shutdown_connections 2024-09-21T06:13:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:51.707+0000 7fb0f1dc7640 1 -- 172.21.15.3:0/780353062 wait complete. 2024-09-21T06:13:53.081 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:52 smithi003 ceph-mon[31227]: pgmap v291: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-21T06:13:53.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:52 smithi137 ceph-mon[34464]: pgmap v291: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-21T06:13:53.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:52 smithi204 ceph-mon[34659]: pgmap v291: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-21T06:13:53.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.172+0000 7f2e29779640 1 -- 172.21.15.3:0/279682113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e241083c0 msgr2=0x7f2e241087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.172+0000 7f2e29779640 1 --2- 172.21.15.3:0/279682113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e241083c0 0x7f2e241087d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009900 tx=0x7f2e1802a410 comp rx=0 tx=0).stop 2024-09-21T06:13:53.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.172+0000 7f2e29779640 1 -- 172.21.15.3:0/279682113 shutdown_connections 2024-09-21T06:13:53.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.172+0000 7f2e29779640 1 --2- 172.21.15.3:0/279682113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24108d10 0x7f2e2410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.172+0000 7f2e29779640 1 --2- 172.21.15.3:0/279682113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e241083c0 0x7f2e241087d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.172+0000 7f2e29779640 1 --2- 172.21.15.3:0/279682113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e24102a80 0x7f2e24102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.172+0000 7f2e29779640 1 -- 172.21.15.3:0/279682113 >> 172.21.15.3:0/279682113 conn(0x7f2e24079740 msgr2=0x7f2e24079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:53.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.172+0000 7f2e29779640 1 -- 172.21.15.3:0/279682113 shutdown_connections 2024-09-21T06:13:53.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.172+0000 7f2e29779640 1 -- 172.21.15.3:0/279682113 wait complete. 2024-09-21T06:13:53.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.172+0000 7f2e29779640 1 Processor -- start 2024-09-21T06:13:53.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.173+0000 7f2e29779640 1 -- start start 2024-09-21T06:13:53.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.173+0000 7f2e29779640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24102a80 0x7f2e2419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.173+0000 7f2e29779640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e241083c0 0x7f2e2419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.173+0000 7f2e29779640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e24108d10 0x7f2e2419be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.173+0000 7f2e29779640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e24112980 con 0x7f2e24108d10 2024-09-21T06:13:53.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.173+0000 7f2e29779640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e24112800 con 0x7f2e24102a80 2024-09-21T06:13:53.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.173+0000 7f2e29779640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e24112b00 con 0x7f2e241083c0 2024-09-21T06:13:53.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.173+0000 7f2e237fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e24108d10 0x7f2e2419be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.173+0000 7f2e237fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e24108d10 0x7f2e2419be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53906/0 (socket says 172.21.15.3:53906) 2024-09-21T06:13:53.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.173+0000 7f2e237fe640 1 -- 172.21.15.3:0/1131891786 learned_addr learned my addr 172.21.15.3:0/1131891786 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:53.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.173+0000 7f2e22ffd640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24102a80 0x7f2e2419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.173+0000 7f2e227fc640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e241083c0 0x7f2e2419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.174+0000 7f2e237fe640 1 -- 172.21.15.3:0/1131891786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e241083c0 msgr2=0x7f2e2419b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.174+0000 7f2e237fe640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e241083c0 0x7f2e2419b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.174+0000 7f2e237fe640 1 -- 172.21.15.3:0/1131891786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24102a80 msgr2=0x7f2e2419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.174+0000 7f2e237fe640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24102a80 0x7f2e2419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.174+0000 7f2e237fe640 1 -- 172.21.15.3:0/1131891786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e241a66e0 con 0x7f2e24108d10 2024-09-21T06:13:53.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.174+0000 7f2e237fe640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e24108d10 0x7f2e2419be50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00d750 tx=0x7f2e1c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:53.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.174+0000 7f2e0bfff640 1 -- 172.21.15.3:0/1131891786 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c017070 con 0x7f2e24108d10 2024-09-21T06:13:53.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.174+0000 7f2e29779640 1 -- 172.21.15.3:0/1131891786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e241a69d0 con 0x7f2e24108d10 2024-09-21T06:13:53.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.174+0000 7f2e0bfff640 1 -- 172.21.15.3:0/1131891786 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1c0054d0 con 0x7f2e24108d10 2024-09-21T06:13:53.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.174+0000 7f2e0bfff640 1 -- 172.21.15.3:0/1131891786 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c004c60 con 0x7f2e24108d10 2024-09-21T06:13:53.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.174+0000 7f2e29779640 1 -- 172.21.15.3:0/1131891786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e241a6ee0 con 0x7f2e24108d10 2024-09-21T06:13:53.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.175+0000 7f2e29779640 1 -- 172.21.15.3:0/1131891786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2e24063ad0 con 0x7f2e24108d10 2024-09-21T06:13:53.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.177+0000 7f2e0bfff640 1 -- 172.21.15.3:0/1131891786 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f2e1c009bd0 con 0x7f2e24108d10 2024-09-21T06:13:53.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.177+0000 7f2e0bfff640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e0c0b65f0 0x7f2e0c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.177+0000 7f2e22ffd640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e0c0b65f0 0x7f2e0c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.178+0000 7f2e0bfff640 1 -- 172.21.15.3:0/1131891786 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2e1c0e7380 con 0x7f2e24108d10 2024-09-21T06:13:53.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.178+0000 7f2e0bfff640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635] conn(0x7f2e0c0c03e0 0x7f2e0c0c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.178+0000 7f2e0bfff640 1 -- 172.21.15.3:0/1131891786 --> [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e1c021e50 con 0x7f2e0c0c03e0 2024-09-21T06:13:53.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.178+0000 7f2e0bfff640 1 -- 172.21.15.3:0/1131891786 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f2e1c0e77d0 con 0x7f2e24108d10 2024-09-21T06:13:53.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.178+0000 7f2e227fc640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635] conn(0x7f2e0c0c03e0 0x7f2e0c0c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.178+0000 7f2e22ffd640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e0c0b65f0 0x7f2e0c0b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e2406c700 tx=0x7f2e14009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:53.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.178+0000 7f2e227fc640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635] conn(0x7f2e0c0c03e0 0x7f2e0c0c28d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:53.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.179+0000 7f2e0bfff640 1 -- 172.21.15.3:0/1131891786 <== osd.3 v2:172.21.15.3:6810/4293741635 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e1c021e50 con 0x7f2e0c0c03e0 2024-09-21T06:13:53.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.193+0000 7f2e29779640 1 -- 172.21.15.3:0/1131891786 --> [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2e24063f00 con 0x7f2e0c0c03e0 2024-09-21T06:13:53.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.194+0000 7f2e0bfff640 1 -- 172.21.15.3:0/1131891786 <== osd.3 v2:172.21.15.3:6810/4293741635 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2e24063f00 con 0x7f2e0c0c03e0 2024-09-21T06:13:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.194+0000 7f2e29779640 1 -- 172.21.15.3:0/1131891786 >> [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635] conn(0x7f2e0c0c03e0 msgr2=0x7f2e0c0c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.194+0000 7f2e29779640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635] conn(0x7f2e0c0c03e0 0x7f2e0c0c28d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.194+0000 7f2e29779640 1 -- 172.21.15.3:0/1131891786 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e0c0b65f0 msgr2=0x7f2e0c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.194+0000 7f2e29779640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e0c0b65f0 0x7f2e0c0b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e2406c700 tx=0x7f2e14009500 comp rx=0 tx=0).stop 2024-09-21T06:13:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.194+0000 7f2e29779640 1 -- 172.21.15.3:0/1131891786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e24108d10 msgr2=0x7f2e2419be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.194+0000 7f2e29779640 1 --2- 172.21.15.3:0/1131891786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e24108d10 0x7f2e2419be50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00d750 tx=0x7f2e1c00dc10 comp rx=0 tx=0).stop 2024-09-21T06:13:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.194+0000 7f2e237fe640 1 -- 172.21.15.3:0/1131891786 reap_dead start 2024-09-21T06:13:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.194+0000 7f2e29779640 1 -- 172.21.15.3:0/1131891786 shutdown_connections 2024-09-21T06:13:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.194+0000 7f2e29779640 1 -- 172.21.15.3:0/1131891786 >> 172.21.15.3:0/1131891786 conn(0x7f2e24079740 msgr2=0x7f2e24104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.195+0000 7f2e29779640 1 -- 172.21.15.3:0/1131891786 shutdown_connections 2024-09-21T06:13:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.195+0000 7f2e29779640 1 -- 172.21.15.3:0/1131891786 wait complete. 2024-09-21T06:13:53.673 INFO:teuthology.orchestra.run.smithi003.stdout:279172874267 2024-09-21T06:13:53.673 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T06:13:53.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.845+0000 7f1fc0d11640 1 -- 172.21.15.3:0/1611904981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fbc10de50 msgr2=0x7f1fbc110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.845+0000 7f1fc0d11640 1 --2- 172.21.15.3:0/1611904981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fbc10de50 0x7f1fbc110320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009ca0 tx=0x7f1fb402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.845+0000 7f1fc0d11640 1 -- 172.21.15.3:0/1611904981 shutdown_connections 2024-09-21T06:13:53.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.845+0000 7f1fc0d11640 1 --2- 172.21.15.3:0/1611904981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fbc1052d0 0x7f1fbc10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.845+0000 7f1fc0d11640 1 --2- 172.21.15.3:0/1611904981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fbc10de50 0x7f1fbc110320 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.845+0000 7f1fc0d11640 1 --2- 172.21.15.3:0/1611904981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fbc1048a0 0x7f1fbc104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.845+0000 7f1fc0d11640 1 -- 172.21.15.3:0/1611904981 >> 172.21.15.3:0/1611904981 conn(0x7f1fbc079740 msgr2=0x7f1fbc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:53.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.845+0000 7f1fc0d11640 1 -- 172.21.15.3:0/1611904981 shutdown_connections 2024-09-21T06:13:53.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.845+0000 7f1fc0d11640 1 -- 172.21.15.3:0/1611904981 wait complete. 2024-09-21T06:13:53.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.846+0000 7f1fc0d11640 1 Processor -- start 2024-09-21T06:13:53.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.846+0000 7f1fc0d11640 1 -- start start 2024-09-21T06:13:53.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.846+0000 7f1fc0d11640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fbc1048a0 0x7f1fbc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.846+0000 7f1fc0d11640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fbc1052d0 0x7f1fbc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.846+0000 7f1fc0d11640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fbc10de50 0x7f1fbc19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.846+0000 7f1fc0d11640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc113a90 con 0x7f1fbc1052d0 2024-09-21T06:13:53.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.846+0000 7f1fc0d11640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc113910 con 0x7f1fbc10de50 2024-09-21T06:13:53.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.846+0000 7f1fc0d11640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc113c10 con 0x7f1fbc1048a0 2024-09-21T06:13:53.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.846+0000 7f1fb9d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fbc1052d0 0x7f1fbc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.846+0000 7f1fb9d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fbc1052d0 0x7f1fbc19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53924/0 (socket says 172.21.15.3:53924) 2024-09-21T06:13:53.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.846+0000 7f1fb9d9b640 1 -- 172.21.15.3:0/2586913290 learned_addr learned my addr 172.21.15.3:0/2586913290 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:53.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fba59c640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fbc1048a0 0x7f1fbc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fbad9d640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fbc10de50 0x7f1fbc19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fb9d9b640 1 -- 172.21.15.3:0/2586913290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fbc1048a0 msgr2=0x7f1fbc19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fb9d9b640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fbc1048a0 0x7f1fbc19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fb9d9b640 1 -- 172.21.15.3:0/2586913290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fbc10de50 msgr2=0x7f1fbc19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fb9d9b640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fbc10de50 0x7f1fbc19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fb9d9b640 1 -- 172.21.15.3:0/2586913290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc1a03b0 con 0x7f1fbc1052d0 2024-09-21T06:13:53.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fba59c640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fbc1048a0 0x7f1fbc19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:53.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fb9d9b640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fbc1052d0 0x7f1fbc19a9e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000e9c0 tx=0x7f1fb000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:53.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fab7fe640 1 -- 172.21.15.3:0/2586913290 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb000cd80 con 0x7f1fbc1052d0 2024-09-21T06:13:53.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fab7fe640 1 -- 172.21.15.3:0/2586913290 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb00054d0 con 0x7f1fbc1052d0 2024-09-21T06:13:53.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fc0d11640 1 -- 172.21.15.3:0/2586913290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fbc1a06a0 con 0x7f1fbc1052d0 2024-09-21T06:13:53.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fab7fe640 1 -- 172.21.15.3:0/2586913290 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0010430 con 0x7f1fbc1052d0 2024-09-21T06:13:53.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.847+0000 7f1fc0d11640 1 -- 172.21.15.3:0/2586913290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fbc1a6e00 con 0x7f1fbc1052d0 2024-09-21T06:13:53.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.848+0000 7f1fc0d11640 1 -- 172.21.15.3:0/2586913290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1fbc108400 con 0x7f1fbc1052d0 2024-09-21T06:13:53.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.850+0000 7f1fab7fe640 1 -- 172.21.15.3:0/2586913290 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f1fb0004f90 con 0x7f1fbc1052d0 2024-09-21T06:13:53.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.851+0000 7f1fab7fe640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1fa40b65f0 0x7f1fa40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.851+0000 7f1fab7fe640 1 -- 172.21.15.3:0/2586913290 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f1fb00e81d0 con 0x7f1fbc1052d0 2024-09-21T06:13:53.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.851+0000 7f1fba59c640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1fa40b65f0 0x7f1fa40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.851+0000 7f1fab7fe640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610] conn(0x7f1fa40c03e0 0x7f1fa40c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.852+0000 7f1fab7fe640 1 -- 172.21.15.3:0/2586913290 --> [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1fb0015e80 con 0x7f1fa40c03e0 2024-09-21T06:13:53.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.852+0000 7f1fab7fe640 1 -- 172.21.15.3:0/2586913290 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f1fb00e8580 con 0x7f1fbc1052d0 2024-09-21T06:13:53.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.852+0000 7f1fbad9d640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610] conn(0x7f1fa40c03e0 0x7f1fa40c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.852+0000 7f1fba59c640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1fa40b65f0 0x7f1fa40b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc06c580 tx=0x7f1fac005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:53.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.852+0000 7f1fbad9d640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610] conn(0x7f1fa40c03e0 0x7f1fa40c28d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:53.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.853+0000 7f1fab7fe640 1 -- 172.21.15.3:0/2586913290 <== osd.11 v2:172.21.15.204:6824/1084308610 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1fb0015e80 con 0x7f1fa40c03e0 2024-09-21T06:13:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.879+0000 7f1fc0d11640 1 -- 172.21.15.3:0/2586913290 --> [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1fbc06be80 con 0x7f1fa40c03e0 2024-09-21T06:13:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.879+0000 7f1fab7fe640 1 -- 172.21.15.3:0/2586913290 <== osd.11 v2:172.21.15.204:6824/1084308610 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1fbc06be80 con 0x7f1fa40c03e0 2024-09-21T06:13:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.880+0000 7f1fc0d11640 1 -- 172.21.15.3:0/2586913290 >> [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610] conn(0x7f1fa40c03e0 msgr2=0x7f1fa40c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.880+0000 7f1fc0d11640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610] conn(0x7f1fa40c03e0 0x7f1fa40c28d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.880+0000 7f1fc0d11640 1 -- 172.21.15.3:0/2586913290 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1fa40b65f0 msgr2=0x7f1fa40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.880+0000 7f1fc0d11640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1fa40b65f0 0x7f1fa40b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc06c580 tx=0x7f1fac005f90 comp rx=0 tx=0).stop 2024-09-21T06:13:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.880+0000 7f1fc0d11640 1 -- 172.21.15.3:0/2586913290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fbc1052d0 msgr2=0x7f1fbc19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.880+0000 7f1fc0d11640 1 --2- 172.21.15.3:0/2586913290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fbc1052d0 0x7f1fbc19a9e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000e9c0 tx=0x7f1fb000ee80 comp rx=0 tx=0).stop 2024-09-21T06:13:53.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.880+0000 7f1fbad9d640 1 -- 172.21.15.3:0/2586913290 reap_dead start 2024-09-21T06:13:53.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.880+0000 7f1fc0d11640 1 -- 172.21.15.3:0/2586913290 shutdown_connections 2024-09-21T06:13:53.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.880+0000 7f1fc0d11640 1 -- 172.21.15.3:0/2586913290 >> 172.21.15.3:0/2586913290 conn(0x7f1fbc079740 msgr2=0x7f1fbc1079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:53.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.880+0000 7f1fc0d11640 1 -- 172.21.15.3:0/2586913290 shutdown_connections 2024-09-21T06:13:53.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:53.880+0000 7f1fc0d11640 1 -- 172.21.15.3:0/2586913290 wait complete. 2024-09-21T06:13:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.370+0000 7f0ca5caa640 1 -- 172.21.15.3:0/88466800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca0109e40 msgr2=0x7f0ca0110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.370+0000 7f0ca5caa640 1 --2- 172.21.15.3:0/88466800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca0109e40 0x7f0ca0110670 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009ca0 tx=0x7f0c9802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.370+0000 7f0ca5caa640 1 -- 172.21.15.3:0/88466800 shutdown_connections 2024-09-21T06:13:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.370+0000 7f0ca5caa640 1 --2- 172.21.15.3:0/88466800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ca01094f0 0x7f0ca0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.370+0000 7f0ca5caa640 1 --2- 172.21.15.3:0/88466800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca0109e40 0x7f0ca0110670 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.370+0000 7f0ca5caa640 1 --2- 172.21.15.3:0/88466800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ca0103bb0 0x7f0ca0103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.370+0000 7f0ca5caa640 1 -- 172.21.15.3:0/88466800 >> 172.21.15.3:0/88466800 conn(0x7f0ca00ff600 msgr2=0x7f0ca0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.370+0000 7f0ca5caa640 1 -- 172.21.15.3:0/88466800 shutdown_connections 2024-09-21T06:13:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.370+0000 7f0ca5caa640 1 -- 172.21.15.3:0/88466800 wait complete. 2024-09-21T06:13:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.371+0000 7f0ca5caa640 1 Processor -- start 2024-09-21T06:13:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.371+0000 7f0ca5caa640 1 -- start start 2024-09-21T06:13:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.371+0000 7f0ca5caa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ca0103bb0 0x7f0ca019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.371+0000 7f0ca5caa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca01094f0 0x7f0ca019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.371+0000 7f0ca5caa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ca0109e40 0x7f0ca019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.371+0000 7f0ca5caa640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca0113b00 con 0x7f0ca01094f0 2024-09-21T06:13:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.371+0000 7f0ca5caa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca0113980 con 0x7f0ca0109e40 2024-09-21T06:13:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.371+0000 7f0ca5caa640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca0113c80 con 0x7f0ca0103bb0 2024-09-21T06:13:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.371+0000 7f0c9effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca01094f0 0x7f0ca019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.371+0000 7f0c9effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca01094f0 0x7f0ca019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53886/0 (socket says 172.21.15.3:53886) 2024-09-21T06:13:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.371+0000 7f0c9effd640 1 -- 172.21.15.3:0/2495286341 learned_addr learned my addr 172.21.15.3:0/2495286341 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0c9ffff640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ca0109e40 0x7f0ca019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0c9f7fe640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ca0103bb0 0x7f0ca019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0c9effd640 1 -- 172.21.15.3:0/2495286341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ca0103bb0 msgr2=0x7f0ca019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0c9effd640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ca0103bb0 0x7f0ca019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0c9effd640 1 -- 172.21.15.3:0/2495286341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ca0109e40 msgr2=0x7f0ca019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0c9effd640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ca0109e40 0x7f0ca019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0c9effd640 1 -- 172.21.15.3:0/2495286341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca01a03e0 con 0x7f0ca01094f0 2024-09-21T06:13:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0c9ffff640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ca0109e40 0x7f0ca019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0c9f7fe640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ca0103bb0 0x7f0ca019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0c9effd640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca01094f0 0x7f0ca019aa10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400e9c0 tx=0x7f0c9400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0c9cff9640 1 -- 172.21.15.3:0/2495286341 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9400ccc0 con 0x7f0ca01094f0 2024-09-21T06:13:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0c9cff9640 1 -- 172.21.15.3:0/2495286341 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c940054d0 con 0x7f0ca01094f0 2024-09-21T06:13:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0ca5caa640 1 -- 172.21.15.3:0/2495286341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca01a06d0 con 0x7f0ca01094f0 2024-09-21T06:13:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0c9cff9640 1 -- 172.21.15.3:0/2495286341 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c94010430 con 0x7f0ca01094f0 2024-09-21T06:13:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.372+0000 7f0ca5caa640 1 -- 172.21.15.3:0/2495286341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca01a6e30 con 0x7f0ca01094f0 2024-09-21T06:13:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.373+0000 7f0ca5caa640 1 -- 172.21.15.3:0/2495286341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ca0063b50 con 0x7f0ca01094f0 2024-09-21T06:13:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.375+0000 7f0c9cff9640 1 -- 172.21.15.3:0/2495286341 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f0c94005000 con 0x7f0ca01094f0 2024-09-21T06:13:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.376+0000 7f0c9cff9640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0c880b6670 0x7f0c880b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.376+0000 7f0c9f7fe640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0c880b6670 0x7f0c880b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.376+0000 7f0c9cff9640 1 -- 172.21.15.3:0/2495286341 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f0c940e7de0 con 0x7f0ca01094f0 2024-09-21T06:13:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.376+0000 7f0c9cff9640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785] conn(0x7f0c880c0460 0x7f0c880c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.376+0000 7f0c9cff9640 1 -- 172.21.15.3:0/2495286341 --> [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c880c3020 con 0x7f0c880c0460 2024-09-21T06:13:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.376+0000 7f0c9cff9640 1 -- 172.21.15.3:0/2495286341 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f0c940e8230 con 0x7f0ca01094f0 2024-09-21T06:13:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.376+0000 7f0c9f7fe640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0c880b6670 0x7f0c880b8b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009b40 tx=0x7f0c90005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.376+0000 7f0c9ffff640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785] conn(0x7f0c880c0460 0x7f0c880c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.377+0000 7f0c9ffff640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785] conn(0x7f0c880c0460 0x7f0c880c2950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.378+0000 7f0c9cff9640 1 -- 172.21.15.3:0/2495286341 <== osd.4 v2:172.21.15.137:6808/2608988785 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c880c3020 con 0x7f0c880c0460 2024-09-21T06:13:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.392+0000 7f0ca5caa640 1 -- 172.21.15.3:0/2495286341 --> [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ca0063fd0 con 0x7f0c880c0460 2024-09-21T06:13:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.393+0000 7f0c9cff9640 1 -- 172.21.15.3:0/2495286341 <== osd.4 v2:172.21.15.137:6808/2608988785 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0ca0063fd0 con 0x7f0c880c0460 2024-09-21T06:13:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.393+0000 7f0ca5caa640 1 -- 172.21.15.3:0/2495286341 >> [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785] conn(0x7f0c880c0460 msgr2=0x7f0c880c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.393+0000 7f0ca5caa640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785] conn(0x7f0c880c0460 0x7f0c880c2950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.393+0000 7f0ca5caa640 1 -- 172.21.15.3:0/2495286341 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0c880b6670 msgr2=0x7f0c880b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.393+0000 7f0ca5caa640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0c880b6670 0x7f0c880b8b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009b40 tx=0x7f0c90005e50 comp rx=0 tx=0).stop 2024-09-21T06:13:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.393+0000 7f0ca5caa640 1 -- 172.21.15.3:0/2495286341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca01094f0 msgr2=0x7f0ca019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.393+0000 7f0ca5caa640 1 --2- 172.21.15.3:0/2495286341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca01094f0 0x7f0ca019aa10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400e9c0 tx=0x7f0c9400ee80 comp rx=0 tx=0).stop 2024-09-21T06:13:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.393+0000 7f0c9ffff640 1 -- 172.21.15.3:0/2495286341 reap_dead start 2024-09-21T06:13:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.393+0000 7f0ca5caa640 1 -- 172.21.15.3:0/2495286341 shutdown_connections 2024-09-21T06:13:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.393+0000 7f0ca5caa640 1 -- 172.21.15.3:0/2495286341 >> 172.21.15.3:0/2495286341 conn(0x7f0ca00ff600 msgr2=0x7f0ca0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.394+0000 7f0ca5caa640 1 -- 172.21.15.3:0/2495286341 shutdown_connections 2024-09-21T06:13:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:52.394+0000 7f0ca5caa640 1 -- 172.21.15.3:0/2495286341 wait complete. 2024-09-21T06:13:54.904 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:54 smithi003 ceph-mon[31227]: pgmap v292: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T06:13:55.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:54 smithi137 ceph-mon[34464]: pgmap v292: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T06:13:55.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:54 smithi204 ceph-mon[34659]: pgmap v292: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T06:13:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.208+0000 7f8fa525b640 1 -- 172.21.15.3:0/3906987771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fa0102a60 msgr2=0x7f8fa0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.208+0000 7f8fa525b640 1 --2- 172.21.15.3:0/3906987771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fa0102a60 0x7f8fa0102e70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009900 tx=0x7f8f9002a410 comp rx=0 tx=0).stop 2024-09-21T06:13:55.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.208+0000 7f8fa525b640 1 -- 172.21.15.3:0/3906987771 shutdown_connections 2024-09-21T06:13:55.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.208+0000 7f8fa525b640 1 --2- 172.21.15.3:0/3906987771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fa01083a0 0x7f8fa01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.208+0000 7f8fa525b640 1 --2- 172.21.15.3:0/3906987771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fa0108cf0 0x7f8fa010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.208+0000 7f8fa525b640 1 --2- 172.21.15.3:0/3906987771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fa0102a60 0x7f8fa0102e70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.208+0000 7f8fa525b640 1 -- 172.21.15.3:0/3906987771 >> 172.21.15.3:0/3906987771 conn(0x7f8fa0079730 msgr2=0x7f8fa0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:55.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.208+0000 7f8fa525b640 1 -- 172.21.15.3:0/3906987771 shutdown_connections 2024-09-21T06:13:55.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.208+0000 7f8fa525b640 1 -- 172.21.15.3:0/3906987771 wait complete. 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.209+0000 7f8fa525b640 1 Processor -- start 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.209+0000 7f8fa525b640 1 -- start start 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8fa525b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fa0102a60 0x7f8fa019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8fa525b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fa01083a0 0x7f8fa019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8fa525b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fa0108cf0 0x7f8fa019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8fa525b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0112960 con 0x7f8fa01083a0 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8fa525b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa01127e0 con 0x7f8fa0108cf0 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8fa525b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0112ae0 con 0x7f8fa0102a60 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8f9e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fa01083a0 0x7f8fa019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8f9e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fa01083a0 0x7f8fa019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53948/0 (socket says 172.21.15.3:53948) 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8f9e59c640 1 -- 172.21.15.3:0/2591693127 learned_addr learned my addr 172.21.15.3:0/2591693127 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8f9f59e640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fa0108cf0 0x7f8fa019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8f9ed9d640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fa0102a60 0x7f8fa019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8f9e59c640 1 -- 172.21.15.3:0/2591693127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fa0102a60 msgr2=0x7f8fa019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8f9e59c640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fa0102a60 0x7f8fa019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8f9e59c640 1 -- 172.21.15.3:0/2591693127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fa0108cf0 msgr2=0x7f8fa019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8f9e59c640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fa0108cf0 0x7f8fa019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8f9e59c640 1 -- 172.21.15.3:0/2591693127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa01a03d0 con 0x7f8fa01083a0 2024-09-21T06:13:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8f9ed9d640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fa0102a60 0x7f8fa019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8f9e59c640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fa01083a0 0x7f8fa019aa00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400e9f0 tx=0x7f8f9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.210+0000 7f8f87fff640 1 -- 172.21.15.3:0/2591693127 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400ccc0 con 0x7f8fa01083a0 2024-09-21T06:13:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.211+0000 7f8f87fff640 1 -- 172.21.15.3:0/2591693127 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f940054d0 con 0x7f8fa01083a0 2024-09-21T06:13:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.211+0000 7f8fa525b640 1 -- 172.21.15.3:0/2591693127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa01a06c0 con 0x7f8fa01083a0 2024-09-21T06:13:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.211+0000 7f8f87fff640 1 -- 172.21.15.3:0/2591693127 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94010430 con 0x7f8fa01083a0 2024-09-21T06:13:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.211+0000 7f8fa525b640 1 -- 172.21.15.3:0/2591693127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa01a6e20 con 0x7f8fa01083a0 2024-09-21T06:13:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.211+0000 7f8fa525b640 1 -- 172.21.15.3:0/2591693127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8fa006b170 con 0x7f8fa01083a0 2024-09-21T06:13:55.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.214+0000 7f8f87fff640 1 -- 172.21.15.3:0/2591693127 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f8f940046c0 con 0x7f8fa01083a0 2024-09-21T06:13:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.214+0000 7f8f87fff640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8f880b65f0 0x7f8f880b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.214+0000 7f8f9ed9d640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8f880b65f0 0x7f8f880b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.214+0000 7f8f87fff640 1 -- 172.21.15.3:0/2591693127 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8f940e8020 con 0x7f8fa01083a0 2024-09-21T06:13:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.215+0000 7f8f87fff640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549] conn(0x7f8f880c03e0 0x7f8f880c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.215+0000 7f8f87fff640 1 -- 172.21.15.3:0/2591693127 --> [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f880c2fa0 con 0x7f8f880c03e0 2024-09-21T06:13:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.215+0000 7f8f87fff640 1 -- 172.21.15.3:0/2591693127 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f8f940e8470 con 0x7f8fa01083a0 2024-09-21T06:13:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.215+0000 7f8f9f59e640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549] conn(0x7f8f880c03e0 0x7f8f880c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.215+0000 7f8f9ed9d640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8f880b65f0 0x7f8f880b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8f90031440 tx=0x7f8f90009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.215+0000 7f8f9f59e640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549] conn(0x7f8f880c03e0 0x7f8f880c28d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.216+0000 7f8f87fff640 1 -- 172.21.15.3:0/2591693127 <== osd.9 v2:172.21.15.3:6826/1238491549 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f880c2fa0 con 0x7f8f880c03e0 2024-09-21T06:13:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.231+0000 7f8fa525b640 1 -- 172.21.15.3:0/2591693127 --> [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8fa0103ce0 con 0x7f8f880c03e0 2024-09-21T06:13:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.232+0000 7f8f87fff640 1 -- 172.21.15.3:0/2591693127 <== osd.9 v2:172.21.15.3:6826/1238491549 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8fa0103ce0 con 0x7f8f880c03e0 2024-09-21T06:13:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.232+0000 7f8fa525b640 1 -- 172.21.15.3:0/2591693127 >> [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549] conn(0x7f8f880c03e0 msgr2=0x7f8f880c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.232+0000 7f8fa525b640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549] conn(0x7f8f880c03e0 0x7f8f880c28d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.232+0000 7f8fa525b640 1 -- 172.21.15.3:0/2591693127 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8f880b65f0 msgr2=0x7f8f880b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.232+0000 7f8fa525b640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8f880b65f0 0x7f8f880b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8f90031440 tx=0x7f8f90009680 comp rx=0 tx=0).stop 2024-09-21T06:13:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.232+0000 7f8fa525b640 1 -- 172.21.15.3:0/2591693127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fa01083a0 msgr2=0x7f8fa019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.232+0000 7f8fa525b640 1 --2- 172.21.15.3:0/2591693127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fa01083a0 0x7f8fa019aa00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400e9f0 tx=0x7f8f9400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.232+0000 7f8f9f59e640 1 -- 172.21.15.3:0/2591693127 reap_dead start 2024-09-21T06:13:55.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.232+0000 7f8fa525b640 1 -- 172.21.15.3:0/2591693127 shutdown_connections 2024-09-21T06:13:55.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.232+0000 7f8fa525b640 1 -- 172.21.15.3:0/2591693127 >> 172.21.15.3:0/2591693127 conn(0x7f8fa0079730 msgr2=0x7f8fa0101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:55.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.233+0000 7f8fa525b640 1 -- 172.21.15.3:0/2591693127 shutdown_connections 2024-09-21T06:13:55.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:55.233+0000 7f8fa525b640 1 -- 172.21.15.3:0/2591693127 wait complete. 2024-09-21T06:13:55.240 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:55.945 INFO:teuthology.orchestra.run.smithi003.stdout:111669149759 2024-09-21T06:13:55.945 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T06:13:56.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.131+0000 7f99634a4640 1 -- 172.21.15.3:0/3269348068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c105220 msgr2=0x7f995c105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.131+0000 7f99634a4640 1 --2- 172.21.15.3:0/3269348068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c105220 0x7f995c105630 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f994c009900 tx=0x7f994c02a410 comp rx=0 tx=0).stop 2024-09-21T06:13:56.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.131+0000 7f99634a4640 1 -- 172.21.15.3:0/3269348068 shutdown_connections 2024-09-21T06:13:56.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.131+0000 7f99634a4640 1 --2- 172.21.15.3:0/3269348068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f995c105b70 0x7f995c10f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.131+0000 7f99634a4640 1 --2- 172.21.15.3:0/3269348068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c105220 0x7f995c105630 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.131+0000 7f99634a4640 1 --2- 172.21.15.3:0/3269348068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f995c103210 0x7f995c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.131+0000 7f99634a4640 1 -- 172.21.15.3:0/3269348068 >> 172.21.15.3:0/3269348068 conn(0x7f995c079ea0 msgr2=0x7f995c07a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.131+0000 7f99634a4640 1 -- 172.21.15.3:0/3269348068 shutdown_connections 2024-09-21T06:13:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.131+0000 7f99634a4640 1 -- 172.21.15.3:0/3269348068 wait complete. 2024-09-21T06:13:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f99634a4640 1 Processor -- start 2024-09-21T06:13:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f99634a4640 1 -- start start 2024-09-21T06:13:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f99634a4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c103210 0x7f995c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f99634a4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f995c105220 0x7f995c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f99634a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f995c105b70 0x7f995c19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f99634a4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f995c112a40 con 0x7f995c103210 2024-09-21T06:13:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f99634a4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1128c0 con 0x7f995c105b70 2024-09-21T06:13:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f99634a4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f995c112bc0 con 0x7f995c105220 2024-09-21T06:13:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f9961240640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c103210 0x7f995c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f9961240640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c103210 0x7f995c19b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36352/0 (socket says 172.21.15.3:36352) 2024-09-21T06:13:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f9961240640 1 -- 172.21.15.3:0/3721151472 learned_addr learned my addr 172.21.15.3:0/3721151472 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f9960a3f640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f995c105220 0x7f995c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f9961a41640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f995c105b70 0x7f995c19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f9961240640 1 -- 172.21.15.3:0/3721151472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f995c105220 msgr2=0x7f995c19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f9961240640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f995c105220 0x7f995c19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f9961240640 1 -- 172.21.15.3:0/3721151472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f995c105b70 msgr2=0x7f995c19bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f9961240640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f995c105b70 0x7f995c19bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.132+0000 7f9961240640 1 -- 172.21.15.3:0/3721151472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c1a66e0 con 0x7f995c103210 2024-09-21T06:13:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.133+0000 7f9961240640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c103210 0x7f995c19b450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f995800d770 tx=0x7f995800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.133+0000 7f99527fc640 1 -- 172.21.15.3:0/3721151472 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958017070 con 0x7f995c103210 2024-09-21T06:13:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.133+0000 7f99527fc640 1 -- 172.21.15.3:0/3721151472 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9958009c30 con 0x7f995c103210 2024-09-21T06:13:56.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.133+0000 7f99634a4640 1 -- 172.21.15.3:0/3721151472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995c1a69d0 con 0x7f995c103210 2024-09-21T06:13:56.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.133+0000 7f99527fc640 1 -- 172.21.15.3:0/3721151472 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958003cc0 con 0x7f995c103210 2024-09-21T06:13:56.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.133+0000 7f99634a4640 1 -- 172.21.15.3:0/3721151472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f995c1a6ee0 con 0x7f995c103210 2024-09-21T06:13:56.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.134+0000 7f99634a4640 1 -- 172.21.15.3:0/3721151472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f995c063ad0 con 0x7f995c103210 2024-09-21T06:13:56.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.136+0000 7f99527fc640 1 -- 172.21.15.3:0/3721151472 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f9958003e60 con 0x7f995c103210 2024-09-21T06:13:56.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.137+0000 7f99527fc640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f99480b6620 0x7f99480b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.137+0000 7f9960a3f640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f99480b6620 0x7f99480b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.137+0000 7f99527fc640 1 -- 172.21.15.3:0/3721151472 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f99580e74f0 con 0x7f995c103210 2024-09-21T06:13:56.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.137+0000 7f99527fc640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496] conn(0x7f99480c0410 0x7f99480c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.137+0000 7f99527fc640 1 -- 172.21.15.3:0/3721151472 --> [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9958021e50 con 0x7f99480c0410 2024-09-21T06:13:56.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.137+0000 7f99527fc640 1 -- 172.21.15.3:0/3721151472 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f99580e78a0 con 0x7f995c103210 2024-09-21T06:13:56.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.138+0000 7f9960a3f640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f99480b6620 0x7f99480b8b30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f994c005940 tx=0x7f994c0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:56.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.138+0000 7f9961a41640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496] conn(0x7f99480c0410 0x7f99480c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.138+0000 7f9961a41640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496] conn(0x7f99480c0410 0x7f99480c2900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:56.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.139+0000 7f99527fc640 1 -- 172.21.15.3:0/3721151472 <== osd.7 v2:172.21.15.137:6816/2086822496 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9958021e50 con 0x7f99480c0410 2024-09-21T06:13:56.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.155+0000 7f99634a4640 1 -- 172.21.15.3:0/3721151472 --> [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f995c063f00 con 0x7f99480c0410 2024-09-21T06:13:56.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.155+0000 7f99527fc640 1 -- 172.21.15.3:0/3721151472 <== osd.7 v2:172.21.15.137:6816/2086822496 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f995c063f00 con 0x7f99480c0410 2024-09-21T06:13:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.156+0000 7f99634a4640 1 -- 172.21.15.3:0/3721151472 >> [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496] conn(0x7f99480c0410 msgr2=0x7f99480c2900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.156+0000 7f99634a4640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496] conn(0x7f99480c0410 0x7f99480c2900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.156+0000 7f99634a4640 1 -- 172.21.15.3:0/3721151472 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f99480b6620 msgr2=0x7f99480b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.156+0000 7f99634a4640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f99480b6620 0x7f99480b8b30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f994c005940 tx=0x7f994c0058d0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.156+0000 7f99634a4640 1 -- 172.21.15.3:0/3721151472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c103210 msgr2=0x7f995c19b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.156+0000 7f99634a4640 1 --2- 172.21.15.3:0/3721151472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c103210 0x7f995c19b450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f995800d770 tx=0x7f995800dc30 comp rx=0 tx=0).stop 2024-09-21T06:13:56.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.156+0000 7f9961a41640 1 -- 172.21.15.3:0/3721151472 reap_dead start 2024-09-21T06:13:56.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.156+0000 7f99634a4640 1 -- 172.21.15.3:0/3721151472 shutdown_connections 2024-09-21T06:13:56.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.156+0000 7f99634a4640 1 -- 172.21.15.3:0/3721151472 >> 172.21.15.3:0/3721151472 conn(0x7f995c079ea0 msgr2=0x7f995c106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:56.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.156+0000 7f99634a4640 1 -- 172.21.15.3:0/3721151472 shutdown_connections 2024-09-21T06:13:56.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.156+0000 7f99634a4640 1 -- 172.21.15.3:0/3721151472 wait complete. 2024-09-21T06:13:56.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.372+0000 7f592337a640 1 -- 172.21.15.3:0/1864826267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f591c10de10 msgr2=0x7f591c1102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.372+0000 7f592337a640 1 --2- 172.21.15.3:0/1864826267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f591c10de10 0x7f591c1102f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5914009ca0 tx=0x7f591402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.372+0000 7f592337a640 1 -- 172.21.15.3:0/1864826267 shutdown_connections 2024-09-21T06:13:56.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.372+0000 7f592337a640 1 --2- 172.21.15.3:0/1864826267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f591c105290 0x7f591c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.372+0000 7f592337a640 1 --2- 172.21.15.3:0/1864826267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f591c10de10 0x7f591c1102f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.372+0000 7f592337a640 1 --2- 172.21.15.3:0/1864826267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f591c104860 0x7f591c104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.372+0000 7f592337a640 1 -- 172.21.15.3:0/1864826267 >> 172.21.15.3:0/1864826267 conn(0x7f591c079730 msgr2=0x7f591c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:56.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.372+0000 7f592337a640 1 -- 172.21.15.3:0/1864826267 shutdown_connections 2024-09-21T06:13:56.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.372+0000 7f592337a640 1 -- 172.21.15.3:0/1864826267 wait complete. 2024-09-21T06:13:56.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.373+0000 7f592337a640 1 Processor -- start 2024-09-21T06:13:56.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.373+0000 7f592337a640 1 -- start start 2024-09-21T06:13:56.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.373+0000 7f592337a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f591c104860 0x7f591c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.373+0000 7f592337a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f591c105290 0x7f591c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.373+0000 7f592337a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f591c10de10 0x7f591c19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.373+0000 7f592337a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f591c113a60 con 0x7f591c105290 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.373+0000 7f592337a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1138e0 con 0x7f591c104860 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.373+0000 7f592337a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f591c113be0 con 0x7f591c10de10 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.373+0000 7f5920915640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f591c105290 0x7f591c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.373+0000 7f5920915640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f591c105290 0x7f591c19a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36362/0 (socket says 172.21.15.3:36362) 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.373+0000 7f5920915640 1 -- 172.21.15.3:0/5664021 learned_addr learned my addr 172.21.15.3:0/5664021 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.373+0000 7f5921116640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f591c104860 0x7f591c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.374+0000 7f5921917640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f591c10de10 0x7f591c19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.374+0000 7f5920915640 1 -- 172.21.15.3:0/5664021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f591c10de10 msgr2=0x7f591c19fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.374+0000 7f5920915640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f591c10de10 0x7f591c19fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.374+0000 7f5920915640 1 -- 172.21.15.3:0/5664021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f591c104860 msgr2=0x7f591c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.374+0000 7f5920915640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f591c104860 0x7f591c19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.374+0000 7f5920915640 1 -- 172.21.15.3:0/5664021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591c1a02a0 con 0x7f591c105290 2024-09-21T06:13:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.374+0000 7f5920915640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f591c105290 0x7f591c19a990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f590c00c8c0 tx=0x7f590c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.374+0000 7f59127fc640 1 -- 172.21.15.3:0/5664021 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c007bb0 con 0x7f591c105290 2024-09-21T06:13:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.374+0000 7f592337a640 1 -- 172.21.15.3:0/5664021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591c1a0590 con 0x7f591c105290 2024-09-21T06:13:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.374+0000 7f59127fc640 1 -- 172.21.15.3:0/5664021 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f590c007d50 con 0x7f591c105290 2024-09-21T06:13:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.374+0000 7f59127fc640 1 -- 172.21.15.3:0/5664021 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c0187c0 con 0x7f591c105290 2024-09-21T06:13:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.374+0000 7f592337a640 1 -- 172.21.15.3:0/5664021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591c1a6db0 con 0x7f591c105290 2024-09-21T06:13:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.375+0000 7f592337a640 1 -- 172.21.15.3:0/5664021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f591c1011e0 con 0x7f591c105290 2024-09-21T06:13:56.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.377+0000 7f59127fc640 1 -- 172.21.15.3:0/5664021 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f590c00fa20 con 0x7f591c105290 2024-09-21T06:13:56.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.378+0000 7f59127fc640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f59080b6620 0x7f59080b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.378+0000 7f5921116640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f59080b6620 0x7f59080b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.378+0000 7f59127fc640 1 -- 172.21.15.3:0/5664021 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f590c0e7ac0 con 0x7f591c105290 2024-09-21T06:13:56.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.378+0000 7f59127fc640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328] conn(0x7f59080c0410 0x7f59080c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.378+0000 7f59127fc640 1 -- 172.21.15.3:0/5664021 --> [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f590c01ae50 con 0x7f59080c0410 2024-09-21T06:13:56.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.378+0000 7f59127fc640 1 -- 172.21.15.3:0/5664021 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f590c0e7f10 con 0x7f591c105290 2024-09-21T06:13:56.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.378+0000 7f5921917640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328] conn(0x7f59080c0410 0x7f59080c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.378+0000 7f5921116640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f59080b6620 0x7f59080b8b30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f59180058c0 tx=0x7f591800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:56.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.379+0000 7f5921917640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328] conn(0x7f59080c0410 0x7f59080c2900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:56.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.379+0000 7f59127fc640 1 -- 172.21.15.3:0/5664021 <== osd.0 v2:172.21.15.3:6802/615312328 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f590c01ae50 con 0x7f59080c0410 2024-09-21T06:13:56.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.393+0000 7f592337a640 1 -- 172.21.15.3:0/5664021 --> [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f591c06b4e0 con 0x7f59080c0410 2024-09-21T06:13:56.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.393+0000 7f59127fc640 1 -- 172.21.15.3:0/5664021 <== osd.0 v2:172.21.15.3:6802/615312328 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f591c06b4e0 con 0x7f59080c0410 2024-09-21T06:13:56.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.394+0000 7f592337a640 1 -- 172.21.15.3:0/5664021 >> [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328] conn(0x7f59080c0410 msgr2=0x7f59080c2900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.394+0000 7f592337a640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328] conn(0x7f59080c0410 0x7f59080c2900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.394+0000 7f592337a640 1 -- 172.21.15.3:0/5664021 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f59080b6620 msgr2=0x7f59080b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.394+0000 7f592337a640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f59080b6620 0x7f59080b8b30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f59180058c0 tx=0x7f591800a2d0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.394+0000 7f592337a640 1 -- 172.21.15.3:0/5664021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f591c105290 msgr2=0x7f591c19a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.394+0000 7f592337a640 1 --2- 172.21.15.3:0/5664021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f591c105290 0x7f591c19a990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f590c00c8c0 tx=0x7f590c00cd80 comp rx=0 tx=0).stop 2024-09-21T06:13:56.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.394+0000 7f5921917640 1 -- 172.21.15.3:0/5664021 reap_dead start 2024-09-21T06:13:56.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.394+0000 7f592337a640 1 -- 172.21.15.3:0/5664021 shutdown_connections 2024-09-21T06:13:56.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.394+0000 7f592337a640 1 -- 172.21.15.3:0/5664021 >> 172.21.15.3:0/5664021 conn(0x7f591c079730 msgr2=0x7f591c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:56.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.394+0000 7f592337a640 1 -- 172.21.15.3:0/5664021 shutdown_connections 2024-09-21T06:13:56.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.394+0000 7f592337a640 1 -- 172.21.15.3:0/5664021 wait complete. 2024-09-21T06:13:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.799+0000 7ff17721c640 1 -- 172.21.15.3:0/2722946322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff170105300 msgr2=0x7ff17010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.799+0000 7ff17721c640 1 --2- 172.21.15.3:0/2722946322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff170105300 0x7ff17010d850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff160009900 tx=0x7ff16002a410 comp rx=0 tx=0).stop 2024-09-21T06:13:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.800+0000 7ff17721c640 1 -- 172.21.15.3:0/2722946322 shutdown_connections 2024-09-21T06:13:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.800+0000 7ff17721c640 1 --2- 172.21.15.3:0/2722946322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff17010de10 0x7ff170110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.800+0000 7ff17721c640 1 --2- 172.21.15.3:0/2722946322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff170105300 0x7ff17010d850 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.800+0000 7ff17721c640 1 --2- 172.21.15.3:0/2722946322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1701049b0 0x7ff170104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.800+0000 7ff17721c640 1 -- 172.21.15.3:0/2722946322 >> 172.21.15.3:0/2722946322 conn(0x7ff170079730 msgr2=0x7ff170079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.800+0000 7ff17721c640 1 -- 172.21.15.3:0/2722946322 shutdown_connections 2024-09-21T06:13:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.800+0000 7ff17721c640 1 -- 172.21.15.3:0/2722946322 wait complete. 2024-09-21T06:13:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.800+0000 7ff17721c640 1 Processor -- start 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.800+0000 7ff17721c640 1 -- start start 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff17721c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1701049b0 0x7ff17019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff17721c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170105300 0x7ff17019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff17721c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17010de10 0x7ff17019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff17721c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff170113a70 con 0x7ff1701049b0 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff17721c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1701138f0 con 0x7ff17010de10 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff17721c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff170113bf0 con 0x7ff170105300 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff174fb8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1701049b0 0x7ff17019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff174fb8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1701049b0 0x7ff17019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36388/0 (socket says 172.21.15.3:36388) 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff174fb8640 1 -- 172.21.15.3:0/4003331113 learned_addr learned my addr 172.21.15.3:0/4003331113 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff167fff640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170105300 0x7ff17019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff1757b9640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17010de10 0x7ff17019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff174fb8640 1 -- 172.21.15.3:0/4003331113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170105300 msgr2=0x7ff17019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff174fb8640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170105300 0x7ff17019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff174fb8640 1 -- 172.21.15.3:0/4003331113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17010de10 msgr2=0x7ff17019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff174fb8640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17010de10 0x7ff17019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff174fb8640 1 -- 172.21.15.3:0/4003331113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1701a03d0 con 0x7ff1701049b0 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff1757b9640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17010de10 0x7ff17019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff174fb8640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1701049b0 0x7ff17019a460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00d770 tx=0x7ff16c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff165ffb640 1 -- 172.21.15.3:0/4003331113 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c017070 con 0x7ff1701049b0 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff17721c640 1 -- 172.21.15.3:0/4003331113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1701a06c0 con 0x7ff1701049b0 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.801+0000 7ff165ffb640 1 -- 172.21.15.3:0/4003331113 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff16c009c30 con 0x7ff1701049b0 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.802+0000 7ff165ffb640 1 -- 172.21.15.3:0/4003331113 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c003cc0 con 0x7ff1701049b0 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.802+0000 7ff17721c640 1 -- 172.21.15.3:0/4003331113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1701a6e20 con 0x7ff1701049b0 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.803+0000 7ff17721c640 1 -- 172.21.15.3:0/4003331113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff170063ad0 con 0x7ff1701049b0 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.804+0000 7ff165ffb640 1 -- 172.21.15.3:0/4003331113 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7ff16c003e60 con 0x7ff1701049b0 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.805+0000 7ff165ffb640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff1580b65f0 0x7ff1580b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.805+0000 7ff167fff640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff1580b65f0 0x7ff1580b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.805+0000 7ff165ffb640 1 -- 172.21.15.3:0/4003331113 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff16c0e7340 con 0x7ff1701049b0 2024-09-21T06:13:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.805+0000 7ff165ffb640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077] conn(0x7ff1580c03e0 0x7ff1580c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.805+0000 7ff165ffb640 1 -- 172.21.15.3:0/4003331113 --> [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1580c2fa0 con 0x7ff1580c03e0 2024-09-21T06:13:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.805+0000 7ff165ffb640 1 -- 172.21.15.3:0/4003331113 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7ff16c0e7790 con 0x7ff1701049b0 2024-09-21T06:13:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.806+0000 7ff167fff640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff1580b65f0 0x7ff1580b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff160009230 tx=0x7ff160031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.806+0000 7ff1757b9640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077] conn(0x7ff1580c03e0 0x7ff1580c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.806+0000 7ff1757b9640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077] conn(0x7ff1580c03e0 0x7ff1580c28d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.807+0000 7ff165ffb640 1 -- 172.21.15.3:0/4003331113 <== osd.2 v2:172.21.15.204:6800/3392577077 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1580c2fa0 con 0x7ff1580c03e0 2024-09-21T06:13:56.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.821+0000 7ff17721c640 1 -- 172.21.15.3:0/4003331113 --> [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff170063f50 con 0x7ff1580c03e0 2024-09-21T06:13:56.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.822+0000 7ff165ffb640 1 -- 172.21.15.3:0/4003331113 <== osd.2 v2:172.21.15.204:6800/3392577077 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff170063f50 con 0x7ff1580c03e0 2024-09-21T06:13:56.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.822+0000 7ff17721c640 1 -- 172.21.15.3:0/4003331113 >> [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077] conn(0x7ff1580c03e0 msgr2=0x7ff1580c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.822+0000 7ff17721c640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077] conn(0x7ff1580c03e0 0x7ff1580c28d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:56.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.822+0000 7ff17721c640 1 -- 172.21.15.3:0/4003331113 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff1580b65f0 msgr2=0x7ff1580b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.822+0000 7ff17721c640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff1580b65f0 0x7ff1580b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff160009230 tx=0x7ff160031400 comp rx=0 tx=0).stop 2024-09-21T06:13:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.822+0000 7ff17721c640 1 -- 172.21.15.3:0/4003331113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1701049b0 msgr2=0x7ff17019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.822+0000 7ff17721c640 1 --2- 172.21.15.3:0/4003331113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1701049b0 0x7ff17019a460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00d770 tx=0x7ff16c00dc30 comp rx=0 tx=0).stop 2024-09-21T06:13:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.822+0000 7ff1757b9640 1 -- 172.21.15.3:0/4003331113 reap_dead start 2024-09-21T06:13:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.822+0000 7ff17721c640 1 -- 172.21.15.3:0/4003331113 shutdown_connections 2024-09-21T06:13:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.822+0000 7ff17721c640 1 -- 172.21.15.3:0/4003331113 >> 172.21.15.3:0/4003331113 conn(0x7ff170079730 msgr2=0x7ff170100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.822+0000 7ff17721c640 1 -- 172.21.15.3:0/4003331113 shutdown_connections 2024-09-21T06:13:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:13:56.822+0000 7ff17721c640 1 -- 172.21.15.3:0/4003331113 wait complete. 2024-09-21T06:13:57.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:56 smithi003 ceph-mon[31227]: pgmap v293: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T06:13:57.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:56 smithi137 ceph-mon[34464]: pgmap v293: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T06:13:57.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:56 smithi204 ceph-mon[34659]: pgmap v293: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T06:13:57.504 INFO:teuthology.orchestra.run.smithi003.stdout:38654705743 2024-09-21T06:13:57.505 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T06:13:58.152 INFO:teuthology.orchestra.run.smithi003.stdout:206158430251 2024-09-21T06:13:58.152 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T06:13:58.804 INFO:teuthology.orchestra.run.smithi003.stdout:257698037793 2024-09-21T06:13:58.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T06:13:59.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:13:58 smithi003 ceph-mon[31227]: pgmap v294: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T06:13:59.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:13:58 smithi137 ceph-mon[34464]: pgmap v294: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T06:13:59.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:13:58 smithi204 ceph-mon[34659]: pgmap v294: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T06:13:59.597 INFO:teuthology.orchestra.run.smithi003.stdout:81604378694 2024-09-21T06:13:59.597 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T06:14:00.042 INFO:teuthology.orchestra.run.smithi003.stdout:304942678038 2024-09-21T06:14:00.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T06:14:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.271+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3279970171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08108480 msgr2=0x7f8a0810ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.271+0000 7f8a0efe6640 1 --2- 172.21.15.3:0/3279970171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08108480 0x7f8a0810ecb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009ca0 tx=0x7f8a0002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.271+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3279970171 shutdown_connections 2024-09-21T06:14:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.271+0000 7f8a0efe6640 1 --2- 172.21.15.3:0/3279970171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a08107b30 0x7f8a08107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.271+0000 7f8a0efe6640 1 --2- 172.21.15.3:0/3279970171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08108480 0x7f8a0810ecb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.271+0000 7f8a0efe6640 1 --2- 172.21.15.3:0/3279970171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a081021f0 0x7f8a08102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.271+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3279970171 >> 172.21.15.3:0/3279970171 conn(0x7f8a08077c60 msgr2=0x7f8a080780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:00.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.271+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3279970171 shutdown_connections 2024-09-21T06:14:00.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.271+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3279970171 wait complete. 2024-09-21T06:14:00.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.272+0000 7f8a0efe6640 1 Processor -- start 2024-09-21T06:14:00.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.272+0000 7f8a0efe6640 1 -- start start 2024-09-21T06:14:00.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.272+0000 7f8a0efe6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a081021f0 0x7f8a0819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.272+0000 7f8a0efe6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a08107b30 0x7f8a0819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.272+0000 7f8a0efe6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a08108480 0x7f8a081a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.272+0000 7f8a0efe6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08113ad0 con 0x7f8a081021f0 2024-09-21T06:14:00.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.272+0000 7f8a0efe6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08113950 con 0x7f8a08107b30 2024-09-21T06:14:00.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.272+0000 7f8a0efe6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08113c50 con 0x7f8a08108480 2024-09-21T06:14:00.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.272+0000 7f8a0cd82640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a081021f0 0x7f8a0819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.272+0000 7f8a0cd82640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a081021f0 0x7f8a0819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36410/0 (socket says 172.21.15.3:36410) 2024-09-21T06:14:00.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.272+0000 7f8a0cd82640 1 -- 172.21.15.3:0/3921588495 learned_addr learned my addr 172.21.15.3:0/3921588495 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:00.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f89fffff640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a08107b30 0x7f8a0819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f8a0d583640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a08108480 0x7f8a081a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f8a0cd82640 1 -- 172.21.15.3:0/3921588495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a08108480 msgr2=0x7f8a081a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f8a0cd82640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a08108480 0x7f8a081a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f8a0cd82640 1 -- 172.21.15.3:0/3921588495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a08107b30 msgr2=0x7f8a0819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f8a0cd82640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a08107b30 0x7f8a0819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f8a0cd82640 1 -- 172.21.15.3:0/3921588495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a081a25f0 con 0x7f8a081021f0 2024-09-21T06:14:00.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f8a0d583640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a08108480 0x7f8a081a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:00.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f8a0cd82640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a081021f0 0x7f8a0819c680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400ddc0 tx=0x7f8a0400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:00.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f89fdffb640 1 -- 172.21.15.3:0/3921588495 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0401b070 con 0x7f8a081021f0 2024-09-21T06:14:00.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f89fdffb640 1 -- 172.21.15.3:0/3921588495 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a0400be40 con 0x7f8a081021f0 2024-09-21T06:14:00.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f89fdffb640 1 -- 172.21.15.3:0/3921588495 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04016a80 con 0x7f8a081021f0 2024-09-21T06:14:00.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3921588495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a081a28e0 con 0x7f8a081021f0 2024-09-21T06:14:00.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.273+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3921588495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a081a9040 con 0x7f8a081021f0 2024-09-21T06:14:00.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.274+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3921588495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a08106320 con 0x7f8a081021f0 2024-09-21T06:14:00.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.276+0000 7f89fdffb640 1 -- 172.21.15.3:0/3921588495 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f8a0401e070 con 0x7f8a081021f0 2024-09-21T06:14:00.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.277+0000 7f89fdffb640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89f00b6670 0x7f89f00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.278+0000 7f89fffff640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89f00b6670 0x7f89f00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.278+0000 7f89fdffb640 1 -- 172.21.15.3:0/3921588495 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8a040e7860 con 0x7f8a081021f0 2024-09-21T06:14:00.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.278+0000 7f89fdffb640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044] conn(0x7f89f00c0460 0x7f89f00c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:00.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.278+0000 7f89fdffb640 1 -- 172.21.15.3:0/3921588495 --> [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89f00c3020 con 0x7f89f00c0460 2024-09-21T06:14:00.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.278+0000 7f89fdffb640 1 -- 172.21.15.3:0/3921588495 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f8a040e7c10 con 0x7f8a081021f0 2024-09-21T06:14:00.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.278+0000 7f89fffff640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89f00b6670 0x7f89f00b8b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f89f8005fd0 tx=0x7f89f8005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:00.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.278+0000 7f8a0d583640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044] conn(0x7f89f00c0460 0x7f89f00c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:00.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.279+0000 7f8a0d583640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044] conn(0x7f89f00c0460 0x7f89f00c2950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:00.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.280+0000 7f89fdffb640 1 -- 172.21.15.3:0/3921588495 <== osd.5 v2:172.21.15.204:6808/3709074044 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f89f00c3020 con 0x7f89f00c0460 2024-09-21T06:14:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.296+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3921588495 --> [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a08103470 con 0x7f89f00c0460 2024-09-21T06:14:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.297+0000 7f89fdffb640 1 -- 172.21.15.3:0/3921588495 <== osd.5 v2:172.21.15.204:6808/3709074044 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8a08103470 con 0x7f89f00c0460 2024-09-21T06:14:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.297+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3921588495 >> [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044] conn(0x7f89f00c0460 msgr2=0x7f89f00c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.297+0000 7f8a0efe6640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044] conn(0x7f89f00c0460 0x7f89f00c2950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.297+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3921588495 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89f00b6670 msgr2=0x7f89f00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.298+0000 7f8a0efe6640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89f00b6670 0x7f89f00b8b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f89f8005fd0 tx=0x7f89f8005850 comp rx=0 tx=0).stop 2024-09-21T06:14:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.298+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3921588495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a081021f0 msgr2=0x7f8a0819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.298+0000 7f8a0efe6640 1 --2- 172.21.15.3:0/3921588495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a081021f0 0x7f8a0819c680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400ddc0 tx=0x7f8a0400a620 comp rx=0 tx=0).stop 2024-09-21T06:14:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.298+0000 7f8a0d583640 1 -- 172.21.15.3:0/3921588495 reap_dead start 2024-09-21T06:14:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.298+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3921588495 shutdown_connections 2024-09-21T06:14:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.298+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3921588495 >> 172.21.15.3:0/3921588495 conn(0x7f8a08077c60 msgr2=0x7f8a08105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.298+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3921588495 shutdown_connections 2024-09-21T06:14:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:00.298+0000 7f8a0efe6640 1 -- 172.21.15.3:0/3921588495 wait complete. 2024-09-21T06:14:00.759 INFO:teuthology.orchestra.run.smithi003.stdout:184683593775 2024-09-21T06:14:00.760 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T06:14:01.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:00 smithi003 ceph-mon[31227]: pgmap v295: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T06:14:01.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:00 smithi137 ceph-mon[34464]: pgmap v295: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T06:14:01.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:00 smithi204 ceph-mon[34659]: pgmap v295: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T06:14:01.607 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:01.607 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:01.619 INFO:teuthology.orchestra.run.smithi003.stdout:137438953530 2024-09-21T06:14:01.619 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T06:14:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:02.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.449+0000 7f3282efb640 1 -- 172.21.15.3:0/1333201574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327c109e40 msgr2=0x7f327c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.449+0000 7f3282efb640 1 --2- 172.21.15.3:0/1333201574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327c109e40 0x7f327c110670 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3274009ca0 tx=0x7f327402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.449+0000 7f3282efb640 1 -- 172.21.15.3:0/1333201574 shutdown_connections 2024-09-21T06:14:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.449+0000 7f3282efb640 1 --2- 172.21.15.3:0/1333201574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327c109e40 0x7f327c110670 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.449+0000 7f3282efb640 1 --2- 172.21.15.3:0/1333201574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f327c1094f0 0x7f327c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.449+0000 7f3282efb640 1 --2- 172.21.15.3:0/1333201574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f327c103bb0 0x7f327c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.449+0000 7f3282efb640 1 -- 172.21.15.3:0/1333201574 >> 172.21.15.3:0/1333201574 conn(0x7f327c0ff600 msgr2=0x7f327c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.449+0000 7f3282efb640 1 -- 172.21.15.3:0/1333201574 shutdown_connections 2024-09-21T06:14:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.449+0000 7f3282efb640 1 -- 172.21.15.3:0/1333201574 wait complete. 2024-09-21T06:14:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.450+0000 7f3282efb640 1 Processor -- start 2024-09-21T06:14:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.450+0000 7f3282efb640 1 -- start start 2024-09-21T06:14:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.450+0000 7f3282efb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f327c103bb0 0x7f327c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.450+0000 7f3282efb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327c1094f0 0x7f327c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.450+0000 7f3282efb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f327c109e40 0x7f327c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.450+0000 7f3282efb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f327c113b00 con 0x7f327c1094f0 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.450+0000 7f3282efb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f327c113980 con 0x7f327c109e40 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.450+0000 7f3282efb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f327c113c80 con 0x7f327c103bb0 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.451+0000 7f327bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327c1094f0 0x7f327c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.451+0000 7f327bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327c1094f0 0x7f327c19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36456/0 (socket says 172.21.15.3:36456) 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.451+0000 7f327bfff640 1 -- 172.21.15.3:0/2828541374 learned_addr learned my addr 172.21.15.3:0/2828541374 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.451+0000 7f3280c97640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f327c103bb0 0x7f327c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.451+0000 7f3281498640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f327c109e40 0x7f327c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.451+0000 7f327bfff640 1 -- 172.21.15.3:0/2828541374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f327c103bb0 msgr2=0x7f327c19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.451+0000 7f327bfff640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f327c103bb0 0x7f327c19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.451+0000 7f327bfff640 1 -- 172.21.15.3:0/2828541374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f327c109e40 msgr2=0x7f327c19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.451+0000 7f327bfff640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f327c109e40 0x7f327c19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.451+0000 7f327bfff640 1 -- 172.21.15.3:0/2828541374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c1a0460 con 0x7f327c1094f0 2024-09-21T06:14:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.451+0000 7f327bfff640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327c1094f0 0x7f327c19aa90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f327000d740 tx=0x7f327000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.451+0000 7f3279ffb640 1 -- 172.21.15.3:0/2828541374 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270017070 con 0x7f327c1094f0 2024-09-21T06:14:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.451+0000 7f3282efb640 1 -- 172.21.15.3:0/2828541374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327c1a0750 con 0x7f327c1094f0 2024-09-21T06:14:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.452+0000 7f3279ffb640 1 -- 172.21.15.3:0/2828541374 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f327000de50 con 0x7f327c1094f0 2024-09-21T06:14:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.452+0000 7f3279ffb640 1 -- 172.21.15.3:0/2828541374 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327000f620 con 0x7f327c1094f0 2024-09-21T06:14:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.452+0000 7f3282efb640 1 -- 172.21.15.3:0/2828541374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f327c1a6eb0 con 0x7f327c1094f0 2024-09-21T06:14:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.453+0000 7f3282efb640 1 -- 172.21.15.3:0/2828541374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f327c063b50 con 0x7f327c1094f0 2024-09-21T06:14:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.455+0000 7f3279ffb640 1 -- 172.21.15.3:0/2828541374 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f327001a070 con 0x7f327c1094f0 2024-09-21T06:14:02.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.456+0000 7f3279ffb640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f32640b65f0 0x7f32640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.456+0000 7f3280c97640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f32640b65f0 0x7f32640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.456+0000 7f3279ffb640 1 -- 172.21.15.3:0/2828541374 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f32700e7510 con 0x7f327c1094f0 2024-09-21T06:14:02.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.456+0000 7f3279ffb640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798] conn(0x7f32640c03e0 0x7f32640c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.456+0000 7f3279ffb640 1 -- 172.21.15.3:0/2828541374 --> [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3270020e50 con 0x7f32640c03e0 2024-09-21T06:14:02.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.456+0000 7f3279ffb640 1 -- 172.21.15.3:0/2828541374 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f32700e7960 con 0x7f327c1094f0 2024-09-21T06:14:02.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.456+0000 7f3280c97640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f32640b65f0 0x7f32640b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f326c005d00 tx=0x7f326c00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.457+0000 7f3281498640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798] conn(0x7f32640c03e0 0x7f32640c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.457+0000 7f3281498640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798] conn(0x7f32640c03e0 0x7f32640c28d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.458+0000 7f3279ffb640 1 -- 172.21.15.3:0/2828541374 <== osd.8 v2:172.21.15.204:6816/153476798 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3270020e50 con 0x7f32640c03e0 2024-09-21T06:14:02.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.473+0000 7f3282efb640 1 -- 172.21.15.3:0/2828541374 --> [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f327c063f80 con 0x7f32640c03e0 2024-09-21T06:14:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.473+0000 7f3279ffb640 1 -- 172.21.15.3:0/2828541374 <== osd.8 v2:172.21.15.204:6816/153476798 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f327c063f80 con 0x7f32640c03e0 2024-09-21T06:14:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.474+0000 7f3282efb640 1 -- 172.21.15.3:0/2828541374 >> [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798] conn(0x7f32640c03e0 msgr2=0x7f32640c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.474+0000 7f3282efb640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798] conn(0x7f32640c03e0 0x7f32640c28d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.474+0000 7f3282efb640 1 -- 172.21.15.3:0/2828541374 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f32640b65f0 msgr2=0x7f32640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.474+0000 7f3282efb640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f32640b65f0 0x7f32640b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f326c005d00 tx=0x7f326c00b410 comp rx=0 tx=0).stop 2024-09-21T06:14:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.474+0000 7f3282efb640 1 -- 172.21.15.3:0/2828541374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327c1094f0 msgr2=0x7f327c19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.474+0000 7f3282efb640 1 --2- 172.21.15.3:0/2828541374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327c1094f0 0x7f327c19aa90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f327000d740 tx=0x7f327000dc00 comp rx=0 tx=0).stop 2024-09-21T06:14:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.474+0000 7f3281498640 1 -- 172.21.15.3:0/2828541374 reap_dead start 2024-09-21T06:14:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.474+0000 7f3282efb640 1 -- 172.21.15.3:0/2828541374 shutdown_connections 2024-09-21T06:14:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.474+0000 7f3282efb640 1 -- 172.21.15.3:0/2828541374 >> 172.21.15.3:0/2828541374 conn(0x7f327c0ff600 msgr2=0x7f327c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.474+0000 7f3282efb640 1 -- 172.21.15.3:0/2828541374 shutdown_connections 2024-09-21T06:14:02.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.474+0000 7f3282efb640 1 -- 172.21.15.3:0/2828541374 wait complete. 2024-09-21T06:14:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.335+0000 7f3897087640 1 -- 172.21.15.3:0/2901755216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3890106c30 msgr2=0x7f3890110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.335+0000 7f3897087640 1 --2- 172.21.15.3:0/2901755216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3890106c30 0x7f3890110680 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3888009ca0 tx=0x7f388802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.335+0000 7f3897087640 1 -- 172.21.15.3:0/2901755216 shutdown_connections 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.335+0000 7f3897087640 1 --2- 172.21.15.3:0/2901755216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38901062e0 0x7f38901066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.335+0000 7f3897087640 1 --2- 172.21.15.3:0/2901755216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3890106c30 0x7f3890110680 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.335+0000 7f3897087640 1 --2- 172.21.15.3:0/2901755216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3890102000 0x7f3890102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.335+0000 7f3897087640 1 -- 172.21.15.3:0/2901755216 >> 172.21.15.3:0/2901755216 conn(0x7f3890079e40 msgr2=0x7f389007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.335+0000 7f3897087640 1 -- 172.21.15.3:0/2901755216 shutdown_connections 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.335+0000 7f3897087640 1 -- 172.21.15.3:0/2901755216 wait complete. 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.336+0000 7f3897087640 1 Processor -- start 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.336+0000 7f3897087640 1 -- start start 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.336+0000 7f3897087640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3890102000 0x7f389019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.336+0000 7f3897087640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38901062e0 0x7f389019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.336+0000 7f3897087640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3890106c30 0x7f389019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.336+0000 7f3897087640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3890113ad0 con 0x7f3890102000 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.336+0000 7f3897087640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3890113950 con 0x7f3890106c30 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.336+0000 7f3897087640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3890113c50 con 0x7f38901062e0 2024-09-21T06:14:02.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.336+0000 7f3894e23640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3890102000 0x7f389019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.336+0000 7f3894e23640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3890102000 0x7f389019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36434/0 (socket says 172.21.15.3:36434) 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.336+0000 7f3894e23640 1 -- 172.21.15.3:0/3495160598 learned_addr learned my addr 172.21.15.3:0/3495160598 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.336+0000 7f388ffff640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38901062e0 0x7f389019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.337+0000 7f3895624640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3890106c30 0x7f389019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.337+0000 7f3894e23640 1 -- 172.21.15.3:0/3495160598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38901062e0 msgr2=0x7f389019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.337+0000 7f3894e23640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38901062e0 0x7f389019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.337+0000 7f3894e23640 1 -- 172.21.15.3:0/3495160598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3890106c30 msgr2=0x7f389019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.337+0000 7f3894e23640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3890106c30 0x7f389019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.337+0000 7f3894e23640 1 -- 172.21.15.3:0/3495160598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38901a03f0 con 0x7f3890102000 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.337+0000 7f3895624640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3890106c30 0x7f389019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.337+0000 7f3894e23640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3890102000 0x7f389019a480 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f388000d770 tx=0x7f388000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.337+0000 7f388dffb640 1 -- 172.21.15.3:0/3495160598 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880017070 con 0x7f3890102000 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.337+0000 7f388dffb640 1 -- 172.21.15.3:0/3495160598 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38800054d0 con 0x7f3890102000 2024-09-21T06:14:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.337+0000 7f3897087640 1 -- 172.21.15.3:0/3495160598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38901a06e0 con 0x7f3890102000 2024-09-21T06:14:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.337+0000 7f388dffb640 1 -- 172.21.15.3:0/3495160598 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880004c30 con 0x7f3890102000 2024-09-21T06:14:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.337+0000 7f3897087640 1 -- 172.21.15.3:0/3495160598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38901a6e40 con 0x7f3890102000 2024-09-21T06:14:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.338+0000 7f3897087640 1 -- 172.21.15.3:0/3495160598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3890063b50 con 0x7f3890102000 2024-09-21T06:14:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.340+0000 7f388dffb640 1 -- 172.21.15.3:0/3495160598 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f3880004dd0 con 0x7f3890102000 2024-09-21T06:14:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.341+0000 7f388dffb640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f38780b65f0 0x7f38780b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.341+0000 7f388ffff640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f38780b65f0 0x7f38780b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.341+0000 7f388dffb640 1 -- 172.21.15.3:0/3495160598 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f38800e74e0 con 0x7f3890102000 2024-09-21T06:14:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.341+0000 7f388dffb640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115] conn(0x7f38780c03e0 0x7f38780c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.341+0000 7f388dffb640 1 -- 172.21.15.3:0/3495160598 --> [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3880021e50 con 0x7f38780c03e0 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.341+0000 7f388dffb640 1 -- 172.21.15.3:0/3495160598 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f38800e7930 con 0x7f3890102000 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.342+0000 7f388ffff640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f38780b65f0 0x7f38780b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f389006b260 tx=0x7f388400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.342+0000 7f3895624640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115] conn(0x7f38780c03e0 0x7f38780c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.342+0000 7f3895624640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115] conn(0x7f38780c03e0 0x7f38780c28d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.343+0000 7f388dffb640 1 -- 172.21.15.3:0/3495160598 <== osd.1 v2:172.21.15.137:6800/4150747115 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3880021e50 con 0x7f38780c03e0 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.358+0000 7f3897087640 1 -- 172.21.15.3:0/3495160598 --> [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3890063fd0 con 0x7f38780c03e0 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.359+0000 7f388dffb640 1 -- 172.21.15.3:0/3495160598 <== osd.1 v2:172.21.15.137:6800/4150747115 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3890063fd0 con 0x7f38780c03e0 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.359+0000 7f3897087640 1 -- 172.21.15.3:0/3495160598 >> [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115] conn(0x7f38780c03e0 msgr2=0x7f38780c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.359+0000 7f3897087640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115] conn(0x7f38780c03e0 0x7f38780c28d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.359+0000 7f3897087640 1 -- 172.21.15.3:0/3495160598 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f38780b65f0 msgr2=0x7f38780b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.359+0000 7f3897087640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f38780b65f0 0x7f38780b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f389006b260 tx=0x7f388400b390 comp rx=0 tx=0).stop 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.359+0000 7f3897087640 1 -- 172.21.15.3:0/3495160598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3890102000 msgr2=0x7f389019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.359+0000 7f3897087640 1 --2- 172.21.15.3:0/3495160598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3890102000 0x7f389019a480 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f388000d770 tx=0x7f388000dc30 comp rx=0 tx=0).stop 2024-09-21T06:14:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.359+0000 7f3895624640 1 -- 172.21.15.3:0/3495160598 reap_dead start 2024-09-21T06:14:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.359+0000 7f3897087640 1 -- 172.21.15.3:0/3495160598 shutdown_connections 2024-09-21T06:14:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.359+0000 7f3897087640 1 -- 172.21.15.3:0/3495160598 >> 172.21.15.3:0/3495160598 conn(0x7f3890079e40 msgr2=0x7f3890107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.360+0000 7f3897087640 1 -- 172.21.15.3:0/3495160598 shutdown_connections 2024-09-21T06:14:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:02.360+0000 7f3897087640 1 -- 172.21.15.3:0/3495160598 wait complete. 2024-09-21T06:14:02.803 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:02.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:02 smithi003 ceph-mon[31227]: pgmap v296: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-21T06:14:03.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:02 smithi137 ceph-mon[34464]: pgmap v296: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-21T06:14:03.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:02 smithi204 ceph-mon[34659]: pgmap v296: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-21T06:14:03.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.436+0000 7f69f31c1640 1 -- 172.21.15.3:0/87223769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69ec102a80 msgr2=0x7f69ec102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:03.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.436+0000 7f69f31c1640 1 --2- 172.21.15.3:0/87223769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69ec102a80 0x7f69ec102e90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009900 tx=0x7f69d402a410 comp rx=0 tx=0).stop 2024-09-21T06:14:03.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.436+0000 7f69f31c1640 1 -- 172.21.15.3:0/87223769 shutdown_connections 2024-09-21T06:14:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.436+0000 7f69f31c1640 1 --2- 172.21.15.3:0/87223769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69ec1083c0 0x7f69ec1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.436+0000 7f69f31c1640 1 --2- 172.21.15.3:0/87223769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ec108d10 0x7f69ec10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.436+0000 7f69f31c1640 1 --2- 172.21.15.3:0/87223769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69ec102a80 0x7f69ec102e90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.436+0000 7f69f31c1640 1 -- 172.21.15.3:0/87223769 >> 172.21.15.3:0/87223769 conn(0x7f69ec079740 msgr2=0x7f69ec079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.436+0000 7f69f31c1640 1 -- 172.21.15.3:0/87223769 shutdown_connections 2024-09-21T06:14:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.436+0000 7f69f31c1640 1 -- 172.21.15.3:0/87223769 wait complete. 2024-09-21T06:14:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f31c1640 1 Processor -- start 2024-09-21T06:14:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f31c1640 1 -- start start 2024-09-21T06:14:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f31c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69ec102a80 0x7f69ec19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f31c1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ec1083c0 0x7f69ec19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f31c1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69ec108d10 0x7f69ec19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f31c1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec113aa0 con 0x7f69ec102a80 2024-09-21T06:14:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f31c1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec113920 con 0x7f69ec1083c0 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f31c1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec113c20 con 0x7f69ec108d10 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f0f5d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69ec102a80 0x7f69ec19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f0f5d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69ec102a80 0x7f69ec19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36474/0 (socket says 172.21.15.3:36474) 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f0f5d640 1 -- 172.21.15.3:0/1169687572 learned_addr learned my addr 172.21.15.3:0/1169687572 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69ebfff640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ec1083c0 0x7f69ec19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f175e640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69ec108d10 0x7f69ec19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f0f5d640 1 -- 172.21.15.3:0/1169687572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69ec108d10 msgr2=0x7f69ec19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f0f5d640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69ec108d10 0x7f69ec19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f0f5d640 1 -- 172.21.15.3:0/1169687572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ec1083c0 msgr2=0x7f69ec19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f0f5d640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ec1083c0 0x7f69ec19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f0f5d640 1 -- 172.21.15.3:0/1169687572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ec1a0460 con 0x7f69ec102a80 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69ebfff640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ec1083c0 0x7f69ec19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.437+0000 7f69f175e640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69ec108d10 0x7f69ec19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.438+0000 7f69f0f5d640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69ec102a80 0x7f69ec19a4f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f69d4005940 tx=0x7f69d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.438+0000 7f69e9ffb640 1 -- 172.21.15.3:0/1169687572 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d403f030 con 0x7f69ec102a80 2024-09-21T06:14:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.438+0000 7f69f31c1640 1 -- 172.21.15.3:0/1169687572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69ec1a06f0 con 0x7f69ec102a80 2024-09-21T06:14:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.438+0000 7f69e9ffb640 1 -- 172.21.15.3:0/1169687572 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69d4003cb0 con 0x7f69ec102a80 2024-09-21T06:14:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.438+0000 7f69e9ffb640 1 -- 172.21.15.3:0/1169687572 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4035070 con 0x7f69ec102a80 2024-09-21T06:14:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.438+0000 7f69f31c1640 1 -- 172.21.15.3:0/1169687572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69ec1a6eb0 con 0x7f69ec102a80 2024-09-21T06:14:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.439+0000 7f69f31c1640 1 -- 172.21.15.3:0/1169687572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69ec102e90 con 0x7f69ec102a80 2024-09-21T06:14:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.441+0000 7f69e9ffb640 1 -- 172.21.15.3:0/1169687572 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f69d4042030 con 0x7f69ec102a80 2024-09-21T06:14:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.441+0000 7f69e9ffb640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f69d80b65f0 0x7f69d80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.442+0000 7f69ebfff640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f69d80b65f0 0x7f69d80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.442+0000 7f69e9ffb640 1 -- 172.21.15.3:0/1169687572 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f69d4108180 con 0x7f69ec102a80 2024-09-21T06:14:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.442+0000 7f69ebfff640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f69d80b65f0 0x7f69d80b8b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f69ec100c70 tx=0x7f69dc009520 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.445+0000 7f69e9ffb640 1 -- 172.21.15.3:0/1169687572 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f69d40ce7d0 con 0x7f69ec102a80 2024-09-21T06:14:03.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.641+0000 7f69f31c1640 1 -- 172.21.15.3:0/1169687572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f69ec10aae0 con 0x7f69ec102a80 2024-09-21T06:14:03.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.642+0000 7f69e9ffb640 1 -- 172.21.15.3:0/1169687572 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f69d40d2bb0 con 0x7f69ec102a80 2024-09-21T06:14:03.644 INFO:teuthology.orchestra.run.smithi003.stdout:279172874269 2024-09-21T06:14:03.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.644+0000 7f69f31c1640 1 -- 172.21.15.3:0/1169687572 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f69d80b65f0 msgr2=0x7f69d80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:03.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.644+0000 7f69f31c1640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f69d80b65f0 0x7f69d80b8b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f69ec100c70 tx=0x7f69dc009520 comp rx=0 tx=0).stop 2024-09-21T06:14:03.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.644+0000 7f69f31c1640 1 -- 172.21.15.3:0/1169687572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69ec102a80 msgr2=0x7f69ec19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:03.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.644+0000 7f69f31c1640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69ec102a80 0x7f69ec19a4f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f69d4005940 tx=0x7f69d40042f0 comp rx=0 tx=0).stop 2024-09-21T06:14:03.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.644+0000 7f69f31c1640 1 -- 172.21.15.3:0/1169687572 shutdown_connections 2024-09-21T06:14:03.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.644+0000 7f69f31c1640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f69d80b65f0 0x7f69d80b8b00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:03.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.644+0000 7f69f31c1640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69ec108d10 0x7f69ec19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:03.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.644+0000 7f69f31c1640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ec1083c0 0x7f69ec19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:03.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.644+0000 7f69f31c1640 1 --2- 172.21.15.3:0/1169687572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69ec102a80 0x7f69ec19a4f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:03.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.644+0000 7f69f31c1640 1 -- 172.21.15.3:0/1169687572 >> 172.21.15.3:0/1169687572 conn(0x7f69ec079740 msgr2=0x7f69ec101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:03.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.644+0000 7f69f31c1640 1 -- 172.21.15.3:0/1169687572 shutdown_connections 2024-09-21T06:14:03.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:03.644+0000 7f69f31c1640 1 -- 172.21.15.3:0/1169687572 wait complete. 2024-09-21T06:14:03.989 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:03 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1169687572' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T06:14:04.021 INFO:teuthology.orchestra.run.smithi003.stdout:60129542220 2024-09-21T06:14:04.021 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T06:14:04.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:03 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1169687572' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T06:14:04.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:03 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1169687572' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T06:14:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.714+0000 7fbafc05b640 1 -- 172.21.15.3:0/2715106298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf41094d0 msgr2=0x7fbaf41098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.714+0000 7fbafc05b640 1 --2- 172.21.15.3:0/2715106298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf41094d0 0x7fbaf41098e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009900 tx=0x7fbaf002a410 comp rx=0 tx=0).stop 2024-09-21T06:14:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.714+0000 7fbafc05b640 1 -- 172.21.15.3:0/2715106298 shutdown_connections 2024-09-21T06:14:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.714+0000 7fbafc05b640 1 --2- 172.21.15.3:0/2715106298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaf4109e20 0x7fbaf4110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.714+0000 7fbafc05b640 1 --2- 172.21.15.3:0/2715106298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf41094d0 0x7fbaf41098e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.714+0000 7fbafc05b640 1 --2- 172.21.15.3:0/2715106298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaf4103b90 0x7fbaf4103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.714+0000 7fbafc05b640 1 -- 172.21.15.3:0/2715106298 >> 172.21.15.3:0/2715106298 conn(0x7fbaf40ff600 msgr2=0x7fbaf4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.714+0000 7fbafc05b640 1 -- 172.21.15.3:0/2715106298 shutdown_connections 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.714+0000 7fbafc05b640 1 -- 172.21.15.3:0/2715106298 wait complete. 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.715+0000 7fbafc05b640 1 Processor -- start 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.715+0000 7fbafc05b640 1 -- start start 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.715+0000 7fbafc05b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaf4103b90 0x7fbaf419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.715+0000 7fbafc05b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaf41094d0 0x7fbaf419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.715+0000 7fbafc05b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf4109e20 0x7fbaf419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.715+0000 7fbafc05b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4113b00 con 0x7fbaf4109e20 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.715+0000 7fbafc05b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4113980 con 0x7fbaf41094d0 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.715+0000 7fbafc05b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4113c80 con 0x7fbaf4103b90 2024-09-21T06:14:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.715+0000 7fbafa5f8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf4109e20 0x7fbaf419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.715+0000 7fbafa5f8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf4109e20 0x7fbaf419fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36490/0 (socket says 172.21.15.3:36490) 2024-09-21T06:14:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.715+0000 7fbafa5f8640 1 -- 172.21.15.3:0/3779515542 learned_addr learned my addr 172.21.15.3:0/3779515542 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.715+0000 7fbaf9df7640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaf4103b90 0x7fbaf419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.715+0000 7fbaf95f6640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaf41094d0 0x7fbaf419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.716+0000 7fbafa5f8640 1 -- 172.21.15.3:0/3779515542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaf4103b90 msgr2=0x7fbaf419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.716+0000 7fbafa5f8640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaf4103b90 0x7fbaf419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.716+0000 7fbafa5f8640 1 -- 172.21.15.3:0/3779515542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaf41094d0 msgr2=0x7fbaf419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.716+0000 7fbafa5f8640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaf41094d0 0x7fbaf419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.716+0000 7fbafa5f8640 1 -- 172.21.15.3:0/3779515542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf41a03c0 con 0x7fbaf4109e20 2024-09-21T06:14:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.716+0000 7fbafa5f8640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf4109e20 0x7fbaf419fd50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00b970 tx=0x7fbaec00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.716+0000 7fbae6ffd640 1 -- 172.21.15.3:0/3779515542 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec019070 con 0x7fbaf4109e20 2024-09-21T06:14:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.716+0000 7fbae6ffd640 1 -- 172.21.15.3:0/3779515542 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaec014440 con 0x7fbaf4109e20 2024-09-21T06:14:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.716+0000 7fbafc05b640 1 -- 172.21.15.3:0/3779515542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf41a06b0 con 0x7fbaf4109e20 2024-09-21T06:14:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.716+0000 7fbae6ffd640 1 -- 172.21.15.3:0/3779515542 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0075d0 con 0x7fbaf4109e20 2024-09-21T06:14:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.716+0000 7fbafc05b640 1 -- 172.21.15.3:0/3779515542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf41a6ed0 con 0x7fbaf4109e20 2024-09-21T06:14:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.717+0000 7fbafc05b640 1 -- 172.21.15.3:0/3779515542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf4103fa0 con 0x7fbaf4109e20 2024-09-21T06:14:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.719+0000 7fbae6ffd640 1 -- 172.21.15.3:0/3779515542 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fbaec01c070 con 0x7fbaf4109e20 2024-09-21T06:14:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.719+0000 7fbae6ffd640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbae00b66a0 0x7fbae00b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.720+0000 7fbaf9df7640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbae00b66a0 0x7fbae00b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.720+0000 7fbae6ffd640 1 -- 172.21.15.3:0/3779515542 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fbaec0e6e90 con 0x7fbaf4109e20 2024-09-21T06:14:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.720+0000 7fbaf9df7640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbae00b66a0 0x7fbae00b8bb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbae800abe0 tx=0x7fbae800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.723+0000 7fbae6ffd640 1 -- 172.21.15.3:0/3779515542 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbaec0b1860 con 0x7fbaf4109e20 2024-09-21T06:14:04.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.914+0000 7fbafc05b640 1 -- 172.21.15.3:0/3779515542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbaf419b640 con 0x7fbaf4109e20 2024-09-21T06:14:04.917 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:04 smithi003 ceph-mon[31227]: pgmap v297: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-21T06:14:04.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.915+0000 7fbae6ffd640 1 -- 172.21.15.3:0/3779515542 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbaf419b640 con 0x7fbaf4109e20 2024-09-21T06:14:04.917 INFO:teuthology.orchestra.run.smithi003.stdout:184683593777 2024-09-21T06:14:04.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.917+0000 7fbafc05b640 1 -- 172.21.15.3:0/3779515542 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbae00b66a0 msgr2=0x7fbae00b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:04.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.917+0000 7fbafc05b640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbae00b66a0 0x7fbae00b8bb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbae800abe0 tx=0x7fbae800a2d0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.917+0000 7fbafc05b640 1 -- 172.21.15.3:0/3779515542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf4109e20 msgr2=0x7fbaf419fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:04.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.917+0000 7fbafc05b640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf4109e20 0x7fbaf419fd50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00b970 tx=0x7fbaec00bd70 comp rx=0 tx=0).stop 2024-09-21T06:14:04.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.917+0000 7fbafc05b640 1 -- 172.21.15.3:0/3779515542 shutdown_connections 2024-09-21T06:14:04.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.917+0000 7fbafc05b640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbae00b66a0 0x7fbae00b8bb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.917+0000 7fbafc05b640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf4109e20 0x7fbaf419fd50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.917+0000 7fbafc05b640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaf41094d0 0x7fbaf419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.917+0000 7fbafc05b640 1 --2- 172.21.15.3:0/3779515542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaf4103b90 0x7fbaf419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.917+0000 7fbafc05b640 1 -- 172.21.15.3:0/3779515542 >> 172.21.15.3:0/3779515542 conn(0x7fbaf40ff600 msgr2=0x7fbaf4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:04.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.917+0000 7fbafc05b640 1 -- 172.21.15.3:0/3779515542 shutdown_connections 2024-09-21T06:14:04.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:04.918+0000 7fbafc05b640 1 -- 172.21.15.3:0/3779515542 wait complete. 2024-09-21T06:14:05.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:04 smithi137 ceph-mon[34464]: pgmap v297: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-21T06:14:05.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:04 smithi204 ceph-mon[34659]: pgmap v297: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-21T06:14:05.657 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874267 got 279172874269 for osd.10 2024-09-21T06:14:05.657 DEBUG:teuthology.parallel:result is None 2024-09-21T06:14:05.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.854+0000 7f1f87595640 1 -- 172.21.15.3:0/2193368249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f801031f0 msgr2=0x7f1f80103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:05.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.854+0000 7f1f87595640 1 --2- 172.21.15.3:0/2193368249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f801031f0 0x7f1f80103600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009900 tx=0x7f1f7c02a410 comp rx=0 tx=0).stop 2024-09-21T06:14:05.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.854+0000 7f1f87595640 1 -- 172.21.15.3:0/2193368249 shutdown_connections 2024-09-21T06:14:05.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.854+0000 7f1f87595640 1 --2- 172.21.15.3:0/2193368249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f801051c0 0x7f1f801055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:05.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.854+0000 7f1f87595640 1 --2- 172.21.15.3:0/2193368249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f80105b10 0x7f1f8010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:05.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.854+0000 7f1f87595640 1 --2- 172.21.15.3:0/2193368249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f801031f0 0x7f1f80103600 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:05.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.854+0000 7f1f87595640 1 -- 172.21.15.3:0/2193368249 >> 172.21.15.3:0/2193368249 conn(0x7f1f80079e40 msgr2=0x7f1f8007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:05.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.855+0000 7f1f87595640 1 -- 172.21.15.3:0/2193368249 shutdown_connections 2024-09-21T06:14:05.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.855+0000 7f1f87595640 1 -- 172.21.15.3:0/2193368249 wait complete. 2024-09-21T06:14:05.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.855+0000 7f1f87595640 1 Processor -- start 2024-09-21T06:14:05.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.855+0000 7f1f87595640 1 -- start start 2024-09-21T06:14:05.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.855+0000 7f1f87595640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f801031f0 0x7f1f8019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:05.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f87595640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f801051c0 0x7f1f8019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:05.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f87595640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f80105b10 0x7f1f8019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:05.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f87595640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801129e0 con 0x7f1f801051c0 2024-09-21T06:14:05.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f87595640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1f80112860 con 0x7f1f801031f0 2024-09-21T06:14:05.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f87595640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f80112b60 con 0x7f1f80105b10 2024-09-21T06:14:05.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f84b30640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f801051c0 0x7f1f8019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:05.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f84b30640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f801051c0 0x7f1f8019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59230/0 (socket says 172.21.15.3:59230) 2024-09-21T06:14:05.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f84b30640 1 -- 172.21.15.3:0/405366781 learned_addr learned my addr 172.21.15.3:0/405366781 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:05.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f85b32640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f80105b10 0x7f1f8019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:05.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f85331640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f801031f0 0x7f1f8019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:05.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f84b30640 1 -- 172.21.15.3:0/405366781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f80105b10 msgr2=0x7f1f8019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:05.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f84b30640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f80105b10 0x7f1f8019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:05.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f84b30640 1 -- 172.21.15.3:0/405366781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f801031f0 msgr2=0x7f1f8019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:05.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f84b30640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f801031f0 0x7f1f8019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:05.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f84b30640 1 -- 172.21.15.3:0/405366781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f801a04f0 con 0x7f1f801051c0 2024-09-21T06:14:05.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f84b30640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f801051c0 0x7f1f8019ab20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000d950 tx=0x7f1f7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:05.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f767fc640 1 -- 172.21.15.3:0/405366781 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70017070 con 0x7f1f801051c0 2024-09-21T06:14:05.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f767fc640 1 -- 172.21.15.3:0/405366781 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f70009c30 con 0x7f1f801051c0 2024-09-21T06:14:05.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f87595640 1 -- 172.21.15.3:0/405366781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f801a07e0 con 0x7f1f801051c0 2024-09-21T06:14:05.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f767fc640 1 -- 172.21.15.3:0/405366781 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f700047a0 con 0x7f1f801051c0 2024-09-21T06:14:05.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.856+0000 7f1f87595640 1 -- 172.21.15.3:0/405366781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f801a6f40 con 0x7f1f801051c0 2024-09-21T06:14:05.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.857+0000 7f1f87595640 1 -- 172.21.15.3:0/405366781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f80103600 con 0x7f1f801051c0 2024-09-21T06:14:05.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.859+0000 7f1f767fc640 1 -- 172.21.15.3:0/405366781 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f1f70004020 con 0x7f1f801051c0 2024-09-21T06:14:05.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.860+0000 7f1f767fc640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1f6c0b6670 0x7f1f6c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:05.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.860+0000 7f1f85331640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1f6c0b6670 0x7f1f6c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:05.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.860+0000 7f1f767fc640 1 -- 172.21.15.3:0/405366781 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f1f700e75b0 con 0x7f1f801051c0 2024-09-21T06:14:05.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.860+0000 7f1f85331640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1f6c0b6670 0x7f1f6c0b8b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c004770 tx=0x7f1f7c0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:05.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:05.863+0000 7f1f767fc640 1 -- 172.21.15.3:0/405366781 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1f70020020 con 0x7f1f801051c0 2024-09-21T06:14:06.000 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:05 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3779515542' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T06:14:06.000 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:05 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:14:06.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.066+0000 7f1f87595640 1 -- 172.21.15.3:0/405366781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1f801a71a0 con 0x7f1f801051c0 2024-09-21T06:14:06.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.067+0000 7f1f767fc640 1 -- 172.21.15.3:0/405366781 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1f700adc20 con 0x7f1f801051c0 2024-09-21T06:14:06.069 INFO:teuthology.orchestra.run.smithi003.stdout:38654705745 2024-09-21T06:14:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.069+0000 7f1f87595640 1 -- 172.21.15.3:0/405366781 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1f6c0b6670 msgr2=0x7f1f6c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.069+0000 7f1f87595640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1f6c0b6670 0x7f1f6c0b8b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c004770 tx=0x7f1f7c0058d0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.069+0000 7f1f87595640 1 -- 172.21.15.3:0/405366781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f801051c0 msgr2=0x7f1f8019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.069+0000 7f1f87595640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f801051c0 0x7f1f8019ab20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000d950 tx=0x7f1f7000de10 comp rx=0 tx=0).stop 2024-09-21T06:14:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.069+0000 7f1f87595640 1 -- 172.21.15.3:0/405366781 shutdown_connections 2024-09-21T06:14:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.069+0000 7f1f87595640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1f6c0b6670 0x7f1f6c0b8b80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.069+0000 7f1f87595640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f80105b10 0x7f1f8019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.069+0000 7f1f87595640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f801051c0 0x7f1f8019ab20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.069+0000 7f1f87595640 1 --2- 172.21.15.3:0/405366781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f801031f0 0x7f1f8019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.069+0000 7f1f87595640 1 -- 172.21.15.3:0/405366781 >> 172.21.15.3:0/405366781 conn(0x7f1f80079e40 msgr2=0x7f1f80106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:06.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.069+0000 7f1f87595640 1 -- 172.21.15.3:0/405366781 shutdown_connections 2024-09-21T06:14:06.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:06.069+0000 7f1f87595640 1 -- 172.21.15.3:0/405366781 wait complete. 2024-09-21T06:14:06.123 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:05 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3779515542' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T06:14:06.123 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:05 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:14:06.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:05 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3779515542' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T06:14:06.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:05 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:14:06.355 INFO:teuthology.orchestra.run.smithi003.stdout:158913790007 2024-09-21T06:14:06.356 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T06:14:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:06.769 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:06.802 INFO:teuthology.orchestra.run.smithi003.stdout:231928234024 2024-09-21T06:14:06.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T06:14:07.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:06 smithi003 ceph-mon[31227]: pgmap v298: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:07.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:06 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/405366781' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T06:14:07.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:06 smithi137 ceph-mon[34464]: pgmap v298: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:07.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:06 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/405366781' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T06:14:07.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:06 smithi204 ceph-mon[34659]: pgmap v298: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:07.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:06 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/405366781' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T06:14:07.490 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593775 got 184683593777 for osd.6 2024-09-21T06:14:07.490 DEBUG:teuthology.parallel:result is None 2024-09-21T06:14:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:07.842 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:07.842 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:08.268 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:08.268 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:08.268 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:08.268 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:08 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:08.268 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:08 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/4084661295' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T06:14:08.301 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705743 got 38654705745 for osd.0 2024-09-21T06:14:08.301 DEBUG:teuthology.parallel:result is None 2024-09-21T06:14:08.322 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:08.323 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:08.323 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:08.323 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:08 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:08.323 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:08 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/4084661295' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T06:14:08.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:08.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:08.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:08.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:08 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:08.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:08 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/4084661295' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T06:14:09.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.083+0000 7f230ccb8640 1 -- 172.21.15.3:0/3909580840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2308105300 msgr2=0x7f230810d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.083+0000 7f230ccb8640 1 --2- 172.21.15.3:0/3909580840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2308105300 0x7f230810d850 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009900 tx=0x7f22fc02a410 comp rx=0 tx=0).stop 2024-09-21T06:14:09.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.083+0000 7f230ccb8640 1 -- 172.21.15.3:0/3909580840 shutdown_connections 2024-09-21T06:14:09.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.083+0000 7f230ccb8640 1 --2- 172.21.15.3:0/3909580840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f230810de10 0x7f2308110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.083+0000 7f230ccb8640 1 --2- 172.21.15.3:0/3909580840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2308105300 0x7f230810d850 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.083+0000 7f230ccb8640 1 --2- 172.21.15.3:0/3909580840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23081049b0 0x7f2308104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.083+0000 7f230ccb8640 1 -- 172.21.15.3:0/3909580840 >> 172.21.15.3:0/3909580840 conn(0x7f2308079730 msgr2=0x7f2308079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:09.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.083+0000 7f230ccb8640 1 -- 172.21.15.3:0/3909580840 shutdown_connections 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.083+0000 7f230ccb8640 1 -- 172.21.15.3:0/3909580840 wait complete. 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.084+0000 7f230ccb8640 1 Processor -- start 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.084+0000 7f230ccb8640 1 -- start start 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.084+0000 7f230ccb8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23081049b0 0x7f230819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.084+0000 7f230ccb8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2308105300 0x7f230819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.084+0000 7f230ccb8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f230810de10 0x7f230819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.084+0000 7f230ccb8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2308113a70 con 0x7f230810de10 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.084+0000 7f230ccb8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23081138f0 con 0x7f23081049b0 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.084+0000 7f230ccb8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2308113bf0 con 0x7f2308105300 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f2306d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f230810de10 0x7f230819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f2306d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f230810de10 0x7f230819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59274/0 (socket says 172.21.15.3:59274) 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f2306d9d640 1 -- 172.21.15.3:0/1502541550 learned_addr learned my addr 172.21.15.3:0/1502541550 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f2305d9b640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2308105300 0x7f230819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f230659c640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23081049b0 0x7f230819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f2306d9d640 1 -- 172.21.15.3:0/1502541550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2308105300 msgr2=0x7f230819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f2306d9d640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2308105300 0x7f230819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f2306d9d640 1 -- 172.21.15.3:0/1502541550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23081049b0 msgr2=0x7f230819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f2306d9d640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23081049b0 0x7f230819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f2306d9d640 1 -- 172.21.15.3:0/1502541550 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23081a0430 con 0x7f230810de10 2024-09-21T06:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f2306d9d640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f230810de10 0x7f230819fd00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f230000eb10 tx=0x7f230000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f22f77fe640 1 -- 172.21.15.3:0/1502541550 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300004b20 con 0x7f230810de10 2024-09-21T06:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f22f77fe640 1 -- 172.21.15.3:0/1502541550 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2300004cc0 con 0x7f230810de10 2024-09-21T06:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f230ccb8640 1 -- 172.21.15.3:0/1502541550 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23081a0720 con 0x7f230810de10 2024-09-21T06:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f22f77fe640 1 -- 172.21.15.3:0/1502541550 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300010670 con 0x7f230810de10 2024-09-21T06:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.085+0000 7f230ccb8640 1 -- 172.21.15.3:0/1502541550 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23081a6e80 con 0x7f230810de10 2024-09-21T06:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.086+0000 7f230ccb8640 1 -- 172.21.15.3:0/1502541550 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2308104dc0 con 0x7f230810de10 2024-09-21T06:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.088+0000 7f22f77fe640 1 -- 172.21.15.3:0/1502541550 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f2300005000 con 0x7f230810de10 2024-09-21T06:14:09.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.089+0000 7f22f77fe640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f22f00b6670 0x7f22f00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.089+0000 7f230659c640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f22f00b6670 0x7f22f00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.089+0000 7f22f77fe640 1 -- 172.21.15.3:0/1502541550 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f23000e7c40 con 0x7f230810de10 2024-09-21T06:14:09.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.089+0000 7f230659c640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f22f00b6670 0x7f22f00b8b80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f22f8005fd0 tx=0x7f22f800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:09.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.092+0000 7f22f77fe640 1 -- 172.21.15.3:0/1502541550 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2300056710 con 0x7f230810de10 2024-09-21T06:14:09.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.283+0000 7f230ccb8640 1 -- 172.21.15.3:0/1502541550 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f230819b5f0 con 0x7f230810de10 2024-09-21T06:14:09.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.283+0000 7f22f77fe640 1 -- 172.21.15.3:0/1502541550 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f230819b5f0 con 0x7f230810de10 2024-09-21T06:14:09.285 INFO:teuthology.orchestra.run.smithi003.stdout:137438953533 2024-09-21T06:14:09.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.285+0000 7f230ccb8640 1 -- 172.21.15.3:0/1502541550 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f22f00b6670 msgr2=0x7f22f00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.285+0000 7f230ccb8640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f22f00b6670 0x7f22f00b8b80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f22f8005fd0 tx=0x7f22f800b410 comp rx=0 tx=0).stop 2024-09-21T06:14:09.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.285+0000 7f230ccb8640 1 -- 172.21.15.3:0/1502541550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f230810de10 msgr2=0x7f230819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.285+0000 7f230ccb8640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f230810de10 0x7f230819fd00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f230000eb10 tx=0x7f230000c2d0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.285+0000 7f230ccb8640 1 -- 172.21.15.3:0/1502541550 shutdown_connections 2024-09-21T06:14:09.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.285+0000 7f230ccb8640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f22f00b6670 0x7f22f00b8b80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.285+0000 7f230ccb8640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2308105300 0x7f230819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.285+0000 7f230ccb8640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f230810de10 0x7f230819fd00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.285+0000 7f230ccb8640 1 --2- 172.21.15.3:0/1502541550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23081049b0 0x7f230819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.285+0000 7f230ccb8640 1 -- 172.21.15.3:0/1502541550 >> 172.21.15.3:0/1502541550 conn(0x7f2308079730 msgr2=0x7f23080ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:09.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.285+0000 7f230ccb8640 1 -- 172.21.15.3:0/1502541550 shutdown_connections 2024-09-21T06:14:09.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:09.285+0000 7f230ccb8640 1 -- 172.21.15.3:0/1502541550 wait complete. 2024-09-21T06:14:09.324 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:09 smithi137 ceph-mon[34464]: pgmap v299: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-21T06:14:09.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:09 smithi003 ceph-mon[31227]: pgmap v299: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-21T06:14:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:09 smithi204 ceph-mon[34659]: pgmap v299: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-21T06:14:10.042 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953530 got 137438953533 for osd.4 2024-09-21T06:14:10.042 DEBUG:teuthology.parallel:result is None 2024-09-21T06:14:10.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.705+0000 7f191f657640 1 -- 172.21.15.3:0/4213437273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1918103b60 msgr2=0x7f1918103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:10.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.705+0000 7f191f657640 1 --2- 172.21.15.3:0/4213437273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1918103b60 0x7f1918103f70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1914009900 tx=0x7f191402a410 comp rx=0 tx=0).stop 2024-09-21T06:14:10.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.705+0000 7f191f657640 1 -- 172.21.15.3:0/4213437273 shutdown_connections 2024-09-21T06:14:10.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.705+0000 7f191f657640 1 --2- 172.21.15.3:0/4213437273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1918109df0 0x7f1918110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:10.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.705+0000 7f191f657640 1 --2- 172.21.15.3:0/4213437273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19181094a0 0x7f19181098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:10.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.705+0000 7f191f657640 1 --2- 172.21.15.3:0/4213437273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1918103b60 0x7f1918103f70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:10.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.705+0000 7f191f657640 1 -- 172.21.15.3:0/4213437273 >> 172.21.15.3:0/4213437273 conn(0x7f191807b850 msgr2=0x7f191807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:10.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.705+0000 7f191f657640 1 -- 172.21.15.3:0/4213437273 shutdown_connections 2024-09-21T06:14:10.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.705+0000 7f191f657640 1 -- 172.21.15.3:0/4213437273 wait complete. 2024-09-21T06:14:10.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191f657640 1 Processor -- start 2024-09-21T06:14:10.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191f657640 1 -- start start 2024-09-21T06:14:10.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191f657640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1918103b60 0x7f191819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:10.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191f657640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19181094a0 0x7f191819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:10.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191f657640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1918109df0 0x7f191819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:10.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191f657640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1918113b00 con 0x7f1918103b60 2024-09-21T06:14:10.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191f657640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1918113980 con 0x7f1918109df0 2024-09-21T06:14:10.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191f657640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1918113c80 con 0x7f19181094a0 2024-09-21T06:14:10.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191d3f3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1918103b60 0x7f191819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:10.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191d3f3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1918103b60 0x7f191819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59258/0 (socket says 172.21.15.3:59258) 2024-09-21T06:14:10.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191d3f3640 1 -- 172.21.15.3:0/4084661295 learned_addr learned my addr 172.21.15.3:0/4084661295 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:10.245 INFO:teuthology.orchestra.run.smithi003.stdout:111669149762 2024-09-21T06:14:10.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191dbf4640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1918109df0 0x7f191819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:10.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191cbf2640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19181094a0 0x7f191819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:10.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191d3f3640 1 -- 172.21.15.3:0/4084661295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19181094a0 msgr2=0x7f191819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:10.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191d3f3640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19181094a0 0x7f191819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:10.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191d3f3640 1 -- 172.21.15.3:0/4084661295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1918109df0 msgr2=0x7f191819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:10.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191d3f3640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1918109df0 0x7f191819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:10.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.706+0000 7f191d3f3640 1 -- 172.21.15.3:0/4084661295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19181a0360 con 0x7f1918103b60 2024-09-21T06:14:10.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.707+0000 7f191d3f3640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1918103b60 0x7f191819a4b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1914005ab0 tx=0x7f1914003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:10.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.707+0000 7f190e7fc640 1 -- 172.21.15.3:0/4084661295 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191403f070 con 0x7f1918103b60 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.707+0000 7f190e7fc640 1 -- 172.21.15.3:0/4084661295 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1914031d60 con 0x7f1918103b60 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.707+0000 7f191f657640 1 -- 172.21.15.3:0/4084661295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19181a05f0 con 0x7f1918103b60 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.707+0000 7f190e7fc640 1 -- 172.21.15.3:0/4084661295 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914035030 con 0x7f1918103b60 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.707+0000 7f191f657640 1 -- 172.21.15.3:0/4084661295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19181a0ad0 con 0x7f1918103b60 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.708+0000 7f191f657640 1 -- 172.21.15.3:0/4084661295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1918103f70 con 0x7f1918103b60 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.709+0000 7f190e7fc640 1 -- 172.21.15.3:0/4084661295 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f1914042070 con 0x7f1918103b60 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.710+0000 7f190e7fc640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f19040b6620 0x7f19040b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.710+0000 7f191cbf2640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f19040b6620 0x7f19040b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.710+0000 7f190e7fc640 1 -- 172.21.15.3:0/4084661295 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f1914108400 con 0x7f1918103b60 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.711+0000 7f191cbf2640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f19040b6620 0x7f19040b8b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1918109200 tx=0x7f190800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.713+0000 7f190e7fc640 1 -- 172.21.15.3:0/4084661295 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f19140d2e30 con 0x7f1918103b60 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.920+0000 7f191f657640 1 -- 172.21.15.3:0/4084661295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f191819b5e0 con 0x7f1918103b60 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.920+0000 7f190e7fc640 1 -- 172.21.15.3:0/4084661295 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f19140d7030 con 0x7f1918103b60 2024-09-21T06:14:10.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.923+0000 7f191f657640 1 -- 172.21.15.3:0/4084661295 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f19040b6620 msgr2=0x7f19040b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:10.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.923+0000 7f191f657640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f19040b6620 0x7f19040b8b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1918109200 tx=0x7f190800b390 comp rx=0 tx=0).stop 2024-09-21T06:14:10.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.923+0000 7f191f657640 1 -- 172.21.15.3:0/4084661295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1918103b60 msgr2=0x7f191819a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:10.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.923+0000 7f191f657640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1918103b60 0x7f191819a4b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1914005ab0 tx=0x7f1914003cd0 comp rx=0 tx=0).stop 2024-09-21T06:14:10.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.923+0000 7f191f657640 1 -- 172.21.15.3:0/4084661295 shutdown_connections 2024-09-21T06:14:10.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.923+0000 7f191f657640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f19040b6620 0x7f19040b8b30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:10.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.923+0000 7f191f657640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19181094a0 0x7f191819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:10.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.923+0000 7f191f657640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1918109df0 0x7f191819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:10.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.923+0000 7f191f657640 1 --2- 172.21.15.3:0/4084661295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1918103b60 0x7f191819a4b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:10.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.923+0000 7f191f657640 1 -- 172.21.15.3:0/4084661295 >> 172.21.15.3:0/4084661295 conn(0x7f191807b850 msgr2=0x7f1918107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:10.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.923+0000 7f191f657640 1 -- 172.21.15.3:0/4084661295 shutdown_connections 2024-09-21T06:14:10.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:07.923+0000 7f191f657640 1 -- 172.21.15.3:0/4084661295 wait complete. 2024-09-21T06:14:10.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:10 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1502541550' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T06:14:10.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:10 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1502541550' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T06:14:10.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:10 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1502541550' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T06:14:10.867 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149759 got 111669149762 for osd.3 2024-09-21T06:14:10.868 DEBUG:teuthology.parallel:result is None 2024-09-21T06:14:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.058+0000 7fd95def5640 1 -- 172.21.15.3:0/1675933775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd958107be0 msgr2=0x7fd9581080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.058+0000 7fd95def5640 1 --2- 172.21.15.3:0/1675933775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd958107be0 0x7fd9581080c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd950009cd0 tx=0x7fd95002a8e0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.058+0000 7fd95def5640 1 -- 172.21.15.3:0/1675933775 shutdown_connections 2024-09-21T06:14:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.058+0000 7fd95def5640 1 --2- 172.21.15.3:0/1675933775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd958107be0 0x7fd9581080c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.058+0000 7fd95def5640 1 --2- 172.21.15.3:0/1675933775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9581030f0 0x7fd958107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.058+0000 7fd95def5640 1 --2- 172.21.15.3:0/1675933775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9581027a0 0x7fd958102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.058+0000 7fd95def5640 1 -- 172.21.15.3:0/1675933775 >> 172.21.15.3:0/1675933775 conn(0x7fd958079730 msgr2=0x7fd958079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.058+0000 7fd95def5640 1 -- 172.21.15.3:0/1675933775 shutdown_connections 2024-09-21T06:14:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.058+0000 7fd95def5640 1 -- 172.21.15.3:0/1675933775 wait complete. 2024-09-21T06:14:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.059+0000 7fd95def5640 1 Processor -- start 2024-09-21T06:14:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd95def5640 1 -- start start 2024-09-21T06:14:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd95def5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9581027a0 0x7fd9581a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd95def5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9581030f0 0x7fd9581a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd95def5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd958107be0 0x7fd9581a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd95def5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd95811a0d0 con 0x7fd9581030f0 2024-09-21T06:14:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd95def5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd958119f50 con 0x7fd958107be0 2024-09-21T06:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd95def5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd95811a250 con 0x7fd9581027a0 2024-09-21T06:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd956ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9581030f0 0x7fd9581a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd956ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9581030f0 0x7fd9581a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59294/0 (socket says 172.21.15.3:59294) 2024-09-21T06:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd956ffd640 1 -- 172.21.15.3:0/1508249687 learned_addr learned my addr 172.21.15.3:0/1508249687 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd957fff640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd958107be0 0x7fd9581a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd9577fe640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9581027a0 0x7fd9581a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd956ffd640 1 -- 172.21.15.3:0/1508249687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9581027a0 msgr2=0x7fd9581a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd956ffd640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9581027a0 0x7fd9581a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd956ffd640 1 -- 172.21.15.3:0/1508249687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd958107be0 msgr2=0x7fd9581a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd956ffd640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd958107be0 0x7fd9581a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.060+0000 7fd956ffd640 1 -- 172.21.15.3:0/1508249687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9581a6a10 con 0x7fd9581030f0 2024-09-21T06:14:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.061+0000 7fd956ffd640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9581030f0 0x7fd9581a1040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00d740 tx=0x7fd94c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.061+0000 7fd954ff9640 1 -- 172.21.15.3:0/1508249687 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c017070 con 0x7fd9581030f0 2024-09-21T06:14:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.061+0000 7fd954ff9640 1 -- 172.21.15.3:0/1508249687 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd94c00de50 con 0x7fd9581030f0 2024-09-21T06:14:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.061+0000 7fd95def5640 1 -- 172.21.15.3:0/1508249687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9581a6d00 con 0x7fd9581030f0 2024-09-21T06:14:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.061+0000 7fd954ff9640 1 -- 172.21.15.3:0/1508249687 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00f620 con 0x7fd9581030f0 2024-09-21T06:14:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.061+0000 7fd95def5640 1 -- 172.21.15.3:0/1508249687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9581ad460 con 0x7fd9581030f0 2024-09-21T06:14:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.062+0000 7fd95def5640 1 -- 172.21.15.3:0/1508249687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd958107620 con 0x7fd9581030f0 2024-09-21T06:14:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.063+0000 7fd954ff9640 1 -- 172.21.15.3:0/1508249687 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fd94c01a070 con 0x7fd9581030f0 2024-09-21T06:14:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.064+0000 7fd954ff9640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd9400b6670 0x7fd9400b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.064+0000 7fd9577fe640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd9400b6670 0x7fd9400b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.065+0000 7fd954ff9640 1 -- 172.21.15.3:0/1508249687 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fd94c0e6f50 con 0x7fd9581030f0 2024-09-21T06:14:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.065+0000 7fd9577fe640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd9400b6670 0x7fd9400b8b80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd948005fd0 tx=0x7fd948005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.069+0000 7fd954ff9640 1 -- 172.21.15.3:0/1508249687 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd94c0ad5a0 con 0x7fd9581030f0 2024-09-21T06:14:11.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.262+0000 7fd95def5640 1 -- 172.21.15.3:0/1508249687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd9581ad740 con 0x7fd9581030f0 2024-09-21T06:14:11.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.262+0000 7fd954ff9640 1 -- 172.21.15.3:0/1508249687 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd94c0b1980 con 0x7fd9581030f0 2024-09-21T06:14:11.264 INFO:teuthology.orchestra.run.smithi003.stdout:304942678041 2024-09-21T06:14:11.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.264+0000 7fd95def5640 1 -- 172.21.15.3:0/1508249687 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd9400b6670 msgr2=0x7fd9400b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.264+0000 7fd95def5640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd9400b6670 0x7fd9400b8b80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd948005fd0 tx=0x7fd948005850 comp rx=0 tx=0).stop 2024-09-21T06:14:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.264+0000 7fd95def5640 1 -- 172.21.15.3:0/1508249687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9581030f0 msgr2=0x7fd9581a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.264+0000 7fd95def5640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9581030f0 0x7fd9581a1040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00d740 tx=0x7fd94c00dc00 comp rx=0 tx=0).stop 2024-09-21T06:14:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.265+0000 7fd95def5640 1 -- 172.21.15.3:0/1508249687 shutdown_connections 2024-09-21T06:14:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.265+0000 7fd95def5640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd9400b6670 0x7fd9400b8b80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.265+0000 7fd95def5640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd958107be0 0x7fd9581a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.265+0000 7fd95def5640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9581030f0 0x7fd9581a1040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.265+0000 7fd95def5640 1 --2- 172.21.15.3:0/1508249687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9581027a0 0x7fd9581a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.265+0000 7fd95def5640 1 -- 172.21.15.3:0/1508249687 >> 172.21.15.3:0/1508249687 conn(0x7fd958079730 msgr2=0x7fd9581055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.265+0000 7fd95def5640 1 -- 172.21.15.3:0/1508249687 shutdown_connections 2024-09-21T06:14:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.265+0000 7fd95def5640 1 -- 172.21.15.3:0/1508249687 wait complete. 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.276+0000 7f4be880d640 1 -- 172.21.15.3:0/3341298766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be41083d0 msgr2=0x7f4be41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.276+0000 7f4be880d640 1 --2- 172.21.15.3:0/3341298766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be41083d0 0x7f4be41087e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009900 tx=0x7f4bd802a410 comp rx=0 tx=0).stop 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.276+0000 7f4be880d640 1 -- 172.21.15.3:0/3341298766 shutdown_connections 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.276+0000 7f4be880d640 1 --2- 172.21.15.3:0/3341298766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4be4108d20 0x7f4be410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.276+0000 7f4be880d640 1 --2- 172.21.15.3:0/3341298766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be41083d0 0x7f4be41087e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.276+0000 7f4be880d640 1 --2- 172.21.15.3:0/3341298766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be4102ae0 0x7f4be4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.276+0000 7f4be880d640 1 -- 172.21.15.3:0/3341298766 >> 172.21.15.3:0/3341298766 conn(0x7f4be40797b0 msgr2=0x7f4be4079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.277+0000 7f4be880d640 1 -- 172.21.15.3:0/3341298766 shutdown_connections 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.277+0000 7f4be880d640 1 -- 172.21.15.3:0/3341298766 wait complete. 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.278+0000 7f4be880d640 1 Processor -- start 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.278+0000 7f4be880d640 1 -- start start 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.278+0000 7f4be880d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be4102ae0 0x7f4be419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.278+0000 7f4be880d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be41083d0 0x7f4be419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.278+0000 7f4be880d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4be4108d20 0x7f4be41a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.278+0000 7f4be880d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4be41129e0 con 0x7f4be41083d0 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.278+0000 7f4be880d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4be4112860 con 0x7f4be4102ae0 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.278+0000 7f4be880d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4be4112b60 con 0x7f4be4108d20 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.278+0000 7f4be1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be41083d0 0x7f4be419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.278+0000 7f4be1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be41083d0 0x7f4be419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59308/0 (socket says 172.21.15.3:59308) 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.278+0000 7f4be1d9b640 1 -- 172.21.15.3:0/3459329111 learned_addr learned my addr 172.21.15.3:0/3459329111 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4be259c640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be4102ae0 0x7f4be419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4be2d9d640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4be4108d20 0x7f4be41a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4be1d9b640 1 -- 172.21.15.3:0/3459329111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4be4108d20 msgr2=0x7f4be41a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4be1d9b640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4be4108d20 0x7f4be41a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4be1d9b640 1 -- 172.21.15.3:0/3459329111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be4102ae0 msgr2=0x7f4be419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4be1d9b640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be4102ae0 0x7f4be419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4be1d9b640 1 -- 172.21.15.3:0/3459329111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be41a25f0 con 0x7f4be41083d0 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4be2d9d640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4be4108d20 0x7f4be41a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4be259c640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be4102ae0 0x7f4be419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4be1d9b640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be41083d0 0x7f4be419cc20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4bd802a930 tx=0x7f4bd8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4bd37fe640 1 -- 172.21.15.3:0/3459329111 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd803f030 con 0x7f4be41083d0 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4bd37fe640 1 -- 172.21.15.3:0/3459329111 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bd8003aa0 con 0x7f4be41083d0 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4be880d640 1 -- 172.21.15.3:0/3459329111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be41a2880 con 0x7f4be41083d0 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4bd37fe640 1 -- 172.21.15.3:0/3459329111 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8035070 con 0x7f4be41083d0 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.279+0000 7f4be880d640 1 -- 172.21.15.3:0/3459329111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be41a9040 con 0x7f4be41083d0 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.281+0000 7f4be880d640 1 -- 172.21.15.3:0/3459329111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be4102ef0 con 0x7f4be41083d0 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.282+0000 7f4bd37fe640 1 -- 172.21.15.3:0/3459329111 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f4bd8042030 con 0x7f4be41083d0 2024-09-21T06:14:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.283+0000 7f4bd37fe640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4bcc0b65f0 0x7f4bcc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.284+0000 7f4be259c640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4bcc0b65f0 0x7f4bcc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.284+0000 7f4bd37fe640 1 -- 172.21.15.3:0/3459329111 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f4bd8107790 con 0x7f4be41083d0 2024-09-21T06:14:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.284+0000 7f4be259c640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4bcc0b65f0 0x7f4bcc0b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400bfd0 tx=0x7f4bd400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:11.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.288+0000 7f4bd37fe640 1 -- 172.21.15.3:0/3459329111 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4bd80cdde0 con 0x7f4be41083d0 2024-09-21T06:14:11.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.478+0000 7f4be880d640 1 -- 172.21.15.3:0/3459329111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4be41a9210 con 0x7f4be41083d0 2024-09-21T06:14:11.480 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:11 smithi003 ceph-mon[31227]: pgmap v300: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:11.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.478+0000 7f4bd37fe640 1 -- 172.21.15.3:0/3459329111 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4bd80d21c0 con 0x7f4be41083d0 2024-09-21T06:14:11.481 INFO:teuthology.orchestra.run.smithi003.stdout:81604378696 2024-09-21T06:14:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.481+0000 7f4be880d640 1 -- 172.21.15.3:0/3459329111 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4bcc0b65f0 msgr2=0x7f4bcc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.481+0000 7f4be880d640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4bcc0b65f0 0x7f4bcc0b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400bfd0 tx=0x7f4bd400b390 comp rx=0 tx=0).stop 2024-09-21T06:14:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.481+0000 7f4be880d640 1 -- 172.21.15.3:0/3459329111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be41083d0 msgr2=0x7f4be419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.481+0000 7f4be880d640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be41083d0 0x7f4be419cc20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4bd802a930 tx=0x7f4bd8004270 comp rx=0 tx=0).stop 2024-09-21T06:14:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.481+0000 7f4be880d640 1 -- 172.21.15.3:0/3459329111 shutdown_connections 2024-09-21T06:14:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.481+0000 7f4be880d640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4bcc0b65f0 0x7f4bcc0b8b00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.481+0000 7f4be880d640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4be4108d20 0x7f4be41a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.481+0000 7f4be880d640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be41083d0 0x7f4be419cc20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.481+0000 7f4be880d640 1 --2- 172.21.15.3:0/3459329111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be4102ae0 0x7f4be419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.481+0000 7f4be880d640 1 -- 172.21.15.3:0/3459329111 >> 172.21.15.3:0/3459329111 conn(0x7f4be40797b0 msgr2=0x7f4be406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.481+0000 7f4be880d640 1 -- 172.21.15.3:0/3459329111 shutdown_connections 2024-09-21T06:14:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.481+0000 7f4be880d640 1 -- 172.21.15.3:0/3459329111 wait complete. 2024-09-21T06:14:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:11 smithi137 ceph-mon[34464]: pgmap v300: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:11 smithi204 ceph-mon[34659]: pgmap v300: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:11.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.905+0000 7feeec71a640 1 -- 172.21.15.3:0/1298731401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4109560 msgr2=0x7feee4109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.905+0000 7feeec71a640 1 --2- 172.21.15.3:0/1298731401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4109560 0x7feee4109970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feed4009900 tx=0x7feed402a410 comp rx=0 tx=0).stop 2024-09-21T06:14:11.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.906+0000 7feeec71a640 1 -- 172.21.15.3:0/1298731401 shutdown_connections 2024-09-21T06:14:11.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.906+0000 7feeec71a640 1 --2- 172.21.15.3:0/1298731401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee4109eb0 0x7feee41106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.906+0000 7feeec71a640 1 --2- 172.21.15.3:0/1298731401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4109560 0x7feee4109970 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.906+0000 7feeec71a640 1 --2- 172.21.15.3:0/1298731401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feee4102080 0x7feee4102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.906+0000 7feeec71a640 1 -- 172.21.15.3:0/1298731401 >> 172.21.15.3:0/1298731401 conn(0x7feee4079ec0 msgr2=0x7feee407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:11.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.906+0000 7feeec71a640 1 -- 172.21.15.3:0/1298731401 shutdown_connections 2024-09-21T06:14:11.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.906+0000 7feeec71a640 1 -- 172.21.15.3:0/1298731401 wait complete. 2024-09-21T06:14:11.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.906+0000 7feeec71a640 1 Processor -- start 2024-09-21T06:14:11.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.906+0000 7feeec71a640 1 -- start start 2024-09-21T06:14:11.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feeec71a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feee4102080 0x7feee419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feeec71a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4109560 0x7feee419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feeec71a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee4109eb0 0x7feee419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feeec71a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feee4113b70 con 0x7feee4109560 2024-09-21T06:14:11.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feeec71a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feee41139f0 con 0x7feee4109eb0 2024-09-21T06:14:11.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feeec71a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feee4113cf0 con 0x7feee4102080 2024-09-21T06:14:11.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feee9cb5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4109560 0x7feee419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feee9cb5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4109560 0x7feee419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59334/0 (socket says 172.21.15.3:59334) 2024-09-21T06:14:11.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feee9cb5640 1 -- 172.21.15.3:0/2975882675 learned_addr learned my addr 172.21.15.3:0/2975882675 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feeea4b6640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feee4102080 0x7feee419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feeeacb7640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee4109eb0 0x7feee419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feee9cb5640 1 -- 172.21.15.3:0/2975882675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feee4102080 msgr2=0x7feee419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feee9cb5640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feee4102080 0x7feee419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feee9cb5640 1 -- 172.21.15.3:0/2975882675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee4109eb0 msgr2=0x7feee419fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feee9cb5640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee4109eb0 0x7feee419fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feee9cb5640 1 -- 172.21.15.3:0/2975882675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee41a0530 con 0x7feee4109560 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feee9cb5640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4109560 0x7feee419ab60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feed4006010 tx=0x7feed4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.907+0000 7feedb7fe640 1 -- 172.21.15.3:0/2975882675 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed403f030 con 0x7feee4109560 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.908+0000 7feeec71a640 1 -- 172.21.15.3:0/2975882675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee41a07c0 con 0x7feee4109560 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.908+0000 7feedb7fe640 1 -- 172.21.15.3:0/2975882675 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed4003aa0 con 0x7feee4109560 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.908+0000 7feedb7fe640 1 -- 172.21.15.3:0/2975882675 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4035070 con 0x7feee4109560 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.908+0000 7feeec71a640 1 -- 172.21.15.3:0/2975882675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee41a6f80 con 0x7feee4109560 2024-09-21T06:14:11.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.909+0000 7feeec71a640 1 -- 172.21.15.3:0/2975882675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee406b8f0 con 0x7feee4109560 2024-09-21T06:14:11.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.911+0000 7feedb7fe640 1 -- 172.21.15.3:0/2975882675 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7feed4042030 con 0x7feee4109560 2024-09-21T06:14:11.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.912+0000 7feedb7fe640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7feed00b65f0 0x7feed00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.912+0000 7feeea4b6640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7feed00b65f0 0x7feed00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.912+0000 7feedb7fe640 1 -- 172.21.15.3:0/2975882675 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7feed4107ed0 con 0x7feee4109560 2024-09-21T06:14:11.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.912+0000 7feeea4b6640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7feed00b65f0 0x7feed00b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feee0005fd0 tx=0x7feee000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:11.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:11.915+0000 7feedb7fe640 1 -- 172.21.15.3:0/2975882675 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feed40ce4c0 con 0x7feee4109560 2024-09-21T06:14:12.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.108+0000 7feeec71a640 1 -- 172.21.15.3:0/2975882675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7feee4102490 con 0x7feee4109560 2024-09-21T06:14:12.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.108+0000 7feedb7fe640 1 -- 172.21.15.3:0/2975882675 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feee4102490 con 0x7feee4109560 2024-09-21T06:14:12.110 INFO:teuthology.orchestra.run.smithi003.stdout:257698037796 2024-09-21T06:14:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.111+0000 7feeec71a640 1 -- 172.21.15.3:0/2975882675 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7feed00b65f0 msgr2=0x7feed00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.111+0000 7feeec71a640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7feed00b65f0 0x7feed00b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feee0005fd0 tx=0x7feee000b390 comp rx=0 tx=0).stop 2024-09-21T06:14:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.111+0000 7feeec71a640 1 -- 172.21.15.3:0/2975882675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4109560 msgr2=0x7feee419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.111+0000 7feeec71a640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4109560 0x7feee419ab60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feed4006010 tx=0x7feed4004060 comp rx=0 tx=0).stop 2024-09-21T06:14:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.111+0000 7feeec71a640 1 -- 172.21.15.3:0/2975882675 shutdown_connections 2024-09-21T06:14:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.111+0000 7feeec71a640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7feed00b65f0 0x7feed00b8b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.111+0000 7feeec71a640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee4109eb0 0x7feee419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.111+0000 7feeec71a640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4109560 0x7feee419ab60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:12.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.111+0000 7feeec71a640 1 --2- 172.21.15.3:0/2975882675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feee4102080 0x7feee419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:12.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.111+0000 7feeec71a640 1 -- 172.21.15.3:0/2975882675 >> 172.21.15.3:0/2975882675 conn(0x7feee4079ec0 msgr2=0x7feee41055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:12.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.111+0000 7feeec71a640 1 -- 172.21.15.3:0/2975882675 shutdown_connections 2024-09-21T06:14:12.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:12.111+0000 7feeec71a640 1 -- 172.21.15.3:0/2975882675 wait complete. 2024-09-21T06:14:12.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:12 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1508249687' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T06:14:12.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:12 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3459329111' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T06:14:12.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:12 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2975882675' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T06:14:12.404 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378694 got 81604378696 for osd.2 2024-09-21T06:14:12.404 DEBUG:teuthology.parallel:result is None 2024-09-21T06:14:12.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:12 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1508249687' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T06:14:12.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:12 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3459329111' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T06:14:12.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:12 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2975882675' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T06:14:12.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:12 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1508249687' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T06:14:12.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:12 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3459329111' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T06:14:12.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:12 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2975882675' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T06:14:12.898 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678038 got 304942678041 for osd.11 2024-09-21T06:14:12.898 DEBUG:teuthology.parallel:result is None 2024-09-21T06:14:13.288 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:13.515 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:13 smithi003 ceph-mon[31227]: pgmap v301: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:13.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:13 smithi137 ceph-mon[34464]: pgmap v301: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:13.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:13 smithi204 ceph-mon[34659]: pgmap v301: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:13.800 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037793 got 257698037796 for osd.9 2024-09-21T06:14:13.800 DEBUG:teuthology.parallel:result is None 2024-09-21T06:14:15.268 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:15 smithi003 ceph-mon[31227]: pgmap v302: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T06:14:15.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.266+0000 7f13251a1640 1 -- 172.21.15.3:0/15738968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201047c0 msgr2=0x7f1320104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.266+0000 7f13251a1640 1 --2- 172.21.15.3:0/15738968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201047c0 0x7f1320104bd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1310009900 tx=0x7f131002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.266+0000 7f13251a1640 1 -- 172.21.15.3:0/15738968 shutdown_connections 2024-09-21T06:14:15.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.266+0000 7f13251a1640 1 --2- 172.21.15.3:0/15738968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1320105110 0x7f132010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.266+0000 7f13251a1640 1 --2- 172.21.15.3:0/15738968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f132010dc20 0x7f1320110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.266+0000 7f13251a1640 1 --2- 172.21.15.3:0/15738968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201047c0 0x7f1320104bd0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.266+0000 7f13251a1640 1 -- 172.21.15.3:0/15738968 >> 172.21.15.3:0/15738968 conn(0x7f13200fe2c0 msgr2=0x7f1320100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.266+0000 7f13251a1640 1 -- 172.21.15.3:0/15738968 shutdown_connections 2024-09-21T06:14:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.266+0000 7f13251a1640 1 -- 172.21.15.3:0/15738968 wait complete. 2024-09-21T06:14:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.267+0000 7f13251a1640 1 Processor -- start 2024-09-21T06:14:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.267+0000 7f13251a1640 1 -- start start 2024-09-21T06:14:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f13251a1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201047c0 0x7f132019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f13251a1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1320105110 0x7f132019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f13251a1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f132010dc20 0x7f132019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f13251a1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f13201138a0 con 0x7f13201047c0 2024-09-21T06:14:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f13251a1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1320113720 con 0x7f1320105110 2024-09-21T06:14:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f13251a1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1320113a20 con 0x7f132010dc20 2024-09-21T06:14:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f131ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201047c0 0x7f132019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f131ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201047c0 0x7f132019a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50452/0 (socket says 172.21.15.3:50452) 2024-09-21T06:14:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f131ffff640 1 -- 172.21.15.3:0/2499215991 learned_addr learned my addr 172.21.15.3:0/2499215991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f13249a0640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f132010dc20 0x7f132019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f131f7fe640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1320105110 0x7f132019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f131ffff640 1 -- 172.21.15.3:0/2499215991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f132010dc20 msgr2=0x7f132019fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f131ffff640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f132010dc20 0x7f132019fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f131ffff640 1 -- 172.21.15.3:0/2499215991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1320105110 msgr2=0x7f132019a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f131ffff640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1320105110 0x7f132019a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f131ffff640 1 -- 172.21.15.3:0/2499215991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13201a01c0 con 0x7f13201047c0 2024-09-21T06:14:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.268+0000 7f131ffff640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201047c0 0x7f132019a310 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1310005940 tx=0x7f13100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.269+0000 7f131d7fa640 1 -- 172.21.15.3:0/2499215991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131003f030 con 0x7f13201047c0 2024-09-21T06:14:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.269+0000 7f131d7fa640 1 -- 172.21.15.3:0/2499215991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1310003cb0 con 0x7f13201047c0 2024-09-21T06:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.269+0000 7f13251a1640 1 -- 172.21.15.3:0/2499215991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13201a0450 con 0x7f13201047c0 2024-09-21T06:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.269+0000 7f131d7fa640 1 -- 172.21.15.3:0/2499215991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1310035070 con 0x7f13201047c0 2024-09-21T06:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.269+0000 7f13251a1640 1 -- 172.21.15.3:0/2499215991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13201a0930 con 0x7f13201047c0 2024-09-21T06:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.270+0000 7f13251a1640 1 -- 172.21.15.3:0/2499215991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1320104bd0 con 0x7f13201047c0 2024-09-21T06:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.271+0000 7f131d7fa640 1 -- 172.21.15.3:0/2499215991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f1310042030 con 0x7f13201047c0 2024-09-21T06:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.272+0000 7f131d7fa640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f13080b6620 0x7f13080b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.272+0000 7f131f7fe640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f13080b6620 0x7f13080b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.272+0000 7f131d7fa640 1 -- 172.21.15.3:0/2499215991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f1310107e90 con 0x7f13201047c0 2024-09-21T06:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.273+0000 7f131f7fe640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f13080b6620 0x7f13080b8b30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1314005fd0 tx=0x7f1314005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.275+0000 7f131d7fa640 1 -- 172.21.15.3:0/2499215991 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f13100ce4e0 con 0x7f13201047c0 2024-09-21T06:14:15.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.465+0000 7f13251a1640 1 -- 172.21.15.3:0/2499215991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f13201a6e00 con 0x7f13201047c0 2024-09-21T06:14:15.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.466+0000 7f131d7fa640 1 -- 172.21.15.3:0/2499215991 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f13100d28c0 con 0x7f13201047c0 2024-09-21T06:14:15.468 INFO:teuthology.orchestra.run.smithi003.stdout:206158430254 2024-09-21T06:14:15.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.468+0000 7f13251a1640 1 -- 172.21.15.3:0/2499215991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f13080b6620 msgr2=0x7f13080b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.468+0000 7f13251a1640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f13080b6620 0x7f13080b8b30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1314005fd0 tx=0x7f1314005850 comp rx=0 tx=0).stop 2024-09-21T06:14:15.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.468+0000 7f13251a1640 1 -- 172.21.15.3:0/2499215991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201047c0 msgr2=0x7f132019a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.468+0000 7f13251a1640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201047c0 0x7f132019a310 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1310005940 tx=0x7f13100042f0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.469+0000 7f13251a1640 1 -- 172.21.15.3:0/2499215991 shutdown_connections 2024-09-21T06:14:15.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.469+0000 7f13251a1640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f13080b6620 0x7f13080b8b30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.469+0000 7f13251a1640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f132010dc20 0x7f132019fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.469+0000 7f13251a1640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1320105110 0x7f132019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.469+0000 7f13251a1640 1 --2- 172.21.15.3:0/2499215991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201047c0 0x7f132019a310 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.469+0000 7f13251a1640 1 -- 172.21.15.3:0/2499215991 >> 172.21.15.3:0/2499215991 conn(0x7f13200fe2c0 msgr2=0x7f1320107780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:15.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.469+0000 7f13251a1640 1 -- 172.21.15.3:0/2499215991 shutdown_connections 2024-09-21T06:14:15.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.469+0000 7f13251a1640 1 -- 172.21.15.3:0/2499215991 wait complete. 2024-09-21T06:14:15.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.512+0000 7ff9065ed640 1 -- 172.21.15.3:0/3988983647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff900108420 msgr2=0x7ff900108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.512+0000 7ff9065ed640 1 --2- 172.21.15.3:0/3988983647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff900108420 0x7ff900108830 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009900 tx=0x7ff8f402a410 comp rx=0 tx=0).stop 2024-09-21T06:14:15.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.512+0000 7ff9065ed640 1 -- 172.21.15.3:0/3988983647 shutdown_connections 2024-09-21T06:14:15.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.512+0000 7ff9065ed640 1 --2- 172.21.15.3:0/3988983647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff900108d70 0x7ff90010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.512+0000 7ff9065ed640 1 --2- 172.21.15.3:0/3988983647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff900108420 0x7ff900108830 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.512+0000 7ff9065ed640 1 --2- 172.21.15.3:0/3988983647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff900102ae0 0x7ff900102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.512+0000 7ff9065ed640 1 -- 172.21.15.3:0/3988983647 >> 172.21.15.3:0/3988983647 conn(0x7ff900079770 msgr2=0x7ff900079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.513+0000 7ff9065ed640 1 -- 172.21.15.3:0/3988983647 shutdown_connections 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.513+0000 7ff9065ed640 1 -- 172.21.15.3:0/3988983647 wait complete. 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.514+0000 7ff9065ed640 1 Processor -- start 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.514+0000 7ff9065ed640 1 -- start start 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.514+0000 7ff9065ed640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff900102ae0 0x7ff90019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.514+0000 7ff9065ed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff900108420 0x7ff90019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.514+0000 7ff9065ed640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff900108d70 0x7ff90019bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.514+0000 7ff9065ed640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff900112a30 con 0x7ff900108420 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.514+0000 7ff9065ed640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff9001128b0 con 0x7ff900102ae0 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.514+0000 7ff9065ed640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff900112bb0 con 0x7ff900108d70 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.514+0000 7ff8ff7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff900108420 0x7ff90019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff8ff7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff900108420 0x7ff90019b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50458/0 (socket says 172.21.15.3:50458) 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff8ff7fe640 1 -- 172.21.15.3:0/3195614281 learned_addr learned my addr 172.21.15.3:0/3195614281 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff8fffff640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff900102ae0 0x7ff90019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff904b8a640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff900108d70 0x7ff90019bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff8ff7fe640 1 -- 172.21.15.3:0/3195614281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff900108d70 msgr2=0x7ff90019bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff8ff7fe640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff900108d70 0x7ff90019bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff8ff7fe640 1 -- 172.21.15.3:0/3195614281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff900102ae0 msgr2=0x7ff90019b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff8ff7fe640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff900102ae0 0x7ff90019b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff8ff7fe640 1 -- 172.21.15.3:0/3195614281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9001a6600 con 0x7ff900108420 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff8ff7fe640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff900108420 0x7ff90019b880 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff8f402a930 tx=0x7ff8f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff8fd7fa640 1 -- 172.21.15.3:0/3195614281 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f403f030 con 0x7ff900108420 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff8fd7fa640 1 -- 172.21.15.3:0/3195614281 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8f4003aa0 con 0x7ff900108420 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff9065ed640 1 -- 172.21.15.3:0/3195614281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9001a6890 con 0x7ff900108420 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff8fd7fa640 1 -- 172.21.15.3:0/3195614281 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4035070 con 0x7ff900108420 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.515+0000 7ff9065ed640 1 -- 172.21.15.3:0/3195614281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9001a6d70 con 0x7ff900108420 2024-09-21T06:14:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.517+0000 7ff9065ed640 1 -- 172.21.15.3:0/3195614281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff900103d60 con 0x7ff900108420 2024-09-21T06:14:15.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.520+0000 7ff8fd7fa640 1 -- 172.21.15.3:0/3195614281 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7ff8f4042030 con 0x7ff900108420 2024-09-21T06:14:15.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.521+0000 7ff8fd7fa640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff8e80b65f0 0x7ff8e80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.521+0000 7ff8fd7fa640 1 -- 172.21.15.3:0/3195614281 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff8f4108150 con 0x7ff900108420 2024-09-21T06:14:15.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.521+0000 7ff8fffff640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff8e80b65f0 0x7ff8e80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.522+0000 7ff8fffff640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff8e80b65f0 0x7ff8e80b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff90006c6e0 tx=0x7ff8f000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:15.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.524+0000 7ff8fd7fa640 1 -- 172.21.15.3:0/3195614281 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff8f40ce740 con 0x7ff900108420 2024-09-21T06:14:15.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:15 smithi137 ceph-mon[34464]: pgmap v302: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T06:14:15.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:15 smithi204 ceph-mon[34659]: pgmap v302: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T06:14:15.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.716+0000 7ff9065ed640 1 -- 172.21.15.3:0/3195614281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff900102ef0 con 0x7ff900108420 2024-09-21T06:14:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.716+0000 7ff8fd7fa640 1 -- 172.21.15.3:0/3195614281 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff900102ef0 con 0x7ff900108420 2024-09-21T06:14:15.719 INFO:teuthology.orchestra.run.smithi003.stdout:158913790010 2024-09-21T06:14:15.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.719+0000 7ff9065ed640 1 -- 172.21.15.3:0/3195614281 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff8e80b65f0 msgr2=0x7ff8e80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.719+0000 7ff9065ed640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff8e80b65f0 0x7ff8e80b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff90006c6e0 tx=0x7ff8f000b390 comp rx=0 tx=0).stop 2024-09-21T06:14:15.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.719+0000 7ff9065ed640 1 -- 172.21.15.3:0/3195614281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff900108420 msgr2=0x7ff90019b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.719+0000 7ff9065ed640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff900108420 0x7ff90019b880 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff8f402a930 tx=0x7ff8f4004270 comp rx=0 tx=0).stop 2024-09-21T06:14:15.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.719+0000 7ff9065ed640 1 -- 172.21.15.3:0/3195614281 shutdown_connections 2024-09-21T06:14:15.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.719+0000 7ff9065ed640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff8e80b65f0 0x7ff8e80b8b00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.719+0000 7ff9065ed640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff900108d70 0x7ff90019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.719+0000 7ff9065ed640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff900108420 0x7ff90019b880 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.719+0000 7ff9065ed640 1 --2- 172.21.15.3:0/3195614281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff900102ae0 0x7ff90019b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.719+0000 7ff9065ed640 1 -- 172.21.15.3:0/3195614281 >> 172.21.15.3:0/3195614281 conn(0x7ff900079770 msgr2=0x7ff9001044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:15.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.719+0000 7ff9065ed640 1 -- 172.21.15.3:0/3195614281 shutdown_connections 2024-09-21T06:14:15.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:15.720+0000 7ff9065ed640 1 -- 172.21.15.3:0/3195614281 wait complete. 2024-09-21T06:14:16.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.400+0000 7fb89ebca640 1 -- 172.21.15.3:0/2578783281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb89810de30 msgr2=0x7fb898110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.401+0000 7fb89ebca640 1 --2- 172.21.15.3:0/2578783281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb89810de30 0x7fb898110300 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb890009ca0 tx=0x7fb89002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.401+0000 7fb89ebca640 1 -- 172.21.15.3:0/2578783281 shutdown_connections 2024-09-21T06:14:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.401+0000 7fb89ebca640 1 --2- 172.21.15.3:0/2578783281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8981052b0 0x7fb89810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.401+0000 7fb89ebca640 1 --2- 172.21.15.3:0/2578783281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb89810de30 0x7fb898110300 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.401+0000 7fb89ebca640 1 --2- 172.21.15.3:0/2578783281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb898104880 0x7fb898104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.401+0000 7fb89ebca640 1 -- 172.21.15.3:0/2578783281 >> 172.21.15.3:0/2578783281 conn(0x7fb898079730 msgr2=0x7fb898079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.401+0000 7fb89ebca640 1 -- 172.21.15.3:0/2578783281 shutdown_connections 2024-09-21T06:14:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.401+0000 7fb89ebca640 1 -- 172.21.15.3:0/2578783281 wait complete. 2024-09-21T06:14:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.401+0000 7fb89ebca640 1 Processor -- start 2024-09-21T06:14:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.401+0000 7fb89ebca640 1 -- start start 2024-09-21T06:14:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89ebca640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb898104880 0x7fb898102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89ebca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8981052b0 0x7fb898102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89ebca640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb89810de30 0x7fb898103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89ebca640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb898113a70 con 0x7fb898104880 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89ebca640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981138f0 con 0x7fb8981052b0 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89ebca640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb898113bf0 con 0x7fb89810de30 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89c966640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb898104880 0x7fb898102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89c966640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb898104880 0x7fb898102790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50466/0 (socket says 172.21.15.3:50466) 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89c966640 1 -- 172.21.15.3:0/3087061903 learned_addr learned my addr 172.21.15.3:0/3087061903 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb88ffff640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8981052b0 0x7fb898102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89d167640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb89810de30 0x7fb898103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89c966640 1 -- 172.21.15.3:0/3087061903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb89810de30 msgr2=0x7fb898103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89c966640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb89810de30 0x7fb898103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89c966640 1 -- 172.21.15.3:0/3087061903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8981052b0 msgr2=0x7fb898102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89c966640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8981052b0 0x7fb898102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89c966640 1 -- 172.21.15.3:0/3087061903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8981a85f0 con 0x7fb898104880 2024-09-21T06:14:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.402+0000 7fb89d167640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb89810de30 0x7fb898103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.403+0000 7fb89c966640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb898104880 0x7fb898102790 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb89400d770 tx=0x7fb89400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.403+0000 7fb88dffb640 1 -- 172.21.15.3:0/3087061903 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894017070 con 0x7fb898104880 2024-09-21T06:14:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.403+0000 7fb88dffb640 1 -- 172.21.15.3:0/3087061903 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb894009cd0 con 0x7fb898104880 2024-09-21T06:14:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.403+0000 7fb88dffb640 1 -- 172.21.15.3:0/3087061903 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894003c60 con 0x7fb898104880 2024-09-21T06:14:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.403+0000 7fb89ebca640 1 -- 172.21.15.3:0/3087061903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8981a88e0 con 0x7fb898104880 2024-09-21T06:14:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.403+0000 7fb89ebca640 1 -- 172.21.15.3:0/3087061903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8981a8dd0 con 0x7fb898104880 2024-09-21T06:14:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.404+0000 7fb89ebca640 1 -- 172.21.15.3:0/3087061903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb898104c90 con 0x7fb898104880 2024-09-21T06:14:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.406+0000 7fb88dffb640 1 -- 172.21.15.3:0/3087061903 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fb894004340 con 0x7fb898104880 2024-09-21T06:14:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.407+0000 7fb88dffb640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb8800b65f0 0x7fb8800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.407+0000 7fb88ffff640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb8800b65f0 0x7fb8800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.407+0000 7fb88dffb640 1 -- 172.21.15.3:0/3087061903 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb8940e73c0 con 0x7fb898104880 2024-09-21T06:14:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.407+0000 7fb88ffff640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb8800b65f0 0x7fb8800b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb898100110 tx=0x7fb88800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.410+0000 7fb88dffb640 1 -- 172.21.15.3:0/3087061903 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8940ada10 con 0x7fb898104880 2024-09-21T06:14:16.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:16 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2499215991' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T06:14:16.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:16 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3195614281' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T06:14:16.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.601+0000 7fb89ebca640 1 -- 172.21.15.3:0/3087061903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb8981a9260 con 0x7fb898104880 2024-09-21T06:14:16.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.602+0000 7fb88dffb640 1 -- 172.21.15.3:0/3087061903 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb8940b1df0 con 0x7fb898104880 2024-09-21T06:14:16.604 INFO:teuthology.orchestra.run.smithi003.stdout:231928234026 2024-09-21T06:14:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.604+0000 7fb89ebca640 1 -- 172.21.15.3:0/3087061903 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb8800b65f0 msgr2=0x7fb8800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.604+0000 7fb89ebca640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb8800b65f0 0x7fb8800b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb898100110 tx=0x7fb88800a2d0 comp rx=0 tx=0).stop 2024-09-21T06:14:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.604+0000 7fb89ebca640 1 -- 172.21.15.3:0/3087061903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb898104880 msgr2=0x7fb898102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.604+0000 7fb89ebca640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb898104880 0x7fb898102790 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb89400d770 tx=0x7fb89400db70 comp rx=0 tx=0).stop 2024-09-21T06:14:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.604+0000 7fb89ebca640 1 -- 172.21.15.3:0/3087061903 shutdown_connections 2024-09-21T06:14:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.604+0000 7fb89ebca640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb8800b65f0 0x7fb8800b8b00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.604+0000 7fb89ebca640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb89810de30 0x7fb898103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.604+0000 7fb89ebca640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8981052b0 0x7fb898102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.604+0000 7fb89ebca640 1 --2- 172.21.15.3:0/3087061903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb898104880 0x7fb898102790 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.604+0000 7fb89ebca640 1 -- 172.21.15.3:0/3087061903 >> 172.21.15.3:0/3087061903 conn(0x7fb898079730 msgr2=0x7fb898107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.604+0000 7fb89ebca640 1 -- 172.21.15.3:0/3087061903 shutdown_connections 2024-09-21T06:14:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.604+0000 7fb89ebca640 1 -- 172.21.15.3:0/3087061903 wait complete. 2024-09-21T06:14:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:16 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2499215991' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T06:14:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:16 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3195614281' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T06:14:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:16 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2499215991' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T06:14:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:16 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3195614281' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T06:14:17.307 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:17 smithi003 ceph-mon[31227]: pgmap v303: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:17.307 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:17 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3087061903' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T06:14:17.308 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:17 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/4279724304' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T06:14:17.337 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430251 got 206158430254 for osd.7 2024-09-21T06:14:17.338 DEBUG:teuthology.parallel:result is None 2024-09-21T06:14:17.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.761+0000 7fe9c89e0640 1 -- 172.21.15.3:0/2453067513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c4103b90 msgr2=0x7fe9c4103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:17.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.761+0000 7fe9c89e0640 1 --2- 172.21.15.3:0/2453067513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c4103b90 0x7fe9c4103fa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009900 tx=0x7fe9b402a410 comp rx=0 tx=0).stop 2024-09-21T06:14:17.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.761+0000 7fe9c89e0640 1 -- 172.21.15.3:0/2453067513 shutdown_connections 2024-09-21T06:14:17.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.761+0000 7fe9c89e0640 1 --2- 172.21.15.3:0/2453067513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9c41094d0 0x7fe9c41098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.761+0000 7fe9c89e0640 1 --2- 172.21.15.3:0/2453067513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9c4109e20 0x7fe9c4110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.761+0000 7fe9c89e0640 1 --2- 172.21.15.3:0/2453067513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c4103b90 0x7fe9c4103fa0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.761+0000 7fe9c89e0640 1 -- 172.21.15.3:0/2453067513 >> 172.21.15.3:0/2453067513 conn(0x7fe9c40ff5e0 msgr2=0x7fe9c4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.761+0000 7fe9c89e0640 1 -- 172.21.15.3:0/2453067513 shutdown_connections 2024-09-21T06:14:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.761+0000 7fe9c89e0640 1 -- 172.21.15.3:0/2453067513 wait complete. 2024-09-21T06:14:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.762+0000 7fe9c89e0640 1 Processor -- start 2024-09-21T06:14:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.762+0000 7fe9c89e0640 1 -- start start 2024-09-21T06:14:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.762+0000 7fe9c89e0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9c4103b90 0x7fe9c419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.762+0000 7fe9c89e0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9c41094d0 0x7fe9c419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.762+0000 7fe9c89e0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c4109e20 0x7fe9c419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.762+0000 7fe9c89e0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c4113ae0 con 0x7fe9c4109e20 2024-09-21T06:14:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.762+0000 7fe9c89e0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c4113960 con 0x7fe9c4103b90 2024-09-21T06:14:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.762+0000 7fe9c89e0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c4113c60 con 0x7fe9c41094d0 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.762+0000 7fe9c2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c4109e20 0x7fe9c419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.762+0000 7fe9c2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c4109e20 0x7fe9c419fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50478/0 (socket says 172.21.15.3:50478) 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.762+0000 7fe9c2d9d640 1 -- 172.21.15.3:0/4279724304 learned_addr learned my addr 172.21.15.3:0/4279724304 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9c1d9b640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9c41094d0 0x7fe9c419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9c259c640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9c4103b90 0x7fe9c419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9c2d9d640 1 -- 172.21.15.3:0/4279724304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9c41094d0 msgr2=0x7fe9c419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9c2d9d640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9c41094d0 0x7fe9c419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9c2d9d640 1 -- 172.21.15.3:0/4279724304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9c4103b90 msgr2=0x7fe9c419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9c2d9d640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9c4103b90 0x7fe9c419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9c2d9d640 1 -- 172.21.15.3:0/4279724304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c41a0420 con 0x7fe9c4109e20 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9c259c640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9c4103b90 0x7fe9c419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9c2d9d640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c4109e20 0x7fe9c419fcf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00eaa0 tx=0x7fe9bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9b37fe640 1 -- 172.21.15.3:0/4279724304 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc00cdb0 con 0x7fe9c4109e20 2024-09-21T06:14:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9b37fe640 1 -- 172.21.15.3:0/4279724304 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9bc005020 con 0x7fe9c4109e20 2024-09-21T06:14:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9c89e0640 1 -- 172.21.15.3:0/4279724304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c41a0710 con 0x7fe9c4109e20 2024-09-21T06:14:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9b37fe640 1 -- 172.21.15.3:0/4279724304 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0116d0 con 0x7fe9c4109e20 2024-09-21T06:14:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.763+0000 7fe9c89e0640 1 -- 172.21.15.3:0/4279724304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c41a6e70 con 0x7fe9c4109e20 2024-09-21T06:14:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.765+0000 7fe9c89e0640 1 -- 172.21.15.3:0/4279724304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c4103fa0 con 0x7fe9c4109e20 2024-09-21T06:14:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.766+0000 7fe9b37fe640 1 -- 172.21.15.3:0/4279724304 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fe9bc0051d0 con 0x7fe9c4109e20 2024-09-21T06:14:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.767+0000 7fe9b37fe640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe9ac0b65f0 0x7fe9ac0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.767+0000 7fe9c259c640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe9ac0b65f0 0x7fe9ac0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.767+0000 7fe9b37fe640 1 -- 172.21.15.3:0/4279724304 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe9bc0e77e0 con 0x7fe9c4109e20 2024-09-21T06:14:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.767+0000 7fe9c259c640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe9ac0b65f0 0x7fe9ac0b8b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4031440 tx=0x7fe9b4009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.771+0000 7fe9b37fe640 1 -- 172.21.15.3:0/4279724304 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe9bc016030 con 0x7fe9c4109e20 2024-09-21T06:14:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.969+0000 7fe9c89e0640 1 -- 172.21.15.3:0/4279724304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe9c410b660 con 0x7fe9c4109e20 2024-09-21T06:14:17.609 INFO:teuthology.orchestra.run.smithi003.stdout:60129542223 2024-09-21T06:14:17.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.969+0000 7fe9b37fe640 1 -- 172.21.15.3:0/4279724304 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe9bc0ade30 con 0x7fe9c4109e20 2024-09-21T06:14:17.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.972+0000 7fe9c89e0640 1 -- 172.21.15.3:0/4279724304 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe9ac0b65f0 msgr2=0x7fe9ac0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:17.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.972+0000 7fe9c89e0640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe9ac0b65f0 0x7fe9ac0b8b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4031440 tx=0x7fe9b4009680 comp rx=0 tx=0).stop 2024-09-21T06:14:17.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.972+0000 7fe9c89e0640 1 -- 172.21.15.3:0/4279724304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c4109e20 msgr2=0x7fe9c419fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:17.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.972+0000 7fe9c89e0640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c4109e20 0x7fe9c419fcf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00eaa0 tx=0x7fe9bc00ef60 comp rx=0 tx=0).stop 2024-09-21T06:14:17.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.972+0000 7fe9c89e0640 1 -- 172.21.15.3:0/4279724304 shutdown_connections 2024-09-21T06:14:17.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.972+0000 7fe9c89e0640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe9ac0b65f0 0x7fe9ac0b8b00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:17.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.972+0000 7fe9c89e0640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9c41094d0 0x7fe9c419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:17.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.972+0000 7fe9c89e0640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c4109e20 0x7fe9c419fcf0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:17.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.972+0000 7fe9c89e0640 1 --2- 172.21.15.3:0/4279724304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9c4103b90 0x7fe9c419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:17.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.972+0000 7fe9c89e0640 1 -- 172.21.15.3:0/4279724304 >> 172.21.15.3:0/4279724304 conn(0x7fe9c40ff5e0 msgr2=0x7fe9c4101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:17.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.972+0000 7fe9c89e0640 1 -- 172.21.15.3:0/4279724304 shutdown_connections 2024-09-21T06:14:17.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:16.972+0000 7fe9c89e0640 1 -- 172.21.15.3:0/4279724304 wait complete. 2024-09-21T06:14:17.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:17 smithi137 ceph-mon[34464]: pgmap v303: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:17.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:17 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3087061903' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T06:14:17.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:17 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/4279724304' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T06:14:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:17 smithi204 ceph-mon[34659]: pgmap v303: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:17 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3087061903' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T06:14:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:17 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/4279724304' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T06:14:18.284 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542220 got 60129542223 for osd.1 2024-09-21T06:14:18.284 DEBUG:teuthology.parallel:result is None 2024-09-21T06:14:18.802 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234024 got 231928234026 for osd.8 2024-09-21T06:14:18.802 DEBUG:teuthology.parallel:result is None 2024-09-21T06:14:19.250 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790007 got 158913790010 for osd.5 2024-09-21T06:14:19.250 DEBUG:teuthology.parallel:result is None 2024-09-21T06:14:19.250 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T06:14:19.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:14:19.547 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:19.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:19 smithi003 ceph-mon[31227]: pgmap v304: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:19.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:19 smithi137 ceph-mon[34464]: pgmap v304: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:19 smithi204 ceph-mon[34659]: pgmap v304: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.051+0000 7f50e6e68640 1 -- 172.21.15.3:0/1222597616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50e0105330 msgr2=0x7f50e010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.051+0000 7f50e6e68640 1 --2- 172.21.15.3:0/1222597616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50e0105330 0x7f50e010d880 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009ca0 tx=0x7f50d002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.051+0000 7f50e6e68640 1 -- 172.21.15.3:0/1222597616 shutdown_connections 2024-09-21T06:14:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.051+0000 7f50e6e68640 1 --2- 172.21.15.3:0/1222597616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e010de40 0x7f50e0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.051+0000 7f50e6e68640 1 --2- 172.21.15.3:0/1222597616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50e0105330 0x7f50e010d880 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.051+0000 7f50e6e68640 1 --2- 172.21.15.3:0/1222597616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50e01049e0 0x7f50e0104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.051+0000 7f50e6e68640 1 -- 172.21.15.3:0/1222597616 >> 172.21.15.3:0/1222597616 conn(0x7f50e0079750 msgr2=0x7f50e0079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.051+0000 7f50e6e68640 1 -- 172.21.15.3:0/1222597616 shutdown_connections 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.051+0000 7f50e6e68640 1 -- 172.21.15.3:0/1222597616 wait complete. 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.052+0000 7f50e6e68640 1 Processor -- start 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.052+0000 7f50e6e68640 1 -- start start 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.052+0000 7f50e6e68640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e01049e0 0x7f50e019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.052+0000 7f50e6e68640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50e0105330 0x7f50e019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.052+0000 7f50e6e68640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50e010de40 0x7f50e019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.052+0000 7f50e6e68640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f50e0113aa0 con 0x7f50e0105330 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.052+0000 7f50e6e68640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f50e0113920 con 0x7f50e010de40 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.052+0000 7f50e6e68640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50e0113c20 con 0x7f50e01049e0 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.052+0000 7f50dffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50e0105330 0x7f50e019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.052+0000 7f50dffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50e0105330 0x7f50e019a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50494/0 (socket says 172.21.15.3:50494) 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.052+0000 7f50dffff640 1 -- 172.21.15.3:0/2761113242 learned_addr learned my addr 172.21.15.3:0/2761113242 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50e4c04640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e01049e0 0x7f50e019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50e5405640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50e010de40 0x7f50e019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50dffff640 1 -- 172.21.15.3:0/2761113242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e01049e0 msgr2=0x7f50e019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50dffff640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e01049e0 0x7f50e019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50dffff640 1 -- 172.21.15.3:0/2761113242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50e010de40 msgr2=0x7f50e019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50dffff640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50e010de40 0x7f50e019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50dffff640 1 -- 172.21.15.3:0/2761113242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e01a02c0 con 0x7f50e0105330 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50e5405640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50e010de40 0x7f50e019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50dffff640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50e0105330 0x7f50e019a9b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f50d002ade0 tx=0x7f50d0031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50ddffb640 1 -- 172.21.15.3:0/2761113242 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0003d20 con 0x7f50e0105330 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50e6e68640 1 -- 172.21.15.3:0/2761113242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e01a0550 con 0x7f50e0105330 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50ddffb640 1 -- 172.21.15.3:0/2761113242 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d00044f0 con 0x7f50e0105330 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50ddffb640 1 -- 172.21.15.3:0/2761113242 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d002fd40 con 0x7f50e0105330 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.053+0000 7f50e6e68640 1 -- 172.21.15.3:0/2761113242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e01a0a90 con 0x7f50e0105330 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.054+0000 7f50e6e68640 1 -- 172.21.15.3:0/2761113242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50e006c280 con 0x7f50e0105330 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.057+0000 7f50ddffb640 1 -- 172.21.15.3:0/2761113242 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f50d0004040 con 0x7f50e0105330 2024-09-21T06:14:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.058+0000 7f50ddffb640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50cc0b6620 0x7f50cc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:20.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.058+0000 7f50e4c04640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50cc0b6620 0x7f50cc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:20.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.058+0000 7f50ddffb640 1 -- 172.21.15.3:0/2761113242 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f50d0108bd0 con 0x7f50e0105330 2024-09-21T06:14:20.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.058+0000 7f50e4c04640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50cc0b6620 0x7f50cc0b8b30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f50e006c980 tx=0x7f50c8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:20.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.061+0000 7f50ddffb640 1 -- 172.21.15.3:0/2761113242 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50d00cf1c0 con 0x7f50e0105330 2024-09-21T06:14:20.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.249+0000 7f50e6e68640 1 -- 172.21.15.3:0/2761113242 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f50e0101d40 con 0x7f50cc0b6620 2024-09-21T06:14:20.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.253+0000 7f50ddffb640 1 -- 172.21.15.3:0/2761113242 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259020 (secure 0 0 0) 0x7f50e0101d40 con 0x7f50cc0b6620 2024-09-21T06:14:20.257 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:20.258 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-21T06:14:20.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.256+0000 7f50e6e68640 1 -- 172.21.15.3:0/2761113242 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50cc0b6620 msgr2=0x7f50cc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:20.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.256+0000 7f50e6e68640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50cc0b6620 0x7f50cc0b8b30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f50e006c980 tx=0x7f50c8005f90 comp rx=0 tx=0).stop 2024-09-21T06:14:20.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.256+0000 7f50e6e68640 1 -- 172.21.15.3:0/2761113242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50e0105330 msgr2=0x7f50e019a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:20.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.256+0000 7f50e6e68640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50e0105330 0x7f50e019a9b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f50d002ade0 tx=0x7f50d0031a10 comp rx=0 tx=0).stop 2024-09-21T06:14:20.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.257+0000 7f50e6e68640 1 -- 172.21.15.3:0/2761113242 shutdown_connections 2024-09-21T06:14:20.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.257+0000 7f50e6e68640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50cc0b6620 0x7f50cc0b8b30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:20.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.257+0000 7f50e6e68640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50e010de40 0x7f50e019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:20.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.257+0000 7f50e6e68640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50e0105330 0x7f50e019a9b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:20.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.257+0000 7f50e6e68640 1 --2- 172.21.15.3:0/2761113242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e01049e0 0x7f50e019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:20.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.257+0000 7f50e6e68640 1 -- 172.21.15.3:0/2761113242 >> 172.21.15.3:0/2761113242 conn(0x7f50e0079750 msgr2=0x7f50e0100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:20.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.257+0000 7f50e6e68640 1 -- 172.21.15.3:0/2761113242 shutdown_connections 2024-09-21T06:14:20.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:20.257+0000 7f50e6e68640 1 -- 172.21.15.3:0/2761113242 wait complete. 2024-09-21T06:14:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:20 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:20 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:20 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:14:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:20 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:14:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:20 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:20.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:20 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:20.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:20 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:20.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:20 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:14:20.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:20 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:14:20.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:20 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:20 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:20 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:20 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:14:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:20 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:14:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:20 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:14:20.987 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-21T06:14:19.635406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664163,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":653,"num_read_kb":434,"num_write":372,"num_write_kb":2295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2953564,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359976,"kb_used_data":10716,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581848,"statfs":{"total":1151940427776,"available":1151571812352,"internally_reserved":0,"allocated":10973184,"data_stored":7103769,"data_compressed":177687,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003531"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607638+0000","last_change":"2024-09-21T06:13:10.515115+0000","last_active":"2024-09-21T06:13:24.607638+0000","last_peered":"2024-09-21T06:13:24.607638+0000","last_clean":"2024-09-21T06:13:24.607638+0000","last_became_active":"2024-09-21T06:13:10.514553+0000","last_became_peered":"2024-09-21T06:13:10.514553+0000","last_unstale":"2024-09-21T06:13:24.607638+0000","last_undegraded":"2024-09-21T06:13:24.607638+0000","last_fullsized":"2024-09-21T06:13:24.607638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:33.268926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609526+0000","last_change":"2024-09-21T06:13:10.511469+0000","last_active":"2024-09-21T06:13:24.609526+0000","last_peered":"2024-09-21T06:13:24.609526+0000","last_clean":"2024-09-21T06:13:24.609526+0000","last_became_active":"2024-09-21T06:13:10.511102+0000","last_became_peered":"2024-09-21T06:13:10.511102+0000","last_unstale":"2024-09-21T06:13:24.609526+0000","last_undegraded":"2024-09-21T06:13:24.609526+0000","last_fullsized":"2024-09-21T06:13:24.609526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:12.483821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609288+0000","last_change":"2024-09-21T06:13:10.520090+0000","last_active":"2024-09-21T06:13:24.609288+0000","last_peered":"2024-09-21T06:13:24.609288+0000","last_clean":"2024-09-21T06:13:24.609288+0000","last_became_active":"2024-09-21T06:13:10.519758+0000","last_became_peered":"2024-09-21T06:13:10.519758+0000","last_unstale":"2024-09-21T06:13:24.609288+0000","last_undegraded":"2024-09-21T06:13:24.609288+0000","last_fullsized":"2024-09-21T06:13:24.609288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:20.836847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.617812+0000","last_change":"2024-09-21T06:13:10.511732+0000","last_active":"2024-09-21T06:13:24.617812+0000","last_peered":"2024-09-21T06:13:24.617812+0000","last_clean":"2024-09-21T06:13:24.617812+0000","last_became_active":"2024-09-21T06:13:10.511377+0000","last_became_peered":"2024-09-21T06:13:10.511377+0000","last_unstale":"2024-09-21T06:13:24.617812+0000","last_undegraded":"2024-09-21T06:13:24.617812+0000","last_fullsized":"2024-09-21T06:13:24.617812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:18.500739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607690+0000","last_change":"2024-09-21T06:13:10.510464+0000","last_active":"2024-09-21T06:13:24.607690+0000","last_peered":"2024-09-21T06:13:24.607690+0000","last_clean":"2024-09-21T06:13:24.607690+0000","last_became_active":"2024-09-21T06:13:10.509979+0000","last_became_peered":"2024-09-21T06:13:10.509979+0000","last_unstale":"2024-09-21T06:13:24.607690+0000","last_undegraded":"2024-09-21T06:13:24.607690+0000","last_fullsized":"2024-09-21T06:13:24.607690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:56.871043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608937+0000","last_change":"2024-09-21T06:13:24.608832+0000","last_active":"2024-09-21T06:13:24.608937+0000","last_peered":"2024-09-21T06:13:24.608937+0000","last_clean":"2024-09-21T06:13:24.608937+0000","last_became_active":"2024-09-21T06:13:24.608439+0000","last_became_peered":"2024-09-21T06:13:24.608439+0000","last_unstale":"2024-09-21T06:13:24.608937+0000","last_undegraded":"2024-09-21T06:13:24.608937+0000","last_fullsized":"2024-09-21T06:13:24.608937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:41.515894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609585+0000","last_change":"2024-09-21T06:13:10.511391+0000","last_active":"2024-09-21T06:13:24.609585+0000","last_peered":"2024-09-21T06:13:24.609585+0000","last_clean":"2024-09-21T06:13:24.609585+0000","last_became_active":"2024-09-21T06:13:10.510733+0000","last_became_peered":"2024-09-21T06:13:10.510733+0000","last_unstale":"2024-09-21T06:13:24.609585+0000","last_undegraded":"2024-09-21T06:13:24.609585+0000","last_fullsized":"2024-09-21T06:13:24.609585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:24.071860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609673+0000","last_change":"2024-09-21T06:13:10.510886+0000","last_active":"2024-09-21T06:13:24.609673+0000","last_peered":"2024-09-21T06:13:24.609673+0000","last_clean":"2024-09-21T06:13:24.609673+0000","last_became_active":"2024-09-21T06:13:10.510159+0000","last_became_peered":"2024-09-21T06:13:10.510159+0000","last_unstale":"2024-09-21T06:13:24.609673+0000","last_undegraded":"2024-09-21T06:13:24.609673+0000","last_fullsized":"2024-09-21T06:13:24.609673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:37.124848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598718+0000","last_change":"2024-09-21T06:13:10.510466+0000","last_active":"2024-09-21T06:13:24.598718+0000","last_peered":"2024-09-21T06:13:24.598718+0000","last_clean":"2024-09-21T06:13:24.598718+0000","last_became_active":"2024-09-21T06:13:10.509928+0000","last_became_peered":"2024-09-21T06:13:10.509928+0000","last_unstale":"2024-09-21T06:13:24.598718+0000","last_undegraded":"2024-09-21T06:13:24.598718+0000","last_fullsized":"2024-09-21T06:13:24.598718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:06.156505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600080+0000","last_change":"2024-09-21T06:13:10.510622+0000","last_active":"2024-09-21T06:13:24.600080+0000","last_peered":"2024-09-21T06:13:24.600080+0000","last_clean":"2024-09-21T06:13:24.600080+0000","last_became_active":"2024-09-21T06:13:10.510206+0000","last_became_peered":"2024-09-21T06:13:10.510206+0000","last_unstale":"2024-09-21T06:13:24.600080+0000","last_undegraded":"2024-09-21T06:13:24.600080+0000","last_fullsized":"2024-09-21T06:13:24.600080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:03.223451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610031+0000","last_change":"2024-09-21T06:13:10.517525+0000","last_active":"2024-09-21T06:13:24.610031+0000","last_peered":"2024-09-21T06:13:24.610031+0000","last_clean":"2024-09-21T06:13:24.610031+0000","last_became_active":"2024-09-21T06:13:10.517144+0000","last_became_peered":"2024-09-21T06:13:10.517144+0000","last_unstale":"2024-09-21T06:13:24.610031+0000","last_undegraded":"2024-09-21T06:13:24.610031+0000","last_fullsized":"2024-09-21T06:13:24.610031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:46.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.617837+0000","last_change":"2024-09-21T06:13:10.515537+0000","last_active":"2024-09-21T06:13:24.617837+0000","last_peered":"2024-09-21T06:13:24.617837+0000","last_clean":"2024-09-21T06:13:24.617837+0000","last_became_active":"2024-09-21T06:13:10.515063+0000","last_became_peered":"2024-09-21T06:13:10.515063+0000","last_unstale":"2024-09-21T06:13:24.617837+0000","last_undegraded":"2024-09-21T06:13:24.617837+0000","last_fullsized":"2024-09-21T06:13:24.617837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:33.718471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597513+0000","last_change":"2024-09-21T06:13:10.509368+0000","last_active":"2024-09-21T06:13:24.597513+0000","last_peered":"2024-09-21T06:13:24.597513+0000","last_clean":"2024-09-21T06:13:24.597513+0000","last_became_active":"2024-09-21T06:13:10.508982+0000","last_became_peered":"2024-09-21T06:13:10.508982+0000","last_unstale":"2024-09-21T06:13:24.597513+0000","last_undegraded":"2024-09-21T06:13:24.597513+0000","last_fullsized":"2024-09-21T06:13:24.597513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:54.430889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608602+0000","last_change":"2024-09-21T06:13:10.506559+0000","last_active":"2024-09-21T06:13:24.608602+0000","last_peered":"2024-09-21T06:13:24.608602+0000","last_clean":"2024-09-21T06:13:24.608602+0000","last_became_active":"2024-09-21T06:13:10.506161+0000","last_became_peered":"2024-09-21T06:13:10.506161+0000","last_unstale":"2024-09-21T06:13:24.608602+0000","last_undegraded":"2024-09-21T06:13:24.608602+0000","last_fullsized":"2024-09-21T06:13:24.608602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:19.870292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608304+0000","last_change":"2024-09-21T06:13:10.516000+0000","last_active":"2024-09-21T06:13:24.608304+0000","last_peered":"2024-09-21T06:13:24.608304+0000","last_clean":"2024-09-21T06:13:24.608304+0000","last_became_active":"2024-09-21T06:13:10.515543+0000","last_became_peered":"2024-09-21T06:13:10.515543+0000","last_unstale":"2024-09-21T06:13:24.608304+0000","last_undegraded":"2024-09-21T06:13:24.608304+0000","last_fullsized":"2024-09-21T06:13:24.608304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:48.968489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598302+0000","last_change":"2024-09-21T06:13:10.513753+0000","last_active":"2024-09-21T06:13:24.598302+0000","last_peered":"2024-09-21T06:13:24.598302+0000","last_clean":"2024-09-21T06:13:24.598302+0000","last_became_active":"2024-09-21T06:13:10.511079+0000","last_became_peered":"2024-09-21T06:13:10.511079+0000","last_unstale":"2024-09-21T06:13:24.598302+0000","last_undegraded":"2024-09-21T06:13:24.598302+0000","last_fullsized":"2024-09-21T06:13:24.598302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:25.320041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607768+0000","last_change":"2024-09-21T06:13:10.515222+0000","last_active":"2024-09-21T06:13:24.607768+0000","last_peered":"2024-09-21T06:13:24.607768+0000","last_clean":"2024-09-21T06:13:24.607768+0000","last_became_active":"2024-09-21T06:13:10.514604+0000","last_became_peered":"2024-09-21T06:13:10.514604+0000","last_unstale":"2024-09-21T06:13:24.607768+0000","last_undegraded":"2024-09-21T06:13:24.607768+0000","last_fullsized":"2024-09-21T06:13:24.607768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:03.165695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600779+0000","last_change":"2024-09-21T06:13:10.518334+0000","last_active":"2024-09-21T06:13:24.600779+0000","last_peered":"2024-09-21T06:13:24.600779+0000","last_clean":"2024-09-21T06:13:24.600779+0000","last_became_active":"2024-09-21T06:13:10.517956+0000","last_became_peered":"2024-09-21T06:13:10.517956+0000","last_unstale":"2024-09-21T06:13:24.600779+0000","last_undegraded":"2024-09-21T06:13:24.600779+0000","last_fullsized":"2024-09-21T06:13:24.600779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:05.094454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604993+0000","last_change":"2024-09-21T06:13:10.508763+0000","last_active":"2024-09-21T06:13:24.604993+0000","last_peered":"2024-09-21T06:13:24.604993+0000","last_clean":"2024-09-21T06:13:24.604993+0000","last_became_active":"2024-09-21T06:13:10.508371+0000","last_became_peered":"2024-09-21T06:13:10.508371+0000","last_unstale":"2024-09-21T06:13:24.604993+0000","last_undegraded":"2024-09-21T06:13:24.604993+0000","last_fullsized":"2024-09-21T06:13:24.604993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:50.602692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607019+0000","last_change":"2024-09-21T06:13:10.503465+0000","last_active":"2024-09-21T06:13:24.607019+0000","last_peered":"2024-09-21T06:13:24.607019+0000","last_clean":"2024-09-21T06:13:24.607019+0000","last_became_active":"2024-09-21T06:13:10.503072+0000","last_became_peered":"2024-09-21T06:13:10.503072+0000","last_unstale":"2024-09-21T06:13:24.607019+0000","last_undegraded":"2024-09-21T06:13:24.607019+0000","last_fullsized":"2024-09-21T06:13:24.607019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:36.368987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597579+0000","last_change":"2024-09-21T06:13:10.512841+0000","last_active":"2024-09-21T06:13:24.597579+0000","last_peered":"2024-09-21T06:13:24.597579+0000","last_clean":"2024-09-21T06:13:24.597579+0000","last_became_active":"2024-09-21T06:13:10.512407+0000","last_became_peered":"2024-09-21T06:13:10.512407+0000","last_unstale":"2024-09-21T06:13:24.597579+0000","last_undegraded":"2024-09-21T06:13:24.597579+0000","last_fullsized":"2024-09-21T06:13:24.597579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:09.855608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598794+0000","last_change":"2024-09-21T06:13:10.510088+0000","last_active":"2024-09-21T06:13:24.598794+0000","last_peered":"2024-09-21T06:13:24.598794+0000","last_clean":"2024-09-21T06:13:24.598794+0000","last_became_active":"2024-09-21T06:13:10.507904+0000","last_became_peered":"2024-09-21T06:13:10.507904+0000","last_unstale":"2024-09-21T06:13:24.598794+0000","last_undegraded":"2024-09-21T06:13:24.598794+0000","last_fullsized":"2024-09-21T06:13:24.598794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:49.404402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.611267+0000","last_change":"2024-09-21T06:13:10.517720+0000","last_active":"2024-09-21T06:13:24.611267+0000","last_peered":"2024-09-21T06:13:24.611267+0000","last_clean":"2024-09-21T06:13:24.611267+0000","last_became_active":"2024-09-21T06:13:10.514641+0000","last_became_peered":"2024-09-21T06:13:10.514641+0000","last_unstale":"2024-09-21T06:13:24.611267+0000","last_undegraded":"2024-09-21T06:13:24.611267+0000","last_fullsized":"2024-09-21T06:13:24.611267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:40.639665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602839+0000","last_change":"2024-09-21T06:13:10.512234+0000","last_active":"2024-09-21T06:13:24.602839+0000","last_peered":"2024-09-21T06:13:24.602839+0000","last_clean":"2024-09-21T06:13:24.602839+0000","last_became_active":"2024-09-21T06:13:10.511594+0000","last_became_peered":"2024-09-21T06:13:10.511594+0000","last_unstale":"2024-09-21T06:13:24.602839+0000","last_undegraded":"2024-09-21T06:13:24.602839+0000","last_fullsized":"2024-09-21T06:13:24.602839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:04.410342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610455+0000","last_change":"2024-09-21T06:13:10.513216+0000","last_active":"2024-09-21T06:13:24.610455+0000","last_peered":"2024-09-21T06:13:24.610455+0000","last_clean":"2024-09-21T06:13:24.610455+0000","last_became_active":"2024-09-21T06:13:10.512830+0000","last_became_peered":"2024-09-21T06:13:10.512830+0000","last_unstale":"2024-09-21T06:13:24.610455+0000","last_undegraded":"2024-09-21T06:13:24.610455+0000","last_fullsized":"2024-09-21T06:13:24.610455+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:46.418161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.605196+0000","last_change":"2024-09-21T06:13:10.518863+0000","last_active":"2024-09-21T06:13:24.605196+0000","last_peered":"2024-09-21T06:13:24.605196+0000","last_clean":"2024-09-21T06:13:24.605196+0000","last_became_active":"2024-09-21T06:13:10.518492+0000","last_became_peered":"2024-09-21T06:13:10.518492+0000","last_unstale":"2024-09-21T06:13:24.605196+0000","last_undegraded":"2024-09-21T06:13:24.605196+0000","last_fullsized":"2024-09-21T06:13:24.605196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:23.770763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608235+0000","last_change":"2024-09-21T06:13:10.508538+0000","last_active":"2024-09-21T06:13:24.608235+0000","last_peered":"2024-09-21T06:13:24.608235+0000","last_clean":"2024-09-21T06:13:24.608235+0000","last_became_active":"2024-09-21T06:13:10.508017+0000","last_became_peered":"2024-09-21T06:13:10.508017+0000","last_unstale":"2024-09-21T06:13:24.608235+0000","last_undegraded":"2024-09-21T06:13:24.608235+0000","last_fullsized":"2024-09-21T06:13:24.608235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:00.168741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597567+0000","last_change":"2024-09-21T06:13:10.516734+0000","last_active":"2024-09-21T06:13:24.597567+0000","last_peered":"2024-09-21T06:13:24.597567+0000","last_clean":"2024-09-21T06:13:24.597567+0000","last_became_active":"2024-09-21T06:13:10.516362+0000","last_became_peered":"2024-09-21T06:13:10.516362+0000","last_unstale":"2024-09-21T06:13:24.597567+0000","last_undegraded":"2024-09-21T06:13:24.597567+0000","last_fullsized":"2024-09-21T06:13:24.597567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:49.792011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608841+0000","last_change":"2024-09-21T06:13:10.513143+0000","last_active":"2024-09-21T06:13:24.608841+0000","last_peered":"2024-09-21T06:13:24.608841+0000","last_clean":"2024-09-21T06:13:24.608841+0000","last_became_active":"2024-09-21T06:13:10.512338+0000","last_became_peered":"2024-09-21T06:13:10.512338+0000","last_unstale":"2024-09-21T06:13:24.608841+0000","last_undegraded":"2024-09-21T06:13:24.608841+0000","last_fullsized":"2024-09-21T06:13:24.608841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:02.963860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602980+0000","last_change":"2024-09-21T06:13:10.513429+0000","last_active":"2024-09-21T06:13:24.602980+0000","last_peered":"2024-09-21T06:13:24.602980+0000","last_clean":"2024-09-21T06:13:24.602980+0000","last_became_active":"2024-09-21T06:13:10.512995+0000","last_became_peered":"2024-09-21T06:13:10.512995+0000","last_unstale":"2024-09-21T06:13:24.602980+0000","last_undegraded":"2024-09-21T06:13:24.602980+0000","last_fullsized":"2024-09-21T06:13:24.602980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:39.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603628+0000","last_change":"2024-09-21T06:13:10.509580+0000","last_active":"2024-09-21T06:13:24.603628+0000","last_peered":"2024-09-21T06:13:24.603628+0000","last_clean":"2024-09-21T06:13:24.603628+0000","last_became_active":"2024-09-21T06:13:10.509179+0000","last_became_peered":"2024-09-21T06:13:10.509179+0000","last_unstale":"2024-09-21T06:13:24.603628+0000","last_undegraded":"2024-09-21T06:13:24.603628+0000","last_fullsized":"2024-09-21T06:13:24.603628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:39.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.606565+0000","last_change":"2024-09-21T06:13:10.508358+0000","last_active":"2024-09-21T06:13:24.606565+0000","last_peered":"2024-09-21T06:13:24.606565+0000","last_clean":"2024-09-21T06:13:24.606565+0000","last_became_active":"2024-09-21T06:13:10.507952+0000","last_became_peered":"2024-09-21T06:13:10.507952+0000","last_unstale":"2024-09-21T06:13:24.606565+0000","last_undegraded":"2024-09-21T06:13:24.606565+0000","last_fullsized":"2024-09-21T06:13:24.606565+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:55.565203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607615+0000","last_change":"2024-09-21T06:13:10.510580+0000","last_active":"2024-09-21T06:13:24.607615+0000","last_peered":"2024-09-21T06:13:24.607615+0000","last_clean":"2024-09-21T06:13:24.607615+0000","last_became_active":"2024-09-21T06:13:10.510175+0000","last_became_peered":"2024-09-21T06:13:10.510175+0000","last_unstale":"2024-09-21T06:13:24.607615+0000","last_undegraded":"2024-09-21T06:13:24.607615+0000","last_fullsized":"2024-09-21T06:13:24.607615+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:49.467057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":87,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:16.693979+0000","last_change":"2024-09-21T06:13:08.485527+0000","last_active":"2024-09-21T06:14:16.693979+0000","last_peered":"2024-09-21T06:14:16.693979+0000","last_clean":"2024-09-21T06:14:16.693979+0000","last_became_active":"2024-09-21T06:13:08.485094+0000","last_became_peered":"2024-09-21T06:13:08.485094+0000","last_unstale":"2024-09-21T06:14:16.693979+0000","last_undegraded":"2024-09-21T06:14:16.693979+0000","last_fullsized":"2024-09-21T06:14:16.693979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:48.285638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603582+0000","last_change":"2024-09-21T06:13:10.507977+0000","last_active":"2024-09-21T06:13:24.603582+0000","last_peered":"2024-09-21T06:13:24.603582+0000","last_clean":"2024-09-21T06:13:24.603582+0000","last_became_active":"2024-09-21T06:13:10.507594+0000","last_became_peered":"2024-09-21T06:13:10.507594+0000","last_unstale":"2024-09-21T06:13:24.603582+0000","last_undegraded":"2024-09-21T06:13:24.603582+0000","last_fullsized":"2024-09-21T06:13:24.603582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:57.318240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.599690+0000","last_change":"2024-09-21T06:13:08.491237+0000","last_active":"2024-09-21T06:13:24.599690+0000","last_peered":"2024-09-21T06:13:24.599690+0000","last_clean":"2024-09-21T06:13:24.599690+0000","last_became_active":"2024-09-21T06:13:08.490908+0000","last_became_peered":"2024-09-21T06:13:08.490908+0000","last_unstale":"2024-09-21T06:13:24.599690+0000","last_undegraded":"2024-09-21T06:13:24.599690+0000","last_fullsized":"2024-09-21T06:13:24.599690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:53.185029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597228+0000","last_change":"2024-09-21T06:13:08.488721+0000","last_active":"2024-09-21T06:13:24.597228+0000","last_peered":"2024-09-21T06:13:24.597228+0000","last_clean":"2024-09-21T06:13:24.597228+0000","last_became_active":"2024-09-21T06:13:08.488347+0000","last_became_peered":"2024-09-21T06:13:08.488347+0000","last_unstale":"2024-09-21T06:13:24.597228+0000","last_undegraded":"2024-09-21T06:13:24.597228+0000","last_fullsized":"2024-09-21T06:13:24.597228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:27.008479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609911+0000","last_change":"2024-09-21T06:13:10.506527+0000","last_active":"2024-09-21T06:13:24.609911+0000","last_peered":"2024-09-21T06:13:24.609911+0000","last_clean":"2024-09-21T06:13:24.609911+0000","last_became_active":"2024-09-21T06:13:10.506163+0000","last_became_peered":"2024-09-21T06:13:10.506163+0000","last_unstale":"2024-09-21T06:13:24.609911+0000","last_undegraded":"2024-09-21T06:13:24.609911+0000","last_fullsized":"2024-09-21T06:13:24.609911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.244097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604150+0000","last_change":"2024-09-21T06:13:08.489292+0000","last_active":"2024-09-21T06:13:24.604150+0000","last_peered":"2024-09-21T06:13:24.604150+0000","last_clean":"2024-09-21T06:13:24.604150+0000","last_became_active":"2024-09-21T06:13:08.488918+0000","last_became_peered":"2024-09-21T06:13:08.488918+0000","last_unstale":"2024-09-21T06:13:24.604150+0000","last_undegraded":"2024-09-21T06:13:24.604150+0000","last_fullsized":"2024-09-21T06:13:24.604150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:37.992260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600648+0000","last_change":"2024-09-21T06:13:10.505027+0000","last_active":"2024-09-21T06:13:24.600648+0000","last_peered":"2024-09-21T06:13:24.600648+0000","last_clean":"2024-09-21T06:13:24.600648+0000","last_became_active":"2024-09-21T06:13:10.504669+0000","last_became_peered":"2024-09-21T06:13:10.504669+0000","last_unstale":"2024-09-21T06:13:24.600648+0000","last_undegraded":"2024-09-21T06:13:24.600648+0000","last_fullsized":"2024-09-21T06:13:24.600648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:43.927229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":132,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:15.691583+0000","last_change":"2024-09-21T06:13:08.491101+0000","last_active":"2024-09-21T06:14:15.691583+0000","last_peered":"2024-09-21T06:14:15.691583+0000","last_clean":"2024-09-21T06:14:15.691583+0000","last_became_active":"2024-09-21T06:13:08.490607+0000","last_became_peered":"2024-09-21T06:13:08.490607+0000","last_unstale":"2024-09-21T06:14:15.691583+0000","last_undegraded":"2024-09-21T06:14:15.691583+0000","last_fullsized":"2024-09-21T06:14:15.691583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:16.758463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598830+0000","last_change":"2024-09-21T06:13:10.510453+0000","last_active":"2024-09-21T06:13:24.598830+0000","last_peered":"2024-09-21T06:13:24.598830+0000","last_clean":"2024-09-21T06:13:24.598830+0000","last_became_active":"2024-09-21T06:13:10.508513+0000","last_became_peered":"2024-09-21T06:13:10.508513+0000","last_unstale":"2024-09-21T06:13:24.598830+0000","last_undegraded":"2024-09-21T06:13:24.598830+0000","last_fullsized":"2024-09-21T06:13:24.598830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:48.388140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609878+0000","last_change":"2024-09-21T06:13:08.488714+0000","last_active":"2024-09-21T06:13:24.609878+0000","last_peered":"2024-09-21T06:13:24.609878+0000","last_clean":"2024-09-21T06:13:24.609878+0000","last_became_active":"2024-09-21T06:13:08.488363+0000","last_became_peered":"2024-09-21T06:13:08.488363+0000","last_unstale":"2024-09-21T06:13:24.609878+0000","last_undegraded":"2024-09-21T06:13:24.609878+0000","last_fullsized":"2024-09-21T06:13:24.609878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:33.626165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609602+0000","last_change":"2024-09-21T06:13:10.508046+0000","last_active":"2024-09-21T06:13:24.609602+0000","last_peered":"2024-09-21T06:13:24.609602+0000","last_clean":"2024-09-21T06:13:24.609602+0000","last_became_active":"2024-09-21T06:13:10.507660+0000","last_became_peered":"2024-09-21T06:13:10.507660+0000","last_unstale":"2024-09-21T06:13:24.609602+0000","last_undegraded":"2024-09-21T06:13:24.609602+0000","last_fullsized":"2024-09-21T06:13:24.609602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:35.466965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603272+0000","last_change":"2024-09-21T06:13:08.562863+0000","last_active":"2024-09-21T06:13:24.603272+0000","last_peered":"2024-09-21T06:13:24.603272+0000","last_clean":"2024-09-21T06:13:24.603272+0000","last_became_active":"2024-09-21T06:13:08.562424+0000","last_became_peered":"2024-09-21T06:13:08.562424+0000","last_unstale":"2024-09-21T06:13:24.603272+0000","last_undegraded":"2024-09-21T06:13:24.603272+0000","last_fullsized":"2024-09-21T06:13:24.603272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:43.251932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598595+0000","last_change":"2024-09-21T06:13:10.509818+0000","last_active":"2024-09-21T06:13:24.598595+0000","last_peered":"2024-09-21T06:13:24.598595+0000","last_clean":"2024-09-21T06:13:24.598595+0000","last_became_active":"2024-09-21T06:13:10.509468+0000","last_became_peered":"2024-09-21T06:13:10.509468+0000","last_unstale":"2024-09-21T06:13:24.598595+0000","last_undegraded":"2024-09-21T06:13:24.598595+0000","last_fullsized":"2024-09-21T06:13:24.598595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:25.130868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'31","reported_seq":187,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:15.693259+0000","last_change":"2024-09-21T06:13:24.633807+0000","last_active":"2024-09-21T06:14:15.693259+0000","last_peered":"2024-09-21T06:14:15.693259+0000","last_clean":"2024-09-21T06:14:15.693259+0000","last_became_active":"2024-09-21T06:13:24.618599+0000","last_became_peered":"2024-09-21T06:13:24.618599+0000","last_unstale":"2024-09-21T06:14:15.693259+0000","last_undegraded":"2024-09-21T06:14:15.693259+0000","last_fullsized":"2024-09-21T06:14:15.693259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:07.449253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65966,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":119,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608537+0000","last_change":"2024-09-21T06:13:10.511360+0000","last_active":"2024-09-21T06:13:24.608537+0000","last_peered":"2024-09-21T06:13:24.608537+0000","last_clean":"2024-09-21T06:13:24.608537+0000","last_became_active":"2024-09-21T06:13:10.510980+0000","last_became_peered":"2024-09-21T06:13:10.510980+0000","last_unstale":"2024-09-21T06:13:24.608537+0000","last_undegraded":"2024-09-21T06:13:24.608537+0000","last_fullsized":"2024-09-21T06:13:24.608537+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:43.859758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604427+0000","last_change":"2024-09-21T06:13:08.486262+0000","last_active":"2024-09-21T06:13:24.604427+0000","last_peered":"2024-09-21T06:13:24.604427+0000","last_clean":"2024-09-21T06:13:24.604427+0000","last_became_active":"2024-09-21T06:13:08.485509+0000","last_became_peered":"2024-09-21T06:13:08.485509+0000","last_unstale":"2024-09-21T06:13:24.604427+0000","last_undegraded":"2024-09-21T06:13:24.604427+0000","last_fullsized":"2024-09-21T06:13:24.604427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:31.726746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597324+0000","last_change":"2024-09-21T06:13:10.506894+0000","last_active":"2024-09-21T06:13:24.597324+0000","last_peered":"2024-09-21T06:13:24.597324+0000","last_clean":"2024-09-21T06:13:24.597324+0000","last_became_active":"2024-09-21T06:13:10.506489+0000","last_became_peered":"2024-09-21T06:13:10.506489+0000","last_unstale":"2024-09-21T06:13:24.597324+0000","last_undegraded":"2024-09-21T06:13:24.597324+0000","last_fullsized":"2024-09-21T06:13:24.597324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:58.755601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603141+0000","last_change":"2024-09-21T06:13:08.483429+0000","last_active":"2024-09-21T06:13:24.603141+0000","last_peered":"2024-09-21T06:13:24.603141+0000","last_clean":"2024-09-21T06:13:24.603141+0000","last_became_active":"2024-09-21T06:13:08.483028+0000","last_became_peered":"2024-09-21T06:13:08.483028+0000","last_unstale":"2024-09-21T06:13:24.603141+0000","last_undegraded":"2024-09-21T06:13:24.603141+0000","last_fullsized":"2024-09-21T06:13:24.603141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:33.258154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609758+0000","last_change":"2024-09-21T06:13:10.513398+0000","last_active":"2024-09-21T06:13:24.609758+0000","last_peered":"2024-09-21T06:13:24.609758+0000","last_clean":"2024-09-21T06:13:24.609758+0000","last_became_active":"2024-09-21T06:13:10.512713+0000","last_became_peered":"2024-09-21T06:13:10.512713+0000","last_unstale":"2024-09-21T06:13:24.609758+0000","last_undegraded":"2024-09-21T06:13:24.609758+0000","last_fullsized":"2024-09-21T06:13:24.609758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:21.026984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":85,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:15.692383+0000","last_change":"2024-09-21T06:13:08.493771+0000","last_active":"2024-09-21T06:14:15.692383+0000","last_peered":"2024-09-21T06:14:15.692383+0000","last_clean":"2024-09-21T06:14:15.692383+0000","last_became_active":"2024-09-21T06:13:08.493380+0000","last_became_peered":"2024-09-21T06:13:08.493380+0000","last_unstale":"2024-09-21T06:14:15.692383+0000","last_undegraded":"2024-09-21T06:14:15.692383+0000","last_fullsized":"2024-09-21T06:14:15.692383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:12.552137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609811+0000","last_change":"2024-09-21T06:13:10.511679+0000","last_active":"2024-09-21T06:13:24.609811+0000","last_peered":"2024-09-21T06:13:24.609811+0000","last_clean":"2024-09-21T06:13:24.609811+0000","last_became_active":"2024-09-21T06:13:10.511270+0000","last_became_peered":"2024-09-21T06:13:10.511270+0000","last_unstale":"2024-09-21T06:13:24.609811+0000","last_undegraded":"2024-09-21T06:13:24.609811+0000","last_fullsized":"2024-09-21T06:13:24.609811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:10.521853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.627560+0000","last_change":"2024-09-21T06:13:24.627467+0000","last_active":"2024-09-21T06:13:24.627560+0000","last_peered":"2024-09-21T06:13:24.627560+0000","last_clean":"2024-09-21T06:13:24.627560+0000","last_became_active":"2024-09-21T06:13:24.610263+0000","last_became_peered":"2024-09-21T06:13:24.610263+0000","last_unstale":"2024-09-21T06:13:24.627560+0000","last_undegraded":"2024-09-21T06:13:24.627560+0000","last_fullsized":"2024-09-21T06:13:24.627560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:26.472147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602912+0000","last_change":"2024-09-21T06:13:10.517214+0000","last_active":"2024-09-21T06:13:24.602912+0000","last_peered":"2024-09-21T06:13:24.602912+0000","last_clean":"2024-09-21T06:13:24.602912+0000","last_became_active":"2024-09-21T06:13:10.516828+0000","last_became_peered":"2024-09-21T06:13:10.516828+0000","last_unstale":"2024-09-21T06:13:24.602912+0000","last_undegraded":"2024-09-21T06:13:24.602912+0000","last_fullsized":"2024-09-21T06:13:24.602912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:58.077670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608703+0000","last_change":"2024-09-21T06:13:08.489153+0000","last_active":"2024-09-21T06:13:24.608703+0000","last_peered":"2024-09-21T06:13:24.608703+0000","last_clean":"2024-09-21T06:13:24.608703+0000","last_became_active":"2024-09-21T06:13:08.488732+0000","last_became_peered":"2024-09-21T06:13:08.488732+0000","last_unstale":"2024-09-21T06:13:24.608703+0000","last_undegraded":"2024-09-21T06:13:24.608703+0000","last_fullsized":"2024-09-21T06:13:24.608703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:22.848136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610561+0000","last_change":"2024-09-21T06:13:24.610441+0000","last_active":"2024-09-21T06:13:24.610561+0000","last_peered":"2024-09-21T06:13:24.610561+0000","last_clean":"2024-09-21T06:13:24.610561+0000","last_became_active":"2024-09-21T06:13:24.609992+0000","last_became_peered":"2024-09-21T06:13:24.609992+0000","last_unstale":"2024-09-21T06:13:24.610561+0000","last_undegraded":"2024-09-21T06:13:24.610561+0000","last_fullsized":"2024-09-21T06:13:24.610561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:59.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597725+0000","last_change":"2024-09-21T06:13:08.484514+0000","last_active":"2024-09-21T06:13:24.597725+0000","last_peered":"2024-09-21T06:13:24.597725+0000","last_clean":"2024-09-21T06:13:24.597725+0000","last_became_active":"2024-09-21T06:13:08.484134+0000","last_became_peered":"2024-09-21T06:13:08.484134+0000","last_unstale":"2024-09-21T06:13:24.597725+0000","last_undegraded":"2024-09-21T06:13:24.597725+0000","last_fullsized":"2024-09-21T06:13:24.597725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:45.124734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610027+0000","last_change":"2024-09-21T06:13:10.503798+0000","last_active":"2024-09-21T06:13:24.610027+0000","last_peered":"2024-09-21T06:13:24.610027+0000","last_clean":"2024-09-21T06:13:24.610027+0000","last_became_active":"2024-09-21T06:13:10.502418+0000","last_became_peered":"2024-09-21T06:13:10.502418+0000","last_unstale":"2024-09-21T06:13:24.610027+0000","last_undegraded":"2024-09-21T06:13:24.610027+0000","last_fullsized":"2024-09-21T06:13:24.610027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:57.575569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603439+0000","last_change":"2024-09-21T06:13:08.487191+0000","last_active":"2024-09-21T06:13:24.603439+0000","last_peered":"2024-09-21T06:13:24.603439+0000","last_clean":"2024-09-21T06:13:24.603439+0000","last_became_active":"2024-09-21T06:13:08.486812+0000","last_became_peered":"2024-09-21T06:13:08.486812+0000","last_unstale":"2024-09-21T06:13:24.603439+0000","last_undegraded":"2024-09-21T06:13:24.603439+0000","last_fullsized":"2024-09-21T06:13:24.603439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:51.989708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610179+0000","last_change":"2024-09-21T06:13:10.511678+0000","last_active":"2024-09-21T06:13:24.610179+0000","last_peered":"2024-09-21T06:13:24.610179+0000","last_clean":"2024-09-21T06:13:24.610179+0000","last_became_active":"2024-09-21T06:13:10.510408+0000","last_became_peered":"2024-09-21T06:13:10.510408+0000","last_unstale":"2024-09-21T06:13:24.610179+0000","last_undegraded":"2024-09-21T06:13:24.610179+0000","last_fullsized":"2024-09-21T06:13:24.610179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:58.711190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'6","reported_seq":82,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:12.687569+0000","last_change":"2024-09-21T06:13:08.560894+0000","last_active":"2024-09-21T06:14:12.687569+0000","last_peered":"2024-09-21T06:14:12.687569+0000","last_clean":"2024-09-21T06:14:12.687569+0000","last_became_active":"2024-09-21T06:13:08.560423+0000","last_became_peered":"2024-09-21T06:13:08.560423+0000","last_unstale":"2024-09-21T06:14:12.687569+0000","last_undegraded":"2024-09-21T06:14:12.687569+0000","last_fullsized":"2024-09-21T06:14:12.687569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:53.015711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610081+0000","last_change":"2024-09-21T06:13:10.516009+0000","last_active":"2024-09-21T06:13:24.610081+0000","last_peered":"2024-09-21T06:13:24.610081+0000","last_clean":"2024-09-21T06:13:24.610081+0000","last_became_active":"2024-09-21T06:13:10.515669+0000","last_became_peered":"2024-09-21T06:13:10.515669+0000","last_unstale":"2024-09-21T06:13:24.610081+0000","last_undegraded":"2024-09-21T06:13:24.610081+0000","last_fullsized":"2024-09-21T06:13:24.610081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:54.134564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":99,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610317+0000","last_change":"2024-09-21T06:11:53.408030+0000","last_active":"2024-09-21T06:13:24.610317+0000","last_peered":"2024-09-21T06:13:24.610317+0000","last_clean":"2024-09-21T06:13:24.610317+0000","last_became_active":"2024-09-21T06:11:53.402296+0000","last_became_peered":"2024-09-21T06:11:53.402296+0000","last_unstale":"2024-09-21T06:13:24.610317+0000","last_undegraded":"2024-09-21T06:13:24.610317+0000","last_fullsized":"2024-09-21T06:13:24.610317+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_clean_scrub_stamp":"2024-09-21T06:08:26.098183+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:08.239872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616959+0000","last_change":"2024-09-21T06:13:08.489209+0000","last_active":"2024-09-21T06:13:24.616959+0000","last_peered":"2024-09-21T06:13:24.616959+0000","last_clean":"2024-09-21T06:13:24.616959+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.616959+0000","last_undegraded":"2024-09-21T06:13:24.616959+0000","last_fullsized":"2024-09-21T06:13:24.616959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:38.798499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607003+0000","last_change":"2024-09-21T06:13:24.606487+0000","last_active":"2024-09-21T06:13:24.607003+0000","last_peered":"2024-09-21T06:13:24.607003+0000","last_clean":"2024-09-21T06:13:24.607003+0000","last_became_active":"2024-09-21T06:13:24.606141+0000","last_became_peered":"2024-09-21T06:13:24.606141+0000","last_unstale":"2024-09-21T06:13:24.607003+0000","last_undegraded":"2024-09-21T06:13:24.607003+0000","last_fullsized":"2024-09-21T06:13:24.607003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:24.040770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609976+0000","last_change":"2024-09-21T06:13:08.489206+0000","last_active":"2024-09-21T06:13:24.609976+0000","last_peered":"2024-09-21T06:13:24.609976+0000","last_clean":"2024-09-21T06:13:24.609976+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.609976+0000","last_undegraded":"2024-09-21T06:13:24.609976+0000","last_fullsized":"2024-09-21T06:13:24.609976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.627305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597214+0000","last_change":"2024-09-21T06:13:10.513999+0000","last_active":"2024-09-21T06:13:24.597214+0000","last_peered":"2024-09-21T06:13:24.597214+0000","last_clean":"2024-09-21T06:13:24.597214+0000","last_became_active":"2024-09-21T06:13:10.513657+0000","last_became_peered":"2024-09-21T06:13:10.513657+0000","last_unstale":"2024-09-21T06:13:24.597214+0000","last_undegraded":"2024-09-21T06:13:24.597214+0000","last_fullsized":"2024-09-21T06:13:24.597214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:05.536921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'5","reported_seq":84,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:17.694312+0000","last_change":"2024-09-21T06:13:08.560811+0000","last_active":"2024-09-21T06:14:17.694312+0000","last_peered":"2024-09-21T06:14:17.694312+0000","last_clean":"2024-09-21T06:14:17.694312+0000","last_became_active":"2024-09-21T06:13:08.560145+0000","last_became_peered":"2024-09-21T06:13:08.560145+0000","last_unstale":"2024-09-21T06:14:17.694312+0000","last_undegraded":"2024-09-21T06:14:17.694312+0000","last_fullsized":"2024-09-21T06:14:17.694312+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:24.550008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.613627+0000","last_change":"2024-09-21T06:13:24.613495+0000","last_active":"2024-09-21T06:13:24.613627+0000","last_peered":"2024-09-21T06:13:24.613627+0000","last_clean":"2024-09-21T06:13:24.613627+0000","last_became_active":"2024-09-21T06:13:24.612634+0000","last_became_peered":"2024-09-21T06:13:24.612634+0000","last_unstale":"2024-09-21T06:13:24.613627+0000","last_undegraded":"2024-09-21T06:13:24.613627+0000","last_fullsized":"2024-09-21T06:13:24.613627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.863141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'5","reported_seq":85,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:17.691999+0000","last_change":"2024-09-21T06:13:08.486973+0000","last_active":"2024-09-21T06:14:17.691999+0000","last_peered":"2024-09-21T06:14:17.691999+0000","last_clean":"2024-09-21T06:14:17.691999+0000","last_became_active":"2024-09-21T06:13:08.486602+0000","last_became_peered":"2024-09-21T06:13:08.486602+0000","last_unstale":"2024-09-21T06:14:17.691999+0000","last_undegraded":"2024-09-21T06:14:17.691999+0000","last_fullsized":"2024-09-21T06:14:17.691999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:41.804955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.599893+0000","last_change":"2024-09-21T06:13:10.509499+0000","last_active":"2024-09-21T06:13:24.599893+0000","last_peered":"2024-09-21T06:13:24.599893+0000","last_clean":"2024-09-21T06:13:24.599893+0000","last_became_active":"2024-09-21T06:13:10.508566+0000","last_became_peered":"2024-09-21T06:13:10.508566+0000","last_unstale":"2024-09-21T06:13:24.599893+0000","last_undegraded":"2024-09-21T06:13:24.599893+0000","last_fullsized":"2024-09-21T06:13:24.599893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:12.701631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600737+0000","last_change":"2024-09-21T06:13:08.561378+0000","last_active":"2024-09-21T06:13:24.600737+0000","last_peered":"2024-09-21T06:13:24.600737+0000","last_clean":"2024-09-21T06:13:24.600737+0000","last_became_active":"2024-09-21T06:13:08.561002+0000","last_became_peered":"2024-09-21T06:13:08.561002+0000","last_unstale":"2024-09-21T06:13:24.600737+0000","last_undegraded":"2024-09-21T06:13:24.600737+0000","last_fullsized":"2024-09-21T06:13:24.600737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:11.325377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602794+0000","last_change":"2024-09-21T06:13:24.602347+0000","last_active":"2024-09-21T06:13:24.602794+0000","last_peered":"2024-09-21T06:13:24.602794+0000","last_clean":"2024-09-21T06:13:24.602794+0000","last_became_active":"2024-09-21T06:13:24.601978+0000","last_became_peered":"2024-09-21T06:13:24.601978+0000","last_unstale":"2024-09-21T06:13:24.602794+0000","last_undegraded":"2024-09-21T06:13:24.602794+0000","last_fullsized":"2024-09-21T06:13:24.602794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:34.860096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608569+0000","last_change":"2024-09-21T06:13:08.561937+0000","last_active":"2024-09-21T06:13:24.608569+0000","last_peered":"2024-09-21T06:13:24.608569+0000","last_clean":"2024-09-21T06:13:24.608569+0000","last_became_active":"2024-09-21T06:13:08.560547+0000","last_became_peered":"2024-09-21T06:13:08.560547+0000","last_unstale":"2024-09-21T06:13:24.608569+0000","last_undegraded":"2024-09-21T06:13:24.608569+0000","last_fullsized":"2024-09-21T06:13:24.608569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:29.702291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597628+0000","last_change":"2024-09-21T06:13:10.516833+0000","last_active":"2024-09-21T06:13:24.597628+0000","last_peered":"2024-09-21T06:13:24.597628+0000","last_clean":"2024-09-21T06:13:24.597628+0000","last_became_active":"2024-09-21T06:13:10.516486+0000","last_became_peered":"2024-09-21T06:13:10.516486+0000","last_unstale":"2024-09-21T06:13:24.597628+0000","last_undegraded":"2024-09-21T06:13:24.597628+0000","last_fullsized":"2024-09-21T06:13:24.597628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:05.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'16","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:37.499820+0000","last_change":"2024-09-21T06:13:08.483271+0000","last_active":"2024-09-21T06:13:37.499820+0000","last_peered":"2024-09-21T06:13:37.499820+0000","last_clean":"2024-09-21T06:13:37.499820+0000","last_became_active":"2024-09-21T06:13:08.482951+0000","last_became_peered":"2024-09-21T06:13:08.482951+0000","last_unstale":"2024-09-21T06:13:37.499820+0000","last_undegraded":"2024-09-21T06:13:37.499820+0000","last_fullsized":"2024-09-21T06:13:37.499820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:17.537988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610524+0000","last_change":"2024-09-21T06:13:10.509125+0000","last_active":"2024-09-21T06:13:24.610524+0000","last_peered":"2024-09-21T06:13:24.610524+0000","last_clean":"2024-09-21T06:13:24.610524+0000","last_became_active":"2024-09-21T06:13:10.508773+0000","last_became_peered":"2024-09-21T06:13:10.508773+0000","last_unstale":"2024-09-21T06:13:24.610524+0000","last_undegraded":"2024-09-21T06:13:24.610524+0000","last_fullsized":"2024-09-21T06:13:24.610524+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:44.570477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616932+0000","last_change":"2024-09-21T06:13:08.497885+0000","last_active":"2024-09-21T06:13:24.616932+0000","last_peered":"2024-09-21T06:13:24.616932+0000","last_clean":"2024-09-21T06:13:24.616932+0000","last_became_active":"2024-09-21T06:13:08.497593+0000","last_became_peered":"2024-09-21T06:13:08.497593+0000","last_unstale":"2024-09-21T06:13:24.616932+0000","last_undegraded":"2024-09-21T06:13:24.616932+0000","last_fullsized":"2024-09-21T06:13:24.616932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:48.361175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609527+0000","last_change":"2024-09-21T06:13:10.514430+0000","last_active":"2024-09-21T06:13:24.609527+0000","last_peered":"2024-09-21T06:13:24.609527+0000","last_clean":"2024-09-21T06:13:24.609527+0000","last_became_active":"2024-09-21T06:13:10.514063+0000","last_became_peered":"2024-09-21T06:13:10.514063+0000","last_unstale":"2024-09-21T06:13:24.609527+0000","last_undegraded":"2024-09-21T06:13:24.609527+0000","last_fullsized":"2024-09-21T06:13:24.609527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:37.036497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598415+0000","last_change":"2024-09-21T06:13:08.560564+0000","last_active":"2024-09-21T06:13:24.598415+0000","last_peered":"2024-09-21T06:13:24.598415+0000","last_clean":"2024-09-21T06:13:24.598415+0000","last_became_active":"2024-09-21T06:13:08.560339+0000","last_became_peered":"2024-09-21T06:13:08.560339+0000","last_unstale":"2024-09-21T06:13:24.598415+0000","last_undegraded":"2024-09-21T06:13:24.598415+0000","last_fullsized":"2024-09-21T06:13:24.598415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:55.378911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.605861+0000","last_change":"2024-09-21T06:13:10.511354+0000","last_active":"2024-09-21T06:13:24.605861+0000","last_peered":"2024-09-21T06:13:24.605861+0000","last_clean":"2024-09-21T06:13:24.605861+0000","last_became_active":"2024-09-21T06:13:10.510946+0000","last_became_peered":"2024-09-21T06:13:10.510946+0000","last_unstale":"2024-09-21T06:13:24.605861+0000","last_undegraded":"2024-09-21T06:13:24.605861+0000","last_fullsized":"2024-09-21T06:13:24.605861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:11.535461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610883+0000","last_change":"2024-09-21T06:13:08.485578+0000","last_active":"2024-09-21T06:13:24.610883+0000","last_peered":"2024-09-21T06:13:24.610883+0000","last_clean":"2024-09-21T06:13:24.610883+0000","last_became_active":"2024-09-21T06:13:08.485170+0000","last_became_peered":"2024-09-21T06:13:08.485170+0000","last_unstale":"2024-09-21T06:13:24.610883+0000","last_undegraded":"2024-09-21T06:13:24.610883+0000","last_fullsized":"2024-09-21T06:13:24.610883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:39.000114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.606691+0000","last_change":"2024-09-21T06:13:10.510360+0000","last_active":"2024-09-21T06:13:24.606691+0000","last_peered":"2024-09-21T06:13:24.606691+0000","last_clean":"2024-09-21T06:13:24.606691+0000","last_became_active":"2024-09-21T06:13:10.509887+0000","last_became_peered":"2024-09-21T06:13:10.509887+0000","last_unstale":"2024-09-21T06:13:24.606691+0000","last_undegraded":"2024-09-21T06:13:24.606691+0000","last_fullsized":"2024-09-21T06:13:24.606691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:58.135706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.625360+0000","last_change":"2024-09-21T06:13:24.625284+0000","last_active":"2024-09-21T06:13:24.625360+0000","last_peered":"2024-09-21T06:13:24.625360+0000","last_clean":"2024-09-21T06:13:24.625360+0000","last_became_active":"2024-09-21T06:13:24.612704+0000","last_became_peered":"2024-09-21T06:13:24.612704+0000","last_unstale":"2024-09-21T06:13:24.625360+0000","last_undegraded":"2024-09-21T06:13:24.625360+0000","last_fullsized":"2024-09-21T06:13:24.625360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:29.385258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.611377+0000","last_change":"2024-09-21T06:13:10.514128+0000","last_active":"2024-09-21T06:13:24.611377+0000","last_peered":"2024-09-21T06:13:24.611377+0000","last_clean":"2024-09-21T06:13:24.611377+0000","last_became_active":"2024-09-21T06:13:10.513790+0000","last_became_peered":"2024-09-21T06:13:10.513790+0000","last_unstale":"2024-09-21T06:13:24.611377+0000","last_undegraded":"2024-09-21T06:13:24.611377+0000","last_fullsized":"2024-09-21T06:13:24.611377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:33.407189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607762+0000","last_change":"2024-09-21T06:13:08.489105+0000","last_active":"2024-09-21T06:13:24.607762+0000","last_peered":"2024-09-21T06:13:24.607762+0000","last_clean":"2024-09-21T06:13:24.607762+0000","last_became_active":"2024-09-21T06:13:08.487110+0000","last_became_peered":"2024-09-21T06:13:08.487110+0000","last_unstale":"2024-09-21T06:13:24.607762+0000","last_undegraded":"2024-09-21T06:13:24.607762+0000","last_fullsized":"2024-09-21T06:13:24.607762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:24.849365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610194+0000","last_change":"2024-09-21T06:13:10.513751+0000","last_active":"2024-09-21T06:13:24.610194+0000","last_peered":"2024-09-21T06:13:24.610194+0000","last_clean":"2024-09-21T06:13:24.610194+0000","last_became_active":"2024-09-21T06:13:10.513376+0000","last_became_peered":"2024-09-21T06:13:10.513376+0000","last_unstale":"2024-09-21T06:13:24.610194+0000","last_undegraded":"2024-09-21T06:13:24.610194+0000","last_fullsized":"2024-09-21T06:13:24.610194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.364959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603110+0000","last_change":"2024-09-21T06:13:10.519359+0000","last_active":"2024-09-21T06:13:24.603110+0000","last_peered":"2024-09-21T06:13:24.603110+0000","last_clean":"2024-09-21T06:13:24.603110+0000","last_became_active":"2024-09-21T06:13:10.518518+0000","last_became_peered":"2024-09-21T06:13:10.518518+0000","last_unstale":"2024-09-21T06:13:24.603110+0000","last_undegraded":"2024-09-21T06:13:24.603110+0000","last_fullsized":"2024-09-21T06:13:24.603110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:55.554710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608166+0000","last_change":"2024-09-21T06:13:08.488122+0000","last_active":"2024-09-21T06:13:24.608166+0000","last_peered":"2024-09-21T06:13:24.608166+0000","last_clean":"2024-09-21T06:13:24.608166+0000","last_became_active":"2024-09-21T06:13:08.487679+0000","last_became_peered":"2024-09-21T06:13:08.487679+0000","last_unstale":"2024-09-21T06:13:24.608166+0000","last_undegraded":"2024-09-21T06:13:24.608166+0000","last_fullsized":"2024-09-21T06:13:24.608166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:12.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598434+0000","last_change":"2024-09-21T06:13:10.516794+0000","last_active":"2024-09-21T06:13:24.598434+0000","last_peered":"2024-09-21T06:13:24.598434+0000","last_clean":"2024-09-21T06:13:24.598434+0000","last_became_active":"2024-09-21T06:13:10.516297+0000","last_became_peered":"2024-09-21T06:13:10.516297+0000","last_unstale":"2024-09-21T06:13:24.598434+0000","last_undegraded":"2024-09-21T06:13:24.598434+0000","last_fullsized":"2024-09-21T06:13:24.598434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:19.873742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.621488+0000","last_change":"2024-09-21T06:13:24.621358+0000","last_active":"2024-09-21T06:13:24.621488+0000","last_peered":"2024-09-21T06:13:24.621488+0000","last_clean":"2024-09-21T06:13:24.621488+0000","last_became_active":"2024-09-21T06:13:24.611058+0000","last_became_peered":"2024-09-21T06:13:24.611058+0000","last_unstale":"2024-09-21T06:13:24.621488+0000","last_undegraded":"2024-09-21T06:13:24.621488+0000","last_fullsized":"2024-09-21T06:13:24.621488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:28.759029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609712+0000","last_change":"2024-09-21T06:13:10.516339+0000","last_active":"2024-09-21T06:13:24.609712+0000","last_peered":"2024-09-21T06:13:24.609712+0000","last_clean":"2024-09-21T06:13:24.609712+0000","last_became_active":"2024-09-21T06:13:10.515397+0000","last_became_peered":"2024-09-21T06:13:10.515397+0000","last_unstale":"2024-09-21T06:13:24.609712+0000","last_undegraded":"2024-09-21T06:13:24.609712+0000","last_fullsized":"2024-09-21T06:13:24.609712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:56.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610156+0000","last_change":"2024-09-21T06:13:08.560494+0000","last_active":"2024-09-21T06:13:24.610156+0000","last_peered":"2024-09-21T06:13:24.610156+0000","last_clean":"2024-09-21T06:13:24.610156+0000","last_became_active":"2024-09-21T06:13:08.560118+0000","last_became_peered":"2024-09-21T06:13:08.560118+0000","last_unstale":"2024-09-21T06:13:24.610156+0000","last_undegraded":"2024-09-21T06:13:24.610156+0000","last_fullsized":"2024-09-21T06:13:24.610156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:05.597233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610032+0000","last_change":"2024-09-21T06:13:10.515780+0000","last_active":"2024-09-21T06:13:24.610032+0000","last_peered":"2024-09-21T06:13:24.610032+0000","last_clean":"2024-09-21T06:13:24.610032+0000","last_became_active":"2024-09-21T06:13:10.515437+0000","last_became_peered":"2024-09-21T06:13:10.515437+0000","last_unstale":"2024-09-21T06:13:24.610032+0000","last_undegraded":"2024-09-21T06:13:24.610032+0000","last_fullsized":"2024-09-21T06:13:24.610032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:04.848003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'13","reported_seq":90,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:14.686753+0000","last_change":"2024-09-21T06:13:08.483300+0000","last_active":"2024-09-21T06:14:14.686753+0000","last_peered":"2024-09-21T06:14:14.686753+0000","last_clean":"2024-09-21T06:14:14.686753+0000","last_became_active":"2024-09-21T06:13:08.482908+0000","last_became_peered":"2024-09-21T06:13:08.482908+0000","last_unstale":"2024-09-21T06:14:14.686753+0000","last_undegraded":"2024-09-21T06:14:14.686753+0000","last_fullsized":"2024-09-21T06:14:14.686753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:09.202075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73561,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":572,"num_read_kb":365,"num_write":247,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1724,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":220683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282438,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678043,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":864256,"data_stored":427523,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874272,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1084518,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037798,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":427179,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234027,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":427440,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":1020349,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593781,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":426779,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1019636,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":492997,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":426487,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378698,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":428582,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":493770,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":892928,"data_stored":428509,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":68093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:14:20.991 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:14:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:21.649 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:21 smithi003 ceph-mon[31227]: pgmap v305: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:21.650 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:21 smithi003 ceph-mon[31227]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:14:21.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.739+0000 7f58a3d76640 1 -- 172.21.15.3:0/979225974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f589c1083c0 msgr2=0x7f589c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:21.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.739+0000 7f58a3d76640 1 --2- 172.21.15.3:0/979225974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f589c1083c0 0x7f589c1087d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f588c009900 tx=0x7f588c02a410 comp rx=0 tx=0).stop 2024-09-21T06:14:21.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.739+0000 7f58a3d76640 1 -- 172.21.15.3:0/979225974 shutdown_connections 2024-09-21T06:14:21.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.739+0000 7f58a3d76640 1 --2- 172.21.15.3:0/979225974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f589c108d10 0x7f589c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:21.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.739+0000 7f58a3d76640 1 --2- 172.21.15.3:0/979225974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f589c1083c0 0x7f589c1087d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:21.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.739+0000 7f58a3d76640 1 --2- 172.21.15.3:0/979225974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f589c102a80 0x7f589c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:21.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.739+0000 7f58a3d76640 1 -- 172.21.15.3:0/979225974 >> 172.21.15.3:0/979225974 conn(0x7f589c079740 msgr2=0x7f589c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:21.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.739+0000 7f58a3d76640 1 -- 172.21.15.3:0/979225974 shutdown_connections 2024-09-21T06:14:21.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.739+0000 7f58a3d76640 1 -- 172.21.15.3:0/979225974 wait complete. 2024-09-21T06:14:21.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.740+0000 7f58a3d76640 1 Processor -- start 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.740+0000 7f58a3d76640 1 -- start start 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.740+0000 7f58a3d76640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f589c102a80 0x7f589c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.740+0000 7f58a3d76640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f589c1083c0 0x7f589c19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.740+0000 7f58a3d76640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f589c108d10 0x7f589c19bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.740+0000 7f58a3d76640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f589c112980 con 0x7f589c1083c0 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.740+0000 7f58a3d76640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f589c112800 con 0x7f589c102a80 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.740+0000 7f58a3d76640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f589c112b00 con 0x7f589c108d10 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.740+0000 7f58a1311640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f589c1083c0 0x7f589c19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f58a1311640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f589c1083c0 0x7f589c19b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50514/0 (socket says 172.21.15.3:50514) 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f58a1311640 1 -- 172.21.15.3:0/64342936 learned_addr learned my addr 172.21.15.3:0/64342936 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f58a1b12640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f589c102a80 0x7f589c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f58a2313640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f589c108d10 0x7f589c19bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f58a1311640 1 -- 172.21.15.3:0/64342936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f589c108d10 msgr2=0x7f589c19bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f58a1311640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f589c108d10 0x7f589c19bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f58a1311640 1 -- 172.21.15.3:0/64342936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f589c102a80 msgr2=0x7f589c19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f58a1311640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f589c102a80 0x7f589c19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f58a1311640 1 -- 172.21.15.3:0/64342936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c1a6630 con 0x7f589c1083c0 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f58a2313640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f589c108d10 0x7f589c19bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f58a1311640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f589c1083c0 0x7f589c19b8b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f588c032fd0 tx=0x7f588c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f5892ffd640 1 -- 172.21.15.3:0/64342936 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c03f030 con 0x7f589c1083c0 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f5892ffd640 1 -- 172.21.15.3:0/64342936 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f588c004490 con 0x7f589c1083c0 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f58a3d76640 1 -- 172.21.15.3:0/64342936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589c1a68c0 con 0x7f589c1083c0 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f5892ffd640 1 -- 172.21.15.3:0/64342936 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c035070 con 0x7f589c1083c0 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.741+0000 7f58a3d76640 1 -- 172.21.15.3:0/64342936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589c1a6df0 con 0x7f589c1083c0 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.742+0000 7f58a3d76640 1 -- 172.21.15.3:0/64342936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589c103d00 con 0x7f589c1083c0 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.744+0000 7f5892ffd640 1 -- 172.21.15.3:0/64342936 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f588c042030 con 0x7f589c1083c0 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.745+0000 7f5892ffd640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f58880b6670 0x7f58880b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:21.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.745+0000 7f58a1b12640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f58880b6670 0x7f58880b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:21.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.745+0000 7f5892ffd640 1 -- 172.21.15.3:0/64342936 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f588c107ee0 con 0x7f589c1083c0 2024-09-21T06:14:21.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.746+0000 7f58a1b12640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f58880b6670 0x7f58880b8b80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f589c06c6c0 tx=0x7f589800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:21.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.748+0000 7f5892ffd640 1 -- 172.21.15.3:0/64342936 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f588c0ce530 con 0x7f589c1083c0 2024-09-21T06:14:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:21 smithi137 ceph-mon[34464]: pgmap v305: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:21 smithi137 ceph-mon[34464]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:14:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:21 smithi204 ceph-mon[34659]: pgmap v305: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:21 smithi204 ceph-mon[34659]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:14:21.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.937+0000 7f58a3d76640 1 -- 172.21.15.3:0/64342936 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f589c063800 con 0x7f58880b6670 2024-09-21T06:14:21.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.941+0000 7f5892ffd640 1 -- 172.21.15.3:0/64342936 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259020 (secure 0 0 0) 0x7f589c063800 con 0x7f58880b6670 2024-09-21T06:14:21.948 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:21.949 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-21T06:14:21.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.944+0000 7f58a3d76640 1 -- 172.21.15.3:0/64342936 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f58880b6670 msgr2=0x7f58880b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:21.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.944+0000 7f58a3d76640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f58880b6670 0x7f58880b8b80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f589c06c6c0 tx=0x7f589800b410 comp rx=0 tx=0).stop 2024-09-21T06:14:21.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.944+0000 7f58a3d76640 1 -- 172.21.15.3:0/64342936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f589c1083c0 msgr2=0x7f589c19b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:21.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.944+0000 7f58a3d76640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f589c1083c0 0x7f589c19b8b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f588c032fd0 tx=0x7f588c004270 comp rx=0 tx=0).stop 2024-09-21T06:14:21.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.944+0000 7f58a3d76640 1 -- 172.21.15.3:0/64342936 shutdown_connections 2024-09-21T06:14:21.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.944+0000 7f58a3d76640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f58880b6670 0x7f58880b8b80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:21.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.944+0000 7f58a3d76640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f589c108d10 0x7f589c19bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:21.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.944+0000 7f58a3d76640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f589c1083c0 0x7f589c19b8b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:21.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.944+0000 7f58a3d76640 1 --2- 172.21.15.3:0/64342936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f589c102a80 0x7f589c19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:21.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.944+0000 7f58a3d76640 1 -- 172.21.15.3:0/64342936 >> 172.21.15.3:0/64342936 conn(0x7f589c079740 msgr2=0x7f589c104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.945+0000 7f58a3d76640 1 -- 172.21.15.3:0/64342936 shutdown_connections 2024-09-21T06:14:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:21.945+0000 7f58a3d76640 1 -- 172.21.15.3:0/64342936 wait complete. 2024-09-21T06:14:22.659 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-21T06:14:21.636010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664163,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":683,"num_read_kb":454,"num_write":372,"num_write_kb":2295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2953564,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359976,"kb_used_data":10716,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581848,"statfs":{"total":1151940427776,"available":1151571812352,"internally_reserved":0,"allocated":10973184,"data_stored":7103769,"data_compressed":177687,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003613"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607638+0000","last_change":"2024-09-21T06:13:10.515115+0000","last_active":"2024-09-21T06:13:24.607638+0000","last_peered":"2024-09-21T06:13:24.607638+0000","last_clean":"2024-09-21T06:13:24.607638+0000","last_became_active":"2024-09-21T06:13:10.514553+0000","last_became_peered":"2024-09-21T06:13:10.514553+0000","last_unstale":"2024-09-21T06:13:24.607638+0000","last_undegraded":"2024-09-21T06:13:24.607638+0000","last_fullsized":"2024-09-21T06:13:24.607638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:33.268926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609526+0000","last_change":"2024-09-21T06:13:10.511469+0000","last_active":"2024-09-21T06:13:24.609526+0000","last_peered":"2024-09-21T06:13:24.609526+0000","last_clean":"2024-09-21T06:13:24.609526+0000","last_became_active":"2024-09-21T06:13:10.511102+0000","last_became_peered":"2024-09-21T06:13:10.511102+0000","last_unstale":"2024-09-21T06:13:24.609526+0000","last_undegraded":"2024-09-21T06:13:24.609526+0000","last_fullsized":"2024-09-21T06:13:24.609526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:12.483821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609288+0000","last_change":"2024-09-21T06:13:10.520090+0000","last_active":"2024-09-21T06:13:24.609288+0000","last_peered":"2024-09-21T06:13:24.609288+0000","last_clean":"2024-09-21T06:13:24.609288+0000","last_became_active":"2024-09-21T06:13:10.519758+0000","last_became_peered":"2024-09-21T06:13:10.519758+0000","last_unstale":"2024-09-21T06:13:24.609288+0000","last_undegraded":"2024-09-21T06:13:24.609288+0000","last_fullsized":"2024-09-21T06:13:24.609288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:20.836847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.617812+0000","last_change":"2024-09-21T06:13:10.511732+0000","last_active":"2024-09-21T06:13:24.617812+0000","last_peered":"2024-09-21T06:13:24.617812+0000","last_clean":"2024-09-21T06:13:24.617812+0000","last_became_active":"2024-09-21T06:13:10.511377+0000","last_became_peered":"2024-09-21T06:13:10.511377+0000","last_unstale":"2024-09-21T06:13:24.617812+0000","last_undegraded":"2024-09-21T06:13:24.617812+0000","last_fullsized":"2024-09-21T06:13:24.617812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:18.500739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607690+0000","last_change":"2024-09-21T06:13:10.510464+0000","last_active":"2024-09-21T06:13:24.607690+0000","last_peered":"2024-09-21T06:13:24.607690+0000","last_clean":"2024-09-21T06:13:24.607690+0000","last_became_active":"2024-09-21T06:13:10.509979+0000","last_became_peered":"2024-09-21T06:13:10.509979+0000","last_unstale":"2024-09-21T06:13:24.607690+0000","last_undegraded":"2024-09-21T06:13:24.607690+0000","last_fullsized":"2024-09-21T06:13:24.607690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:56.871043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608937+0000","last_change":"2024-09-21T06:13:24.608832+0000","last_active":"2024-09-21T06:13:24.608937+0000","last_peered":"2024-09-21T06:13:24.608937+0000","last_clean":"2024-09-21T06:13:24.608937+0000","last_became_active":"2024-09-21T06:13:24.608439+0000","last_became_peered":"2024-09-21T06:13:24.608439+0000","last_unstale":"2024-09-21T06:13:24.608937+0000","last_undegraded":"2024-09-21T06:13:24.608937+0000","last_fullsized":"2024-09-21T06:13:24.608937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:41.515894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609585+0000","last_change":"2024-09-21T06:13:10.511391+0000","last_active":"2024-09-21T06:13:24.609585+0000","last_peered":"2024-09-21T06:13:24.609585+0000","last_clean":"2024-09-21T06:13:24.609585+0000","last_became_active":"2024-09-21T06:13:10.510733+0000","last_became_peered":"2024-09-21T06:13:10.510733+0000","last_unstale":"2024-09-21T06:13:24.609585+0000","last_undegraded":"2024-09-21T06:13:24.609585+0000","last_fullsized":"2024-09-21T06:13:24.609585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:24.071860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609673+0000","last_change":"2024-09-21T06:13:10.510886+0000","last_active":"2024-09-21T06:13:24.609673+0000","last_peered":"2024-09-21T06:13:24.609673+0000","last_clean":"2024-09-21T06:13:24.609673+0000","last_became_active":"2024-09-21T06:13:10.510159+0000","last_became_peered":"2024-09-21T06:13:10.510159+0000","last_unstale":"2024-09-21T06:13:24.609673+0000","last_undegraded":"2024-09-21T06:13:24.609673+0000","last_fullsized":"2024-09-21T06:13:24.609673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:37.124848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598718+0000","last_change":"2024-09-21T06:13:10.510466+0000","last_active":"2024-09-21T06:13:24.598718+0000","last_peered":"2024-09-21T06:13:24.598718+0000","last_clean":"2024-09-21T06:13:24.598718+0000","last_became_active":"2024-09-21T06:13:10.509928+0000","last_became_peered":"2024-09-21T06:13:10.509928+0000","last_unstale":"2024-09-21T06:13:24.598718+0000","last_undegraded":"2024-09-21T06:13:24.598718+0000","last_fullsized":"2024-09-21T06:13:24.598718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:06.156505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600080+0000","last_change":"2024-09-21T06:13:10.510622+0000","last_active":"2024-09-21T06:13:24.600080+0000","last_peered":"2024-09-21T06:13:24.600080+0000","last_clean":"2024-09-21T06:13:24.600080+0000","last_became_active":"2024-09-21T06:13:10.510206+0000","last_became_peered":"2024-09-21T06:13:10.510206+0000","last_unstale":"2024-09-21T06:13:24.600080+0000","last_undegraded":"2024-09-21T06:13:24.600080+0000","last_fullsized":"2024-09-21T06:13:24.600080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:03.223451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610031+0000","last_change":"2024-09-21T06:13:10.517525+0000","last_active":"2024-09-21T06:13:24.610031+0000","last_peered":"2024-09-21T06:13:24.610031+0000","last_clean":"2024-09-21T06:13:24.610031+0000","last_became_active":"2024-09-21T06:13:10.517144+0000","last_became_peered":"2024-09-21T06:13:10.517144+0000","last_unstale":"2024-09-21T06:13:24.610031+0000","last_undegraded":"2024-09-21T06:13:24.610031+0000","last_fullsized":"2024-09-21T06:13:24.610031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:46.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.617837+0000","last_change":"2024-09-21T06:13:10.515537+0000","last_active":"2024-09-21T06:13:24.617837+0000","last_peered":"2024-09-21T06:13:24.617837+0000","last_clean":"2024-09-21T06:13:24.617837+0000","last_became_active":"2024-09-21T06:13:10.515063+0000","last_became_peered":"2024-09-21T06:13:10.515063+0000","last_unstale":"2024-09-21T06:13:24.617837+0000","last_undegraded":"2024-09-21T06:13:24.617837+0000","last_fullsized":"2024-09-21T06:13:24.617837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:33.718471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597513+0000","last_change":"2024-09-21T06:13:10.509368+0000","last_active":"2024-09-21T06:13:24.597513+0000","last_peered":"2024-09-21T06:13:24.597513+0000","last_clean":"2024-09-21T06:13:24.597513+0000","last_became_active":"2024-09-21T06:13:10.508982+0000","last_became_peered":"2024-09-21T06:13:10.508982+0000","last_unstale":"2024-09-21T06:13:24.597513+0000","last_undegraded":"2024-09-21T06:13:24.597513+0000","last_fullsized":"2024-09-21T06:13:24.597513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:54.430889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608602+0000","last_change":"2024-09-21T06:13:10.506559+0000","last_active":"2024-09-21T06:13:24.608602+0000","last_peered":"2024-09-21T06:13:24.608602+0000","last_clean":"2024-09-21T06:13:24.608602+0000","last_became_active":"2024-09-21T06:13:10.506161+0000","last_became_peered":"2024-09-21T06:13:10.506161+0000","last_unstale":"2024-09-21T06:13:24.608602+0000","last_undegraded":"2024-09-21T06:13:24.608602+0000","last_fullsized":"2024-09-21T06:13:24.608602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:19.870292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608304+0000","last_change":"2024-09-21T06:13:10.516000+0000","last_active":"2024-09-21T06:13:24.608304+0000","last_peered":"2024-09-21T06:13:24.608304+0000","last_clean":"2024-09-21T06:13:24.608304+0000","last_became_active":"2024-09-21T06:13:10.515543+0000","last_became_peered":"2024-09-21T06:13:10.515543+0000","last_unstale":"2024-09-21T06:13:24.608304+0000","last_undegraded":"2024-09-21T06:13:24.608304+0000","last_fullsized":"2024-09-21T06:13:24.608304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:48.968489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598302+0000","last_change":"2024-09-21T06:13:10.513753+0000","last_active":"2024-09-21T06:13:24.598302+0000","last_peered":"2024-09-21T06:13:24.598302+0000","last_clean":"2024-09-21T06:13:24.598302+0000","last_became_active":"2024-09-21T06:13:10.511079+0000","last_became_peered":"2024-09-21T06:13:10.511079+0000","last_unstale":"2024-09-21T06:13:24.598302+0000","last_undegraded":"2024-09-21T06:13:24.598302+0000","last_fullsized":"2024-09-21T06:13:24.598302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:25.320041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607768+0000","last_change":"2024-09-21T06:13:10.515222+0000","last_active":"2024-09-21T06:13:24.607768+0000","last_peered":"2024-09-21T06:13:24.607768+0000","last_clean":"2024-09-21T06:13:24.607768+0000","last_became_active":"2024-09-21T06:13:10.514604+0000","last_became_peered":"2024-09-21T06:13:10.514604+0000","last_unstale":"2024-09-21T06:13:24.607768+0000","last_undegraded":"2024-09-21T06:13:24.607768+0000","last_fullsized":"2024-09-21T06:13:24.607768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:03.165695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600779+0000","last_change":"2024-09-21T06:13:10.518334+0000","last_active":"2024-09-21T06:13:24.600779+0000","last_peered":"2024-09-21T06:13:24.600779+0000","last_clean":"2024-09-21T06:13:24.600779+0000","last_became_active":"2024-09-21T06:13:10.517956+0000","last_became_peered":"2024-09-21T06:13:10.517956+0000","last_unstale":"2024-09-21T06:13:24.600779+0000","last_undegraded":"2024-09-21T06:13:24.600779+0000","last_fullsized":"2024-09-21T06:13:24.600779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:05.094454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604993+0000","last_change":"2024-09-21T06:13:10.508763+0000","last_active":"2024-09-21T06:13:24.604993+0000","last_peered":"2024-09-21T06:13:24.604993+0000","last_clean":"2024-09-21T06:13:24.604993+0000","last_became_active":"2024-09-21T06:13:10.508371+0000","last_became_peered":"2024-09-21T06:13:10.508371+0000","last_unstale":"2024-09-21T06:13:24.604993+0000","last_undegraded":"2024-09-21T06:13:24.604993+0000","last_fullsized":"2024-09-21T06:13:24.604993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:50.602692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607019+0000","last_change":"2024-09-21T06:13:10.503465+0000","last_active":"2024-09-21T06:13:24.607019+0000","last_peered":"2024-09-21T06:13:24.607019+0000","last_clean":"2024-09-21T06:13:24.607019+0000","last_became_active":"2024-09-21T06:13:10.503072+0000","last_became_peered":"2024-09-21T06:13:10.503072+0000","last_unstale":"2024-09-21T06:13:24.607019+0000","last_undegraded":"2024-09-21T06:13:24.607019+0000","last_fullsized":"2024-09-21T06:13:24.607019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:36.368987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597579+0000","last_change":"2024-09-21T06:13:10.512841+0000","last_active":"2024-09-21T06:13:24.597579+0000","last_peered":"2024-09-21T06:13:24.597579+0000","last_clean":"2024-09-21T06:13:24.597579+0000","last_became_active":"2024-09-21T06:13:10.512407+0000","last_became_peered":"2024-09-21T06:13:10.512407+0000","last_unstale":"2024-09-21T06:13:24.597579+0000","last_undegraded":"2024-09-21T06:13:24.597579+0000","last_fullsized":"2024-09-21T06:13:24.597579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:09.855608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598794+0000","last_change":"2024-09-21T06:13:10.510088+0000","last_active":"2024-09-21T06:13:24.598794+0000","last_peered":"2024-09-21T06:13:24.598794+0000","last_clean":"2024-09-21T06:13:24.598794+0000","last_became_active":"2024-09-21T06:13:10.507904+0000","last_became_peered":"2024-09-21T06:13:10.507904+0000","last_unstale":"2024-09-21T06:13:24.598794+0000","last_undegraded":"2024-09-21T06:13:24.598794+0000","last_fullsized":"2024-09-21T06:13:24.598794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:49.404402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.611267+0000","last_change":"2024-09-21T06:13:10.517720+0000","last_active":"2024-09-21T06:13:24.611267+0000","last_peered":"2024-09-21T06:13:24.611267+0000","last_clean":"2024-09-21T06:13:24.611267+0000","last_became_active":"2024-09-21T06:13:10.514641+0000","last_became_peered":"2024-09-21T06:13:10.514641+0000","last_unstale":"2024-09-21T06:13:24.611267+0000","last_undegraded":"2024-09-21T06:13:24.611267+0000","last_fullsized":"2024-09-21T06:13:24.611267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:40.639665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602839+0000","last_change":"2024-09-21T06:13:10.512234+0000","last_active":"2024-09-21T06:13:24.602839+0000","last_peered":"2024-09-21T06:13:24.602839+0000","last_clean":"2024-09-21T06:13:24.602839+0000","last_became_active":"2024-09-21T06:13:10.511594+0000","last_became_peered":"2024-09-21T06:13:10.511594+0000","last_unstale":"2024-09-21T06:13:24.602839+0000","last_undegraded":"2024-09-21T06:13:24.602839+0000","last_fullsized":"2024-09-21T06:13:24.602839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:04.410342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610455+0000","last_change":"2024-09-21T06:13:10.513216+0000","last_active":"2024-09-21T06:13:24.610455+0000","last_peered":"2024-09-21T06:13:24.610455+0000","last_clean":"2024-09-21T06:13:24.610455+0000","last_became_active":"2024-09-21T06:13:10.512830+0000","last_became_peered":"2024-09-21T06:13:10.512830+0000","last_unstale":"2024-09-21T06:13:24.610455+0000","last_undegraded":"2024-09-21T06:13:24.610455+0000","last_fullsized":"2024-09-21T06:13:24.610455+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:46.418161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.605196+0000","last_change":"2024-09-21T06:13:10.518863+0000","last_active":"2024-09-21T06:13:24.605196+0000","last_peered":"2024-09-21T06:13:24.605196+0000","last_clean":"2024-09-21T06:13:24.605196+0000","last_became_active":"2024-09-21T06:13:10.518492+0000","last_became_peered":"2024-09-21T06:13:10.518492+0000","last_unstale":"2024-09-21T06:13:24.605196+0000","last_undegraded":"2024-09-21T06:13:24.605196+0000","last_fullsized":"2024-09-21T06:13:24.605196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:23.770763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608235+0000","last_change":"2024-09-21T06:13:10.508538+0000","last_active":"2024-09-21T06:13:24.608235+0000","last_peered":"2024-09-21T06:13:24.608235+0000","last_clean":"2024-09-21T06:13:24.608235+0000","last_became_active":"2024-09-21T06:13:10.508017+0000","last_became_peered":"2024-09-21T06:13:10.508017+0000","last_unstale":"2024-09-21T06:13:24.608235+0000","last_undegraded":"2024-09-21T06:13:24.608235+0000","last_fullsized":"2024-09-21T06:13:24.608235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:00.168741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597567+0000","last_change":"2024-09-21T06:13:10.516734+0000","last_active":"2024-09-21T06:13:24.597567+0000","last_peered":"2024-09-21T06:13:24.597567+0000","last_clean":"2024-09-21T06:13:24.597567+0000","last_became_active":"2024-09-21T06:13:10.516362+0000","last_became_peered":"2024-09-21T06:13:10.516362+0000","last_unstale":"2024-09-21T06:13:24.597567+0000","last_undegraded":"2024-09-21T06:13:24.597567+0000","last_fullsized":"2024-09-21T06:13:24.597567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:49.792011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608841+0000","last_change":"2024-09-21T06:13:10.513143+0000","last_active":"2024-09-21T06:13:24.608841+0000","last_peered":"2024-09-21T06:13:24.608841+0000","last_clean":"2024-09-21T06:13:24.608841+0000","last_became_active":"2024-09-21T06:13:10.512338+0000","last_became_peered":"2024-09-21T06:13:10.512338+0000","last_unstale":"2024-09-21T06:13:24.608841+0000","last_undegraded":"2024-09-21T06:13:24.608841+0000","last_fullsized":"2024-09-21T06:13:24.608841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:02.963860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602980+0000","last_change":"2024-09-21T06:13:10.513429+0000","last_active":"2024-09-21T06:13:24.602980+0000","last_peered":"2024-09-21T06:13:24.602980+0000","last_clean":"2024-09-21T06:13:24.602980+0000","last_became_active":"2024-09-21T06:13:10.512995+0000","last_became_peered":"2024-09-21T06:13:10.512995+0000","last_unstale":"2024-09-21T06:13:24.602980+0000","last_undegraded":"2024-09-21T06:13:24.602980+0000","last_fullsized":"2024-09-21T06:13:24.602980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:39.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603628+0000","last_change":"2024-09-21T06:13:10.509580+0000","last_active":"2024-09-21T06:13:24.603628+0000","last_peered":"2024-09-21T06:13:24.603628+0000","last_clean":"2024-09-21T06:13:24.603628+0000","last_became_active":"2024-09-21T06:13:10.509179+0000","last_became_peered":"2024-09-21T06:13:10.509179+0000","last_unstale":"2024-09-21T06:13:24.603628+0000","last_undegraded":"2024-09-21T06:13:24.603628+0000","last_fullsized":"2024-09-21T06:13:24.603628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:39.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.606565+0000","last_change":"2024-09-21T06:13:10.508358+0000","last_active":"2024-09-21T06:13:24.606565+0000","last_peered":"2024-09-21T06:13:24.606565+0000","last_clean":"2024-09-21T06:13:24.606565+0000","last_became_active":"2024-09-21T06:13:10.507952+0000","last_became_peered":"2024-09-21T06:13:10.507952+0000","last_unstale":"2024-09-21T06:13:24.606565+0000","last_undegraded":"2024-09-21T06:13:24.606565+0000","last_fullsized":"2024-09-21T06:13:24.606565+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:55.565203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607615+0000","last_change":"2024-09-21T06:13:10.510580+0000","last_active":"2024-09-21T06:13:24.607615+0000","last_peered":"2024-09-21T06:13:24.607615+0000","last_clean":"2024-09-21T06:13:24.607615+0000","last_became_active":"2024-09-21T06:13:10.510175+0000","last_became_peered":"2024-09-21T06:13:10.510175+0000","last_unstale":"2024-09-21T06:13:24.607615+0000","last_undegraded":"2024-09-21T06:13:24.607615+0000","last_fullsized":"2024-09-21T06:13:24.607615+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:49.467057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":87,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:16.693979+0000","last_change":"2024-09-21T06:13:08.485527+0000","last_active":"2024-09-21T06:14:16.693979+0000","last_peered":"2024-09-21T06:14:16.693979+0000","last_clean":"2024-09-21T06:14:16.693979+0000","last_became_active":"2024-09-21T06:13:08.485094+0000","last_became_peered":"2024-09-21T06:13:08.485094+0000","last_unstale":"2024-09-21T06:14:16.693979+0000","last_undegraded":"2024-09-21T06:14:16.693979+0000","last_fullsized":"2024-09-21T06:14:16.693979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:48.285638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603582+0000","last_change":"2024-09-21T06:13:10.507977+0000","last_active":"2024-09-21T06:13:24.603582+0000","last_peered":"2024-09-21T06:13:24.603582+0000","last_clean":"2024-09-21T06:13:24.603582+0000","last_became_active":"2024-09-21T06:13:10.507594+0000","last_became_peered":"2024-09-21T06:13:10.507594+0000","last_unstale":"2024-09-21T06:13:24.603582+0000","last_undegraded":"2024-09-21T06:13:24.603582+0000","last_fullsized":"2024-09-21T06:13:24.603582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:57.318240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.599690+0000","last_change":"2024-09-21T06:13:08.491237+0000","last_active":"2024-09-21T06:13:24.599690+0000","last_peered":"2024-09-21T06:13:24.599690+0000","last_clean":"2024-09-21T06:13:24.599690+0000","last_became_active":"2024-09-21T06:13:08.490908+0000","last_became_peered":"2024-09-21T06:13:08.490908+0000","last_unstale":"2024-09-21T06:13:24.599690+0000","last_undegraded":"2024-09-21T06:13:24.599690+0000","last_fullsized":"2024-09-21T06:13:24.599690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:53.185029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597228+0000","last_change":"2024-09-21T06:13:08.488721+0000","last_active":"2024-09-21T06:13:24.597228+0000","last_peered":"2024-09-21T06:13:24.597228+0000","last_clean":"2024-09-21T06:13:24.597228+0000","last_became_active":"2024-09-21T06:13:08.488347+0000","last_became_peered":"2024-09-21T06:13:08.488347+0000","last_unstale":"2024-09-21T06:13:24.597228+0000","last_undegraded":"2024-09-21T06:13:24.597228+0000","last_fullsized":"2024-09-21T06:13:24.597228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:27.008479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609911+0000","last_change":"2024-09-21T06:13:10.506527+0000","last_active":"2024-09-21T06:13:24.609911+0000","last_peered":"2024-09-21T06:13:24.609911+0000","last_clean":"2024-09-21T06:13:24.609911+0000","last_became_active":"2024-09-21T06:13:10.506163+0000","last_became_peered":"2024-09-21T06:13:10.506163+0000","last_unstale":"2024-09-21T06:13:24.609911+0000","last_undegraded":"2024-09-21T06:13:24.609911+0000","last_fullsized":"2024-09-21T06:13:24.609911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.244097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604150+0000","last_change":"2024-09-21T06:13:08.489292+0000","last_active":"2024-09-21T06:13:24.604150+0000","last_peered":"2024-09-21T06:13:24.604150+0000","last_clean":"2024-09-21T06:13:24.604150+0000","last_became_active":"2024-09-21T06:13:08.488918+0000","last_became_peered":"2024-09-21T06:13:08.488918+0000","last_unstale":"2024-09-21T06:13:24.604150+0000","last_undegraded":"2024-09-21T06:13:24.604150+0000","last_fullsized":"2024-09-21T06:13:24.604150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:37.992260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600648+0000","last_change":"2024-09-21T06:13:10.505027+0000","last_active":"2024-09-21T06:13:24.600648+0000","last_peered":"2024-09-21T06:13:24.600648+0000","last_clean":"2024-09-21T06:13:24.600648+0000","last_became_active":"2024-09-21T06:13:10.504669+0000","last_became_peered":"2024-09-21T06:13:10.504669+0000","last_unstale":"2024-09-21T06:13:24.600648+0000","last_undegraded":"2024-09-21T06:13:24.600648+0000","last_fullsized":"2024-09-21T06:13:24.600648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:43.927229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":142,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:20.699467+0000","last_change":"2024-09-21T06:13:08.491101+0000","last_active":"2024-09-21T06:14:20.699467+0000","last_peered":"2024-09-21T06:14:20.699467+0000","last_clean":"2024-09-21T06:14:20.699467+0000","last_became_active":"2024-09-21T06:13:08.490607+0000","last_became_peered":"2024-09-21T06:13:08.490607+0000","last_unstale":"2024-09-21T06:14:20.699467+0000","last_undegraded":"2024-09-21T06:14:20.699467+0000","last_fullsized":"2024-09-21T06:14:20.699467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:16.758463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598830+0000","last_change":"2024-09-21T06:13:10.510453+0000","last_active":"2024-09-21T06:13:24.598830+0000","last_peered":"2024-09-21T06:13:24.598830+0000","last_clean":"2024-09-21T06:13:24.598830+0000","last_became_active":"2024-09-21T06:13:10.508513+0000","last_became_peered":"2024-09-21T06:13:10.508513+0000","last_unstale":"2024-09-21T06:13:24.598830+0000","last_undegraded":"2024-09-21T06:13:24.598830+0000","last_fullsized":"2024-09-21T06:13:24.598830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:48.388140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609878+0000","last_change":"2024-09-21T06:13:08.488714+0000","last_active":"2024-09-21T06:13:24.609878+0000","last_peered":"2024-09-21T06:13:24.609878+0000","last_clean":"2024-09-21T06:13:24.609878+0000","last_became_active":"2024-09-21T06:13:08.488363+0000","last_became_peered":"2024-09-21T06:13:08.488363+0000","last_unstale":"2024-09-21T06:13:24.609878+0000","last_undegraded":"2024-09-21T06:13:24.609878+0000","last_fullsized":"2024-09-21T06:13:24.609878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:33.626165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609602+0000","last_change":"2024-09-21T06:13:10.508046+0000","last_active":"2024-09-21T06:13:24.609602+0000","last_peered":"2024-09-21T06:13:24.609602+0000","last_clean":"2024-09-21T06:13:24.609602+0000","last_became_active":"2024-09-21T06:13:10.507660+0000","last_became_peered":"2024-09-21T06:13:10.507660+0000","last_unstale":"2024-09-21T06:13:24.609602+0000","last_undegraded":"2024-09-21T06:13:24.609602+0000","last_fullsized":"2024-09-21T06:13:24.609602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:35.466965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603272+0000","last_change":"2024-09-21T06:13:08.562863+0000","last_active":"2024-09-21T06:13:24.603272+0000","last_peered":"2024-09-21T06:13:24.603272+0000","last_clean":"2024-09-21T06:13:24.603272+0000","last_became_active":"2024-09-21T06:13:08.562424+0000","last_became_peered":"2024-09-21T06:13:08.562424+0000","last_unstale":"2024-09-21T06:13:24.603272+0000","last_undegraded":"2024-09-21T06:13:24.603272+0000","last_fullsized":"2024-09-21T06:13:24.603272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:43.251932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598595+0000","last_change":"2024-09-21T06:13:10.509818+0000","last_active":"2024-09-21T06:13:24.598595+0000","last_peered":"2024-09-21T06:13:24.598595+0000","last_clean":"2024-09-21T06:13:24.598595+0000","last_became_active":"2024-09-21T06:13:10.509468+0000","last_became_peered":"2024-09-21T06:13:10.509468+0000","last_unstale":"2024-09-21T06:13:24.598595+0000","last_undegraded":"2024-09-21T06:13:24.598595+0000","last_fullsized":"2024-09-21T06:13:24.598595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:25.130868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'31","reported_seq":197,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:20.699471+0000","last_change":"2024-09-21T06:13:24.633807+0000","last_active":"2024-09-21T06:14:20.699471+0000","last_peered":"2024-09-21T06:14:20.699471+0000","last_clean":"2024-09-21T06:14:20.699471+0000","last_became_active":"2024-09-21T06:13:24.618599+0000","last_became_peered":"2024-09-21T06:13:24.618599+0000","last_unstale":"2024-09-21T06:14:20.699471+0000","last_undegraded":"2024-09-21T06:14:20.699471+0000","last_fullsized":"2024-09-21T06:14:20.699471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:07.449253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65966,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":122,"num_read_kb":124,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608537+0000","last_change":"2024-09-21T06:13:10.511360+0000","last_active":"2024-09-21T06:13:24.608537+0000","last_peered":"2024-09-21T06:13:24.608537+0000","last_clean":"2024-09-21T06:13:24.608537+0000","last_became_active":"2024-09-21T06:13:10.510980+0000","last_became_peered":"2024-09-21T06:13:10.510980+0000","last_unstale":"2024-09-21T06:13:24.608537+0000","last_undegraded":"2024-09-21T06:13:24.608537+0000","last_fullsized":"2024-09-21T06:13:24.608537+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:43.859758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604427+0000","last_change":"2024-09-21T06:13:08.486262+0000","last_active":"2024-09-21T06:13:24.604427+0000","last_peered":"2024-09-21T06:13:24.604427+0000","last_clean":"2024-09-21T06:13:24.604427+0000","last_became_active":"2024-09-21T06:13:08.485509+0000","last_became_peered":"2024-09-21T06:13:08.485509+0000","last_unstale":"2024-09-21T06:13:24.604427+0000","last_undegraded":"2024-09-21T06:13:24.604427+0000","last_fullsized":"2024-09-21T06:13:24.604427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:31.726746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597324+0000","last_change":"2024-09-21T06:13:10.506894+0000","last_active":"2024-09-21T06:13:24.597324+0000","last_peered":"2024-09-21T06:13:24.597324+0000","last_clean":"2024-09-21T06:13:24.597324+0000","last_became_active":"2024-09-21T06:13:10.506489+0000","last_became_peered":"2024-09-21T06:13:10.506489+0000","last_unstale":"2024-09-21T06:13:24.597324+0000","last_undegraded":"2024-09-21T06:13:24.597324+0000","last_fullsized":"2024-09-21T06:13:24.597324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:58.755601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603141+0000","last_change":"2024-09-21T06:13:08.483429+0000","last_active":"2024-09-21T06:13:24.603141+0000","last_peered":"2024-09-21T06:13:24.603141+0000","last_clean":"2024-09-21T06:13:24.603141+0000","last_became_active":"2024-09-21T06:13:08.483028+0000","last_became_peered":"2024-09-21T06:13:08.483028+0000","last_unstale":"2024-09-21T06:13:24.603141+0000","last_undegraded":"2024-09-21T06:13:24.603141+0000","last_fullsized":"2024-09-21T06:13:24.603141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:33.258154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609758+0000","last_change":"2024-09-21T06:13:10.513398+0000","last_active":"2024-09-21T06:13:24.609758+0000","last_peered":"2024-09-21T06:13:24.609758+0000","last_clean":"2024-09-21T06:13:24.609758+0000","last_became_active":"2024-09-21T06:13:10.512713+0000","last_became_peered":"2024-09-21T06:13:10.512713+0000","last_unstale":"2024-09-21T06:13:24.609758+0000","last_undegraded":"2024-09-21T06:13:24.609758+0000","last_fullsized":"2024-09-21T06:13:24.609758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:21.026984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":85,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:15.692383+0000","last_change":"2024-09-21T06:13:08.493771+0000","last_active":"2024-09-21T06:14:15.692383+0000","last_peered":"2024-09-21T06:14:15.692383+0000","last_clean":"2024-09-21T06:14:15.692383+0000","last_became_active":"2024-09-21T06:13:08.493380+0000","last_became_peered":"2024-09-21T06:13:08.493380+0000","last_unstale":"2024-09-21T06:14:15.692383+0000","last_undegraded":"2024-09-21T06:14:15.692383+0000","last_fullsized":"2024-09-21T06:14:15.692383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:12.552137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609811+0000","last_change":"2024-09-21T06:13:10.511679+0000","last_active":"2024-09-21T06:13:24.609811+0000","last_peered":"2024-09-21T06:13:24.609811+0000","last_clean":"2024-09-21T06:13:24.609811+0000","last_became_active":"2024-09-21T06:13:10.511270+0000","last_became_peered":"2024-09-21T06:13:10.511270+0000","last_unstale":"2024-09-21T06:13:24.609811+0000","last_undegraded":"2024-09-21T06:13:24.609811+0000","last_fullsized":"2024-09-21T06:13:24.609811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:10.521853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.627560+0000","last_change":"2024-09-21T06:13:24.627467+0000","last_active":"2024-09-21T06:13:24.627560+0000","last_peered":"2024-09-21T06:13:24.627560+0000","last_clean":"2024-09-21T06:13:24.627560+0000","last_became_active":"2024-09-21T06:13:24.610263+0000","last_became_peered":"2024-09-21T06:13:24.610263+0000","last_unstale":"2024-09-21T06:13:24.627560+0000","last_undegraded":"2024-09-21T06:13:24.627560+0000","last_fullsized":"2024-09-21T06:13:24.627560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:26.472147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602912+0000","last_change":"2024-09-21T06:13:10.517214+0000","last_active":"2024-09-21T06:13:24.602912+0000","last_peered":"2024-09-21T06:13:24.602912+0000","last_clean":"2024-09-21T06:13:24.602912+0000","last_became_active":"2024-09-21T06:13:10.516828+0000","last_became_peered":"2024-09-21T06:13:10.516828+0000","last_unstale":"2024-09-21T06:13:24.602912+0000","last_undegraded":"2024-09-21T06:13:24.602912+0000","last_fullsized":"2024-09-21T06:13:24.602912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:58.077670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608703+0000","last_change":"2024-09-21T06:13:08.489153+0000","last_active":"2024-09-21T06:13:24.608703+0000","last_peered":"2024-09-21T06:13:24.608703+0000","last_clean":"2024-09-21T06:13:24.608703+0000","last_became_active":"2024-09-21T06:13:08.488732+0000","last_became_peered":"2024-09-21T06:13:08.488732+0000","last_unstale":"2024-09-21T06:13:24.608703+0000","last_undegraded":"2024-09-21T06:13:24.608703+0000","last_fullsized":"2024-09-21T06:13:24.608703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:22.848136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610561+0000","last_change":"2024-09-21T06:13:24.610441+0000","last_active":"2024-09-21T06:13:24.610561+0000","last_peered":"2024-09-21T06:13:24.610561+0000","last_clean":"2024-09-21T06:13:24.610561+0000","last_became_active":"2024-09-21T06:13:24.609992+0000","last_became_peered":"2024-09-21T06:13:24.609992+0000","last_unstale":"2024-09-21T06:13:24.610561+0000","last_undegraded":"2024-09-21T06:13:24.610561+0000","last_fullsized":"2024-09-21T06:13:24.610561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:59.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597725+0000","last_change":"2024-09-21T06:13:08.484514+0000","last_active":"2024-09-21T06:13:24.597725+0000","last_peered":"2024-09-21T06:13:24.597725+0000","last_clean":"2024-09-21T06:13:24.597725+0000","last_became_active":"2024-09-21T06:13:08.484134+0000","last_became_peered":"2024-09-21T06:13:08.484134+0000","last_unstale":"2024-09-21T06:13:24.597725+0000","last_undegraded":"2024-09-21T06:13:24.597725+0000","last_fullsized":"2024-09-21T06:13:24.597725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:45.124734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610027+0000","last_change":"2024-09-21T06:13:10.503798+0000","last_active":"2024-09-21T06:13:24.610027+0000","last_peered":"2024-09-21T06:13:24.610027+0000","last_clean":"2024-09-21T06:13:24.610027+0000","last_became_active":"2024-09-21T06:13:10.502418+0000","last_became_peered":"2024-09-21T06:13:10.502418+0000","last_unstale":"2024-09-21T06:13:24.610027+0000","last_undegraded":"2024-09-21T06:13:24.610027+0000","last_fullsized":"2024-09-21T06:13:24.610027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:57.575569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603439+0000","last_change":"2024-09-21T06:13:08.487191+0000","last_active":"2024-09-21T06:13:24.603439+0000","last_peered":"2024-09-21T06:13:24.603439+0000","last_clean":"2024-09-21T06:13:24.603439+0000","last_became_active":"2024-09-21T06:13:08.486812+0000","last_became_peered":"2024-09-21T06:13:08.486812+0000","last_unstale":"2024-09-21T06:13:24.603439+0000","last_undegraded":"2024-09-21T06:13:24.603439+0000","last_fullsized":"2024-09-21T06:13:24.603439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:51.989708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610179+0000","last_change":"2024-09-21T06:13:10.511678+0000","last_active":"2024-09-21T06:13:24.610179+0000","last_peered":"2024-09-21T06:13:24.610179+0000","last_clean":"2024-09-21T06:13:24.610179+0000","last_became_active":"2024-09-21T06:13:10.510408+0000","last_became_peered":"2024-09-21T06:13:10.510408+0000","last_unstale":"2024-09-21T06:13:24.610179+0000","last_undegraded":"2024-09-21T06:13:24.610179+0000","last_fullsized":"2024-09-21T06:13:24.610179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:58.711190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'6","reported_seq":87,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:17.695371+0000","last_change":"2024-09-21T06:13:08.560894+0000","last_active":"2024-09-21T06:14:17.695371+0000","last_peered":"2024-09-21T06:14:17.695371+0000","last_clean":"2024-09-21T06:14:17.695371+0000","last_became_active":"2024-09-21T06:13:08.560423+0000","last_became_peered":"2024-09-21T06:13:08.560423+0000","last_unstale":"2024-09-21T06:14:17.695371+0000","last_undegraded":"2024-09-21T06:14:17.695371+0000","last_fullsized":"2024-09-21T06:14:17.695371+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:53.015711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610081+0000","last_change":"2024-09-21T06:13:10.516009+0000","last_active":"2024-09-21T06:13:24.610081+0000","last_peered":"2024-09-21T06:13:24.610081+0000","last_clean":"2024-09-21T06:13:24.610081+0000","last_became_active":"2024-09-21T06:13:10.515669+0000","last_became_peered":"2024-09-21T06:13:10.515669+0000","last_unstale":"2024-09-21T06:13:24.610081+0000","last_undegraded":"2024-09-21T06:13:24.610081+0000","last_fullsized":"2024-09-21T06:13:24.610081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:54.134564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":99,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610317+0000","last_change":"2024-09-21T06:11:53.408030+0000","last_active":"2024-09-21T06:13:24.610317+0000","last_peered":"2024-09-21T06:13:24.610317+0000","last_clean":"2024-09-21T06:13:24.610317+0000","last_became_active":"2024-09-21T06:11:53.402296+0000","last_became_peered":"2024-09-21T06:11:53.402296+0000","last_unstale":"2024-09-21T06:13:24.610317+0000","last_undegraded":"2024-09-21T06:13:24.610317+0000","last_fullsized":"2024-09-21T06:13:24.610317+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_clean_scrub_stamp":"2024-09-21T06:08:26.098183+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:08.239872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616959+0000","last_change":"2024-09-21T06:13:08.489209+0000","last_active":"2024-09-21T06:13:24.616959+0000","last_peered":"2024-09-21T06:13:24.616959+0000","last_clean":"2024-09-21T06:13:24.616959+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.616959+0000","last_undegraded":"2024-09-21T06:13:24.616959+0000","last_fullsized":"2024-09-21T06:13:24.616959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:38.798499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607003+0000","last_change":"2024-09-21T06:13:24.606487+0000","last_active":"2024-09-21T06:13:24.607003+0000","last_peered":"2024-09-21T06:13:24.607003+0000","last_clean":"2024-09-21T06:13:24.607003+0000","last_became_active":"2024-09-21T06:13:24.606141+0000","last_became_peered":"2024-09-21T06:13:24.606141+0000","last_unstale":"2024-09-21T06:13:24.607003+0000","last_undegraded":"2024-09-21T06:13:24.607003+0000","last_fullsized":"2024-09-21T06:13:24.607003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:24.040770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609976+0000","last_change":"2024-09-21T06:13:08.489206+0000","last_active":"2024-09-21T06:13:24.609976+0000","last_peered":"2024-09-21T06:13:24.609976+0000","last_clean":"2024-09-21T06:13:24.609976+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.609976+0000","last_undegraded":"2024-09-21T06:13:24.609976+0000","last_fullsized":"2024-09-21T06:13:24.609976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.627305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597214+0000","last_change":"2024-09-21T06:13:10.513999+0000","last_active":"2024-09-21T06:13:24.597214+0000","last_peered":"2024-09-21T06:13:24.597214+0000","last_clean":"2024-09-21T06:13:24.597214+0000","last_became_active":"2024-09-21T06:13:10.513657+0000","last_became_peered":"2024-09-21T06:13:10.513657+0000","last_unstale":"2024-09-21T06:13:24.597214+0000","last_undegraded":"2024-09-21T06:13:24.597214+0000","last_fullsized":"2024-09-21T06:13:24.597214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:05.536921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'5","reported_seq":84,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:17.694312+0000","last_change":"2024-09-21T06:13:08.560811+0000","last_active":"2024-09-21T06:14:17.694312+0000","last_peered":"2024-09-21T06:14:17.694312+0000","last_clean":"2024-09-21T06:14:17.694312+0000","last_became_active":"2024-09-21T06:13:08.560145+0000","last_became_peered":"2024-09-21T06:13:08.560145+0000","last_unstale":"2024-09-21T06:14:17.694312+0000","last_undegraded":"2024-09-21T06:14:17.694312+0000","last_fullsized":"2024-09-21T06:14:17.694312+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:24.550008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.613627+0000","last_change":"2024-09-21T06:13:24.613495+0000","last_active":"2024-09-21T06:13:24.613627+0000","last_peered":"2024-09-21T06:13:24.613627+0000","last_clean":"2024-09-21T06:13:24.613627+0000","last_became_active":"2024-09-21T06:13:24.612634+0000","last_became_peered":"2024-09-21T06:13:24.612634+0000","last_unstale":"2024-09-21T06:13:24.613627+0000","last_undegraded":"2024-09-21T06:13:24.613627+0000","last_fullsized":"2024-09-21T06:13:24.613627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.863141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'5","reported_seq":85,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:17.691999+0000","last_change":"2024-09-21T06:13:08.486973+0000","last_active":"2024-09-21T06:14:17.691999+0000","last_peered":"2024-09-21T06:14:17.691999+0000","last_clean":"2024-09-21T06:14:17.691999+0000","last_became_active":"2024-09-21T06:13:08.486602+0000","last_became_peered":"2024-09-21T06:13:08.486602+0000","last_unstale":"2024-09-21T06:14:17.691999+0000","last_undegraded":"2024-09-21T06:14:17.691999+0000","last_fullsized":"2024-09-21T06:14:17.691999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:41.804955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.599893+0000","last_change":"2024-09-21T06:13:10.509499+0000","last_active":"2024-09-21T06:13:24.599893+0000","last_peered":"2024-09-21T06:13:24.599893+0000","last_clean":"2024-09-21T06:13:24.599893+0000","last_became_active":"2024-09-21T06:13:10.508566+0000","last_became_peered":"2024-09-21T06:13:10.508566+0000","last_unstale":"2024-09-21T06:13:24.599893+0000","last_undegraded":"2024-09-21T06:13:24.599893+0000","last_fullsized":"2024-09-21T06:13:24.599893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:12.701631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600737+0000","last_change":"2024-09-21T06:13:08.561378+0000","last_active":"2024-09-21T06:13:24.600737+0000","last_peered":"2024-09-21T06:13:24.600737+0000","last_clean":"2024-09-21T06:13:24.600737+0000","last_became_active":"2024-09-21T06:13:08.561002+0000","last_became_peered":"2024-09-21T06:13:08.561002+0000","last_unstale":"2024-09-21T06:13:24.600737+0000","last_undegraded":"2024-09-21T06:13:24.600737+0000","last_fullsized":"2024-09-21T06:13:24.600737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:11.325377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.602794+0000","last_change":"2024-09-21T06:13:24.602347+0000","last_active":"2024-09-21T06:13:24.602794+0000","last_peered":"2024-09-21T06:13:24.602794+0000","last_clean":"2024-09-21T06:13:24.602794+0000","last_became_active":"2024-09-21T06:13:24.601978+0000","last_became_peered":"2024-09-21T06:13:24.601978+0000","last_unstale":"2024-09-21T06:13:24.602794+0000","last_undegraded":"2024-09-21T06:13:24.602794+0000","last_fullsized":"2024-09-21T06:13:24.602794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:34.860096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608569+0000","last_change":"2024-09-21T06:13:08.561937+0000","last_active":"2024-09-21T06:13:24.608569+0000","last_peered":"2024-09-21T06:13:24.608569+0000","last_clean":"2024-09-21T06:13:24.608569+0000","last_became_active":"2024-09-21T06:13:08.560547+0000","last_became_peered":"2024-09-21T06:13:08.560547+0000","last_unstale":"2024-09-21T06:13:24.608569+0000","last_undegraded":"2024-09-21T06:13:24.608569+0000","last_fullsized":"2024-09-21T06:13:24.608569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:29.702291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597628+0000","last_change":"2024-09-21T06:13:10.516833+0000","last_active":"2024-09-21T06:13:24.597628+0000","last_peered":"2024-09-21T06:13:24.597628+0000","last_clean":"2024-09-21T06:13:24.597628+0000","last_became_active":"2024-09-21T06:13:10.516486+0000","last_became_peered":"2024-09-21T06:13:10.516486+0000","last_unstale":"2024-09-21T06:13:24.597628+0000","last_undegraded":"2024-09-21T06:13:24.597628+0000","last_fullsized":"2024-09-21T06:13:24.597628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:05.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'16","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:37.499820+0000","last_change":"2024-09-21T06:13:08.483271+0000","last_active":"2024-09-21T06:13:37.499820+0000","last_peered":"2024-09-21T06:13:37.499820+0000","last_clean":"2024-09-21T06:13:37.499820+0000","last_became_active":"2024-09-21T06:13:08.482951+0000","last_became_peered":"2024-09-21T06:13:08.482951+0000","last_unstale":"2024-09-21T06:13:37.499820+0000","last_undegraded":"2024-09-21T06:13:37.499820+0000","last_fullsized":"2024-09-21T06:13:37.499820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:17.537988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610524+0000","last_change":"2024-09-21T06:13:10.509125+0000","last_active":"2024-09-21T06:13:24.610524+0000","last_peered":"2024-09-21T06:13:24.610524+0000","last_clean":"2024-09-21T06:13:24.610524+0000","last_became_active":"2024-09-21T06:13:10.508773+0000","last_became_peered":"2024-09-21T06:13:10.508773+0000","last_unstale":"2024-09-21T06:13:24.610524+0000","last_undegraded":"2024-09-21T06:13:24.610524+0000","last_fullsized":"2024-09-21T06:13:24.610524+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:44.570477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616932+0000","last_change":"2024-09-21T06:13:08.497885+0000","last_active":"2024-09-21T06:13:24.616932+0000","last_peered":"2024-09-21T06:13:24.616932+0000","last_clean":"2024-09-21T06:13:24.616932+0000","last_became_active":"2024-09-21T06:13:08.497593+0000","last_became_peered":"2024-09-21T06:13:08.497593+0000","last_unstale":"2024-09-21T06:13:24.616932+0000","last_undegraded":"2024-09-21T06:13:24.616932+0000","last_fullsized":"2024-09-21T06:13:24.616932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:48.361175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609527+0000","last_change":"2024-09-21T06:13:10.514430+0000","last_active":"2024-09-21T06:13:24.609527+0000","last_peered":"2024-09-21T06:13:24.609527+0000","last_clean":"2024-09-21T06:13:24.609527+0000","last_became_active":"2024-09-21T06:13:10.514063+0000","last_became_peered":"2024-09-21T06:13:10.514063+0000","last_unstale":"2024-09-21T06:13:24.609527+0000","last_undegraded":"2024-09-21T06:13:24.609527+0000","last_fullsized":"2024-09-21T06:13:24.609527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:37.036497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598415+0000","last_change":"2024-09-21T06:13:08.560564+0000","last_active":"2024-09-21T06:13:24.598415+0000","last_peered":"2024-09-21T06:13:24.598415+0000","last_clean":"2024-09-21T06:13:24.598415+0000","last_became_active":"2024-09-21T06:13:08.560339+0000","last_became_peered":"2024-09-21T06:13:08.560339+0000","last_unstale":"2024-09-21T06:13:24.598415+0000","last_undegraded":"2024-09-21T06:13:24.598415+0000","last_fullsized":"2024-09-21T06:13:24.598415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:55.378911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.605861+0000","last_change":"2024-09-21T06:13:10.511354+0000","last_active":"2024-09-21T06:13:24.605861+0000","last_peered":"2024-09-21T06:13:24.605861+0000","last_clean":"2024-09-21T06:13:24.605861+0000","last_became_active":"2024-09-21T06:13:10.510946+0000","last_became_peered":"2024-09-21T06:13:10.510946+0000","last_unstale":"2024-09-21T06:13:24.605861+0000","last_undegraded":"2024-09-21T06:13:24.605861+0000","last_fullsized":"2024-09-21T06:13:24.605861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:11.535461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610883+0000","last_change":"2024-09-21T06:13:08.485578+0000","last_active":"2024-09-21T06:13:24.610883+0000","last_peered":"2024-09-21T06:13:24.610883+0000","last_clean":"2024-09-21T06:13:24.610883+0000","last_became_active":"2024-09-21T06:13:08.485170+0000","last_became_peered":"2024-09-21T06:13:08.485170+0000","last_unstale":"2024-09-21T06:13:24.610883+0000","last_undegraded":"2024-09-21T06:13:24.610883+0000","last_fullsized":"2024-09-21T06:13:24.610883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:39.000114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.606691+0000","last_change":"2024-09-21T06:13:10.510360+0000","last_active":"2024-09-21T06:13:24.606691+0000","last_peered":"2024-09-21T06:13:24.606691+0000","last_clean":"2024-09-21T06:13:24.606691+0000","last_became_active":"2024-09-21T06:13:10.509887+0000","last_became_peered":"2024-09-21T06:13:10.509887+0000","last_unstale":"2024-09-21T06:13:24.606691+0000","last_undegraded":"2024-09-21T06:13:24.606691+0000","last_fullsized":"2024-09-21T06:13:24.606691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:58.135706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.625360+0000","last_change":"2024-09-21T06:13:24.625284+0000","last_active":"2024-09-21T06:13:24.625360+0000","last_peered":"2024-09-21T06:13:24.625360+0000","last_clean":"2024-09-21T06:13:24.625360+0000","last_became_active":"2024-09-21T06:13:24.612704+0000","last_became_peered":"2024-09-21T06:13:24.612704+0000","last_unstale":"2024-09-21T06:13:24.625360+0000","last_undegraded":"2024-09-21T06:13:24.625360+0000","last_fullsized":"2024-09-21T06:13:24.625360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:29.385258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.611377+0000","last_change":"2024-09-21T06:13:10.514128+0000","last_active":"2024-09-21T06:13:24.611377+0000","last_peered":"2024-09-21T06:13:24.611377+0000","last_clean":"2024-09-21T06:13:24.611377+0000","last_became_active":"2024-09-21T06:13:10.513790+0000","last_became_peered":"2024-09-21T06:13:10.513790+0000","last_unstale":"2024-09-21T06:13:24.611377+0000","last_undegraded":"2024-09-21T06:13:24.611377+0000","last_fullsized":"2024-09-21T06:13:24.611377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:33.407189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607762+0000","last_change":"2024-09-21T06:13:08.489105+0000","last_active":"2024-09-21T06:13:24.607762+0000","last_peered":"2024-09-21T06:13:24.607762+0000","last_clean":"2024-09-21T06:13:24.607762+0000","last_became_active":"2024-09-21T06:13:08.487110+0000","last_became_peered":"2024-09-21T06:13:08.487110+0000","last_unstale":"2024-09-21T06:13:24.607762+0000","last_undegraded":"2024-09-21T06:13:24.607762+0000","last_fullsized":"2024-09-21T06:13:24.607762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:24.849365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610194+0000","last_change":"2024-09-21T06:13:10.513751+0000","last_active":"2024-09-21T06:13:24.610194+0000","last_peered":"2024-09-21T06:13:24.610194+0000","last_clean":"2024-09-21T06:13:24.610194+0000","last_became_active":"2024-09-21T06:13:10.513376+0000","last_became_peered":"2024-09-21T06:13:10.513376+0000","last_unstale":"2024-09-21T06:13:24.610194+0000","last_undegraded":"2024-09-21T06:13:24.610194+0000","last_fullsized":"2024-09-21T06:13:24.610194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.364959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603110+0000","last_change":"2024-09-21T06:13:10.519359+0000","last_active":"2024-09-21T06:13:24.603110+0000","last_peered":"2024-09-21T06:13:24.603110+0000","last_clean":"2024-09-21T06:13:24.603110+0000","last_became_active":"2024-09-21T06:13:10.518518+0000","last_became_peered":"2024-09-21T06:13:10.518518+0000","last_unstale":"2024-09-21T06:13:24.603110+0000","last_undegraded":"2024-09-21T06:13:24.603110+0000","last_fullsized":"2024-09-21T06:13:24.603110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:55.554710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608166+0000","last_change":"2024-09-21T06:13:08.488122+0000","last_active":"2024-09-21T06:13:24.608166+0000","last_peered":"2024-09-21T06:13:24.608166+0000","last_clean":"2024-09-21T06:13:24.608166+0000","last_became_active":"2024-09-21T06:13:08.487679+0000","last_became_peered":"2024-09-21T06:13:08.487679+0000","last_unstale":"2024-09-21T06:13:24.608166+0000","last_undegraded":"2024-09-21T06:13:24.608166+0000","last_fullsized":"2024-09-21T06:13:24.608166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:12.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598434+0000","last_change":"2024-09-21T06:13:10.516794+0000","last_active":"2024-09-21T06:13:24.598434+0000","last_peered":"2024-09-21T06:13:24.598434+0000","last_clean":"2024-09-21T06:13:24.598434+0000","last_became_active":"2024-09-21T06:13:10.516297+0000","last_became_peered":"2024-09-21T06:13:10.516297+0000","last_unstale":"2024-09-21T06:13:24.598434+0000","last_undegraded":"2024-09-21T06:13:24.598434+0000","last_fullsized":"2024-09-21T06:13:24.598434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:19.873742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.621488+0000","last_change":"2024-09-21T06:13:24.621358+0000","last_active":"2024-09-21T06:13:24.621488+0000","last_peered":"2024-09-21T06:13:24.621488+0000","last_clean":"2024-09-21T06:13:24.621488+0000","last_became_active":"2024-09-21T06:13:24.611058+0000","last_became_peered":"2024-09-21T06:13:24.611058+0000","last_unstale":"2024-09-21T06:13:24.621488+0000","last_undegraded":"2024-09-21T06:13:24.621488+0000","last_fullsized":"2024-09-21T06:13:24.621488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:28.759029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609712+0000","last_change":"2024-09-21T06:13:10.516339+0000","last_active":"2024-09-21T06:13:24.609712+0000","last_peered":"2024-09-21T06:13:24.609712+0000","last_clean":"2024-09-21T06:13:24.609712+0000","last_became_active":"2024-09-21T06:13:10.515397+0000","last_became_peered":"2024-09-21T06:13:10.515397+0000","last_unstale":"2024-09-21T06:13:24.609712+0000","last_undegraded":"2024-09-21T06:13:24.609712+0000","last_fullsized":"2024-09-21T06:13:24.609712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:56.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610156+0000","last_change":"2024-09-21T06:13:08.560494+0000","last_active":"2024-09-21T06:13:24.610156+0000","last_peered":"2024-09-21T06:13:24.610156+0000","last_clean":"2024-09-21T06:13:24.610156+0000","last_became_active":"2024-09-21T06:13:08.560118+0000","last_became_peered":"2024-09-21T06:13:08.560118+0000","last_unstale":"2024-09-21T06:13:24.610156+0000","last_undegraded":"2024-09-21T06:13:24.610156+0000","last_fullsized":"2024-09-21T06:13:24.610156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:05.597233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610032+0000","last_change":"2024-09-21T06:13:10.515780+0000","last_active":"2024-09-21T06:13:24.610032+0000","last_peered":"2024-09-21T06:13:24.610032+0000","last_clean":"2024-09-21T06:13:24.610032+0000","last_became_active":"2024-09-21T06:13:10.515437+0000","last_became_peered":"2024-09-21T06:13:10.515437+0000","last_unstale":"2024-09-21T06:13:24.610032+0000","last_undegraded":"2024-09-21T06:13:24.610032+0000","last_fullsized":"2024-09-21T06:13:24.610032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:04.848003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'13","reported_seq":95,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:19.694411+0000","last_change":"2024-09-21T06:13:08.483300+0000","last_active":"2024-09-21T06:14:19.694411+0000","last_peered":"2024-09-21T06:14:19.694411+0000","last_clean":"2024-09-21T06:14:19.694411+0000","last_became_active":"2024-09-21T06:13:08.482908+0000","last_became_peered":"2024-09-21T06:13:08.482908+0000","last_unstale":"2024-09-21T06:14:19.694411+0000","last_undegraded":"2024-09-21T06:14:19.694411+0000","last_fullsized":"2024-09-21T06:14:19.694411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:09.202075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73561,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":602,"num_read_kb":385,"num_write":247,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1724,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":220683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282438,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678043,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":864256,"data_stored":427523,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874273,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1084518,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037799,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":427179,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234028,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":427440,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":1020349,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593781,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":426779,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1019636,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":492997,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":426487,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378699,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":428582,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542224,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":493770,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":892928,"data_stored":428509,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":68093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:14:22.663 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T06:14:22.663 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T06:14:22.663 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T06:14:22.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T06:14:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.484+0000 7f88913fd640 1 -- 172.21.15.3:0/3174653655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f888c1083c0 msgr2=0x7f888c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.484+0000 7f88913fd640 1 --2- 172.21.15.3:0/3174653655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f888c1083c0 0x7f888c1087d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8880009900 tx=0x7f888002a410 comp rx=0 tx=0).stop 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.484+0000 7f88913fd640 1 -- 172.21.15.3:0/3174653655 shutdown_connections 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.484+0000 7f88913fd640 1 --2- 172.21.15.3:0/3174653655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f888c108d10 0x7f888c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.484+0000 7f88913fd640 1 --2- 172.21.15.3:0/3174653655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f888c1083c0 0x7f888c1087d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.484+0000 7f88913fd640 1 --2- 172.21.15.3:0/3174653655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c102a80 0x7f888c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.484+0000 7f88913fd640 1 -- 172.21.15.3:0/3174653655 >> 172.21.15.3:0/3174653655 conn(0x7f888c079740 msgr2=0x7f888c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.484+0000 7f88913fd640 1 -- 172.21.15.3:0/3174653655 shutdown_connections 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.484+0000 7f88913fd640 1 -- 172.21.15.3:0/3174653655 wait complete. 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.485+0000 7f88913fd640 1 Processor -- start 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.485+0000 7f88913fd640 1 -- start start 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.485+0000 7f88913fd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c102a80 0x7f888c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.485+0000 7f88913fd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f888c1083c0 0x7f888c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.485+0000 7f88913fd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f888c108d10 0x7f888c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.485+0000 7f88913fd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f888c112980 con 0x7f888c108d10 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.485+0000 7f88913fd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f888c112800 con 0x7f888c102a80 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.485+0000 7f88913fd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f888c112b00 con 0x7f888c1083c0 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f888b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f888c108d10 0x7f888c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f888b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f888c108d10 0x7f888c19eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50530/0 (socket says 172.21.15.3:50530) 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f888b7fe640 1 -- 172.21.15.3:0/3392326149 learned_addr learned my addr 172.21.15.3:0/3392326149 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f888affd640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c102a80 0x7f888c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f888a7fc640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f888c1083c0 0x7f888c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f888b7fe640 1 -- 172.21.15.3:0/3392326149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f888c1083c0 msgr2=0x7f888c1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f888b7fe640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f888c1083c0 0x7f888c1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f888b7fe640 1 -- 172.21.15.3:0/3392326149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c102a80 msgr2=0x7f888c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f888b7fe640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c102a80 0x7f888c199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f888b7fe640 1 -- 172.21.15.3:0/3392326149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888c19f1c0 con 0x7f888c108d10 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f888b7fe640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f888c108d10 0x7f888c19eb50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f888400c970 tx=0x7f888400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f8873fff640 1 -- 172.21.15.3:0/3392326149 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884007bb0 con 0x7f888c108d10 2024-09-21T06:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f88913fd640 1 -- 172.21.15.3:0/3392326149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888c19f4b0 con 0x7f888c108d10 2024-09-21T06:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f8873fff640 1 -- 172.21.15.3:0/3392326149 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8884007d50 con 0x7f888c108d10 2024-09-21T06:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f8873fff640 1 -- 172.21.15.3:0/3392326149 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884018770 con 0x7f888c108d10 2024-09-21T06:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.486+0000 7f88913fd640 1 -- 172.21.15.3:0/3392326149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f888c1a5cd0 con 0x7f888c108d10 2024-09-21T06:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.487+0000 7f88913fd640 1 -- 172.21.15.3:0/3392326149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f888c103d00 con 0x7f888c108d10 2024-09-21T06:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.490+0000 7f8873fff640 1 -- 172.21.15.3:0/3392326149 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f888400f990 con 0x7f888c108d10 2024-09-21T06:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.490+0000 7f8873fff640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f88740b6620 0x7f88740b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.490+0000 7f888affd640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f88740b6620 0x7f88740b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.491+0000 7f8873fff640 1 -- 172.21.15.3:0/3392326149 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f88840e7650 con 0x7f888c108d10 2024-09-21T06:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.491+0000 7f888affd640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f88740b6620 0x7f88740b8b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f887c007ec0 tx=0x7f887c0075b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.493+0000 7f8873fff640 1 -- 172.21.15.3:0/3392326149 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8884014030 con 0x7f888c108d10 2024-09-21T06:14:23.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.722+0000 7f88913fd640 1 -- 172.21.15.3:0/3392326149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f888c102e90 con 0x7f888c108d10 2024-09-21T06:14:23.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.722+0000 7f8873fff640 1 -- 172.21.15.3:0/3392326149 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f88840add30 con 0x7f888c108d10 2024-09-21T06:14:23.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:23 smithi003 ceph-mon[31227]: pgmap v306: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:23.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:23 smithi003 ceph-mon[31227]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:14:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:23.725 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T06:14:23.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.724+0000 7f88913fd640 1 -- 172.21.15.3:0/3392326149 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f88740b6620 msgr2=0x7f88740b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:23.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.724+0000 7f88913fd640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f88740b6620 0x7f88740b8b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f887c007ec0 tx=0x7f887c0075b0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.724+0000 7f88913fd640 1 -- 172.21.15.3:0/3392326149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f888c108d10 msgr2=0x7f888c19eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.724+0000 7f88913fd640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f888c108d10 0x7f888c19eb50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f888400c970 tx=0x7f888400ce30 comp rx=0 tx=0).stop 2024-09-21T06:14:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.724+0000 7f88913fd640 1 -- 172.21.15.3:0/3392326149 shutdown_connections 2024-09-21T06:14:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.724+0000 7f88913fd640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f88740b6620 0x7f88740b8b30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.724+0000 7f88913fd640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f888c1083c0 0x7f888c1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.724+0000 7f88913fd640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f888c108d10 0x7f888c19eb50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.724+0000 7f88913fd640 1 --2- 172.21.15.3:0/3392326149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c102a80 0x7f888c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.724+0000 7f88913fd640 1 -- 172.21.15.3:0/3392326149 >> 172.21.15.3:0/3392326149 conn(0x7f888c079740 msgr2=0x7f888c1043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.725+0000 7f88913fd640 1 -- 172.21.15.3:0/3392326149 shutdown_connections 2024-09-21T06:14:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:23.725+0000 7f88913fd640 1 -- 172.21.15.3:0/3392326149 wait complete. 2024-09-21T06:14:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:23 smithi137 ceph-mon[34464]: pgmap v306: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:23 smithi137 ceph-mon[34464]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:14:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:23 smithi204 ceph-mon[34659]: pgmap v306: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:23 smithi204 ceph-mon[34659]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:14:24.427 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T06:14:24.428 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:14:24.686 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:24.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:24 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3392326149' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T06:14:24.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:24 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3392326149' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T06:14:24.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:24 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3392326149' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T06:14:25.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.165+0000 7fc7095af640 1 -- 172.21.15.3:0/2797636609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7041094a0 msgr2=0x7fc7041098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:25.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.165+0000 7fc7095af640 1 --2- 172.21.15.3:0/2797636609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7041094a0 0x7fc7041098b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009900 tx=0x7fc6ec02a410 comp rx=0 tx=0).stop 2024-09-21T06:14:25.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.165+0000 7fc7095af640 1 -- 172.21.15.3:0/2797636609 shutdown_connections 2024-09-21T06:14:25.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.165+0000 7fc7095af640 1 --2- 172.21.15.3:0/2797636609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc704109df0 0x7fc704110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:25.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.165+0000 7fc7095af640 1 --2- 172.21.15.3:0/2797636609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7041094a0 0x7fc7041098b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:25.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.165+0000 7fc7095af640 1 --2- 172.21.15.3:0/2797636609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc704103b60 0x7fc704103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:25.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.165+0000 7fc7095af640 1 -- 172.21.15.3:0/2797636609 >> 172.21.15.3:0/2797636609 conn(0x7fc70407b850 msgr2=0x7fc70407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:25.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.165+0000 7fc7095af640 1 -- 172.21.15.3:0/2797636609 shutdown_connections 2024-09-21T06:14:25.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.165+0000 7fc7095af640 1 -- 172.21.15.3:0/2797636609 wait complete. 2024-09-21T06:14:25.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.166+0000 7fc7095af640 1 Processor -- start 2024-09-21T06:14:25.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.166+0000 7fc7095af640 1 -- start start 2024-09-21T06:14:25.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7095af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc704103b60 0x7fc704075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:25.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7095af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7041094a0 0x7fc704075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:25.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7095af640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc704109df0 0x7fc704076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7095af640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc704113ad0 con 0x7fc704109df0 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7095af640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc704113950 con 0x7fc704103b60 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7095af640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc704113c50 con 0x7fc7041094a0 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7037fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc704109df0 0x7fc704076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7037fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc704109df0 0x7fc704076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50552/0 (socket says 172.21.15.3:50552) 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7037fe640 1 -- 172.21.15.3:0/608215948 learned_addr learned my addr 172.21.15.3:0/608215948 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc702ffd640 1 --2- 172.21.15.3:0/608215948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc704103b60 0x7fc704075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7037fe640 1 -- 172.21.15.3:0/608215948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7041094a0 msgr2=0x7fc704075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7037fe640 1 --2- 172.21.15.3:0/608215948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7041094a0 0x7fc704075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7037fe640 1 -- 172.21.15.3:0/608215948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc704103b60 msgr2=0x7fc704075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7037fe640 1 --2- 172.21.15.3:0/608215948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc704103b60 0x7fc704075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7037fe640 1 -- 172.21.15.3:0/608215948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7041a88c0 con 0x7fc704109df0 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7037fe640 1 --2- 172.21.15.3:0/608215948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc704109df0 0x7fc704076190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00b970 tx=0x7fc6fc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:25.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc700ff9640 1 -- 172.21.15.3:0/608215948 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc019070 con 0x7fc704109df0 2024-09-21T06:14:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc700ff9640 1 -- 172.21.15.3:0/608215948 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6fc014440 con 0x7fc704109df0 2024-09-21T06:14:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7095af640 1 -- 172.21.15.3:0/608215948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7041a8bb0 con 0x7fc704109df0 2024-09-21T06:14:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc700ff9640 1 -- 172.21.15.3:0/608215948 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc0075d0 con 0x7fc704109df0 2024-09-21T06:14:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.167+0000 7fc7095af640 1 -- 172.21.15.3:0/608215948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7041a90f0 con 0x7fc704109df0 2024-09-21T06:14:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.169+0000 7fc7095af640 1 -- 172.21.15.3:0/608215948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc704103f70 con 0x7fc704109df0 2024-09-21T06:14:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.170+0000 7fc700ff9640 1 -- 172.21.15.3:0/608215948 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fc6fc01c070 con 0x7fc704109df0 2024-09-21T06:14:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.171+0000 7fc700ff9640 1 --2- 172.21.15.3:0/608215948 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6f00b6670 0x7fc6f00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.171+0000 7fc702ffd640 1 --2- 172.21.15.3:0/608215948 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6f00b6670 0x7fc6f00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.171+0000 7fc700ff9640 1 -- 172.21.15.3:0/608215948 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fc6fc0e6f70 con 0x7fc704109df0 2024-09-21T06:14:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.172+0000 7fc702ffd640 1 --2- 172.21.15.3:0/608215948 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6f00b6670 0x7fc6f00b8b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4005fd0 tx=0x7fc6f4005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.174+0000 7fc700ff9640 1 -- 172.21.15.3:0/608215948 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc6fc0ad5c0 con 0x7fc704109df0 2024-09-21T06:14:25.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.405+0000 7fc7095af640 1 -- 172.21.15.3:0/608215948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc704076a10 con 0x7fc704109df0 2024-09-21T06:14:25.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.405+0000 7fc700ff9640 1 -- 172.21.15.3:0/608215948 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11962 (secure 0 0 0) 0x7fc704076a10 con 0x7fc704109df0 2024-09-21T06:14:25.407 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:25.408 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-09-21T06:13:23:600046+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:13:23.600022+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:14:25.408 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-09-21T06:14:25.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.408+0000 7fc7095af640 1 -- 172.21.15.3:0/608215948 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6f00b6670 msgr2=0x7fc6f00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:25.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.408+0000 7fc7095af640 1 --2- 172.21.15.3:0/608215948 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6f00b6670 0x7fc6f00b8b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4005fd0 tx=0x7fc6f4005850 comp rx=0 tx=0).stop 2024-09-21T06:14:25.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.408+0000 7fc7095af640 1 -- 172.21.15.3:0/608215948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc704109df0 msgr2=0x7fc704076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:25.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.408+0000 7fc7095af640 1 --2- 172.21.15.3:0/608215948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc704109df0 0x7fc704076190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00b970 tx=0x7fc6fc00be30 comp rx=0 tx=0).stop 2024-09-21T06:14:25.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.408+0000 7fc7095af640 1 -- 172.21.15.3:0/608215948 shutdown_connections 2024-09-21T06:14:25.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.408+0000 7fc7095af640 1 --2- 172.21.15.3:0/608215948 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6f00b6670 0x7fc6f00b8b80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:25.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.408+0000 7fc7095af640 1 --2- 172.21.15.3:0/608215948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7041094a0 0x7fc704075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:25.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.408+0000 7fc7095af640 1 --2- 172.21.15.3:0/608215948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc704109df0 0x7fc704076190 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:25.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.408+0000 7fc7095af640 1 --2- 172.21.15.3:0/608215948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc704103b60 0x7fc704075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:25.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.408+0000 7fc7095af640 1 -- 172.21.15.3:0/608215948 >> 172.21.15.3:0/608215948 conn(0x7fc70407b850 msgr2=0x7fc704107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:25.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.408+0000 7fc7095af640 1 -- 172.21.15.3:0/608215948 shutdown_connections 2024-09-21T06:14:25.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:25.408+0000 7fc7095af640 1 -- 172.21.15.3:0/608215948 wait complete. 2024-09-21T06:14:25.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:25 smithi003 ceph-mon[31227]: pgmap v307: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:25.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:25 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/608215948' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:25 smithi137 ceph-mon[34464]: pgmap v307: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:25 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/608215948' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:25 smithi204 ceph-mon[34659]: pgmap v307: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:25 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/608215948' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:26.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:14:26.347 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.863+0000 7f81373e4640 1 -- 172.21.15.3:0/4262469646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81301083d0 msgr2=0x7f81301087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.863+0000 7f81373e4640 1 --2- 172.21.15.3:0/4262469646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81301083d0 0x7f81301087e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8120009900 tx=0x7f812002a410 comp rx=0 tx=0).stop 2024-09-21T06:14:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.863+0000 7f81373e4640 1 -- 172.21.15.3:0/4262469646 shutdown_connections 2024-09-21T06:14:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.863+0000 7f81373e4640 1 --2- 172.21.15.3:0/4262469646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8130108d20 0x7f813010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.863+0000 7f81373e4640 1 --2- 172.21.15.3:0/4262469646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81301083d0 0x7f81301087e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.863+0000 7f81373e4640 1 --2- 172.21.15.3:0/4262469646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8130102ae0 0x7f8130102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.863+0000 7f81373e4640 1 -- 172.21.15.3:0/4262469646 >> 172.21.15.3:0/4262469646 conn(0x7f8130079760 msgr2=0x7f8130079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.863+0000 7f81373e4640 1 -- 172.21.15.3:0/4262469646 shutdown_connections 2024-09-21T06:14:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.863+0000 7f81373e4640 1 -- 172.21.15.3:0/4262469646 wait complete. 2024-09-21T06:14:26.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.864+0000 7f81373e4640 1 Processor -- start 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.864+0000 7f81373e4640 1 -- start start 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.864+0000 7f81373e4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8130102ae0 0x7f813019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.864+0000 7f81373e4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81301083d0 0x7f813019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.864+0000 7f81373e4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8130108d20 0x7f813019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.864+0000 7f81373e4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81301129d0 con 0x7f81301083d0 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.864+0000 7f81373e4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8130112850 con 0x7f8130102ae0 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.864+0000 7f81373e4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8130112b50 con 0x7f8130108d20 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.864+0000 7f813497f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81301083d0 0x7f813019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f813497f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81301083d0 0x7f813019aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36584/0 (socket says 172.21.15.3:36584) 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f813497f640 1 -- 172.21.15.3:0/1886133592 learned_addr learned my addr 172.21.15.3:0/1886133592 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f8135180640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8130102ae0 0x7f813019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f8135981640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8130108d20 0x7f813019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f813497f640 1 -- 172.21.15.3:0/1886133592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8130108d20 msgr2=0x7f813019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f813497f640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8130108d20 0x7f813019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f813497f640 1 -- 172.21.15.3:0/1886133592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8130102ae0 msgr2=0x7f813019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f813497f640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8130102ae0 0x7f813019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f813497f640 1 -- 172.21.15.3:0/1886133592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81301a0480 con 0x7f81301083d0 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f813497f640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81301083d0 0x7f813019aab0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8120032fd0 tx=0x7f8120004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f81267fc640 1 -- 172.21.15.3:0/1886133592 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812003f030 con 0x7f81301083d0 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f81267fc640 1 -- 172.21.15.3:0/1886133592 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8120004490 con 0x7f81301083d0 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f81373e4640 1 -- 172.21.15.3:0/1886133592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81301a0710 con 0x7f81301083d0 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f81267fc640 1 -- 172.21.15.3:0/1886133592 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120035070 con 0x7f81301083d0 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.865+0000 7f81373e4640 1 -- 172.21.15.3:0/1886133592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81301a6ed0 con 0x7f81301083d0 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.867+0000 7f81373e4640 1 -- 172.21.15.3:0/1886133592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8130103d10 con 0x7f81301083d0 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.868+0000 7f81267fc640 1 -- 172.21.15.3:0/1886133592 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f8120042030 con 0x7f81301083d0 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.869+0000 7f81267fc640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f811c0b6670 0x7f811c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.869+0000 7f8135180640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f811c0b6670 0x7f811c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.870+0000 7f81267fc640 1 -- 172.21.15.3:0/1886133592 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8120107ed0 con 0x7f81301083d0 2024-09-21T06:14:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.870+0000 7f8135180640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f811c0b6670 0x7f811c0b8b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f812c009890 tx=0x7f812c008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:26.873+0000 7f81267fc640 1 -- 172.21.15.3:0/1886133592 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81200ce520 con 0x7f81301083d0 2024-09-21T06:14:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.098+0000 7f81373e4640 1 -- 172.21.15.3:0/1886133592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8130102ef0 con 0x7f81301083d0 2024-09-21T06:14:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.099+0000 7f81267fc640 1 -- 172.21.15.3:0/1886133592 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11962 (secure 0 0 0) 0x7f8130102ef0 con 0x7f81301083d0 2024-09-21T06:14:27.101 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:27.101 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-09-21T06:13:23:600046+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:13:23.600022+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:14:27.101 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-09-21T06:14:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.101+0000 7f81373e4640 1 -- 172.21.15.3:0/1886133592 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f811c0b6670 msgr2=0x7f811c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.101+0000 7f81373e4640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f811c0b6670 0x7f811c0b8b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f812c009890 tx=0x7f812c008040 comp rx=0 tx=0).stop 2024-09-21T06:14:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.101+0000 7f81373e4640 1 -- 172.21.15.3:0/1886133592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81301083d0 msgr2=0x7f813019aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.101+0000 7f81373e4640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81301083d0 0x7f813019aab0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8120032fd0 tx=0x7f8120004270 comp rx=0 tx=0).stop 2024-09-21T06:14:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.101+0000 7f81373e4640 1 -- 172.21.15.3:0/1886133592 shutdown_connections 2024-09-21T06:14:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.101+0000 7f81373e4640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f811c0b6670 0x7f811c0b8b80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.101+0000 7f81373e4640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8130108d20 0x7f813019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.101+0000 7f81373e4640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81301083d0 0x7f813019aab0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.101+0000 7f81373e4640 1 --2- 172.21.15.3:0/1886133592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8130102ae0 0x7f813019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.101+0000 7f81373e4640 1 -- 172.21.15.3:0/1886133592 >> 172.21.15.3:0/1886133592 conn(0x7f8130079760 msgr2=0x7f813006be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.101+0000 7f81373e4640 1 -- 172.21.15.3:0/1886133592 shutdown_connections 2024-09-21T06:14:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:27.101+0000 7f81373e4640 1 -- 172.21.15.3:0/1886133592 wait complete. 2024-09-21T06:14:27.822 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:27 smithi003 ceph-mon[31227]: pgmap v308: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:27.823 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:27 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1886133592' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:27.854 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:14:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:27 smithi137 ceph-mon[34464]: pgmap v308: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:27 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1886133592' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:27 smithi204 ceph-mon[34659]: pgmap v308: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:27 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1886133592' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:28.112 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.635+0000 7f455a341640 1 -- 172.21.15.3:0/2178157223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45541083c0 msgr2=0x7f45541087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.635+0000 7f455a341640 1 --2- 172.21.15.3:0/2178157223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45541083c0 0x7f45541087d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4548009900 tx=0x7f454802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.635+0000 7f455a341640 1 -- 172.21.15.3:0/2178157223 shutdown_connections 2024-09-21T06:14:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.635+0000 7f455a341640 1 --2- 172.21.15.3:0/2178157223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554108d10 0x7f455410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.635+0000 7f455a341640 1 --2- 172.21.15.3:0/2178157223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45541083c0 0x7f45541087d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.635+0000 7f455a341640 1 --2- 172.21.15.3:0/2178157223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554102a80 0x7f4554102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.635+0000 7f455a341640 1 -- 172.21.15.3:0/2178157223 >> 172.21.15.3:0/2178157223 conn(0x7f4554079740 msgr2=0x7f4554079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.635+0000 7f455a341640 1 -- 172.21.15.3:0/2178157223 shutdown_connections 2024-09-21T06:14:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.635+0000 7f455a341640 1 -- 172.21.15.3:0/2178157223 wait complete. 2024-09-21T06:14:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.636+0000 7f455a341640 1 Processor -- start 2024-09-21T06:14:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.636+0000 7f455a341640 1 -- start start 2024-09-21T06:14:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.636+0000 7f455a341640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102a80 0x7f455419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.636+0000 7f455a341640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45541083c0 0x7f455419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.636+0000 7f455a341640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4554108d10 0x7f455419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.636+0000 7f455a341640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f45541129c0 con 0x7f4554108d10 2024-09-21T06:14:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.636+0000 7f455a341640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4554112840 con 0x7f4554102a80 2024-09-21T06:14:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.636+0000 7f455a341640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4554112b40 con 0x7f45541083c0 2024-09-21T06:14:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f45588de640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4554108d10 0x7f455419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f45588de640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4554108d10 0x7f455419fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36598/0 (socket says 172.21.15.3:36598) 2024-09-21T06:14:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f45588de640 1 -- 172.21.15.3:0/2119816735 learned_addr learned my addr 172.21.15.3:0/2119816735 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f45537fe640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45541083c0 0x7f455419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f4553fff640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102a80 0x7f455419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f45588de640 1 -- 172.21.15.3:0/2119816735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45541083c0 msgr2=0x7f455419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f45588de640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45541083c0 0x7f455419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f45588de640 1 -- 172.21.15.3:0/2119816735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102a80 msgr2=0x7f455419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f45588de640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102a80 0x7f455419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f45588de640 1 -- 172.21.15.3:0/2119816735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45541a0410 con 0x7f4554108d10 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f4553fff640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102a80 0x7f455419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f45588de640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4554108d10 0x7f455419fda0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f454c00b970 tx=0x7f454c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f45517fa640 1 -- 172.21.15.3:0/2119816735 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c019070 con 0x7f4554108d10 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f45517fa640 1 -- 172.21.15.3:0/2119816735 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454c014440 con 0x7f4554108d10 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f455a341640 1 -- 172.21.15.3:0/2119816735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45541a0700 con 0x7f4554108d10 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f45517fa640 1 -- 172.21.15.3:0/2119816735 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0075d0 con 0x7f4554108d10 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.637+0000 7f455a341640 1 -- 172.21.15.3:0/2119816735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45541a0bc0 con 0x7f4554108d10 2024-09-21T06:14:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.638+0000 7f455a341640 1 -- 172.21.15.3:0/2119816735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4554103d00 con 0x7f4554108d10 2024-09-21T06:14:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.640+0000 7f45517fa640 1 -- 172.21.15.3:0/2119816735 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f454c01c070 con 0x7f4554108d10 2024-09-21T06:14:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.641+0000 7f45517fa640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f453c0b6620 0x7f453c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.641+0000 7f4553fff640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f453c0b6620 0x7f453c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.641+0000 7f45517fa640 1 -- 172.21.15.3:0/2119816735 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f454c0e6f60 con 0x7f4554108d10 2024-09-21T06:14:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.641+0000 7f4553fff640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f453c0b6620 0x7f453c0b8b30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f454400abe0 tx=0x7f454400a5c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.644+0000 7f45517fa640 1 -- 172.21.15.3:0/2119816735 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f454c0b1930 con 0x7f4554108d10 2024-09-21T06:14:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.832+0000 7f455a341640 1 -- 172.21.15.3:0/2119816735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4554102e90 con 0x7f4554108d10 2024-09-21T06:14:28.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.833+0000 7f45517fa640 1 -- 172.21.15.3:0/2119816735 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23048 (secure 0 0 0) 0x7f454c0b6870 con 0x7f4554108d10 2024-09-21T06:14:28.836 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:28.836 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","created":"2024-09-21T06:04:24.293786+0000","modified":"2024-09-21T06:13:24.594225+0000","last_up_change":"2024-09-21T06:12:17.781243+0000","last_in_change":"2024-09-21T06:11:54.727435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:08:26.015803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:13:07.084406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:13:09.149891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51736f81-f410-48ee-a8f3-121289607309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6803","nonce":615312328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6805","nonce":615312328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6809","nonce":615312328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6807","nonce":615312328}]},"public_addr":"172.21.15.3:6803/615312328","cluster_addr":"172.21.15.3:6805/615312328","heartbeat_back_addr":"172.21.15.3:6809/615312328","heartbeat_front_addr":"172.21.15.3:6807/615312328","state":["exists","up"]},{"osd":1,"uuid":"55696a37-34c1-482a-8561-7d08f0aa29fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6801","nonce":4150747115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6803","nonce":4150747115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6807","nonce":4150747115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6805","nonce":4150747115}]},"public_addr":"172.21.15.137:6801/4150747115","cluster_addr":"172.21.15.137:6803/4150747115","heartbeat_back_addr":"172.21.15.137:6807/4150747115","heartbeat_front_addr":"172.21.15.137:6805/4150747115","state":["exists","up"]},{"osd":2,"uuid":"8a81d34b-464b-46a2-8ef2-5223b5947004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6801","nonce":3392577077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6803","nonce":3392577077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6807","nonce":3392577077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6805","nonce":3392577077}]},"public_addr":"172.21.15.204:6801/3392577077","cluster_addr":"172.21.15.204:6803/3392577077","heartbeat_back_addr":"172.21.15.204:6807/3392577077","heartbeat_front_addr":"172.21.15.204:6805/3392577077","state":["exists","up"]},{"osd":3,"uuid":"94ab3938-7a70-44cb-99bf-9d2dedc4eb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6811","nonce":4293741635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6813","nonce":4293741635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6817","nonce":4293741635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6815","nonce":4293741635}]},"public_addr":"172.21.15.3:6811/4293741635","cluster_addr":"172.21.15.3:6813/4293741635","heartbeat_back_addr":"172.21.15.3:6817/4293741635","heartbeat_front_addr":"172.21.15.3:6815/4293741635","state":["exists","up"]},{"osd":4,"uuid":"38551535-d692-401c-aad0-bee38f3026d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6809","nonce":2608988785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6811","nonce":2608988785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6815","nonce":2608988785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6813","nonce":2608988785}]},"public_addr":"172.21.15.137:6809/2608988785","cluster_addr":"172.21.15.137:6811/2608988785","heartbeat_back_addr":"172.21.15.137:6815/2608988785","heartbeat_front_addr":"172.21.15.137:6813/2608988785","state":["exists","up"]},{"osd":5,"uuid":"3d4b785b-28d0-4654-a9b1-4c5c905f4bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6809","nonce":3709074044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6811","nonce":3709074044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6815","nonce":3709074044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6813","nonce":3709074044}]},"public_addr":"172.21.15.204:6809/3709074044","cluster_addr":"172.21.15.204:6811/3709074044","heartbeat_back_addr":"172.21.15.204:6815/3709074044","heartbeat_front_addr":"172.21.15.204:6813/3709074044","state":["exists","up"]},{"osd":6,"uuid":"ba21dbe9-ab43-4b7e-9446-2929ee874c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6819","nonce":3862646594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6821","nonce":3862646594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6825","nonce":3862646594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6823","nonce":3862646594}]},"public_addr":"172.21.15.3:6819/3862646594","cluster_addr":"172.21.15.3:6821/3862646594","heartbeat_back_addr":"172.21.15.3:6825/3862646594","heartbeat_front_addr":"172.21.15.3:6823/3862646594","state":["exists","up"]},{"osd":7,"uuid":"9610fe5c-93df-49f9-b8bd-5cb480e0871b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6817","nonce":2086822496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6819","nonce":2086822496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6823","nonce":2086822496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6821","nonce":2086822496}]},"public_addr":"172.21.15.137:6817/2086822496","cluster_addr":"172.21.15.137:6819/2086822496","heartbeat_back_addr":"172.21.15.137:6823/2086822496","heartbeat_front_addr":"172.21.15.137:6821/2086822496","state":["exists","up"]},{"osd":8,"uuid":"49f6e9b4-6d14-434e-8b42-8b7ffeb2a072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6817","nonce":153476798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6819","nonce":153476798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6823","nonce":153476798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6821","nonce":153476798}]},"public_addr":"172.21.15.204:6817/153476798","cluster_addr":"172.21.15.204:6819/153476798","heartbeat_back_addr":"172.21.15.204:6823/153476798","heartbeat_front_addr":"172.21.15.204:6821/153476798","state":["exists","up"]},{"osd":9,"uuid":"633f67a7-b759-4ea6-82ed-3c42b3fb70ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6827","nonce":1238491549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6829","nonce":1238491549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6833","nonce":1238491549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6831","nonce":1238491549}]},"public_addr":"172.21.15.3:6827/1238491549","cluster_addr":"172.21.15.3:6829/1238491549","heartbeat_back_addr":"172.21.15.3:6833/1238491549","heartbeat_front_addr":"172.21.15.3:6831/1238491549","state":["exists","up"]},{"osd":10,"uuid":"94b65a48-ee87-4e59-9296-a72a22787f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6825","nonce":3486552683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6827","nonce":3486552683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6831","nonce":3486552683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6829","nonce":3486552683}]},"public_addr":"172.21.15.137:6825/3486552683","cluster_addr":"172.21.15.137:6827/3486552683","heartbeat_back_addr":"172.21.15.137:6831/3486552683","heartbeat_front_addr":"172.21.15.137:6829/3486552683","state":["exists","up"]},{"osd":11,"uuid":"c5cc2850-9062-4b8e-beb3-bef0db30a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6825","nonce":1084308610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6827","nonce":1084308610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6831","nonce":1084308610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6829","nonce":1084308610}]},"public_addr":"172.21.15.204:6825/1084308610","cluster_addr":"172.21.15.204:6827/1084308610","heartbeat_back_addr":"172.21.15.204:6831/1084308610","heartbeat_front_addr":"172.21.15.204:6829/1084308610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:31.898745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.089965+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:22.367464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:48.543248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:14.687331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:41.243579+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:07.290227+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:32.930046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:59.197638+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:24.315219+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:50.333639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:12:16.453640+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1573374138":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1595702358":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/547897695":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/3673347975":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/141972779":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6800/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:6801/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/2363985440":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/2138752581":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1820734047":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/2929116087":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/45453677":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/2929116087":"2024-09-22T06:05:21.689191+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:14:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.836+0000 7f455a341640 1 -- 172.21.15.3:0/2119816735 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f453c0b6620 msgr2=0x7f453c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.836+0000 7f455a341640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f453c0b6620 0x7f453c0b8b30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f454400abe0 tx=0x7f454400a5c0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.836+0000 7f455a341640 1 -- 172.21.15.3:0/2119816735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4554108d10 msgr2=0x7f455419fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.836+0000 7f455a341640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4554108d10 0x7f455419fda0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f454c00b970 tx=0x7f454c00be30 comp rx=0 tx=0).stop 2024-09-21T06:14:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.836+0000 7f455a341640 1 -- 172.21.15.3:0/2119816735 shutdown_connections 2024-09-21T06:14:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.836+0000 7f455a341640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f453c0b6620 0x7f453c0b8b30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.836+0000 7f455a341640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45541083c0 0x7f455419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.836+0000 7f455a341640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4554108d10 0x7f455419fda0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.836+0000 7f455a341640 1 --2- 172.21.15.3:0/2119816735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102a80 0x7f455419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.836+0000 7f455a341640 1 -- 172.21.15.3:0/2119816735 >> 172.21.15.3:0/2119816735 conn(0x7f4554079740 msgr2=0x7f45541019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.836+0000 7f455a341640 1 -- 172.21.15.3:0/2119816735 shutdown_connections 2024-09-21T06:14:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:28.836+0000 7f455a341640 1 -- 172.21.15.3:0/2119816735 wait complete. 2024-09-21T06:14:29.463 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:14:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:29.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:29 smithi003 ceph-mon[31227]: pgmap v309: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:29.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:29 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2119816735' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:14:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:29 smithi137 ceph-mon[34464]: pgmap v309: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:29 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2119816735' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:14:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:29 smithi204 ceph-mon[34659]: pgmap v309: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:29 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2119816735' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:14:30.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.243+0000 7fd086b11640 1 -- 172.21.15.3:0/1211375727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd080108d20 msgr2=0x7fd08010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:30.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.243+0000 7fd086b11640 1 --2- 172.21.15.3:0/1211375727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd080108d20 0x7fd08010f550 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd078009ca0 tx=0x7fd07802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:30.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.243+0000 7fd086b11640 1 -- 172.21.15.3:0/1211375727 shutdown_connections 2024-09-21T06:14:30.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.243+0000 7fd086b11640 1 --2- 172.21.15.3:0/1211375727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd080108d20 0x7fd08010f550 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:30.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.243+0000 7fd086b11640 1 --2- 172.21.15.3:0/1211375727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0801083d0 0x7fd0801087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:30.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.243+0000 7fd086b11640 1 --2- 172.21.15.3:0/1211375727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd080102ae0 0x7fd080102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:30.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.243+0000 7fd086b11640 1 -- 172.21.15.3:0/1211375727 >> 172.21.15.3:0/1211375727 conn(0x7fd080079760 msgr2=0x7fd080079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.243+0000 7fd086b11640 1 -- 172.21.15.3:0/1211375727 shutdown_connections 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.243+0000 7fd086b11640 1 -- 172.21.15.3:0/1211375727 wait complete. 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd086b11640 1 Processor -- start 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd086b11640 1 -- start start 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd086b11640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd080102ae0 0x7fd0800ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd086b11640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0801083d0 0x7fd0800ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd086b11640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd080108d20 0x7fd0801000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd086b11640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd080112990 con 0x7fd080108d20 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd086b11640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd080112810 con 0x7fd080102ae0 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd086b11640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd080112b10 con 0x7fd0801083d0 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd0850ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd080108d20 0x7fd0801000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd0850ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd080108d20 0x7fd0801000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36624/0 (socket says 172.21.15.3:36624) 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd0850ae640 1 -- 172.21.15.3:0/460153597 learned_addr learned my addr 172.21.15.3:0/460153597 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd0848ad640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd080102ae0 0x7fd0800ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd0850ae640 1 -- 172.21.15.3:0/460153597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0801083d0 msgr2=0x7fd0800ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd0850ae640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0801083d0 0x7fd0800ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.244+0000 7fd0850ae640 1 -- 172.21.15.3:0/460153597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd080102ae0 msgr2=0x7fd0800ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.245+0000 7fd0850ae640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd080102ae0 0x7fd0800ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.245+0000 7fd0850ae640 1 -- 172.21.15.3:0/460153597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0801a8670 con 0x7fd080108d20 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.245+0000 7fd0848ad640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd080102ae0 0x7fd0800ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.245+0000 7fd0850ae640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd080108d20 0x7fd0801000f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd078005cc0 tx=0x7fd07802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.245+0000 7fd075ffb640 1 -- 172.21.15.3:0/460153597 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078003bf0 con 0x7fd080108d20 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.245+0000 7fd086b11640 1 -- 172.21.15.3:0/460153597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0801a8900 con 0x7fd080108d20 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.245+0000 7fd075ffb640 1 -- 172.21.15.3:0/460153597 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd078003d90 con 0x7fd080108d20 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.245+0000 7fd075ffb640 1 -- 172.21.15.3:0/460153597 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07802fbc0 con 0x7fd080108d20 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.245+0000 7fd086b11640 1 -- 172.21.15.3:0/460153597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0801a8e90 con 0x7fd080108d20 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.246+0000 7fd086b11640 1 -- 172.21.15.3:0/460153597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd080103d10 con 0x7fd080108d20 2024-09-21T06:14:30.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.248+0000 7fd075ffb640 1 -- 172.21.15.3:0/460153597 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fd07802fd60 con 0x7fd080108d20 2024-09-21T06:14:30.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.249+0000 7fd075ffb640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0680b65f0 0x7fd0680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:30.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.249+0000 7fd0848ad640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0680b65f0 0x7fd0680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:30.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.249+0000 7fd075ffb640 1 -- 172.21.15.3:0/460153597 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fd078108d00 con 0x7fd080108d20 2024-09-21T06:14:30.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.249+0000 7fd0848ad640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0680b65f0 0x7fd0680b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd0801010c0 tx=0x7fd07c005e70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:30.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.252+0000 7fd075ffb640 1 -- 172.21.15.3:0/460153597 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd0780cf350 con 0x7fd080108d20 2024-09-21T06:14:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.477+0000 7fd086b11640 1 -- 172.21.15.3:0/460153597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd080100970 con 0x7fd080108d20 2024-09-21T06:14:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.478+0000 7fd075ffb640 1 -- 172.21.15.3:0/460153597 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11962 (secure 0 0 0) 0x7fd080100970 con 0x7fd080108d20 2024-09-21T06:14:30.480 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:30.480 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-09-21T06:13:23:600046+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:13:23.600022+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:14:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-09-21T06:14:30.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.480+0000 7fd086b11640 1 -- 172.21.15.3:0/460153597 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0680b65f0 msgr2=0x7fd0680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:30.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.480+0000 7fd086b11640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0680b65f0 0x7fd0680b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd0801010c0 tx=0x7fd07c005e70 comp rx=0 tx=0).stop 2024-09-21T06:14:30.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.480+0000 7fd086b11640 1 -- 172.21.15.3:0/460153597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd080108d20 msgr2=0x7fd0801000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.480+0000 7fd086b11640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd080108d20 0x7fd0801000f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd078005cc0 tx=0x7fd07802f380 comp rx=0 tx=0).stop 2024-09-21T06:14:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.480+0000 7fd086b11640 1 -- 172.21.15.3:0/460153597 shutdown_connections 2024-09-21T06:14:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.480+0000 7fd086b11640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0680b65f0 0x7fd0680b8b00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.480+0000 7fd086b11640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0801083d0 0x7fd0800ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.480+0000 7fd086b11640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd080108d20 0x7fd0801000f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.480+0000 7fd086b11640 1 --2- 172.21.15.3:0/460153597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd080102ae0 0x7fd0800ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.480+0000 7fd086b11640 1 -- 172.21.15.3:0/460153597 >> 172.21.15.3:0/460153597 conn(0x7fd080079760 msgr2=0x7fd0801018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.481+0000 7fd086b11640 1 -- 172.21.15.3:0/460153597 shutdown_connections 2024-09-21T06:14:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:30.481+0000 7fd086b11640 1 -- 172.21.15.3:0/460153597 wait complete. 2024-09-21T06:14:31.181 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T06:13:11.205233+0000', 'modified': '2024-09-21T06:13:23.600022+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24704, 'mds_1': 24691, 'mds_2': 24679, 'mds_3': 24674, 'mds_4': 24670}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14844': {'gid': 14844, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.3:6839/1325295587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6838', 'nonce': 1325295587}, {'type': 'v1', 'addr': '172.21.15.3:6839', 'nonce': 1325295587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14856': {'gid': 14856, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.3:6841/2403205583', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6840', 'nonce': 2403205583}, {'type': 'v1', 'addr': '172.21.15.3:6841', 'nonce': 2403205583}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24620': {'gid': 24620, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6833/4270441897', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6832', 'nonce': 4270441897}, {'type': 'v1', 'addr': '172.21.15.137:6833', 'nonce': 4270441897}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24631': {'gid': 24631, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/2480313554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 2480313554}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 2480313554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24644': {'gid': 24644, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/399369618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 399369618}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 399369618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24670': {'gid': 24670, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.204:6835/879721642', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 879721642}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 879721642}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24674': {'gid': 24674, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.137:6837/44063020', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 44063020}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 44063020}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24679': {'gid': 24679, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.204:6837/3211759973', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 3211759973}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 3211759973}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24691': {'gid': 24691, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.204:6839/2228257050', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 2228257050}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 2228257050}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24704': {'gid': 24704, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.137:6839/2663570031', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 2663570031}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 2663570031}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24704, 'qdb_cluster': [24704, 24691, 24679, 24674, 24670]} 2024-09-21T06:14:31.181 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T06:14:31.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.f status 2024-09-21T06:14:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:31.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:31 smithi003 ceph-mon[31227]: pgmap v310: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:31.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:31 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/460153597' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:31.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:31 smithi137 ceph-mon[34464]: pgmap v310: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:31.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:31 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/460153597' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:31.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:31 smithi204 ceph-mon[34659]: pgmap v310: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:31.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:31 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/460153597' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.927+0000 7f0def222640 1 -- 172.21.15.3:0/39340771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de8102ae0 msgr2=0x7f0de8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.927+0000 7f0def222640 1 --2- 172.21.15.3:0/39340771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de8102ae0 0x7f0de8102ef0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009900 tx=0x7f0dd002a410 comp rx=0 tx=0).stop 2024-09-21T06:14:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.927+0000 7f0def222640 1 -- 172.21.15.3:0/39340771 shutdown_connections 2024-09-21T06:14:31.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.927+0000 7f0def222640 1 --2- 172.21.15.3:0/39340771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de8108d20 0x7f0de810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.927+0000 7f0def222640 1 --2- 172.21.15.3:0/39340771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0de81083d0 0x7f0de81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.927+0000 7f0def222640 1 --2- 172.21.15.3:0/39340771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de8102ae0 0x7f0de8102ef0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.927+0000 7f0def222640 1 -- 172.21.15.3:0/39340771 >> 172.21.15.3:0/39340771 conn(0x7f0de8079760 msgr2=0x7f0de8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:31.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.928+0000 7f0def222640 1 -- 172.21.15.3:0/39340771 shutdown_connections 2024-09-21T06:14:31.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.928+0000 7f0def222640 1 -- 172.21.15.3:0/39340771 wait complete. 2024-09-21T06:14:31.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.928+0000 7f0def222640 1 Processor -- start 2024-09-21T06:14:31.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.928+0000 7f0def222640 1 -- start start 2024-09-21T06:14:31.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0def222640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de8102ae0 0x7f0de819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0def222640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de81083d0 0x7f0de819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0def222640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0de8108d20 0x7f0de819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0def222640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8112990 con 0x7f0de81083d0 2024-09-21T06:14:31.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0def222640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8112810 con 0x7f0de8108d20 2024-09-21T06:14:31.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0def222640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8112b10 con 0x7f0de8102ae0 2024-09-21T06:14:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0de7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de81083d0 0x7f0de819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0de7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de81083d0 0x7f0de819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36636/0 (socket says 172.21.15.3:36636) 2024-09-21T06:14:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0de7fff640 1 -- 172.21.15.3:0/3034875623 learned_addr learned my addr 172.21.15.3:0/3034875623 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0ded7bf640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0de8108d20 0x7f0de819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0decfbe640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de8102ae0 0x7f0de819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0de7fff640 1 -- 172.21.15.3:0/3034875623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de8102ae0 msgr2=0x7f0de819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0de7fff640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de8102ae0 0x7f0de819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0de7fff640 1 -- 172.21.15.3:0/3034875623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0de8108d20 msgr2=0x7f0de819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0de7fff640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0de8108d20 0x7f0de819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0de7fff640 1 -- 172.21.15.3:0/3034875623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de81a0340 con 0x7f0de81083d0 2024-09-21T06:14:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0decfbe640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de8102ae0 0x7f0de819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0de7fff640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de81083d0 0x7f0de819aa30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800ea80 tx=0x7f0dd800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0de5ffb640 1 -- 172.21.15.3:0/3034875623 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd800ce40 con 0x7f0de81083d0 2024-09-21T06:14:31.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0def222640 1 -- 172.21.15.3:0/3034875623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de81a0630 con 0x7f0de81083d0 2024-09-21T06:14:31.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0de5ffb640 1 -- 172.21.15.3:0/3034875623 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd8011920 con 0x7f0de81083d0 2024-09-21T06:14:31.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.929+0000 7f0de5ffb640 1 -- 172.21.15.3:0/3034875623 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd801aa20 con 0x7f0de81083d0 2024-09-21T06:14:31.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.930+0000 7f0def222640 1 -- 172.21.15.3:0/3034875623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de81a6e50 con 0x7f0de81083d0 2024-09-21T06:14:31.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.932+0000 7f0df5796740 1 Processor -- start 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.932+0000 7f0de5ffb640 1 -- 172.21.15.3:0/3034875623 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f0dd801abc0 con 0x7f0de81083d0 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.932+0000 7f0df5796740 1 -- start start 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.932+0000 7f0df5796740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb7ef60 0x55f2bbf24f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.932+0000 7f0df5796740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf25440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0df5796740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb5e280 0x55f2bbf25980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0df5796740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f2bbe9cc10 con 0x55f2bbb560a0 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0df5796740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2bbe8c040 con 0x55f2bbb7ef60 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0df5796740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f2bbe8fe50 con 0x55f2bbb5e280 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0decfbe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf25440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0decfbe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf25440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36652/0 (socket says 172.21.15.3:36652) 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0decfbe640 1 -- 172.21.15.3:0/1836388692 learned_addr learned my addr 172.21.15.3:0/1836388692 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0ded7bf640 1 --2- 172.21.15.3:0/1836388692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb7ef60 0x55f2bbf24f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0de7fff640 1 --2- 172.21.15.3:0/1836388692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb5e280 0x55f2bbf25980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0decfbe640 1 -- 172.21.15.3:0/1836388692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb5e280 msgr2=0x55f2bbf25980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0decfbe640 1 --2- 172.21.15.3:0/1836388692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb5e280 0x55f2bbf25980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0decfbe640 1 -- 172.21.15.3:0/1836388692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb7ef60 msgr2=0x55f2bbf24f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0decfbe640 1 --2- 172.21.15.3:0/1836388692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb7ef60 0x55f2bbf24f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0decfbe640 1 -- 172.21.15.3:0/1836388692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2bbbea540 con 0x55f2bbb560a0 2024-09-21T06:14:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0ded7bf640 1 --2- 172.21.15.3:0/1836388692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb7ef60 0x55f2bbf24f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0de5ffb640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0dd40b6620 0x7f0dd40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0decfbe640 1 --2- 172.21.15.3:0/1836388692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf25440 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0000c00 tx=0x7f0dd0032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e3083314d5e5ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.933+0000 7f0ded7bf640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0dd40b6620 0x7f0dd40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.934+0000 7f0de5ffb640 1 -- 172.21.15.3:0/3034875623 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f0dd80e7bd0 con 0x7f0de81083d0 2024-09-21T06:14:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.934+0000 7f0ddeffd640 1 -- 172.21.15.3:0/1836388692 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd003f070 con 0x55f2bbb560a0 2024-09-21T06:14:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.934+0000 7f0ddeffd640 1 -- 172.21.15.3:0/1836388692 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd00316a0 con 0x55f2bbb560a0 2024-09-21T06:14:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.934+0000 7f0ddeffd640 1 -- 172.21.15.3:0/1836388692 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0035070 con 0x55f2bbb560a0 2024-09-21T06:14:31.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.935+0000 7f0df5796740 1 -- 172.21.15.3:0/1836388692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 msgr2=0x55f2bbf25440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:31.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.935+0000 7f0df5796740 1 --2- 172.21.15.3:0/1836388692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf25440 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0000c00 tx=0x7f0dd0032d00 comp rx=0 tx=0).stop 2024-09-21T06:14:31.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.935+0000 7f0ded7bf640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0dd40b6620 0x7f0dd40b8b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0de0005f30 tx=0x7f0de0005ec0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.935+0000 7f0df5796740 1 -- 172.21.15.3:0/1836388692 shutdown_connections 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.935+0000 7f0df5796740 1 --2- 172.21.15.3:0/1836388692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb5e280 0x55f2bbf25980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.935+0000 7f0df5796740 1 --2- 172.21.15.3:0/1836388692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf25440 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.935+0000 7f0df5796740 1 --2- 172.21.15.3:0/1836388692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb7ef60 0x55f2bbf24f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.935+0000 7f0df5796740 1 -- 172.21.15.3:0/1836388692 >> 172.21.15.3:0/1836388692 conn(0x55f2bbb73350 msgr2=0x55f2bbf22df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.935+0000 7f0df5796740 1 -- 172.21.15.3:0/1836388692 shutdown_connections 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.935+0000 7f0df5796740 1 -- 172.21.15.3:0/1836388692 wait complete. 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.935+0000 7f0df5796740 1 -- start start 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.935+0000 7f0df5796740 1 Processor -- start 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0df5796740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb560a0 0x55f2bbf30a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0df5796740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb5e280 0x55f2bbf30f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0df5796740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb7ef60 0x55f2bbf3c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0df5796740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f2bbced5f0 con 0x55f2bbb7ef60 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0df5796740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2bbe27010 con 0x55f2bbb560a0 2024-09-21T06:14:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0df5796740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f2bbf33680 con 0x55f2bbb5e280 2024-09-21T06:14:31.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0df5796740 10 client.0 _ll_register_callbacks cb 0x55f2bbe86340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:31.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0ded7bf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb7ef60 0x55f2bbf3c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0ded7bf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb7ef60 0x55f2bbf3c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36656/0 (socket says 172.21.15.3:36656) 2024-09-21T06:14:31.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0ded7bf640 1 -- 172.21.15.3:0/3732159982 learned_addr learned my addr 172.21.15.3:0/3732159982 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:31.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0decfbe640 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb560a0 0x55f2bbf30a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0de7fff640 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb5e280 0x55f2bbf30f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0ded7bf640 1 -- 172.21.15.3:0/3732159982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb5e280 msgr2=0x55f2bbf30f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:31.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0ded7bf640 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb5e280 0x55f2bbf30f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0ded7bf640 1 -- 172.21.15.3:0/3732159982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb560a0 msgr2=0x55f2bbf30a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:31.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0ded7bf640 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb560a0 0x55f2bbf30a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0ded7bf640 1 -- 172.21.15.3:0/3732159982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2bbe54a00 con 0x55f2bbb7ef60 2024-09-21T06:14:31.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0decfbe640 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb560a0 0x55f2bbf30a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:31.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0ded7bf640 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb7ef60 0x55f2bbf3c9a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0de000c620 tx=0x7f0de00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5eff892639399a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.936+0000 7f0dc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0dc7fff640 1 -- 172.21.15.3:0/3732159982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de000ee10 con 0x55f2bbb7ef60 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0dc7fff640 1 -- 172.21.15.3:0/3732159982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f0de001e420 con 0x55f2bbb7ef60 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0df5796740 1 -- 172.21.15.3:0/3732159982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dd00316a0 con 0x55f2bbb7ef60 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0dc7fff640 1 -- 172.21.15.3:0/3732159982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0de001e620 con 0x55f2bbb7ef60 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0df5796740 10 client.15222 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0df5796740 1 -- 172.21.15.3:0/3732159982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2bbf3f600 con 0x55f2bbb7ef60 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0dc7fff640 1 -- 172.21.15.3:0/3732159982 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd00316a0 con 0x55f2bbb7ef60 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0dc7fff640 1 -- 172.21.15.3:0/3732159982 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0de002f8a0 con 0x55f2bbb7ef60 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0df5796740 10 client.15222 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0df5796740 10 client.15222 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0df5796740 20 client.15222 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0df5796740 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f2bbb3d8e0 0x55f2bbf430b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0df5796740 4 client.15222 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.937+0000 7f0df5796740 1 -- 172.21.15.3:0/3732159982 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2bbe9eb60 con 0x55f2bbb3d8e0 2024-09-21T06:14:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.938+0000 7f0decfbe640 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f2bbb3d8e0 0x55f2bbf430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.938+0000 7f0decfbe640 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f2bbb3d8e0 0x55f2bbf430b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cc1fa7b9b3fdc9fd server_cookie=d564a3492ea93b20 in_seq=0 out_seq=0 2024-09-21T06:14:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.938+0000 7f0dc7fff640 10 client.15222 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:14:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.939+0000 7f0dc7fff640 1 -- 172.21.15.3:0/3732159982 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f2bbe9eb60 con 0x55f2bbb3d8e0 2024-09-21T06:14:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.939+0000 7f0dc7fff640 10 client.15222 handle_command_reply: tid=0 2024-09-21T06:14:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.939+0000 7f0df5796740 1 client.15222 shutdown 2024-09-21T06:14:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.939+0000 7f0df5796740 1 -- 172.21.15.3:0/3732159982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb7ef60 msgr2=0x55f2bbf3c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.939+0000 7f0df5796740 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb7ef60 0x55f2bbf3c9a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0de000c620 tx=0x7f0de00046c0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.939+0000 7f0df5796740 1 -- 172.21.15.3:0/3732159982 shutdown_connections 2024-09-21T06:14:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.939+0000 7f0df5796740 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f2bbb3d8e0 0x55f2bbf430b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.939+0000 7f0dc7fff640 0 client.15222 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:14:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.939+0000 7f0df5796740 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb5e280 0x55f2bbf30f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.940+0000 7f0df5796740 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb7ef60 0x55f2bbf3c9a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.940+0000 7f0df5796740 1 --2- 172.21.15.3:0/3732159982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb560a0 0x55f2bbf30a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.940+0000 7f0df5796740 1 -- 172.21.15.3:0/3732159982 >> 172.21.15.3:0/3732159982 conn(0x55f2bbb73350 msgr2=0x55f2bbf28090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.940+0000 7f0df5796740 1 -- 172.21.15.3:0/3732159982 shutdown_connections 2024-09-21T06:14:31.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.940+0000 7f0df5796740 1 -- 172.21.15.3:0/3732159982 wait complete. 2024-09-21T06:14:31.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.940+0000 7f0df5796740 20 client.15222 trim_cache size 0 max 16384 2024-09-21T06:14:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.958+0000 7f0df5796740 1 Processor -- start 2024-09-21T06:14:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.958+0000 7f0df5796740 1 -- start start 2024-09-21T06:14:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.958+0000 7f0df5796740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb3d8e0 0x55f2bbf309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.958+0000 7f0df5796740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf30f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.958+0000 7f0df5796740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb5e280 0x55f2bbf2b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.958+0000 7f0df5796740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f2bbf413f0 con 0x55f2bbb560a0 2024-09-21T06:14:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.958+0000 7f0df5796740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2bbf41ac0 con 0x55f2bbb5e280 2024-09-21T06:14:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.958+0000 7f0df5796740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f2bbf42190 con 0x55f2bbb3d8e0 2024-09-21T06:14:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.958+0000 7f0de7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf30f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.958+0000 7f0de7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf30f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36666/0 (socket says 172.21.15.3:36666) 2024-09-21T06:14:31.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.958+0000 7f0de7fff640 1 -- 172.21.15.3:0/1273101881 learned_addr learned my addr 172.21.15.3:0/1273101881 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:31.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.958+0000 7f0ded7bf640 1 --2- 172.21.15.3:0/1273101881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb5e280 0x55f2bbf2b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.959+0000 7f0de7fff640 1 -- 172.21.15.3:0/1273101881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb3d8e0 msgr2=0x55f2bbf309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:31.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.959+0000 7f0decfbe640 1 --2- 172.21.15.3:0/1273101881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb3d8e0 0x55f2bbf309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.959+0000 7f0de7fff640 1 --2- 172.21.15.3:0/1273101881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb3d8e0 0x55f2bbf309e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.959+0000 7f0de7fff640 1 -- 172.21.15.3:0/1273101881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb5e280 msgr2=0x55f2bbf2b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:31.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.959+0000 7f0de7fff640 1 --2- 172.21.15.3:0/1273101881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb5e280 0x55f2bbf2b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.959+0000 7f0de7fff640 1 -- 172.21.15.3:0/1273101881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2bbf1d080 con 0x55f2bbb560a0 2024-09-21T06:14:31.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.959+0000 7f0ded7bf640 1 --2- 172.21.15.3:0/1273101881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb5e280 0x55f2bbf2b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:31.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.959+0000 7f0de7fff640 1 --2- 172.21.15.3:0/1273101881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf30f20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80aa0f0 tx=0x7f0dd801aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189d3a7e1261bd52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:31.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.959+0000 7f0dde7fc640 1 -- 172.21.15.3:0/1273101881 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd80ad8f0 con 0x55f2bbb560a0 2024-09-21T06:14:31.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.959+0000 7f0dde7fc640 1 -- 172.21.15.3:0/1273101881 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd80ad630 con 0x55f2bbb560a0 2024-09-21T06:14:31.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.959+0000 7f0dde7fc640 1 -- 172.21.15.3:0/1273101881 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd80a9290 con 0x55f2bbb560a0 2024-09-21T06:14:31.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.960+0000 7f0df5796740 1 -- 172.21.15.3:0/1273101881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 msgr2=0x55f2bbf30f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:31.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.960+0000 7f0df5796740 1 --2- 172.21.15.3:0/1273101881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf30f20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80aa0f0 tx=0x7f0dd801aeb0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.960+0000 7f0df5796740 1 -- 172.21.15.3:0/1273101881 shutdown_connections 2024-09-21T06:14:31.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.960+0000 7f0df5796740 1 --2- 172.21.15.3:0/1273101881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb5e280 0x55f2bbf2b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.960+0000 7f0df5796740 1 --2- 172.21.15.3:0/1273101881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf30f20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.960+0000 7f0df5796740 1 --2- 172.21.15.3:0/1273101881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb3d8e0 0x55f2bbf309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.960+0000 7f0df5796740 1 -- 172.21.15.3:0/1273101881 >> 172.21.15.3:0/1273101881 conn(0x55f2bbb73350 msgr2=0x55f2bbf40a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.960+0000 7f0df5796740 1 -- 172.21.15.3:0/1273101881 shutdown_connections 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.960+0000 7f0df5796740 1 -- 172.21.15.3:0/1273101881 wait complete. 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.960+0000 7f0df5796740 1 -- start start 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.960+0000 7f0df5796740 1 Processor -- start 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0df5796740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb3d8e0 0x55f2bbf45a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0df5796740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf45f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0df5796740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb5e280 0x55f2bbf4a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0df5796740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f2bbf42f30 con 0x55f2bbb560a0 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0df5796740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2bbf42860 con 0x55f2bbb5e280 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0df5796740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f2bbf43600 con 0x55f2bbb3d8e0 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0df5796740 10 client.0 _ll_register_callbacks cb 0x7f0de000c490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:31.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0de7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf45f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0de7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf45f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36678/0 (socket says 172.21.15.3:36678) 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0de7fff640 1 -- 172.21.15.3:0/3009984690 learned_addr learned my addr 172.21.15.3:0/3009984690 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0ded7bf640 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb5e280 0x55f2bbf4a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0decfbe640 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb3d8e0 0x55f2bbf45a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0de7fff640 1 -- 172.21.15.3:0/3009984690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb3d8e0 msgr2=0x55f2bbf45a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0de7fff640 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb3d8e0 0x55f2bbf45a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0de7fff640 1 -- 172.21.15.3:0/3009984690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb5e280 msgr2=0x55f2bbf4a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0de7fff640 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb5e280 0x55f2bbf4a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0de7fff640 1 -- 172.21.15.3:0/3009984690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2bbf4ae10 con 0x55f2bbb560a0 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0ded7bf640 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb5e280 0x55f2bbf4a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0de7fff640 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf45f90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80e5520 tx=0x7f0dd80ad960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ecaa83f1ac34e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.961+0000 7f0dc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0dc7fff640 1 -- 172.21.15.3:0/3009984690 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd80b2070 con 0x55f2bbb560a0 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0dc7fff640 1 -- 172.21.15.3:0/3009984690 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f0dd80a9d90 con 0x55f2bbb560a0 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0df5796740 1 -- 172.21.15.3:0/3009984690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dd80ad630 con 0x55f2bbb560a0 2024-09-21T06:14:31.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0dc7fff640 1 -- 172.21.15.3:0/3009984690 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0dd80ad290 con 0x55f2bbb560a0 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0df5796740 10 client.15228 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0df5796740 1 -- 172.21.15.3:0/3009984690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2bbf4c000 con 0x55f2bbb560a0 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0dc7fff640 1 -- 172.21.15.3:0/3009984690 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd80ad630 con 0x55f2bbb560a0 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0dc7fff640 1 -- 172.21.15.3:0/3009984690 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0dd8022030 con 0x55f2bbb560a0 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0df5796740 10 client.15228 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0df5796740 10 client.15228 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0df5796740 20 client.15228 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0df5796740 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f2bbb7ef60 0x55f2bbfb5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0df5796740 4 client.15228 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.962+0000 7f0df5796740 1 -- 172.21.15.3:0/3009984690 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "status"}) -- 0x55f2bbf41240 con 0x55f2bbb7ef60 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.963+0000 7f0decfbe640 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f2bbb7ef60 0x55f2bbfb5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.963+0000 7f0dc7fff640 10 client.15228 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.963+0000 7f0decfbe640 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f2bbb7ef60 0x55f2bbfb5a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7ef3552d36eeb21e server_cookie=9b8ee48f96462cbc in_seq=0 out_seq=0 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.963+0000 7f0dc7fff640 1 -- 172.21.15.3:0/3009984690 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f2bbf41240 con 0x55f2bbb7ef60 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.963+0000 7f0dc7fff640 10 client.15228 handle_command_reply: tid=0 2024-09-21T06:14:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.963+0000 7f0df5796740 1 client.15228 shutdown 2024-09-21T06:14:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.963+0000 7f0df5796740 1 -- 172.21.15.3:0/3009984690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 msgr2=0x55f2bbf45f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.963+0000 7f0df5796740 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf45f90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80e5520 tx=0x7f0dd80ad960 comp rx=0 tx=0).stop 2024-09-21T06:14:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.963+0000 7f0df5796740 1 -- 172.21.15.3:0/3009984690 shutdown_connections 2024-09-21T06:14:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.963+0000 7f0df5796740 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f2bbb7ef60 0x55f2bbfb5a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0df5796740 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2bbb5e280 0x55f2bbf4a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0df5796740 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2bbb560a0 0x55f2bbf45f90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0dc7fff640 0 client.15228 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:14:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0df5796740 1 --2- 172.21.15.3:0/3009984690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2bbb3d8e0 0x55f2bbf45a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0df5796740 1 -- 172.21.15.3:0/3009984690 >> 172.21.15.3:0/3009984690 conn(0x55f2bbb73350 msgr2=0x55f2bbf40a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0df5796740 1 -- 172.21.15.3:0/3009984690 shutdown_connections 2024-09-21T06:14:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0df5796740 1 -- 172.21.15.3:0/3009984690 wait complete. 2024-09-21T06:14:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0df5796740 20 client.15228 trim_cache size 0 max 16384 2024-09-21T06:14:31.970 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:14:31.971 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "38d3c75c-77df-11ef-baf5-efdc52797490", 2024-09-21T06:14:31.971 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 4, 2024-09-21T06:14:31.971 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24670, 2024-09-21T06:14:31.971 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-21T06:14:31.971 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-21T06:14:31.971 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-21T06:14:31.971 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 69.376583784000005, 2024-09-21T06:14:31.971 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2024-09-21T06:14:31.971 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 79, 2024-09-21T06:14:31.971 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T06:14:31.971 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 108.80695277700001 2024-09-21T06:14:31.971 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:14:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0def222640 1 -- 172.21.15.3:0/3034875623 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0dd40b6620 msgr2=0x7f0dd40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0def222640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0dd40b6620 0x7f0dd40b8b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0de0005f30 tx=0x7f0de0005ec0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0def222640 1 -- 172.21.15.3:0/3034875623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de81083d0 msgr2=0x7f0de819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0def222640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de81083d0 0x7f0de819aa30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800ea80 tx=0x7f0dd800ef40 comp rx=0 tx=0).stop 2024-09-21T06:14:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0def222640 1 -- 172.21.15.3:0/3034875623 shutdown_connections 2024-09-21T06:14:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0def222640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0dd40b6620 0x7f0dd40b8b30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0def222640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0de8108d20 0x7f0de819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0def222640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de81083d0 0x7f0de819aa30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0def222640 1 --2- 172.21.15.3:0/3034875623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de8102ae0 0x7f0de819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0def222640 1 -- 172.21.15.3:0/3034875623 >> 172.21.15.3:0/3034875623 conn(0x7f0de8079760 msgr2=0x7f0de8101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.964+0000 7f0def222640 1 -- 172.21.15.3:0/3034875623 shutdown_connections 2024-09-21T06:14:31.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:31.965+0000 7f0def222640 1 -- 172.21.15.3:0/3034875623 wait complete. 2024-09-21T06:14:32.763 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T06:14:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:33.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.495+0000 7f5f5f392640 1 -- 172.21.15.3:0/748961312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f581083c0 msgr2=0x7f5f581087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:33.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.495+0000 7f5f5f392640 1 --2- 172.21.15.3:0/748961312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f581083c0 0x7f5f581087d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009900 tx=0x7f5f4802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.495+0000 7f5f5f392640 1 -- 172.21.15.3:0/748961312 shutdown_connections 2024-09-21T06:14:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.495+0000 7f5f5f392640 1 --2- 172.21.15.3:0/748961312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f58108d10 0x7f5f5810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.495+0000 7f5f5f392640 1 --2- 172.21.15.3:0/748961312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f581083c0 0x7f5f581087d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.495+0000 7f5f5f392640 1 --2- 172.21.15.3:0/748961312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f58102a80 0x7f5f58102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.495+0000 7f5f5f392640 1 -- 172.21.15.3:0/748961312 >> 172.21.15.3:0/748961312 conn(0x7f5f58079740 msgr2=0x7f5f58079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.495+0000 7f5f5f392640 1 -- 172.21.15.3:0/748961312 shutdown_connections 2024-09-21T06:14:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.495+0000 7f5f5f392640 1 -- 172.21.15.3:0/748961312 wait complete. 2024-09-21T06:14:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.495+0000 7f5f5f392640 1 Processor -- start 2024-09-21T06:14:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.495+0000 7f5f5f392640 1 -- start start 2024-09-21T06:14:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5f392640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f58102a80 0x7f5f5819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5f392640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f581083c0 0x7f5f5819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5f392640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f58108d10 0x7f5f5819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5f392640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5f58113af0 con 0x7f5f581083c0 2024-09-21T06:14:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5f392640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f58113970 con 0x7f5f58102a80 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5f392640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f58113c70 con 0x7f5f58108d10 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5c92d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f581083c0 0x7f5f5819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5c92d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f581083c0 0x7f5f5819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36692/0 (socket says 172.21.15.3:36692) 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5c92d640 1 -- 172.21.15.3:0/1379770312 learned_addr learned my addr 172.21.15.3:0/1379770312 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5d12e640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f58102a80 0x7f5f5819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5d92f640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f58108d10 0x7f5f5819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5c92d640 1 -- 172.21.15.3:0/1379770312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f58108d10 msgr2=0x7f5f5819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5c92d640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f58108d10 0x7f5f5819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5c92d640 1 -- 172.21.15.3:0/1379770312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f58102a80 msgr2=0x7f5f5819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5c92d640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f58102a80 0x7f5f5819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5c92d640 1 -- 172.21.15.3:0/1379770312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f581a03d0 con 0x7f5f581083c0 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5d92f640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f58108d10 0x7f5f5819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.496+0000 7f5f5c92d640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f581083c0 0x7f5f5819aac0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f48005ab0 tx=0x7f5f48003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.497+0000 7f5f4e7fc640 1 -- 172.21.15.3:0/1379770312 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4803f070 con 0x7f5f581083c0 2024-09-21T06:14:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.497+0000 7f5f4e7fc640 1 -- 172.21.15.3:0/1379770312 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f48031d60 con 0x7f5f581083c0 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.497+0000 7f5f5f392640 1 -- 172.21.15.3:0/1379770312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f581a0660 con 0x7f5f581083c0 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.497+0000 7f5f4e7fc640 1 -- 172.21.15.3:0/1379770312 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48035030 con 0x7f5f581083c0 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.497+0000 7f5f5f392640 1 -- 172.21.15.3:0/1379770312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f581a0b90 con 0x7f5f581083c0 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f65906740 1 Processor -- start 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f65906740 1 -- start start 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f4e7fc640 1 -- 172.21.15.3:0/1379770312 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f5f48042070 con 0x7f5f581083c0 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f65906740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 0x556595f46f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f65906740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 0x556595f47440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f65906740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b80280 0x556595f47980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f65906740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556595ebec10 con 0x556595b780a0 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f65906740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556595eae040 con 0x556595ba0f60 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f65906740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556595eb1e50 con 0x556595b80280 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f5d12e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 0x556595f47440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f5d12e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 0x556595f47440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36702/0 (socket says 172.21.15.3:36702) 2024-09-21T06:14:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f5d12e640 1 -- 172.21.15.3:0/3628078522 learned_addr learned my addr 172.21.15.3:0/3628078522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f5d92f640 1 --2- 172.21.15.3:0/3628078522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 0x556595f46f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f5c92d640 1 --2- 172.21.15.3:0/3628078522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b80280 0x556595f47980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f5d12e640 1 -- 172.21.15.3:0/3628078522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b80280 msgr2=0x556595f47980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f5d12e640 1 --2- 172.21.15.3:0/3628078522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b80280 0x556595f47980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f5d12e640 1 -- 172.21.15.3:0/3628078522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 msgr2=0x556595f46f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f5d12e640 1 --2- 172.21.15.3:0/3628078522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 0x556595f46f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.500+0000 7f5f5d12e640 1 -- 172.21.15.3:0/3628078522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556595c0c540 con 0x556595b780a0 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.501+0000 7f5f5d92f640 1 --2- 172.21.15.3:0/3628078522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 0x556595f46f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.501+0000 7f5f5d12e640 1 --2- 172.21.15.3:0/3628078522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 0x556595f47440 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400dce0 tx=0x7f5f5400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b2a1556432c0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.501+0000 7f5f4e7fc640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5f440b66a0 0x7f5f440b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.501+0000 7f5f5d92f640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5f440b66a0 0x7f5f440b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.501+0000 7f5f437fe640 1 -- 172.21.15.3:0/3628078522 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54011440 con 0x556595b780a0 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.501+0000 7f5f4e7fc640 1 -- 172.21.15.3:0/1379770312 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5f48107ac0 con 0x7f5f581083c0 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.501+0000 7f5f437fe640 1 -- 172.21.15.3:0/3628078522 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f54011a90 con 0x556595b780a0 2024-09-21T06:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.501+0000 7f5f437fe640 1 -- 172.21.15.3:0/3628078522 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5401ba70 con 0x556595b780a0 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.502+0000 7f5f65906740 1 -- 172.21.15.3:0/3628078522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 msgr2=0x556595f47440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.502+0000 7f5f65906740 1 --2- 172.21.15.3:0/3628078522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 0x556595f47440 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400dce0 tx=0x7f5f5400f730 comp rx=0 tx=0).stop 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.502+0000 7f5f5d92f640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5f440b66a0 0x7f5f440b8bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009e40 tx=0x7f5f50009390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.502+0000 7f5f65906740 1 -- 172.21.15.3:0/3628078522 shutdown_connections 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.502+0000 7f5f65906740 1 --2- 172.21.15.3:0/3628078522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b80280 0x556595f47980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.502+0000 7f5f65906740 1 --2- 172.21.15.3:0/3628078522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 0x556595f47440 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.502+0000 7f5f65906740 1 --2- 172.21.15.3:0/3628078522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 0x556595f46f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.502+0000 7f5f65906740 1 -- 172.21.15.3:0/3628078522 >> 172.21.15.3:0/3628078522 conn(0x556595b95350 msgr2=0x556595f44df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.502+0000 7f5f65906740 1 -- 172.21.15.3:0/3628078522 shutdown_connections 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.502+0000 7f5f65906740 1 -- 172.21.15.3:0/3628078522 wait complete. 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.502+0000 7f5f65906740 1 -- start start 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f65906740 1 Processor -- start 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f65906740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595b780a0 0x556595f56e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f65906740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 0x556595f57350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f65906740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595ba0f60 0x556595f62d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f65906740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556595d0f5f0 con 0x556595b80280 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f65906740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556595e49010 con 0x556595b780a0 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f65906740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556595f59a10 con 0x556595ba0f60 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f65906740 10 client.0 _ll_register_callbacks cb 0x556595ea8340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f5c92d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 0x556595f57350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f5c92d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 0x556595f57350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36706/0 (socket says 172.21.15.3:36706) 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f5c92d640 1 -- 172.21.15.3:0/2642715098 learned_addr learned my addr 172.21.15.3:0/2642715098 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f5c92d640 1 -- 172.21.15.3:0/2642715098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595ba0f60 msgr2=0x556595f62d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f5c92d640 1 --2- 172.21.15.3:0/2642715098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595ba0f60 0x556595f62d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f5c92d640 1 -- 172.21.15.3:0/2642715098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595b780a0 msgr2=0x556595f56e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f5d12e640 1 --2- 172.21.15.3:0/2642715098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595b780a0 0x556595f56e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f5c92d640 1 --2- 172.21.15.3:0/2642715098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595b780a0 0x556595f56e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.503+0000 7f5f5c92d640 1 -- 172.21.15.3:0/2642715098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556595e76a00 con 0x556595b80280 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f5c92d640 1 --2- 172.21.15.3:0/2642715098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 0x556595f57350 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f480cd9a0 tx=0x7f5f4803b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de55a5a418a370b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f40ff9640 1 -- 172.21.15.3:0/2642715098 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48049030 con 0x556595b80280 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f40ff9640 1 -- 172.21.15.3:0/2642715098 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5f480318b0 con 0x556595b80280 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f65906740 1 -- 172.21.15.3:0/2642715098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f54011a90 con 0x556595b80280 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f40ff9640 1 -- 172.21.15.3:0/2642715098 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f480cd360 con 0x556595b80280 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f65906740 10 client.15240 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f65906740 1 -- 172.21.15.3:0/2642715098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556595f65920 con 0x556595b80280 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f40ff9640 1 -- 172.21.15.3:0/2642715098 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f54011a90 con 0x556595b80280 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f40ff9640 1 -- 172.21.15.3:0/2642715098 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5f480c9360 con 0x556595b80280 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f65906740 10 client.15240 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f65906740 10 client.15240 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f65906740 20 client.15240 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f65906740 1 --2- 172.21.15.3:0/2642715098 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556595b5f8e0 0x556595f69490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f65906740 4 client.15240 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.504+0000 7f5f65906740 1 -- 172.21.15.3:0/2642715098 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556595ec0b60 con 0x556595b5f8e0 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.505+0000 7f5f5d12e640 1 --2- 172.21.15.3:0/2642715098 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556595b5f8e0 0x556595f69490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.505+0000 7f5f5d12e640 1 --2- 172.21.15.3:0/2642715098 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556595b5f8e0 0x556595f69490 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3f55d8fd3cdb360b server_cookie=ce47fb4476661362 in_seq=0 out_seq=0 2024-09-21T06:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.505+0000 7f5f40ff9640 10 client.15240 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.506+0000 7f5f40ff9640 1 -- 172.21.15.3:0/2642715098 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556595ec0b60 con 0x556595b5f8e0 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.506+0000 7f5f40ff9640 10 client.15240 handle_command_reply: tid=0 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.506+0000 7f5f65906740 1 client.15240 shutdown 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.506+0000 7f5f65906740 1 -- 172.21.15.3:0/2642715098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 msgr2=0x556595f57350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.506+0000 7f5f65906740 1 --2- 172.21.15.3:0/2642715098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 0x556595f57350 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f480cd9a0 tx=0x7f5f4803b890 comp rx=0 tx=0).stop 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.506+0000 7f5f65906740 1 -- 172.21.15.3:0/2642715098 shutdown_connections 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.506+0000 7f5f65906740 1 --2- 172.21.15.3:0/2642715098 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556595b5f8e0 0x556595f69490 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.506+0000 7f5f65906740 1 --2- 172.21.15.3:0/2642715098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595ba0f60 0x556595f62d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.506+0000 7f5f65906740 1 --2- 172.21.15.3:0/2642715098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 0x556595f57350 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.506+0000 7f5f40ff9640 0 client.15240 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.506+0000 7f5f65906740 1 --2- 172.21.15.3:0/2642715098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595b780a0 0x556595f56e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.506+0000 7f5f65906740 1 -- 172.21.15.3:0/2642715098 >> 172.21.15.3:0/2642715098 conn(0x556595b95350 msgr2=0x556595f4a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.506+0000 7f5f65906740 1 -- 172.21.15.3:0/2642715098 shutdown_connections 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.507+0000 7f5f65906740 1 -- 172.21.15.3:0/2642715098 wait complete. 2024-09-21T06:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.507+0000 7f5f65906740 20 client.15240 trim_cache size 0 max 16384 2024-09-21T06:14:33.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f65906740 1 Processor -- start 2024-09-21T06:14:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f65906740 1 -- start start 2024-09-21T06:14:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f65906740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 0x556595f62cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f65906740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b80280 0x556595f631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f65906740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 0x556595f51dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f65906740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556595f677d0 con 0x556595b780a0 2024-09-21T06:14:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f65906740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556595f67ea0 con 0x556595ba0f60 2024-09-21T06:14:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f65906740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556595f68570 con 0x556595b80280 2024-09-21T06:14:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f5d12e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 0x556595f62cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f5d12e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 0x556595f62cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36722/0 (socket says 172.21.15.3:36722) 2024-09-21T06:14:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f5d12e640 1 -- 172.21.15.3:0/3072165064 learned_addr learned my addr 172.21.15.3:0/3072165064 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f5c92d640 1 --2- 172.21.15.3:0/3072165064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b80280 0x556595f631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f5d12e640 1 -- 172.21.15.3:0/3072165064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b80280 msgr2=0x556595f631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f5d12e640 1 --2- 172.21.15.3:0/3072165064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b80280 0x556595f631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f5d92f640 1 --2- 172.21.15.3:0/3072165064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 0x556595f51dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.525+0000 7f5f5d12e640 1 -- 172.21.15.3:0/3072165064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 msgr2=0x556595f51dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:33.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.526+0000 7f5f5d12e640 1 --2- 172.21.15.3:0/3072165064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 0x556595f51dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.526+0000 7f5f5d12e640 1 -- 172.21.15.3:0/3072165064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556595f55110 con 0x556595b780a0 2024-09-21T06:14:33.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.526+0000 7f5f5d12e640 1 --2- 172.21.15.3:0/3072165064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 0x556595f62cb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5f5401b580 tx=0x7f5f5400ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf6b924cc16197 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:33.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.526+0000 7f5f42ffd640 1 -- 172.21.15.3:0/3072165064 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5400de50 con 0x556595b780a0 2024-09-21T06:14:33.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.526+0000 7f5f42ffd640 1 -- 172.21.15.3:0/3072165064 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f54005040 con 0x556595b780a0 2024-09-21T06:14:33.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.526+0000 7f5f42ffd640 1 -- 172.21.15.3:0/3072165064 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5401b920 con 0x556595b780a0 2024-09-21T06:14:33.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.527+0000 7f5f65906740 1 -- 172.21.15.3:0/3072165064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 msgr2=0x556595f62cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:33.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.527+0000 7f5f65906740 1 --2- 172.21.15.3:0/3072165064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 0x556595f62cb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5f5401b580 tx=0x7f5f5400ff60 comp rx=0 tx=0).stop 2024-09-21T06:14:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.527+0000 7f5f65906740 1 -- 172.21.15.3:0/3072165064 shutdown_connections 2024-09-21T06:14:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.527+0000 7f5f65906740 1 --2- 172.21.15.3:0/3072165064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b80280 0x556595f631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.527+0000 7f5f65906740 1 --2- 172.21.15.3:0/3072165064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 0x556595f51dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.527+0000 7f5f65906740 1 --2- 172.21.15.3:0/3072165064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b780a0 0x556595f62cb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.527+0000 7f5f65906740 1 -- 172.21.15.3:0/3072165064 >> 172.21.15.3:0/3072165064 conn(0x556595b95350 msgr2=0x556595f66c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.527+0000 7f5f65906740 1 -- 172.21.15.3:0/3072165064 shutdown_connections 2024-09-21T06:14:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.527+0000 7f5f65906740 1 -- 172.21.15.3:0/3072165064 wait complete. 2024-09-21T06:14:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.527+0000 7f5f65906740 1 -- start start 2024-09-21T06:14:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.527+0000 7f5f65906740 1 Processor -- start 2024-09-21T06:14:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f65906740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b780a0 0x556595f71df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f65906740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 0x556595f6b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f65906740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 0x556595f6b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f65906740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556595f69310 con 0x556595b80280 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f65906740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556595f68c40 con 0x556595ba0f60 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f65906740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556595f699e0 con 0x556595b780a0 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f65906740 10 client.0 _ll_register_callbacks cb 0x7f5f48005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f5c92d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 0x556595f6b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f5c92d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 0x556595f6b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36728/0 (socket says 172.21.15.3:36728) 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f5c92d640 1 -- 172.21.15.3:0/1569397419 learned_addr learned my addr 172.21.15.3:0/1569397419 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f5d92f640 1 --2- 172.21.15.3:0/1569397419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 0x556595f6b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f5d12e640 1 --2- 172.21.15.3:0/1569397419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b780a0 0x556595f71df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f5c92d640 1 -- 172.21.15.3:0/1569397419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b780a0 msgr2=0x556595f71df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f5c92d640 1 --2- 172.21.15.3:0/1569397419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b780a0 0x556595f71df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f5c92d640 1 -- 172.21.15.3:0/1569397419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 msgr2=0x556595f6b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f5c92d640 1 --2- 172.21.15.3:0/1569397419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 0x556595f6b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.528+0000 7f5f5c92d640 1 -- 172.21.15.3:0/1569397419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556595f6be50 con 0x556595b80280 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f5c92d640 1 --2- 172.21.15.3:0/1569397419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 0x556595f6b2a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5f48000c00 tx=0x7f5f4802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe0b457933f46eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f40ff9640 1 -- 172.21.15.3:0/1569397419 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48049070 con 0x556595b80280 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f40ff9640 1 -- 172.21.15.3:0/1569397419 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5f480ccc70 con 0x556595b80280 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f65906740 1 -- 172.21.15.3:0/1569397419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f54005040 con 0x556595b80280 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f40ff9640 1 -- 172.21.15.3:0/1569397419 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f480cd180 con 0x556595b80280 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f65906740 10 client.15246 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f65906740 1 -- 172.21.15.3:0/1569397419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556595f71860 con 0x556595b80280 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f40ff9640 1 -- 172.21.15.3:0/1569397419 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f54005040 con 0x556595b80280 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f40ff9640 1 -- 172.21.15.3:0/1569397419 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5f480d3680 con 0x556595b80280 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f65906740 10 client.15246 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f65906740 10 client.15246 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f65906740 20 client.15246 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f65906740 1 --2- 172.21.15.3:0/1569397419 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556595b34c90 0x556595fd8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f65906740 4 client.15246 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.529+0000 7f5f65906740 1 -- 172.21.15.3:0/1569397419 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "status"}) -- 0x556595ebc610 con 0x556595b34c90 2024-09-21T06:14:33.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.530+0000 7f5f5d12e640 1 --2- 172.21.15.3:0/1569397419 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556595b34c90 0x556595fd8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.530+0000 7f5f40ff9640 10 client.15246 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.530+0000 7f5f5d12e640 1 --2- 172.21.15.3:0/1569397419 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556595b34c90 0x556595fd8040 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3f886f66ae72ad3e server_cookie=541e13d3b4d99ad in_seq=0 out_seq=0 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.530+0000 7f5f40ff9640 1 -- 172.21.15.3:0/1569397419 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556595ebc610 con 0x556595b34c90 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.530+0000 7f5f40ff9640 10 client.15246 handle_command_reply: tid=0 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.530+0000 7f5f65906740 1 client.15246 shutdown 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.530+0000 7f5f65906740 1 -- 172.21.15.3:0/1569397419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 msgr2=0x556595f6b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.530+0000 7f5f65906740 1 --2- 172.21.15.3:0/1569397419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 0x556595f6b2a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5f48000c00 tx=0x7f5f4802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f65906740 1 -- 172.21.15.3:0/1569397419 shutdown_connections 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f65906740 1 --2- 172.21.15.3:0/1569397419 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556595b34c90 0x556595fd8040 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f65906740 1 --2- 172.21.15.3:0/1569397419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556595ba0f60 0x556595f6b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f65906740 1 --2- 172.21.15.3:0/1569397419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556595b80280 0x556595f6b2a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f65906740 1 --2- 172.21.15.3:0/1569397419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556595b780a0 0x556595f71df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f40ff9640 0 client.15246 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:14:33.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f65906740 1 -- 172.21.15.3:0/1569397419 >> 172.21.15.3:0/1569397419 conn(0x556595b95350 msgr2=0x556595f51dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:33.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f65906740 1 -- 172.21.15.3:0/1569397419 shutdown_connections 2024-09-21T06:14:33.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f65906740 1 -- 172.21.15.3:0/1569397419 wait complete. 2024-09-21T06:14:33.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f65906740 20 client.15246 trim_cache size 0 max 16384 2024-09-21T06:14:33.537 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:14:33.537 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "38d3c75c-77df-11ef-baf5-efdc52797490", 2024-09-21T06:14:33.537 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 3, 2024-09-21T06:14:33.537 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24674, 2024-09-21T06:14:33.537 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-21T06:14:33.537 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-21T06:14:33.537 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-21T06:14:33.537 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 71.959509030000007, 2024-09-21T06:14:33.537 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2024-09-21T06:14:33.538 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 79, 2024-09-21T06:14:33.538 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T06:14:33.538 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 105.093685369 2024-09-21T06:14:33.538 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:14:33.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f5f392640 1 -- 172.21.15.3:0/1379770312 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5f440b66a0 msgr2=0x7f5f440b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:33.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f5f392640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5f440b66a0 0x7f5f440b8bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009e40 tx=0x7f5f50009390 comp rx=0 tx=0).stop 2024-09-21T06:14:33.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f5f392640 1 -- 172.21.15.3:0/1379770312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f581083c0 msgr2=0x7f5f5819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:33.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.531+0000 7f5f5f392640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f581083c0 0x7f5f5819aac0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f48005ab0 tx=0x7f5f48003cd0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.532+0000 7f5f5f392640 1 -- 172.21.15.3:0/1379770312 shutdown_connections 2024-09-21T06:14:33.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.532+0000 7f5f5f392640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5f440b66a0 0x7f5f440b8bb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.532+0000 7f5f5f392640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f58108d10 0x7f5f5819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.532+0000 7f5f5f392640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f581083c0 0x7f5f5819aac0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.532+0000 7f5f5f392640 1 --2- 172.21.15.3:0/1379770312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f58102a80 0x7f5f5819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:33.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.532+0000 7f5f5f392640 1 -- 172.21.15.3:0/1379770312 >> 172.21.15.3:0/1379770312 conn(0x7f5f58079740 msgr2=0x7f5f58104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:33.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.532+0000 7f5f5f392640 1 -- 172.21.15.3:0/1379770312 shutdown_connections 2024-09-21T06:14:33.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:33.532+0000 7f5f5f392640 1 -- 172.21.15.3:0/1379770312 wait complete. 2024-09-21T06:14:33.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:33 smithi003 ceph-mon[31227]: pgmap v311: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:33.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:33 smithi137 ceph-mon[34464]: pgmap v311: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:33 smithi204 ceph-mon[34659]: pgmap v311: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:34.319 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T06:14:34.581 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.081+0000 7f2bd82b3640 1 -- 172.21.15.3:0/1247318408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bd0109df0 msgr2=0x7f2bd0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.081+0000 7f2bd82b3640 1 --2- 172.21.15.3:0/1247318408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bd0109df0 0x7f2bd0110620 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009ca0 tx=0x7f2bc802a8e0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.081+0000 7f2bd82b3640 1 -- 172.21.15.3:0/1247318408 shutdown_connections 2024-09-21T06:14:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.081+0000 7f2bd82b3640 1 --2- 172.21.15.3:0/1247318408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bd01094a0 0x7f2bd01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.081+0000 7f2bd82b3640 1 --2- 172.21.15.3:0/1247318408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bd0109df0 0x7f2bd0110620 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.081+0000 7f2bd82b3640 1 --2- 172.21.15.3:0/1247318408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bd0103b60 0x7f2bd0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.081+0000 7f2bd82b3640 1 -- 172.21.15.3:0/1247318408 >> 172.21.15.3:0/1247318408 conn(0x7f2bd007b850 msgr2=0x7f2bd007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.081+0000 7f2bd82b3640 1 -- 172.21.15.3:0/1247318408 shutdown_connections 2024-09-21T06:14:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.081+0000 7f2bd82b3640 1 -- 172.21.15.3:0/1247318408 wait complete. 2024-09-21T06:14:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.082+0000 7f2bd82b3640 1 Processor -- start 2024-09-21T06:14:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.082+0000 7f2bd82b3640 1 -- start start 2024-09-21T06:14:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.082+0000 7f2bd82b3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bd0103b60 0x7f2bd0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.082+0000 7f2bd82b3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bd01094a0 0x7f2bd0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.082+0000 7f2bd82b3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bd0109df0 0x7f2bd0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.082+0000 7f2bd82b3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0113af0 con 0x7f2bd0109df0 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.082+0000 7f2bd82b3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0113970 con 0x7f2bd01094a0 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.082+0000 7f2bd82b3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0113c70 con 0x7f2bd0103b60 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.082+0000 7f2bd6850640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bd0109df0 0x7f2bd0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.082+0000 7f2bd6850640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bd0109df0 0x7f2bd0076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36752/0 (socket says 172.21.15.3:36752) 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.082+0000 7f2bd6850640 1 -- 172.21.15.3:0/2148734971 learned_addr learned my addr 172.21.15.3:0/2148734971 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.083+0000 7f2bd584e640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bd01094a0 0x7f2bd0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.083+0000 7f2bd6850640 1 -- 172.21.15.3:0/2148734971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bd0103b60 msgr2=0x7f2bd0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.083+0000 7f2bd604f640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bd0103b60 0x7f2bd0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.083+0000 7f2bd6850640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bd0103b60 0x7f2bd0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.083+0000 7f2bd6850640 1 -- 172.21.15.3:0/2148734971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bd01094a0 msgr2=0x7f2bd0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.083+0000 7f2bd6850640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bd01094a0 0x7f2bd0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.083+0000 7f2bd6850640 1 -- 172.21.15.3:0/2148734971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd01a87a0 con 0x7f2bd0109df0 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.083+0000 7f2bd604f640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bd0103b60 0x7f2bd0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.083+0000 7f2bd6850640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bd0109df0 0x7f2bd0076190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8005cc0 tx=0x7f2bc80039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.083+0000 7f2bc77fe640 1 -- 172.21.15.3:0/2148734971 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8003de0 con 0x7f2bd0109df0 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.084+0000 7f2bd82b3640 1 -- 172.21.15.3:0/2148734971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd01a8a30 con 0x7f2bd0109df0 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.084+0000 7f2bc77fe640 1 -- 172.21.15.3:0/2148734971 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc80044f0 con 0x7f2bd0109df0 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.084+0000 7f2bc77fe640 1 -- 172.21.15.3:0/2148734971 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8030cb0 con 0x7f2bd0109df0 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.084+0000 7f2bd82b3640 1 -- 172.21.15.3:0/2148734971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd01a8fc0 con 0x7f2bd0109df0 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.087+0000 7f2bde827740 1 Processor -- start 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.087+0000 7f2bc77fe640 1 -- 172.21.15.3:0/2148734971 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f2bc803a400 con 0x7f2bd0109df0 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.087+0000 7f2bde827740 1 -- start start 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bde827740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 0x55a889a06f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bde827740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8896380a0 0x55a889a07440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bde827740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a889640280 0x55a889a07980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bde827740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a88997ec10 con 0x55a889660f60 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bde827740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a88996e040 con 0x55a889640280 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bde827740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a889971e50 con 0x55a8896380a0 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bd584e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 0x55a889a06f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bd584e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 0x55a889a06f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36762/0 (socket says 172.21.15.3:36762) 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bd584e640 1 -- 172.21.15.3:0/2270192623 learned_addr learned my addr 172.21.15.3:0/2270192623 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bd604f640 1 --2- 172.21.15.3:0/2270192623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8896380a0 0x55a889a07440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bd584e640 1 -- 172.21.15.3:0/2270192623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8896380a0 msgr2=0x55a889a07440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bd584e640 1 --2- 172.21.15.3:0/2270192623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8896380a0 0x55a889a07440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bd584e640 1 -- 172.21.15.3:0/2270192623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a889640280 msgr2=0x55a889a07980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bd584e640 1 --2- 172.21.15.3:0/2270192623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a889640280 0x55a889a07980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bd584e640 1 -- 172.21.15.3:0/2270192623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8896cc540 con 0x55a889660f60 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bc77fe640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2bbc0b65f0 0x7f2bbc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bd584e640 1 --2- 172.21.15.3:0/2270192623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 0x55a889a06f00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009a80 tx=0x7f2bc0009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506ef77808ba5da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:35.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bd604f640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2bbc0b65f0 0x7f2bbc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.088+0000 7f2bc4d9a640 1 -- 172.21.15.3:0/2270192623 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0018020 con 0x55a889660f60 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.089+0000 7f2bc77fe640 1 -- 172.21.15.3:0/2148734971 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2bc8108a50 con 0x7f2bd0109df0 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.089+0000 7f2bc4d9a640 1 -- 172.21.15.3:0/2270192623 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc0010dc0 con 0x55a889660f60 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.089+0000 7f2bc4d9a640 1 -- 172.21.15.3:0/2270192623 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00136d0 con 0x55a889660f60 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.089+0000 7f2bde827740 1 -- 172.21.15.3:0/2270192623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 msgr2=0x55a889a06f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.089+0000 7f2bde827740 1 --2- 172.21.15.3:0/2270192623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 0x55a889a06f00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009a80 tx=0x7f2bc0009f40 comp rx=0 tx=0).stop 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.089+0000 7f2bd604f640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2bbc0b65f0 0x7f2bbc0b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55a889a05b80 tx=0x7f2bcc009330 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.089+0000 7f2bde827740 1 -- 172.21.15.3:0/2270192623 shutdown_connections 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 --2- 172.21.15.3:0/2270192623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8896380a0 0x55a889a07440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 --2- 172.21.15.3:0/2270192623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a889640280 0x55a889a07980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 --2- 172.21.15.3:0/2270192623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 0x55a889a06f00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 -- 172.21.15.3:0/2270192623 >> 172.21.15.3:0/2270192623 conn(0x55a889655350 msgr2=0x55a889a04df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 -- 172.21.15.3:0/2270192623 shutdown_connections 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 -- 172.21.15.3:0/2270192623 wait complete. 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 -- start start 2024-09-21T06:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 Processor -- start 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8896380a0 0x55a889a12780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889640280 0x55a889a12d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889660f60 0x55a889a1ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a8897cf5f0 con 0x55a889640280 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a889909010 con 0x55a8896380a0 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.090+0000 7f2bde827740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a889a156e0 con 0x55a889660f60 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bde827740 10 client.0 _ll_register_callbacks cb 0x55a889968340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bd584e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889640280 0x55a889a12d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bd584e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889640280 0x55a889a12d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36772/0 (socket says 172.21.15.3:36772) 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bd584e640 1 -- 172.21.15.3:0/2270889541 learned_addr learned my addr 172.21.15.3:0/2270889541 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bd6850640 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889660f60 0x55a889a1ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bd604f640 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8896380a0 0x55a889a12780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bd584e640 1 -- 172.21.15.3:0/2270889541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889660f60 msgr2=0x55a889a1ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bd584e640 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889660f60 0x55a889a1ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bd584e640 1 -- 172.21.15.3:0/2270889541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8896380a0 msgr2=0x55a889a12780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bd584e640 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8896380a0 0x55a889a12780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bd584e640 1 -- 172.21.15.3:0/2270889541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a889936a00 con 0x55a889640280 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bd604f640 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8896380a0 0x55a889a12780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bd584e640 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889640280 0x55a889a12d20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0004700 tx=0x7f2bc001d6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450e78bf5df9f1a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bb9ffb640 1 -- 172.21.15.3:0/2270889541 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0018070 con 0x55a889640280 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bb9ffb640 1 -- 172.21.15.3:0/2270889541 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2bc001db00 con 0x55a889640280 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.091+0000 7f2bde827740 1 -- 172.21.15.3:0/2270889541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc0010dc0 con 0x55a889640280 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.092+0000 7f2bb9ffb640 1 -- 172.21.15.3:0/2270889541 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2bc0014a60 con 0x55a889640280 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.092+0000 7f2bde827740 10 client.15258 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.092+0000 7f2bde827740 1 -- 172.21.15.3:0/2270889541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a889a215a0 con 0x55a889640280 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.092+0000 7f2bb9ffb640 1 -- 172.21.15.3:0/2270889541 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc0010dc0 con 0x55a889640280 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.092+0000 7f2bb9ffb640 1 -- 172.21.15.3:0/2270889541 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2bc0024020 con 0x55a889640280 2024-09-21T06:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.092+0000 7f2bde827740 10 client.15258 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.092+0000 7f2bde827740 10 client.15258 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.092+0000 7f2bde827740 20 client.15258 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.092+0000 7f2bde827740 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55a88961f8e0 0x55a889a24c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.092+0000 7f2bde827740 4 client.15258 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.092+0000 7f2bde827740 1 -- 172.21.15.3:0/2270889541 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a889980b60 con 0x55a88961f8e0 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.093+0000 7f2bd604f640 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55a88961f8e0 0x55a889a24c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.093+0000 7f2bb9ffb640 10 client.15258 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.093+0000 7f2bd604f640 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55a88961f8e0 0x55a889a24c60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=901cfef6fedc77c4 server_cookie=de9d6ea9e9266ffa in_seq=0 out_seq=0 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bb9ffb640 1 -- 172.21.15.3:0/2270889541 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a889980b60 con 0x55a88961f8e0 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bb9ffb640 10 client.15258 handle_command_reply: tid=0 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bde827740 1 client.15258 shutdown 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bde827740 1 -- 172.21.15.3:0/2270889541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889640280 msgr2=0x55a889a12d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bde827740 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889640280 0x55a889a12d20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0004700 tx=0x7f2bc001d6d0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bde827740 1 -- 172.21.15.3:0/2270889541 shutdown_connections 2024-09-21T06:14:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bde827740 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55a88961f8e0 0x55a889a24c60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bde827740 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889660f60 0x55a889a1ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bde827740 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889640280 0x55a889a12d20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bde827740 1 --2- 172.21.15.3:0/2270889541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8896380a0 0x55a889a12780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bb9ffb640 0 client.15258 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:14:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bde827740 1 -- 172.21.15.3:0/2270889541 >> 172.21.15.3:0/2270889541 conn(0x55a889655350 msgr2=0x55a889a0a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bde827740 1 -- 172.21.15.3:0/2270889541 shutdown_connections 2024-09-21T06:14:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bde827740 1 -- 172.21.15.3:0/2270889541 wait complete. 2024-09-21T06:14:35.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.094+0000 7f2bde827740 20 client.15258 trim_cache size 0 max 16384 2024-09-21T06:14:35.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.112+0000 7f2bde827740 1 Processor -- start 2024-09-21T06:14:35.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.112+0000 7f2bde827740 1 -- start start 2024-09-21T06:14:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bde827740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a8896380a0 0x55a889a0d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bde827740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889640280 0x55a889a1e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bde827740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a889660f60 0x55a889a1eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bde827740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a889a22fa0 con 0x55a8896380a0 2024-09-21T06:14:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bde827740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a889a23670 con 0x55a889660f60 2024-09-21T06:14:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bde827740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a889a23d40 con 0x55a889640280 2024-09-21T06:14:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bd604f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a8896380a0 0x55a889a0d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bd604f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a8896380a0 0x55a889a0d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36774/0 (socket says 172.21.15.3:36774) 2024-09-21T06:14:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bd604f640 1 -- 172.21.15.3:0/15888941 learned_addr learned my addr 172.21.15.3:0/15888941 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bd6850640 1 --2- 172.21.15.3:0/15888941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a889660f60 0x55a889a1eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bd584e640 1 --2- 172.21.15.3:0/15888941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889640280 0x55a889a1e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bd604f640 1 -- 172.21.15.3:0/15888941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889640280 msgr2=0x55a889a1e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bd604f640 1 --2- 172.21.15.3:0/15888941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889640280 0x55a889a1e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bd604f640 1 -- 172.21.15.3:0/15888941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a889660f60 msgr2=0x55a889a1eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bd604f640 1 --2- 172.21.15.3:0/15888941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a889660f60 0x55a889a1eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bd604f640 1 -- 172.21.15.3:0/15888941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a889a09d60 con 0x55a8896380a0 2024-09-21T06:14:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bd604f640 1 --2- 172.21.15.3:0/15888941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a8896380a0 0x55a889a0d020 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0077160 tx=0x7f2bcc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47359427004e17ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.113+0000 7f2bbbfff640 1 -- 172.21.15.3:0/15888941 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc004220 con 0x55a8896380a0 2024-09-21T06:14:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.114+0000 7f2bbbfff640 1 -- 172.21.15.3:0/15888941 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bcc0043c0 con 0x55a8896380a0 2024-09-21T06:14:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.114+0000 7f2bbbfff640 1 -- 172.21.15.3:0/15888941 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc0208b0 con 0x55a8896380a0 2024-09-21T06:14:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.114+0000 7f2bde827740 1 -- 172.21.15.3:0/15888941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a8896380a0 msgr2=0x55a889a0d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.114+0000 7f2bde827740 1 --2- 172.21.15.3:0/15888941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a8896380a0 0x55a889a0d020 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0077160 tx=0x7f2bcc016000 comp rx=0 tx=0).stop 2024-09-21T06:14:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.114+0000 7f2bde827740 1 -- 172.21.15.3:0/15888941 shutdown_connections 2024-09-21T06:14:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.114+0000 7f2bde827740 1 --2- 172.21.15.3:0/15888941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889640280 0x55a889a1e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.114+0000 7f2bde827740 1 --2- 172.21.15.3:0/15888941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a889660f60 0x55a889a1eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.114+0000 7f2bde827740 1 --2- 172.21.15.3:0/15888941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a8896380a0 0x55a889a0d020 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.114+0000 7f2bde827740 1 -- 172.21.15.3:0/15888941 >> 172.21.15.3:0/15888941 conn(0x55a889655350 msgr2=0x55a889a11000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.114+0000 7f2bde827740 1 -- 172.21.15.3:0/15888941 shutdown_connections 2024-09-21T06:14:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.114+0000 7f2bde827740 1 -- 172.21.15.3:0/15888941 wait complete. 2024-09-21T06:14:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.115+0000 7f2bde827740 1 -- start start 2024-09-21T06:14:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.115+0000 7f2bde827740 1 Processor -- start 2024-09-21T06:14:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.115+0000 7f2bde827740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8896380a0 0x55a889a270f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.115+0000 7f2bde827740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889640280 0x55a889a27630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.115+0000 7f2bde827740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 0x55a889a27b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.115+0000 7f2bde827740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a889a24ae0 con 0x55a889660f60 2024-09-21T06:14:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.115+0000 7f2bde827740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a889a24410 con 0x55a8896380a0 2024-09-21T06:14:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.115+0000 7f2bde827740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a889a251b0 con 0x55a889640280 2024-09-21T06:14:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.115+0000 7f2bde827740 10 client.0 _ll_register_callbacks cb 0x7f2bc0009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.115+0000 7f2bd6850640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 0x55a889a27b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.115+0000 7f2bd6850640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 0x55a889a27b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36786/0 (socket says 172.21.15.3:36786) 2024-09-21T06:14:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.115+0000 7f2bd6850640 1 -- 172.21.15.3:0/3448034683 learned_addr learned my addr 172.21.15.3:0/3448034683 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.115+0000 7f2bd584e640 1 --2- 172.21.15.3:0/3448034683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889640280 0x55a889a27630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bd604f640 1 --2- 172.21.15.3:0/3448034683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8896380a0 0x55a889a270f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bd6850640 1 -- 172.21.15.3:0/3448034683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889640280 msgr2=0x55a889a27630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bd6850640 1 --2- 172.21.15.3:0/3448034683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889640280 0x55a889a27630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bd6850640 1 -- 172.21.15.3:0/3448034683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8896380a0 msgr2=0x55a889a270f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bd6850640 1 --2- 172.21.15.3:0/3448034683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8896380a0 0x55a889a270f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bd6850640 1 -- 172.21.15.3:0/3448034683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a889a2cab0 con 0x55a889660f60 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bd6850640 1 --2- 172.21.15.3:0/3448034683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 0x55a889a27b90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80ce6a0 tx=0x7f2bc81067a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740d088f3727a744 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bb9ffb640 1 -- 172.21.15.3:0/3448034683 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80cf0a0 con 0x55a889660f60 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bb9ffb640 1 -- 172.21.15.3:0/3448034683 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2bc80cac10 con 0x55a889660f60 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bde827740 1 -- 172.21.15.3:0/3448034683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bcc0043c0 con 0x55a889660f60 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bb9ffb640 1 -- 172.21.15.3:0/3448034683 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2bc80098b0 con 0x55a889660f60 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bde827740 10 client.15264 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bde827740 1 -- 172.21.15.3:0/3448034683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a889a905d0 con 0x55a889660f60 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.116+0000 7f2bb9ffb640 1 -- 172.21.15.3:0/3448034683 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc80b8080 con 0x55a889660f60 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.117+0000 7f2bb9ffb640 1 -- 172.21.15.3:0/3448034683 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2bc80cdd20 con 0x55a889660f60 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.117+0000 7f2bde827740 10 client.15264 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.117+0000 7f2bde827740 10 client.15264 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:14:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.117+0000 7f2bde827740 20 client.15264 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.117+0000 7f2bde827740 1 --2- 172.21.15.3:0/3448034683 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55a8895f4c90 0x55a889a93ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.117+0000 7f2bde827740 4 client.15264 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:14:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.117+0000 7f2bde827740 1 -- 172.21.15.3:0/3448034683 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "status"}) -- 0x55a88997c610 con 0x55a8895f4c90 2024-09-21T06:14:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.117+0000 7f2bd604f640 1 --2- 172.21.15.3:0/3448034683 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55a8895f4c90 0x55a889a93ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bd604f640 1 --2- 172.21.15.3:0/3448034683 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55a8895f4c90 0x55a889a93ec0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=869432a63cc3eb92 server_cookie=611eeb09e537791c in_seq=0 out_seq=0 2024-09-21T06:14:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bb9ffb640 10 client.15264 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:14:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bb9ffb640 1 -- 172.21.15.3:0/3448034683 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a88997c610 con 0x55a8895f4c90 2024-09-21T06:14:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bb9ffb640 10 client.15264 handle_command_reply: tid=0 2024-09-21T06:14:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bde827740 1 client.15264 shutdown 2024-09-21T06:14:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bde827740 1 -- 172.21.15.3:0/3448034683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 msgr2=0x55a889a27b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bde827740 1 --2- 172.21.15.3:0/3448034683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 0x55a889a27b90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80ce6a0 tx=0x7f2bc81067a0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bde827740 1 -- 172.21.15.3:0/3448034683 shutdown_connections 2024-09-21T06:14:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bde827740 1 --2- 172.21.15.3:0/3448034683 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55a8895f4c90 0x55a889a93ec0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bde827740 1 --2- 172.21.15.3:0/3448034683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a889640280 0x55a889a27630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bde827740 1 --2- 172.21.15.3:0/3448034683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a889660f60 0x55a889a27b90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bde827740 1 --2- 172.21.15.3:0/3448034683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8896380a0 0x55a889a270f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bde827740 1 -- 172.21.15.3:0/3448034683 >> 172.21.15.3:0/3448034683 conn(0x55a889655350 msgr2=0x55a889a04600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bb9ffb640 0 client.15264 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:14:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bde827740 1 -- 172.21.15.3:0/3448034683 shutdown_connections 2024-09-21T06:14:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bde827740 1 -- 172.21.15.3:0/3448034683 wait complete. 2024-09-21T06:14:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.118+0000 7f2bde827740 20 client.15264 trim_cache size 0 max 16384 2024-09-21T06:14:35.125 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:14:35.126 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "38d3c75c-77df-11ef-baf5-efdc52797490", 2024-09-21T06:14:35.126 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-09-21T06:14:35.126 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24679, 2024-09-21T06:14:35.126 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-21T06:14:35.126 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-21T06:14:35.126 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-21T06:14:35.126 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 74.554219509999996, 2024-09-21T06:14:35.126 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2024-09-21T06:14:35.126 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 79, 2024-09-21T06:14:35.126 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T06:14:35.126 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 109.39695501999999 2024-09-21T06:14:35.126 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:14:35.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.119+0000 7f2bd82b3640 1 -- 172.21.15.3:0/2148734971 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2bbc0b65f0 msgr2=0x7f2bbc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:35.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.119+0000 7f2bd82b3640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2bbc0b65f0 0x7f2bbc0b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55a889a05b80 tx=0x7f2bcc009330 comp rx=0 tx=0).stop 2024-09-21T06:14:35.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.119+0000 7f2bd82b3640 1 -- 172.21.15.3:0/2148734971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bd0109df0 msgr2=0x7f2bd0076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:35.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.119+0000 7f2bd82b3640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bd0109df0 0x7f2bd0076190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8005cc0 tx=0x7f2bc80039f0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.119+0000 7f2bd82b3640 1 -- 172.21.15.3:0/2148734971 shutdown_connections 2024-09-21T06:14:35.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.119+0000 7f2bd82b3640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2bbc0b65f0 0x7f2bbc0b8b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.119+0000 7f2bd82b3640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bd0109df0 0x7f2bd0076190 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.119+0000 7f2bd82b3640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bd01094a0 0x7f2bd0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.119+0000 7f2bd82b3640 1 --2- 172.21.15.3:0/2148734971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bd0103b60 0x7f2bd0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:35.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.119+0000 7f2bd82b3640 1 -- 172.21.15.3:0/2148734971 >> 172.21.15.3:0/2148734971 conn(0x7f2bd007b850 msgr2=0x7f2bd0107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:35.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.119+0000 7f2bd82b3640 1 -- 172.21.15.3:0/2148734971 shutdown_connections 2024-09-21T06:14:35.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:35.119+0000 7f2bd82b3640 1 -- 172.21.15.3:0/2148734971 wait complete. 2024-09-21T06:14:35.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:35 smithi003 ceph-mon[31227]: pgmap v312: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:35.825 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T06:14:35.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:35 smithi137 ceph-mon[34464]: pgmap v312: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:35.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:35 smithi204 ceph-mon[34659]: pgmap v312: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:36.083 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:36.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.582+0000 7f18b72d7640 1 -- 172.21.15.3:0/694149723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b01052a0 msgr2=0x7f18b010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:36.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.582+0000 7f18b72d7640 1 --2- 172.21.15.3:0/694149723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b01052a0 0x7f18b010d930 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009900 tx=0x7f18a002a410 comp rx=0 tx=0).stop 2024-09-21T06:14:36.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.582+0000 7f18b72d7640 1 -- 172.21.15.3:0/694149723 shutdown_connections 2024-09-21T06:14:36.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.582+0000 7f18b72d7640 1 --2- 172.21.15.3:0/694149723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b010de70 0x7f18b0110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.582+0000 7f18b72d7640 1 --2- 172.21.15.3:0/694149723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b01052a0 0x7f18b010d930 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.582+0000 7f18b72d7640 1 --2- 172.21.15.3:0/694149723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18b01048c0 0x7f18b0104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.582+0000 7f18b72d7640 1 -- 172.21.15.3:0/694149723 >> 172.21.15.3:0/694149723 conn(0x7f18b0079760 msgr2=0x7f18b0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.582+0000 7f18b72d7640 1 -- 172.21.15.3:0/694149723 shutdown_connections 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.582+0000 7f18b72d7640 1 -- 172.21.15.3:0/694149723 wait complete. 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.583+0000 7f18b72d7640 1 Processor -- start 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.583+0000 7f18b72d7640 1 -- start start 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.583+0000 7f18b72d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18b01048c0 0x7f18b019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.583+0000 7f18b72d7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b01052a0 0x7f18b019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.583+0000 7f18b72d7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b010de70 0x7f18b019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.583+0000 7f18b72d7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0113b10 con 0x7f18b01052a0 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.583+0000 7f18b72d7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0113990 con 0x7f18b010de70 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.583+0000 7f18b72d7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0113c90 con 0x7f18b01048c0 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.583+0000 7f18b4872640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b01052a0 0x7f18b019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.583+0000 7f18b4872640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b01052a0 0x7f18b019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33394/0 (socket says 172.21.15.3:33394) 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.583+0000 7f18b4872640 1 -- 172.21.15.3:0/2740487388 learned_addr learned my addr 172.21.15.3:0/2740487388 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:36.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18b5073640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18b01048c0 0x7f18b019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18b5874640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b010de70 0x7f18b019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18b4872640 1 -- 172.21.15.3:0/2740487388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18b01048c0 msgr2=0x7f18b019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:36.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18b4872640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18b01048c0 0x7f18b019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18b4872640 1 -- 172.21.15.3:0/2740487388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b010de70 msgr2=0x7f18b019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:36.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18b4872640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b010de70 0x7f18b019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18b4872640 1 -- 172.21.15.3:0/2740487388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b01a0530 con 0x7f18b01052a0 2024-09-21T06:14:36.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18b5874640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b010de70 0x7f18b019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:36.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18b4872640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b01052a0 0x7f18b019ab60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f18a002a930 tx=0x7f18a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:36.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18a67fc640 1 -- 172.21.15.3:0/2740487388 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a003f030 con 0x7f18b01052a0 2024-09-21T06:14:36.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18b72d7640 1 -- 172.21.15.3:0/2740487388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b01a07c0 con 0x7f18b01052a0 2024-09-21T06:14:36.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18b72d7640 1 -- 172.21.15.3:0/2740487388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b01a6f80 con 0x7f18b01052a0 2024-09-21T06:14:36.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18a67fc640 1 -- 172.21.15.3:0/2740487388 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a0003cb0 con 0x7f18b01052a0 2024-09-21T06:14:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.584+0000 7f18a67fc640 1 -- 172.21.15.3:0/2740487388 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0049060 con 0x7f18b01052a0 2024-09-21T06:14:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.587+0000 7f18bd84b740 1 Processor -- start 2024-09-21T06:14:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.587+0000 7f18bd84b740 1 -- start start 2024-09-21T06:14:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.587+0000 7f18a67fc640 1 -- 172.21.15.3:0/2740487388 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f18a0042030 con 0x7f18b01052a0 2024-09-21T06:14:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.587+0000 7f18bd84b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a82f60 0x564084e28f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.587+0000 7f18bd84b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a5a0a0 0x564084e29440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.587+0000 7f18bd84b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a62280 0x564084e29980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.587+0000 7f18bd84b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564084da0c10 con 0x564084a82f60 2024-09-21T06:14:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.587+0000 7f18bd84b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564084d90040 con 0x564084a62280 2024-09-21T06:14:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.587+0000 7f18bd84b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564084d93e50 con 0x564084a5a0a0 2024-09-21T06:14:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18b5874640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a82f60 0x564084e28f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18b5874640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a82f60 0x564084e28f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33408/0 (socket says 172.21.15.3:33408) 2024-09-21T06:14:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18b5874640 1 -- 172.21.15.3:0/304383887 learned_addr learned my addr 172.21.15.3:0/304383887 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18b4872640 1 --2- 172.21.15.3:0/304383887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a62280 0x564084e29980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18b5073640 1 --2- 172.21.15.3:0/304383887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a5a0a0 0x564084e29440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18b5874640 1 -- 172.21.15.3:0/304383887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a5a0a0 msgr2=0x564084e29440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18b5874640 1 --2- 172.21.15.3:0/304383887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a5a0a0 0x564084e29440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18b5874640 1 -- 172.21.15.3:0/304383887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a62280 msgr2=0x564084e29980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18b5874640 1 --2- 172.21.15.3:0/304383887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a62280 0x564084e29980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18b5874640 1 -- 172.21.15.3:0/304383887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564084aee540 con 0x564084a82f60 2024-09-21T06:14:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18b5874640 1 --2- 172.21.15.3:0/304383887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a82f60 0x564084e28f00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f18a8008dd0 tx=0x7f18a800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa7dae7198c279a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18a67fc640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f189c0b65f0 0x7f189c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f189b7fe640 1 -- 172.21.15.3:0/304383887 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a800f830 con 0x564084a82f60 2024-09-21T06:14:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18b5073640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f189c0b65f0 0x7f189c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f18a67fc640 1 -- 172.21.15.3:0/2740487388 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f18a0107430 con 0x7f18b01052a0 2024-09-21T06:14:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f189b7fe640 1 -- 172.21.15.3:0/304383887 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a800de20 con 0x564084a82f60 2024-09-21T06:14:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.588+0000 7f189b7fe640 1 -- 172.21.15.3:0/304383887 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8018e40 con 0x564084a82f60 2024-09-21T06:14:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.589+0000 7f18bd84b740 1 -- 172.21.15.3:0/304383887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a82f60 msgr2=0x564084e28f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.589+0000 7f18b5073640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f189c0b65f0 0x7f189c0b8b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x564084e27b80 tx=0x7f18ac008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.589+0000 7f18bd84b740 1 --2- 172.21.15.3:0/304383887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a82f60 0x564084e28f00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f18a8008dd0 tx=0x7f18a800d710 comp rx=0 tx=0).stop 2024-09-21T06:14:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.589+0000 7f18bd84b740 1 -- 172.21.15.3:0/304383887 shutdown_connections 2024-09-21T06:14:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.589+0000 7f18bd84b740 1 --2- 172.21.15.3:0/304383887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a5a0a0 0x564084e29440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.589+0000 7f18bd84b740 1 --2- 172.21.15.3:0/304383887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a62280 0x564084e29980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.589+0000 7f18bd84b740 1 --2- 172.21.15.3:0/304383887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a82f60 0x564084e28f00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.589+0000 7f18bd84b740 1 -- 172.21.15.3:0/304383887 >> 172.21.15.3:0/304383887 conn(0x564084a77350 msgr2=0x564084e26df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.589+0000 7f18bd84b740 1 -- 172.21.15.3:0/304383887 shutdown_connections 2024-09-21T06:14:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.589+0000 7f18bd84b740 1 -- 172.21.15.3:0/304383887 wait complete. 2024-09-21T06:14:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.590+0000 7f18bd84b740 1 -- start start 2024-09-21T06:14:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.590+0000 7f18bd84b740 1 Processor -- start 2024-09-21T06:14:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.590+0000 7f18bd84b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 0x564084e347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.590+0000 7f18bd84b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a62280 0x564084e34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.590+0000 7f18bd84b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a82f60 0x564084e40a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.590+0000 7f18bd84b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564084bf15f0 con 0x564084a5a0a0 2024-09-21T06:14:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.590+0000 7f18bd84b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564084d2b010 con 0x564084a82f60 2024-09-21T06:14:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.590+0000 7f18bd84b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564084e376e0 con 0x564084a62280 2024-09-21T06:14:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.590+0000 7f18bd84b740 10 client.0 _ll_register_callbacks cb 0x564084d8a340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18b5073640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 0x564084e347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18b5073640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 0x564084e347c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33424/0 (socket says 172.21.15.3:33424) 2024-09-21T06:14:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18b5073640 1 -- 172.21.15.3:0/2544214927 learned_addr learned my addr 172.21.15.3:0/2544214927 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18b4872640 1 --2- 172.21.15.3:0/2544214927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a62280 0x564084e34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18b5874640 1 --2- 172.21.15.3:0/2544214927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a82f60 0x564084e40a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18b5073640 1 -- 172.21.15.3:0/2544214927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a62280 msgr2=0x564084e34da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18b5073640 1 --2- 172.21.15.3:0/2544214927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a62280 0x564084e34da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18b5073640 1 -- 172.21.15.3:0/2544214927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a82f60 msgr2=0x564084e40a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18b5073640 1 --2- 172.21.15.3:0/2544214927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a82f60 0x564084e40a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18b5073640 1 -- 172.21.15.3:0/2544214927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564084d58a00 con 0x564084a5a0a0 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18b5073640 1 --2- 172.21.15.3:0/2544214927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 0x564084e347c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f18ac016a00 tx=0x7f18ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28acfba046ff447 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f1898ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f1898ff9640 1 -- 172.21.15.3:0/2544214927 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac01c050 con 0x564084a5a0a0 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f1898ff9640 1 -- 172.21.15.3:0/2544214927 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f18ac014b40 con 0x564084a5a0a0 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18bd84b740 1 -- 172.21.15.3:0/2544214927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a800de20 con 0x564084a5a0a0 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f1898ff9640 1 -- 172.21.15.3:0/2544214927 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18ac010e70 con 0x564084a5a0a0 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18bd84b740 10 client.15276 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.591+0000 7f18bd84b740 1 -- 172.21.15.3:0/2544214927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564084e436c0 con 0x564084a5a0a0 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.592+0000 7f1898ff9640 1 -- 172.21.15.3:0/2544214927 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a800de20 con 0x564084a5a0a0 2024-09-21T06:14:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.592+0000 7f1898ff9640 1 -- 172.21.15.3:0/2544214927 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f18ac006520 con 0x564084a5a0a0 2024-09-21T06:14:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.592+0000 7f18bd84b740 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.592+0000 7f18bd84b740 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T06:14:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.592+0000 7f18bd84b740 20 client.15276 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.592+0000 7f18bd84b740 1 --2- 172.21.15.3:0/2544214927 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564084a418e0 0x564084e471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.592+0000 7f18bd84b740 4 client.15276 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.592+0000 7f18bd84b740 1 -- 172.21.15.3:0/2544214927 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564084da2b60 con 0x564084a418e0 2024-09-21T06:14:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.592+0000 7f18b4872640 1 --2- 172.21.15.3:0/2544214927 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564084a418e0 0x564084e471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.593+0000 7f18b4872640 1 --2- 172.21.15.3:0/2544214927 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564084a418e0 0x564084e471e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=956e37ea987b8e1b server_cookie=72e9bad3d22c3b66 in_seq=0 out_seq=0 2024-09-21T06:14:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.593+0000 7f1898ff9640 10 client.15276 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:14:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.593+0000 7f1898ff9640 1 -- 172.21.15.3:0/2544214927 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564084da2b60 con 0x564084a418e0 2024-09-21T06:14:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.593+0000 7f1898ff9640 10 client.15276 handle_command_reply: tid=0 2024-09-21T06:14:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.593+0000 7f18bd84b740 1 client.15276 shutdown 2024-09-21T06:14:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.594+0000 7f18bd84b740 1 -- 172.21.15.3:0/2544214927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 msgr2=0x564084e347c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:36.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.594+0000 7f18bd84b740 1 --2- 172.21.15.3:0/2544214927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 0x564084e347c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f18ac016a00 tx=0x7f18ac0046c0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.594+0000 7f18bd84b740 1 -- 172.21.15.3:0/2544214927 shutdown_connections 2024-09-21T06:14:36.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.594+0000 7f18bd84b740 1 --2- 172.21.15.3:0/2544214927 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564084a418e0 0x564084e471e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.594+0000 7f18bd84b740 1 --2- 172.21.15.3:0/2544214927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a62280 0x564084e34da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.594+0000 7f18bd84b740 1 --2- 172.21.15.3:0/2544214927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a82f60 0x564084e40a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.594+0000 7f1898ff9640 0 client.15276 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:14:36.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.594+0000 7f18bd84b740 1 --2- 172.21.15.3:0/2544214927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 0x564084e347c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.594+0000 7f18bd84b740 1 -- 172.21.15.3:0/2544214927 >> 172.21.15.3:0/2544214927 conn(0x564084a77350 msgr2=0x564084e2c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:36.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.594+0000 7f18bd84b740 1 -- 172.21.15.3:0/2544214927 shutdown_connections 2024-09-21T06:14:36.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.594+0000 7f18bd84b740 1 -- 172.21.15.3:0/2544214927 wait complete. 2024-09-21T06:14:36.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.594+0000 7f18bd84b740 20 client.15276 trim_cache size 0 max 16384 2024-09-21T06:14:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.613+0000 7f18bd84b740 1 Processor -- start 2024-09-21T06:14:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.613+0000 7f18bd84b740 1 -- start start 2024-09-21T06:14:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.613+0000 7f18bd84b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a5a0a0 0x564084e409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.613+0000 7f18bd84b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a62280 0x564084e40f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.613+0000 7f18bd84b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a82f60 0x564084e26600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.613+0000 7f18bd84b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564084e45520 con 0x564084a62280 2024-09-21T06:14:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.613+0000 7f18bd84b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564084e45bf0 con 0x564084a5a0a0 2024-09-21T06:14:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.613+0000 7f18bd84b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564084e462c0 con 0x564084a82f60 2024-09-21T06:14:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.613+0000 7f18b4872640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a62280 0x564084e40f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.613+0000 7f18b4872640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a62280 0x564084e40f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33428/0 (socket says 172.21.15.3:33428) 2024-09-21T06:14:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.613+0000 7f18b4872640 1 -- 172.21.15.3:0/821266426 learned_addr learned my addr 172.21.15.3:0/821266426 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.613+0000 7f18b5073640 1 --2- 172.21.15.3:0/821266426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a5a0a0 0x564084e409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.613+0000 7f18b5874640 1 --2- 172.21.15.3:0/821266426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a82f60 0x564084e26600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.614+0000 7f18b4872640 1 -- 172.21.15.3:0/821266426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a82f60 msgr2=0x564084e26600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.614+0000 7f18b4872640 1 --2- 172.21.15.3:0/821266426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a82f60 0x564084e26600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.614+0000 7f18b4872640 1 -- 172.21.15.3:0/821266426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a5a0a0 msgr2=0x564084e409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.614+0000 7f18b4872640 1 --2- 172.21.15.3:0/821266426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a5a0a0 0x564084e409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.614+0000 7f18b4872640 1 -- 172.21.15.3:0/821266426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564084e2bda0 con 0x564084a62280 2024-09-21T06:14:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.614+0000 7f18b4872640 1 --2- 172.21.15.3:0/821266426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a62280 0x564084e40f10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f18a0000c00 tx=0x7f18a0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2087ab98d43ce0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.614+0000 7f189affd640 1 -- 172.21.15.3:0/821266426 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0035040 con 0x564084a62280 2024-09-21T06:14:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.614+0000 7f189affd640 1 -- 172.21.15.3:0/821266426 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a00cce90 con 0x564084a62280 2024-09-21T06:14:36.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.614+0000 7f189affd640 1 -- 172.21.15.3:0/821266426 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a00d6da0 con 0x564084a62280 2024-09-21T06:14:36.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.615+0000 7f18bd84b740 1 -- 172.21.15.3:0/821266426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a62280 msgr2=0x564084e40f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:36.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.615+0000 7f18bd84b740 1 --2- 172.21.15.3:0/821266426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a62280 0x564084e40f10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f18a0000c00 tx=0x7f18a0031400 comp rx=0 tx=0).stop 2024-09-21T06:14:36.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.615+0000 7f18bd84b740 1 -- 172.21.15.3:0/821266426 shutdown_connections 2024-09-21T06:14:36.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.615+0000 7f18bd84b740 1 --2- 172.21.15.3:0/821266426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a82f60 0x564084e26600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.615+0000 7f18bd84b740 1 --2- 172.21.15.3:0/821266426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a62280 0x564084e40f10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.615+0000 7f18bd84b740 1 --2- 172.21.15.3:0/821266426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a5a0a0 0x564084e409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.615+0000 7f18bd84b740 1 -- 172.21.15.3:0/821266426 >> 172.21.15.3:0/821266426 conn(0x564084a77350 msgr2=0x564084e33040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.615+0000 7f18bd84b740 1 -- 172.21.15.3:0/821266426 shutdown_connections 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.615+0000 7f18bd84b740 1 -- 172.21.15.3:0/821266426 wait complete. 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.615+0000 7f18bd84b740 1 -- start start 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18bd84b740 1 Processor -- start 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18bd84b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 0x564084e4fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18bd84b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a62280 0x564084e49150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18bd84b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a82f60 0x564084e496d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18bd84b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564084e47060 con 0x564084a5a0a0 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18bd84b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564084e46990 con 0x564084a82f60 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18bd84b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564084e47730 con 0x564084a62280 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18bd84b740 10 client.0 _ll_register_callbacks cb 0x7f18ac00b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18b5073640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 0x564084e4fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18b5073640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 0x564084e4fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33432/0 (socket says 172.21.15.3:33432) 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18b5073640 1 -- 172.21.15.3:0/3442038826 learned_addr learned my addr 172.21.15.3:0/3442038826 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18b4872640 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a62280 0x564084e49150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18b5874640 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a82f60 0x564084e496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18b5073640 1 -- 172.21.15.3:0/3442038826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a62280 msgr2=0x564084e49150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18b5073640 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a62280 0x564084e49150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18b5073640 1 -- 172.21.15.3:0/3442038826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a82f60 msgr2=0x564084e496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18b5073640 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a82f60 0x564084e496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.616+0000 7f18b5073640 1 -- 172.21.15.3:0/3442038826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564084e49e60 con 0x564084a5a0a0 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.617+0000 7f18b5874640 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a82f60 0x564084e496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.617+0000 7f18b5073640 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 0x564084e4fca0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0214e0 tx=0x7f18ac021810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0ad8383bcade15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.617+0000 7f1898ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.617+0000 7f1898ff9640 1 -- 172.21.15.3:0/3442038826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac03a070 con 0x564084a5a0a0 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.617+0000 7f1898ff9640 1 -- 172.21.15.3:0/3442038826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f18ac021c10 con 0x564084a5a0a0 2024-09-21T06:14:36.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.617+0000 7f18bd84b740 1 -- 172.21.15.3:0/3442038826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a00cce90 con 0x564084a5a0a0 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.617+0000 7f1898ff9640 1 -- 172.21.15.3:0/3442038826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18ac01c040 con 0x564084a5a0a0 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.617+0000 7f18bd84b740 10 client.15282 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.617+0000 7f18bd84b740 1 -- 172.21.15.3:0/3442038826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564084eb26b0 con 0x564084a5a0a0 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.617+0000 7f1898ff9640 1 -- 172.21.15.3:0/3442038826 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a00cce90 con 0x564084a5a0a0 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.618+0000 7f1898ff9640 1 -- 172.21.15.3:0/3442038826 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f18ac034430 con 0x564084a5a0a0 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.618+0000 7f18bd84b740 10 client.15282 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.618+0000 7f18bd84b740 10 client.15282 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.618+0000 7f18bd84b740 20 client.15282 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.618+0000 7f18bd84b740 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564084a16c90 0x564084eb5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.618+0000 7f18bd84b740 4 client.15282 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.618+0000 7f18bd84b740 1 -- 172.21.15.3:0/3442038826 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "status"}) -- 0x564084d9e610 con 0x564084a16c90 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.618+0000 7f18b4872640 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564084a16c90 0x564084eb5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.618+0000 7f18b4872640 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564084a16c90 0x564084eb5f80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=339a3810d545a025 server_cookie=7d12f49cdaa916a in_seq=0 out_seq=0 2024-09-21T06:14:36.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.618+0000 7f1898ff9640 10 client.15282 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f1898ff9640 1 -- 172.21.15.3:0/3442038826 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564084d9e610 con 0x564084a16c90 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f1898ff9640 10 client.15282 handle_command_reply: tid=0 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f18bd84b740 1 client.15282 shutdown 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f18bd84b740 1 -- 172.21.15.3:0/3442038826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 msgr2=0x564084e4fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f18bd84b740 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 0x564084e4fca0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0214e0 tx=0x7f18ac021810 comp rx=0 tx=0).stop 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f18bd84b740 1 -- 172.21.15.3:0/3442038826 shutdown_connections 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f18bd84b740 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564084a16c90 0x564084eb5f80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f18bd84b740 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564084a62280 0x564084e49150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f18bd84b740 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564084a82f60 0x564084e496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f1898ff9640 0 client.15282 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f18bd84b740 1 --2- 172.21.15.3:0/3442038826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564084a5a0a0 0x564084e4fca0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f18bd84b740 1 -- 172.21.15.3:0/3442038826 >> 172.21.15.3:0/3442038826 conn(0x564084a77350 msgr2=0x564084e26600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f18bd84b740 1 -- 172.21.15.3:0/3442038826 shutdown_connections 2024-09-21T06:14:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f18bd84b740 1 -- 172.21.15.3:0/3442038826 wait complete. 2024-09-21T06:14:36.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.619+0000 7f18bd84b740 20 client.15282 trim_cache size 0 max 16384 2024-09-21T06:14:36.626 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:14:36.626 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "38d3c75c-77df-11ef-baf5-efdc52797490", 2024-09-21T06:14:36.626 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-09-21T06:14:36.626 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24691, 2024-09-21T06:14:36.626 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-21T06:14:36.626 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-21T06:14:36.626 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-21T06:14:36.626 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 77.069957008000003, 2024-09-21T06:14:36.626 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2024-09-21T06:14:36.627 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 79, 2024-09-21T06:14:36.627 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T06:14:36.627 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 105.54128241399999 2024-09-21T06:14:36.627 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:14:36.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.620+0000 7f18b72d7640 1 -- 172.21.15.3:0/2740487388 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f189c0b65f0 msgr2=0x7f189c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:36.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.620+0000 7f18b72d7640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f189c0b65f0 0x7f189c0b8b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x564084e27b80 tx=0x7f18ac008040 comp rx=0 tx=0).stop 2024-09-21T06:14:36.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.620+0000 7f18b72d7640 1 -- 172.21.15.3:0/2740487388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b01052a0 msgr2=0x7f18b019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.620+0000 7f18b72d7640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b01052a0 0x7f18b019ab60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f18a002a930 tx=0x7f18a0004270 comp rx=0 tx=0).stop 2024-09-21T06:14:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.620+0000 7f18b72d7640 1 -- 172.21.15.3:0/2740487388 shutdown_connections 2024-09-21T06:14:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.620+0000 7f18b72d7640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f189c0b65f0 0x7f189c0b8b00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.620+0000 7f18b72d7640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b010de70 0x7f18b019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.620+0000 7f18b72d7640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b01052a0 0x7f18b019ab60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.620+0000 7f18b72d7640 1 --2- 172.21.15.3:0/2740487388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18b01048c0 0x7f18b019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.620+0000 7f18b72d7640 1 -- 172.21.15.3:0/2740487388 >> 172.21.15.3:0/2740487388 conn(0x7f18b0079760 msgr2=0x7f18b0100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.620+0000 7f18b72d7640 1 -- 172.21.15.3:0/2740487388 shutdown_connections 2024-09-21T06:14:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:36.620+0000 7f18b72d7640 1 -- 172.21.15.3:0/2740487388 wait complete. 2024-09-21T06:14:37.326 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T06:14:37.582 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:37.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:37 smithi003 ceph-mon[31227]: pgmap v313: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:37.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:37 smithi137 ceph-mon[34464]: pgmap v313: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:37.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:37 smithi204 ceph-mon[34659]: pgmap v313: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.065+0000 7f82fb567640 1 -- 172.21.15.3:0/2250631753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f4102290 msgr2=0x7f82f41026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.065+0000 7f82fb567640 1 --2- 172.21.15.3:0/2250631753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f4102290 0x7f82f41026a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009900 tx=0x7f82f002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.065+0000 7f82fb567640 1 -- 172.21.15.3:0/2250631753 shutdown_connections 2024-09-21T06:14:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.065+0000 7f82fb567640 1 --2- 172.21.15.3:0/2250631753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4107bd0 0x7f82f4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.065+0000 7f82fb567640 1 --2- 172.21.15.3:0/2250631753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82f4108520 0x7f82f410ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.065+0000 7f82fb567640 1 --2- 172.21.15.3:0/2250631753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f4102290 0x7f82f41026a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.065+0000 7f82fb567640 1 -- 172.21.15.3:0/2250631753 >> 172.21.15.3:0/2250631753 conn(0x7f82f4079ea0 msgr2=0x7f82f407a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.065+0000 7f82fb567640 1 -- 172.21.15.3:0/2250631753 shutdown_connections 2024-09-21T06:14:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.065+0000 7f82fb567640 1 -- 172.21.15.3:0/2250631753 wait complete. 2024-09-21T06:14:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.066+0000 7f82fb567640 1 Processor -- start 2024-09-21T06:14:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.066+0000 7f82fb567640 1 -- start start 2024-09-21T06:14:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.066+0000 7f82fb567640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82f4102290 0x7f82f41993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.066+0000 7f82fb567640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4107bd0 0x7f82f4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.066+0000 7f82fb567640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f4108520 0x7f82f419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.066+0000 7f82fb567640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4112a60 con 0x7f82f4108520 2024-09-21T06:14:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.066+0000 7f82fb567640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41128e0 con 0x7f82f4102290 2024-09-21T06:14:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.066+0000 7f82fb567640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4112be0 con 0x7f82f4107bd0 2024-09-21T06:14:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.066+0000 7f82f9b04640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f4108520 0x7f82f419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.066+0000 7f82f9b04640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f4108520 0x7f82f419ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33460/0 (socket says 172.21.15.3:33460) 2024-09-21T06:14:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82f9b04640 1 -- 172.21.15.3:0/2470132808 learned_addr learned my addr 172.21.15.3:0/2470132808 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82f9303640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82f4102290 0x7f82f41993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82f8b02640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4107bd0 0x7f82f4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82f9b04640 1 -- 172.21.15.3:0/2470132808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4107bd0 msgr2=0x7f82f4199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82f9b04640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4107bd0 0x7f82f4199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82f9b04640 1 -- 172.21.15.3:0/2470132808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82f4102290 msgr2=0x7f82f41993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82f9b04640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82f4102290 0x7f82f41993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82f9b04640 1 -- 172.21.15.3:0/2470132808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f419f2a0 con 0x7f82f4108520 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82f9b04640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f4108520 0x7f82f419ec30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00cce0 tx=0x7f82ec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82ea7fc640 1 -- 172.21.15.3:0/2470132808 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec00f450 con 0x7f82f4108520 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82ea7fc640 1 -- 172.21.15.3:0/2470132808 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82ec00faa0 con 0x7f82f4108520 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82fb567640 1 -- 172.21.15.3:0/2470132808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f419f590 con 0x7f82f4108520 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82ea7fc640 1 -- 172.21.15.3:0/2470132808 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec018b70 con 0x7f82f4108520 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.067+0000 7f82fb567640 1 -- 172.21.15.3:0/2470132808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f41a5db0 con 0x7f82f4108520 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.070+0000 7f8301adb740 1 Processor -- start 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.070+0000 7f82ea7fc640 1 -- 172.21.15.3:0/2470132808 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f82ec018d10 con 0x7f82f4108520 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.070+0000 7f8301adb740 1 -- start start 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.070+0000 7f8301adb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3768df60 0x555a37a369e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.070+0000 7f8301adb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a376650a0 0x555a37a33f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.070+0000 7f8301adb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.070+0000 7f8301adb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555a379abc10 con 0x555a3766d280 2024-09-21T06:14:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.070+0000 7f8301adb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555a3799b040 con 0x555a3768df60 2024-09-21T06:14:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.070+0000 7f8301adb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555a3799ee50 con 0x555a376650a0 2024-09-21T06:14:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82f8b02640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82f8b02640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3768df60 0x555a37a369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82f9303640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a376650a0 0x555a37a33f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82f8b02640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3768df60 0x555a37a369e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:53644/0 (socket says 172.21.15.3:53644) 2024-09-21T06:14:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82f8b02640 1 -- 172.21.15.3:0/2808292702 learned_addr learned my addr 172.21.15.3:0/2808292702 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82f8b02640 1 -- 172.21.15.3:0/2808292702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a376650a0 msgr2=0x555a37a33f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82f8b02640 1 --2- 172.21.15.3:0/2808292702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a376650a0 0x555a37a33f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82ea7fc640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82e00b66a0 0x7f82e00b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82f8b02640 1 -- 172.21.15.3:0/2808292702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 msgr2=0x555a37a345b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82f8b02640 1 --2- 172.21.15.3:0/2808292702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a345b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82f8b02640 1 -- 172.21.15.3:0/2808292702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a376f9540 con 0x555a3768df60 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82f9b04640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82e00b66a0 0x7f82e00b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82f8b02640 1 --2- 172.21.15.3:0/2808292702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3768df60 0x555a37a369e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82e4013980 tx=0x7f82e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc6d2ba074e52013 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82ea7fc640 1 -- 172.21.15.3:0/2470132808 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f82ec0e79c0 con 0x7f82f4108520 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.071+0000 7f82df7fe640 1 -- 172.21.15.3:0/2808292702 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e40109e0 con 0x555a3768df60 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.072+0000 7f82df7fe640 1 -- 172.21.15.3:0/2808292702 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e4011440 con 0x555a3768df60 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.072+0000 7f82df7fe640 1 -- 172.21.15.3:0/2808292702 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e400e5f0 con 0x555a3768df60 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.072+0000 7f8301adb740 1 -- 172.21.15.3:0/2808292702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3768df60 msgr2=0x555a37a369e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.072+0000 7f82f9b04640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82e00b66a0 0x7f82e00b8bb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f82ec01b000 tx=0x7f82ec016f60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.072+0000 7f8301adb740 1 --2- 172.21.15.3:0/2808292702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3768df60 0x555a37a369e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82e4013980 tx=0x7f82e40046c0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.072+0000 7f8301adb740 1 -- 172.21.15.3:0/2808292702 shutdown_connections 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.072+0000 7f8301adb740 1 --2- 172.21.15.3:0/2808292702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a376650a0 0x555a37a33f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.072+0000 7f8301adb740 1 --2- 172.21.15.3:0/2808292702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a345b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.072+0000 7f8301adb740 1 --2- 172.21.15.3:0/2808292702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3768df60 0x555a37a369e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.072+0000 7f8301adb740 1 -- 172.21.15.3:0/2808292702 >> 172.21.15.3:0/2808292702 conn(0x555a37682350 msgr2=0x555a37a31df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.072+0000 7f8301adb740 1 -- 172.21.15.3:0/2808292702 shutdown_connections 2024-09-21T06:14:38.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.072+0000 7f8301adb740 1 -- 172.21.15.3:0/2808292702 wait complete. 2024-09-21T06:14:38.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.073+0000 7f8301adb740 1 -- start start 2024-09-21T06:14:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.073+0000 7f8301adb740 1 Processor -- start 2024-09-21T06:14:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f8301adb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a376650a0 0x555a37a43d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f8301adb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a44340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f8301adb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3768df60 0x555a37a4fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f8301adb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555a377fc5f0 con 0x555a3766d280 2024-09-21T06:14:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f8301adb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555a37936010 con 0x555a3768df60 2024-09-21T06:14:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f8301adb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555a37a46a60 con 0x555a376650a0 2024-09-21T06:14:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f8301adb740 10 client.0 _ll_register_callbacks cb 0x555a37995340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f82f9303640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a44340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f82f9303640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a44340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33476/0 (socket says 172.21.15.3:33476) 2024-09-21T06:14:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f82f9303640 1 -- 172.21.15.3:0/2865163938 learned_addr learned my addr 172.21.15.3:0/2865163938 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f82f8b02640 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3768df60 0x555a37a4fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f82f8b02640 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a376650a0 0x555a37a43d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f82f9303640 1 -- 172.21.15.3:0/2865163938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a376650a0 msgr2=0x555a37a43d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f82f9303640 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a376650a0 0x555a37a43d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f82f9303640 1 -- 172.21.15.3:0/2865163938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3768df60 msgr2=0x555a37a4fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f82f9303640 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3768df60 0x555a37a4fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f82f9303640 1 -- 172.21.15.3:0/2865163938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a37963a00 con 0x555a3766d280 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.074+0000 7f82f8b02640 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a376650a0 0x555a37a43d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.075+0000 7f82f9303640 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a44340 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f82f002a910 tx=0x7f82f00316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8467b1ccd5f75df8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.075+0000 7f82dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.075+0000 7f82dcff9640 1 -- 172.21.15.3:0/2865163938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0035040 con 0x555a3766d280 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.075+0000 7f82dcff9640 1 -- 172.21.15.3:0/2865163938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f82f0004040 con 0x555a3766d280 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.075+0000 7f8301adb740 1 -- 172.21.15.3:0/2865163938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e4011440 con 0x555a3766d280 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.075+0000 7f82dcff9640 1 -- 172.21.15.3:0/2865163938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82f0004240 con 0x555a3766d280 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.075+0000 7f8301adb740 10 client.15294 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.075+0000 7f8301adb740 1 -- 172.21.15.3:0/2865163938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a37a52960 con 0x555a3766d280 2024-09-21T06:14:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.075+0000 7f82dcff9640 1 -- 172.21.15.3:0/2865163938 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e4011440 con 0x555a3766d280 2024-09-21T06:14:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.076+0000 7f82dcff9640 1 -- 172.21.15.3:0/2865163938 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f82f00465a0 con 0x555a3766d280 2024-09-21T06:14:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.076+0000 7f8301adb740 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.076+0000 7f8301adb740 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:14:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.076+0000 7f8301adb740 20 client.15294 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.076+0000 7f8301adb740 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555a3764c8e0 0x555a37a563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.076+0000 7f8301adb740 4 client.15294 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.076+0000 7f8301adb740 1 -- 172.21.15.3:0/2865163938 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a379adb60 con 0x555a3764c8e0 2024-09-21T06:14:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.076+0000 7f82f8b02640 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555a3764c8e0 0x555a37a563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.076+0000 7f82f8b02640 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555a3764c8e0 0x555a37a563a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7918dc5c39f4ccb3 server_cookie=3f480585a5a5eef2 in_seq=0 out_seq=0 2024-09-21T06:14:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.076+0000 7f82dcff9640 10 client.15294 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:14:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.077+0000 7f82dcff9640 1 -- 172.21.15.3:0/2865163938 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a379adb60 con 0x555a3764c8e0 2024-09-21T06:14:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.077+0000 7f82dcff9640 10 client.15294 handle_command_reply: tid=0 2024-09-21T06:14:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.077+0000 7f8301adb740 1 client.15294 shutdown 2024-09-21T06:14:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.078+0000 7f8301adb740 1 -- 172.21.15.3:0/2865163938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 msgr2=0x555a37a44340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.078+0000 7f8301adb740 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a44340 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f82f002a910 tx=0x7f82f00316d0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.078+0000 7f8301adb740 1 -- 172.21.15.3:0/2865163938 shutdown_connections 2024-09-21T06:14:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.078+0000 7f8301adb740 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555a3764c8e0 0x555a37a563a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.078+0000 7f82dcff9640 0 client.15294 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:14:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.078+0000 7f8301adb740 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3768df60 0x555a37a4fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.078+0000 7f8301adb740 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a44340 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.078+0000 7f8301adb740 1 --2- 172.21.15.3:0/2865163938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a376650a0 0x555a37a43d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.078+0000 7f8301adb740 1 -- 172.21.15.3:0/2865163938 >> 172.21.15.3:0/2865163938 conn(0x555a37682350 msgr2=0x555a37a303b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.078+0000 7f8301adb740 1 -- 172.21.15.3:0/2865163938 shutdown_connections 2024-09-21T06:14:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.078+0000 7f8301adb740 1 -- 172.21.15.3:0/2865163938 wait complete. 2024-09-21T06:14:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.078+0000 7f8301adb740 20 client.15294 trim_cache size 0 max 16384 2024-09-21T06:14:38.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.096+0000 7f8301adb740 1 Processor -- start 2024-09-21T06:14:38.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.096+0000 7f8301adb740 1 -- start start 2024-09-21T06:14:38.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f8301adb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3764c8e0 0x555a37a41860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f8301adb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a376650a0 0x555a37a4fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f8301adb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a3766d280 0x555a37a50240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f8301adb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555a37a546e0 con 0x555a376650a0 2024-09-21T06:14:38.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f8301adb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555a37a54db0 con 0x555a3764c8e0 2024-09-21T06:14:38.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f8301adb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555a37a55480 con 0x555a3766d280 2024-09-21T06:14:38.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82f9303640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a376650a0 0x555a37a4fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82f9303640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a376650a0 0x555a37a4fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33488/0 (socket says 172.21.15.3:33488) 2024-09-21T06:14:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82f9303640 1 -- 172.21.15.3:0/4156681592 learned_addr learned my addr 172.21.15.3:0/4156681592 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82f8b02640 1 --2- 172.21.15.3:0/4156681592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3764c8e0 0x555a37a41860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82f9303640 1 -- 172.21.15.3:0/4156681592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a3766d280 msgr2=0x555a37a50240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82f9303640 1 --2- 172.21.15.3:0/4156681592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a3766d280 0x555a37a50240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82f9303640 1 -- 172.21.15.3:0/4156681592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3764c8e0 msgr2=0x555a37a41860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82f9303640 1 --2- 172.21.15.3:0/4156681592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3764c8e0 0x555a37a41860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82f9303640 1 -- 172.21.15.3:0/4156681592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a37a45020 con 0x555a376650a0 2024-09-21T06:14:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82f8b02640 1 --2- 172.21.15.3:0/4156681592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3764c8e0 0x555a37a41860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82f9303640 1 --2- 172.21.15.3:0/4156681592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a376650a0 0x555a37a4fd00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009340 tx=0x7f82f0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64bd08b1da4c512 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82deffd640 1 -- 172.21.15.3:0/4156681592 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0000e60 con 0x555a376650a0 2024-09-21T06:14:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82deffd640 1 -- 172.21.15.3:0/4156681592 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82f0046b90 con 0x555a376650a0 2024-09-21T06:14:38.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.097+0000 7f82deffd640 1 -- 172.21.15.3:0/4156681592 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0035030 con 0x555a376650a0 2024-09-21T06:14:38.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.098+0000 7f8301adb740 1 -- 172.21.15.3:0/4156681592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a376650a0 msgr2=0x555a37a4fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:38.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.098+0000 7f8301adb740 1 --2- 172.21.15.3:0/4156681592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a376650a0 0x555a37a4fd00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009340 tx=0x7f82f0031400 comp rx=0 tx=0).stop 2024-09-21T06:14:38.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.098+0000 7f8301adb740 1 -- 172.21.15.3:0/4156681592 shutdown_connections 2024-09-21T06:14:38.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.098+0000 7f8301adb740 1 --2- 172.21.15.3:0/4156681592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a3766d280 0x555a37a50240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.098+0000 7f8301adb740 1 --2- 172.21.15.3:0/4156681592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a376650a0 0x555a37a4fd00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.098+0000 7f8301adb740 1 --2- 172.21.15.3:0/4156681592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a3764c8e0 0x555a37a41860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.098+0000 7f8301adb740 1 -- 172.21.15.3:0/4156681592 >> 172.21.15.3:0/4156681592 conn(0x555a37682350 msgr2=0x555a37a2cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:38.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.098+0000 7f8301adb740 1 -- 172.21.15.3:0/4156681592 shutdown_connections 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.098+0000 7f8301adb740 1 -- 172.21.15.3:0/4156681592 wait complete. 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.099+0000 7f8301adb740 1 -- start start 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.099+0000 7f8301adb740 1 Processor -- start 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.099+0000 7f8301adb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a3764c8e0 0x555a37a311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.099+0000 7f8301adb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a376650a0 0x555a37a31700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.099+0000 7f8301adb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a32d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.099+0000 7f8301adb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555a37a56220 con 0x555a3766d280 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.099+0000 7f8301adb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555a37a55b50 con 0x555a376650a0 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.099+0000 7f8301adb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555a37a568f0 con 0x555a3764c8e0 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.099+0000 7f8301adb740 10 client.0 _ll_register_callbacks cb 0x7f82f0031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.099+0000 7f82f8b02640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a32d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.099+0000 7f82f8b02640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a32d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33502/0 (socket says 172.21.15.3:33502) 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.099+0000 7f82f8b02640 1 -- 172.21.15.3:0/3211859216 learned_addr learned my addr 172.21.15.3:0/3211859216 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f82f9303640 1 --2- 172.21.15.3:0/3211859216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a376650a0 0x555a37a31700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f82f8b02640 1 --2- 172.21.15.3:0/3211859216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a3764c8e0 0x555a37a311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f82f8b02640 1 -- 172.21.15.3:0/3211859216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a3764c8e0 msgr2=0x555a37a311c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f82f8b02640 1 --2- 172.21.15.3:0/3211859216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a3764c8e0 0x555a37a311c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f82f8b02640 1 -- 172.21.15.3:0/3211859216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a376650a0 msgr2=0x555a37a31700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f82f8b02640 1 --2- 172.21.15.3:0/3211859216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a376650a0 0x555a37a31700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f82f8b02640 1 -- 172.21.15.3:0/3211859216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a37a31d70 con 0x555a3766d280 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f82f8b02640 1 --2- 172.21.15.3:0/3211859216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a32d40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x555a37a34580 tx=0x7f82e4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd2fc8b4a3c2af6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f82dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f82dcff9640 1 -- 172.21.15.3:0/3211859216 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4013190 con 0x555a3766d280 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f82dcff9640 1 -- 172.21.15.3:0/3211859216 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f82e402df90 con 0x555a3766d280 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f8301adb740 1 -- 172.21.15.3:0/3211859216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82f0046b90 con 0x555a3766d280 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f82dcff9640 1 -- 172.21.15.3:0/3211859216 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82e403a050 con 0x555a3766d280 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f8301adb740 10 client.15300 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.100+0000 7f8301adb740 1 -- 172.21.15.3:0/3211859216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a37a5da90 con 0x555a3766d280 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.101+0000 7f82dcff9640 1 -- 172.21.15.3:0/3211859216 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82f0046b90 con 0x555a3766d280 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.101+0000 7f82dcff9640 1 -- 172.21.15.3:0/3211859216 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f82e4011440 con 0x555a3766d280 2024-09-21T06:14:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.101+0000 7f8301adb740 10 client.15300 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.101+0000 7f8301adb740 10 client.15300 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.101+0000 7f8301adb740 20 client.15300 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.101+0000 7f8301adb740 1 --2- 172.21.15.3:0/3211859216 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555a3768df60 0x555a37ac53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.101+0000 7f8301adb740 4 client.15300 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.101+0000 7f8301adb740 1 -- 172.21.15.3:0/3211859216 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "status"}) -- 0x555a379a9610 con 0x555a3768df60 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.101+0000 7f82f9303640 1 --2- 172.21.15.3:0/3211859216 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555a3768df60 0x555a37ac53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.101+0000 7f82f9303640 1 --2- 172.21.15.3:0/3211859216 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555a3768df60 0x555a37ac53c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0fbc21add1c5d7a server_cookie=ab1426c443ac3d54 in_seq=0 out_seq=0 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.101+0000 7f82dcff9640 10 client.15300 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f82dcff9640 1 -- 172.21.15.3:0/3211859216 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555a379a9610 con 0x555a3768df60 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f82dcff9640 10 client.15300 handle_command_reply: tid=0 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f8301adb740 1 client.15300 shutdown 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f8301adb740 1 -- 172.21.15.3:0/3211859216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 msgr2=0x555a37a32d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f8301adb740 1 --2- 172.21.15.3:0/3211859216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a32d40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x555a37a34580 tx=0x7f82e4015040 comp rx=0 tx=0).stop 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f8301adb740 1 -- 172.21.15.3:0/3211859216 shutdown_connections 2024-09-21T06:14:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f8301adb740 1 --2- 172.21.15.3:0/3211859216 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555a3768df60 0x555a37ac53c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f8301adb740 1 --2- 172.21.15.3:0/3211859216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555a3766d280 0x555a37a32d40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f8301adb740 1 --2- 172.21.15.3:0/3211859216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a376650a0 0x555a37a31700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f82dcff9640 0 client.15300 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:14:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f8301adb740 1 --2- 172.21.15.3:0/3211859216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a3764c8e0 0x555a37a311c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f8301adb740 1 -- 172.21.15.3:0/3211859216 >> 172.21.15.3:0/3211859216 conn(0x555a37682350 msgr2=0x555a37a3a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f8301adb740 1 -- 172.21.15.3:0/3211859216 shutdown_connections 2024-09-21T06:14:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f8301adb740 1 -- 172.21.15.3:0/3211859216 wait complete. 2024-09-21T06:14:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.102+0000 7f8301adb740 20 client.15300 trim_cache size 0 max 16384 2024-09-21T06:14:38.110 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:14:38.110 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "38d3c75c-77df-11ef-baf5-efdc52797490", 2024-09-21T06:14:38.110 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-21T06:14:38.110 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24704, 2024-09-21T06:14:38.111 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-21T06:14:38.111 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-21T06:14:38.111 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-21T06:14:38.111 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 86.612975176999996, 2024-09-21T06:14:38.111 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2024-09-21T06:14:38.111 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 79, 2024-09-21T06:14:38.111 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T06:14:38.111 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 97.200461058000002 2024-09-21T06:14:38.111 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:14:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.103+0000 7f82fb567640 1 -- 172.21.15.3:0/2470132808 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82e00b66a0 msgr2=0x7f82e00b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.103+0000 7f82fb567640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82e00b66a0 0x7f82e00b8bb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f82ec01b000 tx=0x7f82ec016f60 comp rx=0 tx=0).stop 2024-09-21T06:14:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.103+0000 7f82fb567640 1 -- 172.21.15.3:0/2470132808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f4108520 msgr2=0x7f82f419ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.103+0000 7f82fb567640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f4108520 0x7f82f419ec30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00cce0 tx=0x7f82ec00a620 comp rx=0 tx=0).stop 2024-09-21T06:14:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.103+0000 7f82fb567640 1 -- 172.21.15.3:0/2470132808 shutdown_connections 2024-09-21T06:14:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.103+0000 7f82fb567640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82e00b66a0 0x7f82e00b8bb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.103+0000 7f82fb567640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f4107bd0 0x7f82f4199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.103+0000 7f82fb567640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f4108520 0x7f82f419ec30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.103+0000 7f82fb567640 1 --2- 172.21.15.3:0/2470132808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82f4102290 0x7f82f41993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.103+0000 7f82fb567640 1 -- 172.21.15.3:0/2470132808 >> 172.21.15.3:0/2470132808 conn(0x7f82f4079ea0 msgr2=0x7f82f4105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.103+0000 7f82fb567640 1 -- 172.21.15.3:0/2470132808 shutdown_connections 2024-09-21T06:14:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:38.103+0000 7f82fb567640 1 -- 172.21.15.3:0/2470132808 wait complete. 2024-09-21T06:14:38.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:38 smithi137 ceph-mon[34464]: pgmap v314: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:38.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:38 smithi003 ceph-mon[31227]: pgmap v314: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:38.805 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:38 smithi204 ceph-mon[34659]: pgmap v314: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:38.827 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T06:14:38.827 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T06:14:38.839 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-09-21T06:14:38.840 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T06:14:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.549+0000 7f5416432640 1 -- 172.21.15.3:0/1016958267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5410102a60 msgr2=0x7f5410102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.549+0000 7f5416432640 1 --2- 172.21.15.3:0/1016958267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5410102a60 0x7f5410102e70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5400009900 tx=0x7f540002a410 comp rx=0 tx=0).stop 2024-09-21T06:14:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.550+0000 7f5416432640 1 -- 172.21.15.3:0/1016958267 shutdown_connections 2024-09-21T06:14:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.550+0000 7f5416432640 1 --2- 172.21.15.3:0/1016958267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5410108cf0 0x7f541010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.550+0000 7f5416432640 1 --2- 172.21.15.3:0/1016958267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54101083a0 0x7f54101087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.550+0000 7f5416432640 1 --2- 172.21.15.3:0/1016958267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5410102a60 0x7f5410102e70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.550+0000 7f5416432640 1 -- 172.21.15.3:0/1016958267 >> 172.21.15.3:0/1016958267 conn(0x7f5410079730 msgr2=0x7f5410079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.550+0000 7f5416432640 1 -- 172.21.15.3:0/1016958267 shutdown_connections 2024-09-21T06:14:39.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.550+0000 7f5416432640 1 -- 172.21.15.3:0/1016958267 wait complete. 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.550+0000 7f5416432640 1 Processor -- start 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.550+0000 7f5416432640 1 -- start start 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f5416432640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5410102a60 0x7f541019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f5416432640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54101083a0 0x7f541019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f5416432640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5410108cf0 0x7f54101a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f5416432640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5410113a80 con 0x7f5410108cf0 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f5416432640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5410113900 con 0x7f5410102a60 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f5416432640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5410113c00 con 0x7f54101083a0 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f54149cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5410108cf0 0x7f54101a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f54149cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5410108cf0 0x7f54101a1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33508/0 (socket says 172.21.15.3:33508) 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f54149cf640 1 -- 172.21.15.3:0/2566220899 learned_addr learned my addr 172.21.15.3:0/2566220899 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f540ffff640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5410102a60 0x7f541019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f540f7fe640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54101083a0 0x7f541019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f54149cf640 1 -- 172.21.15.3:0/2566220899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54101083a0 msgr2=0x7f541019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:39.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f54149cf640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54101083a0 0x7f541019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f54149cf640 1 -- 172.21.15.3:0/2566220899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5410102a60 msgr2=0x7f541019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f54149cf640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5410102a60 0x7f541019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f54149cf640 1 -- 172.21.15.3:0/2566220899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54101a2560 con 0x7f5410108cf0 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.551+0000 7f540ffff640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5410102a60 0x7f541019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.552+0000 7f54149cf640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5410108cf0 0x7f54101a1ef0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f540800cce0 tx=0x7f540800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.552+0000 7f540d7fa640 1 -- 172.21.15.3:0/2566220899 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540800f450 con 0x7f5410108cf0 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.552+0000 7f540d7fa640 1 -- 172.21.15.3:0/2566220899 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f540800faa0 con 0x7f5410108cf0 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.552+0000 7f5416432640 1 -- 172.21.15.3:0/2566220899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54101a2850 con 0x7f5410108cf0 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.552+0000 7f540d7fa640 1 -- 172.21.15.3:0/2566220899 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408018b70 con 0x7f5410108cf0 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.552+0000 7f5416432640 1 -- 172.21.15.3:0/2566220899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54101a9070 con 0x7f5410108cf0 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.553+0000 7f5416432640 1 -- 172.21.15.3:0/2566220899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5410103ce0 con 0x7f5410108cf0 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.555+0000 7f540d7fa640 1 -- 172.21.15.3:0/2566220899 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f5408018d10 con 0x7f5410108cf0 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.556+0000 7f540d7fa640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f53f80b6620 0x7f53f80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.556+0000 7f540ffff640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f53f80b6620 0x7f53f80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:39.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.556+0000 7f540d7fa640 1 -- 172.21.15.3:0/2566220899 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f54080e7b90 con 0x7f5410108cf0 2024-09-21T06:14:39.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.557+0000 7f540ffff640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f53f80b6620 0x7f53f80b8b30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5400031440 tx=0x7f5400009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:39.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.560+0000 7f540d7fa640 1 -- 172.21.15.3:0/2566220899 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5408014030 con 0x7f5410108cf0 2024-09-21T06:14:39.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.768+0000 7f5416432640 1 -- 172.21.15.3:0/2566220899 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f541010ef00 con 0x7f53f80b6620 2024-09-21T06:14:39.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.769+0000 7f540d7fa640 1 -- 172.21.15.3:0/2566220899 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f541010ef00 con 0x7f53f80b6620 2024-09-21T06:14:39.771 INFO:teuthology.orchestra.run.smithi003.stdout:Backend: cephadm 2024-09-21T06:14:39.771 INFO:teuthology.orchestra.run.smithi003.stdout:Available: Yes 2024-09-21T06:14:39.771 INFO:teuthology.orchestra.run.smithi003.stdout:Paused: No 2024-09-21T06:14:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.771+0000 7f5416432640 1 -- 172.21.15.3:0/2566220899 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f53f80b6620 msgr2=0x7f53f80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.771+0000 7f5416432640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f53f80b6620 0x7f53f80b8b30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5400031440 tx=0x7f5400009680 comp rx=0 tx=0).stop 2024-09-21T06:14:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.771+0000 7f5416432640 1 -- 172.21.15.3:0/2566220899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5410108cf0 msgr2=0x7f54101a1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.771+0000 7f5416432640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5410108cf0 0x7f54101a1ef0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f540800cce0 tx=0x7f540800a620 comp rx=0 tx=0).stop 2024-09-21T06:14:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.771+0000 7f5416432640 1 -- 172.21.15.3:0/2566220899 shutdown_connections 2024-09-21T06:14:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.771+0000 7f5416432640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f53f80b6620 0x7f53f80b8b30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.771+0000 7f5416432640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54101083a0 0x7f541019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.771+0000 7f5416432640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5410108cf0 0x7f54101a1ef0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.771+0000 7f5416432640 1 --2- 172.21.15.3:0/2566220899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5410102a60 0x7f541019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.771+0000 7f5416432640 1 -- 172.21.15.3:0/2566220899 >> 172.21.15.3:0/2566220899 conn(0x7f5410079730 msgr2=0x7f541010e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.771+0000 7f5416432640 1 -- 172.21.15.3:0/2566220899 shutdown_connections 2024-09-21T06:14:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:39.771+0000 7f5416432640 1 -- 172.21.15.3:0/2566220899 wait complete. 2024-09-21T06:14:40.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T06:14:40.803 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:41.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.301+0000 7fed4278b640 1 -- 172.21.15.3:0/1094853151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c102a60 msgr2=0x7fed3c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.301+0000 7fed4278b640 1 --2- 172.21.15.3:0/1094853151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c102a60 0x7fed3c102e70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009900 tx=0x7fed2c02a410 comp rx=0 tx=0).stop 2024-09-21T06:14:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.301+0000 7fed4278b640 1 -- 172.21.15.3:0/1094853151 shutdown_connections 2024-09-21T06:14:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.301+0000 7fed4278b640 1 --2- 172.21.15.3:0/1094853151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed3c1083a0 0x7fed3c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.301+0000 7fed4278b640 1 --2- 172.21.15.3:0/1094853151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed3c108cf0 0x7fed3c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.301+0000 7fed4278b640 1 --2- 172.21.15.3:0/1094853151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c102a60 0x7fed3c102e70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.301+0000 7fed4278b640 1 -- 172.21.15.3:0/1094853151 >> 172.21.15.3:0/1094853151 conn(0x7fed3c079730 msgr2=0x7fed3c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.301+0000 7fed4278b640 1 -- 172.21.15.3:0/1094853151 shutdown_connections 2024-09-21T06:14:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.301+0000 7fed4278b640 1 -- 172.21.15.3:0/1094853151 wait complete. 2024-09-21T06:14:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.302+0000 7fed4278b640 1 Processor -- start 2024-09-21T06:14:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.302+0000 7fed4278b640 1 -- start start 2024-09-21T06:14:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.302+0000 7fed4278b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c102a60 0x7fed3c199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.302+0000 7fed4278b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed3c1083a0 0x7fed3c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.302+0000 7fed4278b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed3c108cf0 0x7fed3c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.302+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c102a60 0x7fed3c199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.302+0000 7fed4278b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c112960 con 0x7fed3c102a60 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.302+0000 7fed4278b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1127e0 con 0x7fed3c108cf0 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.302+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c102a60 0x7fed3c199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33518/0 (socket says 172.21.15.3:33518) 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.302+0000 7fed3bfff640 1 -- 172.21.15.3:0/872719577 learned_addr learned my addr 172.21.15.3:0/872719577 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.302+0000 7fed4278b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c112ae0 con 0x7fed3c1083a0 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.303+0000 7fed40d28640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed3c108cf0 0x7fed3c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.303+0000 7fed3b7fe640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed3c1083a0 0x7fed3c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.303+0000 7fed3bfff640 1 -- 172.21.15.3:0/872719577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed3c1083a0 msgr2=0x7fed3c1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.303+0000 7fed3bfff640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed3c1083a0 0x7fed3c1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.303+0000 7fed3bfff640 1 -- 172.21.15.3:0/872719577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed3c108cf0 msgr2=0x7fed3c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.303+0000 7fed3bfff640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed3c108cf0 0x7fed3c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.303+0000 7fed3bfff640 1 -- 172.21.15.3:0/872719577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c1a86e0 con 0x7fed3c102a60 2024-09-21T06:14:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.303+0000 7fed3bfff640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c102a60 0x7fed3c199060 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fed2c005940 tx=0x7fed2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:41.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.303+0000 7fed397fa640 1 -- 172.21.15.3:0/872719577 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c03f030 con 0x7fed3c102a60 2024-09-21T06:14:41.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.303+0000 7fed397fa640 1 -- 172.21.15.3:0/872719577 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed2c003cb0 con 0x7fed3c102a60 2024-09-21T06:14:41.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.303+0000 7fed4278b640 1 -- 172.21.15.3:0/872719577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3c1a8910 con 0x7fed3c102a60 2024-09-21T06:14:41.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.303+0000 7fed397fa640 1 -- 172.21.15.3:0/872719577 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c035070 con 0x7fed3c102a60 2024-09-21T06:14:41.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.304+0000 7fed4278b640 1 -- 172.21.15.3:0/872719577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3c1a8df0 con 0x7fed3c102a60 2024-09-21T06:14:41.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.304+0000 7fed4278b640 1 -- 172.21.15.3:0/872719577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed3c102e70 con 0x7fed3c102a60 2024-09-21T06:14:41.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.306+0000 7fed397fa640 1 -- 172.21.15.3:0/872719577 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fed2c042030 con 0x7fed3c102a60 2024-09-21T06:14:41.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.307+0000 7fed397fa640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fed240b65f0 0x7fed240b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:41.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.307+0000 7fed3b7fe640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fed240b65f0 0x7fed240b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:41.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.307+0000 7fed397fa640 1 -- 172.21.15.3:0/872719577 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fed2c108080 con 0x7fed3c102a60 2024-09-21T06:14:41.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.307+0000 7fed3b7fe640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fed240b65f0 0x7fed240b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fed3c100c50 tx=0x7fed3000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:41.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.311+0000 7fed397fa640 1 -- 172.21.15.3:0/872719577 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fed2c0ce6d0 con 0x7fed3c102a60 2024-09-21T06:14:41.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.510+0000 7fed4278b640 1 -- 172.21.15.3:0/872719577 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fed3c10ab20 con 0x7fed240b65f0 2024-09-21T06:14:41.512 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:41 smithi003 ceph-mon[31227]: pgmap v315: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:41.512 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:41 smithi003 ceph-mon[31227]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.523+0000 7fed397fa640 1 -- 172.21.15.3:0/872719577 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fed3c10ab20 con 0x7fed240b65f0 2024-09-21T06:14:41.525 INFO:teuthology.orchestra.run.smithi003.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T06:14:41.525 INFO:teuthology.orchestra.run.smithi003.stdout:mds.a smithi003 running (103s) 22s ago 103s 84.6M - 19.3.0-5120-g18432218 ddce307e1f32 5e6cd1bf61a6 2024-09-21T06:14:41.525 INFO:teuthology.orchestra.run.smithi003.stdout:mds.b smithi137 running (101s) 34s ago 100s 96.5M - 19.3.0-5120-g18432218 ddce307e1f32 35f23dd78ded 2024-09-21T06:14:41.525 INFO:teuthology.orchestra.run.smithi003.stdout:mds.c smithi204 running (116s) 34s ago 115s 87.9M - 19.3.0-5120-g18432218 ddce307e1f32 a9f07c06da34 2024-09-21T06:14:41.525 INFO:teuthology.orchestra.run.smithi003.stdout:mds.d smithi003 running (2m) 22s ago 2m 82.0M - 19.3.0-5120-g18432218 ddce307e1f32 e72b103822bf 2024-09-21T06:14:41.525 INFO:teuthology.orchestra.run.smithi003.stdout:mds.e smithi137 running (2m) 34s ago 2m 87.8M - 19.3.0-5120-g18432218 ddce307e1f32 367b67e56167 2024-09-21T06:14:41.525 INFO:teuthology.orchestra.run.smithi003.stdout:mds.f smithi204 running (118s) 34s ago 118s 85.6M - 19.3.0-5120-g18432218 ddce307e1f32 682f556a8944 2024-09-21T06:14:41.525 INFO:teuthology.orchestra.run.smithi003.stdout:mds.g smithi003 running (107s) 22s ago 107s 86.9M - 19.3.0-5120-g18432218 ddce307e1f32 9257d07f9b96 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:mds.h smithi137 running (113s) 34s ago 113s 87.8M - 19.3.0-5120-g18432218 ddce307e1f32 155436ebcdc7 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:mds.i smithi204 running (110s) 34s ago 110s 85.6M - 19.3.0-5120-g18432218 ddce307e1f32 648de31a88de 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:mds.j smithi003 running (2m) 22s ago 2m 83.2M - 19.3.0-5120-g18432218 ddce307e1f32 1b18b84989ff 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:mds.k smithi137 running (2m) 34s ago 2m 84.1M - 19.3.0-5120-g18432218 ddce307e1f32 b1d6d9ac62cb 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:mds.l smithi204 running (2m) 34s ago 2m 84.7M - 19.3.0-5120-g18432218 ddce307e1f32 3debaa0031e2 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.x smithi003 *:9283,8765 running (10m) 22s ago 10m 587M - 19.3.0-5120-g18432218 ddce307e1f32 46bb2d2a398f 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.y smithi137 *:8443,8765 running (8m) 34s ago 8m 524M - 19.3.0-5120-g18432218 ddce307e1f32 f23cce124baf 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.z smithi204 *:8443,8765 running (8m) 34s ago 8m 528M - 19.3.0-5120-g18432218 ddce307e1f32 42ac69c4e6c1 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:mon.a smithi003 running (10m) 22s ago 10m 121M 2048M 19.3.0-5120-g18432218 ddce307e1f32 6ceb763afa3c 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:mon.b smithi137 running (8m) 34s ago 8m 111M 2048M 19.3.0-5120-g18432218 ddce307e1f32 cc17861276f7 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:mon.c smithi204 running (8m) 34s ago 8m 111M 2048M 19.3.0-5120-g18432218 ddce307e1f32 ee491905d688 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:osd.0 smithi003 running (7m) 22s ago 7m 115M 4273M 19.3.0-5120-g18432218 ddce307e1f32 cf8ad698682f 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:osd.1 smithi137 running (6m) 34s ago 6m 113M 4272M 19.3.0-5120-g18432218 ddce307e1f32 c859ed03c62a 2024-09-21T06:14:41.526 INFO:teuthology.orchestra.run.smithi003.stdout:osd.2 smithi204 running (6m) 34s ago 6m 127M 4272M 19.3.0-5120-g18432218 ddce307e1f32 44dacea92457 2024-09-21T06:14:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:osd.3 smithi003 running (6m) 22s ago 5m 120M 4273M 19.3.0-5120-g18432218 ddce307e1f32 6a67cbf73f84 2024-09-21T06:14:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:osd.4 smithi137 running (5m) 34s ago 5m 109M 4272M 19.3.0-5120-g18432218 ddce307e1f32 33a468151e93 2024-09-21T06:14:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:osd.5 smithi204 running (5m) 34s ago 5m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 9b33dcb743cf 2024-09-21T06:14:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:osd.6 smithi003 running (4m) 22s ago 4m 119M 4273M 19.3.0-5120-g18432218 ddce307e1f32 ba155ce0469a 2024-09-21T06:14:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:osd.7 smithi137 running (4m) 34s ago 4m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 97fd481db612 2024-09-21T06:14:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:osd.8 smithi204 running (3m) 34s ago 3m 120M 4272M 19.3.0-5120-g18432218 ddce307e1f32 7f21676818a5 2024-09-21T06:14:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:osd.9 smithi003 running (3m) 22s ago 3m 119M 4273M 19.3.0-5120-g18432218 ddce307e1f32 29941d79a932 2024-09-21T06:14:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:osd.10 smithi137 running (2m) 34s ago 2m 110M 4272M 19.3.0-5120-g18432218 ddce307e1f32 b5881673a662 2024-09-21T06:14:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:osd.11 smithi204 running (2m) 34s ago 2m 108M 4272M 19.3.0-5120-g18432218 ddce307e1f32 897464b2b4c5 2024-09-21T06:14:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.525+0000 7fed4278b640 1 -- 172.21.15.3:0/872719577 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fed240b65f0 msgr2=0x7fed240b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.525+0000 7fed4278b640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fed240b65f0 0x7fed240b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fed3c100c50 tx=0x7fed3000a2d0 comp rx=0 tx=0).stop 2024-09-21T06:14:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.525+0000 7fed4278b640 1 -- 172.21.15.3:0/872719577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c102a60 msgr2=0x7fed3c199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.525+0000 7fed4278b640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c102a60 0x7fed3c199060 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fed2c005940 tx=0x7fed2c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:14:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.525+0000 7fed4278b640 1 -- 172.21.15.3:0/872719577 shutdown_connections 2024-09-21T06:14:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.525+0000 7fed4278b640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fed240b65f0 0x7fed240b8b00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.525+0000 7fed4278b640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed3c1083a0 0x7fed3c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.525+0000 7fed4278b640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed3c108cf0 0x7fed3c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.525+0000 7fed4278b640 1 --2- 172.21.15.3:0/872719577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c102a60 0x7fed3c199060 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:41.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.525+0000 7fed4278b640 1 -- 172.21.15.3:0/872719577 >> 172.21.15.3:0/872719577 conn(0x7fed3c079730 msgr2=0x7fed3c1013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:41.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.525+0000 7fed4278b640 1 -- 172.21.15.3:0/872719577 shutdown_connections 2024-09-21T06:14:41.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:41.525+0000 7fed4278b640 1 -- 172.21.15.3:0/872719577 wait complete. 2024-09-21T06:14:41.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:41 smithi137 ceph-mon[34464]: pgmap v315: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:41.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:41 smithi137 ceph-mon[34464]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:41 smithi204 ceph-mon[34659]: pgmap v315: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:41 smithi204 ceph-mon[34659]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:42.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T06:14:42.557 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:43.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.052+0000 7f73112ec640 1 -- 172.21.15.3:0/2960787197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c103160 msgr2=0x7f730c103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.052+0000 7f73112ec640 1 --2- 172.21.15.3:0/2960787197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c103160 0x7f730c103570 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009cc0 tx=0x7f72fc02caf0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.052+0000 7f73112ec640 1 -- 172.21.15.3:0/2960787197 shutdown_connections 2024-09-21T06:14:43.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.052+0000 7f73112ec640 1 --2- 172.21.15.3:0/2960787197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f730c105170 0x7f730c105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.052+0000 7f73112ec640 1 --2- 172.21.15.3:0/2960787197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f730c105ac0 0x7f730c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.052+0000 7f73112ec640 1 --2- 172.21.15.3:0/2960787197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c103160 0x7f730c103570 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.052+0000 7f73112ec640 1 -- 172.21.15.3:0/2960787197 >> 172.21.15.3:0/2960787197 conn(0x7f730c077c60 msgr2=0x7f730c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:43.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.052+0000 7f73112ec640 1 -- 172.21.15.3:0/2960787197 shutdown_connections 2024-09-21T06:14:43.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.052+0000 7f73112ec640 1 -- 172.21.15.3:0/2960787197 wait complete. 2024-09-21T06:14:43.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.053+0000 7f73112ec640 1 Processor -- start 2024-09-21T06:14:43.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.053+0000 7f73112ec640 1 -- start start 2024-09-21T06:14:43.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.053+0000 7f73112ec640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c103160 0x7f730c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.053+0000 7f73112ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f730c105170 0x7f730c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.053+0000 7f73112ec640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f730c105ac0 0x7f730c19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.053+0000 7f73112ec640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1118c0 con 0x7f730c103160 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.053+0000 7f73112ec640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f730c111740 con 0x7f730c105170 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.053+0000 7f73112ec640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f730c111a40 con 0x7f730c105ac0 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.053+0000 7f730affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c103160 0x7f730c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.053+0000 7f730affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c103160 0x7f730c1982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33536/0 (socket says 172.21.15.3:33536) 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.053+0000 7f730affd640 1 -- 172.21.15.3:0/824764920 learned_addr learned my addr 172.21.15.3:0/824764920 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f730a7fc640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f730c105170 0x7f730c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f730b7fe640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f730c105ac0 0x7f730c19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f730affd640 1 -- 172.21.15.3:0/824764920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f730c105ac0 msgr2=0x7f730c19db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f730affd640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f730c105ac0 0x7f730c19db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f730affd640 1 -- 172.21.15.3:0/824764920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f730c105170 msgr2=0x7f730c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f730affd640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f730c105170 0x7f730c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f730affd640 1 -- 172.21.15.3:0/824764920 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730c19e1a0 con 0x7f730c103160 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f730a7fc640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f730c105170 0x7f730c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f730b7fe640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f730c105ac0 0x7f730c19db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f730affd640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c103160 0x7f730c1982f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f72fc005e30 tx=0x7f72fc02f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f72f3fff640 1 -- 172.21.15.3:0/824764920 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc046070 con 0x7f730c103160 2024-09-21T06:14:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f72f3fff640 1 -- 172.21.15.3:0/824764920 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72fc02fc00 con 0x7f730c103160 2024-09-21T06:14:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f73112ec640 1 -- 172.21.15.3:0/824764920 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730c19e430 con 0x7f730c103160 2024-09-21T06:14:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f72f3fff640 1 -- 172.21.15.3:0/824764920 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc02ea40 con 0x7f730c103160 2024-09-21T06:14:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.054+0000 7f73112ec640 1 -- 172.21.15.3:0/824764920 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f730c19e960 con 0x7f730c103160 2024-09-21T06:14:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.055+0000 7f73112ec640 1 -- 172.21.15.3:0/824764920 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f730c103570 con 0x7f730c103160 2024-09-21T06:14:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.057+0000 7f72f3fff640 1 -- 172.21.15.3:0/824764920 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f72fc049070 con 0x7f730c103160 2024-09-21T06:14:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.057+0000 7f72f3fff640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f72f40b66a0 0x7f72f40b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.058+0000 7f730a7fc640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f72f40b66a0 0x7f72f40b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.058+0000 7f72f3fff640 1 -- 172.21.15.3:0/824764920 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f72fc109650 con 0x7f730c103160 2024-09-21T06:14:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.058+0000 7f730a7fc640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f72f40b66a0 0x7f72f40b8bb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f730c101cb0 tx=0x7f7300005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.062+0000 7f72f3fff640 1 -- 172.21.15.3:0/824764920 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f730c19e960 con 0x7f730c103160 2024-09-21T06:14:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.261+0000 7f73112ec640 1 -- 172.21.15.3:0/824764920 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f730c063800 con 0x7f72f40b66a0 2024-09-21T06:14:43.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.267+0000 7f72f3fff640 1 -- 172.21.15.3:0/824764920 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f730c063800 con 0x7f72f40b66a0 2024-09-21T06:14:43.269 INFO:teuthology.orchestra.run.smithi003.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T06:14:43.269 INFO:teuthology.orchestra.run.smithi003.stdout:mds.all 12/12 36s ago 2m smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-21T06:14:43.269 INFO:teuthology.orchestra.run.smithi003.stdout:mgr 3/3 36s ago 8m smithi003=x;smithi137=y;smithi204=z;count:3 2024-09-21T06:14:43.269 INFO:teuthology.orchestra.run.smithi003.stdout:mon 3/3 36s ago 8m smithi003:172.21.15.3=a;smithi137:172.21.15.137=b;smithi204:172.21.15.204=c;count:3 2024-09-21T06:14:43.269 INFO:teuthology.orchestra.run.smithi003.stdout:osd 12 36s ago - 2024-09-21T06:14:43.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.269+0000 7f73112ec640 1 -- 172.21.15.3:0/824764920 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f72f40b66a0 msgr2=0x7f72f40b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.269+0000 7f73112ec640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f72f40b66a0 0x7f72f40b8bb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f730c101cb0 tx=0x7f7300005e50 comp rx=0 tx=0).stop 2024-09-21T06:14:43.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.269+0000 7f73112ec640 1 -- 172.21.15.3:0/824764920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c103160 msgr2=0x7f730c1982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.269+0000 7f73112ec640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c103160 0x7f730c1982f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f72fc005e30 tx=0x7f72fc02f800 comp rx=0 tx=0).stop 2024-09-21T06:14:43.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.269+0000 7f73112ec640 1 -- 172.21.15.3:0/824764920 shutdown_connections 2024-09-21T06:14:43.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.269+0000 7f73112ec640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f72f40b66a0 0x7f72f40b8bb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.269+0000 7f73112ec640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f730c105ac0 0x7f730c19db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.269+0000 7f73112ec640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f730c105170 0x7f730c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.269+0000 7f73112ec640 1 --2- 172.21.15.3:0/824764920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730c103160 0x7f730c1982f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.269+0000 7f73112ec640 1 -- 172.21.15.3:0/824764920 >> 172.21.15.3:0/824764920 conn(0x7f730c077c60 msgr2=0x7f730c106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:43.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.269+0000 7f73112ec640 1 -- 172.21.15.3:0/824764920 shutdown_connections 2024-09-21T06:14:43.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:43.270+0000 7f73112ec640 1 -- 172.21.15.3:0/824764920 wait complete. 2024-09-21T06:14:43.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:43 smithi003 ceph-mon[31227]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:43.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:43 smithi003 ceph-mon[31227]: pgmap v316: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:43.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:43 smithi137 ceph-mon[34464]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:43 smithi137 ceph-mon[34464]: pgmap v316: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:43.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:43 smithi204 ceph-mon[34659]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:43.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:43 smithi204 ceph-mon[34659]: pgmap v316: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:44.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T06:14:44.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:44 smithi003 ceph-mon[31227]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:44.358 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:44 smithi137 ceph-mon[34464]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:44.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:44 smithi204 ceph-mon[34659]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.863+0000 7f7fa690f640 1 -- 172.21.15.3:0/1286476879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa0108d20 msgr2=0x7f7fa010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.863+0000 7f7fa690f640 1 --2- 172.21.15.3:0/1286476879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa0108d20 0x7f7fa010f550 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009ca0 tx=0x7f7f9802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.863+0000 7f7fa690f640 1 -- 172.21.15.3:0/1286476879 shutdown_connections 2024-09-21T06:14:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.863+0000 7f7fa690f640 1 --2- 172.21.15.3:0/1286476879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7fa01083d0 0x7f7fa01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.863+0000 7f7fa690f640 1 --2- 172.21.15.3:0/1286476879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa0108d20 0x7f7fa010f550 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.863+0000 7f7fa690f640 1 --2- 172.21.15.3:0/1286476879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fa0102a90 0x7f7fa0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.863+0000 7f7fa690f640 1 -- 172.21.15.3:0/1286476879 >> 172.21.15.3:0/1286476879 conn(0x7f7fa0079750 msgr2=0x7f7fa0079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.863+0000 7f7fa690f640 1 -- 172.21.15.3:0/1286476879 shutdown_connections 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.863+0000 7f7fa690f640 1 -- 172.21.15.3:0/1286476879 wait complete. 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.864+0000 7f7fa690f640 1 Processor -- start 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.864+0000 7f7fa690f640 1 -- start start 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.864+0000 7f7fa690f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa0102a90 0x7f7fa0199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.864+0000 7f7fa690f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7fa01083d0 0x7f7fa0199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.864+0000 7f7fa690f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fa0108d20 0x7f7fa019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.864+0000 7f7fa690f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0112990 con 0x7f7fa0102a90 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.864+0000 7f7fa690f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0112810 con 0x7f7fa0108d20 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.864+0000 7f7fa690f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0112b10 con 0x7f7fa01083d0 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.864+0000 7f7f9ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa0102a90 0x7f7fa0199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:44.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.864+0000 7f7f9ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa0102a90 0x7f7fa0199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33554/0 (socket says 172.21.15.3:33554) 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.864+0000 7f7f9ffff640 1 -- 172.21.15.3:0/1783241619 learned_addr learned my addr 172.21.15.3:0/1783241619 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.865+0000 7f7f9f7fe640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7fa01083d0 0x7f7fa0199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.865+0000 7f7fa4eac640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fa0108d20 0x7f7fa019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.865+0000 7f7f9ffff640 1 -- 172.21.15.3:0/1783241619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7fa01083d0 msgr2=0x7f7fa0199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.865+0000 7f7f9ffff640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7fa01083d0 0x7f7fa0199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.865+0000 7f7f9ffff640 1 -- 172.21.15.3:0/1783241619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fa0108d20 msgr2=0x7f7fa019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.865+0000 7f7f9ffff640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fa0108d20 0x7f7fa019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.865+0000 7f7f9ffff640 1 -- 172.21.15.3:0/1783241619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa019f2d0 con 0x7f7fa0102a90 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.865+0000 7f7f9ffff640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa0102a90 0x7f7fa0199360 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009970 tx=0x7f7f90009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.865+0000 7f7f9d7fa640 1 -- 172.21.15.3:0/1783241619 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9000d020 con 0x7f7fa0102a90 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.865+0000 7f7f9d7fa640 1 -- 172.21.15.3:0/1783241619 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f9000d1c0 con 0x7f7fa0102a90 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.865+0000 7f7f9d7fa640 1 -- 172.21.15.3:0/1783241619 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9000cc50 con 0x7f7fa0102a90 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.865+0000 7f7fa690f640 1 -- 172.21.15.3:0/1783241619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa019f5c0 con 0x7f7fa0102a90 2024-09-21T06:14:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.865+0000 7f7fa690f640 1 -- 172.21.15.3:0/1783241619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa006bb80 con 0x7f7fa0102a90 2024-09-21T06:14:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.866+0000 7f7fa690f640 1 -- 172.21.15.3:0/1783241619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa0103d10 con 0x7f7fa0102a90 2024-09-21T06:14:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.868+0000 7f7f9d7fa640 1 -- 172.21.15.3:0/1783241619 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f7f90007860 con 0x7f7fa0102a90 2024-09-21T06:14:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.869+0000 7f7f9d7fa640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7f880b65f0 0x7f7f880b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.869+0000 7f7f9f7fe640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7f880b65f0 0x7f7f880b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.869+0000 7f7f9d7fa640 1 -- 172.21.15.3:0/1783241619 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f7f900e9000 con 0x7f7fa0102a90 2024-09-21T06:14:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.869+0000 7f7f9f7fe640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7f880b65f0 0x7f7f880b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7f94005fd0 tx=0x7f7f94005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:44.872+0000 7f7f9d7fa640 1 -- 172.21.15.3:0/1783241619 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7f900af5f0 con 0x7f7fa0102a90 2024-09-21T06:14:45.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.065+0000 7f7fa690f640 1 -- 172.21.15.3:0/1783241619 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7fa006c680 con 0x7f7f880b65f0 2024-09-21T06:14:45.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.067+0000 7f7f9d7fa640 1 -- 172.21.15.3:0/1783241619 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7fa006c680 con 0x7f7f880b65f0 2024-09-21T06:14:45.069 INFO:teuthology.orchestra.run.smithi003.stdout:HOST ADDR LABELS STATUS 2024-09-21T06:14:45.069 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 172.21.15.3 2024-09-21T06:14:45.069 INFO:teuthology.orchestra.run.smithi003.stdout:smithi137 172.21.15.137 2024-09-21T06:14:45.069 INFO:teuthology.orchestra.run.smithi003.stdout:smithi204 172.21.15.204 2024-09-21T06:14:45.069 INFO:teuthology.orchestra.run.smithi003.stdout:3 hosts in cluster 2024-09-21T06:14:45.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.070+0000 7f7fa690f640 1 -- 172.21.15.3:0/1783241619 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7f880b65f0 msgr2=0x7f7f880b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:45.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.070+0000 7f7fa690f640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7f880b65f0 0x7f7f880b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7f94005fd0 tx=0x7f7f94005850 comp rx=0 tx=0).stop 2024-09-21T06:14:45.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.070+0000 7f7fa690f640 1 -- 172.21.15.3:0/1783241619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa0102a90 msgr2=0x7f7fa0199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:45.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.070+0000 7f7fa690f640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa0102a90 0x7f7fa0199360 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009970 tx=0x7f7f90009e30 comp rx=0 tx=0).stop 2024-09-21T06:14:45.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.070+0000 7f7fa690f640 1 -- 172.21.15.3:0/1783241619 shutdown_connections 2024-09-21T06:14:45.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.070+0000 7f7fa690f640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7f880b65f0 0x7f7f880b8b00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.070+0000 7f7fa690f640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7fa01083d0 0x7f7fa0199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.070+0000 7f7fa690f640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fa0108d20 0x7f7fa019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.070+0000 7f7fa690f640 1 --2- 172.21.15.3:0/1783241619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fa0102a90 0x7f7fa0199360 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.070+0000 7f7fa690f640 1 -- 172.21.15.3:0/1783241619 >> 172.21.15.3:0/1783241619 conn(0x7f7fa0079750 msgr2=0x7f7fa01043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:45.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.070+0000 7f7fa690f640 1 -- 172.21.15.3:0/1783241619 shutdown_connections 2024-09-21T06:14:45.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:45.070+0000 7f7fa690f640 1 -- 172.21.15.3:0/1783241619 wait complete. 2024-09-21T06:14:45.339 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:45 smithi003 ceph-mon[31227]: pgmap v317: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:45 smithi137 ceph-mon[34464]: pgmap v317: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:45 smithi204 ceph-mon[34659]: pgmap v317: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:45.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T06:14:45.989 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:46.369 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:46 smithi003 ceph-mon[31227]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:46.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.535+0000 7fd279b74640 1 -- 172.21.15.3:0/855946066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2741083a0 msgr2=0x7fd2741087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:46.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.535+0000 7fd279b74640 1 --2- 172.21.15.3:0/855946066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2741083a0 0x7fd2741087b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009900 tx=0x7fd25c02a410 comp rx=0 tx=0).stop 2024-09-21T06:14:46.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.535+0000 7fd279b74640 1 -- 172.21.15.3:0/855946066 shutdown_connections 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.535+0000 7fd279b74640 1 --2- 172.21.15.3:0/855946066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd274108cf0 0x7fd27410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.535+0000 7fd279b74640 1 --2- 172.21.15.3:0/855946066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2741083a0 0x7fd2741087b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.535+0000 7fd279b74640 1 --2- 172.21.15.3:0/855946066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd274102a60 0x7fd274102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.535+0000 7fd279b74640 1 -- 172.21.15.3:0/855946066 >> 172.21.15.3:0/855946066 conn(0x7fd274079730 msgr2=0x7fd274079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.535+0000 7fd279b74640 1 -- 172.21.15.3:0/855946066 shutdown_connections 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.535+0000 7fd279b74640 1 -- 172.21.15.3:0/855946066 wait complete. 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.536+0000 7fd279b74640 1 Processor -- start 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.536+0000 7fd279b74640 1 -- start start 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.536+0000 7fd279b74640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd274102a60 0x7fd274199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.536+0000 7fd279b74640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2741083a0 0x7fd274199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.536+0000 7fd279b74640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd274108cf0 0x7fd27419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.536+0000 7fd279b74640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd274112960 con 0x7fd274102a60 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.536+0000 7fd279b74640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd2741127e0 con 0x7fd274108cf0 2024-09-21T06:14:46.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.536+0000 7fd279b74640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd274112ae0 con 0x7fd2741083a0 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.536+0000 7fd2737fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd274102a60 0x7fd274199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.536+0000 7fd2737fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd274102a60 0x7fd274199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33188/0 (socket says 172.21.15.3:33188) 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.536+0000 7fd2737fe640 1 -- 172.21.15.3:0/436958305 learned_addr learned my addr 172.21.15.3:0/436958305 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.537+0000 7fd273fff640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd274108cf0 0x7fd27419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.537+0000 7fd272ffd640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2741083a0 0x7fd274199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.537+0000 7fd2737fe640 1 -- 172.21.15.3:0/436958305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2741083a0 msgr2=0x7fd274199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.537+0000 7fd2737fe640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2741083a0 0x7fd274199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.537+0000 7fd2737fe640 1 -- 172.21.15.3:0/436958305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd274108cf0 msgr2=0x7fd27419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.537+0000 7fd2737fe640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd274108cf0 0x7fd27419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.537+0000 7fd2737fe640 1 -- 172.21.15.3:0/436958305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27419f340 con 0x7fd274102a60 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.537+0000 7fd2737fe640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd274102a60 0x7fd274199490 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd26400eab0 tx=0x7fd26400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.537+0000 7fd270ff9640 1 -- 172.21.15.3:0/436958305 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26400ce40 con 0x7fd274102a60 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.537+0000 7fd279b74640 1 -- 172.21.15.3:0/436958305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27419f630 con 0x7fd274102a60 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.537+0000 7fd270ff9640 1 -- 172.21.15.3:0/436958305 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd264011920 con 0x7fd274102a60 2024-09-21T06:14:46.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.537+0000 7fd270ff9640 1 -- 172.21.15.3:0/436958305 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264010a30 con 0x7fd274102a60 2024-09-21T06:14:46.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.537+0000 7fd279b74640 1 -- 172.21.15.3:0/436958305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27419fae0 con 0x7fd274102a60 2024-09-21T06:14:46.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.538+0000 7fd279b74640 1 -- 172.21.15.3:0/436958305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd274102e70 con 0x7fd274102a60 2024-09-21T06:14:46.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.541+0000 7fd270ff9640 1 -- 172.21.15.3:0/436958305 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fd264010bd0 con 0x7fd274102a60 2024-09-21T06:14:46.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.542+0000 7fd270ff9640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd2600b6620 0x7fd2600b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:46.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.542+0000 7fd272ffd640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd2600b6620 0x7fd2600b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:46.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.542+0000 7fd270ff9640 1 -- 172.21.15.3:0/436958305 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fd2640e7c00 con 0x7fd274102a60 2024-09-21T06:14:46.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.542+0000 7fd272ffd640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd2600b6620 0x7fd2600b8b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd25c000c00 tx=0x7fd25c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:46.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.545+0000 7fd270ff9640 1 -- 172.21.15.3:0/436958305 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd264016030 con 0x7fd274102a60 2024-09-21T06:14:46.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:46 smithi137 ceph-mon[34464]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:46.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:46 smithi204 ceph-mon[34659]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:46.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.743+0000 7fd279b74640 1 -- 172.21.15.3:0/436958305 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fd27410a650 con 0x7fd2600b6620 2024-09-21T06:14:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.745+0000 7fd270ff9640 1 -- 172.21.15.3:0/436958305 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fd27410a650 con 0x7fd2600b6620 2024-09-21T06:14:46.746 INFO:teuthology.orchestra.run.smithi003.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T06:14:46.746 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009Z400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T06:14:46.747 INFO:teuthology.orchestra.run.smithi003.stdout:smithi137 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233004A400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T06:14:46.747 INFO:teuthology.orchestra.run.smithi003.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T06:14:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.747+0000 7fd279b74640 1 -- 172.21.15.3:0/436958305 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd2600b6620 msgr2=0x7fd2600b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.747+0000 7fd279b74640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd2600b6620 0x7fd2600b8b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd25c000c00 tx=0x7fd25c031400 comp rx=0 tx=0).stop 2024-09-21T06:14:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.747+0000 7fd279b74640 1 -- 172.21.15.3:0/436958305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd274102a60 msgr2=0x7fd274199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.747+0000 7fd279b74640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd274102a60 0x7fd274199490 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd26400eab0 tx=0x7fd26400ef70 comp rx=0 tx=0).stop 2024-09-21T06:14:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.747+0000 7fd279b74640 1 -- 172.21.15.3:0/436958305 shutdown_connections 2024-09-21T06:14:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.747+0000 7fd279b74640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd2600b6620 0x7fd2600b8b30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.747+0000 7fd279b74640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2741083a0 0x7fd274199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.747+0000 7fd279b74640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd274108cf0 0x7fd27419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:46.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.747+0000 7fd279b74640 1 --2- 172.21.15.3:0/436958305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd274102a60 0x7fd274199490 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:46.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.747+0000 7fd279b74640 1 -- 172.21.15.3:0/436958305 >> 172.21.15.3:0/436958305 conn(0x7fd274079730 msgr2=0x7fd2741044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:46.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.747+0000 7fd279b74640 1 -- 172.21.15.3:0/436958305 shutdown_connections 2024-09-21T06:14:46.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:46.747+0000 7fd279b74640 1 -- 172.21.15.3:0/436958305 wait complete. 2024-09-21T06:14:47.419 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:47 smithi003 ceph-mon[31227]: pgmap v318: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:47.453 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T06:14:47.464 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-09-21T06:14:47.464 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T06:14:47.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:47 smithi137 ceph-mon[34464]: pgmap v318: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:47.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:47 smithi204 ceph-mon[34659]: pgmap v318: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:47.731 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.214+0000 7f91e7475640 1 -- 172.21.15.3:0/341932760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91e010de40 msgr2=0x7f91e0110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.214+0000 7f91e7475640 1 --2- 172.21.15.3:0/341932760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91e010de40 0x7f91e0110310 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009ca0 tx=0x7f91d802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.214+0000 7f91e7475640 1 -- 172.21.15.3:0/341932760 shutdown_connections 2024-09-21T06:14:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.214+0000 7f91e7475640 1 --2- 172.21.15.3:0/341932760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91e010de40 0x7f91e0110310 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.214+0000 7f91e7475640 1 --2- 172.21.15.3:0/341932760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e0105360 0x7f91e010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.214+0000 7f91e7475640 1 --2- 172.21.15.3:0/341932760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e0104a10 0x7f91e0104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.214+0000 7f91e7475640 1 -- 172.21.15.3:0/341932760 >> 172.21.15.3:0/341932760 conn(0x7f91e0079720 msgr2=0x7f91e0079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.214+0000 7f91e7475640 1 -- 172.21.15.3:0/341932760 shutdown_connections 2024-09-21T06:14:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.214+0000 7f91e7475640 1 -- 172.21.15.3:0/341932760 wait complete. 2024-09-21T06:14:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.215+0000 7f91e7475640 1 Processor -- start 2024-09-21T06:14:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.215+0000 7f91e7475640 1 -- start start 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.215+0000 7f91e7475640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e0104a10 0x7f91e019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.215+0000 7f91e7475640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91e0105360 0x7f91e019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.215+0000 7f91e7475640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e010de40 0x7f91e019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.215+0000 7f91e7475640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0113aa0 con 0x7f91e0105360 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.215+0000 7f91e7475640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0113920 con 0x7f91e010de40 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.215+0000 7f91e7475640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0113c20 con 0x7f91e0104a10 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.215+0000 7f91e4a10640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91e0105360 0x7f91e019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.215+0000 7f91e4a10640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91e0105360 0x7f91e019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33204/0 (socket says 172.21.15.3:33204) 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.215+0000 7f91e4a10640 1 -- 172.21.15.3:0/1573641125 learned_addr learned my addr 172.21.15.3:0/1573641125 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.215+0000 7f91e5a12640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e010de40 0x7f91e019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.215+0000 7f91e5211640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e0104a10 0x7f91e019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.216+0000 7f91e4a10640 1 -- 172.21.15.3:0/1573641125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e0104a10 msgr2=0x7f91e019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.216+0000 7f91e4a10640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e0104a10 0x7f91e019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.216+0000 7f91e4a10640 1 -- 172.21.15.3:0/1573641125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e010de40 msgr2=0x7f91e019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:48.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.216+0000 7f91e4a10640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e010de40 0x7f91e019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:48.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.216+0000 7f91e4a10640 1 -- 172.21.15.3:0/1573641125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e01a03c0 con 0x7f91e0105360 2024-09-21T06:14:48.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.216+0000 7f91e5a12640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e010de40 0x7f91e019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:48.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.216+0000 7f91e4a10640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91e0105360 0x7f91e019a9f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f91d000d740 tx=0x7f91d000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:48.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.216+0000 7f91d67fc640 1 -- 172.21.15.3:0/1573641125 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0017070 con 0x7f91e0105360 2024-09-21T06:14:48.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.216+0000 7f91d67fc640 1 -- 172.21.15.3:0/1573641125 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d000de50 con 0x7f91e0105360 2024-09-21T06:14:48.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.216+0000 7f91e7475640 1 -- 172.21.15.3:0/1573641125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e01a06b0 con 0x7f91e0105360 2024-09-21T06:14:48.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.216+0000 7f91d67fc640 1 -- 172.21.15.3:0/1573641125 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d000f620 con 0x7f91e0105360 2024-09-21T06:14:48.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.216+0000 7f91e7475640 1 -- 172.21.15.3:0/1573641125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e01a6e10 con 0x7f91e0105360 2024-09-21T06:14:48.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.217+0000 7f91e7475640 1 -- 172.21.15.3:0/1573641125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e0104e20 con 0x7f91e0105360 2024-09-21T06:14:48.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.220+0000 7f91d67fc640 1 -- 172.21.15.3:0/1573641125 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f91d001a070 con 0x7f91e0105360 2024-09-21T06:14:48.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.220+0000 7f91d67fc640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f91cc0b6670 0x7f91cc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:48.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.220+0000 7f91e5211640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f91cc0b6670 0x7f91cc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:48.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.220+0000 7f91d67fc640 1 -- 172.21.15.3:0/1573641125 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f91d00e6fe0 con 0x7f91e0105360 2024-09-21T06:14:48.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.221+0000 7f91e5211640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f91cc0b6670 0x7f91cc0b8b80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f91dc005fd0 tx=0x7f91dc005e20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:48.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.223+0000 7f91d67fc640 1 -- 172.21.15.3:0/1573641125 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f91d00ad690 con 0x7f91e0105360 2024-09-21T06:14:48.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.449+0000 7f91e7475640 1 -- 172.21.15.3:0/1573641125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f91e01a7040 con 0x7f91e0105360 2024-09-21T06:14:48.452 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:48 smithi003 ceph-mon[31227]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:48.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.450+0000 7f91d67fc640 1 -- 172.21.15.3:0/1573641125 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3121 (secure 0 0 0) 0x7f91d00b1a70 con 0x7f91e0105360 2024-09-21T06:14:48.453 INFO:teuthology.orchestra.run.smithi003.stdout:e27 2024-09-21T06:14:48.453 INFO:teuthology.orchestra.run.smithi003.stdout:btime 2024-09-21T06:13:23:600046+0000 2024-09-21T06:14:48.453 INFO:teuthology.orchestra.run.smithi003.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T06:14:48.453 INFO:teuthology.orchestra.run.smithi003.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:14:48.453 INFO:teuthology.orchestra.run.smithi003.stdout:legacy client fscid: 1 2024-09-21T06:14:48.453 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:48.453 INFO:teuthology.orchestra.run.smithi003.stdout:Filesystem 'cephfs' (1) 2024-09-21T06:14:48.453 INFO:teuthology.orchestra.run.smithi003.stdout:fs_name cephfs 2024-09-21T06:14:48.454 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 27 2024-09-21T06:14:48.454 INFO:teuthology.orchestra.run.smithi003.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T06:14:48.454 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-09-21T06:13:11.205233+0000 2024-09-21T06:14:48.454 INFO:teuthology.orchestra.run.smithi003.stdout:modified 2024-09-21T06:13:23.600022+0000 2024-09-21T06:14:48.454 INFO:teuthology.orchestra.run.smithi003.stdout:tableserver 0 2024-09-21T06:14:48.454 INFO:teuthology.orchestra.run.smithi003.stdout:root 0 2024-09-21T06:14:48.454 INFO:teuthology.orchestra.run.smithi003.stdout:session_timeout 300 2024-09-21T06:14:48.454 INFO:teuthology.orchestra.run.smithi003.stdout:session_autoclose 300 2024-09-21T06:14:48.454 INFO:teuthology.orchestra.run.smithi003.stdout:max_file_size 1099511627776 2024-09-21T06:14:48.454 INFO:teuthology.orchestra.run.smithi003.stdout:max_xattr_size 65536 2024-09-21T06:14:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:required_client_features {} 2024-09-21T06:14:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:last_failure 0 2024-09-21T06:14:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:last_failure_osd_epoch 0 2024-09-21T06:14:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:14:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:max_mds 5 2024-09-21T06:14:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:in 0,1,2,3,4 2024-09-21T06:14:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:up {0=24704,1=24691,2=24679,3=24674,4=24670} 2024-09-21T06:14:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:failed 2024-09-21T06:14:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:damaged 2024-09-21T06:14:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:stopped 2024-09-21T06:14:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:data_pools [3] 2024-09-21T06:14:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:metadata_pool 2 2024-09-21T06:14:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:inline_data disabled 2024-09-21T06:14:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:balancer 2024-09-21T06:14:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:bal_rank_mask -1 2024-09-21T06:14:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:standby_count_wanted 0 2024-09-21T06:14:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:qdb_cluster leader: 24704 members: 24704,24691,24679,24674,24670 2024-09-21T06:14:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b{0:24704} state up:active seq 4 addr [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:14:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.e{0:24644} state up:standby-replay seq 1 addr [v2:172.21.15.137:6834/399369618,v1:172.21.15.137:6835/399369618] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:14:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.i{1:24691} state up:active seq 9 addr [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:14:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.l{1:24631} state up:standby-replay seq 1 addr [v2:172.21.15.204:6832/2480313554,v1:172.21.15.204:6833/2480313554] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:14:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c{2:24679} state up:active seq 10 addr [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:14:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.k{2:24620} state up:standby-replay seq 1 addr [v2:172.21.15.137:6832/4270441897,v1:172.21.15.137:6833/4270441897] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:14:48.457 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.h{3:24674} state up:active seq 10 addr [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:14:48.457 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a{3:14856} state up:standby-replay seq 1 addr [v2:172.21.15.3:6840/2403205583,v1:172.21.15.3:6841/2403205583] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:14:48.457 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.f{4:24670} state up:active seq 11 addr [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:14:48.457 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.g{4:14844} state up:standby-replay seq 1 addr [v2:172.21.15.3:6838/1325295587,v1:172.21.15.3:6839/1325295587] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:14:48.457 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:48.457 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:48.457 INFO:teuthology.orchestra.run.smithi003.stdout:Standby daemons: 2024-09-21T06:14:48.457 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:48.457 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.j{-1:14781} state up:standby seq 1 addr [v2:172.21.15.3:6834/601337393,v1:172.21.15.3:6835/601337393] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:14:48.457 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d{-1:14793} state up:standby seq 1 addr [v2:172.21.15.3:6836/2476398034,v1:172.21.15.3:6837/2476398034] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:14:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-09-21T06:14:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.452+0000 7f91e7475640 1 -- 172.21.15.3:0/1573641125 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f91cc0b6670 msgr2=0x7f91cc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.452+0000 7f91e7475640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f91cc0b6670 0x7f91cc0b8b80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f91dc005fd0 tx=0x7f91dc005e20 comp rx=0 tx=0).stop 2024-09-21T06:14:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.452+0000 7f91e7475640 1 -- 172.21.15.3:0/1573641125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91e0105360 msgr2=0x7f91e019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.452+0000 7f91e7475640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91e0105360 0x7f91e019a9f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f91d000d740 tx=0x7f91d000dc00 comp rx=0 tx=0).stop 2024-09-21T06:14:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.452+0000 7f91e7475640 1 -- 172.21.15.3:0/1573641125 shutdown_connections 2024-09-21T06:14:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.452+0000 7f91e7475640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f91cc0b6670 0x7f91cc0b8b80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.452+0000 7f91e7475640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e010de40 0x7f91e019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.452+0000 7f91e7475640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91e0105360 0x7f91e019a9f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.452+0000 7f91e7475640 1 --2- 172.21.15.3:0/1573641125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e0104a10 0x7f91e019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.452+0000 7f91e7475640 1 -- 172.21.15.3:0/1573641125 >> 172.21.15.3:0/1573641125 conn(0x7f91e0079720 msgr2=0x7f91e0107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.452+0000 7f91e7475640 1 -- 172.21.15.3:0/1573641125 shutdown_connections 2024-09-21T06:14:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:48.452+0000 7f91e7475640 1 -- 172.21.15.3:0/1573641125 wait complete. 2024-09-21T06:14:48.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:48 smithi137 ceph-mon[34464]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:48.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:48 smithi204 ceph-mon[34659]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:14:49.170 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T06:14:49.432 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:49.539 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:49 smithi003 ceph-mon[31227]: pgmap v319: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:49.539 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:49 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1573641125' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T06:14:49.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:49 smithi137 ceph-mon[34464]: pgmap v319: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:49.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:49 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1573641125' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T06:14:49.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:49 smithi204 ceph-mon[34659]: pgmap v319: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:49.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:49 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1573641125' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T06:14:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.900+0000 7fc76bc51640 1 -- 172.21.15.3:0/3091338359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc764102fa0 msgr2=0x7fc76410fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.900+0000 7fc76bc51640 1 --2- 172.21.15.3:0/3091338359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc764102fa0 0x7fc76410fcc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc754009900 tx=0x7fc75402a480 comp rx=0 tx=0).stop 2024-09-21T06:14:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.900+0000 7fc76bc51640 1 -- 172.21.15.3:0/3091338359 shutdown_connections 2024-09-21T06:14:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.900+0000 7fc76bc51640 1 --2- 172.21.15.3:0/3091338359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc764110200 0x7fc764112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.900+0000 7fc76bc51640 1 --2- 172.21.15.3:0/3091338359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc764102fa0 0x7fc76410fcc0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.900+0000 7fc76bc51640 1 --2- 172.21.15.3:0/3091338359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc764102650 0x7fc764102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.900+0000 7fc76bc51640 1 -- 172.21.15.3:0/3091338359 >> 172.21.15.3:0/3091338359 conn(0x7fc764079760 msgr2=0x7fc764079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.900+0000 7fc76bc51640 1 -- 172.21.15.3:0/3091338359 shutdown_connections 2024-09-21T06:14:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.900+0000 7fc76bc51640 1 -- 172.21.15.3:0/3091338359 wait complete. 2024-09-21T06:14:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc76bc51640 1 Processor -- start 2024-09-21T06:14:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc76bc51640 1 -- start start 2024-09-21T06:14:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc76bc51640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc764102650 0x7fc76419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc76bc51640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc764102fa0 0x7fc76419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc76bc51640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc764110200 0x7fc7641a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc76bc51640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc764115d40 con 0x7fc764102fa0 2024-09-21T06:14:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc76bc51640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc764115bc0 con 0x7fc764110200 2024-09-21T06:14:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc76bc51640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc764115ec0 con 0x7fc764102650 2024-09-21T06:14:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc7691ec640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc764102fa0 0x7fc76419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc7691ec640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc764102fa0 0x7fc76419cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33216/0 (socket says 172.21.15.3:33216) 2024-09-21T06:14:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc7691ec640 1 -- 172.21.15.3:0/3123303982 learned_addr learned my addr 172.21.15.3:0/3123303982 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc7699ed640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc764102650 0x7fc76419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc76a1ee640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc764110200 0x7fc7641a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc7691ec640 1 -- 172.21.15.3:0/3123303982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc764102650 msgr2=0x7fc76419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc7691ec640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc764102650 0x7fc76419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc7691ec640 1 -- 172.21.15.3:0/3123303982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc764110200 msgr2=0x7fc7641a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc7691ec640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc764110200 0x7fc7641a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc7691ec640 1 -- 172.21.15.3:0/3123303982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7641a2740 con 0x7fc764102fa0 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc7699ed640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc764102650 0x7fc76419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.901+0000 7fc76a1ee640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc764110200 0x7fc7641a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.902+0000 7fc7691ec640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc764102fa0 0x7fc76419cd70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc754006010 tx=0x7fc7540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.902+0000 7fc75affd640 1 -- 172.21.15.3:0/3123303982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75403f030 con 0x7fc764102fa0 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.902+0000 7fc75affd640 1 -- 172.21.15.3:0/3123303982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc754003800 con 0x7fc764102fa0 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.902+0000 7fc76bc51640 1 -- 172.21.15.3:0/3123303982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7641a29d0 con 0x7fc764102fa0 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.902+0000 7fc75affd640 1 -- 172.21.15.3:0/3123303982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754035070 con 0x7fc764102fa0 2024-09-21T06:14:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.902+0000 7fc76bc51640 1 -- 172.21.15.3:0/3123303982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7641a9190 con 0x7fc764102fa0 2024-09-21T06:14:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.903+0000 7fc76bc51640 1 -- 172.21.15.3:0/3123303982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76406c1f0 con 0x7fc764102fa0 2024-09-21T06:14:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.905+0000 7fc75affd640 1 -- 172.21.15.3:0/3123303982 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fc754003cb0 con 0x7fc764102fa0 2024-09-21T06:14:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.906+0000 7fc75affd640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc7500b65f0 0x7fc7500b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.906+0000 7fc7699ed640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc7500b65f0 0x7fc7500b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.906+0000 7fc75affd640 1 -- 172.21.15.3:0/3123303982 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fc754108260 con 0x7fc764102fa0 2024-09-21T06:14:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.906+0000 7fc7699ed640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc7500b65f0 0x7fc7500b8b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc760006fd0 tx=0x7fc760008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:49.909+0000 7fc75affd640 1 -- 172.21.15.3:0/3123303982 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc7540ce8b0 con 0x7fc764102fa0 2024-09-21T06:14:50.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.099+0000 7fc76bc51640 1 -- 172.21.15.3:0/3123303982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fc764102ae0 con 0x7fc764102fa0 2024-09-21T06:14:50.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.100+0000 7fc75affd640 1 -- 172.21.15.3:0/3123303982 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v79) ==== 56+0+5666 (secure 0 0 0) 0x7fc7540d2c90 con 0x7fc764102fa0 2024-09-21T06:14:50.101 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 79 2024-09-21T06:14:50.102 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:14:50.102 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-09-21T06:04:24.293786+0000 2024-09-21T06:14:50.102 INFO:teuthology.orchestra.run.smithi003.stdout:modified 2024-09-21T06:13:24.594225+0000 2024-09-21T06:14:50.102 INFO:teuthology.orchestra.run.smithi003.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T06:14:50.102 INFO:teuthology.orchestra.run.smithi003.stdout:crush_version 26 2024-09-21T06:14:50.102 INFO:teuthology.orchestra.run.smithi003.stdout:full_ratio 0.95 2024-09-21T06:14:50.102 INFO:teuthology.orchestra.run.smithi003.stdout:backfillfull_ratio 0.9 2024-09-21T06:14:50.102 INFO:teuthology.orchestra.run.smithi003.stdout:nearfull_ratio 0.85 2024-09-21T06:14:50.102 INFO:teuthology.orchestra.run.smithi003.stdout:require_min_compat_client luminous 2024-09-21T06:14:50.102 INFO:teuthology.orchestra.run.smithi003.stdout:min_compat_client luminous 2024-09-21T06:14:50.102 INFO:teuthology.orchestra.run.smithi003.stdout:require_osd_release squid 2024-09-21T06:14:50.103 INFO:teuthology.orchestra.run.smithi003.stdout:stretch_mode_enabled false 2024-09-21T06:14:50.103 INFO:teuthology.orchestra.run.smithi003.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T06:14:50.103 INFO:teuthology.orchestra.run.smithi003.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T06:14:50.103 INFO:teuthology.orchestra.run.smithi003.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-21T06:14:50.103 INFO:teuthology.orchestra.run.smithi003.stdout:max_osd 12 2024-09-21T06:14:50.103 INFO:teuthology.orchestra.run.smithi003.stdout:osd.0 up in weight 1 up_from 9 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6802/615312328,v1:172.21.15.3:6803/615312328] [v2:172.21.15.3:6804/615312328,v1:172.21.15.3:6805/615312328] exists,up 51736f81-f410-48ee-a8f3-121289607309 2024-09-21T06:14:50.103 INFO:teuthology.orchestra.run.smithi003.stdout:osd.1 up in weight 1 up_from 14 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6800/4150747115,v1:172.21.15.137:6801/4150747115] [v2:172.21.15.137:6802/4150747115,v1:172.21.15.137:6803/4150747115] exists,up 55696a37-34c1-482a-8561-7d08f0aa29fa 2024-09-21T06:14:50.103 INFO:teuthology.orchestra.run.smithi003.stdout:osd.2 up in weight 1 up_from 19 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/3392577077,v1:172.21.15.204:6801/3392577077] [v2:172.21.15.204:6802/3392577077,v1:172.21.15.204:6803/3392577077] exists,up 8a81d34b-464b-46a2-8ef2-5223b5947004 2024-09-21T06:14:50.103 INFO:teuthology.orchestra.run.smithi003.stdout:osd.3 up in weight 1 up_from 26 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6810/4293741635,v1:172.21.15.3:6811/4293741635] [v2:172.21.15.3:6812/4293741635,v1:172.21.15.3:6813/4293741635] exists,up 94ab3938-7a70-44cb-99bf-9d2dedc4eb87 2024-09-21T06:14:50.103 INFO:teuthology.orchestra.run.smithi003.stdout:osd.4 up in weight 1 up_from 32 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6808/2608988785,v1:172.21.15.137:6809/2608988785] [v2:172.21.15.137:6810/2608988785,v1:172.21.15.137:6811/2608988785] exists,up 38551535-d692-401c-aad0-bee38f3026d4 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:osd.5 up in weight 1 up_from 37 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/3709074044,v1:172.21.15.204:6809/3709074044] [v2:172.21.15.204:6810/3709074044,v1:172.21.15.204:6811/3709074044] exists,up 3d4b785b-28d0-4654-a9b1-4c5c905f4bcc 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:osd.6 up in weight 1 up_from 43 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6818/3862646594,v1:172.21.15.3:6819/3862646594] [v2:172.21.15.3:6820/3862646594,v1:172.21.15.3:6821/3862646594] exists,up ba21dbe9-ab43-4b7e-9446-2929ee874c43 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:osd.7 up in weight 1 up_from 48 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6816/2086822496,v1:172.21.15.137:6817/2086822496] [v2:172.21.15.137:6818/2086822496,v1:172.21.15.137:6819/2086822496] exists,up 9610fe5c-93df-49f9-b8bd-5cb480e0871b 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:osd.8 up in weight 1 up_from 54 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/153476798,v1:172.21.15.204:6817/153476798] [v2:172.21.15.204:6818/153476798,v1:172.21.15.204:6819/153476798] exists,up 49f6e9b4-6d14-434e-8b42-8b7ffeb2a072 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:osd.9 up in weight 1 up_from 60 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6826/1238491549,v1:172.21.15.3:6827/1238491549] [v2:172.21.15.3:6828/1238491549,v1:172.21.15.3:6829/1238491549] exists,up 633f67a7-b759-4ea6-82ed-3c42b3fb70ff 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:osd.10 up in weight 1 up_from 65 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6824/3486552683,v1:172.21.15.137:6825/3486552683] [v2:172.21.15.137:6826/3486552683,v1:172.21.15.137:6827/3486552683] exists,up 94b65a48-ee87-4e59-9296-a72a22787f03 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:osd.11 up in weight 1 up_from 71 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/1084308610,v1:172.21.15.204:6825/1084308610] [v2:172.21.15.204:6826/1084308610,v1:172.21.15.204:6827/1084308610] exists,up c5cc2850-9062-4b8e-beb3-bef0db30a410 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.a [2,11] 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.10 [2,1] 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.14 [2,4] 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.1d [8,11] 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.0 [4,7] 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.2 [3,9] 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.5 [3,9] 2024-09-21T06:14:50.104 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.e [4,6,5,9] 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.3a [4,8] 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/1573374138 expires 2024-09-22T06:06:23.529208+0000 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6801/986982328 expires 2024-09-22T06:06:23.529208+0000 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/1595702358 expires 2024-09-22T06:04:55.356595+0000 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/547897695 expires 2024-09-22T06:04:55.356595+0000 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/3673347975 expires 2024-09-22T06:04:55.356595+0000 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/141972779 expires 2024-09-22T06:06:23.529208+0000 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6800/1707771276 expires 2024-09-22T06:04:55.356595+0000 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6801/1707771276 expires 2024-09-22T06:04:55.356595+0000 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/2363985440 expires 2024-09-22T06:05:21.689191+0000 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/2138752581 expires 2024-09-22T06:05:21.689191+0000 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6800/986982328 expires 2024-09-22T06:06:23.529208+0000 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/1820734047 expires 2024-09-22T06:05:21.689191+0000 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6800/2929116087 expires 2024-09-22T06:05:21.689191+0000 2024-09-21T06:14:50.105 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/45453677 expires 2024-09-22T06:06:23.529208+0000 2024-09-21T06:14:50.106 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6801/2929116087 expires 2024-09-22T06:05:21.689191+0000 2024-09-21T06:14:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.102+0000 7fc76bc51640 1 -- 172.21.15.3:0/3123303982 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc7500b65f0 msgr2=0x7fc7500b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.102+0000 7fc76bc51640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc7500b65f0 0x7fc7500b8b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc760006fd0 tx=0x7fc760008040 comp rx=0 tx=0).stop 2024-09-21T06:14:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.102+0000 7fc76bc51640 1 -- 172.21.15.3:0/3123303982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc764102fa0 msgr2=0x7fc76419cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.102+0000 7fc76bc51640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc764102fa0 0x7fc76419cd70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc754006010 tx=0x7fc7540042f0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.102+0000 7fc76bc51640 1 -- 172.21.15.3:0/3123303982 shutdown_connections 2024-09-21T06:14:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.102+0000 7fc76bc51640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc7500b65f0 0x7fc7500b8b00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.102+0000 7fc76bc51640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc764110200 0x7fc7641a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.102+0000 7fc76bc51640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc764102fa0 0x7fc76419cd70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.102+0000 7fc76bc51640 1 --2- 172.21.15.3:0/3123303982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc764102650 0x7fc76419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.102+0000 7fc76bc51640 1 -- 172.21.15.3:0/3123303982 >> 172.21.15.3:0/3123303982 conn(0x7fc764079760 msgr2=0x7fc7641116d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.102+0000 7fc76bc51640 1 -- 172.21.15.3:0/3123303982 shutdown_connections 2024-09-21T06:14:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:50.102+0000 7fc76bc51640 1 -- 172.21.15.3:0/3123303982 wait complete. 2024-09-21T06:14:50.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:50 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3123303982' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T06:14:50.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:50 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3123303982' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T06:14:50.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:50 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3123303982' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T06:14:50.964 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T06:14:50.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:14:51.238 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:51.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:51 smithi003 ceph-mon[31227]: pgmap v320: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:51.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:51 smithi137 ceph-mon[34464]: pgmap v320: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:51.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:51 smithi204 ceph-mon[34659]: pgmap v320: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:14:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.733+0000 7f4f9cee3640 1 -- 172.21.15.3:0/4260476302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f981083a0 msgr2=0x7f4f981087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.733+0000 7f4f9cee3640 1 --2- 172.21.15.3:0/4260476302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f981083a0 0x7f4f981087b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009900 tx=0x7f4f8c02a410 comp rx=0 tx=0).stop 2024-09-21T06:14:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.733+0000 7f4f9cee3640 1 -- 172.21.15.3:0/4260476302 shutdown_connections 2024-09-21T06:14:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.733+0000 7f4f9cee3640 1 --2- 172.21.15.3:0/4260476302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f98108cf0 0x7f4f9810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.733+0000 7f4f9cee3640 1 --2- 172.21.15.3:0/4260476302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f981083a0 0x7f4f981087b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.733+0000 7f4f9cee3640 1 --2- 172.21.15.3:0/4260476302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f98102a60 0x7f4f98102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.733+0000 7f4f9cee3640 1 -- 172.21.15.3:0/4260476302 >> 172.21.15.3:0/4260476302 conn(0x7f4f98079730 msgr2=0x7f4f98079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.733+0000 7f4f9cee3640 1 -- 172.21.15.3:0/4260476302 shutdown_connections 2024-09-21T06:14:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.733+0000 7f4f9cee3640 1 -- 172.21.15.3:0/4260476302 wait complete. 2024-09-21T06:14:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.734+0000 7f4f9cee3640 1 Processor -- start 2024-09-21T06:14:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.734+0000 7f4f9cee3640 1 -- start start 2024-09-21T06:14:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.734+0000 7f4f9cee3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f98102a60 0x7f4f981050e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.734+0000 7f4f9cee3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f981083a0 0x7f4f98105620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.734+0000 7f4f9cee3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f98108cf0 0x7f4f98105b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.734+0000 7f4f9cee3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4f981129a0 con 0x7f4f981083a0 2024-09-21T06:14:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.734+0000 7f4f9cee3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4f98112820 con 0x7f4f98108cf0 2024-09-21T06:14:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.734+0000 7f4f9cee3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f98112b20 con 0x7f4f98102a60 2024-09-21T06:14:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.734+0000 7f4f95d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f981083a0 0x7f4f98105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.734+0000 7f4f95d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f981083a0 0x7f4f98105620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33234/0 (socket says 172.21.15.3:33234) 2024-09-21T06:14:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.734+0000 7f4f95d9b640 1 -- 172.21.15.3:0/3409723924 learned_addr learned my addr 172.21.15.3:0/3409723924 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.734+0000 7f4f96d9d640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f98108cf0 0x7f4f98105b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.735+0000 7f4f9659c640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f98102a60 0x7f4f981050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.735+0000 7f4f95d9b640 1 -- 172.21.15.3:0/3409723924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f98102a60 msgr2=0x7f4f981050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.735+0000 7f4f95d9b640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f98102a60 0x7f4f981050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.735+0000 7f4f95d9b640 1 -- 172.21.15.3:0/3409723924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f98108cf0 msgr2=0x7f4f98105b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:51.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.735+0000 7f4f95d9b640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f98108cf0 0x7f4f98105b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:51.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.735+0000 7f4f95d9b640 1 -- 172.21.15.3:0/3409723924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f981a8770 con 0x7f4f981083a0 2024-09-21T06:14:51.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.735+0000 7f4f95d9b640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f981083a0 0x7f4f98105620 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c032fd0 tx=0x7f4f8c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:51.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.735+0000 7f4f877fe640 1 -- 172.21.15.3:0/3409723924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c03f030 con 0x7f4f981083a0 2024-09-21T06:14:51.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.735+0000 7f4f877fe640 1 -- 172.21.15.3:0/3409723924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8c004490 con 0x7f4f981083a0 2024-09-21T06:14:51.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.735+0000 7f4f9cee3640 1 -- 172.21.15.3:0/3409723924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f981a8a00 con 0x7f4f981083a0 2024-09-21T06:14:51.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.735+0000 7f4f877fe640 1 -- 172.21.15.3:0/3409723924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c035070 con 0x7f4f981083a0 2024-09-21T06:14:51.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.735+0000 7f4f9cee3640 1 -- 172.21.15.3:0/3409723924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f981a8f60 con 0x7f4f981083a0 2024-09-21T06:14:51.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.736+0000 7f4f9cee3640 1 -- 172.21.15.3:0/3409723924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f98102e70 con 0x7f4f981083a0 2024-09-21T06:14:51.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.738+0000 7f4f877fe640 1 -- 172.21.15.3:0/3409723924 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f4f8c042030 con 0x7f4f981083a0 2024-09-21T06:14:51.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.739+0000 7f4f877fe640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f800b65f0 0x7f4f800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:51.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.739+0000 7f4f9659c640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f800b65f0 0x7f4f800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:51.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.739+0000 7f4f877fe640 1 -- 172.21.15.3:0/3409723924 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f4f8c107e40 con 0x7f4f981083a0 2024-09-21T06:14:51.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.740+0000 7f4f9659c640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f800b65f0 0x7f4f800b8b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f98106b30 tx=0x7f4f88009110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:51.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.743+0000 7f4f877fe640 1 -- 172.21.15.3:0/3409723924 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4f8c0ce490 con 0x7f4f981083a0 2024-09-21T06:14:51.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.973+0000 7f4f9cee3640 1 -- 172.21.15.3:0/3409723924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f981a92e0 con 0x7f4f981083a0 2024-09-21T06:14:51.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.973+0000 7f4f877fe640 1 -- 172.21.15.3:0/3409723924 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11962 (secure 0 0 0) 0x7f4f8c0d2870 con 0x7f4f981083a0 2024-09-21T06:14:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:51.976 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-09-21T06:13:23:600046+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:13:23.600022+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:14:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-09-21T06:14:51.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.976+0000 7f4f9cee3640 1 -- 172.21.15.3:0/3409723924 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f800b65f0 msgr2=0x7f4f800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:51.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.976+0000 7f4f9cee3640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f800b65f0 0x7f4f800b8b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f98106b30 tx=0x7f4f88009110 comp rx=0 tx=0).stop 2024-09-21T06:14:51.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.976+0000 7f4f9cee3640 1 -- 172.21.15.3:0/3409723924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f981083a0 msgr2=0x7f4f98105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:51.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.976+0000 7f4f9cee3640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f981083a0 0x7f4f98105620 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c032fd0 tx=0x7f4f8c004270 comp rx=0 tx=0).stop 2024-09-21T06:14:51.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.976+0000 7f4f9cee3640 1 -- 172.21.15.3:0/3409723924 shutdown_connections 2024-09-21T06:14:51.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.976+0000 7f4f9cee3640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f800b65f0 0x7f4f800b8b00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:51.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.976+0000 7f4f9cee3640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f98108cf0 0x7f4f98105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:51.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.976+0000 7f4f9cee3640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f981083a0 0x7f4f98105620 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:51.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.976+0000 7f4f9cee3640 1 --2- 172.21.15.3:0/3409723924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f98102a60 0x7f4f981050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:51.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.976+0000 7f4f9cee3640 1 -- 172.21.15.3:0/3409723924 >> 172.21.15.3:0/3409723924 conn(0x7f4f98079730 msgr2=0x7f4f98100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:51.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.976+0000 7f4f9cee3640 1 -- 172.21.15.3:0/3409723924 shutdown_connections 2024-09-21T06:14:51.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:51.976+0000 7f4f9cee3640 1 -- 172.21.15.3:0/3409723924 wait complete. 2024-09-21T06:14:52.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:52 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3409723924' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:52.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:52 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3409723924' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:52.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:52 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3409723924' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:52.807 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:14:53.064 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:53.454 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:53 smithi003 ceph-mon[31227]: pgmap v321: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.545+0000 7fab30892640 1 -- 172.21.15.3:0/3792393636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2c1083c0 msgr2=0x7fab2c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.545+0000 7fab30892640 1 --2- 172.21.15.3:0/3792393636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2c1083c0 0x7fab2c1087d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fab20009900 tx=0x7fab2002a410 comp rx=0 tx=0).stop 2024-09-21T06:14:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.545+0000 7fab30892640 1 -- 172.21.15.3:0/3792393636 shutdown_connections 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.545+0000 7fab30892640 1 --2- 172.21.15.3:0/3792393636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab2c108d10 0x7fab2c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.545+0000 7fab30892640 1 --2- 172.21.15.3:0/3792393636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2c1083c0 0x7fab2c1087d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.545+0000 7fab30892640 1 --2- 172.21.15.3:0/3792393636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab2c102a80 0x7fab2c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.545+0000 7fab30892640 1 -- 172.21.15.3:0/3792393636 >> 172.21.15.3:0/3792393636 conn(0x7fab2c079740 msgr2=0x7fab2c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.545+0000 7fab30892640 1 -- 172.21.15.3:0/3792393636 shutdown_connections 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.545+0000 7fab30892640 1 -- 172.21.15.3:0/3792393636 wait complete. 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.546+0000 7fab30892640 1 Processor -- start 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.546+0000 7fab30892640 1 -- start start 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.546+0000 7fab30892640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab2c102a80 0x7fab2c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.546+0000 7fab30892640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2c1083c0 0x7fab2c19b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.546+0000 7fab30892640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab2c108d10 0x7fab2c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.546+0000 7fab30892640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1129d0 con 0x7fab2c1083c0 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.546+0000 7fab30892640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c112850 con 0x7fab2c108d10 2024-09-21T06:14:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.546+0000 7fab30892640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c112b50 con 0x7fab2c102a80 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab29d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2c1083c0 0x7fab2c19b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab29d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2c1083c0 0x7fab2c19b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33244/0 (socket says 172.21.15.3:33244) 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab29d9b640 1 -- 172.21.15.3:0/2218157226 learned_addr learned my addr 172.21.15.3:0/2218157226 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab2a59c640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab2c102a80 0x7fab2c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab2ad9d640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab2c108d10 0x7fab2c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab29d9b640 1 -- 172.21.15.3:0/2218157226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab2c102a80 msgr2=0x7fab2c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab29d9b640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab2c102a80 0x7fab2c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab29d9b640 1 -- 172.21.15.3:0/2218157226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab2c108d10 msgr2=0x7fab2c19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab29d9b640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab2c108d10 0x7fab2c19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab29d9b640 1 -- 172.21.15.3:0/2218157226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c1a6650 con 0x7fab2c1083c0 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab29d9b640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2c1083c0 0x7fab2c19b880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fab20031440 tx=0x7fab20003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab1b7fe640 1 -- 172.21.15.3:0/2218157226 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2003f070 con 0x7fab2c1083c0 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab30892640 1 -- 172.21.15.3:0/2218157226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2c1a68e0 con 0x7fab2c1083c0 2024-09-21T06:14:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab1b7fe640 1 -- 172.21.15.3:0/2218157226 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab20031d60 con 0x7fab2c1083c0 2024-09-21T06:14:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab1b7fe640 1 -- 172.21.15.3:0/2218157226 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20035030 con 0x7fab2c1083c0 2024-09-21T06:14:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.547+0000 7fab30892640 1 -- 172.21.15.3:0/2218157226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2c1a6e10 con 0x7fab2c1083c0 2024-09-21T06:14:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.548+0000 7fab30892640 1 -- 172.21.15.3:0/2218157226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2c103d00 con 0x7fab2c1083c0 2024-09-21T06:14:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.551+0000 7fab1b7fe640 1 -- 172.21.15.3:0/2218157226 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7fab20042070 con 0x7fab2c1083c0 2024-09-21T06:14:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.552+0000 7fab1b7fe640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fab140b65f0 0x7fab140b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.552+0000 7fab2a59c640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fab140b65f0 0x7fab140b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.552+0000 7fab1b7fe640 1 -- 172.21.15.3:0/2218157226 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fab20108350 con 0x7fab2c1083c0 2024-09-21T06:14:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.552+0000 7fab2a59c640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fab140b65f0 0x7fab140b8b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab1c005c40 tx=0x7fab1c00a590 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.555+0000 7fab1b7fe640 1 -- 172.21.15.3:0/2218157226 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab200ce940 con 0x7fab2c1083c0 2024-09-21T06:14:53.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:53 smithi137 ceph-mon[34464]: pgmap v321: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:53.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:53 smithi204 ceph-mon[34659]: pgmap v321: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T06:14:53.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.783+0000 7fab30892640 1 -- 172.21.15.3:0/2218157226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab2c102e90 con 0x7fab2c1083c0 2024-09-21T06:14:53.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.784+0000 7fab1b7fe640 1 -- 172.21.15.3:0/2218157226 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11962 (secure 0 0 0) 0x7fab2c102e90 con 0x7fab2c1083c0 2024-09-21T06:14:53.786 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:53.786 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-09-21T06:13:23:600046+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:13:23.600022+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:14:53.786 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-09-21T06:14:53.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.786+0000 7fab30892640 1 -- 172.21.15.3:0/2218157226 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fab140b65f0 msgr2=0x7fab140b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:53.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.786+0000 7fab30892640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fab140b65f0 0x7fab140b8b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab1c005c40 tx=0x7fab1c00a590 comp rx=0 tx=0).stop 2024-09-21T06:14:53.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.786+0000 7fab30892640 1 -- 172.21.15.3:0/2218157226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2c1083c0 msgr2=0x7fab2c19b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:53.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.786+0000 7fab30892640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2c1083c0 0x7fab2c19b880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fab20031440 tx=0x7fab20003d50 comp rx=0 tx=0).stop 2024-09-21T06:14:53.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.786+0000 7fab30892640 1 -- 172.21.15.3:0/2218157226 shutdown_connections 2024-09-21T06:14:53.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.786+0000 7fab30892640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fab140b65f0 0x7fab140b8b00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:53.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.786+0000 7fab30892640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab2c108d10 0x7fab2c19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:53.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.786+0000 7fab30892640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2c1083c0 0x7fab2c19b880 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:53.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.786+0000 7fab30892640 1 --2- 172.21.15.3:0/2218157226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab2c102a80 0x7fab2c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:53.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.786+0000 7fab30892640 1 -- 172.21.15.3:0/2218157226 >> 172.21.15.3:0/2218157226 conn(0x7fab2c079740 msgr2=0x7fab2c104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:53.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.787+0000 7fab30892640 1 -- 172.21.15.3:0/2218157226 shutdown_connections 2024-09-21T06:14:53.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:53.787+0000 7fab30892640 1 -- 172.21.15.3:0/2218157226 wait complete. 2024-09-21T06:14:54.560 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:54 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2218157226' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:54.591 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:14:54.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:54 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2218157226' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:54.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:54 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2218157226' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:54.851 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:55.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.297+0000 7f5a788a3640 1 -- 172.21.15.3:0/417620162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a74105270 msgr2=0x7f5a7410d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.297+0000 7f5a788a3640 1 --2- 172.21.15.3:0/417620162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a74105270 0x7f5a7410d8b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009900 tx=0x7f5a6802a410 comp rx=0 tx=0).stop 2024-09-21T06:14:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.297+0000 7f5a788a3640 1 -- 172.21.15.3:0/417620162 shutdown_connections 2024-09-21T06:14:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.297+0000 7f5a788a3640 1 --2- 172.21.15.3:0/417620162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a7410ddf0 0x7f5a74110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.297+0000 7f5a788a3640 1 --2- 172.21.15.3:0/417620162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a74105270 0x7f5a7410d8b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.297+0000 7f5a788a3640 1 --2- 172.21.15.3:0/417620162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a74104840 0x7f5a74104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.297+0000 7f5a788a3640 1 -- 172.21.15.3:0/417620162 >> 172.21.15.3:0/417620162 conn(0x7f5a74079730 msgr2=0x7f5a74079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.297+0000 7f5a788a3640 1 -- 172.21.15.3:0/417620162 shutdown_connections 2024-09-21T06:14:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.298+0000 7f5a788a3640 1 -- 172.21.15.3:0/417620162 wait complete. 2024-09-21T06:14:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.298+0000 7f5a788a3640 1 Processor -- start 2024-09-21T06:14:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.298+0000 7f5a788a3640 1 -- start start 2024-09-21T06:14:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a788a3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a74104840 0x7f5a7419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a788a3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a74105270 0x7f5a7419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a788a3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a7410ddf0 0x7f5a7419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a788a3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74113a70 con 0x7f5a74104840 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a788a3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741138f0 con 0x7f5a7410ddf0 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a788a3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74113bf0 con 0x7f5a74105270 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a7259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a74104840 0x7f5a7419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a7259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a74104840 0x7f5a7419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51788/0 (socket says 172.21.15.3:51788) 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a7259c640 1 -- 172.21.15.3:0/714094973 learned_addr learned my addr 172.21.15.3:0/714094973 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a71d9b640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a74105270 0x7f5a7419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a72d9d640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a7410ddf0 0x7f5a7419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a7259c640 1 -- 172.21.15.3:0/714094973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a74105270 msgr2=0x7f5a7419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a7259c640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a74105270 0x7f5a7419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a7259c640 1 -- 172.21.15.3:0/714094973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a7410ddf0 msgr2=0x7f5a7419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a7259c640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a7410ddf0 0x7f5a7419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a7259c640 1 -- 172.21.15.3:0/714094973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a741a0470 con 0x7f5a74104840 2024-09-21T06:14:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a7259c640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a74104840 0x7f5a7419a500 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400e9c0 tx=0x7f5a6400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.299+0000 7f5a637fe640 1 -- 172.21.15.3:0/714094973 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6400cd80 con 0x7f5a74104840 2024-09-21T06:14:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.300+0000 7f5a637fe640 1 -- 172.21.15.3:0/714094973 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a640035f0 con 0x7f5a74104840 2024-09-21T06:14:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.300+0000 7f5a788a3640 1 -- 172.21.15.3:0/714094973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a741a0760 con 0x7f5a74104840 2024-09-21T06:14:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.300+0000 7f5a637fe640 1 -- 172.21.15.3:0/714094973 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64010690 con 0x7f5a74104840 2024-09-21T06:14:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.300+0000 7f5a788a3640 1 -- 172.21.15.3:0/714094973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a741a6ec0 con 0x7f5a74104840 2024-09-21T06:14:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.301+0000 7f5a788a3640 1 -- 172.21.15.3:0/714094973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a74104c50 con 0x7f5a74104840 2024-09-21T06:14:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.302+0000 7f5a637fe640 1 -- 172.21.15.3:0/714094973 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 154579+0+0 (secure 0 0 0) 0x7f5a640037a0 con 0x7f5a74104840 2024-09-21T06:14:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.303+0000 7f5a637fe640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5a5c0b6670 0x7f5a5c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.303+0000 7f5a71d9b640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5a5c0b6670 0x7f5a5c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.303+0000 7f5a637fe640 1 -- 172.21.15.3:0/714094973 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5a640e7af0 con 0x7f5a74104840 2024-09-21T06:14:55.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.304+0000 7f5a71d9b640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5a5c0b6670 0x7f5a5c0b8b80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009230 tx=0x7f5a68031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:55.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.307+0000 7f5a637fe640 1 -- 172.21.15.3:0/714094973 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5a64056710 con 0x7f5a74104840 2024-09-21T06:14:55.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.494+0000 7f5a788a3640 1 -- 172.21.15.3:0/714094973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a7419b630 con 0x7f5a74104840 2024-09-21T06:14:55.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.495+0000 7f5a637fe640 1 -- 172.21.15.3:0/714094973 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23048 (secure 0 0 0) 0x7f5a640ae140 con 0x7f5a74104840 2024-09-21T06:14:55.497 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:14:55.497 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","created":"2024-09-21T06:04:24.293786+0000","modified":"2024-09-21T06:13:24.594225+0000","last_up_change":"2024-09-21T06:12:17.781243+0000","last_in_change":"2024-09-21T06:11:54.727435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:08:26.015803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:13:07.084406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:13:09.149891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51736f81-f410-48ee-a8f3-121289607309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6803","nonce":615312328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6805","nonce":615312328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6809","nonce":615312328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6807","nonce":615312328}]},"public_addr":"172.21.15.3:6803/615312328","cluster_addr":"172.21.15.3:6805/615312328","heartbeat_back_addr":"172.21.15.3:6809/615312328","heartbeat_front_addr":"172.21.15.3:6807/615312328","state":["exists","up"]},{"osd":1,"uuid":"55696a37-34c1-482a-8561-7d08f0aa29fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6801","nonce":4150747115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6803","nonce":4150747115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6807","nonce":4150747115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6805","nonce":4150747115}]},"public_addr":"172.21.15.137:6801/4150747115","cluster_addr":"172.21.15.137:6803/4150747115","heartbeat_back_addr":"172.21.15.137:6807/4150747115","heartbeat_front_addr":"172.21.15.137:6805/4150747115","state":["exists","up"]},{"osd":2,"uuid":"8a81d34b-464b-46a2-8ef2-5223b5947004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6801","nonce":3392577077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6803","nonce":3392577077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6807","nonce":3392577077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6805","nonce":3392577077}]},"public_addr":"172.21.15.204:6801/3392577077","cluster_addr":"172.21.15.204:6803/3392577077","heartbeat_back_addr":"172.21.15.204:6807/3392577077","heartbeat_front_addr":"172.21.15.204:6805/3392577077","state":["exists","up"]},{"osd":3,"uuid":"94ab3938-7a70-44cb-99bf-9d2dedc4eb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6811","nonce":4293741635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6813","nonce":4293741635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6817","nonce":4293741635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6815","nonce":4293741635}]},"public_addr":"172.21.15.3:6811/4293741635","cluster_addr":"172.21.15.3:6813/4293741635","heartbeat_back_addr":"172.21.15.3:6817/4293741635","heartbeat_front_addr":"172.21.15.3:6815/4293741635","state":["exists","up"]},{"osd":4,"uuid":"38551535-d692-401c-aad0-bee38f3026d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6809","nonce":2608988785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6811","nonce":2608988785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6815","nonce":2608988785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6813","nonce":2608988785}]},"public_addr":"172.21.15.137:6809/2608988785","cluster_addr":"172.21.15.137:6811/2608988785","heartbeat_back_addr":"172.21.15.137:6815/2608988785","heartbeat_front_addr":"172.21.15.137:6813/2608988785","state":["exists","up"]},{"osd":5,"uuid":"3d4b785b-28d0-4654-a9b1-4c5c905f4bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6809","nonce":3709074044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6811","nonce":3709074044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6815","nonce":3709074044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6813","nonce":3709074044}]},"public_addr":"172.21.15.204:6809/3709074044","cluster_addr":"172.21.15.204:6811/3709074044","heartbeat_back_addr":"172.21.15.204:6815/3709074044","heartbeat_front_addr":"172.21.15.204:6813/3709074044","state":["exists","up"]},{"osd":6,"uuid":"ba21dbe9-ab43-4b7e-9446-2929ee874c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6819","nonce":3862646594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6821","nonce":3862646594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6825","nonce":3862646594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6823","nonce":3862646594}]},"public_addr":"172.21.15.3:6819/3862646594","cluster_addr":"172.21.15.3:6821/3862646594","heartbeat_back_addr":"172.21.15.3:6825/3862646594","heartbeat_front_addr":"172.21.15.3:6823/3862646594","state":["exists","up"]},{"osd":7,"uuid":"9610fe5c-93df-49f9-b8bd-5cb480e0871b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6817","nonce":2086822496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6819","nonce":2086822496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6823","nonce":2086822496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6821","nonce":2086822496}]},"public_addr":"172.21.15.137:6817/2086822496","cluster_addr":"172.21.15.137:6819/2086822496","heartbeat_back_addr":"172.21.15.137:6823/2086822496","heartbeat_front_addr":"172.21.15.137:6821/2086822496","state":["exists","up"]},{"osd":8,"uuid":"49f6e9b4-6d14-434e-8b42-8b7ffeb2a072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6817","nonce":153476798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6819","nonce":153476798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6823","nonce":153476798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6821","nonce":153476798}]},"public_addr":"172.21.15.204:6817/153476798","cluster_addr":"172.21.15.204:6819/153476798","heartbeat_back_addr":"172.21.15.204:6823/153476798","heartbeat_front_addr":"172.21.15.204:6821/153476798","state":["exists","up"]},{"osd":9,"uuid":"633f67a7-b759-4ea6-82ed-3c42b3fb70ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6827","nonce":1238491549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6829","nonce":1238491549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6833","nonce":1238491549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6831","nonce":1238491549}]},"public_addr":"172.21.15.3:6827/1238491549","cluster_addr":"172.21.15.3:6829/1238491549","heartbeat_back_addr":"172.21.15.3:6833/1238491549","heartbeat_front_addr":"172.21.15.3:6831/1238491549","state":["exists","up"]},{"osd":10,"uuid":"94b65a48-ee87-4e59-9296-a72a22787f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6825","nonce":3486552683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6827","nonce":3486552683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6831","nonce":3486552683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6829","nonce":3486552683}]},"public_addr":"172.21.15.137:6825/3486552683","cluster_addr":"172.21.15.137:6827/3486552683","heartbeat_back_addr":"172.21.15.137:6831/3486552683","heartbeat_front_addr":"172.21.15.137:6829/3486552683","state":["exists","up"]},{"osd":11,"uuid":"c5cc2850-9062-4b8e-beb3-bef0db30a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6825","nonce":1084308610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6827","nonce":1084308610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6831","nonce":1084308610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6829","nonce":1084308610}]},"public_addr":"172.21.15.204:6825/1084308610","cluster_addr":"172.21.15.204:6827/1084308610","heartbeat_back_addr":"172.21.15.204:6831/1084308610","heartbeat_front_addr":"172.21.15.204:6829/1084308610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:31.898745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.089965+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:22.367464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:48.543248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:14.687331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:41.243579+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:07.290227+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:32.930046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:59.197638+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:24.315219+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:50.333639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:12:16.453640+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1573374138":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1595702358":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/547897695":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/3673347975":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/141972779":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6800/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:6801/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/2363985440":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/2138752581":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1820734047":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/2929116087":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/45453677":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/2929116087":"2024-09-22T06:05:21.689191+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:14:55.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.497+0000 7f5a788a3640 1 -- 172.21.15.3:0/714094973 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5a5c0b6670 msgr2=0x7f5a5c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.497+0000 7f5a788a3640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5a5c0b6670 0x7f5a5c0b8b80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009230 tx=0x7f5a68031400 comp rx=0 tx=0).stop 2024-09-21T06:14:55.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.497+0000 7f5a788a3640 1 -- 172.21.15.3:0/714094973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a74104840 msgr2=0x7f5a7419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.497+0000 7f5a788a3640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a74104840 0x7f5a7419a500 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400e9c0 tx=0x7f5a6400ee80 comp rx=0 tx=0).stop 2024-09-21T06:14:55.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.497+0000 7f5a788a3640 1 -- 172.21.15.3:0/714094973 shutdown_connections 2024-09-21T06:14:55.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.497+0000 7f5a788a3640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5a5c0b6670 0x7f5a5c0b8b80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.497+0000 7f5a788a3640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a74105270 0x7f5a7419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.497+0000 7f5a788a3640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a7410ddf0 0x7f5a7419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.497+0000 7f5a788a3640 1 --2- 172.21.15.3:0/714094973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a74104840 0x7f5a7419a500 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.497+0000 7f5a788a3640 1 -- 172.21.15.3:0/714094973 >> 172.21.15.3:0/714094973 conn(0x7f5a74079730 msgr2=0x7f5a74100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:55.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.497+0000 7f5a788a3640 1 -- 172.21.15.3:0/714094973 shutdown_connections 2024-09-21T06:14:55.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:55.497+0000 7f5a788a3640 1 -- 172.21.15.3:0/714094973 wait complete. 2024-09-21T06:14:55.501 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:55 smithi003 ceph-mon[31227]: pgmap v322: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:55 smithi137 ceph-mon[34464]: pgmap v322: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:55 smithi204 ceph-mon[34659]: pgmap v322: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:14:56.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T06:13:11.205233+0000', 'modified': '2024-09-21T06:13:23.600022+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24704, 'mds_1': 24691, 'mds_2': 24679, 'mds_3': 24674, 'mds_4': 24670}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14844': {'gid': 14844, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.3:6839/1325295587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6838', 'nonce': 1325295587}, {'type': 'v1', 'addr': '172.21.15.3:6839', 'nonce': 1325295587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14856': {'gid': 14856, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.3:6841/2403205583', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6840', 'nonce': 2403205583}, {'type': 'v1', 'addr': '172.21.15.3:6841', 'nonce': 2403205583}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24620': {'gid': 24620, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6833/4270441897', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6832', 'nonce': 4270441897}, {'type': 'v1', 'addr': '172.21.15.137:6833', 'nonce': 4270441897}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24631': {'gid': 24631, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/2480313554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 2480313554}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 2480313554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24644': {'gid': 24644, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/399369618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 399369618}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 399369618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24670': {'gid': 24670, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.204:6835/879721642', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 879721642}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 879721642}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24674': {'gid': 24674, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.137:6837/44063020', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 44063020}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 44063020}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24679': {'gid': 24679, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.204:6837/3211759973', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 3211759973}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 3211759973}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24691': {'gid': 24691, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.204:6839/2228257050', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 2228257050}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 2228257050}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24704': {'gid': 24704, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.137:6839/2663570031', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 2663570031}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 2663570031}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24704, 'qdb_cluster': [24704, 24691, 24679, 24674, 24670]} 2024-09-21T06:14:56.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T06:14:56.191 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.f status 2024-09-21T06:14:56.455 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:56.577 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:56 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/714094973' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:14:56.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:56 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/714094973' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:14:56.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:56 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/714094973' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:14:56.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.947+0000 7f95754ab640 1 -- 172.21.15.3:0/2951365185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95701083c0 msgr2=0x7f95701087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:56.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.947+0000 7f95754ab640 1 --2- 172.21.15.3:0/2951365185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95701083c0 0x7f95701087d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9564009900 tx=0x7f956402a410 comp rx=0 tx=0).stop 2024-09-21T06:14:56.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.947+0000 7f95754ab640 1 -- 172.21.15.3:0/2951365185 shutdown_connections 2024-09-21T06:14:56.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.947+0000 7f95754ab640 1 --2- 172.21.15.3:0/2951365185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9570108d10 0x7f957010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.947+0000 7f95754ab640 1 --2- 172.21.15.3:0/2951365185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95701083c0 0x7f95701087d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.947+0000 7f95754ab640 1 --2- 172.21.15.3:0/2951365185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9570102a80 0x7f9570102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.947+0000 7f95754ab640 1 -- 172.21.15.3:0/2951365185 >> 172.21.15.3:0/2951365185 conn(0x7f9570079740 msgr2=0x7f9570079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:56.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.947+0000 7f95754ab640 1 -- 172.21.15.3:0/2951365185 shutdown_connections 2024-09-21T06:14:56.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.947+0000 7f95754ab640 1 -- 172.21.15.3:0/2951365185 wait complete. 2024-09-21T06:14:56.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.948+0000 7f95754ab640 1 Processor -- start 2024-09-21T06:14:56.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.948+0000 7f95754ab640 1 -- start start 2024-09-21T06:14:56.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.948+0000 7f95754ab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9570102a80 0x7f957019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.948+0000 7f95754ab640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95701083c0 0x7f957019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.948+0000 7f95754ab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9570108d10 0x7f957019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.948+0000 7f95754ab640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9570113aa0 con 0x7f9570108d10 2024-09-21T06:14:56.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.948+0000 7f95754ab640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9570113920 con 0x7f9570102a80 2024-09-21T06:14:56.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.948+0000 7f95754ab640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9570113c20 con 0x7f95701083c0 2024-09-21T06:14:56.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.948+0000 7f956f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9570108d10 0x7f957019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f956f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9570108d10 0x7f957019fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51794/0 (socket says 172.21.15.3:51794) 2024-09-21T06:14:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f956f7fe640 1 -- 172.21.15.3:0/2037219679 learned_addr learned my addr 172.21.15.3:0/2037219679 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f956e7fc640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95701083c0 0x7f957019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f956effd640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9570102a80 0x7f957019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f956f7fe640 1 -- 172.21.15.3:0/2037219679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95701083c0 msgr2=0x7f957019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f956f7fe640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95701083c0 0x7f957019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f956f7fe640 1 -- 172.21.15.3:0/2037219679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9570102a80 msgr2=0x7f957019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f956f7fe640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9570102a80 0x7f957019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f956f7fe640 1 -- 172.21.15.3:0/2037219679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95701a03e0 con 0x7f9570108d10 2024-09-21T06:14:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f956f7fe640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9570108d10 0x7f957019fcb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f956800eaa0 tx=0x7f956800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f9557fff640 1 -- 172.21.15.3:0/2037219679 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956800cdb0 con 0x7f9570108d10 2024-09-21T06:14:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f9557fff640 1 -- 172.21.15.3:0/2037219679 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9568005020 con 0x7f9570108d10 2024-09-21T06:14:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f95754ab640 1 -- 172.21.15.3:0/2037219679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95701a06d0 con 0x7f9570108d10 2024-09-21T06:14:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f9557fff640 1 -- 172.21.15.3:0/2037219679 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95680116d0 con 0x7f9570108d10 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.949+0000 7f95754ab640 1 -- 172.21.15.3:0/2037219679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95701a6e30 con 0x7f9570108d10 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.952+0000 7f957ba1f740 1 Processor -- start 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.952+0000 7f9557fff640 1 -- 172.21.15.3:0/2037219679 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 154528+0+0 (secure 0 0 0) 0x7f9568011870 con 0x7f9570108d10 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.952+0000 7f957ba1f740 1 -- start start 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f957ba1f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5822f60 0x5654f5bc8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f957ba1f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5bc9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f957ba1f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5802280 0x5654f5bc9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f957ba1f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5654f5b40c10 con 0x5654f57fa0a0 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f957ba1f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5654f5b30040 con 0x5654f5822f60 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f957ba1f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5654f5b33e50 con 0x5654f5802280 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f956effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5bc9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f956effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5bc9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51808/0 (socket says 172.21.15.3:51808) 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f956effd640 1 -- 172.21.15.3:0/2143403517 learned_addr learned my addr 172.21.15.3:0/2143403517 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f956e7fc640 1 --2- 172.21.15.3:0/2143403517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5822f60 0x5654f5bc8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f956e7fc640 1 --2- 172.21.15.3:0/2143403517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5802280 0x5654f5bc9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f956effd640 1 -- 172.21.15.3:0/2143403517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5802280 msgr2=0x5654f5bc9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f956effd640 1 --2- 172.21.15.3:0/2143403517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5802280 0x5654f5bc9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f956effd640 1 -- 172.21.15.3:0/2143403517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5822f60 msgr2=0x5654f5bc8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f956effd640 1 --2- 172.21.15.3:0/2143403517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5822f60 0x5654f5bc8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f956effd640 1 -- 172.21.15.3:0/2143403517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f588e540 con 0x5654f57fa0a0 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f956e7fc640 1 --2- 172.21.15.3:0/2143403517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5802280 0x5654f5bc9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f9557fff640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f95580b6560 0x7f95580b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f956e7fc640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f95580b6560 0x7f95580b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f956effd640 1 --2- 172.21.15.3:0/2143403517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5bc9440 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f956000bd10 tx=0x7f95600107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b54fc8303cbca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.953+0000 7f9557fff640 1 -- 172.21.15.3:0/2037219679 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f95680e7520 con 0x7f9570108d10 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.954+0000 7f95557fa640 1 -- 172.21.15.3:0/2143403517 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560004030 con 0x5654f57fa0a0 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.954+0000 7f95557fa640 1 -- 172.21.15.3:0/2143403517 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95600035f0 con 0x5654f57fa0a0 2024-09-21T06:14:56.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.954+0000 7f95557fa640 1 -- 172.21.15.3:0/2143403517 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95600117b0 con 0x5654f57fa0a0 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.955+0000 7f956e7fc640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f95580b6560 0x7f95580b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9564000c00 tx=0x7f9564005780 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.955+0000 7f957ba1f740 1 -- 172.21.15.3:0/2143403517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 msgr2=0x5654f5bc9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.955+0000 7f957ba1f740 1 --2- 172.21.15.3:0/2143403517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5bc9440 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f956000bd10 tx=0x7f95600107a0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.955+0000 7f957ba1f740 1 -- 172.21.15.3:0/2143403517 shutdown_connections 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.955+0000 7f957ba1f740 1 --2- 172.21.15.3:0/2143403517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5802280 0x5654f5bc9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.955+0000 7f957ba1f740 1 --2- 172.21.15.3:0/2143403517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5bc9440 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.955+0000 7f957ba1f740 1 --2- 172.21.15.3:0/2143403517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5822f60 0x5654f5bc8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.955+0000 7f957ba1f740 1 -- 172.21.15.3:0/2143403517 >> 172.21.15.3:0/2143403517 conn(0x5654f5817350 msgr2=0x5654f5bc6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.955+0000 7f957ba1f740 1 -- 172.21.15.3:0/2143403517 shutdown_connections 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.955+0000 7f957ba1f740 1 -- 172.21.15.3:0/2143403517 wait complete. 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.955+0000 7f957ba1f740 1 -- start start 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.955+0000 7f957ba1f740 1 Processor -- start 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.955+0000 7f957ba1f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5bd4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f957ba1f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5802280 0x5654f5bd4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f957ba1f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5822f60 0x5654f5be0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f957ba1f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5654f59915f0 con 0x5654f57fa0a0 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f957ba1f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5654f5acb010 con 0x5654f5802280 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f957ba1f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5654f5bd76e0 con 0x5654f5822f60 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f957ba1f740 10 client.0 _ll_register_callbacks cb 0x5654f5b2a340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f956effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5bd4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f956effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5bd4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51824/0 (socket says 172.21.15.3:51824) 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f956effd640 1 -- 172.21.15.3:0/1027093912 learned_addr learned my addr 172.21.15.3:0/1027093912 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f956e7fc640 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5802280 0x5654f5bd4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f956f7fe640 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5822f60 0x5654f5be0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f956effd640 1 -- 172.21.15.3:0/1027093912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5822f60 msgr2=0x5654f5be0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f956effd640 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5822f60 0x5654f5be0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f956effd640 1 -- 172.21.15.3:0/1027093912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5802280 msgr2=0x5654f5bd4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f956effd640 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5802280 0x5654f5bd4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f956effd640 1 -- 172.21.15.3:0/1027093912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f5af8a00 con 0x5654f57fa0a0 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f956f7fe640 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5822f60 0x5654f5be0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f956e7fc640 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5802280 0x5654f5bd4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f956effd640 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5bd4a00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9560005920 tx=0x7f956000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d462bb8aea85c186 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f9552ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f9552ff5640 1 -- 172.21.15.3:0/1027093912 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560029060 con 0x5654f57fa0a0 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f9552ff5640 1 -- 172.21.15.3:0/1027093912 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f9560004000 con 0x5654f57fa0a0 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.956+0000 7f957ba1f740 1 -- 172.21.15.3:0/1027093912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95600035f0 con 0x5654f57fa0a0 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.957+0000 7f9552ff5640 1 -- 172.21.15.3:0/1027093912 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9560004200 con 0x5654f57fa0a0 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.957+0000 7f957ba1f740 10 client.15372 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.957+0000 7f957ba1f740 1 -- 172.21.15.3:0/1027093912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654f5be3600 con 0x5654f57fa0a0 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.957+0000 7f9552ff5640 1 -- 172.21.15.3:0/1027093912 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95600035f0 con 0x5654f57fa0a0 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.957+0000 7f9552ff5640 1 -- 172.21.15.3:0/1027093912 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9560023030 con 0x5654f57fa0a0 2024-09-21T06:14:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.957+0000 7f957ba1f740 10 client.15372 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.957+0000 7f957ba1f740 10 client.15372 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.957+0000 7f957ba1f740 20 client.15372 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.957+0000 7f957ba1f740 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5654f57e18e0 0x5654f5be6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.957+0000 7f957ba1f740 4 client.15372 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.957+0000 7f957ba1f740 1 -- 172.21.15.3:0/1027093912 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654f5b42b60 con 0x5654f57e18e0 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.958+0000 7f956e7fc640 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5654f57e18e0 0x5654f5be6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.958+0000 7f9552ff5640 10 client.15372 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.958+0000 7f956e7fc640 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5654f57e18e0 0x5654f5be6c50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5947baa803414ebd server_cookie=c8adbecaeade6604 in_seq=0 out_seq=0 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f9552ff5640 1 -- 172.21.15.3:0/1027093912 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5654f5b42b60 con 0x5654f57e18e0 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f9552ff5640 10 client.15372 handle_command_reply: tid=0 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f957ba1f740 1 client.15372 shutdown 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f957ba1f740 1 -- 172.21.15.3:0/1027093912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 msgr2=0x5654f5bd4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f957ba1f740 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5bd4a00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9560005920 tx=0x7f956000b8e0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f957ba1f740 1 -- 172.21.15.3:0/1027093912 shutdown_connections 2024-09-21T06:14:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f957ba1f740 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5654f57e18e0 0x5654f5be6c50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f957ba1f740 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5822f60 0x5654f5be0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f957ba1f740 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5802280 0x5654f5bd4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f9552ff5640 0 client.15372 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:14:56.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f957ba1f740 1 --2- 172.21.15.3:0/1027093912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5bd4a00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f957ba1f740 1 -- 172.21.15.3:0/1027093912 >> 172.21.15.3:0/1027093912 conn(0x5654f5817350 msgr2=0x5654f5bcc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:56.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f957ba1f740 1 -- 172.21.15.3:0/1027093912 shutdown_connections 2024-09-21T06:14:56.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f957ba1f740 1 -- 172.21.15.3:0/1027093912 wait complete. 2024-09-21T06:14:56.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.959+0000 7f957ba1f740 20 client.15372 trim_cache size 0 max 16384 2024-09-21T06:14:56.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.977+0000 7f957ba1f740 1 Processor -- start 2024-09-21T06:14:56.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.977+0000 7f957ba1f740 1 -- start start 2024-09-21T06:14:56.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f957ba1f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5be0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f957ba1f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5802280 0x5654f5be0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f957ba1f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5822f60 0x5654f5bd3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f957ba1f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5654f5be4f90 con 0x5654f57fa0a0 2024-09-21T06:14:56.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f957ba1f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5654f5be5660 con 0x5654f5822f60 2024-09-21T06:14:56.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f957ba1f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5654f5be5d30 con 0x5654f5802280 2024-09-21T06:14:56.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f956effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5be0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f956effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5be0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51826/0 (socket says 172.21.15.3:51826) 2024-09-21T06:14:56.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f956effd640 1 -- 172.21.15.3:0/2574098520 learned_addr learned my addr 172.21.15.3:0/2574098520 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:56.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f956effd640 1 -- 172.21.15.3:0/2574098520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5802280 msgr2=0x5654f5be0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:56.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f956e7fc640 1 --2- 172.21.15.3:0/2574098520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5802280 0x5654f5be0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f956effd640 1 --2- 172.21.15.3:0/2574098520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5802280 0x5654f5be0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f956effd640 1 -- 172.21.15.3:0/2574098520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5822f60 msgr2=0x5654f5bd3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:56.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f956f7fe640 1 --2- 172.21.15.3:0/2574098520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5822f60 0x5654f5bd3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f956effd640 1 --2- 172.21.15.3:0/2574098520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5822f60 0x5654f5bd3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f956effd640 1 -- 172.21.15.3:0/2574098520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f5bc1080 con 0x5654f57fa0a0 2024-09-21T06:14:56.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f956f7fe640 1 --2- 172.21.15.3:0/2574098520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5822f60 0x5654f5bd3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:56.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.978+0000 7f956effd640 1 --2- 172.21.15.3:0/2574098520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5be0970 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9560011010 tx=0x7f95600107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8c881cb6469cae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:56.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.979+0000 7f9554ff9640 1 -- 172.21.15.3:0/2574098520 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560029070 con 0x5654f57fa0a0 2024-09-21T06:14:56.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.979+0000 7f9554ff9640 1 -- 172.21.15.3:0/2574098520 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9560004850 con 0x5654f57fa0a0 2024-09-21T06:14:56.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.979+0000 7f9554ff9640 1 -- 172.21.15.3:0/2574098520 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560005060 con 0x5654f57fa0a0 2024-09-21T06:14:56.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.979+0000 7f957ba1f740 1 -- 172.21.15.3:0/2574098520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 msgr2=0x5654f5be0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:56.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.979+0000 7f957ba1f740 1 --2- 172.21.15.3:0/2574098520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5be0970 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9560011010 tx=0x7f95600107a0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.979+0000 7f957ba1f740 1 -- 172.21.15.3:0/2574098520 shutdown_connections 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.979+0000 7f957ba1f740 1 --2- 172.21.15.3:0/2574098520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f5802280 0x5654f5be0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.979+0000 7f957ba1f740 1 --2- 172.21.15.3:0/2574098520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5822f60 0x5654f5bd3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.979+0000 7f957ba1f740 1 --2- 172.21.15.3:0/2574098520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f57fa0a0 0x5654f5be0970 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.979+0000 7f957ba1f740 1 -- 172.21.15.3:0/2574098520 >> 172.21.15.3:0/2574098520 conn(0x5654f5817350 msgr2=0x5654f5bd49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f957ba1f740 1 -- 172.21.15.3:0/2574098520 shutdown_connections 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f957ba1f740 1 -- 172.21.15.3:0/2574098520 wait complete. 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f957ba1f740 1 -- start start 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f957ba1f740 1 Processor -- start 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f957ba1f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f57fa0a0 0x5654f5be91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f957ba1f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5802280 0x5654f5be9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f957ba1f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f5822f60 0x5654f5bee920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f957ba1f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5654f5be6ad0 con 0x5654f5822f60 2024-09-21T06:14:56.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f957ba1f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5654f5be6400 con 0x5654f5802280 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f957ba1f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5654f5be71a0 con 0x5654f57fa0a0 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f957ba1f740 10 client.0 _ll_register_callbacks cb 0x7f9564005e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f956f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f5822f60 0x5654f5bee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f956f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f5822f60 0x5654f5bee920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51842/0 (socket says 172.21.15.3:51842) 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f956f7fe640 1 -- 172.21.15.3:0/3931060927 learned_addr learned my addr 172.21.15.3:0/3931060927 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.980+0000 7f956e7fc640 1 --2- 172.21.15.3:0/3931060927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5802280 0x5654f5be9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f956effd640 1 --2- 172.21.15.3:0/3931060927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f57fa0a0 0x5654f5be91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f956f7fe640 1 -- 172.21.15.3:0/3931060927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f57fa0a0 msgr2=0x5654f5be91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f956f7fe640 1 --2- 172.21.15.3:0/3931060927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f57fa0a0 0x5654f5be91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f956f7fe640 1 -- 172.21.15.3:0/3931060927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5802280 msgr2=0x5654f5be9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f956f7fe640 1 --2- 172.21.15.3:0/3931060927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5802280 0x5654f5be9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f956f7fe640 1 -- 172.21.15.3:0/3931060927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f5be9f70 con 0x5654f5822f60 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f956f7fe640 1 --2- 172.21.15.3:0/3931060927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f5822f60 0x5654f5bee920 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f95680aa8f0 tx=0x7f9568005450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a3f445b1683d5d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:56.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f9552ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f9552ff5640 1 -- 172.21.15.3:0/3931060927 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956801f070 con 0x5654f5822f60 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f9552ff5640 1 -- 172.21.15.3:0/3931060927 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f9568011b60 con 0x5654f5822f60 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f957ba1f740 1 -- 172.21.15.3:0/3931060927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9560004850 con 0x5654f5822f60 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f9552ff5640 1 -- 172.21.15.3:0/3931060927 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f95680b20b0 con 0x5654f5822f60 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f957ba1f740 10 client.15378 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.981+0000 7f957ba1f740 1 -- 172.21.15.3:0/3931060927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654f5c525d0 con 0x5654f5822f60 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.982+0000 7f9552ff5640 1 -- 172.21.15.3:0/3931060927 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9560004850 con 0x5654f5822f60 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.982+0000 7f9552ff5640 1 -- 172.21.15.3:0/3931060927 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f956800f040 con 0x5654f5822f60 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.982+0000 7f957ba1f740 10 client.15378 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.982+0000 7f957ba1f740 10 client.15378 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.982+0000 7f957ba1f740 20 client.15378 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.982+0000 7f957ba1f740 1 --2- 172.21.15.3:0/3931060927 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5654f57b6c90 0x5654f5c55f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.982+0000 7f957ba1f740 4 client.15378 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:14:56.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.982+0000 7f957ba1f740 1 -- 172.21.15.3:0/3931060927 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "status"}) -- 0x5654f5b3e610 con 0x5654f57b6c90 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.982+0000 7f956effd640 1 --2- 172.21.15.3:0/3931060927 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5654f57b6c90 0x5654f5c55f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f9552ff5640 10 client.15378 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f956effd640 1 --2- 172.21.15.3:0/3931060927 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5654f57b6c90 0x5654f5c55f10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1db57b21f4785df4 server_cookie=95ccc4cd15e82f75 in_seq=0 out_seq=0 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f9552ff5640 1 -- 172.21.15.3:0/3931060927 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5654f5b3e610 con 0x5654f57b6c90 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f9552ff5640 10 client.15378 handle_command_reply: tid=0 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f957ba1f740 1 client.15378 shutdown 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f957ba1f740 1 -- 172.21.15.3:0/3931060927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f5822f60 msgr2=0x5654f5bee920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f957ba1f740 1 --2- 172.21.15.3:0/3931060927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f5822f60 0x5654f5bee920 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f95680aa8f0 tx=0x7f9568005450 comp rx=0 tx=0).stop 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f957ba1f740 1 -- 172.21.15.3:0/3931060927 shutdown_connections 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f957ba1f740 1 --2- 172.21.15.3:0/3931060927 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5654f57b6c90 0x5654f5c55f10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f957ba1f740 1 --2- 172.21.15.3:0/3931060927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654f5822f60 0x5654f5bee920 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f957ba1f740 1 --2- 172.21.15.3:0/3931060927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654f5802280 0x5654f5be9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f957ba1f740 1 --2- 172.21.15.3:0/3931060927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654f57fa0a0 0x5654f5be91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f957ba1f740 1 -- 172.21.15.3:0/3931060927 >> 172.21.15.3:0/3931060927 conn(0x5654f5817350 msgr2=0x5654f5bc6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:56.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f9552ff5640 0 client.15378 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:14:56.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f957ba1f740 1 -- 172.21.15.3:0/3931060927 shutdown_connections 2024-09-21T06:14:56.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f957ba1f740 1 -- 172.21.15.3:0/3931060927 wait complete. 2024-09-21T06:14:56.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.983+0000 7f957ba1f740 20 client.15378 trim_cache size 0 max 16384 2024-09-21T06:14:56.990 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:14:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "38d3c75c-77df-11ef-baf5-efdc52797490", 2024-09-21T06:14:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 4, 2024-09-21T06:14:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24670, 2024-09-21T06:14:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-21T06:14:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-21T06:14:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-21T06:14:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 94.396097104999996, 2024-09-21T06:14:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2024-09-21T06:14:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 79, 2024-09-21T06:14:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T06:14:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 133.82646632699999 2024-09-21T06:14:56.991 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:14:56.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.984+0000 7f95754ab640 1 -- 172.21.15.3:0/2037219679 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f95580b6560 msgr2=0x7f95580b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:56.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.984+0000 7f95754ab640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f95580b6560 0x7f95580b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9564000c00 tx=0x7f9564005780 comp rx=0 tx=0).stop 2024-09-21T06:14:56.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.984+0000 7f95754ab640 1 -- 172.21.15.3:0/2037219679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9570108d10 msgr2=0x7f957019fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:56.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.984+0000 7f95754ab640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9570108d10 0x7f957019fcb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f956800eaa0 tx=0x7f956800ef60 comp rx=0 tx=0).stop 2024-09-21T06:14:56.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.984+0000 7f95754ab640 1 -- 172.21.15.3:0/2037219679 shutdown_connections 2024-09-21T06:14:56.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.984+0000 7f95754ab640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f95580b6560 0x7f95580b8a70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.984+0000 7f95754ab640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95701083c0 0x7f957019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.984+0000 7f95754ab640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9570108d10 0x7f957019fcb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.984+0000 7f95754ab640 1 --2- 172.21.15.3:0/2037219679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9570102a80 0x7f957019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:56.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.984+0000 7f95754ab640 1 -- 172.21.15.3:0/2037219679 >> 172.21.15.3:0/2037219679 conn(0x7f9570079740 msgr2=0x7f9570104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:56.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.984+0000 7f95754ab640 1 -- 172.21.15.3:0/2037219679 shutdown_connections 2024-09-21T06:14:56.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:56.984+0000 7f95754ab640 1 -- 172.21.15.3:0/2037219679 wait complete. 2024-09-21T06:14:57.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:57 smithi003 ceph-mon[31227]: pgmap v323: 97 pgs: 97 active+clean; 649 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:14:57.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:57 smithi003 ceph-mon[31227]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T06:14:57.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:57 smithi137 ceph-mon[34464]: pgmap v323: 97 pgs: 97 active+clean; 649 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:14:57.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:57 smithi137 ceph-mon[34464]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T06:14:57.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:57 smithi204 ceph-mon[34659]: pgmap v323: 97 pgs: 97 active+clean; 649 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:14:57.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:57 smithi204 ceph-mon[34659]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T06:14:57.758 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T06:14:58.016 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.517+0000 7fe7ae9b3640 1 -- 172.21.15.3:0/2918062793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a81083d0 msgr2=0x7fe7a81087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.517+0000 7fe7ae9b3640 1 --2- 172.21.15.3:0/2918062793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a81083d0 0x7fe7a81087e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe79c009900 tx=0x7fe79c02a410 comp rx=0 tx=0).stop 2024-09-21T06:14:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.517+0000 7fe7ae9b3640 1 -- 172.21.15.3:0/2918062793 shutdown_connections 2024-09-21T06:14:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.517+0000 7fe7ae9b3640 1 --2- 172.21.15.3:0/2918062793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a8108d20 0x7fe7a810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.517+0000 7fe7ae9b3640 1 --2- 172.21.15.3:0/2918062793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a81083d0 0x7fe7a81087e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.517+0000 7fe7ae9b3640 1 --2- 172.21.15.3:0/2918062793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7a8102ae0 0x7fe7a8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.517+0000 7fe7ae9b3640 1 -- 172.21.15.3:0/2918062793 >> 172.21.15.3:0/2918062793 conn(0x7fe7a8079760 msgr2=0x7fe7a8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.517+0000 7fe7ae9b3640 1 -- 172.21.15.3:0/2918062793 shutdown_connections 2024-09-21T06:14:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.517+0000 7fe7ae9b3640 1 -- 172.21.15.3:0/2918062793 wait complete. 2024-09-21T06:14:58.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.517+0000 7fe7ae9b3640 1 Processor -- start 2024-09-21T06:14:58.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7ae9b3640 1 -- start start 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7ae9b3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7a8102ae0 0x7fe7a819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7ae9b3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a81083d0 0x7fe7a819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7ae9b3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a8108d20 0x7fe7a819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7ae9b3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a8112990 con 0x7fe7a81083d0 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7ae9b3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a8112810 con 0x7fe7a8108d20 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7ae9b3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a8112b10 con 0x7fe7a8102ae0 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7a77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a81083d0 0x7fe7a819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7a77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a81083d0 0x7fe7a819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51858/0 (socket says 172.21.15.3:51858) 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7a77fe640 1 -- 172.21.15.3:0/1863333468 learned_addr learned my addr 172.21.15.3:0/1863333468 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7a7fff640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7a8102ae0 0x7fe7a819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7acf50640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a8108d20 0x7fe7a819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7a77fe640 1 -- 172.21.15.3:0/1863333468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7a8102ae0 msgr2=0x7fe7a819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7a77fe640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7a8102ae0 0x7fe7a819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7a77fe640 1 -- 172.21.15.3:0/1863333468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a8108d20 msgr2=0x7fe7a819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7a77fe640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a8108d20 0x7fe7a819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7a77fe640 1 -- 172.21.15.3:0/1863333468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a81a0380 con 0x7fe7a81083d0 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7a7fff640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7a8102ae0 0x7fe7a819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.518+0000 7fe7acf50640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a8108d20 0x7fe7a819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.519+0000 7fe7a77fe640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a81083d0 0x7fe7a819aa70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe79c032fd0 tx=0x7fe79c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.519+0000 7fe7a57fa640 1 -- 172.21.15.3:0/1863333468 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c03f030 con 0x7fe7a81083d0 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.519+0000 7fe7a57fa640 1 -- 172.21.15.3:0/1863333468 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe79c004490 con 0x7fe7a81083d0 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.519+0000 7fe7a57fa640 1 -- 172.21.15.3:0/1863333468 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c035070 con 0x7fe7a81083d0 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.519+0000 7fe7ae9b3640 1 -- 172.21.15.3:0/1863333468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a81a0610 con 0x7fe7a81083d0 2024-09-21T06:14:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.519+0000 7fe7ae9b3640 1 -- 172.21.15.3:0/1863333468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a81a0b40 con 0x7fe7a81083d0 2024-09-21T06:14:58.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.522+0000 7fe7b4f27740 1 Processor -- start 2024-09-21T06:14:58.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.522+0000 7fe7b4f27740 1 -- start start 2024-09-21T06:14:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7b4f27740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 0x55aaf73f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7b4f27740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 0x55aaf73f9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7b4f27740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 0x55aaf73f9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7b4f27740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55aaf7370c10 con 0x55aaf7052f60 2024-09-21T06:14:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7b4f27740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aaf7360040 con 0x55aaf702a0a0 2024-09-21T06:14:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7b4f27740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aaf7363e50 con 0x55aaf7032280 2024-09-21T06:14:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7acf50640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 0x55aaf73f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7acf50640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 0x55aaf73f8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51874/0 (socket says 172.21.15.3:51874) 2024-09-21T06:14:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7acf50640 1 -- 172.21.15.3:0/2445970737 learned_addr learned my addr 172.21.15.3:0/2445970737 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7a57fa640 1 -- 172.21.15.3:0/1863333468 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 154528+0+0 (secure 0 0 0) 0x7fe79c042030 con 0x7fe7a81083d0 2024-09-21T06:14:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7a7fff640 1 --2- 172.21.15.3:0/2445970737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 0x55aaf73f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7acf50640 1 -- 172.21.15.3:0/2445970737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 msgr2=0x55aaf73f9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7acf50640 1 --2- 172.21.15.3:0/2445970737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 0x55aaf73f9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7acf50640 1 -- 172.21.15.3:0/2445970737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 msgr2=0x55aaf73f9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7acf50640 1 --2- 172.21.15.3:0/2445970737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 0x55aaf73f9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7acf50640 1 -- 172.21.15.3:0/2445970737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf70be540 con 0x55aaf7052f60 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.523+0000 7fe7acf50640 1 --2- 172.21.15.3:0/2445970737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 0x55aaf73f8f00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009ed0 tx=0x7fe7a000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbe2961fe27783d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.524+0000 7fe78e7fc640 1 -- 172.21.15.3:0/2445970737 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0004d20 con 0x55aaf7052f60 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.524+0000 7fe78e7fc640 1 -- 172.21.15.3:0/2445970737 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a00054d0 con 0x55aaf7052f60 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.524+0000 7fe78e7fc640 1 -- 172.21.15.3:0/2445970737 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0011800 con 0x55aaf7052f60 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.524+0000 7fe7a57fa640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe7900b6610 0x7fe7900b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.524+0000 7fe7a7fff640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe7900b6610 0x7fe7900b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.524+0000 7fe7a57fa640 1 -- 172.21.15.3:0/1863333468 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe79c107940 con 0x7fe7a81083d0 2024-09-21T06:14:58.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.525+0000 7fe7b4f27740 1 -- 172.21.15.3:0/2445970737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 msgr2=0x55aaf73f8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:58.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.525+0000 7fe7a7fff640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe7900b6610 0x7fe7900b8b20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe798005fd0 tx=0x7fe79800b470 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:58.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.525+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/2445970737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 0x55aaf73f8f00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009ed0 tx=0x7fe7a000f730 comp rx=0 tx=0).stop 2024-09-21T06:14:58.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.525+0000 7fe7b4f27740 1 -- 172.21.15.3:0/2445970737 shutdown_connections 2024-09-21T06:14:58.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.525+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/2445970737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 0x55aaf73f9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.525+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/2445970737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 0x55aaf73f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.525+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/2445970737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 0x55aaf73f8f00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.525+0000 7fe7b4f27740 1 -- 172.21.15.3:0/2445970737 >> 172.21.15.3:0/2445970737 conn(0x55aaf7047350 msgr2=0x55aaf73f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.525+0000 7fe7b4f27740 1 -- 172.21.15.3:0/2445970737 shutdown_connections 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.525+0000 7fe7b4f27740 1 -- 172.21.15.3:0/2445970737 wait complete. 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.525+0000 7fe7b4f27740 1 -- start start 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.525+0000 7fe7b4f27740 1 Processor -- start 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7b4f27740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf702a0a0 0x55aaf7408e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7b4f27740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 0x55aaf74093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7b4f27740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf7052f60 0x55aaf7414d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7b4f27740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55aaf71c15f0 con 0x55aaf702a0a0 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7b4f27740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aaf72fb010 con 0x55aaf7052f60 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7b4f27740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aaf740ba10 con 0x55aaf7032280 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7b4f27740 10 client.0 _ll_register_callbacks cb 0x55aaf735a340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf702a0a0 0x55aaf7408e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf702a0a0 0x55aaf7408e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51890/0 (socket says 172.21.15.3:51890) 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7a7fff640 1 -- 172.21.15.3:0/1119420146 learned_addr learned my addr 172.21.15.3:0/1119420146 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7a77fe640 1 --2- 172.21.15.3:0/1119420146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 0x55aaf74093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7acf50640 1 --2- 172.21.15.3:0/1119420146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf7052f60 0x55aaf7414d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7a7fff640 1 -- 172.21.15.3:0/1119420146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 msgr2=0x55aaf74093b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7a7fff640 1 --2- 172.21.15.3:0/1119420146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 0x55aaf74093b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7a7fff640 1 -- 172.21.15.3:0/1119420146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf7052f60 msgr2=0x55aaf7414d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7a7fff640 1 --2- 172.21.15.3:0/1119420146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf7052f60 0x55aaf7414d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7a7fff640 1 -- 172.21.15.3:0/1119420146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf7328a00 con 0x55aaf702a0a0 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7a7fff640 1 --2- 172.21.15.3:0/1119420146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf702a0a0 0x55aaf7408e70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe79800b700 tx=0x7fe7980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35a983a3418b593 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.526+0000 7fe7877fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7877fe640 1 -- 172.21.15.3:0/1119420146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79800ec90 con 0x55aaf702a0a0 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7877fe640 1 -- 172.21.15.3:0/1119420146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe79800ee30 con 0x55aaf702a0a0 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7b4f27740 1 -- 172.21.15.3:0/1119420146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7a00054d0 con 0x55aaf702a0a0 2024-09-21T06:14:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7877fe640 1 -- 172.21.15.3:0/1119420146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe798005020 con 0x55aaf702a0a0 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7b4f27740 10 client.15390 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7b4f27740 1 -- 172.21.15.3:0/1119420146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aaf7417a40 con 0x55aaf702a0a0 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7877fe640 1 -- 172.21.15.3:0/1119420146 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a00054d0 con 0x55aaf702a0a0 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7877fe640 1 -- 172.21.15.3:0/1119420146 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe798019050 con 0x55aaf702a0a0 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7b4f27740 10 client.15390 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7b4f27740 10 client.15390 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7b4f27740 20 client.15390 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/1119420146 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55aaf70118e0 0x55aaf741b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7b4f27740 4 client.15390 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.527+0000 7fe7b4f27740 1 -- 172.21.15.3:0/1119420146 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaf7372b60 con 0x55aaf70118e0 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.528+0000 7fe7a77fe640 1 --2- 172.21.15.3:0/1119420146 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55aaf70118e0 0x55aaf741b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.528+0000 7fe7877fe640 10 client.15390 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:14:58.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.528+0000 7fe7a77fe640 1 --2- 172.21.15.3:0/1119420146 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55aaf70118e0 0x55aaf741b480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1f57bd69754a1ecc server_cookie=cf61272d2924bf37 in_seq=0 out_seq=0 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7877fe640 1 -- 172.21.15.3:0/1119420146 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aaf7372b60 con 0x55aaf70118e0 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7877fe640 10 client.15390 handle_command_reply: tid=0 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7b4f27740 1 client.15390 shutdown 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7b4f27740 1 -- 172.21.15.3:0/1119420146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf702a0a0 msgr2=0x55aaf7408e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/1119420146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf702a0a0 0x55aaf7408e70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe79800b700 tx=0x7fe7980046c0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7b4f27740 1 -- 172.21.15.3:0/1119420146 shutdown_connections 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/1119420146 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55aaf70118e0 0x55aaf741b480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/1119420146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 0x55aaf74093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/1119420146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf7052f60 0x55aaf7414d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7877fe640 0 client.15390 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/1119420146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf702a0a0 0x55aaf7408e70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7b4f27740 1 -- 172.21.15.3:0/1119420146 >> 172.21.15.3:0/1119420146 conn(0x55aaf7047350 msgr2=0x55aaf73fc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7b4f27740 1 -- 172.21.15.3:0/1119420146 shutdown_connections 2024-09-21T06:14:58.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7b4f27740 1 -- 172.21.15.3:0/1119420146 wait complete. 2024-09-21T06:14:58.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.529+0000 7fe7b4f27740 20 client.15390 trim_cache size 0 max 16384 2024-09-21T06:14:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7b4f27740 1 Processor -- start 2024-09-21T06:14:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7b4f27740 1 -- start start 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7b4f27740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 0x55aaf7403e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7b4f27740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7032280 0x55aaf7404360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7b4f27740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7052f60 0x55aaf7414ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7b4f27740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55aaf74197c0 con 0x55aaf7032280 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7b4f27740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aaf7419e90 con 0x55aaf702a0a0 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7b4f27740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aaf741a560 con 0x55aaf7052f60 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7a77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7032280 0x55aaf7404360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7a77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7032280 0x55aaf7404360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51892/0 (socket says 172.21.15.3:51892) 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7a77fe640 1 -- 172.21.15.3:0/3915131491 learned_addr learned my addr 172.21.15.3:0/3915131491 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7a7fff640 1 --2- 172.21.15.3:0/3915131491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 0x55aaf7403e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7acf50640 1 --2- 172.21.15.3:0/3915131491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7052f60 0x55aaf7414ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7a77fe640 1 -- 172.21.15.3:0/3915131491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7052f60 msgr2=0x55aaf7414ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7a77fe640 1 --2- 172.21.15.3:0/3915131491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7052f60 0x55aaf7414ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7a77fe640 1 -- 172.21.15.3:0/3915131491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 msgr2=0x55aaf7403e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7a77fe640 1 --2- 172.21.15.3:0/3915131491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 0x55aaf7403e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7a77fe640 1 -- 172.21.15.3:0/3915131491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf73f7a90 con 0x55aaf7032280 2024-09-21T06:14:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7a7fff640 1 --2- 172.21.15.3:0/3915131491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 0x55aaf7403e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7acf50640 1 --2- 172.21.15.3:0/3915131491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7052f60 0x55aaf7414ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.548+0000 7fe7a77fe640 1 --2- 172.21.15.3:0/3915131491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7032280 0x55aaf7404360 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0cac60 tx=0x7fe79c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83311dea0e83b5c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.549+0000 7fe78dffb640 1 -- 172.21.15.3:0/3915131491 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c049040 con 0x55aaf7032280 2024-09-21T06:14:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.549+0000 7fe78dffb640 1 -- 172.21.15.3:0/3915131491 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe79c0ce0f0 con 0x55aaf7032280 2024-09-21T06:14:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.549+0000 7fe78dffb640 1 -- 172.21.15.3:0/3915131491 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c0d20b0 con 0x55aaf7032280 2024-09-21T06:14:58.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.549+0000 7fe7b4f27740 1 -- 172.21.15.3:0/3915131491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7032280 msgr2=0x55aaf7404360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:58.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.549+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/3915131491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7032280 0x55aaf7404360 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0cac60 tx=0x7fe79c040000 comp rx=0 tx=0).stop 2024-09-21T06:14:58.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.549+0000 7fe7b4f27740 1 -- 172.21.15.3:0/3915131491 shutdown_connections 2024-09-21T06:14:58.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.549+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/3915131491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7052f60 0x55aaf7414ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.549+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/3915131491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7032280 0x55aaf7404360 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.549+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/3915131491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 0x55aaf7403e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.549+0000 7fe7b4f27740 1 -- 172.21.15.3:0/3915131491 >> 172.21.15.3:0/3915131491 conn(0x55aaf7047350 msgr2=0x55aaf73f1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.550+0000 7fe7b4f27740 1 -- 172.21.15.3:0/3915131491 shutdown_connections 2024-09-21T06:14:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.550+0000 7fe7b4f27740 1 -- 172.21.15.3:0/3915131491 wait complete. 2024-09-21T06:14:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.550+0000 7fe7b4f27740 1 -- start start 2024-09-21T06:14:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.550+0000 7fe7b4f27740 1 Processor -- start 2024-09-21T06:14:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.550+0000 7fe7b4f27740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 0x55aaf7423c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.550+0000 7fe7b4f27740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 0x55aaf741d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.550+0000 7fe7b4f27740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 0x55aaf741d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.550+0000 7fe7b4f27740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55aaf741b300 con 0x55aaf7052f60 2024-09-21T06:14:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7b4f27740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aaf741ac30 con 0x55aaf702a0a0 2024-09-21T06:14:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7b4f27740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aaf741b9d0 con 0x55aaf7032280 2024-09-21T06:14:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7b4f27740 10 client.0 _ll_register_callbacks cb 0x7fe79800ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7acf50640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 0x55aaf741d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7acf50640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 0x55aaf741d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51896/0 (socket says 172.21.15.3:51896) 2024-09-21T06:14:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 0x55aaf7423c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7acf50640 1 -- 172.21.15.3:0/3587488565 learned_addr learned my addr 172.21.15.3:0/3587488565 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:14:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7a77fe640 1 --2- 172.21.15.3:0/3587488565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 0x55aaf741d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7acf50640 1 -- 172.21.15.3:0/3587488565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 msgr2=0x55aaf741d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7acf50640 1 --2- 172.21.15.3:0/3587488565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 0x55aaf741d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7acf50640 1 -- 172.21.15.3:0/3587488565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 msgr2=0x55aaf7423c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7acf50640 1 --2- 172.21.15.3:0/3587488565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 0x55aaf7423c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7acf50640 1 -- 172.21.15.3:0/3587488565 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf741db70 con 0x55aaf7052f60 2024-09-21T06:14:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7acf50640 1 --2- 172.21.15.3:0/3587488565 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 0x55aaf741d630 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009890 tx=0x7fe7a0004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5502e8d18ffbdedb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7877fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:14:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7877fe640 1 -- 172.21.15.3:0/3587488565 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0029060 con 0x55aaf7052f60 2024-09-21T06:14:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7877fe640 1 -- 172.21.15.3:0/3587488565 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe7a0005200 con 0x55aaf7052f60 2024-09-21T06:14:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.551+0000 7fe7b4f27740 1 -- 172.21.15.3:0/3587488565 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe79c0ce0f0 con 0x55aaf7052f60 2024-09-21T06:14:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.552+0000 7fe7877fe640 1 -- 172.21.15.3:0/3587488565 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7a0005440 con 0x55aaf7052f60 2024-09-21T06:14:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.552+0000 7fe7b4f27740 10 client.15396 fetch_fsmap learned FSMap version 27 2024-09-21T06:14:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.552+0000 7fe7b4f27740 1 -- 172.21.15.3:0/3587488565 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aaf7423490 con 0x55aaf7052f60 2024-09-21T06:14:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.552+0000 7fe7877fe640 1 -- 172.21.15.3:0/3587488565 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe79c0ce0f0 con 0x55aaf7052f60 2024-09-21T06:14:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.552+0000 7fe7877fe640 1 -- 172.21.15.3:0/3587488565 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe7a0019430 con 0x55aaf7052f60 2024-09-21T06:14:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.552+0000 7fe7b4f27740 10 client.15396 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:14:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.552+0000 7fe7b4f27740 10 client.15396 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T06:14:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.552+0000 7fe7b4f27740 20 client.15396 populate_metadata read hostname 'smithi003' 2024-09-21T06:14:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.552+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/3587488565 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55aaf6fe6c90 0x55aaf7489bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.552+0000 7fe7b4f27740 4 client.15396 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:14:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.552+0000 7fe7b4f27740 1 -- 172.21.15.3:0/3587488565 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "status"}) -- 0x55aaf736e610 con 0x55aaf6fe6c90 2024-09-21T06:14:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.552+0000 7fe7a7fff640 1 --2- 172.21.15.3:0/3587488565 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55aaf6fe6c90 0x55aaf7489bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7a7fff640 1 --2- 172.21.15.3:0/3587488565 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55aaf6fe6c90 0x55aaf7489bf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4be78cdfee06ad35 server_cookie=57725d3cccc68aee in_seq=0 out_seq=0 2024-09-21T06:14:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7877fe640 10 client.15396 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:14:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7877fe640 1 -- 172.21.15.3:0/3587488565 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55aaf736e610 con 0x55aaf6fe6c90 2024-09-21T06:14:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7877fe640 10 client.15396 handle_command_reply: tid=0 2024-09-21T06:14:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7b4f27740 1 client.15396 shutdown 2024-09-21T06:14:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7b4f27740 1 -- 172.21.15.3:0/3587488565 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 msgr2=0x55aaf741d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/3587488565 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 0x55aaf741d630 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009890 tx=0x7fe7a0004bf0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7b4f27740 1 -- 172.21.15.3:0/3587488565 shutdown_connections 2024-09-21T06:14:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/3587488565 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55aaf6fe6c90 0x55aaf7489bf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/3587488565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaf7032280 0x55aaf741d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/3587488565 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aaf7052f60 0x55aaf741d630 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7b4f27740 1 --2- 172.21.15.3:0/3587488565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaf702a0a0 0x55aaf7423c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7b4f27740 1 -- 172.21.15.3:0/3587488565 >> 172.21.15.3:0/3587488565 conn(0x55aaf7047350 msgr2=0x55aaf73f1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7877fe640 0 client.15396 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:14:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7b4f27740 1 -- 172.21.15.3:0/3587488565 shutdown_connections 2024-09-21T06:14:58.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7b4f27740 1 -- 172.21.15.3:0/3587488565 wait complete. 2024-09-21T06:14:58.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.553+0000 7fe7b4f27740 20 client.15396 trim_cache size 0 max 16384 2024-09-21T06:14:58.562 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:14:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "38d3c75c-77df-11ef-baf5-efdc52797490", 2024-09-21T06:14:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 3, 2024-09-21T06:14:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24674, 2024-09-21T06:14:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-21T06:14:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-21T06:14:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-21T06:14:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 96.982153936000003, 2024-09-21T06:14:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2024-09-21T06:14:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 79, 2024-09-21T06:14:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T06:14:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 130.11632967899999 2024-09-21T06:14:58.563 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:14:58.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.554+0000 7fe7ae9b3640 1 -- 172.21.15.3:0/1863333468 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe7900b6610 msgr2=0x7fe7900b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:58.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.554+0000 7fe7ae9b3640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe7900b6610 0x7fe7900b8b20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe798005fd0 tx=0x7fe79800b470 comp rx=0 tx=0).stop 2024-09-21T06:14:58.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.554+0000 7fe7ae9b3640 1 -- 172.21.15.3:0/1863333468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a81083d0 msgr2=0x7fe7a819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:58.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.554+0000 7fe7ae9b3640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a81083d0 0x7fe7a819aa70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe79c032fd0 tx=0x7fe79c004270 comp rx=0 tx=0).stop 2024-09-21T06:14:58.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.554+0000 7fe7ae9b3640 1 -- 172.21.15.3:0/1863333468 shutdown_connections 2024-09-21T06:14:58.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.554+0000 7fe7ae9b3640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe7900b6610 0x7fe7900b8b20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.554+0000 7fe7ae9b3640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a8108d20 0x7fe7a819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.554+0000 7fe7ae9b3640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a81083d0 0x7fe7a819aa70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.554+0000 7fe7ae9b3640 1 --2- 172.21.15.3:0/1863333468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7a8102ae0 0x7fe7a819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:58.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.554+0000 7fe7ae9b3640 1 -- 172.21.15.3:0/1863333468 >> 172.21.15.3:0/1863333468 conn(0x7fe7a8079760 msgr2=0x7fe7a8101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:58.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.554+0000 7fe7ae9b3640 1 -- 172.21.15.3:0/1863333468 shutdown_connections 2024-09-21T06:14:58.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:14:58.554+0000 7fe7ae9b3640 1 -- 172.21.15.3:0/1863333468 wait complete. 2024-09-21T06:14:59.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:14:59 smithi003 ceph-mon[31227]: pgmap v324: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T06:14:59.427 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T06:14:59.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:14:59 smithi137 ceph-mon[34464]: pgmap v324: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T06:14:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:14:59 smithi204 ceph-mon[34659]: pgmap v324: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T06:14:59.685 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:00.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.193+0000 7f0939a26640 1 -- 172.21.15.3:0/4247858452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09341094f0 msgr2=0x7f0934109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.193+0000 7f0939a26640 1 --2- 172.21.15.3:0/4247858452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09341094f0 0x7f0934109900 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0928009900 tx=0x7f092802a410 comp rx=0 tx=0).stop 2024-09-21T06:15:00.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.194+0000 7f0939a26640 1 -- 172.21.15.3:0/4247858452 shutdown_connections 2024-09-21T06:15:00.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.194+0000 7f0939a26640 1 --2- 172.21.15.3:0/4247858452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0934109e40 0x7f0934110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.194+0000 7f0939a26640 1 --2- 172.21.15.3:0/4247858452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09341094f0 0x7f0934109900 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.194+0000 7f0939a26640 1 --2- 172.21.15.3:0/4247858452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0934103bb0 0x7f0934103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.194+0000 7f0939a26640 1 -- 172.21.15.3:0/4247858452 >> 172.21.15.3:0/4247858452 conn(0x7f09340ff600 msgr2=0x7f0934101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:00.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.194+0000 7f0939a26640 1 -- 172.21.15.3:0/4247858452 shutdown_connections 2024-09-21T06:15:00.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.194+0000 7f0939a26640 1 -- 172.21.15.3:0/4247858452 wait complete. 2024-09-21T06:15:00.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.194+0000 7f0939a26640 1 Processor -- start 2024-09-21T06:15:00.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f0939a26640 1 -- start start 2024-09-21T06:15:00.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f0939a26640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0934103bb0 0x7f093419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f0939a26640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09341094f0 0x7f093419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f0939a26640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0934109e40 0x7f093419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f0939a26640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0934113b00 con 0x7f0934109e40 2024-09-21T06:15:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f0939a26640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0934113980 con 0x7f0934103bb0 2024-09-21T06:15:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f0939a26640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0934113c80 con 0x7f09341094f0 2024-09-21T06:15:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f09337fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0934109e40 0x7f093419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f09337fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0934109e40 0x7f093419fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51910/0 (socket says 172.21.15.3:51910) 2024-09-21T06:15:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f09337fe640 1 -- 172.21.15.3:0/4189088929 learned_addr learned my addr 172.21.15.3:0/4189088929 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f09327fc640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09341094f0 0x7f093419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f0932ffd640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0934103bb0 0x7f093419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f09337fe640 1 -- 172.21.15.3:0/4189088929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09341094f0 msgr2=0x7f093419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f09337fe640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09341094f0 0x7f093419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f09337fe640 1 -- 172.21.15.3:0/4189088929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0934103bb0 msgr2=0x7f093419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f09337fe640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0934103bb0 0x7f093419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.195+0000 7f09337fe640 1 -- 172.21.15.3:0/4189088929 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09341a03a0 con 0x7f0934109e40 2024-09-21T06:15:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.196+0000 7f09337fe640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0934109e40 0x7f093419fd30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f092c00cce0 tx=0x7f092c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.196+0000 7f0938a24640 1 -- 172.21.15.3:0/4189088929 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c00f450 con 0x7f0934109e40 2024-09-21T06:15:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.196+0000 7f0938a24640 1 -- 172.21.15.3:0/4189088929 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f092c00faa0 con 0x7f0934109e40 2024-09-21T06:15:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.196+0000 7f0939a26640 1 -- 172.21.15.3:0/4189088929 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09341a0690 con 0x7f0934109e40 2024-09-21T06:15:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.196+0000 7f0938a24640 1 -- 172.21.15.3:0/4189088929 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c018b70 con 0x7f0934109e40 2024-09-21T06:15:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.196+0000 7f0939a26640 1 -- 172.21.15.3:0/4189088929 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09341a6eb0 con 0x7f0934109e40 2024-09-21T06:15:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.198+0000 7f093ff9a740 1 Processor -- start 2024-09-21T06:15:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.199+0000 7f0938a24640 1 -- 172.21.15.3:0/4189088929 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 154528+0+0 (secure 0 0 0) 0x7f092c018d10 con 0x7f0934109e40 2024-09-21T06:15:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.199+0000 7f093ff9a740 1 -- start start 2024-09-21T06:15:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.199+0000 7f093ff9a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575e8ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.199+0000 7f093ff9a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575e90440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.199+0000 7f093ff9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 0x55b575e90980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.199+0000 7f093ff9a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b575e07c10 con 0x55b575ae9f60 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.199+0000 7f093ff9a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b575df7040 con 0x55b575ac10a0 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.199+0000 7f093ff9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b575dfae50 con 0x55b575ac9280 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.199+0000 7f09327fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575e8ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.199+0000 7f09327fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575e8ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51922/0 (socket says 172.21.15.3:51922) 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.199+0000 7f09327fc640 1 -- 172.21.15.3:0/2028249702 learned_addr learned my addr 172.21.15.3:0/2028249702 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f0932ffd640 1 --2- 172.21.15.3:0/2028249702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575e90440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f09327fc640 1 --2- 172.21.15.3:0/2028249702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 0x55b575e90980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f09327fc640 1 -- 172.21.15.3:0/2028249702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 msgr2=0x55b575e90980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f09327fc640 1 --2- 172.21.15.3:0/2028249702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 0x55b575e90980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f09327fc640 1 -- 172.21.15.3:0/2028249702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 msgr2=0x55b575e90440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f09327fc640 1 --2- 172.21.15.3:0/2028249702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575e90440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f09327fc640 1 -- 172.21.15.3:0/2028249702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b575b55540 con 0x55b575ae9f60 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f0932ffd640 1 --2- 172.21.15.3:0/2028249702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575e90440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f0938a24640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f091c0b6610 0x7f091c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f09327fc640 1 --2- 172.21.15.3:0/2028249702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575e8ff00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0928009900 tx=0x7f0928000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8b6c6402c7401c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:00.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f0932ffd640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f091c0b6610 0x7f091c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f0938a24640 1 -- 172.21.15.3:0/4189088929 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f092c0e7750 con 0x7f0934109e40 2024-09-21T06:15:00.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f0919ffb640 1 -- 172.21.15.3:0/2028249702 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928035040 con 0x55b575ae9f60 2024-09-21T06:15:00.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f0919ffb640 1 -- 172.21.15.3:0/2028249702 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0928042070 con 0x55b575ae9f60 2024-09-21T06:15:00.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.200+0000 7f0919ffb640 1 -- 172.21.15.3:0/2028249702 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09280055c0 con 0x55b575ae9f60 2024-09-21T06:15:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.201+0000 7f093ff9a740 1 -- 172.21.15.3:0/2028249702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 msgr2=0x55b575e8ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.201+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2028249702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575e8ff00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0928009900 tx=0x7f0928000bc0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.201+0000 7f0932ffd640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f091c0b6610 0x7f091c0b8b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55b575e8eb80 tx=0x7f092400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.201+0000 7f093ff9a740 1 -- 172.21.15.3:0/2028249702 shutdown_connections 2024-09-21T06:15:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.201+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2028249702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 0x55b575e90980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.201+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2028249702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575e90440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.201+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2028249702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575e8ff00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.201+0000 7f093ff9a740 1 -- 172.21.15.3:0/2028249702 >> 172.21.15.3:0/2028249702 conn(0x55b575ade350 msgr2=0x55b575e8ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.201+0000 7f093ff9a740 1 -- 172.21.15.3:0/2028249702 shutdown_connections 2024-09-21T06:15:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.201+0000 7f093ff9a740 1 -- 172.21.15.3:0/2028249702 wait complete. 2024-09-21T06:15:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.202+0000 7f093ff9a740 1 -- start start 2024-09-21T06:15:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.202+0000 7f093ff9a740 1 Processor -- start 2024-09-21T06:15:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.202+0000 7f093ff9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac10a0 0x55b575e9fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.202+0000 7f093ff9a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac9280 0x55b575ea0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.202+0000 7f093ff9a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575eabd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.202+0000 7f093ff9a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b575c585f0 con 0x55b575ae9f60 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.202+0000 7f093ff9a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b575d92010 con 0x55b575ac9280 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.202+0000 7f093ff9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b575ea29e0 con 0x55b575ac10a0 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.202+0000 7f093ff9a740 10 client.0 _ll_register_callbacks cb 0x55b575df1340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.202+0000 7f09337fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575eabd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.202+0000 7f09337fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575eabd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51928/0 (socket says 172.21.15.3:51928) 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.202+0000 7f09337fe640 1 -- 172.21.15.3:0/2834661345 learned_addr learned my addr 172.21.15.3:0/2834661345 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.202+0000 7f09327fc640 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac9280 0x55b575ea0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f0932ffd640 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac10a0 0x55b575e9fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f09337fe640 1 -- 172.21.15.3:0/2834661345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac10a0 msgr2=0x55b575e9fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f09337fe640 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac10a0 0x55b575e9fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f09337fe640 1 -- 172.21.15.3:0/2834661345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac9280 msgr2=0x55b575ea0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f09337fe640 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac9280 0x55b575ea0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f09337fe640 1 -- 172.21.15.3:0/2834661345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b575dbfa00 con 0x55b575ae9f60 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f09327fc640 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac9280 0x55b575ea0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f09337fe640 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575eabd40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f092c0ad500 tx=0x7f092c0ad210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2930158d839649 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f0912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f0912ffd640 1 -- 172.21.15.3:0/2834661345 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c0b2070 con 0x55b575ae9f60 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f0912ffd640 1 -- 172.21.15.3:0/2834661345 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f092c0aeb30 con 0x55b575ae9f60 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f093ff9a740 1 -- 172.21.15.3:0/2834661345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0928042070 con 0x55b575ae9f60 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f0912ffd640 1 -- 172.21.15.3:0/2834661345 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f092c0adf00 con 0x55b575ae9f60 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f093ff9a740 10 client.15408 fetch_fsmap learned FSMap version 27 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f093ff9a740 1 -- 172.21.15.3:0/2834661345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b575eae940 con 0x55b575ae9f60 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.203+0000 7f0912ffd640 1 -- 172.21.15.3:0/2834661345 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0928042070 con 0x55b575ae9f60 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.204+0000 7f0912ffd640 1 -- 172.21.15.3:0/2834661345 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f092c0a91e0 con 0x55b575ae9f60 2024-09-21T06:15:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.204+0000 7f093ff9a740 10 client.15408 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.204+0000 7f093ff9a740 10 client.15408 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.204+0000 7f093ff9a740 20 client.15408 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.204+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b575aa88e0 0x55b575eb2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.204+0000 7f093ff9a740 4 client.15408 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.204+0000 7f093ff9a740 1 -- 172.21.15.3:0/2834661345 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b575e09b60 con 0x55b575aa88e0 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.204+0000 7f0932ffd640 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b575aa88e0 0x55b575eb2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.205+0000 7f0912ffd640 10 client.15408 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.205+0000 7f0932ffd640 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b575aa88e0 0x55b575eb2340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c55d8d8bec7ce968 server_cookie=bac3fde04795dcc in_seq=0 out_seq=0 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.205+0000 7f0912ffd640 1 -- 172.21.15.3:0/2834661345 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b575e09b60 con 0x55b575aa88e0 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.205+0000 7f0912ffd640 10 client.15408 handle_command_reply: tid=0 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.205+0000 7f093ff9a740 1 client.15408 shutdown 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.206+0000 7f093ff9a740 1 -- 172.21.15.3:0/2834661345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 msgr2=0x55b575eabd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.206+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575eabd40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f092c0ad500 tx=0x7f092c0ad210 comp rx=0 tx=0).stop 2024-09-21T06:15:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.206+0000 7f093ff9a740 1 -- 172.21.15.3:0/2834661345 shutdown_connections 2024-09-21T06:15:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.206+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b575aa88e0 0x55b575eb2340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.206+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575eabd40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.206+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac9280 0x55b575ea0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.206+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2834661345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac10a0 0x55b575e9fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.206+0000 7f093ff9a740 1 -- 172.21.15.3:0/2834661345 >> 172.21.15.3:0/2834661345 conn(0x55b575ade350 msgr2=0x55b575e8c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.206+0000 7f0912ffd640 0 client.15408 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:15:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.206+0000 7f093ff9a740 1 -- 172.21.15.3:0/2834661345 shutdown_connections 2024-09-21T06:15:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.206+0000 7f093ff9a740 1 -- 172.21.15.3:0/2834661345 wait complete. 2024-09-21T06:15:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.206+0000 7f093ff9a740 20 client.15408 trim_cache size 0 max 16384 2024-09-21T06:15:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.224+0000 7f093ff9a740 1 Processor -- start 2024-09-21T06:15:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.224+0000 7f093ff9a740 1 -- start start 2024-09-21T06:15:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.224+0000 7f093ff9a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575eabcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.224+0000 7f093ff9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 0x55b575eac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.224+0000 7f093ff9a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575e9ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.224+0000 7f093ff9a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b575eb0680 con 0x55b575ae9f60 2024-09-21T06:15:00.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.224+0000 7f093ff9a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b575eb0d50 con 0x55b575ac10a0 2024-09-21T06:15:00.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.224+0000 7f093ff9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b575eb1420 con 0x55b575ac9280 2024-09-21T06:15:00.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.224+0000 7f09337fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575e9ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.224+0000 7f09337fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575e9ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51932/0 (socket says 172.21.15.3:51932) 2024-09-21T06:15:00.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.224+0000 7f09337fe640 1 -- 172.21.15.3:0/3464522461 learned_addr learned my addr 172.21.15.3:0/3464522461 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:00.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.225+0000 7f0932ffd640 1 --2- 172.21.15.3:0/3464522461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575eabcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.225+0000 7f09327fc640 1 --2- 172.21.15.3:0/3464522461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 0x55b575eac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.225+0000 7f09337fe640 1 -- 172.21.15.3:0/3464522461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 msgr2=0x55b575eac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.225+0000 7f09337fe640 1 --2- 172.21.15.3:0/3464522461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 0x55b575eac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.225+0000 7f09337fe640 1 -- 172.21.15.3:0/3464522461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 msgr2=0x55b575eabcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.225+0000 7f09337fe640 1 --2- 172.21.15.3:0/3464522461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575eabcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.225+0000 7f09337fe640 1 -- 172.21.15.3:0/3464522461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b575ea04c0 con 0x55b575ae9f60 2024-09-21T06:15:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.225+0000 7f09327fc640 1 --2- 172.21.15.3:0/3464522461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 0x55b575eac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.225+0000 7f0932ffd640 1 --2- 172.21.15.3:0/3464522461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575eabcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.225+0000 7f09337fe640 1 --2- 172.21.15.3:0/3464522461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575e9ad40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f092c0e51c0 tx=0x7f092c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1b0cc7cd5e84de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.225+0000 7f09197fa640 1 -- 172.21.15.3:0/3464522461 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c0b2040 con 0x55b575ae9f60 2024-09-21T06:15:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.225+0000 7f09197fa640 1 -- 172.21.15.3:0/3464522461 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f092c0d2020 con 0x55b575ae9f60 2024-09-21T06:15:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.225+0000 7f09197fa640 1 -- 172.21.15.3:0/3464522461 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c0aca60 con 0x55b575ae9f60 2024-09-21T06:15:00.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.226+0000 7f093ff9a740 1 -- 172.21.15.3:0/3464522461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 msgr2=0x55b575e9ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.226+0000 7f093ff9a740 1 --2- 172.21.15.3:0/3464522461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575e9ad40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f092c0e51c0 tx=0x7f092c012000 comp rx=0 tx=0).stop 2024-09-21T06:15:00.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.226+0000 7f093ff9a740 1 -- 172.21.15.3:0/3464522461 shutdown_connections 2024-09-21T06:15:00.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.226+0000 7f093ff9a740 1 --2- 172.21.15.3:0/3464522461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 0x55b575eac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.226+0000 7f093ff9a740 1 --2- 172.21.15.3:0/3464522461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575e9ad40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.226+0000 7f093ff9a740 1 --2- 172.21.15.3:0/3464522461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575eabcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.226+0000 7f093ff9a740 1 -- 172.21.15.3:0/3464522461 >> 172.21.15.3:0/3464522461 conn(0x55b575ade350 msgr2=0x55b575e88ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:00.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.226+0000 7f093ff9a740 1 -- 172.21.15.3:0/3464522461 shutdown_connections 2024-09-21T06:15:00.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.226+0000 7f093ff9a740 1 -- 172.21.15.3:0/3464522461 wait complete. 2024-09-21T06:15:00.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.226+0000 7f093ff9a740 1 -- start start 2024-09-21T06:15:00.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f093ff9a740 1 Processor -- start 2024-09-21T06:15:00.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f093ff9a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575eba950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f093ff9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 0x55b575ebaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f093ff9a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575eb3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f093ff9a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b575eb21c0 con 0x55b575ae9f60 2024-09-21T06:15:00.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f093ff9a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b575eb1af0 con 0x55b575ac10a0 2024-09-21T06:15:00.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f093ff9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b575eb2890 con 0x55b575ac9280 2024-09-21T06:15:00.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f093ff9a740 10 client.0 _ll_register_callbacks cb 0x7f092c00af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:00.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f09337fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575eb3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f09337fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575eb3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51938/0 (socket says 172.21.15.3:51938) 2024-09-21T06:15:00.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f09337fe640 1 -- 172.21.15.3:0/2204866998 learned_addr learned my addr 172.21.15.3:0/2204866998 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f09327fc640 1 --2- 172.21.15.3:0/2204866998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 0x55b575ebaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f0932ffd640 1 --2- 172.21.15.3:0/2204866998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575eba950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f09337fe640 1 -- 172.21.15.3:0/2204866998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 msgr2=0x55b575ebaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f09337fe640 1 --2- 172.21.15.3:0/2204866998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 0x55b575ebaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f09337fe640 1 -- 172.21.15.3:0/2204866998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 msgr2=0x55b575eba950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f09337fe640 1 --2- 172.21.15.3:0/2204866998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575eba950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f09337fe640 1 -- 172.21.15.3:0/2204866998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b575eb4740 con 0x55b575ae9f60 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f09337fe640 1 --2- 172.21.15.3:0/2204866998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575eb3fb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f092c0a9d80 tx=0x7f092c0a96a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7bb1bb7db826cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.227+0000 7f0912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f0912ffd640 1 -- 172.21.15.3:0/2204866998 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c0b2040 con 0x55b575ae9f60 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f0912ffd640 1 -- 172.21.15.3:0/2204866998 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f092c0acc70 con 0x55b575ae9f60 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f093ff9a740 1 -- 172.21.15.3:0/2204866998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f092c0d2020 con 0x55b575ae9f60 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f0912ffd640 1 -- 172.21.15.3:0/2204866998 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f092c0cf320 con 0x55b575ae9f60 2024-09-21T06:15:00.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f093ff9a740 10 client.15414 fetch_fsmap learned FSMap version 27 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f093ff9a740 1 -- 172.21.15.3:0/2204866998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b575eb9fc0 con 0x55b575ae9f60 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f0912ffd640 1 -- 172.21.15.3:0/2204866998 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f092c0d2020 con 0x55b575ae9f60 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f0912ffd640 1 -- 172.21.15.3:0/2204866998 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f092c004a40 con 0x55b575ae9f60 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f093ff9a740 10 client.15414 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f093ff9a740 10 client.15414 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f093ff9a740 20 client.15414 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2204866998 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b575a7dc90 0x55b575f206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f093ff9a740 4 client.15414 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.228+0000 7f093ff9a740 1 -- 172.21.15.3:0/2204866998 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "status"}) -- 0x55b575e05610 con 0x55b575a7dc90 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.229+0000 7f0932ffd640 1 --2- 172.21.15.3:0/2204866998 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b575a7dc90 0x55b575f206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.229+0000 7f0932ffd640 1 --2- 172.21.15.3:0/2204866998 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b575a7dc90 0x55b575f206f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f32cc5ae341f48c server_cookie=40cb1185a4abaa26 in_seq=0 out_seq=0 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.229+0000 7f0912ffd640 10 client.15414 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.229+0000 7f0912ffd640 1 -- 172.21.15.3:0/2204866998 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b575e05610 con 0x55b575a7dc90 2024-09-21T06:15:00.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.229+0000 7f0912ffd640 10 client.15414 handle_command_reply: tid=0 2024-09-21T06:15:00.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.229+0000 7f093ff9a740 1 client.15414 shutdown 2024-09-21T06:15:00.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.229+0000 7f093ff9a740 1 -- 172.21.15.3:0/2204866998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 msgr2=0x55b575eb3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.229+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2204866998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575eb3fb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f092c0a9d80 tx=0x7f092c0a96a0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.229+0000 7f093ff9a740 1 -- 172.21.15.3:0/2204866998 shutdown_connections 2024-09-21T06:15:00.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.229+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2204866998 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b575a7dc90 0x55b575f206f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2204866998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b575ac9280 0x55b575ebaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2204866998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b575ae9f60 0x55b575eb3fb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f093ff9a740 1 --2- 172.21.15.3:0/2204866998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b575ac10a0 0x55b575eba950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f093ff9a740 1 -- 172.21.15.3:0/2204866998 >> 172.21.15.3:0/2204866998 conn(0x55b575ade350 msgr2=0x55b575e88ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:00.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f0912ffd640 0 client.15414 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:15:00.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f093ff9a740 1 -- 172.21.15.3:0/2204866998 shutdown_connections 2024-09-21T06:15:00.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f093ff9a740 1 -- 172.21.15.3:0/2204866998 wait complete. 2024-09-21T06:15:00.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f093ff9a740 20 client.15414 trim_cache size 0 max 16384 2024-09-21T06:15:00.238 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:15:00.238 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "38d3c75c-77df-11ef-baf5-efdc52797490", 2024-09-21T06:15:00.238 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-09-21T06:15:00.238 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24679, 2024-09-21T06:15:00.238 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-21T06:15:00.238 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-21T06:15:00.238 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-21T06:15:00.238 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 99.665617862999994, 2024-09-21T06:15:00.238 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2024-09-21T06:15:00.238 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 79, 2024-09-21T06:15:00.238 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T06:15:00.239 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 134.50835305499999 2024-09-21T06:15:00.239 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:15:00.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f0939a26640 1 -- 172.21.15.3:0/4189088929 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f091c0b6610 msgr2=0x7f091c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f0939a26640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f091c0b6610 0x7f091c0b8b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55b575e8eb80 tx=0x7f092400b390 comp rx=0 tx=0).stop 2024-09-21T06:15:00.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f0939a26640 1 -- 172.21.15.3:0/4189088929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0934109e40 msgr2=0x7f093419fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f0939a26640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0934109e40 0x7f093419fd30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f092c00cce0 tx=0x7f092c00a620 comp rx=0 tx=0).stop 2024-09-21T06:15:00.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f0939a26640 1 -- 172.21.15.3:0/4189088929 shutdown_connections 2024-09-21T06:15:00.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f0939a26640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f091c0b6610 0x7f091c0b8b20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f0939a26640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09341094f0 0x7f093419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f0939a26640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0934109e40 0x7f093419fd30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f0939a26640 1 --2- 172.21.15.3:0/4189088929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0934103bb0 0x7f093419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f0939a26640 1 -- 172.21.15.3:0/4189088929 >> 172.21.15.3:0/4189088929 conn(0x7f09340ff600 msgr2=0x7f0934101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:00.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.230+0000 7f0939a26640 1 -- 172.21.15.3:0/4189088929 shutdown_connections 2024-09-21T06:15:00.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:00.231+0000 7f0939a26640 1 -- 172.21.15.3:0/4189088929 wait complete. 2024-09-21T06:15:01.115 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T06:15:01.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:01 smithi003 ceph-mon[31227]: pgmap v325: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:15:01.373 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:01 smithi137 ceph-mon[34464]: pgmap v325: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:15:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:01 smithi204 ceph-mon[34659]: pgmap v325: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:15:01.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.886+0000 7fdfe13b4640 1 -- 172.21.15.3:0/1655702793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfdc108cf0 msgr2=0x7fdfdc10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:01.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.886+0000 7fdfe13b4640 1 --2- 172.21.15.3:0/1655702793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfdc108cf0 0x7fdfdc10f520 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009ca0 tx=0x7fdfcc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.886+0000 7fdfe13b4640 1 -- 172.21.15.3:0/1655702793 shutdown_connections 2024-09-21T06:15:01.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.886+0000 7fdfe13b4640 1 --2- 172.21.15.3:0/1655702793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfdc108cf0 0x7fdfdc10f520 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.886+0000 7fdfe13b4640 1 --2- 172.21.15.3:0/1655702793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfdc1083a0 0x7fdfdc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.886+0000 7fdfe13b4640 1 --2- 172.21.15.3:0/1655702793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfdc102a60 0x7fdfdc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.886+0000 7fdfe13b4640 1 -- 172.21.15.3:0/1655702793 >> 172.21.15.3:0/1655702793 conn(0x7fdfdc079730 msgr2=0x7fdfdc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.886+0000 7fdfe13b4640 1 -- 172.21.15.3:0/1655702793 shutdown_connections 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.886+0000 7fdfe13b4640 1 -- 172.21.15.3:0/1655702793 wait complete. 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.887+0000 7fdfe13b4640 1 Processor -- start 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.887+0000 7fdfe13b4640 1 -- start start 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.887+0000 7fdfe13b4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfdc102a60 0x7fdfdc199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.887+0000 7fdfe13b4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfdc1083a0 0x7fdfdc1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.887+0000 7fdfe13b4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfdc108cf0 0x7fdfdc19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.887+0000 7fdfe13b4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc112960 con 0x7fdfdc1083a0 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.887+0000 7fdfe13b4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1127e0 con 0x7fdfdc102a60 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.887+0000 7fdfe13b4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc112ae0 con 0x7fdfdc108cf0 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfda7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfdc1083a0 0x7fdfdc1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfda7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfdc1083a0 0x7fdfdc1998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51956/0 (socket says 172.21.15.3:51956) 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfda7fc640 1 -- 172.21.15.3:0/2356144844 learned_addr learned my addr 172.21.15.3:0/2356144844 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfdaffd640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfdc102a60 0x7fdfdc199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfdb7fe640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfdc108cf0 0x7fdfdc19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfda7fc640 1 -- 172.21.15.3:0/2356144844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfdc108cf0 msgr2=0x7fdfdc19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfda7fc640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfdc108cf0 0x7fdfdc19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfda7fc640 1 -- 172.21.15.3:0/2356144844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfdc102a60 msgr2=0x7fdfdc199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfda7fc640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfdc102a60 0x7fdfdc199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfda7fc640 1 -- 172.21.15.3:0/2356144844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfdc19f1c0 con 0x7fdfdc1083a0 2024-09-21T06:15:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfda7fc640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfdc1083a0 0x7fdfdc1998b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4005fd0 tx=0x7fdfc400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfd3fff640 1 -- 172.21.15.3:0/2356144844 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4017070 con 0x7fdfdc1083a0 2024-09-21T06:15:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfd3fff640 1 -- 172.21.15.3:0/2356144844 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc400de50 con 0x7fdfdc1083a0 2024-09-21T06:15:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfe13b4640 1 -- 172.21.15.3:0/2356144844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfdc19f4b0 con 0x7fdfdc1083a0 2024-09-21T06:15:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfd3fff640 1 -- 172.21.15.3:0/2356144844 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4003c20 con 0x7fdfdc1083a0 2024-09-21T06:15:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.888+0000 7fdfe13b4640 1 -- 172.21.15.3:0/2356144844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfdc1a5cd0 con 0x7fdfdc1083a0 2024-09-21T06:15:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.891+0000 7fdfe7928740 1 Processor -- start 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.891+0000 7fdfe7928740 1 -- start start 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.891+0000 7fdfd3fff640 1 -- 172.21.15.3:0/2356144844 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 154528+0+0 (secure 0 0 0) 0x7fdfc4003dc0 con 0x7fdfdc1083a0 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfe7928740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 0x56249d9e3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfe7928740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d6150a0 0x56249d9e4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfe7928740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d61d280 0x56249d9e4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfe7928740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56249d95bc10 con 0x56249d63df60 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfe7928740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56249d94b040 con 0x56249d61d280 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfe7928740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56249d94ee50 con 0x56249d6150a0 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfdb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 0x56249d9e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfdb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 0x56249d9e3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51962/0 (socket says 172.21.15.3:51962) 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfdb7fe640 1 -- 172.21.15.3:0/2617020230 learned_addr learned my addr 172.21.15.3:0/2617020230 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfdaffd640 1 --2- 172.21.15.3:0/2617020230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d6150a0 0x56249d9e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfda7fc640 1 --2- 172.21.15.3:0/2617020230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d61d280 0x56249d9e4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfdb7fe640 1 -- 172.21.15.3:0/2617020230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d6150a0 msgr2=0x56249d9e4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfdb7fe640 1 --2- 172.21.15.3:0/2617020230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d6150a0 0x56249d9e4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfdb7fe640 1 -- 172.21.15.3:0/2617020230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d61d280 msgr2=0x56249d9e4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfdb7fe640 1 --2- 172.21.15.3:0/2617020230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d61d280 0x56249d9e4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfdb7fe640 1 -- 172.21.15.3:0/2617020230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56249d6a9540 con 0x56249d63df60 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.892+0000 7fdfda7fc640 1 --2- 172.21.15.3:0/2617020230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d61d280 0x56249d9e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.893+0000 7fdfdb7fe640 1 --2- 172.21.15.3:0/2617020230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 0x56249d9e3f00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009920 tx=0x7fdfcc009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01e7ee703c4ac36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.893+0000 7fdfd3fff640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdfc80b6590 0x7fdfc80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.893+0000 7fdfd17fa640 1 -- 172.21.15.3:0/2617020230 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc031e30 con 0x56249d63df60 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.893+0000 7fdfdaffd640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdfc80b6590 0x7fdfc80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.893+0000 7fdfd17fa640 1 -- 172.21.15.3:0/2617020230 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc036030 con 0x56249d63df60 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.893+0000 7fdfd17fa640 1 -- 172.21.15.3:0/2617020230 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0042a0 con 0x56249d63df60 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.893+0000 7fdfd3fff640 1 -- 172.21.15.3:0/2356144844 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fdfc40e6fb0 con 0x7fdfdc1083a0 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.894+0000 7fdfe7928740 1 -- 172.21.15.3:0/2617020230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 msgr2=0x56249d9e3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.894+0000 7fdfe7928740 1 --2- 172.21.15.3:0/2617020230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 0x56249d9e3f00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009920 tx=0x7fdfcc009950 comp rx=0 tx=0).stop 2024-09-21T06:15:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.894+0000 7fdfdaffd640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdfc80b6590 0x7fdfc80b8aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400ad30 tx=0x7fdfd400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.894+0000 7fdfe7928740 1 -- 172.21.15.3:0/2617020230 shutdown_connections 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.894+0000 7fdfe7928740 1 --2- 172.21.15.3:0/2617020230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d6150a0 0x56249d9e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.894+0000 7fdfe7928740 1 --2- 172.21.15.3:0/2617020230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d61d280 0x56249d9e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.894+0000 7fdfe7928740 1 --2- 172.21.15.3:0/2617020230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 0x56249d9e3f00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.894+0000 7fdfe7928740 1 -- 172.21.15.3:0/2617020230 >> 172.21.15.3:0/2617020230 conn(0x56249d632350 msgr2=0x56249d9e1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.894+0000 7fdfe7928740 1 -- 172.21.15.3:0/2617020230 shutdown_connections 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.894+0000 7fdfe7928740 1 -- 172.21.15.3:0/2617020230 wait complete. 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.894+0000 7fdfe7928740 1 -- start start 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfe7928740 1 Processor -- start 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfe7928740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d6150a0 0x56249d9efd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfe7928740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d61d280 0x56249d9f0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfe7928740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 0x56249d9f9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfe7928740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56249d7ac5f0 con 0x56249d63df60 2024-09-21T06:15:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfe7928740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56249d8e6010 con 0x56249d6150a0 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfe7928740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56249d9ea350 con 0x56249d61d280 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfe7928740 10 client.0 _ll_register_callbacks cb 0x56249d945340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfdb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 0x56249d9f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfdb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 0x56249d9f9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51974/0 (socket says 172.21.15.3:51974) 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfdb7fe640 1 -- 172.21.15.3:0/1746449661 learned_addr learned my addr 172.21.15.3:0/1746449661 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfda7fc640 1 --2- 172.21.15.3:0/1746449661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d61d280 0x56249d9f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfdaffd640 1 --2- 172.21.15.3:0/1746449661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d6150a0 0x56249d9efd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfdb7fe640 1 -- 172.21.15.3:0/1746449661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d61d280 msgr2=0x56249d9f0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.895+0000 7fdfdb7fe640 1 --2- 172.21.15.3:0/1746449661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d61d280 0x56249d9f0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.896+0000 7fdfdb7fe640 1 -- 172.21.15.3:0/1746449661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d6150a0 msgr2=0x56249d9efd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.896+0000 7fdfdb7fe640 1 --2- 172.21.15.3:0/1746449661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d6150a0 0x56249d9efd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.896+0000 7fdfdb7fe640 1 -- 172.21.15.3:0/1746449661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56249d913a00 con 0x56249d63df60 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.896+0000 7fdfdb7fe640 1 --2- 172.21.15.3:0/1746449661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 0x56249d9f9960 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc005e30 tx=0x7fdfcc031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7048e5b942a20f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.896+0000 7fdfc27fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.896+0000 7fdfc27fc640 1 -- 172.21.15.3:0/1746449661 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc04a040 con 0x56249d63df60 2024-09-21T06:15:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.896+0000 7fdfe7928740 1 -- 172.21.15.3:0/1746449661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfcc036030 con 0x56249d63df60 2024-09-21T06:15:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.896+0000 7fdfc27fc640 1 -- 172.21.15.3:0/1746449661 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fdfcc004200 con 0x56249d63df60 2024-09-21T06:15:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.896+0000 7fdfc27fc640 1 -- 172.21.15.3:0/1746449661 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdfcc031920 con 0x56249d63df60 2024-09-21T06:15:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.896+0000 7fdfe7928740 10 client.15426 fetch_fsmap learned FSMap version 27 2024-09-21T06:15:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.896+0000 7fdfe7928740 1 -- 172.21.15.3:0/1746449661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56249da02750 con 0x56249d63df60 2024-09-21T06:15:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.896+0000 7fdfc27fc640 1 -- 172.21.15.3:0/1746449661 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc036030 con 0x56249d63df60 2024-09-21T06:15:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.897+0000 7fdfc27fc640 1 -- 172.21.15.3:0/1746449661 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdfcc04d030 con 0x56249d63df60 2024-09-21T06:15:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.897+0000 7fdfe7928740 10 client.15426 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:15:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.897+0000 7fdfe7928740 10 client.15426 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T06:15:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.897+0000 7fdfe7928740 20 client.15426 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.897+0000 7fdfe7928740 1 --2- 172.21.15.3:0/1746449661 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56249d5fc8e0 0x56249da05e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.897+0000 7fdfe7928740 4 client.15426 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.897+0000 7fdfe7928740 1 -- 172.21.15.3:0/1746449661 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56249d95db60 con 0x56249d5fc8e0 2024-09-21T06:15:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.897+0000 7fdfdaffd640 1 --2- 172.21.15.3:0/1746449661 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56249d5fc8e0 0x56249da05e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.897+0000 7fdfc27fc640 10 client.15426 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:15:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.897+0000 7fdfdaffd640 1 --2- 172.21.15.3:0/1746449661 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56249d5fc8e0 0x56249da05e30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79b49498963ec2aa server_cookie=fb883a12eb1c56a3 in_seq=0 out_seq=0 2024-09-21T06:15:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.898+0000 7fdfc27fc640 1 -- 172.21.15.3:0/1746449661 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56249d95db60 con 0x56249d5fc8e0 2024-09-21T06:15:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.898+0000 7fdfc27fc640 10 client.15426 handle_command_reply: tid=0 2024-09-21T06:15:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.898+0000 7fdfe7928740 1 client.15426 shutdown 2024-09-21T06:15:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.898+0000 7fdfe7928740 1 -- 172.21.15.3:0/1746449661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 msgr2=0x56249d9f9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.898+0000 7fdfe7928740 1 --2- 172.21.15.3:0/1746449661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 0x56249d9f9960 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc005e30 tx=0x7fdfcc031700 comp rx=0 tx=0).stop 2024-09-21T06:15:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.898+0000 7fdfe7928740 1 -- 172.21.15.3:0/1746449661 shutdown_connections 2024-09-21T06:15:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.898+0000 7fdfe7928740 1 --2- 172.21.15.3:0/1746449661 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56249d5fc8e0 0x56249da05e30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.899+0000 7fdfc27fc640 0 client.15426 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:15:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.899+0000 7fdfe7928740 1 --2- 172.21.15.3:0/1746449661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d61d280 0x56249d9f0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.899+0000 7fdfe7928740 1 --2- 172.21.15.3:0/1746449661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d63df60 0x56249d9f9960 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.899+0000 7fdfe7928740 1 --2- 172.21.15.3:0/1746449661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d6150a0 0x56249d9efd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.899+0000 7fdfe7928740 1 -- 172.21.15.3:0/1746449661 >> 172.21.15.3:0/1746449661 conn(0x56249d632350 msgr2=0x56249d9e58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.899+0000 7fdfe7928740 1 -- 172.21.15.3:0/1746449661 shutdown_connections 2024-09-21T06:15:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.899+0000 7fdfe7928740 1 -- 172.21.15.3:0/1746449661 wait complete. 2024-09-21T06:15:01.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.899+0000 7fdfe7928740 20 client.15426 trim_cache size 0 max 16384 2024-09-21T06:15:01.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.917+0000 7fdfe7928740 1 Processor -- start 2024-09-21T06:15:01.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.917+0000 7fdfe7928740 1 -- start start 2024-09-21T06:15:01.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfe7928740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 0x56249d9ead30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfe7928740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d6150a0 0x56249da00750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfe7928740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d61d280 0x56249d9eb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfe7928740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56249da04170 con 0x56249d5fc8e0 2024-09-21T06:15:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfe7928740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56249da04840 con 0x56249d61d280 2024-09-21T06:15:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfe7928740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56249da04f10 con 0x56249d6150a0 2024-09-21T06:15:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfdaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 0x56249d9ead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfdaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 0x56249d9ead30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51976/0 (socket says 172.21.15.3:51976) 2024-09-21T06:15:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfdaffd640 1 -- 172.21.15.3:0/3238564909 learned_addr learned my addr 172.21.15.3:0/3238564909 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfdb7fe640 1 --2- 172.21.15.3:0/3238564909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d61d280 0x56249d9eb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfda7fc640 1 --2- 172.21.15.3:0/3238564909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d6150a0 0x56249da00750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfdaffd640 1 -- 172.21.15.3:0/3238564909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d6150a0 msgr2=0x56249da00750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfdaffd640 1 --2- 172.21.15.3:0/3238564909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d6150a0 0x56249da00750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfdaffd640 1 -- 172.21.15.3:0/3238564909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d61d280 msgr2=0x56249d9eb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfdaffd640 1 --2- 172.21.15.3:0/3238564909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d61d280 0x56249d9eb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfdaffd640 1 -- 172.21.15.3:0/3238564909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56249d9f4270 con 0x56249d5fc8e0 2024-09-21T06:15:01.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfdaffd640 1 --2- 172.21.15.3:0/3238564909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 0x56249d9ead30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdfd401b760 tx=0x7fdfd4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edaaff2db405d193 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:01.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.918+0000 7fdfd0ff9640 1 -- 172.21.15.3:0/3238564909 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd402a070 con 0x56249d5fc8e0 2024-09-21T06:15:01.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.919+0000 7fdfd0ff9640 1 -- 172.21.15.3:0/3238564909 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd40288e0 con 0x56249d5fc8e0 2024-09-21T06:15:01.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.919+0000 7fdfd0ff9640 1 -- 172.21.15.3:0/3238564909 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd402e8a0 con 0x56249d5fc8e0 2024-09-21T06:15:01.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.919+0000 7fdfe7928740 1 -- 172.21.15.3:0/3238564909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 msgr2=0x56249d9ead30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:01.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.919+0000 7fdfe7928740 1 --2- 172.21.15.3:0/3238564909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 0x56249d9ead30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdfd401b760 tx=0x7fdfd4019000 comp rx=0 tx=0).stop 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.919+0000 7fdfe7928740 1 -- 172.21.15.3:0/3238564909 shutdown_connections 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.919+0000 7fdfe7928740 1 --2- 172.21.15.3:0/3238564909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d6150a0 0x56249da00750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.919+0000 7fdfe7928740 1 --2- 172.21.15.3:0/3238564909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d61d280 0x56249d9eb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.919+0000 7fdfe7928740 1 --2- 172.21.15.3:0/3238564909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 0x56249d9ead30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfe7928740 1 -- 172.21.15.3:0/3238564909 >> 172.21.15.3:0/3238564909 conn(0x56249d632350 msgr2=0x56249d9dcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfe7928740 1 -- 172.21.15.3:0/3238564909 shutdown_connections 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfe7928740 1 -- 172.21.15.3:0/3238564909 wait complete. 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfe7928740 1 -- start start 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfe7928740 1 Processor -- start 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfe7928740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 0x56249d9f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfe7928740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d6150a0 0x56249d9fa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfe7928740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d61d280 0x56249d9fa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfe7928740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56249da05cb0 con 0x56249d5fc8e0 2024-09-21T06:15:01.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfe7928740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56249da055e0 con 0x56249d6150a0 2024-09-21T06:15:01.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfe7928740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56249da06380 con 0x56249d61d280 2024-09-21T06:15:01.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfe7928740 10 client.0 _ll_register_callbacks cb 0x7fdfd400a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:01.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfdaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 0x56249d9f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfdaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 0x56249d9f9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51984/0 (socket says 172.21.15.3:51984) 2024-09-21T06:15:01.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.920+0000 7fdfdaffd640 1 -- 172.21.15.3:0/435541733 learned_addr learned my addr 172.21.15.3:0/435541733 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:01.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfda7fc640 1 --2- 172.21.15.3:0/435541733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d6150a0 0x56249d9fa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfdb7fe640 1 --2- 172.21.15.3:0/435541733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d61d280 0x56249d9fa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfdaffd640 1 -- 172.21.15.3:0/435541733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d61d280 msgr2=0x56249d9fa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:01.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfdaffd640 1 --2- 172.21.15.3:0/435541733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d61d280 0x56249d9fa960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfdaffd640 1 -- 172.21.15.3:0/435541733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d6150a0 msgr2=0x56249d9fa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:01.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfdaffd640 1 --2- 172.21.15.3:0/435541733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d6150a0 0x56249d9fa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfdaffd640 1 -- 172.21.15.3:0/435541733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56249d9ff750 con 0x56249d5fc8e0 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfdaffd640 1 --2- 172.21.15.3:0/435541733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 0x56249d9f9ee0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4030040 tx=0x7fdfd4028430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e24efe77d72262c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfc27fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfc27fc640 1 -- 172.21.15.3:0/435541733 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd402a040 con 0x56249d5fc8e0 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfe7928740 1 -- 172.21.15.3:0/435541733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfd40288e0 con 0x56249d5fc8e0 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfc27fc640 1 -- 172.21.15.3:0/435541733 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fdfd4028c90 con 0x56249d5fc8e0 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfc27fc640 1 -- 172.21.15.3:0/435541733 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdfd4046050 con 0x56249d5fc8e0 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfe7928740 10 client.15432 fetch_fsmap learned FSMap version 27 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.921+0000 7fdfe7928740 1 -- 172.21.15.3:0/435541733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56249da716a0 con 0x56249d5fc8e0 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.922+0000 7fdfc27fc640 1 -- 172.21.15.3:0/435541733 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd40288e0 con 0x56249d5fc8e0 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.922+0000 7fdfc27fc640 1 -- 172.21.15.3:0/435541733 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdfd4004aa0 con 0x56249d5fc8e0 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.922+0000 7fdfe7928740 10 client.15432 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.922+0000 7fdfe7928740 10 client.15432 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.922+0000 7fdfe7928740 20 client.15432 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:01.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.922+0000 7fdfe7928740 1 --2- 172.21.15.3:0/435541733 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56249d63df60 0x56249da74c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.922+0000 7fdfe7928740 4 client.15432 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.922+0000 7fdfe7928740 1 -- 172.21.15.3:0/435541733 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "status"}) -- 0x56249d959610 con 0x56249d63df60 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.922+0000 7fdfda7fc640 1 --2- 172.21.15.3:0/435541733 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56249d63df60 0x56249da74c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfc27fc640 10 client.15432 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfda7fc640 1 --2- 172.21.15.3:0/435541733 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56249d63df60 0x56249da74c50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56c957f31e3624dc server_cookie=b31450bfa9341abc in_seq=0 out_seq=0 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfc27fc640 1 -- 172.21.15.3:0/435541733 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56249d959610 con 0x56249d63df60 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfc27fc640 10 client.15432 handle_command_reply: tid=0 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfe7928740 1 client.15432 shutdown 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfe7928740 1 -- 172.21.15.3:0/435541733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 msgr2=0x56249d9f9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfe7928740 1 --2- 172.21.15.3:0/435541733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 0x56249d9f9ee0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4030040 tx=0x7fdfd4028430 comp rx=0 tx=0).stop 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfe7928740 1 -- 172.21.15.3:0/435541733 shutdown_connections 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfe7928740 1 --2- 172.21.15.3:0/435541733 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56249d63df60 0x56249da74c50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfe7928740 1 --2- 172.21.15.3:0/435541733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56249d61d280 0x56249d9fa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfe7928740 1 --2- 172.21.15.3:0/435541733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56249d6150a0 0x56249d9fa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfc27fc640 0 client.15432 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:15:01.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfe7928740 1 --2- 172.21.15.3:0/435541733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56249d5fc8e0 0x56249d9f9ee0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfe7928740 1 -- 172.21.15.3:0/435541733 >> 172.21.15.3:0/435541733 conn(0x56249d632350 msgr2=0x56249d9ead30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:01.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfe7928740 1 -- 172.21.15.3:0/435541733 shutdown_connections 2024-09-21T06:15:01.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfe7928740 1 -- 172.21.15.3:0/435541733 wait complete. 2024-09-21T06:15:01.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.923+0000 7fdfe7928740 20 client.15432 trim_cache size 0 max 16384 2024-09-21T06:15:01.930 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:15:01.930 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "38d3c75c-77df-11ef-baf5-efdc52797490", 2024-09-21T06:15:01.931 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-09-21T06:15:01.931 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24691, 2024-09-21T06:15:01.931 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-21T06:15:01.931 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-21T06:15:01.931 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-21T06:15:01.931 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 102.374121755, 2024-09-21T06:15:01.931 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2024-09-21T06:15:01.931 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 79, 2024-09-21T06:15:01.931 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T06:15:01.931 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 130.845448202 2024-09-21T06:15:01.931 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:15:01.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.924+0000 7fdfe13b4640 1 -- 172.21.15.3:0/2356144844 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdfc80b6590 msgr2=0x7fdfc80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:01.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.924+0000 7fdfe13b4640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdfc80b6590 0x7fdfc80b8aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400ad30 tx=0x7fdfd400a2d0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.924+0000 7fdfe13b4640 1 -- 172.21.15.3:0/2356144844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfdc1083a0 msgr2=0x7fdfdc1998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:01.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.924+0000 7fdfe13b4640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfdc1083a0 0x7fdfdc1998b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4005fd0 tx=0x7fdfc400dd20 comp rx=0 tx=0).stop 2024-09-21T06:15:01.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.924+0000 7fdfe13b4640 1 -- 172.21.15.3:0/2356144844 shutdown_connections 2024-09-21T06:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.924+0000 7fdfe13b4640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdfc80b6590 0x7fdfc80b8aa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.924+0000 7fdfe13b4640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfdc108cf0 0x7fdfdc19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.924+0000 7fdfe13b4640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfdc1083a0 0x7fdfdc1998b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.924+0000 7fdfe13b4640 1 --2- 172.21.15.3:0/2356144844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfdc102a60 0x7fdfdc199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.924+0000 7fdfe13b4640 1 -- 172.21.15.3:0/2356144844 >> 172.21.15.3:0/2356144844 conn(0x7fdfdc079730 msgr2=0x7fdfdc1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.924+0000 7fdfe13b4640 1 -- 172.21.15.3:0/2356144844 shutdown_connections 2024-09-21T06:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:01.924+0000 7fdfe13b4640 1 -- 172.21.15.3:0/2356144844 wait complete. 2024-09-21T06:15:02.685 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T06:15:02.952 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:03.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:03 smithi003 ceph-mon[31227]: pgmap v326: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T06:15:03.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.411+0000 7f3f2fef3640 1 -- 172.21.15.3:0/3436204480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28108d10 msgr2=0x7f3f2810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.411+0000 7f3f2fef3640 1 --2- 172.21.15.3:0/3436204480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28108d10 0x7f3f2810f540 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009ca0 tx=0x7f3f2002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.411+0000 7f3f2fef3640 1 -- 172.21.15.3:0/3436204480 shutdown_connections 2024-09-21T06:15:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.411+0000 7f3f2fef3640 1 --2- 172.21.15.3:0/3436204480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28108d10 0x7f3f2810f540 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.411+0000 7f3f2fef3640 1 --2- 172.21.15.3:0/3436204480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f281083c0 0x7f3f281087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.411+0000 7f3f2fef3640 1 --2- 172.21.15.3:0/3436204480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f28102a80 0x7f3f28102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.411+0000 7f3f2fef3640 1 -- 172.21.15.3:0/3436204480 >> 172.21.15.3:0/3436204480 conn(0x7f3f28079740 msgr2=0x7f3f28079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.412+0000 7f3f2fef3640 1 -- 172.21.15.3:0/3436204480 shutdown_connections 2024-09-21T06:15:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.412+0000 7f3f2fef3640 1 -- 172.21.15.3:0/3436204480 wait complete. 2024-09-21T06:15:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.412+0000 7f3f2fef3640 1 Processor -- start 2024-09-21T06:15:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.412+0000 7f3f2fef3640 1 -- start start 2024-09-21T06:15:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2fef3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28102a80 0x7f3f2819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2fef3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f281083c0 0x7f3f2819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2fef3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f28108d10 0x7f3f281a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2fef3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3f28112980 con 0x7f3f28102a80 2024-09-21T06:15:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2fef3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3f28112800 con 0x7f3f281083c0 2024-09-21T06:15:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2fef3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3f28112b00 con 0x7f3f28108d10 2024-09-21T06:15:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2dc8f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28102a80 0x7f3f2819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2dc8f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28102a80 0x7f3f2819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52010/0 (socket says 172.21.15.3:52010) 2024-09-21T06:15:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2dc8f640 1 -- 172.21.15.3:0/2466882026 learned_addr learned my addr 172.21.15.3:0/2466882026 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2d48e640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f281083c0 0x7f3f2819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2e490640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f28108d10 0x7f3f281a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2dc8f640 1 -- 172.21.15.3:0/2466882026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f28108d10 msgr2=0x7f3f281a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f28108d10 0x7f3f281a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2dc8f640 1 -- 172.21.15.3:0/2466882026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f281083c0 msgr2=0x7f3f2819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f281083c0 0x7f3f2819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2dc8f640 1 -- 172.21.15.3:0/2466882026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f281a25f0 con 0x7f3f28102a80 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28102a80 0x7f3f2819c680 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400d950 tx=0x7f3f2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f1effd640 1 -- 172.21.15.3:0/2466882026 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24017070 con 0x7f3f28102a80 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f1effd640 1 -- 172.21.15.3:0/2466882026 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f24009c30 con 0x7f3f28102a80 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2fef3640 1 -- 172.21.15.3:0/2466882026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f281a28e0 con 0x7f3f28102a80 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f1effd640 1 -- 172.21.15.3:0/2466882026 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400f620 con 0x7f3f28102a80 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.413+0000 7f3f2fef3640 1 -- 172.21.15.3:0/2466882026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f281a9040 con 0x7f3f28102a80 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.416+0000 7f3f36467740 1 Processor -- start 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.416+0000 7f3f36467740 1 -- start start 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f1effd640 1 -- 172.21.15.3:0/2466882026 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 154528+0+0 (secure 0 0 0) 0x7f3f24010460 con 0x7f3f28102a80 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f36467740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b40f60 0x555634ee6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f36467740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 0x555634ee7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f36467740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b20280 0x555634ee7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f36467740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555634e5ec10 con 0x555634b180a0 2024-09-21T06:15:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f36467740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555634e4e040 con 0x555634b40f60 2024-09-21T06:15:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f36467740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555634e51e50 con 0x555634b20280 2024-09-21T06:15:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f2dc8f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 0x555634ee7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f2dc8f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 0x555634ee7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52024/0 (socket says 172.21.15.3:52024) 2024-09-21T06:15:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f2dc8f640 1 -- 172.21.15.3:0/4020746173 learned_addr learned my addr 172.21.15.3:0/4020746173 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f2e490640 1 --2- 172.21.15.3:0/4020746173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b40f60 0x555634ee6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f2d48e640 1 --2- 172.21.15.3:0/4020746173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b20280 0x555634ee7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f2dc8f640 1 -- 172.21.15.3:0/4020746173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b20280 msgr2=0x555634ee7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/4020746173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b20280 0x555634ee7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f2dc8f640 1 -- 172.21.15.3:0/4020746173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b40f60 msgr2=0x555634ee6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/4020746173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b40f60 0x555634ee6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.417+0000 7f3f2dc8f640 1 -- 172.21.15.3:0/4020746173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555634bac540 con 0x555634b180a0 2024-09-21T06:15:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.418+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/4020746173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 0x555634ee7440 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3f240ef270 tx=0x7f3f240ef730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c4c3cb67121141 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.418+0000 7f3f1effd640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3f140b6560 0x7f3f140b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.418+0000 7f3f2e490640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3f140b6560 0x7f3f140b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.418+0000 7f3f13fff640 1 -- 172.21.15.3:0/4020746173 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f240fa070 con 0x555634b180a0 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.418+0000 7f3f13fff640 1 -- 172.21.15.3:0/4020746173 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f2401e2e0 con 0x555634b180a0 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.418+0000 7f3f1effd640 1 -- 172.21.15.3:0/2466882026 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f3f240e7080 con 0x7f3f28102a80 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.418+0000 7f3f13fff640 1 -- 172.21.15.3:0/4020746173 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f240acf00 con 0x555634b180a0 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.419+0000 7f3f36467740 1 -- 172.21.15.3:0/4020746173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 msgr2=0x555634ee7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.419+0000 7f3f2e490640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3f140b6560 0x7f3f140b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3f20005e30 tx=0x7f3f20005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.419+0000 7f3f36467740 1 --2- 172.21.15.3:0/4020746173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 0x555634ee7440 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3f240ef270 tx=0x7f3f240ef730 comp rx=0 tx=0).stop 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.419+0000 7f3f36467740 1 -- 172.21.15.3:0/4020746173 shutdown_connections 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.419+0000 7f3f36467740 1 --2- 172.21.15.3:0/4020746173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b20280 0x555634ee7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.419+0000 7f3f36467740 1 --2- 172.21.15.3:0/4020746173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 0x555634ee7440 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.419+0000 7f3f36467740 1 --2- 172.21.15.3:0/4020746173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b40f60 0x555634ee6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.419+0000 7f3f36467740 1 -- 172.21.15.3:0/4020746173 >> 172.21.15.3:0/4020746173 conn(0x555634b35350 msgr2=0x555634ee4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.419+0000 7f3f36467740 1 -- 172.21.15.3:0/4020746173 shutdown_connections 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.419+0000 7f3f36467740 1 -- 172.21.15.3:0/4020746173 wait complete. 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.419+0000 7f3f36467740 1 -- start start 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f36467740 1 Processor -- start 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f36467740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b180a0 0x555634ef2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f36467740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 0x555634ef2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f36467740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b40f60 0x555634efea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f36467740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555634caf5f0 con 0x555634b20280 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f36467740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555634de9010 con 0x555634b40f60 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f36467740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555634ef56e0 con 0x555634b180a0 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f36467740 10 client.0 _ll_register_callbacks cb 0x555634e48340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f2d48e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 0x555634ef2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f2d48e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 0x555634ef2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52028/0 (socket says 172.21.15.3:52028) 2024-09-21T06:15:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f2d48e640 1 -- 172.21.15.3:0/1007748004 learned_addr learned my addr 172.21.15.3:0/1007748004 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b180a0 0x555634ef2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f2e490640 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b40f60 0x555634efea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f2d48e640 1 -- 172.21.15.3:0/1007748004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b180a0 msgr2=0x555634ef2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f2d48e640 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b180a0 0x555634ef2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f2d48e640 1 -- 172.21.15.3:0/1007748004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b40f60 msgr2=0x555634efea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.420+0000 7f3f2d48e640 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b40f60 0x555634efea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.421+0000 7f3f2d48e640 1 -- 172.21.15.3:0/1007748004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555634e16a00 con 0x555634b20280 2024-09-21T06:15:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.421+0000 7f3f2e490640 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b40f60 0x555634efea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.421+0000 7f3f2d48e640 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 0x555634ef2f50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800b9f0 tx=0x7f3f1800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247ac8b8e035178c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.421+0000 7f3f117fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.421+0000 7f3f117fa640 1 -- 172.21.15.3:0/1007748004 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18003da0 con 0x555634b20280 2024-09-21T06:15:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.421+0000 7f3f117fa640 1 -- 172.21.15.3:0/1007748004 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f3f18005020 con 0x555634b20280 2024-09-21T06:15:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.421+0000 7f3f36467740 1 -- 172.21.15.3:0/1007748004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f2401e2e0 con 0x555634b20280 2024-09-21T06:15:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.421+0000 7f3f117fa640 1 -- 172.21.15.3:0/1007748004 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f18005220 con 0x555634b20280 2024-09-21T06:15:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.421+0000 7f3f36467740 10 client.15444 fetch_fsmap learned FSMap version 27 2024-09-21T06:15:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.421+0000 7f3f36467740 1 -- 172.21.15.3:0/1007748004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555634f01670 con 0x555634b20280 2024-09-21T06:15:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.421+0000 7f3f117fa640 1 -- 172.21.15.3:0/1007748004 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f2401e2e0 con 0x555634b20280 2024-09-21T06:15:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.421+0000 7f3f117fa640 1 -- 172.21.15.3:0/1007748004 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3f1801b500 con 0x555634b20280 2024-09-21T06:15:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.421+0000 7f3f36467740 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:15:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.422+0000 7f3f36467740 10 client.15444 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:15:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.422+0000 7f3f36467740 20 client.15444 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.422+0000 7f3f36467740 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555634aff8e0 0x555634f05120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.422+0000 7f3f36467740 4 client.15444 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.422+0000 7f3f36467740 1 -- 172.21.15.3:0/1007748004 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555634e60b60 con 0x555634aff8e0 2024-09-21T06:15:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.422+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555634aff8e0 0x555634f05120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.422+0000 7f3f117fa640 10 client.15444 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:03.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.422+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555634aff8e0 0x555634f05120 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb90c930aed1bd49 server_cookie=2b2c344999896954 in_seq=0 out_seq=0 2024-09-21T06:15:03.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.423+0000 7f3f117fa640 1 -- 172.21.15.3:0/1007748004 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555634e60b60 con 0x555634aff8e0 2024-09-21T06:15:03.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.423+0000 7f3f117fa640 10 client.15444 handle_command_reply: tid=0 2024-09-21T06:15:03.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.423+0000 7f3f36467740 1 client.15444 shutdown 2024-09-21T06:15:03.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.423+0000 7f3f36467740 1 -- 172.21.15.3:0/1007748004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 msgr2=0x555634ef2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:03.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.423+0000 7f3f36467740 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 0x555634ef2f50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800b9f0 tx=0x7f3f1800beb0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.423+0000 7f3f36467740 1 -- 172.21.15.3:0/1007748004 shutdown_connections 2024-09-21T06:15:03.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.423+0000 7f3f36467740 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555634aff8e0 0x555634f05120 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.423+0000 7f3f36467740 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b40f60 0x555634efea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.423+0000 7f3f36467740 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 0x555634ef2f50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.423+0000 7f3f117fa640 0 client.15444 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:03.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.423+0000 7f3f36467740 1 --2- 172.21.15.3:0/1007748004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b180a0 0x555634ef2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.423+0000 7f3f36467740 1 -- 172.21.15.3:0/1007748004 >> 172.21.15.3:0/1007748004 conn(0x555634b35350 msgr2=0x555634eea090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:03.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.424+0000 7f3f36467740 1 -- 172.21.15.3:0/1007748004 shutdown_connections 2024-09-21T06:15:03.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.424+0000 7f3f36467740 1 -- 172.21.15.3:0/1007748004 wait complete. 2024-09-21T06:15:03.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.424+0000 7f3f36467740 20 client.15444 trim_cache size 0 max 16384 2024-09-21T06:15:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.442+0000 7f3f36467740 1 Processor -- start 2024-09-21T06:15:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.442+0000 7f3f36467740 1 -- start start 2024-09-21T06:15:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.442+0000 7f3f36467740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 0x555634eed020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.442+0000 7f3f36467740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b20280 0x555634efe970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.442+0000 7f3f36467740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b40f60 0x555634efeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.442+0000 7f3f36467740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555634f03460 con 0x555634b180a0 2024-09-21T06:15:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.442+0000 7f3f36467740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555634f03b30 con 0x555634b20280 2024-09-21T06:15:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.442+0000 7f3f36467740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555634f04200 con 0x555634b40f60 2024-09-21T06:15:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f2dc8f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 0x555634eed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f2dc8f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 0x555634eed020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52040/0 (socket says 172.21.15.3:52040) 2024-09-21T06:15:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f2dc8f640 1 -- 172.21.15.3:0/3880130875 learned_addr learned my addr 172.21.15.3:0/3880130875 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f2e490640 1 --2- 172.21.15.3:0/3880130875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b40f60 0x555634efeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f2d48e640 1 --2- 172.21.15.3:0/3880130875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b20280 0x555634efe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f2dc8f640 1 -- 172.21.15.3:0/3880130875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b40f60 msgr2=0x555634efeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/3880130875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b40f60 0x555634efeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f2dc8f640 1 -- 172.21.15.3:0/3880130875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b20280 msgr2=0x555634efe970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/3880130875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b20280 0x555634efe970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f2dc8f640 1 -- 172.21.15.3:0/3880130875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555634ee9790 con 0x555634b180a0 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/3880130875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 0x555634eed020 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3f24000c00 tx=0x7f3f240b4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f3542eb46b7b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f137fe640 1 -- 172.21.15.3:0/3880130875 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24020040 con 0x555634b180a0 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f137fe640 1 -- 172.21.15.3:0/3880130875 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f240ea930 con 0x555634b180a0 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.443+0000 7f3f137fe640 1 -- 172.21.15.3:0/3880130875 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24061980 con 0x555634b180a0 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.444+0000 7f3f36467740 1 -- 172.21.15.3:0/3880130875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 msgr2=0x555634eed020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.444+0000 7f3f36467740 1 --2- 172.21.15.3:0/3880130875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 0x555634eed020 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3f24000c00 tx=0x7f3f240b4f90 comp rx=0 tx=0).stop 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.444+0000 7f3f36467740 1 -- 172.21.15.3:0/3880130875 shutdown_connections 2024-09-21T06:15:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.444+0000 7f3f36467740 1 --2- 172.21.15.3:0/3880130875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b40f60 0x555634efeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.444+0000 7f3f36467740 1 --2- 172.21.15.3:0/3880130875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b20280 0x555634efe970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.444+0000 7f3f36467740 1 --2- 172.21.15.3:0/3880130875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b180a0 0x555634eed020 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.444+0000 7f3f36467740 1 -- 172.21.15.3:0/3880130875 >> 172.21.15.3:0/3880130875 conn(0x555634b35350 msgr2=0x555634edfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.444+0000 7f3f36467740 1 -- 172.21.15.3:0/3880130875 shutdown_connections 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.444+0000 7f3f36467740 1 -- 172.21.15.3:0/3880130875 wait complete. 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.445+0000 7f3f36467740 1 -- start start 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.445+0000 7f3f36467740 1 Processor -- start 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.445+0000 7f3f36467740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b180a0 0x555634f0d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.445+0000 7f3f36467740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 0x555634f0dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.445+0000 7f3f36467740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b40f60 0x555634f06be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.445+0000 7f3f36467740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555634f04fa0 con 0x555634b20280 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.445+0000 7f3f36467740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555634f048d0 con 0x555634b180a0 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.445+0000 7f3f36467740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555634f05670 con 0x555634b40f60 2024-09-21T06:15:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.445+0000 7f3f36467740 10 client.0 _ll_register_callbacks cb 0x7f3f18004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f2d48e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 0x555634f0dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f2d48e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 0x555634f0dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52044/0 (socket says 172.21.15.3:52044) 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f2d48e640 1 -- 172.21.15.3:0/1351178175 learned_addr learned my addr 172.21.15.3:0/1351178175 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b180a0 0x555634f0d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f2e490640 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b40f60 0x555634f06be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f2d48e640 1 -- 172.21.15.3:0/1351178175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b40f60 msgr2=0x555634f06be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f2d48e640 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b40f60 0x555634f06be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f2d48e640 1 -- 172.21.15.3:0/1351178175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b180a0 msgr2=0x555634f0d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f2d48e640 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b180a0 0x555634f0d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f2d48e640 1 -- 172.21.15.3:0/1351178175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555634f07370 con 0x555634b20280 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b180a0 0x555634f0d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f2d48e640 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 0x555634f0dc80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3f18001eb0 tx=0x7f3f18005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553097ed7dcbd122 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f117fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f117fa640 1 -- 172.21.15.3:0/1351178175 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18028040 con 0x555634b20280 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f117fa640 1 -- 172.21.15.3:0/1351178175 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f3f18020020 con 0x555634b20280 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.446+0000 7f3f36467740 1 -- 172.21.15.3:0/1351178175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f240ea930 con 0x555634b20280 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.447+0000 7f3f117fa640 1 -- 172.21.15.3:0/1351178175 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f1803b050 con 0x555634b20280 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.447+0000 7f3f36467740 10 client.15450 fetch_fsmap learned FSMap version 27 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.447+0000 7f3f36467740 1 -- 172.21.15.3:0/1351178175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555634f0cdb0 con 0x555634b20280 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.447+0000 7f3f117fa640 1 -- 172.21.15.3:0/1351178175 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f240ea930 con 0x555634b20280 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.447+0000 7f3f117fa640 1 -- 172.21.15.3:0/1351178175 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3f18011a40 con 0x555634b20280 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.447+0000 7f3f36467740 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.447+0000 7f3f36467740 10 client.15450 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.447+0000 7f3f36467740 20 client.15450 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.447+0000 7f3f36467740 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555634ad4c90 0x555634f73a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.447+0000 7f3f36467740 4 client.15450 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.447+0000 7f3f36467740 1 -- 172.21.15.3:0/1351178175 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "status"}) -- 0x555634e5c610 con 0x555634ad4c90 2024-09-21T06:15:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.447+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555634ad4c90 0x555634f73a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f2dc8f640 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555634ad4c90 0x555634f73a30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef67dce2da11ca53 server_cookie=3d7afbdc346f659a in_seq=0 out_seq=0 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f117fa640 10 client.15450 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f117fa640 1 -- 172.21.15.3:0/1351178175 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555634e5c610 con 0x555634ad4c90 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f117fa640 10 client.15450 handle_command_reply: tid=0 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f36467740 1 client.15450 shutdown 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f36467740 1 -- 172.21.15.3:0/1351178175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 msgr2=0x555634f0dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f36467740 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 0x555634f0dc80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3f18001eb0 tx=0x7f3f18005670 comp rx=0 tx=0).stop 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f36467740 1 -- 172.21.15.3:0/1351178175 shutdown_connections 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f36467740 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555634ad4c90 0x555634f73a30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f36467740 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555634b40f60 0x555634f06be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f36467740 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555634b20280 0x555634f0dc80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f36467740 1 --2- 172.21.15.3:0/1351178175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555634b180a0 0x555634f0d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f117fa640 0 client.15450 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f36467740 1 -- 172.21.15.3:0/1351178175 >> 172.21.15.3:0/1351178175 conn(0x555634b35350 msgr2=0x555634edfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:03.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f36467740 1 -- 172.21.15.3:0/1351178175 shutdown_connections 2024-09-21T06:15:03.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.448+0000 7f3f36467740 1 -- 172.21.15.3:0/1351178175 wait complete. 2024-09-21T06:15:03.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.449+0000 7f3f36467740 20 client.15450 trim_cache size 0 max 16384 2024-09-21T06:15:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:15:03.460 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "38d3c75c-77df-11ef-baf5-efdc52797490", 2024-09-21T06:15:03.460 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-21T06:15:03.460 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24704, 2024-09-21T06:15:03.460 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-21T06:15:03.460 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-21T06:15:03.460 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-21T06:15:03.461 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 111.95933862699999, 2024-09-21T06:15:03.461 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2024-09-21T06:15:03.461 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 79, 2024-09-21T06:15:03.461 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T06:15:03.461 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 122.546824312 2024-09-21T06:15:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:15:03.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.449+0000 7f3f2fef3640 1 -- 172.21.15.3:0/2466882026 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3f140b6560 msgr2=0x7f3f140b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:03.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.449+0000 7f3f2fef3640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3f140b6560 0x7f3f140b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3f20005e30 tx=0x7f3f20005c30 comp rx=0 tx=0).stop 2024-09-21T06:15:03.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.449+0000 7f3f2fef3640 1 -- 172.21.15.3:0/2466882026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28102a80 msgr2=0x7f3f2819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.449+0000 7f3f2fef3640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28102a80 0x7f3f2819c680 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400d950 tx=0x7f3f2400de10 comp rx=0 tx=0).stop 2024-09-21T06:15:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.449+0000 7f3f2fef3640 1 -- 172.21.15.3:0/2466882026 shutdown_connections 2024-09-21T06:15:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.449+0000 7f3f2fef3640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3f140b6560 0x7f3f140b8a70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.449+0000 7f3f2fef3640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f28108d10 0x7f3f281a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.449+0000 7f3f2fef3640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f281083c0 0x7f3f2819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.449+0000 7f3f2fef3640 1 --2- 172.21.15.3:0/2466882026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28102a80 0x7f3f2819c680 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.449+0000 7f3f2fef3640 1 -- 172.21.15.3:0/2466882026 >> 172.21.15.3:0/2466882026 conn(0x7f3f28079740 msgr2=0x7f3f28104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.449+0000 7f3f2fef3640 1 -- 172.21.15.3:0/2466882026 shutdown_connections 2024-09-21T06:15:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:03.449+0000 7f3f2fef3640 1 -- 172.21.15.3:0/2466882026 wait complete. 2024-09-21T06:15:03.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:03 smithi137 ceph-mon[34464]: pgmap v326: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T06:15:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:03 smithi204 ceph-mon[34659]: pgmap v326: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T06:15:04.368 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T06:15:04.395 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T06:15:04.395 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T06:15:04.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:15:04.395 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-21T06:15:04.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:15:04.415 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-21T06:15:04.487 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2024-09-21T06:15:04.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:15:04.489 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-21T06:15:04.489 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T06:15:04.489 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:15:04.489 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T06:15:04.792 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.257+0000 7f29dc69f640 1 -- 172.21.15.3:0/968221867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d41083c0 msgr2=0x7f29d41087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.257+0000 7f29dc69f640 1 --2- 172.21.15.3:0/968221867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d41083c0 0x7f29d41087d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009900 tx=0x7f29c402a410 comp rx=0 tx=0).stop 2024-09-21T06:15:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.257+0000 7f29dc69f640 1 -- 172.21.15.3:0/968221867 shutdown_connections 2024-09-21T06:15:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.257+0000 7f29dc69f640 1 --2- 172.21.15.3:0/968221867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d4108d10 0x7f29d410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.257+0000 7f29dc69f640 1 --2- 172.21.15.3:0/968221867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d41083c0 0x7f29d41087d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.257+0000 7f29dc69f640 1 --2- 172.21.15.3:0/968221867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29d4102a80 0x7f29d4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.257+0000 7f29dc69f640 1 -- 172.21.15.3:0/968221867 >> 172.21.15.3:0/968221867 conn(0x7f29d4079740 msgr2=0x7f29d4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.257+0000 7f29dc69f640 1 -- 172.21.15.3:0/968221867 shutdown_connections 2024-09-21T06:15:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.257+0000 7f29dc69f640 1 -- 172.21.15.3:0/968221867 wait complete. 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29dc69f640 1 Processor -- start 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29dc69f640 1 -- start start 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29dc69f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29d4102a80 0x7f29d419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29dc69f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d41083c0 0x7f29d419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29dc69f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d4108d10 0x7f29d419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29dc69f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4112980 con 0x7f29d41083c0 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29dc69f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4112800 con 0x7f29d4102a80 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29dc69f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4112b00 con 0x7f29d4108d10 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29d9c3a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d41083c0 0x7f29d419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29d9c3a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d41083c0 0x7f29d419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48146/0 (socket says 172.21.15.3:48146) 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29d9c3a640 1 -- 172.21.15.3:0/2560524423 learned_addr learned my addr 172.21.15.3:0/2560524423 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29da43b640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29d4102a80 0x7f29d419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29dac3c640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d4108d10 0x7f29d419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29d9c3a640 1 -- 172.21.15.3:0/2560524423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d4108d10 msgr2=0x7f29d419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29d9c3a640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d4108d10 0x7f29d419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29d9c3a640 1 -- 172.21.15.3:0/2560524423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29d4102a80 msgr2=0x7f29d419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29d9c3a640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29d4102a80 0x7f29d419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29d9c3a640 1 -- 172.21.15.3:0/2560524423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d41a0570 con 0x7f29d41083c0 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29da43b640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29d4102a80 0x7f29d419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.258+0000 7f29dac3c640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d4108d10 0x7f29d419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.259+0000 7f29d9c3a640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d41083c0 0x7f29d419aba0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f29c4032fd0 tx=0x7f29c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.259+0000 7f29cb7fe640 1 -- 172.21.15.3:0/2560524423 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c403f030 con 0x7f29d41083c0 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.259+0000 7f29dc69f640 1 -- 172.21.15.3:0/2560524423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d41a0800 con 0x7f29d41083c0 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.259+0000 7f29cb7fe640 1 -- 172.21.15.3:0/2560524423 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c4004490 con 0x7f29d41083c0 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.259+0000 7f29cb7fe640 1 -- 172.21.15.3:0/2560524423 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4035070 con 0x7f29d41083c0 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.259+0000 7f29dc69f640 1 -- 172.21.15.3:0/2560524423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d41a6fc0 con 0x7f29d41083c0 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.260+0000 7f29dc69f640 1 -- 172.21.15.3:0/2560524423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d4103d00 con 0x7f29d41083c0 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.262+0000 7f29cb7fe640 1 -- 172.21.15.3:0/2560524423 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 154528+0+0 (secure 0 0 0) 0x7f29c4042030 con 0x7f29d41083c0 2024-09-21T06:15:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.263+0000 7f29cb7fe640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f29c00b6560 0x7f29c00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.263+0000 7f29da43b640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f29c00b6560 0x7f29c00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.263+0000 7f29cb7fe640 1 -- 172.21.15.3:0/2560524423 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f29c4107dc0 con 0x7f29d41083c0 2024-09-21T06:15:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.263+0000 7f29da43b640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f29c00b6560 0x7f29c00b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009b40 tx=0x7f29d0005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.266+0000 7f29cb7fe640 1 -- 172.21.15.3:0/2560524423 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f29c40ce570 con 0x7f29d41083c0 2024-09-21T06:15:05.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.497+0000 7f29dc69f640 1 -- 172.21.15.3:0/2560524423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f29d419b730 con 0x7f29d41083c0 2024-09-21T06:15:05.499 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:05 smithi003 ceph-mon[31227]: pgmap v327: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:15:05.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.497+0000 7f29cb7fe640 1 -- 172.21.15.3:0/2560524423 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f29d419b730 con 0x7f29d41083c0 2024-09-21T06:15:05.499 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:15:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.500+0000 7f29dc69f640 1 -- 172.21.15.3:0/2560524423 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f29c00b6560 msgr2=0x7f29c00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.500+0000 7f29dc69f640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f29c00b6560 0x7f29c00b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009b40 tx=0x7f29d0005e50 comp rx=0 tx=0).stop 2024-09-21T06:15:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.500+0000 7f29dc69f640 1 -- 172.21.15.3:0/2560524423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d41083c0 msgr2=0x7f29d419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.500+0000 7f29dc69f640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d41083c0 0x7f29d419aba0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f29c4032fd0 tx=0x7f29c4004270 comp rx=0 tx=0).stop 2024-09-21T06:15:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.500+0000 7f29dc69f640 1 -- 172.21.15.3:0/2560524423 shutdown_connections 2024-09-21T06:15:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.500+0000 7f29dc69f640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f29c00b6560 0x7f29c00b8a70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.500+0000 7f29dc69f640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d4108d10 0x7f29d419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.500+0000 7f29dc69f640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d41083c0 0x7f29d419aba0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.500+0000 7f29dc69f640 1 --2- 172.21.15.3:0/2560524423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29d4102a80 0x7f29d419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.500+0000 7f29dc69f640 1 -- 172.21.15.3:0/2560524423 >> 172.21.15.3:0/2560524423 conn(0x7f29d4079740 msgr2=0x7f29d4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.500+0000 7f29dc69f640 1 -- 172.21.15.3:0/2560524423 shutdown_connections 2024-09-21T06:15:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:05.500+0000 7f29dc69f640 1 -- 172.21.15.3:0/2560524423 wait complete. 2024-09-21T06:15:05.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:05 smithi137 ceph-mon[34464]: pgmap v327: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:15:05.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:05 smithi204 ceph-mon[34659]: pgmap v327: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:15:06.280 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T06:15:06.280 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:15:06.280 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-21T06:15:06.280 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:15:06.281 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:15:06.281 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T06:15:06.281 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T06:15:06.281 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:15:06.281 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:15:06.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:15:06.281 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-21T06:15:06.299 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:15:06.299 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:15:06.299 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:15:06.299 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:15:06.299 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-21T06:15:06.299 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42106sec preferred_lft 42106sec 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:15:06.300 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-21T06:15:06.301 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-21T06:15:06.301 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T06:15:06.301 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:15:06.301 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-21T06:15:06.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2024-09-21T06:15:06.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2024-09-21T06:15:06.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2024-09-21T06:15:06.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T06:15:06.301 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-21T06:15:06.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:15:06.502 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:06 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2560524423' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:15:06.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:15:06.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:15:06.524 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:15:06.551 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-21T06:15:06.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:15:06.553 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-21T06:15:06.623 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-21T06:15:06.623 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:15:06.623 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T06:15:06.623 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T06:15:06.623 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:15:06.624 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:15:06.624 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-21T06:15:06.624 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:15:06.624 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:15:06.624 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:15:06.624 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-21T06:15:06.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:06 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2560524423' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:15:06.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:06 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2560524423' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:15:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:15:06.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:15:06.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:15:06.930 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-21T06:15:06.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:15:06.947 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-09-21T06:15:07.003 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:15:07.003 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-21T06:15:07.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:15:07.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T06:15:07.003 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-21T06:15:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:15:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:15:07.123 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T06:15:07.123 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:15:07.123 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-21T06:15:07.123 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T06:15:07.123 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T06:15:07.123 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T06:15:07.123 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T06:15:07.124 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T06:15:07.124 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-21T06:15:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:15:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:15:07.249 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:15:07.249 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-21T06:15:07.249 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-21T06:15:07.249 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T06:15:07.249 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-21T06:15:07.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:15:07.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:15:07.329 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:15:07.329 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:15:07.345 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:15:07.345 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:15:07.345 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:15:07.401 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T06:15:07.401 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T06:15:07.402 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/dacd083d-59f5-4b54-9712-1121e8b11822 2024-09-21T06:15:07.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:15:07.402 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/dacd083d-59f5-4b54-9712-1121e8b11822 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-21T06:15:07.464 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:07 smithi003 ceph-mon[31227]: pgmap v328: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:15:07.542 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-21T06:15:07.543 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-21T06:15:07.543 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-21T06:15:07.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:07 smithi137 ceph-mon[34464]: pgmap v328: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:15:07.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:07 smithi204 ceph-mon[34659]: pgmap v328: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:15:07.683 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-21T06:15:07.683 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-21T06:15:07.684 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-21T06:15:07.684 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.137:3300,172.21.15.204:3300" 2024-09-21T06:15:07.684 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:3300,172.21.15.137:3300,172.21.15.204:3300:/volumes/qa/sv_1/dacd083d-59f5-4b54-9712-1121e8b11822 2024-09-21T06:15:07.684 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=38d3c75c-77df-11ef-baf5-efdc52797490" will pass to kernel 2024-09-21T06:15:07.685 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T06:15:07.685 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-09-21T06:15:07.685 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T06:15:07.685 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T06:15:07.685 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T06:15:07.687 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:15:07.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:15:07.687 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:15:07.759 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:15:07.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:15:07.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:15:07.858 INFO:teuthology.orchestra.run.smithi003.stdout:client15459 2024-09-21T06:15:07.858 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:15:07.858 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:15:07.876 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:15:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:15:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:15:07.974 INFO:teuthology.orchestra.run.smithi003.stdout:client15459 2024-09-21T06:15:07.974 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:15:08.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:15:08.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:15:08.032 INFO:teuthology.orchestra.run.smithi003.stdout:client15459 2024-09-21T06:15:08.032 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/38d3c75c-77df-11ef-baf5-efdc52797490.client15459/status') 2024-09-21T06:15:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:15:08.092 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.15459 (3)192.168.0.1:0/1571895111 2024-09-21T06:15:08.092 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2024-09-21T06:15:08.093 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2024-09-21T06:15:08.093 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2024-09-21T06:15:08.093 INFO:teuthology.orchestra.run.smithi003.stderr:70 bytes copied, 2.486e-05 s, 2.8 MB/s 2024-09-21T06:15:08.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:15:08.100 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T06:15:08.109 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T06:15:08.109 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-09-21T06:15:08.109 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-21T06:15:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.221+0000 7fbf56207640 1 -- 172.21.15.3:0/687628548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf50108c60 msgr2=0x7fbf5010f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.221+0000 7fbf56207640 1 --2- 172.21.15.3:0/687628548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf50108c60 0x7fbf5010f490 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009ca0 tx=0x7fbf4802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.221+0000 7fbf56207640 1 -- 172.21.15.3:0/687628548 shutdown_connections 2024-09-21T06:15:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.221+0000 7fbf56207640 1 --2- 172.21.15.3:0/687628548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf50108c60 0x7fbf5010f490 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.221+0000 7fbf56207640 1 --2- 172.21.15.3:0/687628548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf50108310 0x7fbf50108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.221+0000 7fbf56207640 1 --2- 172.21.15.3:0/687628548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf501029e0 0x7fbf50102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.221+0000 7fbf56207640 1 -- 172.21.15.3:0/687628548 >> 172.21.15.3:0/687628548 conn(0x7fbf500fe3e0 msgr2=0x7fbf50100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.221+0000 7fbf56207640 1 -- 172.21.15.3:0/687628548 shutdown_connections 2024-09-21T06:15:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.221+0000 7fbf56207640 1 -- 172.21.15.3:0/687628548 wait complete. 2024-09-21T06:15:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf56207640 1 Processor -- start 2024-09-21T06:15:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf56207640 1 -- start start 2024-09-21T06:15:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf56207640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf501029e0 0x7fbf5019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf56207640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf50108310 0x7fbf5019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf56207640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf50108c60 0x7fbf5019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf56207640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbf50112920 con 0x7fbf501029e0 2024-09-21T06:15:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf56207640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbf501127a0 con 0x7fbf50108310 2024-09-21T06:15:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf56207640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbf50112aa0 con 0x7fbf50108c60 2024-09-21T06:15:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf4f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf501029e0 0x7fbf5019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf4f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf501029e0 0x7fbf5019b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48158/0 (socket says 172.21.15.3:48158) 2024-09-21T06:15:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf4f7fe640 1 -- 172.21.15.3:0/1740677932 learned_addr learned my addr 172.21.15.3:0/1740677932 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf47fff640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf50108310 0x7fbf5019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf4ffff640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf50108c60 0x7fbf5019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf4f7fe640 1 -- 172.21.15.3:0/1740677932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf50108c60 msgr2=0x7fbf5019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf4f7fe640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf50108c60 0x7fbf5019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf4f7fe640 1 -- 172.21.15.3:0/1740677932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf50108310 msgr2=0x7fbf5019b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf4f7fe640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf50108310 0x7fbf5019b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf4f7fe640 1 -- 172.21.15.3:0/1740677932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf501a64e0 con 0x7fbf501029e0 2024-09-21T06:15:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf4ffff640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf50108c60 0x7fbf5019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.222+0000 7fbf47fff640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf50108310 0x7fbf5019b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.223+0000 7fbf4f7fe640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf501029e0 0x7fbf5019b250 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000d770 tx=0x7fbf4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.223+0000 7fbf4d7fa640 1 -- 172.21.15.3:0/1740677932 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40017070 con 0x7fbf501029e0 2024-09-21T06:15:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.223+0000 7fbf56207640 1 -- 172.21.15.3:0/1740677932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf501a67d0 con 0x7fbf501029e0 2024-09-21T06:15:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.223+0000 7fbf4d7fa640 1 -- 172.21.15.3:0/1740677932 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf400054d0 con 0x7fbf501029e0 2024-09-21T06:15:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.223+0000 7fbf4d7fa640 1 -- 172.21.15.3:0/1740677932 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40004c30 con 0x7fbf501029e0 2024-09-21T06:15:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.223+0000 7fbf56207640 1 -- 172.21.15.3:0/1740677932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf501a6cd0 con 0x7fbf501029e0 2024-09-21T06:15:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.224+0000 7fbf56207640 1 -- 172.21.15.3:0/1740677932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf50102df0 con 0x7fbf501029e0 2024-09-21T06:15:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.226+0000 7fbf4d7fa640 1 -- 172.21.15.3:0/1740677932 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 154528+0+0 (secure 0 0 0) 0x7fbf40004dd0 con 0x7fbf501029e0 2024-09-21T06:15:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.226+0000 7fbf4d7fa640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbf3c0b6560 0x7fbf3c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.227+0000 7fbf47fff640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbf3c0b6560 0x7fbf3c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.227+0000 7fbf4d7fa640 1 -- 172.21.15.3:0/1740677932 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fbf400e73f0 con 0x7fbf501029e0 2024-09-21T06:15:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.227+0000 7fbf47fff640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbf3c0b6560 0x7fbf3c0b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbf50101bc0 tx=0x7fbf3800b5f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.229+0000 7fbf4d7fa640 1 -- 172.21.15.3:0/1740677932 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbf400adba0 con 0x7fbf501029e0 2024-09-21T06:15:08.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:08.459+0000 7fbf56207640 1 -- 172.21.15.3:0/1740677932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fbf5002e640 con 0x7fbf501029e0 2024-09-21T06:15:09.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.328+0000 7fbf4d7fa640 1 -- 172.21.15.3:0/1740677932 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7fbf400b1f80 con 0x7fbf501029e0 2024-09-21T06:15:09.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.331+0000 7fbf56207640 1 -- 172.21.15.3:0/1740677932 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbf3c0b6560 msgr2=0x7fbf3c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.331+0000 7fbf56207640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbf3c0b6560 0x7fbf3c0b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbf50101bc0 tx=0x7fbf3800b5f0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.331+0000 7fbf56207640 1 -- 172.21.15.3:0/1740677932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf501029e0 msgr2=0x7fbf5019b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.331+0000 7fbf56207640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf501029e0 0x7fbf5019b250 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000d770 tx=0x7fbf4000dc30 comp rx=0 tx=0).stop 2024-09-21T06:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.331+0000 7fbf56207640 1 -- 172.21.15.3:0/1740677932 shutdown_connections 2024-09-21T06:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.331+0000 7fbf56207640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbf3c0b6560 0x7fbf3c0b8a70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.332+0000 7fbf56207640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf50108c60 0x7fbf5019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.332+0000 7fbf56207640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf50108310 0x7fbf5019b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.332+0000 7fbf56207640 1 --2- 172.21.15.3:0/1740677932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf501029e0 0x7fbf5019b250 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.332+0000 7fbf56207640 1 -- 172.21.15.3:0/1740677932 >> 172.21.15.3:0/1740677932 conn(0x7fbf500fe3e0 msgr2=0x7fbf501058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.332+0000 7fbf56207640 1 -- 172.21.15.3:0/1740677932 shutdown_connections 2024-09-21T06:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.332+0000 7fbf56207640 1 -- 172.21.15.3:0/1740677932 wait complete. 2024-09-21T06:15:09.347 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-21T06:15:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.469+0000 7f9423e5c640 1 -- 172.21.15.3:0/4158492211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941c076100 msgr2=0x7f941c10fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.469+0000 7f9423e5c640 1 --2- 172.21.15.3:0/4158492211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941c076100 0x7f941c10fb10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f940c009900 tx=0x7f940c02a410 comp rx=0 tx=0).stop 2024-09-21T06:15:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.469+0000 7f9423e5c640 1 -- 172.21.15.3:0/4158492211 shutdown_connections 2024-09-21T06:15:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.469+0000 7f9423e5c640 1 --2- 172.21.15.3:0/4158492211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f941c110050 0x7f941c1124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.469+0000 7f9423e5c640 1 --2- 172.21.15.3:0/4158492211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941c076100 0x7f941c10fb10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.469+0000 7f9423e5c640 1 --2- 172.21.15.3:0/4158492211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f941c0757b0 0x7f941c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.469+0000 7f9423e5c640 1 -- 172.21.15.3:0/4158492211 >> 172.21.15.3:0/4158492211 conn(0x7f941c0ff500 msgr2=0x7f941c101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:09.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.469+0000 7f9423e5c640 1 -- 172.21.15.3:0/4158492211 shutdown_connections 2024-09-21T06:15:09.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.469+0000 7f9423e5c640 1 -- 172.21.15.3:0/4158492211 wait complete. 2024-09-21T06:15:09.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.470+0000 7f9423e5c640 1 Processor -- start 2024-09-21T06:15:09.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.470+0000 7f9423e5c640 1 -- start start 2024-09-21T06:15:09.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.470+0000 7f9423e5c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941c0757b0 0x7f941c19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:09.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.470+0000 7f9423e5c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f941c076100 0x7f941c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:09.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.470+0000 7f9423e5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f941c110050 0x7f941c1a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.470+0000 7f9423e5c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f941c115c20 con 0x7f941c0757b0 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.470+0000 7f9423e5c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f941c115aa0 con 0x7f941c110050 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.470+0000 7f9423e5c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f941c115da0 con 0x7f941c076100 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.470+0000 7f9421bf8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941c0757b0 0x7f941c19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.470+0000 7f9421bf8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941c0757b0 0x7f941c19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48166/0 (socket says 172.21.15.3:48166) 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.470+0000 7f9421bf8640 1 -- 172.21.15.3:0/1822152652 learned_addr learned my addr 172.21.15.3:0/1822152652 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.470+0000 7f94223f9640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f941c110050 0x7f941c1a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.471+0000 7f94213f7640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f941c076100 0x7f941c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.471+0000 7f9421bf8640 1 -- 172.21.15.3:0/1822152652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f941c076100 msgr2=0x7f941c19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.471+0000 7f9421bf8640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f941c076100 0x7f941c19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.471+0000 7f9421bf8640 1 -- 172.21.15.3:0/1822152652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f941c110050 msgr2=0x7f941c1a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.471+0000 7f9421bf8640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f941c110050 0x7f941c1a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.471+0000 7f9421bf8640 1 -- 172.21.15.3:0/1822152652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941c1a2570 con 0x7f941c0757b0 2024-09-21T06:15:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.471+0000 7f94223f9640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f941c110050 0x7f941c1a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.471+0000 7f9421bf8640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941c0757b0 0x7f941c19c640 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f941800d950 tx=0x7f941800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.471+0000 7f9412ffd640 1 -- 172.21.15.3:0/1822152652 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418017030 con 0x7f941c0757b0 2024-09-21T06:15:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.471+0000 7f9412ffd640 1 -- 172.21.15.3:0/1822152652 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9418004020 con 0x7f941c0757b0 2024-09-21T06:15:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.471+0000 7f9423e5c640 1 -- 172.21.15.3:0/1822152652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941c1a2860 con 0x7f941c0757b0 2024-09-21T06:15:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.471+0000 7f9412ffd640 1 -- 172.21.15.3:0/1822152652 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94180047a0 con 0x7f941c0757b0 2024-09-21T06:15:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.471+0000 7f9423e5c640 1 -- 172.21.15.3:0/1822152652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f941c1a8ff0 con 0x7f941c0757b0 2024-09-21T06:15:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.472+0000 7f9423e5c640 1 -- 172.21.15.3:0/1822152652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f941c075bc0 con 0x7f941c0757b0 2024-09-21T06:15:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.474+0000 7f9412ffd640 1 -- 172.21.15.3:0/1822152652 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 154528+0+0 (secure 0 0 0) 0x7f9418009a30 con 0x7f941c0757b0 2024-09-21T06:15:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.475+0000 7f9412ffd640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f94080b6560 0x7f94080b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.475+0000 7f94213f7640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f94080b6560 0x7f94080b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.475+0000 7f9412ffd640 1 -- 172.21.15.3:0/1822152652 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f94180e75d0 con 0x7f941c0757b0 2024-09-21T06:15:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.476+0000 7f94213f7640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f94080b6560 0x7f94080b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f940c009230 tx=0x7f940c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.478+0000 7f9412ffd640 1 -- 172.21.15.3:0/1822152652 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94180add80 con 0x7f941c0757b0 2024-09-21T06:15:09.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:09 smithi003 ceph-mon[31227]: pgmap v329: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T06:15:09.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:09 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1740677932' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T06:15:09.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:09 smithi137 ceph-mon[34464]: pgmap v329: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T06:15:09.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:09 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1740677932' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T06:15:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:09 smithi204 ceph-mon[34659]: pgmap v329: 97 pgs: 97 active+clean; 649 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T06:15:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:09 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1740677932' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T06:15:09.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.694+0000 7f9423e5c640 1 -- 172.21.15.3:0/1822152652 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f941c10a300 con 0x7f94080b6560 2024-09-21T06:15:09.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.723+0000 7f9412ffd640 1 -- 172.21.15.3:0/1822152652 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f941c10a300 con 0x7f94080b6560 2024-09-21T06:15:09.725 INFO:teuthology.orchestra.run.smithi003.stdout:{} 2024-09-21T06:15:09.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.725+0000 7f9423e5c640 1 -- 172.21.15.3:0/1822152652 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f94080b6560 msgr2=0x7f94080b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:09.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.725+0000 7f9423e5c640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f94080b6560 0x7f94080b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f940c009230 tx=0x7f940c031400 comp rx=0 tx=0).stop 2024-09-21T06:15:09.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.725+0000 7f9423e5c640 1 -- 172.21.15.3:0/1822152652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941c0757b0 msgr2=0x7f941c19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:09.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.725+0000 7f9423e5c640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941c0757b0 0x7f941c19c640 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f941800d950 tx=0x7f941800de10 comp rx=0 tx=0).stop 2024-09-21T06:15:09.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.725+0000 7f9423e5c640 1 -- 172.21.15.3:0/1822152652 shutdown_connections 2024-09-21T06:15:09.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.725+0000 7f9423e5c640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f94080b6560 0x7f94080b8a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.725+0000 7f9423e5c640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f941c076100 0x7f941c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.725+0000 7f9423e5c640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f941c110050 0x7f941c1a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.725+0000 7f9423e5c640 1 --2- 172.21.15.3:0/1822152652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941c0757b0 0x7f941c19c640 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.725+0000 7f9423e5c640 1 -- 172.21.15.3:0/1822152652 >> 172.21.15.3:0/1822152652 conn(0x7f941c0ff500 msgr2=0x7f941c109bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.725+0000 7f9423e5c640 1 -- 172.21.15.3:0/1822152652 shutdown_connections 2024-09-21T06:15:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.725+0000 7f9423e5c640 1 -- 172.21.15.3:0/1822152652 wait complete. 2024-09-21T06:15:09.738 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T06:15:09.748 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-21T06:15:09.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:15:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.870+0000 7fd785dcc640 1 -- 172.21.15.3:0/4255366477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7801029b0 msgr2=0x7fd780102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.870+0000 7fd785dcc640 1 --2- 172.21.15.3:0/4255366477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7801029b0 0x7fd780102dc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd770009900 tx=0x7fd77002a410 comp rx=0 tx=0).stop 2024-09-21T06:15:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.870+0000 7fd785dcc640 1 -- 172.21.15.3:0/4255366477 shutdown_connections 2024-09-21T06:15:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.870+0000 7fd785dcc640 1 --2- 172.21.15.3:0/4255366477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd780108c30 0x7fd78010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.870+0000 7fd785dcc640 1 --2- 172.21.15.3:0/4255366477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7801082e0 0x7fd7801086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.870+0000 7fd785dcc640 1 --2- 172.21.15.3:0/4255366477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7801029b0 0x7fd780102dc0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.870+0000 7fd785dcc640 1 -- 172.21.15.3:0/4255366477 >> 172.21.15.3:0/4255366477 conn(0x7fd7800fe3b0 msgr2=0x7fd780100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.870+0000 7fd785dcc640 1 -- 172.21.15.3:0/4255366477 shutdown_connections 2024-09-21T06:15:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.870+0000 7fd785dcc640 1 -- 172.21.15.3:0/4255366477 wait complete. 2024-09-21T06:15:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.871+0000 7fd785dcc640 1 Processor -- start 2024-09-21T06:15:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.871+0000 7fd785dcc640 1 -- start start 2024-09-21T06:15:09.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.871+0000 7fd785dcc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7801029b0 0x7fd780199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:09.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.871+0000 7fd785dcc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7801082e0 0x7fd780199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:09.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.871+0000 7fd785dcc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd780108c30 0x7fd78019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:09.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.871+0000 7fd785dcc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd780112930 con 0x7fd780108c30 2024-09-21T06:15:09.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.871+0000 7fd785dcc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd7801127b0 con 0x7fd7801029b0 2024-09-21T06:15:09.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.871+0000 7fd785dcc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd780112ab0 con 0x7fd7801082e0 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd77ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd780108c30 0x7fd78019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd77ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd780108c30 0x7fd78019ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48210/0 (socket says 172.21.15.3:48210) 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd77ffff640 1 -- 172.21.15.3:0/1813044399 learned_addr learned my addr 172.21.15.3:0/1813044399 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd77f7fe640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7801029b0 0x7fd780199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd776ffd640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7801082e0 0x7fd780199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd77ffff640 1 -- 172.21.15.3:0/1813044399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7801082e0 msgr2=0x7fd780199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd77ffff640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7801082e0 0x7fd780199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd77ffff640 1 -- 172.21.15.3:0/1813044399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7801029b0 msgr2=0x7fd780199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd77ffff640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7801029b0 0x7fd780199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd77ffff640 1 -- 172.21.15.3:0/1813044399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78019f230 con 0x7fd780108c30 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd776ffd640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7801082e0 0x7fd780199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd77ffff640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd780108c30 0x7fd78019ebc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd77800cce0 tx=0x7fd77800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd77d7fa640 1 -- 172.21.15.3:0/1813044399 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77800f450 con 0x7fd780108c30 2024-09-21T06:15:09.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd77d7fa640 1 -- 172.21.15.3:0/1813044399 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd77800faa0 con 0x7fd780108c30 2024-09-21T06:15:09.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd785dcc640 1 -- 172.21.15.3:0/1813044399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78019f520 con 0x7fd780108c30 2024-09-21T06:15:09.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd77d7fa640 1 -- 172.21.15.3:0/1813044399 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778018b70 con 0x7fd780108c30 2024-09-21T06:15:09.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.872+0000 7fd785dcc640 1 -- 172.21.15.3:0/1813044399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7801a5d40 con 0x7fd780108c30 2024-09-21T06:15:09.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.873+0000 7fd785dcc640 1 -- 172.21.15.3:0/1813044399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd780102dc0 con 0x7fd780108c30 2024-09-21T06:15:09.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.875+0000 7fd77d7fa640 1 -- 172.21.15.3:0/1813044399 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 154528+0+0 (secure 0 0 0) 0x7fd778018d10 con 0x7fd780108c30 2024-09-21T06:15:09.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.876+0000 7fd77d7fa640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd76c0b6560 0x7fd76c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:09.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.876+0000 7fd77d7fa640 1 -- 172.21.15.3:0/1813044399 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fd7780e7980 con 0x7fd780108c30 2024-09-21T06:15:09.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.876+0000 7fd77f7fe640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd76c0b6560 0x7fd76c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:09.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.877+0000 7fd77f7fe640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd76c0b6560 0x7fd76c0b8a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd770031440 tx=0x7fd770005d10 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:09.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:09.879+0000 7fd77d7fa640 1 -- 172.21.15.3:0/1813044399 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd778014030 con 0x7fd780108c30 2024-09-21T06:15:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.068+0000 7fd785dcc640 1 -- 172.21.15.3:0/1813044399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd78002e640 con 0x7fd780108c30 2024-09-21T06:15:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.069+0000 7fd77d7fa640 1 -- 172.21.15.3:0/1813044399 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23048 (secure 0 0 0) 0x7fd7780ae1c0 con 0x7fd780108c30 2024-09-21T06:15:10.071 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:15:10.071 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","created":"2024-09-21T06:04:24.293786+0000","modified":"2024-09-21T06:13:24.594225+0000","last_up_change":"2024-09-21T06:12:17.781243+0000","last_in_change":"2024-09-21T06:11:54.727435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:08:26.015803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:13:07.084406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:13:09.149891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51736f81-f410-48ee-a8f3-121289607309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6803","nonce":615312328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6805","nonce":615312328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6809","nonce":615312328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6807","nonce":615312328}]},"public_addr":"172.21.15.3:6803/615312328","cluster_addr":"172.21.15.3:6805/615312328","heartbeat_back_addr":"172.21.15.3:6809/615312328","heartbeat_front_addr":"172.21.15.3:6807/615312328","state":["exists","up"]},{"osd":1,"uuid":"55696a37-34c1-482a-8561-7d08f0aa29fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6801","nonce":4150747115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6803","nonce":4150747115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6807","nonce":4150747115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6805","nonce":4150747115}]},"public_addr":"172.21.15.137:6801/4150747115","cluster_addr":"172.21.15.137:6803/4150747115","heartbeat_back_addr":"172.21.15.137:6807/4150747115","heartbeat_front_addr":"172.21.15.137:6805/4150747115","state":["exists","up"]},{"osd":2,"uuid":"8a81d34b-464b-46a2-8ef2-5223b5947004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6801","nonce":3392577077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6803","nonce":3392577077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6807","nonce":3392577077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6805","nonce":3392577077}]},"public_addr":"172.21.15.204:6801/3392577077","cluster_addr":"172.21.15.204:6803/3392577077","heartbeat_back_addr":"172.21.15.204:6807/3392577077","heartbeat_front_addr":"172.21.15.204:6805/3392577077","state":["exists","up"]},{"osd":3,"uuid":"94ab3938-7a70-44cb-99bf-9d2dedc4eb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6811","nonce":4293741635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6813","nonce":4293741635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6817","nonce":4293741635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6815","nonce":4293741635}]},"public_addr":"172.21.15.3:6811/4293741635","cluster_addr":"172.21.15.3:6813/4293741635","heartbeat_back_addr":"172.21.15.3:6817/4293741635","heartbeat_front_addr":"172.21.15.3:6815/4293741635","state":["exists","up"]},{"osd":4,"uuid":"38551535-d692-401c-aad0-bee38f3026d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6809","nonce":2608988785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6811","nonce":2608988785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6815","nonce":2608988785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6813","nonce":2608988785}]},"public_addr":"172.21.15.137:6809/2608988785","cluster_addr":"172.21.15.137:6811/2608988785","heartbeat_back_addr":"172.21.15.137:6815/2608988785","heartbeat_front_addr":"172.21.15.137:6813/2608988785","state":["exists","up"]},{"osd":5,"uuid":"3d4b785b-28d0-4654-a9b1-4c5c905f4bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6809","nonce":3709074044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6811","nonce":3709074044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6815","nonce":3709074044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6813","nonce":3709074044}]},"public_addr":"172.21.15.204:6809/3709074044","cluster_addr":"172.21.15.204:6811/3709074044","heartbeat_back_addr":"172.21.15.204:6815/3709074044","heartbeat_front_addr":"172.21.15.204:6813/3709074044","state":["exists","up"]},{"osd":6,"uuid":"ba21dbe9-ab43-4b7e-9446-2929ee874c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6819","nonce":3862646594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6821","nonce":3862646594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6825","nonce":3862646594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6823","nonce":3862646594}]},"public_addr":"172.21.15.3:6819/3862646594","cluster_addr":"172.21.15.3:6821/3862646594","heartbeat_back_addr":"172.21.15.3:6825/3862646594","heartbeat_front_addr":"172.21.15.3:6823/3862646594","state":["exists","up"]},{"osd":7,"uuid":"9610fe5c-93df-49f9-b8bd-5cb480e0871b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6817","nonce":2086822496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6819","nonce":2086822496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6823","nonce":2086822496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6821","nonce":2086822496}]},"public_addr":"172.21.15.137:6817/2086822496","cluster_addr":"172.21.15.137:6819/2086822496","heartbeat_back_addr":"172.21.15.137:6823/2086822496","heartbeat_front_addr":"172.21.15.137:6821/2086822496","state":["exists","up"]},{"osd":8,"uuid":"49f6e9b4-6d14-434e-8b42-8b7ffeb2a072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6817","nonce":153476798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6819","nonce":153476798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6823","nonce":153476798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6821","nonce":153476798}]},"public_addr":"172.21.15.204:6817/153476798","cluster_addr":"172.21.15.204:6819/153476798","heartbeat_back_addr":"172.21.15.204:6823/153476798","heartbeat_front_addr":"172.21.15.204:6821/153476798","state":["exists","up"]},{"osd":9,"uuid":"633f67a7-b759-4ea6-82ed-3c42b3fb70ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6827","nonce":1238491549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6829","nonce":1238491549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6833","nonce":1238491549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6831","nonce":1238491549}]},"public_addr":"172.21.15.3:6827/1238491549","cluster_addr":"172.21.15.3:6829/1238491549","heartbeat_back_addr":"172.21.15.3:6833/1238491549","heartbeat_front_addr":"172.21.15.3:6831/1238491549","state":["exists","up"]},{"osd":10,"uuid":"94b65a48-ee87-4e59-9296-a72a22787f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6825","nonce":3486552683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6827","nonce":3486552683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6831","nonce":3486552683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6829","nonce":3486552683}]},"public_addr":"172.21.15.137:6825/3486552683","cluster_addr":"172.21.15.137:6827/3486552683","heartbeat_back_addr":"172.21.15.137:6831/3486552683","heartbeat_front_addr":"172.21.15.137:6829/3486552683","state":["exists","up"]},{"osd":11,"uuid":"c5cc2850-9062-4b8e-beb3-bef0db30a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6825","nonce":1084308610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6827","nonce":1084308610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6831","nonce":1084308610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6829","nonce":1084308610}]},"public_addr":"172.21.15.204:6825/1084308610","cluster_addr":"172.21.15.204:6827/1084308610","heartbeat_back_addr":"172.21.15.204:6831/1084308610","heartbeat_front_addr":"172.21.15.204:6829/1084308610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:31.898745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.089965+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:22.367464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:48.543248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:14.687331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:41.243579+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:07.290227+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:32.930046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:59.197638+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:24.315219+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:50.333639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:12:16.453640+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1573374138":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1595702358":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/547897695":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/3673347975":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/141972779":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6800/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:6801/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/2363985440":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/2138752581":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1820734047":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/2929116087":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/45453677":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/2929116087":"2024-09-22T06:05:21.689191+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:15:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.071+0000 7fd785dcc640 1 -- 172.21.15.3:0/1813044399 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd76c0b6560 msgr2=0x7fd76c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.071+0000 7fd785dcc640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd76c0b6560 0x7fd76c0b8a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd770031440 tx=0x7fd770005d10 comp rx=0 tx=0).stop 2024-09-21T06:15:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.071+0000 7fd785dcc640 1 -- 172.21.15.3:0/1813044399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd780108c30 msgr2=0x7fd78019ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.071+0000 7fd785dcc640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd780108c30 0x7fd78019ebc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd77800cce0 tx=0x7fd77800a620 comp rx=0 tx=0).stop 2024-09-21T06:15:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.071+0000 7fd785dcc640 1 -- 172.21.15.3:0/1813044399 shutdown_connections 2024-09-21T06:15:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.071+0000 7fd785dcc640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd76c0b6560 0x7fd76c0b8a70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.071+0000 7fd785dcc640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7801082e0 0x7fd780199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.071+0000 7fd785dcc640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd780108c30 0x7fd78019ebc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.071+0000 7fd785dcc640 1 --2- 172.21.15.3:0/1813044399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7801029b0 0x7fd780199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.071+0000 7fd785dcc640 1 -- 172.21.15.3:0/1813044399 >> 172.21.15.3:0/1813044399 conn(0x7fd7800fe3b0 msgr2=0x7fd780107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.071+0000 7fd785dcc640 1 -- 172.21.15.3:0/1813044399 shutdown_connections 2024-09-21T06:15:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.071+0000 7fd785dcc640 1 -- 172.21.15.3:0/1813044399 wait complete. 2024-09-21T06:15:10.084 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:15:09 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:15:09.699+0000 7fc4decf8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:15:10.084 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:15:09 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:15:09.699+0000 7fc4decf8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:15:10.084 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:15:09 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:15:09.699+0000 7fc4decf8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:15:10.084 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:15:09 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:15:09.699+0000 7fc4decf8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:15:10.084 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:15:09 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:15:09.699+0000 7fc4decf8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:15:10.086 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:08:26.015803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T06:13:07.084406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T06:13:09.149891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:15:10.086 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:15:10.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.201+0000 7f50b9e8b640 1 -- 172.21.15.3:0/2936966691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50b4103ab0 msgr2=0x7f50b4103ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.201+0000 7f50b9e8b640 1 --2- 172.21.15.3:0/2936966691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50b4103ab0 0x7f50b4103ec0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009ca0 tx=0x7f50a402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.201+0000 7f50b9e8b640 1 -- 172.21.15.3:0/2936966691 shutdown_connections 2024-09-21T06:15:10.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.201+0000 7f50b9e8b640 1 --2- 172.21.15.3:0/2936966691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50b41093e0 0x7f50b41097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.201+0000 7f50b9e8b640 1 --2- 172.21.15.3:0/2936966691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4109d30 0x7f50b4110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.201+0000 7f50b9e8b640 1 --2- 172.21.15.3:0/2936966691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50b4103ab0 0x7f50b4103ec0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.201+0000 7f50b9e8b640 1 -- 172.21.15.3:0/2936966691 >> 172.21.15.3:0/2936966691 conn(0x7f50b40ff4b0 msgr2=0x7f50b4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:10.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.201+0000 7f50b9e8b640 1 -- 172.21.15.3:0/2936966691 shutdown_connections 2024-09-21T06:15:10.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.201+0000 7f50b9e8b640 1 -- 172.21.15.3:0/2936966691 wait complete. 2024-09-21T06:15:10.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.202+0000 7f50b9e8b640 1 Processor -- start 2024-09-21T06:15:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.202+0000 7f50b9e8b640 1 -- start start 2024-09-21T06:15:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.202+0000 7f50b9e8b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4103ab0 0x7f50b419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.202+0000 7f50b9e8b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50b41093e0 0x7f50b419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.202+0000 7f50b9e8b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50b4109d30 0x7f50b419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.202+0000 7f50b9e8b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f50b41139f0 con 0x7f50b4109d30 2024-09-21T06:15:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.202+0000 7f50b9e8b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f50b4113870 con 0x7f50b4103ab0 2024-09-21T06:15:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.202+0000 7f50b9e8b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50b4113b70 con 0x7f50b41093e0 2024-09-21T06:15:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.202+0000 7f50b3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50b4109d30 0x7f50b419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.202+0000 7f50b3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50b4109d30 0x7f50b419fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48240/0 (socket says 172.21.15.3:48240) 2024-09-21T06:15:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.202+0000 7f50b3fff640 1 -- 172.21.15.3:0/1159422860 learned_addr learned my addr 172.21.15.3:0/1159422860 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.202+0000 7f50b37fe640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4103ab0 0x7f50b419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.202+0000 7f50b2ffd640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50b41093e0 0x7f50b419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.203+0000 7f50b3fff640 1 -- 172.21.15.3:0/1159422860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50b41093e0 msgr2=0x7f50b419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.203+0000 7f50b3fff640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50b41093e0 0x7f50b419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.203+0000 7f50b3fff640 1 -- 172.21.15.3:0/1159422860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4103ab0 msgr2=0x7f50b419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.203+0000 7f50b3fff640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4103ab0 0x7f50b419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.203+0000 7f50b3fff640 1 -- 172.21.15.3:0/1159422860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b41a0340 con 0x7f50b4109d30 2024-09-21T06:15:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.203+0000 7f50b37fe640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4103ab0 0x7f50b419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.203+0000 7f50b3fff640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50b4109d30 0x7f50b419fc40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00d750 tx=0x7f50ac00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.203+0000 7f50b0ff9640 1 -- 172.21.15.3:0/1159422860 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac017070 con 0x7f50b4109d30 2024-09-21T06:15:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.203+0000 7f50b0ff9640 1 -- 172.21.15.3:0/1159422860 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50ac0054d0 con 0x7f50b4109d30 2024-09-21T06:15:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.203+0000 7f50b9e8b640 1 -- 172.21.15.3:0/1159422860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b41a0630 con 0x7f50b4109d30 2024-09-21T06:15:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.203+0000 7f50b0ff9640 1 -- 172.21.15.3:0/1159422860 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac004c60 con 0x7f50b4109d30 2024-09-21T06:15:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.203+0000 7f50b9e8b640 1 -- 172.21.15.3:0/1159422860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b41a6dc0 con 0x7f50b4109d30 2024-09-21T06:15:10.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.204+0000 7f50b9e8b640 1 -- 172.21.15.3:0/1159422860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b4104d20 con 0x7f50b4109d30 2024-09-21T06:15:10.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.206+0000 7f50b0ff9640 1 -- 172.21.15.3:0/1159422860 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 154528+0+0 (secure 0 0 0) 0x7f50ac009bd0 con 0x7f50b4109d30 2024-09-21T06:15:10.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.206+0000 7f50b0ff9640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50a00b65e0 0x7f50a00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.206+0000 7f50b37fe640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50a00b65e0 0x7f50a00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.207+0000 7f50b0ff9640 1 -- 172.21.15.3:0/1159422860 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f50ac0e6e20 con 0x7f50b4109d30 2024-09-21T06:15:10.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.207+0000 7f50b37fe640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50a00b65e0 0x7f50a00b8af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50a4006010 tx=0x7f50a4000f40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:10.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.211+0000 7f50b0ff9640 1 -- 172.21.15.3:0/1159422860 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50ac0ad5d0 con 0x7f50b4109d30 2024-09-21T06:15:10.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.401+0000 7f50b9e8b640 1 -- 172.21.15.3:0/1159422860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f50b402e640 con 0x7f50b4109d30 2024-09-21T06:15:10.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.401+0000 7f50b0ff9640 1 -- 172.21.15.3:0/1159422860 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f50ac0b19b0 con 0x7f50b4109d30 2024-09-21T06:15:10.403 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-21T06:15:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.404+0000 7f50b9e8b640 1 -- 172.21.15.3:0/1159422860 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50a00b65e0 msgr2=0x7f50a00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.404+0000 7f50b9e8b640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50a00b65e0 0x7f50a00b8af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50a4006010 tx=0x7f50a4000f40 comp rx=0 tx=0).stop 2024-09-21T06:15:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.404+0000 7f50b9e8b640 1 -- 172.21.15.3:0/1159422860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50b4109d30 msgr2=0x7f50b419fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.404+0000 7f50b9e8b640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50b4109d30 0x7f50b419fc40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00d750 tx=0x7f50ac00dc10 comp rx=0 tx=0).stop 2024-09-21T06:15:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.404+0000 7f50b9e8b640 1 -- 172.21.15.3:0/1159422860 shutdown_connections 2024-09-21T06:15:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.404+0000 7f50b9e8b640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50a00b65e0 0x7f50a00b8af0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.404+0000 7f50b9e8b640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50b41093e0 0x7f50b419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.404+0000 7f50b9e8b640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50b4109d30 0x7f50b419fc40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.404+0000 7f50b9e8b640 1 --2- 172.21.15.3:0/1159422860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4103ab0 0x7f50b419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.404+0000 7f50b9e8b640 1 -- 172.21.15.3:0/1159422860 >> 172.21.15.3:0/1159422860 conn(0x7f50b40ff4b0 msgr2=0x7f50b4108a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.404+0000 7f50b9e8b640 1 -- 172.21.15.3:0/1159422860 shutdown_connections 2024-09-21T06:15:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.404+0000 7f50b9e8b640 1 -- 172.21.15.3:0/1159422860 wait complete. 2024-09-21T06:15:10.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T06:15:10.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.535+0000 7f6ceef58640 1 -- 172.21.15.3:0/845226078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce8108c50 msgr2=0x7f6ce810f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.535+0000 7f6ceef58640 1 --2- 172.21.15.3:0/845226078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce8108c50 0x7f6ce810f480 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009ca0 tx=0x7f6ce002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.535+0000 7f6ceef58640 1 -- 172.21.15.3:0/845226078 shutdown_connections 2024-09-21T06:15:10.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.535+0000 7f6ceef58640 1 --2- 172.21.15.3:0/845226078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce8108c50 0x7f6ce810f480 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.535+0000 7f6ceef58640 1 --2- 172.21.15.3:0/845226078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce8108300 0x7f6ce8108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.535+0000 7f6ceef58640 1 --2- 172.21.15.3:0/845226078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce8102a20 0x7f6ce8102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.535+0000 7f6ceef58640 1 -- 172.21.15.3:0/845226078 >> 172.21.15.3:0/845226078 conn(0x7f6ce8079630 msgr2=0x7f6ce8079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:10.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.535+0000 7f6ceef58640 1 -- 172.21.15.3:0/845226078 shutdown_connections 2024-09-21T06:15:10.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.535+0000 7f6ceef58640 1 -- 172.21.15.3:0/845226078 wait complete. 2024-09-21T06:15:10.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.535+0000 7f6ceef58640 1 Processor -- start 2024-09-21T06:15:10.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceef58640 1 -- start start 2024-09-21T06:15:10.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceef58640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce8102a20 0x7f6ce819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceef58640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce8108300 0x7f6ce819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceef58640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce8108c50 0x7f6ce819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceef58640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce8113a30 con 0x7f6ce8102a20 2024-09-21T06:15:10.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceef58640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81138b0 con 0x7f6ce8108c50 2024-09-21T06:15:10.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceef58640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce8113bb0 con 0x7f6ce8108300 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceccf4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce8102a20 0x7f6ce819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceccf4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce8102a20 0x7f6ce819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48266/0 (socket says 172.21.15.3:48266) 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceccf4640 1 -- 172.21.15.3:0/847453618 learned_addr learned my addr 172.21.15.3:0/847453618 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ced4f5640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce8108c50 0x7f6ce819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ce7fff640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce8108300 0x7f6ce819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceccf4640 1 -- 172.21.15.3:0/847453618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce8108300 msgr2=0x7f6ce819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceccf4640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce8108300 0x7f6ce819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceccf4640 1 -- 172.21.15.3:0/847453618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce8108c50 msgr2=0x7f6ce819fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceccf4640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce8108c50 0x7f6ce819fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceccf4640 1 -- 172.21.15.3:0/847453618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce81a0280 con 0x7f6ce8102a20 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.536+0000 7f6ceccf4640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce8102a20 0x7f6ce819a350 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00e9f0 tx=0x7f6cdc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.537+0000 7f6ce5ffb640 1 -- 172.21.15.3:0/847453618 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00cd80 con 0x7f6ce8102a20 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.537+0000 7f6ce5ffb640 1 -- 172.21.15.3:0/847453618 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cdc0054d0 con 0x7f6ce8102a20 2024-09-21T06:15:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.537+0000 7f6ceef58640 1 -- 172.21.15.3:0/847453618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce81a0570 con 0x7f6ce8102a20 2024-09-21T06:15:10.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.537+0000 7f6ce5ffb640 1 -- 172.21.15.3:0/847453618 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc010690 con 0x7f6ce8102a20 2024-09-21T06:15:10.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.537+0000 7f6ceef58640 1 -- 172.21.15.3:0/847453618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce81a6d00 con 0x7f6ce8102a20 2024-09-21T06:15:10.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.538+0000 7f6ceef58640 1 -- 172.21.15.3:0/847453618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce8102e30 con 0x7f6ce8102a20 2024-09-21T06:15:10.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.539+0000 7f6ce5ffb640 1 -- 172.21.15.3:0/847453618 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 154528+0+0 (secure 0 0 0) 0x7f6cdc004e90 con 0x7f6ce8102a20 2024-09-21T06:15:10.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.541+0000 7f6ce5ffb640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6cd80b6560 0x7f6cd80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.541+0000 7f6ce7fff640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6cd80b6560 0x7f6cd80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.541+0000 7f6ce5ffb640 1 -- 172.21.15.3:0/847453618 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f6cdc0e7a00 con 0x7f6ce8102a20 2024-09-21T06:15:10.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.541+0000 7f6ce7fff640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6cd80b6560 0x7f6cd80b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4005fd0 tx=0x7f6cd4005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:10.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.545+0000 7f6ce5ffb640 1 -- 172.21.15.3:0/847453618 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6cdc0ae150 con 0x7f6ce8102a20 2024-09-21T06:15:10.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:10 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1740677932' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T06:15:10.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:10 smithi003 ceph-mon[31227]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:15:10.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:10 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1813044399' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:15:10.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:10 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1740677932' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T06:15:10.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:10 smithi137 ceph-mon[34464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:15:10.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:10 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1813044399' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:15:10.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:10 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1740677932' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T06:15:10.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:10 smithi204 ceph-mon[34659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:15:10.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:10 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1813044399' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:15:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.734+0000 7f6ceef58640 1 -- 172.21.15.3:0/847453618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6ce802e640 con 0x7f6ce8102a20 2024-09-21T06:15:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.735+0000 7f6ce5ffb640 1 -- 172.21.15.3:0/847453618 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7f6cdc0b2530 con 0x7f6ce8102a20 2024-09-21T06:15:10.737 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-21T06:15:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.737+0000 7f6ceef58640 1 -- 172.21.15.3:0/847453618 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6cd80b6560 msgr2=0x7f6cd80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.737+0000 7f6ceef58640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6cd80b6560 0x7f6cd80b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4005fd0 tx=0x7f6cd4005850 comp rx=0 tx=0).stop 2024-09-21T06:15:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.737+0000 7f6ceef58640 1 -- 172.21.15.3:0/847453618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce8102a20 msgr2=0x7f6ce819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.737+0000 7f6ceef58640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce8102a20 0x7f6ce819a350 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00e9f0 tx=0x7f6cdc00edf0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.737+0000 7f6ceef58640 1 -- 172.21.15.3:0/847453618 shutdown_connections 2024-09-21T06:15:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.737+0000 7f6ceef58640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6cd80b6560 0x7f6cd80b8a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.737+0000 7f6ceef58640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce8108300 0x7f6ce819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.737+0000 7f6ceef58640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ce8108c50 0x7f6ce819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.737+0000 7f6ceef58640 1 --2- 172.21.15.3:0/847453618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce8102a20 0x7f6ce819a350 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.737+0000 7f6ceef58640 1 -- 172.21.15.3:0/847453618 >> 172.21.15.3:0/847453618 conn(0x7f6ce8079630 msgr2=0x7f6ce8104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.737+0000 7f6ceef58640 1 -- 172.21.15.3:0/847453618 shutdown_connections 2024-09-21T06:15:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.737+0000 7f6ceef58640 1 -- 172.21.15.3:0/847453618 wait complete. 2024-09-21T06:15:10.751 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T06:15:10.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.868+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2239051658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2d41047c0 msgr2=0x7fe2d4104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.868+0000 7fe2d99b6640 1 --2- 172.21.15.3:0/2239051658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2d41047c0 0x7fe2d4104bd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009900 tx=0x7fe2c402a410 comp rx=0 tx=0).stop 2024-09-21T06:15:10.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.868+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2239051658 shutdown_connections 2024-09-21T06:15:10.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.868+0000 7fe2d99b6640 1 --2- 172.21.15.3:0/2239051658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2d41051a0 0x7fe2d410d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.868+0000 7fe2d99b6640 1 --2- 172.21.15.3:0/2239051658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d410dd70 0x7fe2d4110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.868+0000 7fe2d99b6640 1 --2- 172.21.15.3:0/2239051658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2d41047c0 0x7fe2d4104bd0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.868+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2239051658 >> 172.21.15.3:0/2239051658 conn(0x7fe2d40fe3b0 msgr2=0x7fe2d4100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:10.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.868+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2239051658 shutdown_connections 2024-09-21T06:15:10.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.868+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2239051658 wait complete. 2024-09-21T06:15:10.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d99b6640 1 Processor -- start 2024-09-21T06:15:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d99b6640 1 -- start start 2024-09-21T06:15:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d99b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d41047c0 0x7fe2d41026d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d99b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2d41051a0 0x7fe2d4102c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d99b6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2d410dd70 0x7fe2d4103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d99b6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41139b0 con 0x7fe2d410dd70 2024-09-21T06:15:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d99b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4113830 con 0x7fe2d41047c0 2024-09-21T06:15:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d99b6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4113b30 con 0x7fe2d41051a0 2024-09-21T06:15:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2d410dd70 0x7fe2d4103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2d410dd70 0x7fe2d4103150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48292/0 (socket says 172.21.15.3:48292) 2024-09-21T06:15:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d37fe640 1 -- 172.21.15.3:0/2502196613 learned_addr learned my addr 172.21.15.3:0/2502196613 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d27fc640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2d41051a0 0x7fe2d4102c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d2ffd640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d41047c0 0x7fe2d41026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d37fe640 1 -- 172.21.15.3:0/2502196613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2d41051a0 msgr2=0x7fe2d4102c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d37fe640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2d41051a0 0x7fe2d4102c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d37fe640 1 -- 172.21.15.3:0/2502196613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d41047c0 msgr2=0x7fe2d41026d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d37fe640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d41047c0 0x7fe2d41026d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.869+0000 7fe2d37fe640 1 -- 172.21.15.3:0/2502196613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d41a86c0 con 0x7fe2d410dd70 2024-09-21T06:15:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.870+0000 7fe2d37fe640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2d410dd70 0x7fe2d4103150 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00cce0 tx=0x7fe2cc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.870+0000 7fe2d89b4640 1 -- 172.21.15.3:0/2502196613 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc00f450 con 0x7fe2d410dd70 2024-09-21T06:15:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.870+0000 7fe2d89b4640 1 -- 172.21.15.3:0/2502196613 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2cc00faa0 con 0x7fe2d410dd70 2024-09-21T06:15:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.870+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2502196613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d41a89b0 con 0x7fe2d410dd70 2024-09-21T06:15:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.870+0000 7fe2d89b4640 1 -- 172.21.15.3:0/2502196613 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc018b70 con 0x7fe2d410dd70 2024-09-21T06:15:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.870+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2502196613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d41a8eb0 con 0x7fe2d410dd70 2024-09-21T06:15:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.871+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2502196613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d4104bd0 con 0x7fe2d410dd70 2024-09-21T06:15:10.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.874+0000 7fe2d89b4640 1 -- 172.21.15.3:0/2502196613 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 154528+0+0 (secure 0 0 0) 0x7fe2cc018d10 con 0x7fe2d410dd70 2024-09-21T06:15:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.874+0000 7fe2d89b4640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2bc0b6560 0x7fe2bc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.874+0000 7fe2d2ffd640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2bc0b6560 0x7fe2bc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.874+0000 7fe2d89b4640 1 -- 172.21.15.3:0/2502196613 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe2cc0e7990 con 0x7fe2d410dd70 2024-09-21T06:15:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.875+0000 7fe2d2ffd640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2bc0b6560 0x7fe2bc0b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4006010 tx=0x7fe2c4000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:10.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:10.877+0000 7fe2d89b4640 1 -- 172.21.15.3:0/2502196613 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe2cc014030 con 0x7fe2d410dd70 2024-09-21T06:15:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.065+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2502196613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe2d41a92a0 con 0x7fe2d410dd70 2024-09-21T06:15:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.065+0000 7fe2d89b4640 1 -- 172.21.15.3:0/2502196613 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7fe2cc0ae1d0 con 0x7fe2d410dd70 2024-09-21T06:15:11.067 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-21T06:15:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.068+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2502196613 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2bc0b6560 msgr2=0x7fe2bc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.068+0000 7fe2d99b6640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2bc0b6560 0x7fe2bc0b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4006010 tx=0x7fe2c4000bc0 comp rx=0 tx=0).stop 2024-09-21T06:15:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.068+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2502196613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2d410dd70 msgr2=0x7fe2d4103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.068+0000 7fe2d99b6640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2d410dd70 0x7fe2d4103150 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00cce0 tx=0x7fe2cc00a620 comp rx=0 tx=0).stop 2024-09-21T06:15:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.068+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2502196613 shutdown_connections 2024-09-21T06:15:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.068+0000 7fe2d99b6640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2bc0b6560 0x7fe2bc0b8a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.068+0000 7fe2d99b6640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2d41051a0 0x7fe2d4102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.068+0000 7fe2d99b6640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2d410dd70 0x7fe2d4103150 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.068+0000 7fe2d99b6640 1 --2- 172.21.15.3:0/2502196613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d41047c0 0x7fe2d41026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.068+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2502196613 >> 172.21.15.3:0/2502196613 conn(0x7fe2d40fe3b0 msgr2=0x7fe2d41078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.068+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2502196613 shutdown_connections 2024-09-21T06:15:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.068+0000 7fe2d99b6640 1 -- 172.21.15.3:0/2502196613 wait complete. 2024-09-21T06:15:11.081 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-21T06:15:11.082 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:15:11.338 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:11 smithi003 ceph-mon[31227]: pgmap v330: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T06:15:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:11 smithi003 ceph-mon[31227]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:11 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1159422860' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:15:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:11 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/847453618' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:15:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:11 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2502196613' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:15:11.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:11 smithi137 ceph-mon[34464]: pgmap v330: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T06:15:11.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:11 smithi137 ceph-mon[34464]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:11.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:11 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1159422860' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:15:11.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:11 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/847453618' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:15:11.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:11 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2502196613' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:15:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:11 smithi204 ceph-mon[34659]: pgmap v330: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T06:15:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:11 smithi204 ceph-mon[34659]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:11 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1159422860' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:15:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:11 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/847453618' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:15:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:11 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2502196613' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:15:11.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.845+0000 7fcd386b0640 1 -- 172.21.15.3:0/2329033212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd30103b80 msgr2=0x7fcd30103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:11.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.845+0000 7fcd386b0640 1 --2- 172.21.15.3:0/2329033212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd30103b80 0x7fcd30103f90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009900 tx=0x7fcd2c02a410 comp rx=0 tx=0).stop 2024-09-21T06:15:11.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.845+0000 7fcd386b0640 1 -- 172.21.15.3:0/2329033212 shutdown_connections 2024-09-21T06:15:11.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.845+0000 7fcd386b0640 1 --2- 172.21.15.3:0/2329033212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd30109e10 0x7fcd30110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:11.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.845+0000 7fcd386b0640 1 --2- 172.21.15.3:0/2329033212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd301094c0 0x7fcd301098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:11.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.845+0000 7fcd386b0640 1 --2- 172.21.15.3:0/2329033212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd30103b80 0x7fcd30103f90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:11.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.845+0000 7fcd386b0640 1 -- 172.21.15.3:0/2329033212 >> 172.21.15.3:0/2329033212 conn(0x7fcd300ff610 msgr2=0x7fcd30101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:11.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.845+0000 7fcd386b0640 1 -- 172.21.15.3:0/2329033212 shutdown_connections 2024-09-21T06:15:11.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.845+0000 7fcd386b0640 1 -- 172.21.15.3:0/2329033212 wait complete. 2024-09-21T06:15:11.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.846+0000 7fcd386b0640 1 Processor -- start 2024-09-21T06:15:11.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.846+0000 7fcd386b0640 1 -- start start 2024-09-21T06:15:11.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.846+0000 7fcd386b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd30103b80 0x7fcd3019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:11.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.846+0000 7fcd386b0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd301094c0 0x7fcd3019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:11.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.846+0000 7fcd386b0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd30109e10 0x7fcd3019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:11.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.846+0000 7fcd386b0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcd30113ad0 con 0x7fcd301094c0 2024-09-21T06:15:11.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.846+0000 7fcd386b0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd30113950 con 0x7fcd30103b80 2024-09-21T06:15:11.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.846+0000 7fcd386b0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd30113c50 con 0x7fcd30109e10 2024-09-21T06:15:11.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd35c4b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd301094c0 0x7fcd3019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:11.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd35c4b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd301094c0 0x7fcd3019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48322/0 (socket says 172.21.15.3:48322) 2024-09-21T06:15:11.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd35c4b640 1 -- 172.21.15.3:0/300455 learned_addr learned my addr 172.21.15.3:0/300455 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:11.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd3644c640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd30103b80 0x7fcd3019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:11.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd36c4d640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd30109e10 0x7fcd3019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:11.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd35c4b640 1 -- 172.21.15.3:0/300455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd30109e10 msgr2=0x7fcd3019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:11.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd35c4b640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd30109e10 0x7fcd3019fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:11.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd35c4b640 1 -- 172.21.15.3:0/300455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd30103b80 msgr2=0x7fcd3019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:11.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd35c4b640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd30103b80 0x7fcd3019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:11.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd35c4b640 1 -- 172.21.15.3:0/300455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd301a03f0 con 0x7fcd301094c0 2024-09-21T06:15:11.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd35c4b640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd301094c0 0x7fcd3019aa20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c970 tx=0x7fcd2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:11.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd277fe640 1 -- 172.21.15.3:0/300455 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20007bb0 con 0x7fcd301094c0 2024-09-21T06:15:11.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd277fe640 1 -- 172.21.15.3:0/300455 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd20007d50 con 0x7fcd301094c0 2024-09-21T06:15:11.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd277fe640 1 -- 172.21.15.3:0/300455 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2000f450 con 0x7fcd301094c0 2024-09-21T06:15:11.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd386b0640 1 -- 172.21.15.3:0/300455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd301a06e0 con 0x7fcd301094c0 2024-09-21T06:15:11.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.847+0000 7fcd386b0640 1 -- 172.21.15.3:0/300455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd301a6e40 con 0x7fcd301094c0 2024-09-21T06:15:11.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.849+0000 7fcd386b0640 1 -- 172.21.15.3:0/300455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd30103f90 con 0x7fcd301094c0 2024-09-21T06:15:11.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.850+0000 7fcd277fe640 1 -- 172.21.15.3:0/300455 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 154528+0+0 (secure 0 0 0) 0x7fcd20004eb0 con 0x7fcd301094c0 2024-09-21T06:15:11.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.851+0000 7fcd277fe640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcd1c0b6560 0x7fcd1c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:11.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.851+0000 7fcd3644c640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcd1c0b6560 0x7fcd1c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:11.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.851+0000 7fcd277fe640 1 -- 172.21.15.3:0/300455 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcd200e7100 con 0x7fcd301094c0 2024-09-21T06:15:11.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.851+0000 7fcd3644c640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcd1c0b6560 0x7fcd1c0b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c031440 tx=0x7fcd2c009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:11.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:11.855+0000 7fcd277fe640 1 -- 172.21.15.3:0/300455 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcd20014030 con 0x7fcd301094c0 2024-09-21T06:15:12.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.084+0000 7fcd386b0640 1 -- 172.21.15.3:0/300455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd3002e640 con 0x7fcd301094c0 2024-09-21T06:15:12.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.085+0000 7fcd277fe640 1 -- 172.21.15.3:0/300455 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11962 (secure 0 0 0) 0x7fcd200ad8b0 con 0x7fcd301094c0 2024-09-21T06:15:12.087 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:15:12.087 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"btime":"2024-09-21T06:15:09:322787+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:15:08.461776+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24670,24674,24679,24691,24704]},"id":1}]} 2024-09-21T06:15:12.087 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 28 2024-09-21T06:15:12.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.087+0000 7fcd386b0640 1 -- 172.21.15.3:0/300455 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcd1c0b6560 msgr2=0x7fcd1c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:12.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.087+0000 7fcd386b0640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcd1c0b6560 0x7fcd1c0b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c031440 tx=0x7fcd2c009560 comp rx=0 tx=0).stop 2024-09-21T06:15:12.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.087+0000 7fcd386b0640 1 -- 172.21.15.3:0/300455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd301094c0 msgr2=0x7fcd3019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:12.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.087+0000 7fcd386b0640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd301094c0 0x7fcd3019aa20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c970 tx=0x7fcd2000ce30 comp rx=0 tx=0).stop 2024-09-21T06:15:12.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.087+0000 7fcd386b0640 1 -- 172.21.15.3:0/300455 shutdown_connections 2024-09-21T06:15:12.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.087+0000 7fcd386b0640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcd1c0b6560 0x7fcd1c0b8a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:12.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.087+0000 7fcd386b0640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd30109e10 0x7fcd3019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:12.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.087+0000 7fcd386b0640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd301094c0 0x7fcd3019aa20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:12.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.087+0000 7fcd386b0640 1 --2- 172.21.15.3:0/300455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd30103b80 0x7fcd3019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:12.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.087+0000 7fcd386b0640 1 -- 172.21.15.3:0/300455 >> 172.21.15.3:0/300455 conn(0x7fcd300ff610 msgr2=0x7fcd30101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:12.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.087+0000 7fcd386b0640 1 -- 172.21.15.3:0/300455 shutdown_connections 2024-09-21T06:15:12.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.087+0000 7fcd386b0640 1 -- 172.21.15.3:0/300455 wait complete. 2024-09-21T06:15:12.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:12 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/300455' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:12.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:12 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/300455' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:12.775 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:12 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/300455' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:12.805 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-21T06:15:12.805 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-21T06:15:12.805 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:15:12.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.919+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3387831303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bf81050e0 msgr2=0x7f7bf81054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.919+0000 7f7bfec7e640 1 --2- 172.21.15.3:0/3387831303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bf81050e0 0x7f7bf81054f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009900 tx=0x7f7bec02a410 comp rx=0 tx=0).stop 2024-09-21T06:15:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.919+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3387831303 shutdown_connections 2024-09-21T06:15:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.919+0000 7f7bfec7e640 1 --2- 172.21.15.3:0/3387831303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf8105a30 0x7f7bf810f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.919+0000 7f7bfec7e640 1 --2- 172.21.15.3:0/3387831303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bf81050e0 0x7f7bf81054f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.919+0000 7f7bfec7e640 1 --2- 172.21.15.3:0/3387831303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bf8103120 0x7f7bf8103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.919+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3387831303 >> 172.21.15.3:0/3387831303 conn(0x7f7bf8079d00 msgr2=0x7f7bf807a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.919+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3387831303 shutdown_connections 2024-09-21T06:15:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.919+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3387831303 wait complete. 2024-09-21T06:15:12.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.920+0000 7f7bfec7e640 1 Processor -- start 2024-09-21T06:15:12.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.920+0000 7f7bfec7e640 1 -- start start 2024-09-21T06:15:12.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.920+0000 7f7bfec7e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bf8103120 0x7f7bf819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:12.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.920+0000 7f7bfec7e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bf81050e0 0x7f7bf819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:12.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.920+0000 7f7bfec7e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf8105a30 0x7f7bf819bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:12.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.920+0000 7f7bfec7e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf8112900 con 0x7f7bf8103120 2024-09-21T06:15:12.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.920+0000 7f7bfec7e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf8112780 con 0x7f7bf8105a30 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.920+0000 7f7bfec7e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf8112a80 con 0x7f7bf81050e0 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.920+0000 7f7bfca1a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bf8103120 0x7f7bf819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.920+0000 7f7bfca1a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bf8103120 0x7f7bf819b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48336/0 (socket says 172.21.15.3:48336) 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.920+0000 7f7bfca1a640 1 -- 172.21.15.3:0/3263053920 learned_addr learned my addr 172.21.15.3:0/3263053920 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.921+0000 7f7bf7fff640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bf81050e0 0x7f7bf819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.921+0000 7f7bfca1a640 1 -- 172.21.15.3:0/3263053920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bf81050e0 msgr2=0x7f7bf819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.921+0000 7f7bfca1a640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bf81050e0 0x7f7bf819b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.921+0000 7f7bfca1a640 1 -- 172.21.15.3:0/3263053920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf8105a30 msgr2=0x7f7bf819bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.921+0000 7f7bfca1a640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf8105a30 0x7f7bf819bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.921+0000 7f7bfca1a640 1 -- 172.21.15.3:0/3263053920 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf81a6510 con 0x7f7bf8103120 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.921+0000 7f7bf7fff640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bf81050e0 0x7f7bf819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.921+0000 7f7bfca1a640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bf8103120 0x7f7bf819b310 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c970 tx=0x7f7be800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.921+0000 7f7bf5ffb640 1 -- 172.21.15.3:0/3263053920 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8007c10 con 0x7f7bf8103120 2024-09-21T06:15:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.921+0000 7f7bf5ffb640 1 -- 172.21.15.3:0/3263053920 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be8007db0 con 0x7f7bf8103120 2024-09-21T06:15:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.921+0000 7f7bf5ffb640 1 -- 172.21.15.3:0/3263053920 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be800f870 con 0x7f7bf8103120 2024-09-21T06:15:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.921+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3263053920 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf81a6800 con 0x7f7bf8103120 2024-09-21T06:15:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.921+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3263053920 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf81a6da0 con 0x7f7bf8103120 2024-09-21T06:15:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.922+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3263053920 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf8103530 con 0x7f7bf8103120 2024-09-21T06:15:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.924+0000 7f7bf5ffb640 1 -- 172.21.15.3:0/3263053920 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 154579+0+0 (secure 0 0 0) 0x7f7be800fa10 con 0x7f7bf8103120 2024-09-21T06:15:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.924+0000 7f7bf5ffb640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7be00b6670 0x7f7be00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.924+0000 7f7bf7fff640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7be00b6670 0x7f7be00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.924+0000 7f7bf5ffb640 1 -- 172.21.15.3:0/3263053920 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f7be80e7a00 con 0x7f7bf8103120 2024-09-21T06:15:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.925+0000 7f7bf7fff640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7be00b6670 0x7f7be00b8b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7bec000c00 tx=0x7f7bec031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:12.929+0000 7f7bf5ffb640 1 -- 172.21.15.3:0/3263053920 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7be80ae080 con 0x7f7bf8103120 2024-09-21T06:15:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.113+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3263053920 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7bf8063800 con 0x7f7be00b6670 2024-09-21T06:15:13.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.118+0000 7f7bf5ffb640 1 -- 172.21.15.3:0/3263053920 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259148 (secure 0 0 0) 0x7f7bf8063800 con 0x7f7be00b6670 2024-09-21T06:15:13.123 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:15:13.124 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-21T06:15:13.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.122+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3263053920 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7be00b6670 msgr2=0x7f7be00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:13.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.122+0000 7f7bfec7e640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7be00b6670 0x7f7be00b8b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7bec000c00 tx=0x7f7bec031400 comp rx=0 tx=0).stop 2024-09-21T06:15:13.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.122+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3263053920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bf8103120 msgr2=0x7f7bf819b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:13.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.122+0000 7f7bfec7e640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bf8103120 0x7f7bf819b310 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c970 tx=0x7f7be800ce30 comp rx=0 tx=0).stop 2024-09-21T06:15:13.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.122+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3263053920 shutdown_connections 2024-09-21T06:15:13.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.122+0000 7f7bfec7e640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7be00b6670 0x7f7be00b8b80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.122+0000 7f7bfec7e640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bf81050e0 0x7f7bf819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.122+0000 7f7bfec7e640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf8105a30 0x7f7bf819bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.122+0000 7f7bfec7e640 1 --2- 172.21.15.3:0/3263053920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bf8103120 0x7f7bf819b310 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.122+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3263053920 >> 172.21.15.3:0/3263053920 conn(0x7f7bf8079d00 msgr2=0x7f7bf8108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:13.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.122+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3263053920 shutdown_connections 2024-09-21T06:15:13.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.122+0000 7f7bfec7e640 1 -- 172.21.15.3:0/3263053920 wait complete. 2024-09-21T06:15:13.135 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-21T06:15:11.659039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672711,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1190,"num_read_kb":715,"num_write":397,"num_write_kb":2321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2953564,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":827,"ondisk_log_size":827,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368300,"kb_used_data":10732,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573524,"statfs":{"total":1151940427776,"available":1151563288576,"internally_reserved":0,"allocated":10989568,"data_stored":7114976,"data_compressed":177687,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19139,"internal_metadata":365933885},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7876,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":78,"num_write":14,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011937"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.040716+0000","last_change":"2024-09-21T06:13:10.515115+0000","last_active":"2024-09-21T06:14:54.040716+0000","last_peered":"2024-09-21T06:14:54.040716+0000","last_clean":"2024-09-21T06:14:54.040716+0000","last_became_active":"2024-09-21T06:13:10.514553+0000","last_became_peered":"2024-09-21T06:13:10.514553+0000","last_unstale":"2024-09-21T06:14:54.040716+0000","last_undegraded":"2024-09-21T06:14:54.040716+0000","last_fullsized":"2024-09-21T06:14:54.040716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:33.268926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035807+0000","last_change":"2024-09-21T06:13:10.511469+0000","last_active":"2024-09-21T06:14:54.035807+0000","last_peered":"2024-09-21T06:14:54.035807+0000","last_clean":"2024-09-21T06:14:54.035807+0000","last_became_active":"2024-09-21T06:13:10.511102+0000","last_became_peered":"2024-09-21T06:13:10.511102+0000","last_unstale":"2024-09-21T06:14:54.035807+0000","last_undegraded":"2024-09-21T06:14:54.035807+0000","last_fullsized":"2024-09-21T06:14:54.035807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:12.483821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.040174+0000","last_change":"2024-09-21T06:13:10.520090+0000","last_active":"2024-09-21T06:14:54.040174+0000","last_peered":"2024-09-21T06:14:54.040174+0000","last_clean":"2024-09-21T06:14:54.040174+0000","last_became_active":"2024-09-21T06:13:10.519758+0000","last_became_peered":"2024-09-21T06:13:10.519758+0000","last_unstale":"2024-09-21T06:14:54.040174+0000","last_undegraded":"2024-09-21T06:14:54.040174+0000","last_fullsized":"2024-09-21T06:14:54.040174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:20.836847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.025079+0000","last_change":"2024-09-21T06:13:10.511732+0000","last_active":"2024-09-21T06:14:54.025079+0000","last_peered":"2024-09-21T06:14:54.025079+0000","last_clean":"2024-09-21T06:14:54.025079+0000","last_became_active":"2024-09-21T06:13:10.511377+0000","last_became_peered":"2024-09-21T06:13:10.511377+0000","last_unstale":"2024-09-21T06:14:54.025079+0000","last_undegraded":"2024-09-21T06:14:54.025079+0000","last_fullsized":"2024-09-21T06:14:54.025079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:18.500739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.023035+0000","last_change":"2024-09-21T06:13:10.510464+0000","last_active":"2024-09-21T06:14:54.023035+0000","last_peered":"2024-09-21T06:14:54.023035+0000","last_clean":"2024-09-21T06:14:54.023035+0000","last_became_active":"2024-09-21T06:13:10.509979+0000","last_became_peered":"2024-09-21T06:13:10.509979+0000","last_unstale":"2024-09-21T06:14:54.023035+0000","last_undegraded":"2024-09-21T06:14:54.023035+0000","last_fullsized":"2024-09-21T06:14:54.023035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:56.871043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.032702+0000","last_change":"2024-09-21T06:13:24.608832+0000","last_active":"2024-09-21T06:14:54.032702+0000","last_peered":"2024-09-21T06:14:54.032702+0000","last_clean":"2024-09-21T06:14:54.032702+0000","last_became_active":"2024-09-21T06:13:24.608439+0000","last_became_peered":"2024-09-21T06:13:24.608439+0000","last_unstale":"2024-09-21T06:14:54.032702+0000","last_undegraded":"2024-09-21T06:14:54.032702+0000","last_fullsized":"2024-09-21T06:14:54.032702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:41.515894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.041535+0000","last_change":"2024-09-21T06:13:10.511391+0000","last_active":"2024-09-21T06:14:54.041535+0000","last_peered":"2024-09-21T06:14:54.041535+0000","last_clean":"2024-09-21T06:14:54.041535+0000","last_became_active":"2024-09-21T06:13:10.510733+0000","last_became_peered":"2024-09-21T06:13:10.510733+0000","last_unstale":"2024-09-21T06:14:54.041535+0000","last_undegraded":"2024-09-21T06:14:54.041535+0000","last_fullsized":"2024-09-21T06:14:54.041535+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:24.071860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035636+0000","last_change":"2024-09-21T06:13:10.510886+0000","last_active":"2024-09-21T06:14:54.035636+0000","last_peered":"2024-09-21T06:14:54.035636+0000","last_clean":"2024-09-21T06:14:54.035636+0000","last_became_active":"2024-09-21T06:13:10.510159+0000","last_became_peered":"2024-09-21T06:13:10.510159+0000","last_unstale":"2024-09-21T06:14:54.035636+0000","last_undegraded":"2024-09-21T06:14:54.035636+0000","last_fullsized":"2024-09-21T06:14:54.035636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:37.124848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.045101+0000","last_change":"2024-09-21T06:13:10.510466+0000","last_active":"2024-09-21T06:14:54.045101+0000","last_peered":"2024-09-21T06:14:54.045101+0000","last_clean":"2024-09-21T06:14:54.045101+0000","last_became_active":"2024-09-21T06:13:10.509928+0000","last_became_peered":"2024-09-21T06:13:10.509928+0000","last_unstale":"2024-09-21T06:14:54.045101+0000","last_undegraded":"2024-09-21T06:14:54.045101+0000","last_fullsized":"2024-09-21T06:14:54.045101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:06.156505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.029437+0000","last_change":"2024-09-21T06:13:10.510622+0000","last_active":"2024-09-21T06:14:54.029437+0000","last_peered":"2024-09-21T06:14:54.029437+0000","last_clean":"2024-09-21T06:14:54.029437+0000","last_became_active":"2024-09-21T06:13:10.510206+0000","last_became_peered":"2024-09-21T06:13:10.510206+0000","last_unstale":"2024-09-21T06:14:54.029437+0000","last_undegraded":"2024-09-21T06:14:54.029437+0000","last_fullsized":"2024-09-21T06:14:54.029437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:03.223451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.042280+0000","last_change":"2024-09-21T06:13:10.517525+0000","last_active":"2024-09-21T06:14:54.042280+0000","last_peered":"2024-09-21T06:14:54.042280+0000","last_clean":"2024-09-21T06:14:54.042280+0000","last_became_active":"2024-09-21T06:13:10.517144+0000","last_became_peered":"2024-09-21T06:13:10.517144+0000","last_unstale":"2024-09-21T06:14:54.042280+0000","last_undegraded":"2024-09-21T06:14:54.042280+0000","last_fullsized":"2024-09-21T06:14:54.042280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:46.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037178+0000","last_change":"2024-09-21T06:13:10.515537+0000","last_active":"2024-09-21T06:14:54.037178+0000","last_peered":"2024-09-21T06:14:54.037178+0000","last_clean":"2024-09-21T06:14:54.037178+0000","last_became_active":"2024-09-21T06:13:10.515063+0000","last_became_peered":"2024-09-21T06:13:10.515063+0000","last_unstale":"2024-09-21T06:14:54.037178+0000","last_undegraded":"2024-09-21T06:14:54.037178+0000","last_fullsized":"2024-09-21T06:14:54.037178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:33.718471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.031442+0000","last_change":"2024-09-21T06:13:10.509368+0000","last_active":"2024-09-21T06:14:54.031442+0000","last_peered":"2024-09-21T06:14:54.031442+0000","last_clean":"2024-09-21T06:14:54.031442+0000","last_became_active":"2024-09-21T06:13:10.508982+0000","last_became_peered":"2024-09-21T06:13:10.508982+0000","last_unstale":"2024-09-21T06:14:54.031442+0000","last_undegraded":"2024-09-21T06:14:54.031442+0000","last_fullsized":"2024-09-21T06:14:54.031442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:54.430889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.036256+0000","last_change":"2024-09-21T06:13:10.506559+0000","last_active":"2024-09-21T06:14:54.036256+0000","last_peered":"2024-09-21T06:14:54.036256+0000","last_clean":"2024-09-21T06:14:54.036256+0000","last_became_active":"2024-09-21T06:13:10.506161+0000","last_became_peered":"2024-09-21T06:13:10.506161+0000","last_unstale":"2024-09-21T06:14:54.036256+0000","last_undegraded":"2024-09-21T06:14:54.036256+0000","last_fullsized":"2024-09-21T06:14:54.036256+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:19.870292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037955+0000","last_change":"2024-09-21T06:13:10.516000+0000","last_active":"2024-09-21T06:14:54.037955+0000","last_peered":"2024-09-21T06:14:54.037955+0000","last_clean":"2024-09-21T06:14:54.037955+0000","last_became_active":"2024-09-21T06:13:10.515543+0000","last_became_peered":"2024-09-21T06:13:10.515543+0000","last_unstale":"2024-09-21T06:14:54.037955+0000","last_undegraded":"2024-09-21T06:14:54.037955+0000","last_fullsized":"2024-09-21T06:14:54.037955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:48.968489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.045711+0000","last_change":"2024-09-21T06:13:10.513753+0000","last_active":"2024-09-21T06:14:54.045711+0000","last_peered":"2024-09-21T06:14:54.045711+0000","last_clean":"2024-09-21T06:14:54.045711+0000","last_became_active":"2024-09-21T06:13:10.511079+0000","last_became_peered":"2024-09-21T06:13:10.511079+0000","last_unstale":"2024-09-21T06:14:54.045711+0000","last_undegraded":"2024-09-21T06:14:54.045711+0000","last_fullsized":"2024-09-21T06:14:54.045711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:25.320041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.040850+0000","last_change":"2024-09-21T06:13:10.515222+0000","last_active":"2024-09-21T06:14:54.040850+0000","last_peered":"2024-09-21T06:14:54.040850+0000","last_clean":"2024-09-21T06:14:54.040850+0000","last_became_active":"2024-09-21T06:13:10.514604+0000","last_became_peered":"2024-09-21T06:13:10.514604+0000","last_unstale":"2024-09-21T06:14:54.040850+0000","last_undegraded":"2024-09-21T06:14:54.040850+0000","last_fullsized":"2024-09-21T06:14:54.040850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:03.165695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.038346+0000","last_change":"2024-09-21T06:13:10.518334+0000","last_active":"2024-09-21T06:14:54.038346+0000","last_peered":"2024-09-21T06:14:54.038346+0000","last_clean":"2024-09-21T06:14:54.038346+0000","last_became_active":"2024-09-21T06:13:10.517956+0000","last_became_peered":"2024-09-21T06:13:10.517956+0000","last_unstale":"2024-09-21T06:14:54.038346+0000","last_undegraded":"2024-09-21T06:14:54.038346+0000","last_fullsized":"2024-09-21T06:14:54.038346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:05.094454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.029105+0000","last_change":"2024-09-21T06:13:10.508763+0000","last_active":"2024-09-21T06:14:54.029105+0000","last_peered":"2024-09-21T06:14:54.029105+0000","last_clean":"2024-09-21T06:14:54.029105+0000","last_became_active":"2024-09-21T06:13:10.508371+0000","last_became_peered":"2024-09-21T06:13:10.508371+0000","last_unstale":"2024-09-21T06:14:54.029105+0000","last_undegraded":"2024-09-21T06:14:54.029105+0000","last_fullsized":"2024-09-21T06:14:54.029105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:50.602692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.038111+0000","last_change":"2024-09-21T06:13:10.503465+0000","last_active":"2024-09-21T06:14:54.038111+0000","last_peered":"2024-09-21T06:14:54.038111+0000","last_clean":"2024-09-21T06:14:54.038111+0000","last_became_active":"2024-09-21T06:13:10.503072+0000","last_became_peered":"2024-09-21T06:13:10.503072+0000","last_unstale":"2024-09-21T06:14:54.038111+0000","last_undegraded":"2024-09-21T06:14:54.038111+0000","last_fullsized":"2024-09-21T06:14:54.038111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:36.368987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.032321+0000","last_change":"2024-09-21T06:13:10.512841+0000","last_active":"2024-09-21T06:14:54.032321+0000","last_peered":"2024-09-21T06:14:54.032321+0000","last_clean":"2024-09-21T06:14:54.032321+0000","last_became_active":"2024-09-21T06:13:10.512407+0000","last_became_peered":"2024-09-21T06:13:10.512407+0000","last_unstale":"2024-09-21T06:14:54.032321+0000","last_undegraded":"2024-09-21T06:14:54.032321+0000","last_fullsized":"2024-09-21T06:14:54.032321+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:09.855608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.045911+0000","last_change":"2024-09-21T06:13:10.510088+0000","last_active":"2024-09-21T06:14:54.045911+0000","last_peered":"2024-09-21T06:14:54.045911+0000","last_clean":"2024-09-21T06:14:54.045911+0000","last_became_active":"2024-09-21T06:13:10.507904+0000","last_became_peered":"2024-09-21T06:13:10.507904+0000","last_unstale":"2024-09-21T06:14:54.045911+0000","last_undegraded":"2024-09-21T06:14:54.045911+0000","last_fullsized":"2024-09-21T06:14:54.045911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:49.404402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.033659+0000","last_change":"2024-09-21T06:13:10.517720+0000","last_active":"2024-09-21T06:14:54.033659+0000","last_peered":"2024-09-21T06:14:54.033659+0000","last_clean":"2024-09-21T06:14:54.033659+0000","last_became_active":"2024-09-21T06:13:10.514641+0000","last_became_peered":"2024-09-21T06:13:10.514641+0000","last_unstale":"2024-09-21T06:14:54.033659+0000","last_undegraded":"2024-09-21T06:14:54.033659+0000","last_fullsized":"2024-09-21T06:14:54.033659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:40.639665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.034092+0000","last_change":"2024-09-21T06:13:10.512234+0000","last_active":"2024-09-21T06:14:54.034092+0000","last_peered":"2024-09-21T06:14:54.034092+0000","last_clean":"2024-09-21T06:14:54.034092+0000","last_became_active":"2024-09-21T06:13:10.511594+0000","last_became_peered":"2024-09-21T06:13:10.511594+0000","last_unstale":"2024-09-21T06:14:54.034092+0000","last_undegraded":"2024-09-21T06:14:54.034092+0000","last_fullsized":"2024-09-21T06:14:54.034092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:04.410342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.030782+0000","last_change":"2024-09-21T06:13:10.513216+0000","last_active":"2024-09-21T06:14:54.030782+0000","last_peered":"2024-09-21T06:14:54.030782+0000","last_clean":"2024-09-21T06:14:54.030782+0000","last_became_active":"2024-09-21T06:13:10.512830+0000","last_became_peered":"2024-09-21T06:13:10.512830+0000","last_unstale":"2024-09-21T06:14:54.030782+0000","last_undegraded":"2024-09-21T06:14:54.030782+0000","last_fullsized":"2024-09-21T06:14:54.030782+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:46.418161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.030499+0000","last_change":"2024-09-21T06:13:10.518863+0000","last_active":"2024-09-21T06:14:54.030499+0000","last_peered":"2024-09-21T06:14:54.030499+0000","last_clean":"2024-09-21T06:14:54.030499+0000","last_became_active":"2024-09-21T06:13:10.518492+0000","last_became_peered":"2024-09-21T06:13:10.518492+0000","last_unstale":"2024-09-21T06:14:54.030499+0000","last_undegraded":"2024-09-21T06:14:54.030499+0000","last_fullsized":"2024-09-21T06:14:54.030499+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:23.770763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.031560+0000","last_change":"2024-09-21T06:13:10.508538+0000","last_active":"2024-09-21T06:14:54.031560+0000","last_peered":"2024-09-21T06:14:54.031560+0000","last_clean":"2024-09-21T06:14:54.031560+0000","last_became_active":"2024-09-21T06:13:10.508017+0000","last_became_peered":"2024-09-21T06:13:10.508017+0000","last_unstale":"2024-09-21T06:14:54.031560+0000","last_undegraded":"2024-09-21T06:14:54.031560+0000","last_fullsized":"2024-09-21T06:14:54.031560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:00.168741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.026465+0000","last_change":"2024-09-21T06:13:10.516734+0000","last_active":"2024-09-21T06:14:54.026465+0000","last_peered":"2024-09-21T06:14:54.026465+0000","last_clean":"2024-09-21T06:14:54.026465+0000","last_became_active":"2024-09-21T06:13:10.516362+0000","last_became_peered":"2024-09-21T06:13:10.516362+0000","last_unstale":"2024-09-21T06:14:54.026465+0000","last_undegraded":"2024-09-21T06:14:54.026465+0000","last_fullsized":"2024-09-21T06:14:54.026465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:49.792011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.036688+0000","last_change":"2024-09-21T06:13:10.513143+0000","last_active":"2024-09-21T06:14:54.036688+0000","last_peered":"2024-09-21T06:14:54.036688+0000","last_clean":"2024-09-21T06:14:54.036688+0000","last_became_active":"2024-09-21T06:13:10.512338+0000","last_became_peered":"2024-09-21T06:13:10.512338+0000","last_unstale":"2024-09-21T06:14:54.036688+0000","last_undegraded":"2024-09-21T06:14:54.036688+0000","last_fullsized":"2024-09-21T06:14:54.036688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:02.963860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.022101+0000","last_change":"2024-09-21T06:13:10.513429+0000","last_active":"2024-09-21T06:14:54.022101+0000","last_peered":"2024-09-21T06:14:54.022101+0000","last_clean":"2024-09-21T06:14:54.022101+0000","last_became_active":"2024-09-21T06:13:10.512995+0000","last_became_peered":"2024-09-21T06:13:10.512995+0000","last_unstale":"2024-09-21T06:14:54.022101+0000","last_undegraded":"2024-09-21T06:14:54.022101+0000","last_fullsized":"2024-09-21T06:14:54.022101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:39.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.032522+0000","last_change":"2024-09-21T06:13:10.509580+0000","last_active":"2024-09-21T06:14:54.032522+0000","last_peered":"2024-09-21T06:14:54.032522+0000","last_clean":"2024-09-21T06:14:54.032522+0000","last_became_active":"2024-09-21T06:13:10.509179+0000","last_became_peered":"2024-09-21T06:13:10.509179+0000","last_unstale":"2024-09-21T06:14:54.032522+0000","last_undegraded":"2024-09-21T06:14:54.032522+0000","last_fullsized":"2024-09-21T06:14:54.032522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:39.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.036310+0000","last_change":"2024-09-21T06:13:10.508358+0000","last_active":"2024-09-21T06:14:54.036310+0000","last_peered":"2024-09-21T06:14:54.036310+0000","last_clean":"2024-09-21T06:14:54.036310+0000","last_became_active":"2024-09-21T06:13:10.507952+0000","last_became_peered":"2024-09-21T06:13:10.507952+0000","last_unstale":"2024-09-21T06:14:54.036310+0000","last_undegraded":"2024-09-21T06:14:54.036310+0000","last_fullsized":"2024-09-21T06:14:54.036310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:55.565203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.024319+0000","last_change":"2024-09-21T06:13:10.510580+0000","last_active":"2024-09-21T06:14:54.024319+0000","last_peered":"2024-09-21T06:14:54.024319+0000","last_clean":"2024-09-21T06:14:54.024319+0000","last_became_active":"2024-09-21T06:13:10.510175+0000","last_became_peered":"2024-09-21T06:13:10.510175+0000","last_unstale":"2024-09-21T06:14:54.024319+0000","last_undegraded":"2024-09-21T06:14:54.024319+0000","last_fullsized":"2024-09-21T06:14:54.024319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:49.467057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'8","reported_seq":141,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:07.766031+0000","last_change":"2024-09-21T06:13:08.485527+0000","last_active":"2024-09-21T06:15:07.766031+0000","last_peered":"2024-09-21T06:15:07.766031+0000","last_clean":"2024-09-21T06:15:07.766031+0000","last_became_active":"2024-09-21T06:13:08.485094+0000","last_became_peered":"2024-09-21T06:13:08.485094+0000","last_unstale":"2024-09-21T06:15:07.766031+0000","last_undegraded":"2024-09-21T06:15:07.766031+0000","last_fullsized":"2024-09-21T06:15:07.766031+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:48.285638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.034456+0000","last_change":"2024-09-21T06:13:10.507977+0000","last_active":"2024-09-21T06:14:54.034456+0000","last_peered":"2024-09-21T06:14:54.034456+0000","last_clean":"2024-09-21T06:14:54.034456+0000","last_became_active":"2024-09-21T06:13:10.507594+0000","last_became_peered":"2024-09-21T06:13:10.507594+0000","last_unstale":"2024-09-21T06:14:54.034456+0000","last_undegraded":"2024-09-21T06:14:54.034456+0000","last_fullsized":"2024-09-21T06:14:54.034456+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:57.318240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.599690+0000","last_change":"2024-09-21T06:13:08.491237+0000","last_active":"2024-09-21T06:13:24.599690+0000","last_peered":"2024-09-21T06:13:24.599690+0000","last_clean":"2024-09-21T06:13:24.599690+0000","last_became_active":"2024-09-21T06:13:08.490908+0000","last_became_peered":"2024-09-21T06:13:08.490908+0000","last_unstale":"2024-09-21T06:13:24.599690+0000","last_undegraded":"2024-09-21T06:13:24.599690+0000","last_fullsized":"2024-09-21T06:13:24.599690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:53.185029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597228+0000","last_change":"2024-09-21T06:13:08.488721+0000","last_active":"2024-09-21T06:13:24.597228+0000","last_peered":"2024-09-21T06:13:24.597228+0000","last_clean":"2024-09-21T06:13:24.597228+0000","last_became_active":"2024-09-21T06:13:08.488347+0000","last_became_peered":"2024-09-21T06:13:08.488347+0000","last_unstale":"2024-09-21T06:13:24.597228+0000","last_undegraded":"2024-09-21T06:13:24.597228+0000","last_fullsized":"2024-09-21T06:13:24.597228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:27.008479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.034347+0000","last_change":"2024-09-21T06:13:10.506527+0000","last_active":"2024-09-21T06:14:54.034347+0000","last_peered":"2024-09-21T06:14:54.034347+0000","last_clean":"2024-09-21T06:14:54.034347+0000","last_became_active":"2024-09-21T06:13:10.506163+0000","last_became_peered":"2024-09-21T06:13:10.506163+0000","last_unstale":"2024-09-21T06:14:54.034347+0000","last_undegraded":"2024-09-21T06:14:54.034347+0000","last_fullsized":"2024-09-21T06:14:54.034347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.244097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604150+0000","last_change":"2024-09-21T06:13:08.489292+0000","last_active":"2024-09-21T06:13:24.604150+0000","last_peered":"2024-09-21T06:13:24.604150+0000","last_clean":"2024-09-21T06:13:24.604150+0000","last_became_active":"2024-09-21T06:13:08.488918+0000","last_became_peered":"2024-09-21T06:13:08.488918+0000","last_unstale":"2024-09-21T06:13:24.604150+0000","last_undegraded":"2024-09-21T06:13:24.604150+0000","last_fullsized":"2024-09-21T06:13:24.604150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:37.992260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.036458+0000","last_change":"2024-09-21T06:13:10.505027+0000","last_active":"2024-09-21T06:14:54.036458+0000","last_peered":"2024-09-21T06:14:54.036458+0000","last_clean":"2024-09-21T06:14:54.036458+0000","last_became_active":"2024-09-21T06:13:10.504669+0000","last_became_peered":"2024-09-21T06:13:10.504669+0000","last_unstale":"2024-09-21T06:14:54.036458+0000","last_undegraded":"2024-09-21T06:14:54.036458+0000","last_fullsized":"2024-09-21T06:14:54.036458+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:43.927229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'8","reported_seq":246,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:10.770506+0000","last_change":"2024-09-21T06:13:08.491101+0000","last_active":"2024-09-21T06:15:10.770506+0000","last_peered":"2024-09-21T06:15:10.770506+0000","last_clean":"2024-09-21T06:15:10.770506+0000","last_became_active":"2024-09-21T06:13:08.490607+0000","last_became_peered":"2024-09-21T06:13:08.490607+0000","last_unstale":"2024-09-21T06:15:10.770506+0000","last_undegraded":"2024-09-21T06:15:10.770506+0000","last_fullsized":"2024-09-21T06:15:10.770506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:16.758463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":215,"num_read_kb":215,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.043733+0000","last_change":"2024-09-21T06:13:10.510453+0000","last_active":"2024-09-21T06:14:54.043733+0000","last_peered":"2024-09-21T06:14:54.043733+0000","last_clean":"2024-09-21T06:14:54.043733+0000","last_became_active":"2024-09-21T06:13:10.508513+0000","last_became_peered":"2024-09-21T06:13:10.508513+0000","last_unstale":"2024-09-21T06:14:54.043733+0000","last_undegraded":"2024-09-21T06:14:54.043733+0000","last_fullsized":"2024-09-21T06:14:54.043733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:48.388140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609878+0000","last_change":"2024-09-21T06:13:08.488714+0000","last_active":"2024-09-21T06:13:24.609878+0000","last_peered":"2024-09-21T06:13:24.609878+0000","last_clean":"2024-09-21T06:13:24.609878+0000","last_became_active":"2024-09-21T06:13:08.488363+0000","last_became_peered":"2024-09-21T06:13:08.488363+0000","last_unstale":"2024-09-21T06:13:24.609878+0000","last_undegraded":"2024-09-21T06:13:24.609878+0000","last_fullsized":"2024-09-21T06:13:24.609878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:33.626165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037460+0000","last_change":"2024-09-21T06:13:10.508046+0000","last_active":"2024-09-21T06:14:54.037460+0000","last_peered":"2024-09-21T06:14:54.037460+0000","last_clean":"2024-09-21T06:14:54.037460+0000","last_became_active":"2024-09-21T06:13:10.507660+0000","last_became_peered":"2024-09-21T06:13:10.507660+0000","last_unstale":"2024-09-21T06:14:54.037460+0000","last_undegraded":"2024-09-21T06:14:54.037460+0000","last_fullsized":"2024-09-21T06:14:54.037460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:35.466965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603272+0000","last_change":"2024-09-21T06:13:08.562863+0000","last_active":"2024-09-21T06:13:24.603272+0000","last_peered":"2024-09-21T06:13:24.603272+0000","last_clean":"2024-09-21T06:13:24.603272+0000","last_became_active":"2024-09-21T06:13:08.562424+0000","last_became_peered":"2024-09-21T06:13:08.562424+0000","last_unstale":"2024-09-21T06:13:24.603272+0000","last_undegraded":"2024-09-21T06:13:24.603272+0000","last_fullsized":"2024-09-21T06:13:24.603272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:43.251932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.044625+0000","last_change":"2024-09-21T06:13:10.509818+0000","last_active":"2024-09-21T06:14:54.044625+0000","last_peered":"2024-09-21T06:14:54.044625+0000","last_clean":"2024-09-21T06:14:54.044625+0000","last_became_active":"2024-09-21T06:13:10.509468+0000","last_became_peered":"2024-09-21T06:13:10.509468+0000","last_unstale":"2024-09-21T06:14:54.044625+0000","last_undegraded":"2024-09-21T06:14:54.044625+0000","last_fullsized":"2024-09-21T06:14:54.044625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:25.130868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'38","reported_seq":307,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:10.770919+0000","last_change":"2024-09-21T06:13:24.633807+0000","last_active":"2024-09-21T06:15:10.770919+0000","last_peered":"2024-09-21T06:15:10.770919+0000","last_clean":"2024-09-21T06:15:10.770919+0000","last_became_active":"2024-09-21T06:13:24.618599+0000","last_became_peered":"2024-09-21T06:13:24.618599+0000","last_unstale":"2024-09-21T06:15:10.770919+0000","last_undegraded":"2024-09-21T06:15:10.770919+0000","last_fullsized":"2024-09-21T06:15:10.770919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:07.449253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73783,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":225,"num_read_kb":181,"num_write":40,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.033514+0000","last_change":"2024-09-21T06:13:10.511360+0000","last_active":"2024-09-21T06:14:54.033514+0000","last_peered":"2024-09-21T06:14:54.033514+0000","last_clean":"2024-09-21T06:14:54.033514+0000","last_became_active":"2024-09-21T06:13:10.510980+0000","last_became_peered":"2024-09-21T06:13:10.510980+0000","last_unstale":"2024-09-21T06:14:54.033514+0000","last_undegraded":"2024-09-21T06:14:54.033514+0000","last_fullsized":"2024-09-21T06:14:54.033514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:43.859758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604427+0000","last_change":"2024-09-21T06:13:08.486262+0000","last_active":"2024-09-21T06:13:24.604427+0000","last_peered":"2024-09-21T06:13:24.604427+0000","last_clean":"2024-09-21T06:13:24.604427+0000","last_became_active":"2024-09-21T06:13:08.485509+0000","last_became_peered":"2024-09-21T06:13:08.485509+0000","last_unstale":"2024-09-21T06:13:24.604427+0000","last_undegraded":"2024-09-21T06:13:24.604427+0000","last_fullsized":"2024-09-21T06:13:24.604427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:31.726746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.033404+0000","last_change":"2024-09-21T06:13:10.506894+0000","last_active":"2024-09-21T06:14:54.033404+0000","last_peered":"2024-09-21T06:14:54.033404+0000","last_clean":"2024-09-21T06:14:54.033404+0000","last_became_active":"2024-09-21T06:13:10.506489+0000","last_became_peered":"2024-09-21T06:13:10.506489+0000","last_unstale":"2024-09-21T06:14:54.033404+0000","last_undegraded":"2024-09-21T06:14:54.033404+0000","last_fullsized":"2024-09-21T06:14:54.033404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:58.755601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603141+0000","last_change":"2024-09-21T06:13:08.483429+0000","last_active":"2024-09-21T06:13:24.603141+0000","last_peered":"2024-09-21T06:13:24.603141+0000","last_clean":"2024-09-21T06:13:24.603141+0000","last_became_active":"2024-09-21T06:13:08.483028+0000","last_became_peered":"2024-09-21T06:13:08.483028+0000","last_unstale":"2024-09-21T06:13:24.603141+0000","last_undegraded":"2024-09-21T06:13:24.603141+0000","last_fullsized":"2024-09-21T06:13:24.603141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:33.258154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.042922+0000","last_change":"2024-09-21T06:13:10.513398+0000","last_active":"2024-09-21T06:14:54.042922+0000","last_peered":"2024-09-21T06:14:54.042922+0000","last_clean":"2024-09-21T06:14:54.042922+0000","last_became_active":"2024-09-21T06:13:10.512713+0000","last_became_peered":"2024-09-21T06:13:10.512713+0000","last_unstale":"2024-09-21T06:14:54.042922+0000","last_undegraded":"2024-09-21T06:14:54.042922+0000","last_fullsized":"2024-09-21T06:14:54.042922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:21.026984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'7","reported_seq":137,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:05.762659+0000","last_change":"2024-09-21T06:13:08.493771+0000","last_active":"2024-09-21T06:15:05.762659+0000","last_peered":"2024-09-21T06:15:05.762659+0000","last_clean":"2024-09-21T06:15:05.762659+0000","last_became_active":"2024-09-21T06:13:08.493380+0000","last_became_peered":"2024-09-21T06:13:08.493380+0000","last_unstale":"2024-09-21T06:15:05.762659+0000","last_undegraded":"2024-09-21T06:15:05.762659+0000","last_fullsized":"2024-09-21T06:15:05.762659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:12.552137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.041365+0000","last_change":"2024-09-21T06:13:10.511679+0000","last_active":"2024-09-21T06:14:54.041365+0000","last_peered":"2024-09-21T06:14:54.041365+0000","last_clean":"2024-09-21T06:14:54.041365+0000","last_became_active":"2024-09-21T06:13:10.511270+0000","last_became_peered":"2024-09-21T06:13:10.511270+0000","last_unstale":"2024-09-21T06:14:54.041365+0000","last_undegraded":"2024-09-21T06:14:54.041365+0000","last_fullsized":"2024-09-21T06:14:54.041365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:10.521853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.627560+0000","last_change":"2024-09-21T06:13:24.627467+0000","last_active":"2024-09-21T06:13:24.627560+0000","last_peered":"2024-09-21T06:13:24.627560+0000","last_clean":"2024-09-21T06:13:24.627560+0000","last_became_active":"2024-09-21T06:13:24.610263+0000","last_became_peered":"2024-09-21T06:13:24.610263+0000","last_unstale":"2024-09-21T06:13:24.627560+0000","last_undegraded":"2024-09-21T06:13:24.627560+0000","last_fullsized":"2024-09-21T06:13:24.627560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:26.472147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.032186+0000","last_change":"2024-09-21T06:13:10.517214+0000","last_active":"2024-09-21T06:14:54.032186+0000","last_peered":"2024-09-21T06:14:54.032186+0000","last_clean":"2024-09-21T06:14:54.032186+0000","last_became_active":"2024-09-21T06:13:10.516828+0000","last_became_peered":"2024-09-21T06:13:10.516828+0000","last_unstale":"2024-09-21T06:14:54.032186+0000","last_undegraded":"2024-09-21T06:14:54.032186+0000","last_fullsized":"2024-09-21T06:14:54.032186+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:58.077670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608703+0000","last_change":"2024-09-21T06:13:08.489153+0000","last_active":"2024-09-21T06:13:24.608703+0000","last_peered":"2024-09-21T06:13:24.608703+0000","last_clean":"2024-09-21T06:13:24.608703+0000","last_became_active":"2024-09-21T06:13:08.488732+0000","last_became_peered":"2024-09-21T06:13:08.488732+0000","last_unstale":"2024-09-21T06:13:24.608703+0000","last_undegraded":"2024-09-21T06:13:24.608703+0000","last_fullsized":"2024-09-21T06:13:24.608703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:22.848136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037298+0000","last_change":"2024-09-21T06:13:24.610441+0000","last_active":"2024-09-21T06:14:54.037298+0000","last_peered":"2024-09-21T06:14:54.037298+0000","last_clean":"2024-09-21T06:14:54.037298+0000","last_became_active":"2024-09-21T06:13:24.609992+0000","last_became_peered":"2024-09-21T06:13:24.609992+0000","last_unstale":"2024-09-21T06:14:54.037298+0000","last_undegraded":"2024-09-21T06:14:54.037298+0000","last_fullsized":"2024-09-21T06:14:54.037298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:59.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597725+0000","last_change":"2024-09-21T06:13:08.484514+0000","last_active":"2024-09-21T06:13:24.597725+0000","last_peered":"2024-09-21T06:13:24.597725+0000","last_clean":"2024-09-21T06:13:24.597725+0000","last_became_active":"2024-09-21T06:13:08.484134+0000","last_became_peered":"2024-09-21T06:13:08.484134+0000","last_unstale":"2024-09-21T06:13:24.597725+0000","last_undegraded":"2024-09-21T06:13:24.597725+0000","last_fullsized":"2024-09-21T06:13:24.597725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:45.124734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.039200+0000","last_change":"2024-09-21T06:13:10.503798+0000","last_active":"2024-09-21T06:14:54.039200+0000","last_peered":"2024-09-21T06:14:54.039200+0000","last_clean":"2024-09-21T06:14:54.039200+0000","last_became_active":"2024-09-21T06:13:10.502418+0000","last_became_peered":"2024-09-21T06:13:10.502418+0000","last_unstale":"2024-09-21T06:14:54.039200+0000","last_undegraded":"2024-09-21T06:14:54.039200+0000","last_fullsized":"2024-09-21T06:14:54.039200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:57.575569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603439+0000","last_change":"2024-09-21T06:13:08.487191+0000","last_active":"2024-09-21T06:13:24.603439+0000","last_peered":"2024-09-21T06:13:24.603439+0000","last_clean":"2024-09-21T06:13:24.603439+0000","last_became_active":"2024-09-21T06:13:08.486812+0000","last_became_peered":"2024-09-21T06:13:08.486812+0000","last_unstale":"2024-09-21T06:13:24.603439+0000","last_undegraded":"2024-09-21T06:13:24.603439+0000","last_fullsized":"2024-09-21T06:13:24.603439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:51.989708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.041431+0000","last_change":"2024-09-21T06:13:10.511678+0000","last_active":"2024-09-21T06:14:54.041431+0000","last_peered":"2024-09-21T06:14:54.041431+0000","last_clean":"2024-09-21T06:14:54.041431+0000","last_became_active":"2024-09-21T06:13:10.510408+0000","last_became_peered":"2024-09-21T06:13:10.510408+0000","last_unstale":"2024-09-21T06:14:54.041431+0000","last_undegraded":"2024-09-21T06:14:54.041431+0000","last_fullsized":"2024-09-21T06:14:54.041431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:58.711190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'7","reported_seq":139,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:07.766522+0000","last_change":"2024-09-21T06:13:08.560894+0000","last_active":"2024-09-21T06:15:07.766522+0000","last_peered":"2024-09-21T06:15:07.766522+0000","last_clean":"2024-09-21T06:15:07.766522+0000","last_became_active":"2024-09-21T06:13:08.560423+0000","last_became_peered":"2024-09-21T06:13:08.560423+0000","last_unstale":"2024-09-21T06:15:07.766522+0000","last_undegraded":"2024-09-21T06:15:07.766522+0000","last_fullsized":"2024-09-21T06:15:07.766522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:53.015711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.044084+0000","last_change":"2024-09-21T06:13:10.516009+0000","last_active":"2024-09-21T06:14:54.044084+0000","last_peered":"2024-09-21T06:14:54.044084+0000","last_clean":"2024-09-21T06:14:54.044084+0000","last_became_active":"2024-09-21T06:13:10.515669+0000","last_became_peered":"2024-09-21T06:13:10.515669+0000","last_unstale":"2024-09-21T06:14:54.044084+0000","last_undegraded":"2024-09-21T06:14:54.044084+0000","last_fullsized":"2024-09-21T06:14:54.044084+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:54.134564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":100,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610370+0000","last_change":"2024-09-21T06:11:53.408030+0000","last_active":"2024-09-21T06:13:24.610370+0000","last_peered":"2024-09-21T06:13:24.610370+0000","last_clean":"2024-09-21T06:13:24.610370+0000","last_became_active":"2024-09-21T06:11:53.402296+0000","last_became_peered":"2024-09-21T06:11:53.402296+0000","last_unstale":"2024-09-21T06:13:24.610370+0000","last_undegraded":"2024-09-21T06:13:24.610370+0000","last_fullsized":"2024-09-21T06:13:24.610370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_clean_scrub_stamp":"2024-09-21T06:08:26.098183+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:08.239872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616959+0000","last_change":"2024-09-21T06:13:08.489209+0000","last_active":"2024-09-21T06:13:24.616959+0000","last_peered":"2024-09-21T06:13:24.616959+0000","last_clean":"2024-09-21T06:13:24.616959+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.616959+0000","last_undegraded":"2024-09-21T06:13:24.616959+0000","last_fullsized":"2024-09-21T06:13:24.616959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:38.798499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.038080+0000","last_change":"2024-09-21T06:13:24.606487+0000","last_active":"2024-09-21T06:14:54.038080+0000","last_peered":"2024-09-21T06:14:54.038080+0000","last_clean":"2024-09-21T06:14:54.038080+0000","last_became_active":"2024-09-21T06:13:24.606141+0000","last_became_peered":"2024-09-21T06:13:24.606141+0000","last_unstale":"2024-09-21T06:14:54.038080+0000","last_undegraded":"2024-09-21T06:14:54.038080+0000","last_fullsized":"2024-09-21T06:14:54.038080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:24.040770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610270+0000","last_change":"2024-09-21T06:13:08.489206+0000","last_active":"2024-09-21T06:13:24.610270+0000","last_peered":"2024-09-21T06:13:24.610270+0000","last_clean":"2024-09-21T06:13:24.610270+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.610270+0000","last_undegraded":"2024-09-21T06:13:24.610270+0000","last_fullsized":"2024-09-21T06:13:24.610270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.627305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.027371+0000","last_change":"2024-09-21T06:13:10.513999+0000","last_active":"2024-09-21T06:14:54.027371+0000","last_peered":"2024-09-21T06:14:54.027371+0000","last_clean":"2024-09-21T06:14:54.027371+0000","last_became_active":"2024-09-21T06:13:10.513657+0000","last_became_peered":"2024-09-21T06:13:10.513657+0000","last_unstale":"2024-09-21T06:14:54.027371+0000","last_undegraded":"2024-09-21T06:14:54.027371+0000","last_fullsized":"2024-09-21T06:14:54.027371+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:05.536921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'6","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:07.765185+0000","last_change":"2024-09-21T06:13:08.560811+0000","last_active":"2024-09-21T06:15:07.765185+0000","last_peered":"2024-09-21T06:15:07.765185+0000","last_clean":"2024-09-21T06:15:07.765185+0000","last_became_active":"2024-09-21T06:13:08.560145+0000","last_became_peered":"2024-09-21T06:13:08.560145+0000","last_unstale":"2024-09-21T06:15:07.765185+0000","last_undegraded":"2024-09-21T06:15:07.765185+0000","last_fullsized":"2024-09-21T06:15:07.765185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:24.550008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035762+0000","last_change":"2024-09-21T06:13:24.613495+0000","last_active":"2024-09-21T06:14:54.035762+0000","last_peered":"2024-09-21T06:14:54.035762+0000","last_clean":"2024-09-21T06:14:54.035762+0000","last_became_active":"2024-09-21T06:13:24.612634+0000","last_became_peered":"2024-09-21T06:13:24.612634+0000","last_unstale":"2024-09-21T06:14:54.035762+0000","last_undegraded":"2024-09-21T06:14:54.035762+0000","last_fullsized":"2024-09-21T06:14:54.035762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.863141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'5","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:07.768565+0000","last_change":"2024-09-21T06:13:08.486973+0000","last_active":"2024-09-21T06:15:07.768565+0000","last_peered":"2024-09-21T06:15:07.768565+0000","last_clean":"2024-09-21T06:15:07.768565+0000","last_became_active":"2024-09-21T06:13:08.486602+0000","last_became_peered":"2024-09-21T06:13:08.486602+0000","last_unstale":"2024-09-21T06:15:07.768565+0000","last_undegraded":"2024-09-21T06:15:07.768565+0000","last_fullsized":"2024-09-21T06:15:07.768565+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:41.804955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.029271+0000","last_change":"2024-09-21T06:13:10.509499+0000","last_active":"2024-09-21T06:14:54.029271+0000","last_peered":"2024-09-21T06:14:54.029271+0000","last_clean":"2024-09-21T06:14:54.029271+0000","last_became_active":"2024-09-21T06:13:10.508566+0000","last_became_peered":"2024-09-21T06:13:10.508566+0000","last_unstale":"2024-09-21T06:14:54.029271+0000","last_undegraded":"2024-09-21T06:14:54.029271+0000","last_fullsized":"2024-09-21T06:14:54.029271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:12.701631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600737+0000","last_change":"2024-09-21T06:13:08.561378+0000","last_active":"2024-09-21T06:13:24.600737+0000","last_peered":"2024-09-21T06:13:24.600737+0000","last_clean":"2024-09-21T06:13:24.600737+0000","last_became_active":"2024-09-21T06:13:08.561002+0000","last_became_peered":"2024-09-21T06:13:08.561002+0000","last_unstale":"2024-09-21T06:13:24.600737+0000","last_undegraded":"2024-09-21T06:13:24.600737+0000","last_fullsized":"2024-09-21T06:13:24.600737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:11.325377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.030695+0000","last_change":"2024-09-21T06:13:24.602347+0000","last_active":"2024-09-21T06:14:54.030695+0000","last_peered":"2024-09-21T06:14:54.030695+0000","last_clean":"2024-09-21T06:14:54.030695+0000","last_became_active":"2024-09-21T06:13:24.601978+0000","last_became_peered":"2024-09-21T06:13:24.601978+0000","last_unstale":"2024-09-21T06:14:54.030695+0000","last_undegraded":"2024-09-21T06:14:54.030695+0000","last_fullsized":"2024-09-21T06:14:54.030695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:34.860096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608569+0000","last_change":"2024-09-21T06:13:08.561937+0000","last_active":"2024-09-21T06:13:24.608569+0000","last_peered":"2024-09-21T06:13:24.608569+0000","last_clean":"2024-09-21T06:13:24.608569+0000","last_became_active":"2024-09-21T06:13:08.560547+0000","last_became_peered":"2024-09-21T06:13:08.560547+0000","last_unstale":"2024-09-21T06:13:24.608569+0000","last_undegraded":"2024-09-21T06:13:24.608569+0000","last_fullsized":"2024-09-21T06:13:24.608569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:29.702291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.027723+0000","last_change":"2024-09-21T06:13:10.516833+0000","last_active":"2024-09-21T06:14:54.027723+0000","last_peered":"2024-09-21T06:14:54.027723+0000","last_clean":"2024-09-21T06:14:54.027723+0000","last_became_active":"2024-09-21T06:13:10.516486+0000","last_became_peered":"2024-09-21T06:13:10.516486+0000","last_unstale":"2024-09-21T06:14:54.027723+0000","last_undegraded":"2024-09-21T06:14:54.027723+0000","last_fullsized":"2024-09-21T06:14:54.027723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:05.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'19","reported_seq":42,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:08.513647+0000","last_change":"2024-09-21T06:13:08.483271+0000","last_active":"2024-09-21T06:15:08.513647+0000","last_peered":"2024-09-21T06:15:08.513647+0000","last_clean":"2024-09-21T06:15:08.513647+0000","last_became_active":"2024-09-21T06:13:08.482951+0000","last_became_peered":"2024-09-21T06:13:08.482951+0000","last_unstale":"2024-09-21T06:15:08.513647+0000","last_undegraded":"2024-09-21T06:15:08.513647+0000","last_fullsized":"2024-09-21T06:15:08.513647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:17.537988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.043217+0000","last_change":"2024-09-21T06:13:10.509125+0000","last_active":"2024-09-21T06:14:54.043217+0000","last_peered":"2024-09-21T06:14:54.043217+0000","last_clean":"2024-09-21T06:14:54.043217+0000","last_became_active":"2024-09-21T06:13:10.508773+0000","last_became_peered":"2024-09-21T06:13:10.508773+0000","last_unstale":"2024-09-21T06:14:54.043217+0000","last_undegraded":"2024-09-21T06:14:54.043217+0000","last_fullsized":"2024-09-21T06:14:54.043217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:44.570477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616932+0000","last_change":"2024-09-21T06:13:08.497885+0000","last_active":"2024-09-21T06:13:24.616932+0000","last_peered":"2024-09-21T06:13:24.616932+0000","last_clean":"2024-09-21T06:13:24.616932+0000","last_became_active":"2024-09-21T06:13:08.497593+0000","last_became_peered":"2024-09-21T06:13:08.497593+0000","last_unstale":"2024-09-21T06:13:24.616932+0000","last_undegraded":"2024-09-21T06:13:24.616932+0000","last_fullsized":"2024-09-21T06:13:24.616932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:48.361175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037327+0000","last_change":"2024-09-21T06:13:10.514430+0000","last_active":"2024-09-21T06:14:54.037327+0000","last_peered":"2024-09-21T06:14:54.037327+0000","last_clean":"2024-09-21T06:14:54.037327+0000","last_became_active":"2024-09-21T06:13:10.514063+0000","last_became_peered":"2024-09-21T06:13:10.514063+0000","last_unstale":"2024-09-21T06:14:54.037327+0000","last_undegraded":"2024-09-21T06:14:54.037327+0000","last_fullsized":"2024-09-21T06:14:54.037327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:37.036497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598415+0000","last_change":"2024-09-21T06:13:08.560564+0000","last_active":"2024-09-21T06:13:24.598415+0000","last_peered":"2024-09-21T06:13:24.598415+0000","last_clean":"2024-09-21T06:13:24.598415+0000","last_became_active":"2024-09-21T06:13:08.560339+0000","last_became_peered":"2024-09-21T06:13:08.560339+0000","last_unstale":"2024-09-21T06:13:24.598415+0000","last_undegraded":"2024-09-21T06:13:24.598415+0000","last_fullsized":"2024-09-21T06:13:24.598415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:55.378911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.022629+0000","last_change":"2024-09-21T06:13:10.511354+0000","last_active":"2024-09-21T06:14:54.022629+0000","last_peered":"2024-09-21T06:14:54.022629+0000","last_clean":"2024-09-21T06:14:54.022629+0000","last_became_active":"2024-09-21T06:13:10.510946+0000","last_became_peered":"2024-09-21T06:13:10.510946+0000","last_unstale":"2024-09-21T06:14:54.022629+0000","last_undegraded":"2024-09-21T06:14:54.022629+0000","last_fullsized":"2024-09-21T06:14:54.022629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:11.535461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610883+0000","last_change":"2024-09-21T06:13:08.485578+0000","last_active":"2024-09-21T06:13:24.610883+0000","last_peered":"2024-09-21T06:13:24.610883+0000","last_clean":"2024-09-21T06:13:24.610883+0000","last_became_active":"2024-09-21T06:13:08.485170+0000","last_became_peered":"2024-09-21T06:13:08.485170+0000","last_unstale":"2024-09-21T06:13:24.610883+0000","last_undegraded":"2024-09-21T06:13:24.610883+0000","last_fullsized":"2024-09-21T06:13:24.610883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:39.000114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035886+0000","last_change":"2024-09-21T06:13:10.510360+0000","last_active":"2024-09-21T06:14:54.035886+0000","last_peered":"2024-09-21T06:14:54.035886+0000","last_clean":"2024-09-21T06:14:54.035886+0000","last_became_active":"2024-09-21T06:13:10.509887+0000","last_became_peered":"2024-09-21T06:13:10.509887+0000","last_unstale":"2024-09-21T06:14:54.035886+0000","last_undegraded":"2024-09-21T06:14:54.035886+0000","last_fullsized":"2024-09-21T06:14:54.035886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:58.135706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.625797+0000","last_change":"2024-09-21T06:13:24.625284+0000","last_active":"2024-09-21T06:13:24.625797+0000","last_peered":"2024-09-21T06:13:24.625797+0000","last_clean":"2024-09-21T06:13:24.625797+0000","last_became_active":"2024-09-21T06:13:24.612704+0000","last_became_peered":"2024-09-21T06:13:24.612704+0000","last_unstale":"2024-09-21T06:13:24.625797+0000","last_undegraded":"2024-09-21T06:13:24.625797+0000","last_fullsized":"2024-09-21T06:13:24.625797+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:29.385258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.029990+0000","last_change":"2024-09-21T06:13:10.514128+0000","last_active":"2024-09-21T06:14:54.029990+0000","last_peered":"2024-09-21T06:14:54.029990+0000","last_clean":"2024-09-21T06:14:54.029990+0000","last_became_active":"2024-09-21T06:13:10.513790+0000","last_became_peered":"2024-09-21T06:13:10.513790+0000","last_unstale":"2024-09-21T06:14:54.029990+0000","last_undegraded":"2024-09-21T06:14:54.029990+0000","last_fullsized":"2024-09-21T06:14:54.029990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:33.407189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607762+0000","last_change":"2024-09-21T06:13:08.489105+0000","last_active":"2024-09-21T06:13:24.607762+0000","last_peered":"2024-09-21T06:13:24.607762+0000","last_clean":"2024-09-21T06:13:24.607762+0000","last_became_active":"2024-09-21T06:13:08.487110+0000","last_became_peered":"2024-09-21T06:13:08.487110+0000","last_unstale":"2024-09-21T06:13:24.607762+0000","last_undegraded":"2024-09-21T06:13:24.607762+0000","last_fullsized":"2024-09-21T06:13:24.607762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:24.849365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.044351+0000","last_change":"2024-09-21T06:13:10.513751+0000","last_active":"2024-09-21T06:14:54.044351+0000","last_peered":"2024-09-21T06:14:54.044351+0000","last_clean":"2024-09-21T06:14:54.044351+0000","last_became_active":"2024-09-21T06:13:10.513376+0000","last_became_peered":"2024-09-21T06:13:10.513376+0000","last_unstale":"2024-09-21T06:14:54.044351+0000","last_undegraded":"2024-09-21T06:14:54.044351+0000","last_fullsized":"2024-09-21T06:14:54.044351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.364959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.039103+0000","last_change":"2024-09-21T06:13:10.519359+0000","last_active":"2024-09-21T06:14:54.039103+0000","last_peered":"2024-09-21T06:14:54.039103+0000","last_clean":"2024-09-21T06:14:54.039103+0000","last_became_active":"2024-09-21T06:13:10.518518+0000","last_became_peered":"2024-09-21T06:13:10.518518+0000","last_unstale":"2024-09-21T06:14:54.039103+0000","last_undegraded":"2024-09-21T06:14:54.039103+0000","last_fullsized":"2024-09-21T06:14:54.039103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:55.554710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608166+0000","last_change":"2024-09-21T06:13:08.488122+0000","last_active":"2024-09-21T06:13:24.608166+0000","last_peered":"2024-09-21T06:13:24.608166+0000","last_clean":"2024-09-21T06:13:24.608166+0000","last_became_active":"2024-09-21T06:13:08.487679+0000","last_became_peered":"2024-09-21T06:13:08.487679+0000","last_unstale":"2024-09-21T06:13:24.608166+0000","last_undegraded":"2024-09-21T06:13:24.608166+0000","last_fullsized":"2024-09-21T06:13:24.608166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:12.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.043317+0000","last_change":"2024-09-21T06:13:10.516794+0000","last_active":"2024-09-21T06:14:54.043317+0000","last_peered":"2024-09-21T06:14:54.043317+0000","last_clean":"2024-09-21T06:14:54.043317+0000","last_became_active":"2024-09-21T06:13:10.516297+0000","last_became_peered":"2024-09-21T06:13:10.516297+0000","last_unstale":"2024-09-21T06:14:54.043317+0000","last_undegraded":"2024-09-21T06:14:54.043317+0000","last_fullsized":"2024-09-21T06:14:54.043317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:19.873742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.621488+0000","last_change":"2024-09-21T06:13:24.621358+0000","last_active":"2024-09-21T06:13:24.621488+0000","last_peered":"2024-09-21T06:13:24.621488+0000","last_clean":"2024-09-21T06:13:24.621488+0000","last_became_active":"2024-09-21T06:13:24.611058+0000","last_became_peered":"2024-09-21T06:13:24.611058+0000","last_unstale":"2024-09-21T06:13:24.621488+0000","last_undegraded":"2024-09-21T06:13:24.621488+0000","last_fullsized":"2024-09-21T06:13:24.621488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:28.759029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.040559+0000","last_change":"2024-09-21T06:13:10.516339+0000","last_active":"2024-09-21T06:14:54.040559+0000","last_peered":"2024-09-21T06:14:54.040559+0000","last_clean":"2024-09-21T06:14:54.040559+0000","last_became_active":"2024-09-21T06:13:10.515397+0000","last_became_peered":"2024-09-21T06:13:10.515397+0000","last_unstale":"2024-09-21T06:14:54.040559+0000","last_undegraded":"2024-09-21T06:14:54.040559+0000","last_fullsized":"2024-09-21T06:14:54.040559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:56.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610445+0000","last_change":"2024-09-21T06:13:08.560494+0000","last_active":"2024-09-21T06:13:24.610445+0000","last_peered":"2024-09-21T06:13:24.610445+0000","last_clean":"2024-09-21T06:13:24.610445+0000","last_became_active":"2024-09-21T06:13:08.560118+0000","last_became_peered":"2024-09-21T06:13:08.560118+0000","last_unstale":"2024-09-21T06:13:24.610445+0000","last_undegraded":"2024-09-21T06:13:24.610445+0000","last_fullsized":"2024-09-21T06:13:24.610445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:05.597233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035172+0000","last_change":"2024-09-21T06:13:10.515780+0000","last_active":"2024-09-21T06:14:54.035172+0000","last_peered":"2024-09-21T06:14:54.035172+0000","last_clean":"2024-09-21T06:14:54.035172+0000","last_became_active":"2024-09-21T06:13:10.515437+0000","last_became_peered":"2024-09-21T06:13:10.515437+0000","last_unstale":"2024-09-21T06:14:54.035172+0000","last_undegraded":"2024-09-21T06:14:54.035172+0000","last_fullsized":"2024-09-21T06:14:54.035172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:04.848003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'14","reported_seq":146,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:09.770212+0000","last_change":"2024-09-21T06:13:08.483300+0000","last_active":"2024-09-21T06:15:09.770212+0000","last_peered":"2024-09-21T06:15:09.770212+0000","last_clean":"2024-09-21T06:15:09.770212+0000","last_became_active":"2024-09-21T06:13:08.482908+0000","last_became_peered":"2024-09-21T06:13:08.482908+0000","last_unstale":"2024-09-21T06:15:09.770212+0000","last_undegraded":"2024-09-21T06:15:09.770212+0000","last_fullsized":"2024-09-21T06:15:09.770212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:09.202075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82109,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1109,"num_read_kb":646,"num_write":272,"num_write_kb":169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1724,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":231890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282438,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678053,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":427664,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874283,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1092867,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037809,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":427179,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234038,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":427440,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":1020939,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593791,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":426779,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1020085,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":493387,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":426487,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378709,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":429172,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542234,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":494327,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428650,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":76442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:15:13.140 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:15:13.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.288+0000 7f749de4d640 1 -- 172.21.15.3:0/1019061560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74981093f0 msgr2=0x7f7498109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:13.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.288+0000 7f749de4d640 1 --2- 172.21.15.3:0/1019061560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74981093f0 0x7f7498109800 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f748c009900 tx=0x7f748c02a410 comp rx=0 tx=0).stop 2024-09-21T06:15:13.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.288+0000 7f749de4d640 1 -- 172.21.15.3:0/1019061560 shutdown_connections 2024-09-21T06:15:13.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.288+0000 7f749de4d640 1 --2- 172.21.15.3:0/1019061560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7498109d40 0x7f7498110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.288+0000 7f749de4d640 1 --2- 172.21.15.3:0/1019061560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74981093f0 0x7f7498109800 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.288+0000 7f749de4d640 1 --2- 172.21.15.3:0/1019061560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7498102020 0x7f7498102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.288+0000 7f749de4d640 1 -- 172.21.15.3:0/1019061560 >> 172.21.15.3:0/1019061560 conn(0x7f7498079e50 msgr2=0x7f749807a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:13.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.288+0000 7f749de4d640 1 -- 172.21.15.3:0/1019061560 shutdown_connections 2024-09-21T06:15:13.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.288+0000 7f749de4d640 1 -- 172.21.15.3:0/1019061560 wait complete. 2024-09-21T06:15:13.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.289+0000 7f749de4d640 1 Processor -- start 2024-09-21T06:15:13.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.289+0000 7f749de4d640 1 -- start start 2024-09-21T06:15:13.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.289+0000 7f749de4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7498102020 0x7f749819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:13.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.289+0000 7f749de4d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74981093f0 0x7f749819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:13.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.289+0000 7f749de4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7498109d40 0x7f749819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:13.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.289+0000 7f749de4d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7498113a00 con 0x7f74981093f0 2024-09-21T06:15:13.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.289+0000 7f749de4d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7498113880 con 0x7f7498109d40 2024-09-21T06:15:13.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.289+0000 7f749de4d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7498113b80 con 0x7f7498102020 2024-09-21T06:15:13.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.289+0000 7f7496ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74981093f0 0x7f749819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:13.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.289+0000 7f7496ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74981093f0 0x7f749819a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48354/0 (socket says 172.21.15.3:48354) 2024-09-21T06:15:13.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.289+0000 7f7496ffd640 1 -- 172.21.15.3:0/18470629 learned_addr learned my addr 172.21.15.3:0/18470629 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.289+0000 7f7497fff640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7498109d40 0x7f749819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.289+0000 7f74977fe640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7498102020 0x7f749819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.290+0000 7f7496ffd640 1 -- 172.21.15.3:0/18470629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7498102020 msgr2=0x7f749819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.290+0000 7f7496ffd640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7498102020 0x7f749819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.290+0000 7f7496ffd640 1 -- 172.21.15.3:0/18470629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7498109d40 msgr2=0x7f749819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.290+0000 7f7496ffd640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7498109d40 0x7f749819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.290+0000 7f7496ffd640 1 -- 172.21.15.3:0/18470629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74981a0310 con 0x7f74981093f0 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.290+0000 7f7496ffd640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74981093f0 0x7f749819a980 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f748c02a930 tx=0x7f748c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.290+0000 7f7494ff9640 1 -- 172.21.15.3:0/18470629 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c03f030 con 0x7f74981093f0 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.290+0000 7f7494ff9640 1 -- 172.21.15.3:0/18470629 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f748c004490 con 0x7f74981093f0 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.290+0000 7f7494ff9640 1 -- 172.21.15.3:0/18470629 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c035070 con 0x7f74981093f0 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.290+0000 7f749de4d640 1 -- 172.21.15.3:0/18470629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74981a05a0 con 0x7f74981093f0 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.290+0000 7f749de4d640 1 -- 172.21.15.3:0/18470629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74981a6d90 con 0x7f74981093f0 2024-09-21T06:15:13.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.291+0000 7f749de4d640 1 -- 172.21.15.3:0/18470629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f749806b620 con 0x7f74981093f0 2024-09-21T06:15:13.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.293+0000 7f7494ff9640 1 -- 172.21.15.3:0/18470629 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 154579+0+0 (secure 0 0 0) 0x7f748c042030 con 0x7f74981093f0 2024-09-21T06:15:13.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.294+0000 7f7494ff9640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74800b65f0 0x7f74800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:13.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.294+0000 7f74977fe640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74800b65f0 0x7f74800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:13.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.294+0000 7f7494ff9640 1 -- 172.21.15.3:0/18470629 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f748c1079b0 con 0x7f74981093f0 2024-09-21T06:15:13.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.294+0000 7f74977fe640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74800b65f0 0x7f74800b8b00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7488009b60 tx=0x7f7488005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:13.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.298+0000 7f7494ff9640 1 -- 172.21.15.3:0/18470629 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f748c0ce000 con 0x7f74981093f0 2024-09-21T06:15:13.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.484+0000 7f749de4d640 1 -- 172.21.15.3:0/18470629 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7498063800 con 0x7f74800b65f0 2024-09-21T06:15:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.488+0000 7f7494ff9640 1 -- 172.21.15.3:0/18470629 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259148 (secure 0 0 0) 0x7f7498063800 con 0x7f74800b65f0 2024-09-21T06:15:13.493 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:15:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-21T06:15:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.491+0000 7f749de4d640 1 -- 172.21.15.3:0/18470629 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74800b65f0 msgr2=0x7f74800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.491+0000 7f749de4d640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74800b65f0 0x7f74800b8b00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7488009b60 tx=0x7f7488005f90 comp rx=0 tx=0).stop 2024-09-21T06:15:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.491+0000 7f749de4d640 1 -- 172.21.15.3:0/18470629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74981093f0 msgr2=0x7f749819a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.491+0000 7f749de4d640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74981093f0 0x7f749819a980 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f748c02a930 tx=0x7f748c031a20 comp rx=0 tx=0).stop 2024-09-21T06:15:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.492+0000 7f749de4d640 1 -- 172.21.15.3:0/18470629 shutdown_connections 2024-09-21T06:15:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.492+0000 7f749de4d640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74800b65f0 0x7f74800b8b00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.492+0000 7f749de4d640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7498109d40 0x7f749819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.492+0000 7f749de4d640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74981093f0 0x7f749819a980 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.492+0000 7f749de4d640 1 --2- 172.21.15.3:0/18470629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7498102020 0x7f749819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.492+0000 7f749de4d640 1 -- 172.21.15.3:0/18470629 >> 172.21.15.3:0/18470629 conn(0x7f7498079e50 msgr2=0x7f7498107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.492+0000 7f749de4d640 1 -- 172.21.15.3:0/18470629 shutdown_connections 2024-09-21T06:15:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.492+0000 7f749de4d640 1 -- 172.21.15.3:0/18470629 wait complete. 2024-09-21T06:15:13.504 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:13 smithi003 ceph-mon[31227]: pgmap v331: 97 pgs: 97 active+clean; 657 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-21T06:15:13.504 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:13 smithi003 ceph-mon[31227]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T06:15:13.505 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-21T06:15:11.659039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672711,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1190,"num_read_kb":715,"num_write":397,"num_write_kb":2321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2953564,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":827,"ondisk_log_size":827,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368300,"kb_used_data":10732,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573524,"statfs":{"total":1151940427776,"available":1151563288576,"internally_reserved":0,"allocated":10989568,"data_stored":7114976,"data_compressed":177687,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19139,"internal_metadata":365933885},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7876,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":78,"num_write":14,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011937"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.040716+0000","last_change":"2024-09-21T06:13:10.515115+0000","last_active":"2024-09-21T06:14:54.040716+0000","last_peered":"2024-09-21T06:14:54.040716+0000","last_clean":"2024-09-21T06:14:54.040716+0000","last_became_active":"2024-09-21T06:13:10.514553+0000","last_became_peered":"2024-09-21T06:13:10.514553+0000","last_unstale":"2024-09-21T06:14:54.040716+0000","last_undegraded":"2024-09-21T06:14:54.040716+0000","last_fullsized":"2024-09-21T06:14:54.040716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:33.268926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035807+0000","last_change":"2024-09-21T06:13:10.511469+0000","last_active":"2024-09-21T06:14:54.035807+0000","last_peered":"2024-09-21T06:14:54.035807+0000","last_clean":"2024-09-21T06:14:54.035807+0000","last_became_active":"2024-09-21T06:13:10.511102+0000","last_became_peered":"2024-09-21T06:13:10.511102+0000","last_unstale":"2024-09-21T06:14:54.035807+0000","last_undegraded":"2024-09-21T06:14:54.035807+0000","last_fullsized":"2024-09-21T06:14:54.035807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:12.483821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.040174+0000","last_change":"2024-09-21T06:13:10.520090+0000","last_active":"2024-09-21T06:14:54.040174+0000","last_peered":"2024-09-21T06:14:54.040174+0000","last_clean":"2024-09-21T06:14:54.040174+0000","last_became_active":"2024-09-21T06:13:10.519758+0000","last_became_peered":"2024-09-21T06:13:10.519758+0000","last_unstale":"2024-09-21T06:14:54.040174+0000","last_undegraded":"2024-09-21T06:14:54.040174+0000","last_fullsized":"2024-09-21T06:14:54.040174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:20.836847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.025079+0000","last_change":"2024-09-21T06:13:10.511732+0000","last_active":"2024-09-21T06:14:54.025079+0000","last_peered":"2024-09-21T06:14:54.025079+0000","last_clean":"2024-09-21T06:14:54.025079+0000","last_became_active":"2024-09-21T06:13:10.511377+0000","last_became_peered":"2024-09-21T06:13:10.511377+0000","last_unstale":"2024-09-21T06:14:54.025079+0000","last_undegraded":"2024-09-21T06:14:54.025079+0000","last_fullsized":"2024-09-21T06:14:54.025079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:18.500739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.023035+0000","last_change":"2024-09-21T06:13:10.510464+0000","last_active":"2024-09-21T06:14:54.023035+0000","last_peered":"2024-09-21T06:14:54.023035+0000","last_clean":"2024-09-21T06:14:54.023035+0000","last_became_active":"2024-09-21T06:13:10.509979+0000","last_became_peered":"2024-09-21T06:13:10.509979+0000","last_unstale":"2024-09-21T06:14:54.023035+0000","last_undegraded":"2024-09-21T06:14:54.023035+0000","last_fullsized":"2024-09-21T06:14:54.023035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:56.871043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.032702+0000","last_change":"2024-09-21T06:13:24.608832+0000","last_active":"2024-09-21T06:14:54.032702+0000","last_peered":"2024-09-21T06:14:54.032702+0000","last_clean":"2024-09-21T06:14:54.032702+0000","last_became_active":"2024-09-21T06:13:24.608439+0000","last_became_peered":"2024-09-21T06:13:24.608439+0000","last_unstale":"2024-09-21T06:14:54.032702+0000","last_undegraded":"2024-09-21T06:14:54.032702+0000","last_fullsized":"2024-09-21T06:14:54.032702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:41.515894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.041535+0000","last_change":"2024-09-21T06:13:10.511391+0000","last_active":"2024-09-21T06:14:54.041535+0000","last_peered":"2024-09-21T06:14:54.041535+0000","last_clean":"2024-09-21T06:14:54.041535+0000","last_became_active":"2024-09-21T06:13:10.510733+0000","last_became_peered":"2024-09-21T06:13:10.510733+0000","last_unstale":"2024-09-21T06:14:54.041535+0000","last_undegraded":"2024-09-21T06:14:54.041535+0000","last_fullsized":"2024-09-21T06:14:54.041535+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:24.071860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035636+0000","last_change":"2024-09-21T06:13:10.510886+0000","last_active":"2024-09-21T06:14:54.035636+0000","last_peered":"2024-09-21T06:14:54.035636+0000","last_clean":"2024-09-21T06:14:54.035636+0000","last_became_active":"2024-09-21T06:13:10.510159+0000","last_became_peered":"2024-09-21T06:13:10.510159+0000","last_unstale":"2024-09-21T06:14:54.035636+0000","last_undegraded":"2024-09-21T06:14:54.035636+0000","last_fullsized":"2024-09-21T06:14:54.035636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:37.124848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.045101+0000","last_change":"2024-09-21T06:13:10.510466+0000","last_active":"2024-09-21T06:14:54.045101+0000","last_peered":"2024-09-21T06:14:54.045101+0000","last_clean":"2024-09-21T06:14:54.045101+0000","last_became_active":"2024-09-21T06:13:10.509928+0000","last_became_peered":"2024-09-21T06:13:10.509928+0000","last_unstale":"2024-09-21T06:14:54.045101+0000","last_undegraded":"2024-09-21T06:14:54.045101+0000","last_fullsized":"2024-09-21T06:14:54.045101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:06.156505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.029437+0000","last_change":"2024-09-21T06:13:10.510622+0000","last_active":"2024-09-21T06:14:54.029437+0000","last_peered":"2024-09-21T06:14:54.029437+0000","last_clean":"2024-09-21T06:14:54.029437+0000","last_became_active":"2024-09-21T06:13:10.510206+0000","last_became_peered":"2024-09-21T06:13:10.510206+0000","last_unstale":"2024-09-21T06:14:54.029437+0000","last_undegraded":"2024-09-21T06:14:54.029437+0000","last_fullsized":"2024-09-21T06:14:54.029437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:03.223451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.042280+0000","last_change":"2024-09-21T06:13:10.517525+0000","last_active":"2024-09-21T06:14:54.042280+0000","last_peered":"2024-09-21T06:14:54.042280+0000","last_clean":"2024-09-21T06:14:54.042280+0000","last_became_active":"2024-09-21T06:13:10.517144+0000","last_became_peered":"2024-09-21T06:13:10.517144+0000","last_unstale":"2024-09-21T06:14:54.042280+0000","last_undegraded":"2024-09-21T06:14:54.042280+0000","last_fullsized":"2024-09-21T06:14:54.042280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:46.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037178+0000","last_change":"2024-09-21T06:13:10.515537+0000","last_active":"2024-09-21T06:14:54.037178+0000","last_peered":"2024-09-21T06:14:54.037178+0000","last_clean":"2024-09-21T06:14:54.037178+0000","last_became_active":"2024-09-21T06:13:10.515063+0000","last_became_peered":"2024-09-21T06:13:10.515063+0000","last_unstale":"2024-09-21T06:14:54.037178+0000","last_undegraded":"2024-09-21T06:14:54.037178+0000","last_fullsized":"2024-09-21T06:14:54.037178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:33.718471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.031442+0000","last_change":"2024-09-21T06:13:10.509368+0000","last_active":"2024-09-21T06:14:54.031442+0000","last_peered":"2024-09-21T06:14:54.031442+0000","last_clean":"2024-09-21T06:14:54.031442+0000","last_became_active":"2024-09-21T06:13:10.508982+0000","last_became_peered":"2024-09-21T06:13:10.508982+0000","last_unstale":"2024-09-21T06:14:54.031442+0000","last_undegraded":"2024-09-21T06:14:54.031442+0000","last_fullsized":"2024-09-21T06:14:54.031442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:54.430889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.036256+0000","last_change":"2024-09-21T06:13:10.506559+0000","last_active":"2024-09-21T06:14:54.036256+0000","last_peered":"2024-09-21T06:14:54.036256+0000","last_clean":"2024-09-21T06:14:54.036256+0000","last_became_active":"2024-09-21T06:13:10.506161+0000","last_became_peered":"2024-09-21T06:13:10.506161+0000","last_unstale":"2024-09-21T06:14:54.036256+0000","last_undegraded":"2024-09-21T06:14:54.036256+0000","last_fullsized":"2024-09-21T06:14:54.036256+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:19.870292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037955+0000","last_change":"2024-09-21T06:13:10.516000+0000","last_active":"2024-09-21T06:14:54.037955+0000","last_peered":"2024-09-21T06:14:54.037955+0000","last_clean":"2024-09-21T06:14:54.037955+0000","last_became_active":"2024-09-21T06:13:10.515543+0000","last_became_peered":"2024-09-21T06:13:10.515543+0000","last_unstale":"2024-09-21T06:14:54.037955+0000","last_undegraded":"2024-09-21T06:14:54.037955+0000","last_fullsized":"2024-09-21T06:14:54.037955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:48.968489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.045711+0000","last_change":"2024-09-21T06:13:10.513753+0000","last_active":"2024-09-21T06:14:54.045711+0000","last_peered":"2024-09-21T06:14:54.045711+0000","last_clean":"2024-09-21T06:14:54.045711+0000","last_became_active":"2024-09-21T06:13:10.511079+0000","last_became_peered":"2024-09-21T06:13:10.511079+0000","last_unstale":"2024-09-21T06:14:54.045711+0000","last_undegraded":"2024-09-21T06:14:54.045711+0000","last_fullsized":"2024-09-21T06:14:54.045711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:25.320041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.040850+0000","last_change":"2024-09-21T06:13:10.515222+0000","last_active":"2024-09-21T06:14:54.040850+0000","last_peered":"2024-09-21T06:14:54.040850+0000","last_clean":"2024-09-21T06:14:54.040850+0000","last_became_active":"2024-09-21T06:13:10.514604+0000","last_became_peered":"2024-09-21T06:13:10.514604+0000","last_unstale":"2024-09-21T06:14:54.040850+0000","last_undegraded":"2024-09-21T06:14:54.040850+0000","last_fullsized":"2024-09-21T06:14:54.040850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:03.165695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.038346+0000","last_change":"2024-09-21T06:13:10.518334+0000","last_active":"2024-09-21T06:14:54.038346+0000","last_peered":"2024-09-21T06:14:54.038346+0000","last_clean":"2024-09-21T06:14:54.038346+0000","last_became_active":"2024-09-21T06:13:10.517956+0000","last_became_peered":"2024-09-21T06:13:10.517956+0000","last_unstale":"2024-09-21T06:14:54.038346+0000","last_undegraded":"2024-09-21T06:14:54.038346+0000","last_fullsized":"2024-09-21T06:14:54.038346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:05.094454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.029105+0000","last_change":"2024-09-21T06:13:10.508763+0000","last_active":"2024-09-21T06:14:54.029105+0000","last_peered":"2024-09-21T06:14:54.029105+0000","last_clean":"2024-09-21T06:14:54.029105+0000","last_became_active":"2024-09-21T06:13:10.508371+0000","last_became_peered":"2024-09-21T06:13:10.508371+0000","last_unstale":"2024-09-21T06:14:54.029105+0000","last_undegraded":"2024-09-21T06:14:54.029105+0000","last_fullsized":"2024-09-21T06:14:54.029105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:50.602692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.038111+0000","last_change":"2024-09-21T06:13:10.503465+0000","last_active":"2024-09-21T06:14:54.038111+0000","last_peered":"2024-09-21T06:14:54.038111+0000","last_clean":"2024-09-21T06:14:54.038111+0000","last_became_active":"2024-09-21T06:13:10.503072+0000","last_became_peered":"2024-09-21T06:13:10.503072+0000","last_unstale":"2024-09-21T06:14:54.038111+0000","last_undegraded":"2024-09-21T06:14:54.038111+0000","last_fullsized":"2024-09-21T06:14:54.038111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:36.368987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.032321+0000","last_change":"2024-09-21T06:13:10.512841+0000","last_active":"2024-09-21T06:14:54.032321+0000","last_peered":"2024-09-21T06:14:54.032321+0000","last_clean":"2024-09-21T06:14:54.032321+0000","last_became_active":"2024-09-21T06:13:10.512407+0000","last_became_peered":"2024-09-21T06:13:10.512407+0000","last_unstale":"2024-09-21T06:14:54.032321+0000","last_undegraded":"2024-09-21T06:14:54.032321+0000","last_fullsized":"2024-09-21T06:14:54.032321+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:09.855608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.045911+0000","last_change":"2024-09-21T06:13:10.510088+0000","last_active":"2024-09-21T06:14:54.045911+0000","last_peered":"2024-09-21T06:14:54.045911+0000","last_clean":"2024-09-21T06:14:54.045911+0000","last_became_active":"2024-09-21T06:13:10.507904+0000","last_became_peered":"2024-09-21T06:13:10.507904+0000","last_unstale":"2024-09-21T06:14:54.045911+0000","last_undegraded":"2024-09-21T06:14:54.045911+0000","last_fullsized":"2024-09-21T06:14:54.045911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:49.404402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.033659+0000","last_change":"2024-09-21T06:13:10.517720+0000","last_active":"2024-09-21T06:14:54.033659+0000","last_peered":"2024-09-21T06:14:54.033659+0000","last_clean":"2024-09-21T06:14:54.033659+0000","last_became_active":"2024-09-21T06:13:10.514641+0000","last_became_peered":"2024-09-21T06:13:10.514641+0000","last_unstale":"2024-09-21T06:14:54.033659+0000","last_undegraded":"2024-09-21T06:14:54.033659+0000","last_fullsized":"2024-09-21T06:14:54.033659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:40.639665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.034092+0000","last_change":"2024-09-21T06:13:10.512234+0000","last_active":"2024-09-21T06:14:54.034092+0000","last_peered":"2024-09-21T06:14:54.034092+0000","last_clean":"2024-09-21T06:14:54.034092+0000","last_became_active":"2024-09-21T06:13:10.511594+0000","last_became_peered":"2024-09-21T06:13:10.511594+0000","last_unstale":"2024-09-21T06:14:54.034092+0000","last_undegraded":"2024-09-21T06:14:54.034092+0000","last_fullsized":"2024-09-21T06:14:54.034092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:04.410342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.030782+0000","last_change":"2024-09-21T06:13:10.513216+0000","last_active":"2024-09-21T06:14:54.030782+0000","last_peered":"2024-09-21T06:14:54.030782+0000","last_clean":"2024-09-21T06:14:54.030782+0000","last_became_active":"2024-09-21T06:13:10.512830+0000","last_became_peered":"2024-09-21T06:13:10.512830+0000","last_unstale":"2024-09-21T06:14:54.030782+0000","last_undegraded":"2024-09-21T06:14:54.030782+0000","last_fullsized":"2024-09-21T06:14:54.030782+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:46.418161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.030499+0000","last_change":"2024-09-21T06:13:10.518863+0000","last_active":"2024-09-21T06:14:54.030499+0000","last_peered":"2024-09-21T06:14:54.030499+0000","last_clean":"2024-09-21T06:14:54.030499+0000","last_became_active":"2024-09-21T06:13:10.518492+0000","last_became_peered":"2024-09-21T06:13:10.518492+0000","last_unstale":"2024-09-21T06:14:54.030499+0000","last_undegraded":"2024-09-21T06:14:54.030499+0000","last_fullsized":"2024-09-21T06:14:54.030499+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:23.770763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.031560+0000","last_change":"2024-09-21T06:13:10.508538+0000","last_active":"2024-09-21T06:14:54.031560+0000","last_peered":"2024-09-21T06:14:54.031560+0000","last_clean":"2024-09-21T06:14:54.031560+0000","last_became_active":"2024-09-21T06:13:10.508017+0000","last_became_peered":"2024-09-21T06:13:10.508017+0000","last_unstale":"2024-09-21T06:14:54.031560+0000","last_undegraded":"2024-09-21T06:14:54.031560+0000","last_fullsized":"2024-09-21T06:14:54.031560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:00.168741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.026465+0000","last_change":"2024-09-21T06:13:10.516734+0000","last_active":"2024-09-21T06:14:54.026465+0000","last_peered":"2024-09-21T06:14:54.026465+0000","last_clean":"2024-09-21T06:14:54.026465+0000","last_became_active":"2024-09-21T06:13:10.516362+0000","last_became_peered":"2024-09-21T06:13:10.516362+0000","last_unstale":"2024-09-21T06:14:54.026465+0000","last_undegraded":"2024-09-21T06:14:54.026465+0000","last_fullsized":"2024-09-21T06:14:54.026465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:49.792011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.036688+0000","last_change":"2024-09-21T06:13:10.513143+0000","last_active":"2024-09-21T06:14:54.036688+0000","last_peered":"2024-09-21T06:14:54.036688+0000","last_clean":"2024-09-21T06:14:54.036688+0000","last_became_active":"2024-09-21T06:13:10.512338+0000","last_became_peered":"2024-09-21T06:13:10.512338+0000","last_unstale":"2024-09-21T06:14:54.036688+0000","last_undegraded":"2024-09-21T06:14:54.036688+0000","last_fullsized":"2024-09-21T06:14:54.036688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:02.963860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.022101+0000","last_change":"2024-09-21T06:13:10.513429+0000","last_active":"2024-09-21T06:14:54.022101+0000","last_peered":"2024-09-21T06:14:54.022101+0000","last_clean":"2024-09-21T06:14:54.022101+0000","last_became_active":"2024-09-21T06:13:10.512995+0000","last_became_peered":"2024-09-21T06:13:10.512995+0000","last_unstale":"2024-09-21T06:14:54.022101+0000","last_undegraded":"2024-09-21T06:14:54.022101+0000","last_fullsized":"2024-09-21T06:14:54.022101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:39.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.032522+0000","last_change":"2024-09-21T06:13:10.509580+0000","last_active":"2024-09-21T06:14:54.032522+0000","last_peered":"2024-09-21T06:14:54.032522+0000","last_clean":"2024-09-21T06:14:54.032522+0000","last_became_active":"2024-09-21T06:13:10.509179+0000","last_became_peered":"2024-09-21T06:13:10.509179+0000","last_unstale":"2024-09-21T06:14:54.032522+0000","last_undegraded":"2024-09-21T06:14:54.032522+0000","last_fullsized":"2024-09-21T06:14:54.032522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:39.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.036310+0000","last_change":"2024-09-21T06:13:10.508358+0000","last_active":"2024-09-21T06:14:54.036310+0000","last_peered":"2024-09-21T06:14:54.036310+0000","last_clean":"2024-09-21T06:14:54.036310+0000","last_became_active":"2024-09-21T06:13:10.507952+0000","last_became_peered":"2024-09-21T06:13:10.507952+0000","last_unstale":"2024-09-21T06:14:54.036310+0000","last_undegraded":"2024-09-21T06:14:54.036310+0000","last_fullsized":"2024-09-21T06:14:54.036310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:55.565203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.024319+0000","last_change":"2024-09-21T06:13:10.510580+0000","last_active":"2024-09-21T06:14:54.024319+0000","last_peered":"2024-09-21T06:14:54.024319+0000","last_clean":"2024-09-21T06:14:54.024319+0000","last_became_active":"2024-09-21T06:13:10.510175+0000","last_became_peered":"2024-09-21T06:13:10.510175+0000","last_unstale":"2024-09-21T06:14:54.024319+0000","last_undegraded":"2024-09-21T06:14:54.024319+0000","last_fullsized":"2024-09-21T06:14:54.024319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:49.467057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'8","reported_seq":141,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:07.766031+0000","last_change":"2024-09-21T06:13:08.485527+0000","last_active":"2024-09-21T06:15:07.766031+0000","last_peered":"2024-09-21T06:15:07.766031+0000","last_clean":"2024-09-21T06:15:07.766031+0000","last_became_active":"2024-09-21T06:13:08.485094+0000","last_became_peered":"2024-09-21T06:13:08.485094+0000","last_unstale":"2024-09-21T06:15:07.766031+0000","last_undegraded":"2024-09-21T06:15:07.766031+0000","last_fullsized":"2024-09-21T06:15:07.766031+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:48.285638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.034456+0000","last_change":"2024-09-21T06:13:10.507977+0000","last_active":"2024-09-21T06:14:54.034456+0000","last_peered":"2024-09-21T06:14:54.034456+0000","last_clean":"2024-09-21T06:14:54.034456+0000","last_became_active":"2024-09-21T06:13:10.507594+0000","last_became_peered":"2024-09-21T06:13:10.507594+0000","last_unstale":"2024-09-21T06:14:54.034456+0000","last_undegraded":"2024-09-21T06:14:54.034456+0000","last_fullsized":"2024-09-21T06:14:54.034456+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:57.318240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.599690+0000","last_change":"2024-09-21T06:13:08.491237+0000","last_active":"2024-09-21T06:13:24.599690+0000","last_peered":"2024-09-21T06:13:24.599690+0000","last_clean":"2024-09-21T06:13:24.599690+0000","last_became_active":"2024-09-21T06:13:08.490908+0000","last_became_peered":"2024-09-21T06:13:08.490908+0000","last_unstale":"2024-09-21T06:13:24.599690+0000","last_undegraded":"2024-09-21T06:13:24.599690+0000","last_fullsized":"2024-09-21T06:13:24.599690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:53.185029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597228+0000","last_change":"2024-09-21T06:13:08.488721+0000","last_active":"2024-09-21T06:13:24.597228+0000","last_peered":"2024-09-21T06:13:24.597228+0000","last_clean":"2024-09-21T06:13:24.597228+0000","last_became_active":"2024-09-21T06:13:08.488347+0000","last_became_peered":"2024-09-21T06:13:08.488347+0000","last_unstale":"2024-09-21T06:13:24.597228+0000","last_undegraded":"2024-09-21T06:13:24.597228+0000","last_fullsized":"2024-09-21T06:13:24.597228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:27.008479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.034347+0000","last_change":"2024-09-21T06:13:10.506527+0000","last_active":"2024-09-21T06:14:54.034347+0000","last_peered":"2024-09-21T06:14:54.034347+0000","last_clean":"2024-09-21T06:14:54.034347+0000","last_became_active":"2024-09-21T06:13:10.506163+0000","last_became_peered":"2024-09-21T06:13:10.506163+0000","last_unstale":"2024-09-21T06:14:54.034347+0000","last_undegraded":"2024-09-21T06:14:54.034347+0000","last_fullsized":"2024-09-21T06:14:54.034347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.244097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604150+0000","last_change":"2024-09-21T06:13:08.489292+0000","last_active":"2024-09-21T06:13:24.604150+0000","last_peered":"2024-09-21T06:13:24.604150+0000","last_clean":"2024-09-21T06:13:24.604150+0000","last_became_active":"2024-09-21T06:13:08.488918+0000","last_became_peered":"2024-09-21T06:13:08.488918+0000","last_unstale":"2024-09-21T06:13:24.604150+0000","last_undegraded":"2024-09-21T06:13:24.604150+0000","last_fullsized":"2024-09-21T06:13:24.604150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:37.992260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.036458+0000","last_change":"2024-09-21T06:13:10.505027+0000","last_active":"2024-09-21T06:14:54.036458+0000","last_peered":"2024-09-21T06:14:54.036458+0000","last_clean":"2024-09-21T06:14:54.036458+0000","last_became_active":"2024-09-21T06:13:10.504669+0000","last_became_peered":"2024-09-21T06:13:10.504669+0000","last_unstale":"2024-09-21T06:14:54.036458+0000","last_undegraded":"2024-09-21T06:14:54.036458+0000","last_fullsized":"2024-09-21T06:14:54.036458+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:43.927229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'8","reported_seq":246,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:10.770506+0000","last_change":"2024-09-21T06:13:08.491101+0000","last_active":"2024-09-21T06:15:10.770506+0000","last_peered":"2024-09-21T06:15:10.770506+0000","last_clean":"2024-09-21T06:15:10.770506+0000","last_became_active":"2024-09-21T06:13:08.490607+0000","last_became_peered":"2024-09-21T06:13:08.490607+0000","last_unstale":"2024-09-21T06:15:10.770506+0000","last_undegraded":"2024-09-21T06:15:10.770506+0000","last_fullsized":"2024-09-21T06:15:10.770506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:16.758463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":215,"num_read_kb":215,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.043733+0000","last_change":"2024-09-21T06:13:10.510453+0000","last_active":"2024-09-21T06:14:54.043733+0000","last_peered":"2024-09-21T06:14:54.043733+0000","last_clean":"2024-09-21T06:14:54.043733+0000","last_became_active":"2024-09-21T06:13:10.508513+0000","last_became_peered":"2024-09-21T06:13:10.508513+0000","last_unstale":"2024-09-21T06:14:54.043733+0000","last_undegraded":"2024-09-21T06:14:54.043733+0000","last_fullsized":"2024-09-21T06:14:54.043733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:48.388140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609878+0000","last_change":"2024-09-21T06:13:08.488714+0000","last_active":"2024-09-21T06:13:24.609878+0000","last_peered":"2024-09-21T06:13:24.609878+0000","last_clean":"2024-09-21T06:13:24.609878+0000","last_became_active":"2024-09-21T06:13:08.488363+0000","last_became_peered":"2024-09-21T06:13:08.488363+0000","last_unstale":"2024-09-21T06:13:24.609878+0000","last_undegraded":"2024-09-21T06:13:24.609878+0000","last_fullsized":"2024-09-21T06:13:24.609878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:33.626165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037460+0000","last_change":"2024-09-21T06:13:10.508046+0000","last_active":"2024-09-21T06:14:54.037460+0000","last_peered":"2024-09-21T06:14:54.037460+0000","last_clean":"2024-09-21T06:14:54.037460+0000","last_became_active":"2024-09-21T06:13:10.507660+0000","last_became_peered":"2024-09-21T06:13:10.507660+0000","last_unstale":"2024-09-21T06:14:54.037460+0000","last_undegraded":"2024-09-21T06:14:54.037460+0000","last_fullsized":"2024-09-21T06:14:54.037460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:35.466965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603272+0000","last_change":"2024-09-21T06:13:08.562863+0000","last_active":"2024-09-21T06:13:24.603272+0000","last_peered":"2024-09-21T06:13:24.603272+0000","last_clean":"2024-09-21T06:13:24.603272+0000","last_became_active":"2024-09-21T06:13:08.562424+0000","last_became_peered":"2024-09-21T06:13:08.562424+0000","last_unstale":"2024-09-21T06:13:24.603272+0000","last_undegraded":"2024-09-21T06:13:24.603272+0000","last_fullsized":"2024-09-21T06:13:24.603272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:43.251932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.044625+0000","last_change":"2024-09-21T06:13:10.509818+0000","last_active":"2024-09-21T06:14:54.044625+0000","last_peered":"2024-09-21T06:14:54.044625+0000","last_clean":"2024-09-21T06:14:54.044625+0000","last_became_active":"2024-09-21T06:13:10.509468+0000","last_became_peered":"2024-09-21T06:13:10.509468+0000","last_unstale":"2024-09-21T06:14:54.044625+0000","last_undegraded":"2024-09-21T06:14:54.044625+0000","last_fullsized":"2024-09-21T06:14:54.044625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:25.130868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'38","reported_seq":307,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:10.770919+0000","last_change":"2024-09-21T06:13:24.633807+0000","last_active":"2024-09-21T06:15:10.770919+0000","last_peered":"2024-09-21T06:15:10.770919+0000","last_clean":"2024-09-21T06:15:10.770919+0000","last_became_active":"2024-09-21T06:13:24.618599+0000","last_became_peered":"2024-09-21T06:13:24.618599+0000","last_unstale":"2024-09-21T06:15:10.770919+0000","last_undegraded":"2024-09-21T06:15:10.770919+0000","last_fullsized":"2024-09-21T06:15:10.770919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:07.449253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73783,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":225,"num_read_kb":181,"num_write":40,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.033514+0000","last_change":"2024-09-21T06:13:10.511360+0000","last_active":"2024-09-21T06:14:54.033514+0000","last_peered":"2024-09-21T06:14:54.033514+0000","last_clean":"2024-09-21T06:14:54.033514+0000","last_became_active":"2024-09-21T06:13:10.510980+0000","last_became_peered":"2024-09-21T06:13:10.510980+0000","last_unstale":"2024-09-21T06:14:54.033514+0000","last_undegraded":"2024-09-21T06:14:54.033514+0000","last_fullsized":"2024-09-21T06:14:54.033514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:43.859758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604427+0000","last_change":"2024-09-21T06:13:08.486262+0000","last_active":"2024-09-21T06:13:24.604427+0000","last_peered":"2024-09-21T06:13:24.604427+0000","last_clean":"2024-09-21T06:13:24.604427+0000","last_became_active":"2024-09-21T06:13:08.485509+0000","last_became_peered":"2024-09-21T06:13:08.485509+0000","last_unstale":"2024-09-21T06:13:24.604427+0000","last_undegraded":"2024-09-21T06:13:24.604427+0000","last_fullsized":"2024-09-21T06:13:24.604427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:31.726746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.033404+0000","last_change":"2024-09-21T06:13:10.506894+0000","last_active":"2024-09-21T06:14:54.033404+0000","last_peered":"2024-09-21T06:14:54.033404+0000","last_clean":"2024-09-21T06:14:54.033404+0000","last_became_active":"2024-09-21T06:13:10.506489+0000","last_became_peered":"2024-09-21T06:13:10.506489+0000","last_unstale":"2024-09-21T06:14:54.033404+0000","last_undegraded":"2024-09-21T06:14:54.033404+0000","last_fullsized":"2024-09-21T06:14:54.033404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:58.755601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603141+0000","last_change":"2024-09-21T06:13:08.483429+0000","last_active":"2024-09-21T06:13:24.603141+0000","last_peered":"2024-09-21T06:13:24.603141+0000","last_clean":"2024-09-21T06:13:24.603141+0000","last_became_active":"2024-09-21T06:13:08.483028+0000","last_became_peered":"2024-09-21T06:13:08.483028+0000","last_unstale":"2024-09-21T06:13:24.603141+0000","last_undegraded":"2024-09-21T06:13:24.603141+0000","last_fullsized":"2024-09-21T06:13:24.603141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:33.258154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.042922+0000","last_change":"2024-09-21T06:13:10.513398+0000","last_active":"2024-09-21T06:14:54.042922+0000","last_peered":"2024-09-21T06:14:54.042922+0000","last_clean":"2024-09-21T06:14:54.042922+0000","last_became_active":"2024-09-21T06:13:10.512713+0000","last_became_peered":"2024-09-21T06:13:10.512713+0000","last_unstale":"2024-09-21T06:14:54.042922+0000","last_undegraded":"2024-09-21T06:14:54.042922+0000","last_fullsized":"2024-09-21T06:14:54.042922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:21.026984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'7","reported_seq":137,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:05.762659+0000","last_change":"2024-09-21T06:13:08.493771+0000","last_active":"2024-09-21T06:15:05.762659+0000","last_peered":"2024-09-21T06:15:05.762659+0000","last_clean":"2024-09-21T06:15:05.762659+0000","last_became_active":"2024-09-21T06:13:08.493380+0000","last_became_peered":"2024-09-21T06:13:08.493380+0000","last_unstale":"2024-09-21T06:15:05.762659+0000","last_undegraded":"2024-09-21T06:15:05.762659+0000","last_fullsized":"2024-09-21T06:15:05.762659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:12.552137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.041365+0000","last_change":"2024-09-21T06:13:10.511679+0000","last_active":"2024-09-21T06:14:54.041365+0000","last_peered":"2024-09-21T06:14:54.041365+0000","last_clean":"2024-09-21T06:14:54.041365+0000","last_became_active":"2024-09-21T06:13:10.511270+0000","last_became_peered":"2024-09-21T06:13:10.511270+0000","last_unstale":"2024-09-21T06:14:54.041365+0000","last_undegraded":"2024-09-21T06:14:54.041365+0000","last_fullsized":"2024-09-21T06:14:54.041365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:10.521853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.627560+0000","last_change":"2024-09-21T06:13:24.627467+0000","last_active":"2024-09-21T06:13:24.627560+0000","last_peered":"2024-09-21T06:13:24.627560+0000","last_clean":"2024-09-21T06:13:24.627560+0000","last_became_active":"2024-09-21T06:13:24.610263+0000","last_became_peered":"2024-09-21T06:13:24.610263+0000","last_unstale":"2024-09-21T06:13:24.627560+0000","last_undegraded":"2024-09-21T06:13:24.627560+0000","last_fullsized":"2024-09-21T06:13:24.627560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:26.472147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.032186+0000","last_change":"2024-09-21T06:13:10.517214+0000","last_active":"2024-09-21T06:14:54.032186+0000","last_peered":"2024-09-21T06:14:54.032186+0000","last_clean":"2024-09-21T06:14:54.032186+0000","last_became_active":"2024-09-21T06:13:10.516828+0000","last_became_peered":"2024-09-21T06:13:10.516828+0000","last_unstale":"2024-09-21T06:14:54.032186+0000","last_undegraded":"2024-09-21T06:14:54.032186+0000","last_fullsized":"2024-09-21T06:14:54.032186+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:58.077670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608703+0000","last_change":"2024-09-21T06:13:08.489153+0000","last_active":"2024-09-21T06:13:24.608703+0000","last_peered":"2024-09-21T06:13:24.608703+0000","last_clean":"2024-09-21T06:13:24.608703+0000","last_became_active":"2024-09-21T06:13:08.488732+0000","last_became_peered":"2024-09-21T06:13:08.488732+0000","last_unstale":"2024-09-21T06:13:24.608703+0000","last_undegraded":"2024-09-21T06:13:24.608703+0000","last_fullsized":"2024-09-21T06:13:24.608703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:22.848136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037298+0000","last_change":"2024-09-21T06:13:24.610441+0000","last_active":"2024-09-21T06:14:54.037298+0000","last_peered":"2024-09-21T06:14:54.037298+0000","last_clean":"2024-09-21T06:14:54.037298+0000","last_became_active":"2024-09-21T06:13:24.609992+0000","last_became_peered":"2024-09-21T06:13:24.609992+0000","last_unstale":"2024-09-21T06:14:54.037298+0000","last_undegraded":"2024-09-21T06:14:54.037298+0000","last_fullsized":"2024-09-21T06:14:54.037298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:59.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597725+0000","last_change":"2024-09-21T06:13:08.484514+0000","last_active":"2024-09-21T06:13:24.597725+0000","last_peered":"2024-09-21T06:13:24.597725+0000","last_clean":"2024-09-21T06:13:24.597725+0000","last_became_active":"2024-09-21T06:13:08.484134+0000","last_became_peered":"2024-09-21T06:13:08.484134+0000","last_unstale":"2024-09-21T06:13:24.597725+0000","last_undegraded":"2024-09-21T06:13:24.597725+0000","last_fullsized":"2024-09-21T06:13:24.597725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:45.124734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.039200+0000","last_change":"2024-09-21T06:13:10.503798+0000","last_active":"2024-09-21T06:14:54.039200+0000","last_peered":"2024-09-21T06:14:54.039200+0000","last_clean":"2024-09-21T06:14:54.039200+0000","last_became_active":"2024-09-21T06:13:10.502418+0000","last_became_peered":"2024-09-21T06:13:10.502418+0000","last_unstale":"2024-09-21T06:14:54.039200+0000","last_undegraded":"2024-09-21T06:14:54.039200+0000","last_fullsized":"2024-09-21T06:14:54.039200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:57.575569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603439+0000","last_change":"2024-09-21T06:13:08.487191+0000","last_active":"2024-09-21T06:13:24.603439+0000","last_peered":"2024-09-21T06:13:24.603439+0000","last_clean":"2024-09-21T06:13:24.603439+0000","last_became_active":"2024-09-21T06:13:08.486812+0000","last_became_peered":"2024-09-21T06:13:08.486812+0000","last_unstale":"2024-09-21T06:13:24.603439+0000","last_undegraded":"2024-09-21T06:13:24.603439+0000","last_fullsized":"2024-09-21T06:13:24.603439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:51.989708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.041431+0000","last_change":"2024-09-21T06:13:10.511678+0000","last_active":"2024-09-21T06:14:54.041431+0000","last_peered":"2024-09-21T06:14:54.041431+0000","last_clean":"2024-09-21T06:14:54.041431+0000","last_became_active":"2024-09-21T06:13:10.510408+0000","last_became_peered":"2024-09-21T06:13:10.510408+0000","last_unstale":"2024-09-21T06:14:54.041431+0000","last_undegraded":"2024-09-21T06:14:54.041431+0000","last_fullsized":"2024-09-21T06:14:54.041431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:58.711190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'7","reported_seq":139,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:07.766522+0000","last_change":"2024-09-21T06:13:08.560894+0000","last_active":"2024-09-21T06:15:07.766522+0000","last_peered":"2024-09-21T06:15:07.766522+0000","last_clean":"2024-09-21T06:15:07.766522+0000","last_became_active":"2024-09-21T06:13:08.560423+0000","last_became_peered":"2024-09-21T06:13:08.560423+0000","last_unstale":"2024-09-21T06:15:07.766522+0000","last_undegraded":"2024-09-21T06:15:07.766522+0000","last_fullsized":"2024-09-21T06:15:07.766522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:53.015711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.044084+0000","last_change":"2024-09-21T06:13:10.516009+0000","last_active":"2024-09-21T06:14:54.044084+0000","last_peered":"2024-09-21T06:14:54.044084+0000","last_clean":"2024-09-21T06:14:54.044084+0000","last_became_active":"2024-09-21T06:13:10.515669+0000","last_became_peered":"2024-09-21T06:13:10.515669+0000","last_unstale":"2024-09-21T06:14:54.044084+0000","last_undegraded":"2024-09-21T06:14:54.044084+0000","last_fullsized":"2024-09-21T06:14:54.044084+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:54.134564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":100,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610370+0000","last_change":"2024-09-21T06:11:53.408030+0000","last_active":"2024-09-21T06:13:24.610370+0000","last_peered":"2024-09-21T06:13:24.610370+0000","last_clean":"2024-09-21T06:13:24.610370+0000","last_became_active":"2024-09-21T06:11:53.402296+0000","last_became_peered":"2024-09-21T06:11:53.402296+0000","last_unstale":"2024-09-21T06:13:24.610370+0000","last_undegraded":"2024-09-21T06:13:24.610370+0000","last_fullsized":"2024-09-21T06:13:24.610370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_clean_scrub_stamp":"2024-09-21T06:08:26.098183+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:08.239872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616959+0000","last_change":"2024-09-21T06:13:08.489209+0000","last_active":"2024-09-21T06:13:24.616959+0000","last_peered":"2024-09-21T06:13:24.616959+0000","last_clean":"2024-09-21T06:13:24.616959+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.616959+0000","last_undegraded":"2024-09-21T06:13:24.616959+0000","last_fullsized":"2024-09-21T06:13:24.616959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:38.798499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.038080+0000","last_change":"2024-09-21T06:13:24.606487+0000","last_active":"2024-09-21T06:14:54.038080+0000","last_peered":"2024-09-21T06:14:54.038080+0000","last_clean":"2024-09-21T06:14:54.038080+0000","last_became_active":"2024-09-21T06:13:24.606141+0000","last_became_peered":"2024-09-21T06:13:24.606141+0000","last_unstale":"2024-09-21T06:14:54.038080+0000","last_undegraded":"2024-09-21T06:14:54.038080+0000","last_fullsized":"2024-09-21T06:14:54.038080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:24.040770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610270+0000","last_change":"2024-09-21T06:13:08.489206+0000","last_active":"2024-09-21T06:13:24.610270+0000","last_peered":"2024-09-21T06:13:24.610270+0000","last_clean":"2024-09-21T06:13:24.610270+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.610270+0000","last_undegraded":"2024-09-21T06:13:24.610270+0000","last_fullsized":"2024-09-21T06:13:24.610270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.627305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.027371+0000","last_change":"2024-09-21T06:13:10.513999+0000","last_active":"2024-09-21T06:14:54.027371+0000","last_peered":"2024-09-21T06:14:54.027371+0000","last_clean":"2024-09-21T06:14:54.027371+0000","last_became_active":"2024-09-21T06:13:10.513657+0000","last_became_peered":"2024-09-21T06:13:10.513657+0000","last_unstale":"2024-09-21T06:14:54.027371+0000","last_undegraded":"2024-09-21T06:14:54.027371+0000","last_fullsized":"2024-09-21T06:14:54.027371+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:05.536921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'6","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:07.765185+0000","last_change":"2024-09-21T06:13:08.560811+0000","last_active":"2024-09-21T06:15:07.765185+0000","last_peered":"2024-09-21T06:15:07.765185+0000","last_clean":"2024-09-21T06:15:07.765185+0000","last_became_active":"2024-09-21T06:13:08.560145+0000","last_became_peered":"2024-09-21T06:13:08.560145+0000","last_unstale":"2024-09-21T06:15:07.765185+0000","last_undegraded":"2024-09-21T06:15:07.765185+0000","last_fullsized":"2024-09-21T06:15:07.765185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:24.550008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035762+0000","last_change":"2024-09-21T06:13:24.613495+0000","last_active":"2024-09-21T06:14:54.035762+0000","last_peered":"2024-09-21T06:14:54.035762+0000","last_clean":"2024-09-21T06:14:54.035762+0000","last_became_active":"2024-09-21T06:13:24.612634+0000","last_became_peered":"2024-09-21T06:13:24.612634+0000","last_unstale":"2024-09-21T06:14:54.035762+0000","last_undegraded":"2024-09-21T06:14:54.035762+0000","last_fullsized":"2024-09-21T06:14:54.035762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.863141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'5","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:07.768565+0000","last_change":"2024-09-21T06:13:08.486973+0000","last_active":"2024-09-21T06:15:07.768565+0000","last_peered":"2024-09-21T06:15:07.768565+0000","last_clean":"2024-09-21T06:15:07.768565+0000","last_became_active":"2024-09-21T06:13:08.486602+0000","last_became_peered":"2024-09-21T06:13:08.486602+0000","last_unstale":"2024-09-21T06:15:07.768565+0000","last_undegraded":"2024-09-21T06:15:07.768565+0000","last_fullsized":"2024-09-21T06:15:07.768565+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:41.804955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.029271+0000","last_change":"2024-09-21T06:13:10.509499+0000","last_active":"2024-09-21T06:14:54.029271+0000","last_peered":"2024-09-21T06:14:54.029271+0000","last_clean":"2024-09-21T06:14:54.029271+0000","last_became_active":"2024-09-21T06:13:10.508566+0000","last_became_peered":"2024-09-21T06:13:10.508566+0000","last_unstale":"2024-09-21T06:14:54.029271+0000","last_undegraded":"2024-09-21T06:14:54.029271+0000","last_fullsized":"2024-09-21T06:14:54.029271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:12.701631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600737+0000","last_change":"2024-09-21T06:13:08.561378+0000","last_active":"2024-09-21T06:13:24.600737+0000","last_peered":"2024-09-21T06:13:24.600737+0000","last_clean":"2024-09-21T06:13:24.600737+0000","last_became_active":"2024-09-21T06:13:08.561002+0000","last_became_peered":"2024-09-21T06:13:08.561002+0000","last_unstale":"2024-09-21T06:13:24.600737+0000","last_undegraded":"2024-09-21T06:13:24.600737+0000","last_fullsized":"2024-09-21T06:13:24.600737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:11.325377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.030695+0000","last_change":"2024-09-21T06:13:24.602347+0000","last_active":"2024-09-21T06:14:54.030695+0000","last_peered":"2024-09-21T06:14:54.030695+0000","last_clean":"2024-09-21T06:14:54.030695+0000","last_became_active":"2024-09-21T06:13:24.601978+0000","last_became_peered":"2024-09-21T06:13:24.601978+0000","last_unstale":"2024-09-21T06:14:54.030695+0000","last_undegraded":"2024-09-21T06:14:54.030695+0000","last_fullsized":"2024-09-21T06:14:54.030695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:34.860096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608569+0000","last_change":"2024-09-21T06:13:08.561937+0000","last_active":"2024-09-21T06:13:24.608569+0000","last_peered":"2024-09-21T06:13:24.608569+0000","last_clean":"2024-09-21T06:13:24.608569+0000","last_became_active":"2024-09-21T06:13:08.560547+0000","last_became_peered":"2024-09-21T06:13:08.560547+0000","last_unstale":"2024-09-21T06:13:24.608569+0000","last_undegraded":"2024-09-21T06:13:24.608569+0000","last_fullsized":"2024-09-21T06:13:24.608569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:29.702291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.027723+0000","last_change":"2024-09-21T06:13:10.516833+0000","last_active":"2024-09-21T06:14:54.027723+0000","last_peered":"2024-09-21T06:14:54.027723+0000","last_clean":"2024-09-21T06:14:54.027723+0000","last_became_active":"2024-09-21T06:13:10.516486+0000","last_became_peered":"2024-09-21T06:13:10.516486+0000","last_unstale":"2024-09-21T06:14:54.027723+0000","last_undegraded":"2024-09-21T06:14:54.027723+0000","last_fullsized":"2024-09-21T06:14:54.027723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:05.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'19","reported_seq":42,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:08.513647+0000","last_change":"2024-09-21T06:13:08.483271+0000","last_active":"2024-09-21T06:15:08.513647+0000","last_peered":"2024-09-21T06:15:08.513647+0000","last_clean":"2024-09-21T06:15:08.513647+0000","last_became_active":"2024-09-21T06:13:08.482951+0000","last_became_peered":"2024-09-21T06:13:08.482951+0000","last_unstale":"2024-09-21T06:15:08.513647+0000","last_undegraded":"2024-09-21T06:15:08.513647+0000","last_fullsized":"2024-09-21T06:15:08.513647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:17.537988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.043217+0000","last_change":"2024-09-21T06:13:10.509125+0000","last_active":"2024-09-21T06:14:54.043217+0000","last_peered":"2024-09-21T06:14:54.043217+0000","last_clean":"2024-09-21T06:14:54.043217+0000","last_became_active":"2024-09-21T06:13:10.508773+0000","last_became_peered":"2024-09-21T06:13:10.508773+0000","last_unstale":"2024-09-21T06:14:54.043217+0000","last_undegraded":"2024-09-21T06:14:54.043217+0000","last_fullsized":"2024-09-21T06:14:54.043217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:44.570477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616932+0000","last_change":"2024-09-21T06:13:08.497885+0000","last_active":"2024-09-21T06:13:24.616932+0000","last_peered":"2024-09-21T06:13:24.616932+0000","last_clean":"2024-09-21T06:13:24.616932+0000","last_became_active":"2024-09-21T06:13:08.497593+0000","last_became_peered":"2024-09-21T06:13:08.497593+0000","last_unstale":"2024-09-21T06:13:24.616932+0000","last_undegraded":"2024-09-21T06:13:24.616932+0000","last_fullsized":"2024-09-21T06:13:24.616932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:48.361175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037327+0000","last_change":"2024-09-21T06:13:10.514430+0000","last_active":"2024-09-21T06:14:54.037327+0000","last_peered":"2024-09-21T06:14:54.037327+0000","last_clean":"2024-09-21T06:14:54.037327+0000","last_became_active":"2024-09-21T06:13:10.514063+0000","last_became_peered":"2024-09-21T06:13:10.514063+0000","last_unstale":"2024-09-21T06:14:54.037327+0000","last_undegraded":"2024-09-21T06:14:54.037327+0000","last_fullsized":"2024-09-21T06:14:54.037327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:37.036497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598415+0000","last_change":"2024-09-21T06:13:08.560564+0000","last_active":"2024-09-21T06:13:24.598415+0000","last_peered":"2024-09-21T06:13:24.598415+0000","last_clean":"2024-09-21T06:13:24.598415+0000","last_became_active":"2024-09-21T06:13:08.560339+0000","last_became_peered":"2024-09-21T06:13:08.560339+0000","last_unstale":"2024-09-21T06:13:24.598415+0000","last_undegraded":"2024-09-21T06:13:24.598415+0000","last_fullsized":"2024-09-21T06:13:24.598415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:55.378911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.022629+0000","last_change":"2024-09-21T06:13:10.511354+0000","last_active":"2024-09-21T06:14:54.022629+0000","last_peered":"2024-09-21T06:14:54.022629+0000","last_clean":"2024-09-21T06:14:54.022629+0000","last_became_active":"2024-09-21T06:13:10.510946+0000","last_became_peered":"2024-09-21T06:13:10.510946+0000","last_unstale":"2024-09-21T06:14:54.022629+0000","last_undegraded":"2024-09-21T06:14:54.022629+0000","last_fullsized":"2024-09-21T06:14:54.022629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:11.535461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610883+0000","last_change":"2024-09-21T06:13:08.485578+0000","last_active":"2024-09-21T06:13:24.610883+0000","last_peered":"2024-09-21T06:13:24.610883+0000","last_clean":"2024-09-21T06:13:24.610883+0000","last_became_active":"2024-09-21T06:13:08.485170+0000","last_became_peered":"2024-09-21T06:13:08.485170+0000","last_unstale":"2024-09-21T06:13:24.610883+0000","last_undegraded":"2024-09-21T06:13:24.610883+0000","last_fullsized":"2024-09-21T06:13:24.610883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:39.000114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035886+0000","last_change":"2024-09-21T06:13:10.510360+0000","last_active":"2024-09-21T06:14:54.035886+0000","last_peered":"2024-09-21T06:14:54.035886+0000","last_clean":"2024-09-21T06:14:54.035886+0000","last_became_active":"2024-09-21T06:13:10.509887+0000","last_became_peered":"2024-09-21T06:13:10.509887+0000","last_unstale":"2024-09-21T06:14:54.035886+0000","last_undegraded":"2024-09-21T06:14:54.035886+0000","last_fullsized":"2024-09-21T06:14:54.035886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:58.135706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.625797+0000","last_change":"2024-09-21T06:13:24.625284+0000","last_active":"2024-09-21T06:13:24.625797+0000","last_peered":"2024-09-21T06:13:24.625797+0000","last_clean":"2024-09-21T06:13:24.625797+0000","last_became_active":"2024-09-21T06:13:24.612704+0000","last_became_peered":"2024-09-21T06:13:24.612704+0000","last_unstale":"2024-09-21T06:13:24.625797+0000","last_undegraded":"2024-09-21T06:13:24.625797+0000","last_fullsized":"2024-09-21T06:13:24.625797+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:29.385258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.029990+0000","last_change":"2024-09-21T06:13:10.514128+0000","last_active":"2024-09-21T06:14:54.029990+0000","last_peered":"2024-09-21T06:14:54.029990+0000","last_clean":"2024-09-21T06:14:54.029990+0000","last_became_active":"2024-09-21T06:13:10.513790+0000","last_became_peered":"2024-09-21T06:13:10.513790+0000","last_unstale":"2024-09-21T06:14:54.029990+0000","last_undegraded":"2024-09-21T06:14:54.029990+0000","last_fullsized":"2024-09-21T06:14:54.029990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:33.407189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607762+0000","last_change":"2024-09-21T06:13:08.489105+0000","last_active":"2024-09-21T06:13:24.607762+0000","last_peered":"2024-09-21T06:13:24.607762+0000","last_clean":"2024-09-21T06:13:24.607762+0000","last_became_active":"2024-09-21T06:13:08.487110+0000","last_became_peered":"2024-09-21T06:13:08.487110+0000","last_unstale":"2024-09-21T06:13:24.607762+0000","last_undegraded":"2024-09-21T06:13:24.607762+0000","last_fullsized":"2024-09-21T06:13:24.607762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:24.849365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.044351+0000","last_change":"2024-09-21T06:13:10.513751+0000","last_active":"2024-09-21T06:14:54.044351+0000","last_peered":"2024-09-21T06:14:54.044351+0000","last_clean":"2024-09-21T06:14:54.044351+0000","last_became_active":"2024-09-21T06:13:10.513376+0000","last_became_peered":"2024-09-21T06:13:10.513376+0000","last_unstale":"2024-09-21T06:14:54.044351+0000","last_undegraded":"2024-09-21T06:14:54.044351+0000","last_fullsized":"2024-09-21T06:14:54.044351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.364959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.039103+0000","last_change":"2024-09-21T06:13:10.519359+0000","last_active":"2024-09-21T06:14:54.039103+0000","last_peered":"2024-09-21T06:14:54.039103+0000","last_clean":"2024-09-21T06:14:54.039103+0000","last_became_active":"2024-09-21T06:13:10.518518+0000","last_became_peered":"2024-09-21T06:13:10.518518+0000","last_unstale":"2024-09-21T06:14:54.039103+0000","last_undegraded":"2024-09-21T06:14:54.039103+0000","last_fullsized":"2024-09-21T06:14:54.039103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:55.554710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608166+0000","last_change":"2024-09-21T06:13:08.488122+0000","last_active":"2024-09-21T06:13:24.608166+0000","last_peered":"2024-09-21T06:13:24.608166+0000","last_clean":"2024-09-21T06:13:24.608166+0000","last_became_active":"2024-09-21T06:13:08.487679+0000","last_became_peered":"2024-09-21T06:13:08.487679+0000","last_unstale":"2024-09-21T06:13:24.608166+0000","last_undegraded":"2024-09-21T06:13:24.608166+0000","last_fullsized":"2024-09-21T06:13:24.608166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:12.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.043317+0000","last_change":"2024-09-21T06:13:10.516794+0000","last_active":"2024-09-21T06:14:54.043317+0000","last_peered":"2024-09-21T06:14:54.043317+0000","last_clean":"2024-09-21T06:14:54.043317+0000","last_became_active":"2024-09-21T06:13:10.516297+0000","last_became_peered":"2024-09-21T06:13:10.516297+0000","last_unstale":"2024-09-21T06:14:54.043317+0000","last_undegraded":"2024-09-21T06:14:54.043317+0000","last_fullsized":"2024-09-21T06:14:54.043317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:19.873742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.621488+0000","last_change":"2024-09-21T06:13:24.621358+0000","last_active":"2024-09-21T06:13:24.621488+0000","last_peered":"2024-09-21T06:13:24.621488+0000","last_clean":"2024-09-21T06:13:24.621488+0000","last_became_active":"2024-09-21T06:13:24.611058+0000","last_became_peered":"2024-09-21T06:13:24.611058+0000","last_unstale":"2024-09-21T06:13:24.621488+0000","last_undegraded":"2024-09-21T06:13:24.621488+0000","last_fullsized":"2024-09-21T06:13:24.621488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:28.759029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.040559+0000","last_change":"2024-09-21T06:13:10.516339+0000","last_active":"2024-09-21T06:14:54.040559+0000","last_peered":"2024-09-21T06:14:54.040559+0000","last_clean":"2024-09-21T06:14:54.040559+0000","last_became_active":"2024-09-21T06:13:10.515397+0000","last_became_peered":"2024-09-21T06:13:10.515397+0000","last_unstale":"2024-09-21T06:14:54.040559+0000","last_undegraded":"2024-09-21T06:14:54.040559+0000","last_fullsized":"2024-09-21T06:14:54.040559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:56.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610445+0000","last_change":"2024-09-21T06:13:08.560494+0000","last_active":"2024-09-21T06:13:24.610445+0000","last_peered":"2024-09-21T06:13:24.610445+0000","last_clean":"2024-09-21T06:13:24.610445+0000","last_became_active":"2024-09-21T06:13:08.560118+0000","last_became_peered":"2024-09-21T06:13:08.560118+0000","last_unstale":"2024-09-21T06:13:24.610445+0000","last_undegraded":"2024-09-21T06:13:24.610445+0000","last_fullsized":"2024-09-21T06:13:24.610445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:05.597233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035172+0000","last_change":"2024-09-21T06:13:10.515780+0000","last_active":"2024-09-21T06:14:54.035172+0000","last_peered":"2024-09-21T06:14:54.035172+0000","last_clean":"2024-09-21T06:14:54.035172+0000","last_became_active":"2024-09-21T06:13:10.515437+0000","last_became_peered":"2024-09-21T06:13:10.515437+0000","last_unstale":"2024-09-21T06:14:54.035172+0000","last_undegraded":"2024-09-21T06:14:54.035172+0000","last_fullsized":"2024-09-21T06:14:54.035172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:04.848003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'14","reported_seq":146,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:09.770212+0000","last_change":"2024-09-21T06:13:08.483300+0000","last_active":"2024-09-21T06:15:09.770212+0000","last_peered":"2024-09-21T06:15:09.770212+0000","last_clean":"2024-09-21T06:15:09.770212+0000","last_became_active":"2024-09-21T06:13:08.482908+0000","last_became_peered":"2024-09-21T06:13:08.482908+0000","last_unstale":"2024-09-21T06:15:09.770212+0000","last_undegraded":"2024-09-21T06:15:09.770212+0000","last_fullsized":"2024-09-21T06:15:09.770212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:09.202075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82109,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1109,"num_read_kb":646,"num_write":272,"num_write_kb":169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1724,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":231890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282438,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678053,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":427664,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874283,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1092867,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037809,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":427179,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234038,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":427440,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":1020939,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593791,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":426779,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1020085,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":493387,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":426487,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378709,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":429172,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542234,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":494327,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428650,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":76442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:15:13.509 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-21T06:15:13.509 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:15:13.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:13 smithi137 ceph-mon[34464]: pgmap v331: 97 pgs: 97 active+clean; 657 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-21T06:15:13.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:13 smithi137 ceph-mon[34464]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T06:15:13.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:13 smithi204 ceph-mon[34659]: pgmap v331: 97 pgs: 97 active+clean; 657 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-21T06:15:13.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:13 smithi204 ceph-mon[34659]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T06:15:13.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.657+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/2131771059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a41082f0 msgr2=0x7ff4a4108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:13.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.657+0000 7ff4ab8dd640 1 --2- 172.21.15.3:0/2131771059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a41082f0 0x7ff4a4108700 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009900 tx=0x7ff4a002a410 comp rx=0 tx=0).stop 2024-09-21T06:15:13.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.658+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/2131771059 shutdown_connections 2024-09-21T06:15:13.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.658+0000 7ff4ab8dd640 1 --2- 172.21.15.3:0/2131771059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a4108c40 0x7ff4a410f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.658+0000 7ff4ab8dd640 1 --2- 172.21.15.3:0/2131771059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a41082f0 0x7ff4a4108700 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.658+0000 7ff4ab8dd640 1 --2- 172.21.15.3:0/2131771059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a41029c0 0x7ff4a4102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.658+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/2131771059 >> 172.21.15.3:0/2131771059 conn(0x7ff4a40fe3c0 msgr2=0x7ff4a4100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:13.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.658+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/2131771059 shutdown_connections 2024-09-21T06:15:13.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.658+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/2131771059 wait complete. 2024-09-21T06:15:13.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.658+0000 7ff4ab8dd640 1 Processor -- start 2024-09-21T06:15:13.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4ab8dd640 1 -- start start 2024-09-21T06:15:13.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4ab8dd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a41029c0 0x7ff4a419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:13.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4ab8dd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a41082f0 0x7ff4a419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:13.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4ab8dd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a4108c40 0x7ff4a419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:13.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4ab8dd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a4112900 con 0x7ff4a41082f0 2024-09-21T06:15:13.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4ab8dd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a4112780 con 0x7ff4a4108c40 2024-09-21T06:15:13.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4ab8dd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a4112a80 con 0x7ff4a41029c0 2024-09-21T06:15:13.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4a8e78640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a41082f0 0x7ff4a419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:13.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4a8e78640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a41082f0 0x7ff4a419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48374/0 (socket says 172.21.15.3:48374) 2024-09-21T06:15:13.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4a8e78640 1 -- 172.21.15.3:0/3419878309 learned_addr learned my addr 172.21.15.3:0/3419878309 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4a9679640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a41029c0 0x7ff4a419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4a9e7a640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a4108c40 0x7ff4a419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4a8e78640 1 -- 172.21.15.3:0/3419878309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a41029c0 msgr2=0x7ff4a419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4a8e78640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a41029c0 0x7ff4a419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4a8e78640 1 -- 172.21.15.3:0/3419878309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a4108c40 msgr2=0x7ff4a419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4a8e78640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a4108c40 0x7ff4a419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.659+0000 7ff4a8e78640 1 -- 172.21.15.3:0/3419878309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a41a03a0 con 0x7ff4a41082f0 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.660+0000 7ff4a8e78640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a41082f0 0x7ff4a419aa10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0006010 tx=0x7ff4a0004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.660+0000 7ff4967fc640 1 -- 172.21.15.3:0/3419878309 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a003f030 con 0x7ff4a41082f0 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.660+0000 7ff4967fc640 1 -- 172.21.15.3:0/3419878309 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a0003aa0 con 0x7ff4a41082f0 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.660+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/3419878309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a41a0630 con 0x7ff4a41082f0 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.660+0000 7ff4967fc640 1 -- 172.21.15.3:0/3419878309 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0035070 con 0x7ff4a41082f0 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.660+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/3419878309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a41a6e20 con 0x7ff4a41082f0 2024-09-21T06:15:13.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.661+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/3419878309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a4103c30 con 0x7ff4a41082f0 2024-09-21T06:15:13.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.663+0000 7ff4967fc640 1 -- 172.21.15.3:0/3419878309 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 154579+0+0 (secure 0 0 0) 0x7ff4a0042030 con 0x7ff4a41082f0 2024-09-21T06:15:13.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.664+0000 7ff4967fc640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff4900b65f0 0x7ff4900b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:13.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.664+0000 7ff4a9679640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff4900b65f0 0x7ff4900b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:13.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.664+0000 7ff4967fc640 1 -- 172.21.15.3:0/3419878309 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff4a0107ef0 con 0x7ff4a41082f0 2024-09-21T06:15:13.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.665+0000 7ff4a9679640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff4900b65f0 0x7ff4900b8b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff498009bd0 tx=0x7ff498005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:13.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.667+0000 7ff4967fc640 1 -- 172.21.15.3:0/3419878309 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff4a0139710 con 0x7ff4a41082f0 2024-09-21T06:15:13.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.855+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/3419878309 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4a4063800 con 0x7ff4900b65f0 2024-09-21T06:15:13.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.859+0000 7ff4967fc640 1 -- 172.21.15.3:0/3419878309 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259149 (secure 0 0 0) 0x7ff4a4063800 con 0x7ff4900b65f0 2024-09-21T06:15:13.863 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:15:13.864 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-21T06:15:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.862+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/3419878309 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff4900b65f0 msgr2=0x7ff4900b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.862+0000 7ff4ab8dd640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff4900b65f0 0x7ff4900b8b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff498009bd0 tx=0x7ff498005f90 comp rx=0 tx=0).stop 2024-09-21T06:15:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.862+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/3419878309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a41082f0 msgr2=0x7ff4a419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.862+0000 7ff4ab8dd640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a41082f0 0x7ff4a419aa10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0006010 tx=0x7ff4a0004060 comp rx=0 tx=0).stop 2024-09-21T06:15:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.862+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/3419878309 shutdown_connections 2024-09-21T06:15:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.862+0000 7ff4ab8dd640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff4900b65f0 0x7ff4900b8b00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.862+0000 7ff4ab8dd640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a4108c40 0x7ff4a419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.862+0000 7ff4ab8dd640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a41082f0 0x7ff4a419aa10 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.862+0000 7ff4ab8dd640 1 --2- 172.21.15.3:0/3419878309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a41029c0 0x7ff4a419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.862+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/3419878309 >> 172.21.15.3:0/3419878309 conn(0x7ff4a40fe3c0 msgr2=0x7ff4a4100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.862+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/3419878309 shutdown_connections 2024-09-21T06:15:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:13.862+0000 7ff4ab8dd640 1 -- 172.21.15.3:0/3419878309 wait complete. 2024-09-21T06:15:13.876 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-21T06:15:13.659497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":673493,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1202,"num_read_kb":717,"num_write":399,"num_write_kb":2323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2953564,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":829,"ondisk_log_size":829,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368304,"kb_used_data":10736,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573520,"statfs":{"total":1151940427776,"available":1151563284480,"internally_reserved":0,"allocated":10993664,"data_stored":7123326,"data_compressed":177687,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19139,"internal_metadata":365933885},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8658,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":59,"num_write":16,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011915"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.040716+0000","last_change":"2024-09-21T06:13:10.515115+0000","last_active":"2024-09-21T06:14:54.040716+0000","last_peered":"2024-09-21T06:14:54.040716+0000","last_clean":"2024-09-21T06:14:54.040716+0000","last_became_active":"2024-09-21T06:13:10.514553+0000","last_became_peered":"2024-09-21T06:13:10.514553+0000","last_unstale":"2024-09-21T06:14:54.040716+0000","last_undegraded":"2024-09-21T06:14:54.040716+0000","last_fullsized":"2024-09-21T06:14:54.040716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:33.268926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035807+0000","last_change":"2024-09-21T06:13:10.511469+0000","last_active":"2024-09-21T06:14:54.035807+0000","last_peered":"2024-09-21T06:14:54.035807+0000","last_clean":"2024-09-21T06:14:54.035807+0000","last_became_active":"2024-09-21T06:13:10.511102+0000","last_became_peered":"2024-09-21T06:13:10.511102+0000","last_unstale":"2024-09-21T06:14:54.035807+0000","last_undegraded":"2024-09-21T06:14:54.035807+0000","last_fullsized":"2024-09-21T06:14:54.035807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:12.483821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.040174+0000","last_change":"2024-09-21T06:13:10.520090+0000","last_active":"2024-09-21T06:14:54.040174+0000","last_peered":"2024-09-21T06:14:54.040174+0000","last_clean":"2024-09-21T06:14:54.040174+0000","last_became_active":"2024-09-21T06:13:10.519758+0000","last_became_peered":"2024-09-21T06:13:10.519758+0000","last_unstale":"2024-09-21T06:14:54.040174+0000","last_undegraded":"2024-09-21T06:14:54.040174+0000","last_fullsized":"2024-09-21T06:14:54.040174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:20.836847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.025267+0000","last_change":"2024-09-21T06:13:10.511732+0000","last_active":"2024-09-21T06:14:54.025267+0000","last_peered":"2024-09-21T06:14:54.025267+0000","last_clean":"2024-09-21T06:14:54.025267+0000","last_became_active":"2024-09-21T06:13:10.511377+0000","last_became_peered":"2024-09-21T06:13:10.511377+0000","last_unstale":"2024-09-21T06:14:54.025267+0000","last_undegraded":"2024-09-21T06:14:54.025267+0000","last_fullsized":"2024-09-21T06:14:54.025267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:18.500739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.023035+0000","last_change":"2024-09-21T06:13:10.510464+0000","last_active":"2024-09-21T06:14:54.023035+0000","last_peered":"2024-09-21T06:14:54.023035+0000","last_clean":"2024-09-21T06:14:54.023035+0000","last_became_active":"2024-09-21T06:13:10.509979+0000","last_became_peered":"2024-09-21T06:13:10.509979+0000","last_unstale":"2024-09-21T06:14:54.023035+0000","last_undegraded":"2024-09-21T06:14:54.023035+0000","last_fullsized":"2024-09-21T06:14:54.023035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:56.871043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.032702+0000","last_change":"2024-09-21T06:13:24.608832+0000","last_active":"2024-09-21T06:14:54.032702+0000","last_peered":"2024-09-21T06:14:54.032702+0000","last_clean":"2024-09-21T06:14:54.032702+0000","last_became_active":"2024-09-21T06:13:24.608439+0000","last_became_peered":"2024-09-21T06:13:24.608439+0000","last_unstale":"2024-09-21T06:14:54.032702+0000","last_undegraded":"2024-09-21T06:14:54.032702+0000","last_fullsized":"2024-09-21T06:14:54.032702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:41.515894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.041535+0000","last_change":"2024-09-21T06:13:10.511391+0000","last_active":"2024-09-21T06:14:54.041535+0000","last_peered":"2024-09-21T06:14:54.041535+0000","last_clean":"2024-09-21T06:14:54.041535+0000","last_became_active":"2024-09-21T06:13:10.510733+0000","last_became_peered":"2024-09-21T06:13:10.510733+0000","last_unstale":"2024-09-21T06:14:54.041535+0000","last_undegraded":"2024-09-21T06:14:54.041535+0000","last_fullsized":"2024-09-21T06:14:54.041535+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:24.071860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035636+0000","last_change":"2024-09-21T06:13:10.510886+0000","last_active":"2024-09-21T06:14:54.035636+0000","last_peered":"2024-09-21T06:14:54.035636+0000","last_clean":"2024-09-21T06:14:54.035636+0000","last_became_active":"2024-09-21T06:13:10.510159+0000","last_became_peered":"2024-09-21T06:13:10.510159+0000","last_unstale":"2024-09-21T06:14:54.035636+0000","last_undegraded":"2024-09-21T06:14:54.035636+0000","last_fullsized":"2024-09-21T06:14:54.035636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:37.124848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.045101+0000","last_change":"2024-09-21T06:13:10.510466+0000","last_active":"2024-09-21T06:14:54.045101+0000","last_peered":"2024-09-21T06:14:54.045101+0000","last_clean":"2024-09-21T06:14:54.045101+0000","last_became_active":"2024-09-21T06:13:10.509928+0000","last_became_peered":"2024-09-21T06:13:10.509928+0000","last_unstale":"2024-09-21T06:14:54.045101+0000","last_undegraded":"2024-09-21T06:14:54.045101+0000","last_fullsized":"2024-09-21T06:14:54.045101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:06.156505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.029437+0000","last_change":"2024-09-21T06:13:10.510622+0000","last_active":"2024-09-21T06:14:54.029437+0000","last_peered":"2024-09-21T06:14:54.029437+0000","last_clean":"2024-09-21T06:14:54.029437+0000","last_became_active":"2024-09-21T06:13:10.510206+0000","last_became_peered":"2024-09-21T06:13:10.510206+0000","last_unstale":"2024-09-21T06:14:54.029437+0000","last_undegraded":"2024-09-21T06:14:54.029437+0000","last_fullsized":"2024-09-21T06:14:54.029437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:03.223451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.042280+0000","last_change":"2024-09-21T06:13:10.517525+0000","last_active":"2024-09-21T06:14:54.042280+0000","last_peered":"2024-09-21T06:14:54.042280+0000","last_clean":"2024-09-21T06:14:54.042280+0000","last_became_active":"2024-09-21T06:13:10.517144+0000","last_became_peered":"2024-09-21T06:13:10.517144+0000","last_unstale":"2024-09-21T06:14:54.042280+0000","last_undegraded":"2024-09-21T06:14:54.042280+0000","last_fullsized":"2024-09-21T06:14:54.042280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:46.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037178+0000","last_change":"2024-09-21T06:13:10.515537+0000","last_active":"2024-09-21T06:14:54.037178+0000","last_peered":"2024-09-21T06:14:54.037178+0000","last_clean":"2024-09-21T06:14:54.037178+0000","last_became_active":"2024-09-21T06:13:10.515063+0000","last_became_peered":"2024-09-21T06:13:10.515063+0000","last_unstale":"2024-09-21T06:14:54.037178+0000","last_undegraded":"2024-09-21T06:14:54.037178+0000","last_fullsized":"2024-09-21T06:14:54.037178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:33.718471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.031442+0000","last_change":"2024-09-21T06:13:10.509368+0000","last_active":"2024-09-21T06:14:54.031442+0000","last_peered":"2024-09-21T06:14:54.031442+0000","last_clean":"2024-09-21T06:14:54.031442+0000","last_became_active":"2024-09-21T06:13:10.508982+0000","last_became_peered":"2024-09-21T06:13:10.508982+0000","last_unstale":"2024-09-21T06:14:54.031442+0000","last_undegraded":"2024-09-21T06:14:54.031442+0000","last_fullsized":"2024-09-21T06:14:54.031442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:54.430889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.036256+0000","last_change":"2024-09-21T06:13:10.506559+0000","last_active":"2024-09-21T06:14:54.036256+0000","last_peered":"2024-09-21T06:14:54.036256+0000","last_clean":"2024-09-21T06:14:54.036256+0000","last_became_active":"2024-09-21T06:13:10.506161+0000","last_became_peered":"2024-09-21T06:13:10.506161+0000","last_unstale":"2024-09-21T06:14:54.036256+0000","last_undegraded":"2024-09-21T06:14:54.036256+0000","last_fullsized":"2024-09-21T06:14:54.036256+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:19.870292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037955+0000","last_change":"2024-09-21T06:13:10.516000+0000","last_active":"2024-09-21T06:14:54.037955+0000","last_peered":"2024-09-21T06:14:54.037955+0000","last_clean":"2024-09-21T06:14:54.037955+0000","last_became_active":"2024-09-21T06:13:10.515543+0000","last_became_peered":"2024-09-21T06:13:10.515543+0000","last_unstale":"2024-09-21T06:14:54.037955+0000","last_undegraded":"2024-09-21T06:14:54.037955+0000","last_fullsized":"2024-09-21T06:14:54.037955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:48.968489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.045711+0000","last_change":"2024-09-21T06:13:10.513753+0000","last_active":"2024-09-21T06:14:54.045711+0000","last_peered":"2024-09-21T06:14:54.045711+0000","last_clean":"2024-09-21T06:14:54.045711+0000","last_became_active":"2024-09-21T06:13:10.511079+0000","last_became_peered":"2024-09-21T06:13:10.511079+0000","last_unstale":"2024-09-21T06:14:54.045711+0000","last_undegraded":"2024-09-21T06:14:54.045711+0000","last_fullsized":"2024-09-21T06:14:54.045711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:25.320041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.040850+0000","last_change":"2024-09-21T06:13:10.515222+0000","last_active":"2024-09-21T06:14:54.040850+0000","last_peered":"2024-09-21T06:14:54.040850+0000","last_clean":"2024-09-21T06:14:54.040850+0000","last_became_active":"2024-09-21T06:13:10.514604+0000","last_became_peered":"2024-09-21T06:13:10.514604+0000","last_unstale":"2024-09-21T06:14:54.040850+0000","last_undegraded":"2024-09-21T06:14:54.040850+0000","last_fullsized":"2024-09-21T06:14:54.040850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:03.165695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.038346+0000","last_change":"2024-09-21T06:13:10.518334+0000","last_active":"2024-09-21T06:14:54.038346+0000","last_peered":"2024-09-21T06:14:54.038346+0000","last_clean":"2024-09-21T06:14:54.038346+0000","last_became_active":"2024-09-21T06:13:10.517956+0000","last_became_peered":"2024-09-21T06:13:10.517956+0000","last_unstale":"2024-09-21T06:14:54.038346+0000","last_undegraded":"2024-09-21T06:14:54.038346+0000","last_fullsized":"2024-09-21T06:14:54.038346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:05.094454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.029105+0000","last_change":"2024-09-21T06:13:10.508763+0000","last_active":"2024-09-21T06:14:54.029105+0000","last_peered":"2024-09-21T06:14:54.029105+0000","last_clean":"2024-09-21T06:14:54.029105+0000","last_became_active":"2024-09-21T06:13:10.508371+0000","last_became_peered":"2024-09-21T06:13:10.508371+0000","last_unstale":"2024-09-21T06:14:54.029105+0000","last_undegraded":"2024-09-21T06:14:54.029105+0000","last_fullsized":"2024-09-21T06:14:54.029105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:50.602692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.038111+0000","last_change":"2024-09-21T06:13:10.503465+0000","last_active":"2024-09-21T06:14:54.038111+0000","last_peered":"2024-09-21T06:14:54.038111+0000","last_clean":"2024-09-21T06:14:54.038111+0000","last_became_active":"2024-09-21T06:13:10.503072+0000","last_became_peered":"2024-09-21T06:13:10.503072+0000","last_unstale":"2024-09-21T06:14:54.038111+0000","last_undegraded":"2024-09-21T06:14:54.038111+0000","last_fullsized":"2024-09-21T06:14:54.038111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:36.368987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.032321+0000","last_change":"2024-09-21T06:13:10.512841+0000","last_active":"2024-09-21T06:14:54.032321+0000","last_peered":"2024-09-21T06:14:54.032321+0000","last_clean":"2024-09-21T06:14:54.032321+0000","last_became_active":"2024-09-21T06:13:10.512407+0000","last_became_peered":"2024-09-21T06:13:10.512407+0000","last_unstale":"2024-09-21T06:14:54.032321+0000","last_undegraded":"2024-09-21T06:14:54.032321+0000","last_fullsized":"2024-09-21T06:14:54.032321+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:09.855608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.045911+0000","last_change":"2024-09-21T06:13:10.510088+0000","last_active":"2024-09-21T06:14:54.045911+0000","last_peered":"2024-09-21T06:14:54.045911+0000","last_clean":"2024-09-21T06:14:54.045911+0000","last_became_active":"2024-09-21T06:13:10.507904+0000","last_became_peered":"2024-09-21T06:13:10.507904+0000","last_unstale":"2024-09-21T06:14:54.045911+0000","last_undegraded":"2024-09-21T06:14:54.045911+0000","last_fullsized":"2024-09-21T06:14:54.045911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:49.404402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.033659+0000","last_change":"2024-09-21T06:13:10.517720+0000","last_active":"2024-09-21T06:14:54.033659+0000","last_peered":"2024-09-21T06:14:54.033659+0000","last_clean":"2024-09-21T06:14:54.033659+0000","last_became_active":"2024-09-21T06:13:10.514641+0000","last_became_peered":"2024-09-21T06:13:10.514641+0000","last_unstale":"2024-09-21T06:14:54.033659+0000","last_undegraded":"2024-09-21T06:14:54.033659+0000","last_fullsized":"2024-09-21T06:14:54.033659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:40.639665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.034092+0000","last_change":"2024-09-21T06:13:10.512234+0000","last_active":"2024-09-21T06:14:54.034092+0000","last_peered":"2024-09-21T06:14:54.034092+0000","last_clean":"2024-09-21T06:14:54.034092+0000","last_became_active":"2024-09-21T06:13:10.511594+0000","last_became_peered":"2024-09-21T06:13:10.511594+0000","last_unstale":"2024-09-21T06:14:54.034092+0000","last_undegraded":"2024-09-21T06:14:54.034092+0000","last_fullsized":"2024-09-21T06:14:54.034092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:04.410342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.030782+0000","last_change":"2024-09-21T06:13:10.513216+0000","last_active":"2024-09-21T06:14:54.030782+0000","last_peered":"2024-09-21T06:14:54.030782+0000","last_clean":"2024-09-21T06:14:54.030782+0000","last_became_active":"2024-09-21T06:13:10.512830+0000","last_became_peered":"2024-09-21T06:13:10.512830+0000","last_unstale":"2024-09-21T06:14:54.030782+0000","last_undegraded":"2024-09-21T06:14:54.030782+0000","last_fullsized":"2024-09-21T06:14:54.030782+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:46.418161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.030499+0000","last_change":"2024-09-21T06:13:10.518863+0000","last_active":"2024-09-21T06:14:54.030499+0000","last_peered":"2024-09-21T06:14:54.030499+0000","last_clean":"2024-09-21T06:14:54.030499+0000","last_became_active":"2024-09-21T06:13:10.518492+0000","last_became_peered":"2024-09-21T06:13:10.518492+0000","last_unstale":"2024-09-21T06:14:54.030499+0000","last_undegraded":"2024-09-21T06:14:54.030499+0000","last_fullsized":"2024-09-21T06:14:54.030499+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:23.770763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.031560+0000","last_change":"2024-09-21T06:13:10.508538+0000","last_active":"2024-09-21T06:14:54.031560+0000","last_peered":"2024-09-21T06:14:54.031560+0000","last_clean":"2024-09-21T06:14:54.031560+0000","last_became_active":"2024-09-21T06:13:10.508017+0000","last_became_peered":"2024-09-21T06:13:10.508017+0000","last_unstale":"2024-09-21T06:14:54.031560+0000","last_undegraded":"2024-09-21T06:14:54.031560+0000","last_fullsized":"2024-09-21T06:14:54.031560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:00.168741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.026465+0000","last_change":"2024-09-21T06:13:10.516734+0000","last_active":"2024-09-21T06:14:54.026465+0000","last_peered":"2024-09-21T06:14:54.026465+0000","last_clean":"2024-09-21T06:14:54.026465+0000","last_became_active":"2024-09-21T06:13:10.516362+0000","last_became_peered":"2024-09-21T06:13:10.516362+0000","last_unstale":"2024-09-21T06:14:54.026465+0000","last_undegraded":"2024-09-21T06:14:54.026465+0000","last_fullsized":"2024-09-21T06:14:54.026465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:49.792011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.036688+0000","last_change":"2024-09-21T06:13:10.513143+0000","last_active":"2024-09-21T06:14:54.036688+0000","last_peered":"2024-09-21T06:14:54.036688+0000","last_clean":"2024-09-21T06:14:54.036688+0000","last_became_active":"2024-09-21T06:13:10.512338+0000","last_became_peered":"2024-09-21T06:13:10.512338+0000","last_unstale":"2024-09-21T06:14:54.036688+0000","last_undegraded":"2024-09-21T06:14:54.036688+0000","last_fullsized":"2024-09-21T06:14:54.036688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:02.963860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.022101+0000","last_change":"2024-09-21T06:13:10.513429+0000","last_active":"2024-09-21T06:14:54.022101+0000","last_peered":"2024-09-21T06:14:54.022101+0000","last_clean":"2024-09-21T06:14:54.022101+0000","last_became_active":"2024-09-21T06:13:10.512995+0000","last_became_peered":"2024-09-21T06:13:10.512995+0000","last_unstale":"2024-09-21T06:14:54.022101+0000","last_undegraded":"2024-09-21T06:14:54.022101+0000","last_fullsized":"2024-09-21T06:14:54.022101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:39.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.032522+0000","last_change":"2024-09-21T06:13:10.509580+0000","last_active":"2024-09-21T06:14:54.032522+0000","last_peered":"2024-09-21T06:14:54.032522+0000","last_clean":"2024-09-21T06:14:54.032522+0000","last_became_active":"2024-09-21T06:13:10.509179+0000","last_became_peered":"2024-09-21T06:13:10.509179+0000","last_unstale":"2024-09-21T06:14:54.032522+0000","last_undegraded":"2024-09-21T06:14:54.032522+0000","last_fullsized":"2024-09-21T06:14:54.032522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:39.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.036310+0000","last_change":"2024-09-21T06:13:10.508358+0000","last_active":"2024-09-21T06:14:54.036310+0000","last_peered":"2024-09-21T06:14:54.036310+0000","last_clean":"2024-09-21T06:14:54.036310+0000","last_became_active":"2024-09-21T06:13:10.507952+0000","last_became_peered":"2024-09-21T06:13:10.507952+0000","last_unstale":"2024-09-21T06:14:54.036310+0000","last_undegraded":"2024-09-21T06:14:54.036310+0000","last_fullsized":"2024-09-21T06:14:54.036310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:55.565203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.024319+0000","last_change":"2024-09-21T06:13:10.510580+0000","last_active":"2024-09-21T06:14:54.024319+0000","last_peered":"2024-09-21T06:14:54.024319+0000","last_clean":"2024-09-21T06:14:54.024319+0000","last_became_active":"2024-09-21T06:13:10.510175+0000","last_became_peered":"2024-09-21T06:13:10.510175+0000","last_unstale":"2024-09-21T06:14:54.024319+0000","last_undegraded":"2024-09-21T06:14:54.024319+0000","last_fullsized":"2024-09-21T06:14:54.024319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:49.467057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'9","reported_seq":148,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:11.773087+0000","last_change":"2024-09-21T06:13:08.485527+0000","last_active":"2024-09-21T06:15:11.773087+0000","last_peered":"2024-09-21T06:15:11.773087+0000","last_clean":"2024-09-21T06:15:11.773087+0000","last_became_active":"2024-09-21T06:13:08.485094+0000","last_became_peered":"2024-09-21T06:13:08.485094+0000","last_unstale":"2024-09-21T06:15:11.773087+0000","last_undegraded":"2024-09-21T06:15:11.773087+0000","last_fullsized":"2024-09-21T06:15:11.773087+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:48.285638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.034456+0000","last_change":"2024-09-21T06:13:10.507977+0000","last_active":"2024-09-21T06:14:54.034456+0000","last_peered":"2024-09-21T06:14:54.034456+0000","last_clean":"2024-09-21T06:14:54.034456+0000","last_became_active":"2024-09-21T06:13:10.507594+0000","last_became_peered":"2024-09-21T06:13:10.507594+0000","last_unstale":"2024-09-21T06:14:54.034456+0000","last_undegraded":"2024-09-21T06:14:54.034456+0000","last_fullsized":"2024-09-21T06:14:54.034456+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:57.318240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.599690+0000","last_change":"2024-09-21T06:13:08.491237+0000","last_active":"2024-09-21T06:13:24.599690+0000","last_peered":"2024-09-21T06:13:24.599690+0000","last_clean":"2024-09-21T06:13:24.599690+0000","last_became_active":"2024-09-21T06:13:08.490908+0000","last_became_peered":"2024-09-21T06:13:08.490908+0000","last_unstale":"2024-09-21T06:13:24.599690+0000","last_undegraded":"2024-09-21T06:13:24.599690+0000","last_fullsized":"2024-09-21T06:13:24.599690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:53.185029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597228+0000","last_change":"2024-09-21T06:13:08.488721+0000","last_active":"2024-09-21T06:13:24.597228+0000","last_peered":"2024-09-21T06:13:24.597228+0000","last_clean":"2024-09-21T06:13:24.597228+0000","last_became_active":"2024-09-21T06:13:08.488347+0000","last_became_peered":"2024-09-21T06:13:08.488347+0000","last_unstale":"2024-09-21T06:13:24.597228+0000","last_undegraded":"2024-09-21T06:13:24.597228+0000","last_fullsized":"2024-09-21T06:13:24.597228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:27.008479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.034347+0000","last_change":"2024-09-21T06:13:10.506527+0000","last_active":"2024-09-21T06:14:54.034347+0000","last_peered":"2024-09-21T06:14:54.034347+0000","last_clean":"2024-09-21T06:14:54.034347+0000","last_became_active":"2024-09-21T06:13:10.506163+0000","last_became_peered":"2024-09-21T06:13:10.506163+0000","last_unstale":"2024-09-21T06:14:54.034347+0000","last_undegraded":"2024-09-21T06:14:54.034347+0000","last_fullsized":"2024-09-21T06:14:54.034347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.244097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604150+0000","last_change":"2024-09-21T06:13:08.489292+0000","last_active":"2024-09-21T06:13:24.604150+0000","last_peered":"2024-09-21T06:13:24.604150+0000","last_clean":"2024-09-21T06:13:24.604150+0000","last_became_active":"2024-09-21T06:13:08.488918+0000","last_became_peered":"2024-09-21T06:13:08.488918+0000","last_unstale":"2024-09-21T06:13:24.604150+0000","last_undegraded":"2024-09-21T06:13:24.604150+0000","last_fullsized":"2024-09-21T06:13:24.604150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:37.992260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.036458+0000","last_change":"2024-09-21T06:13:10.505027+0000","last_active":"2024-09-21T06:14:54.036458+0000","last_peered":"2024-09-21T06:14:54.036458+0000","last_clean":"2024-09-21T06:14:54.036458+0000","last_became_active":"2024-09-21T06:13:10.504669+0000","last_became_peered":"2024-09-21T06:13:10.504669+0000","last_unstale":"2024-09-21T06:14:54.036458+0000","last_undegraded":"2024-09-21T06:14:54.036458+0000","last_fullsized":"2024-09-21T06:14:54.036458+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:43.927229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'8","reported_seq":246,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:10.770506+0000","last_change":"2024-09-21T06:13:08.491101+0000","last_active":"2024-09-21T06:15:10.770506+0000","last_peered":"2024-09-21T06:15:10.770506+0000","last_clean":"2024-09-21T06:15:10.770506+0000","last_became_active":"2024-09-21T06:13:08.490607+0000","last_became_peered":"2024-09-21T06:13:08.490607+0000","last_unstale":"2024-09-21T06:15:10.770506+0000","last_undegraded":"2024-09-21T06:15:10.770506+0000","last_fullsized":"2024-09-21T06:15:10.770506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:16.758463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":215,"num_read_kb":215,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.043733+0000","last_change":"2024-09-21T06:13:10.510453+0000","last_active":"2024-09-21T06:14:54.043733+0000","last_peered":"2024-09-21T06:14:54.043733+0000","last_clean":"2024-09-21T06:14:54.043733+0000","last_became_active":"2024-09-21T06:13:10.508513+0000","last_became_peered":"2024-09-21T06:13:10.508513+0000","last_unstale":"2024-09-21T06:14:54.043733+0000","last_undegraded":"2024-09-21T06:14:54.043733+0000","last_fullsized":"2024-09-21T06:14:54.043733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:48.388140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.609878+0000","last_change":"2024-09-21T06:13:08.488714+0000","last_active":"2024-09-21T06:13:24.609878+0000","last_peered":"2024-09-21T06:13:24.609878+0000","last_clean":"2024-09-21T06:13:24.609878+0000","last_became_active":"2024-09-21T06:13:08.488363+0000","last_became_peered":"2024-09-21T06:13:08.488363+0000","last_unstale":"2024-09-21T06:13:24.609878+0000","last_undegraded":"2024-09-21T06:13:24.609878+0000","last_fullsized":"2024-09-21T06:13:24.609878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:33.626165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037460+0000","last_change":"2024-09-21T06:13:10.508046+0000","last_active":"2024-09-21T06:14:54.037460+0000","last_peered":"2024-09-21T06:14:54.037460+0000","last_clean":"2024-09-21T06:14:54.037460+0000","last_became_active":"2024-09-21T06:13:10.507660+0000","last_became_peered":"2024-09-21T06:13:10.507660+0000","last_unstale":"2024-09-21T06:14:54.037460+0000","last_undegraded":"2024-09-21T06:14:54.037460+0000","last_fullsized":"2024-09-21T06:14:54.037460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:35.466965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603272+0000","last_change":"2024-09-21T06:13:08.562863+0000","last_active":"2024-09-21T06:13:24.603272+0000","last_peered":"2024-09-21T06:13:24.603272+0000","last_clean":"2024-09-21T06:13:24.603272+0000","last_became_active":"2024-09-21T06:13:08.562424+0000","last_became_peered":"2024-09-21T06:13:08.562424+0000","last_unstale":"2024-09-21T06:13:24.603272+0000","last_undegraded":"2024-09-21T06:13:24.603272+0000","last_fullsized":"2024-09-21T06:13:24.603272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:43.251932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.044625+0000","last_change":"2024-09-21T06:13:10.509818+0000","last_active":"2024-09-21T06:14:54.044625+0000","last_peered":"2024-09-21T06:14:54.044625+0000","last_clean":"2024-09-21T06:14:54.044625+0000","last_became_active":"2024-09-21T06:13:10.509468+0000","last_became_peered":"2024-09-21T06:13:10.509468+0000","last_unstale":"2024-09-21T06:14:54.044625+0000","last_undegraded":"2024-09-21T06:14:54.044625+0000","last_fullsized":"2024-09-21T06:14:54.044625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:25.130868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'38","reported_seq":307,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:10.770919+0000","last_change":"2024-09-21T06:13:24.633807+0000","last_active":"2024-09-21T06:15:10.770919+0000","last_peered":"2024-09-21T06:15:10.770919+0000","last_clean":"2024-09-21T06:15:10.770919+0000","last_became_active":"2024-09-21T06:13:24.618599+0000","last_became_peered":"2024-09-21T06:13:24.618599+0000","last_unstale":"2024-09-21T06:15:10.770919+0000","last_undegraded":"2024-09-21T06:15:10.770919+0000","last_fullsized":"2024-09-21T06:15:10.770919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:07.449253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73783,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":225,"num_read_kb":181,"num_write":40,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.033514+0000","last_change":"2024-09-21T06:13:10.511360+0000","last_active":"2024-09-21T06:14:54.033514+0000","last_peered":"2024-09-21T06:14:54.033514+0000","last_clean":"2024-09-21T06:14:54.033514+0000","last_became_active":"2024-09-21T06:13:10.510980+0000","last_became_peered":"2024-09-21T06:13:10.510980+0000","last_unstale":"2024-09-21T06:14:54.033514+0000","last_undegraded":"2024-09-21T06:14:54.033514+0000","last_fullsized":"2024-09-21T06:14:54.033514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:43.859758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.604427+0000","last_change":"2024-09-21T06:13:08.486262+0000","last_active":"2024-09-21T06:13:24.604427+0000","last_peered":"2024-09-21T06:13:24.604427+0000","last_clean":"2024-09-21T06:13:24.604427+0000","last_became_active":"2024-09-21T06:13:08.485509+0000","last_became_peered":"2024-09-21T06:13:08.485509+0000","last_unstale":"2024-09-21T06:13:24.604427+0000","last_undegraded":"2024-09-21T06:13:24.604427+0000","last_fullsized":"2024-09-21T06:13:24.604427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:31.726746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.033404+0000","last_change":"2024-09-21T06:13:10.506894+0000","last_active":"2024-09-21T06:14:54.033404+0000","last_peered":"2024-09-21T06:14:54.033404+0000","last_clean":"2024-09-21T06:14:54.033404+0000","last_became_active":"2024-09-21T06:13:10.506489+0000","last_became_peered":"2024-09-21T06:13:10.506489+0000","last_unstale":"2024-09-21T06:14:54.033404+0000","last_undegraded":"2024-09-21T06:14:54.033404+0000","last_fullsized":"2024-09-21T06:14:54.033404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:58.755601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603141+0000","last_change":"2024-09-21T06:13:08.483429+0000","last_active":"2024-09-21T06:13:24.603141+0000","last_peered":"2024-09-21T06:13:24.603141+0000","last_clean":"2024-09-21T06:13:24.603141+0000","last_became_active":"2024-09-21T06:13:08.483028+0000","last_became_peered":"2024-09-21T06:13:08.483028+0000","last_unstale":"2024-09-21T06:13:24.603141+0000","last_undegraded":"2024-09-21T06:13:24.603141+0000","last_fullsized":"2024-09-21T06:13:24.603141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:33.258154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.042922+0000","last_change":"2024-09-21T06:13:10.513398+0000","last_active":"2024-09-21T06:14:54.042922+0000","last_peered":"2024-09-21T06:14:54.042922+0000","last_clean":"2024-09-21T06:14:54.042922+0000","last_became_active":"2024-09-21T06:13:10.512713+0000","last_became_peered":"2024-09-21T06:13:10.512713+0000","last_unstale":"2024-09-21T06:14:54.042922+0000","last_undegraded":"2024-09-21T06:14:54.042922+0000","last_fullsized":"2024-09-21T06:14:54.042922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:21.026984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'8","reported_seq":144,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:10.770724+0000","last_change":"2024-09-21T06:13:08.493771+0000","last_active":"2024-09-21T06:15:10.770724+0000","last_peered":"2024-09-21T06:15:10.770724+0000","last_clean":"2024-09-21T06:15:10.770724+0000","last_became_active":"2024-09-21T06:13:08.493380+0000","last_became_peered":"2024-09-21T06:13:08.493380+0000","last_unstale":"2024-09-21T06:15:10.770724+0000","last_undegraded":"2024-09-21T06:15:10.770724+0000","last_fullsized":"2024-09-21T06:15:10.770724+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:12.552137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.041365+0000","last_change":"2024-09-21T06:13:10.511679+0000","last_active":"2024-09-21T06:14:54.041365+0000","last_peered":"2024-09-21T06:14:54.041365+0000","last_clean":"2024-09-21T06:14:54.041365+0000","last_became_active":"2024-09-21T06:13:10.511270+0000","last_became_peered":"2024-09-21T06:13:10.511270+0000","last_unstale":"2024-09-21T06:14:54.041365+0000","last_undegraded":"2024-09-21T06:14:54.041365+0000","last_fullsized":"2024-09-21T06:14:54.041365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:10.521853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.627560+0000","last_change":"2024-09-21T06:13:24.627467+0000","last_active":"2024-09-21T06:13:24.627560+0000","last_peered":"2024-09-21T06:13:24.627560+0000","last_clean":"2024-09-21T06:13:24.627560+0000","last_became_active":"2024-09-21T06:13:24.610263+0000","last_became_peered":"2024-09-21T06:13:24.610263+0000","last_unstale":"2024-09-21T06:13:24.627560+0000","last_undegraded":"2024-09-21T06:13:24.627560+0000","last_fullsized":"2024-09-21T06:13:24.627560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:26.472147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.032186+0000","last_change":"2024-09-21T06:13:10.517214+0000","last_active":"2024-09-21T06:14:54.032186+0000","last_peered":"2024-09-21T06:14:54.032186+0000","last_clean":"2024-09-21T06:14:54.032186+0000","last_became_active":"2024-09-21T06:13:10.516828+0000","last_became_peered":"2024-09-21T06:13:10.516828+0000","last_unstale":"2024-09-21T06:14:54.032186+0000","last_undegraded":"2024-09-21T06:14:54.032186+0000","last_fullsized":"2024-09-21T06:14:54.032186+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:58.077670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608703+0000","last_change":"2024-09-21T06:13:08.489153+0000","last_active":"2024-09-21T06:13:24.608703+0000","last_peered":"2024-09-21T06:13:24.608703+0000","last_clean":"2024-09-21T06:13:24.608703+0000","last_became_active":"2024-09-21T06:13:08.488732+0000","last_became_peered":"2024-09-21T06:13:08.488732+0000","last_unstale":"2024-09-21T06:13:24.608703+0000","last_undegraded":"2024-09-21T06:13:24.608703+0000","last_fullsized":"2024-09-21T06:13:24.608703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:22.848136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037298+0000","last_change":"2024-09-21T06:13:24.610441+0000","last_active":"2024-09-21T06:14:54.037298+0000","last_peered":"2024-09-21T06:14:54.037298+0000","last_clean":"2024-09-21T06:14:54.037298+0000","last_became_active":"2024-09-21T06:13:24.609992+0000","last_became_peered":"2024-09-21T06:13:24.609992+0000","last_unstale":"2024-09-21T06:14:54.037298+0000","last_undegraded":"2024-09-21T06:14:54.037298+0000","last_fullsized":"2024-09-21T06:14:54.037298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:59.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.597725+0000","last_change":"2024-09-21T06:13:08.484514+0000","last_active":"2024-09-21T06:13:24.597725+0000","last_peered":"2024-09-21T06:13:24.597725+0000","last_clean":"2024-09-21T06:13:24.597725+0000","last_became_active":"2024-09-21T06:13:08.484134+0000","last_became_peered":"2024-09-21T06:13:08.484134+0000","last_unstale":"2024-09-21T06:13:24.597725+0000","last_undegraded":"2024-09-21T06:13:24.597725+0000","last_fullsized":"2024-09-21T06:13:24.597725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:45.124734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.039200+0000","last_change":"2024-09-21T06:13:10.503798+0000","last_active":"2024-09-21T06:14:54.039200+0000","last_peered":"2024-09-21T06:14:54.039200+0000","last_clean":"2024-09-21T06:14:54.039200+0000","last_became_active":"2024-09-21T06:13:10.502418+0000","last_became_peered":"2024-09-21T06:13:10.502418+0000","last_unstale":"2024-09-21T06:14:54.039200+0000","last_undegraded":"2024-09-21T06:14:54.039200+0000","last_fullsized":"2024-09-21T06:14:54.039200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:57.575569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.603439+0000","last_change":"2024-09-21T06:13:08.487191+0000","last_active":"2024-09-21T06:13:24.603439+0000","last_peered":"2024-09-21T06:13:24.603439+0000","last_clean":"2024-09-21T06:13:24.603439+0000","last_became_active":"2024-09-21T06:13:08.486812+0000","last_became_peered":"2024-09-21T06:13:08.486812+0000","last_unstale":"2024-09-21T06:13:24.603439+0000","last_undegraded":"2024-09-21T06:13:24.603439+0000","last_fullsized":"2024-09-21T06:13:24.603439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:51.989708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.041431+0000","last_change":"2024-09-21T06:13:10.511678+0000","last_active":"2024-09-21T06:14:54.041431+0000","last_peered":"2024-09-21T06:14:54.041431+0000","last_clean":"2024-09-21T06:14:54.041431+0000","last_became_active":"2024-09-21T06:13:10.510408+0000","last_became_peered":"2024-09-21T06:13:10.510408+0000","last_unstale":"2024-09-21T06:14:54.041431+0000","last_undegraded":"2024-09-21T06:14:54.041431+0000","last_fullsized":"2024-09-21T06:14:54.041431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:58.711190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'7","reported_seq":139,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:07.766522+0000","last_change":"2024-09-21T06:13:08.560894+0000","last_active":"2024-09-21T06:15:07.766522+0000","last_peered":"2024-09-21T06:15:07.766522+0000","last_clean":"2024-09-21T06:15:07.766522+0000","last_became_active":"2024-09-21T06:13:08.560423+0000","last_became_peered":"2024-09-21T06:13:08.560423+0000","last_unstale":"2024-09-21T06:15:07.766522+0000","last_undegraded":"2024-09-21T06:15:07.766522+0000","last_fullsized":"2024-09-21T06:15:07.766522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:53.015711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.044084+0000","last_change":"2024-09-21T06:13:10.516009+0000","last_active":"2024-09-21T06:14:54.044084+0000","last_peered":"2024-09-21T06:14:54.044084+0000","last_clean":"2024-09-21T06:14:54.044084+0000","last_became_active":"2024-09-21T06:13:10.515669+0000","last_became_peered":"2024-09-21T06:13:10.515669+0000","last_unstale":"2024-09-21T06:14:54.044084+0000","last_undegraded":"2024-09-21T06:14:54.044084+0000","last_fullsized":"2024-09-21T06:14:54.044084+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:54.134564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":100,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610370+0000","last_change":"2024-09-21T06:11:53.408030+0000","last_active":"2024-09-21T06:13:24.610370+0000","last_peered":"2024-09-21T06:13:24.610370+0000","last_clean":"2024-09-21T06:13:24.610370+0000","last_became_active":"2024-09-21T06:11:53.402296+0000","last_became_peered":"2024-09-21T06:11:53.402296+0000","last_unstale":"2024-09-21T06:13:24.610370+0000","last_undegraded":"2024-09-21T06:13:24.610370+0000","last_fullsized":"2024-09-21T06:13:24.610370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:08:26.098183+0000","last_clean_scrub_stamp":"2024-09-21T06:08:26.098183+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:08.239872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616959+0000","last_change":"2024-09-21T06:13:08.489209+0000","last_active":"2024-09-21T06:13:24.616959+0000","last_peered":"2024-09-21T06:13:24.616959+0000","last_clean":"2024-09-21T06:13:24.616959+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.616959+0000","last_undegraded":"2024-09-21T06:13:24.616959+0000","last_fullsized":"2024-09-21T06:13:24.616959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:38.798499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.038080+0000","last_change":"2024-09-21T06:13:24.606487+0000","last_active":"2024-09-21T06:14:54.038080+0000","last_peered":"2024-09-21T06:14:54.038080+0000","last_clean":"2024-09-21T06:14:54.038080+0000","last_became_active":"2024-09-21T06:13:24.606141+0000","last_became_peered":"2024-09-21T06:13:24.606141+0000","last_unstale":"2024-09-21T06:14:54.038080+0000","last_undegraded":"2024-09-21T06:14:54.038080+0000","last_fullsized":"2024-09-21T06:14:54.038080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:24.040770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610270+0000","last_change":"2024-09-21T06:13:08.489206+0000","last_active":"2024-09-21T06:13:24.610270+0000","last_peered":"2024-09-21T06:13:24.610270+0000","last_clean":"2024-09-21T06:13:24.610270+0000","last_became_active":"2024-09-21T06:13:08.488850+0000","last_became_peered":"2024-09-21T06:13:08.488850+0000","last_unstale":"2024-09-21T06:13:24.610270+0000","last_undegraded":"2024-09-21T06:13:24.610270+0000","last_fullsized":"2024-09-21T06:13:24.610270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:41.627305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.027371+0000","last_change":"2024-09-21T06:13:10.513999+0000","last_active":"2024-09-21T06:14:54.027371+0000","last_peered":"2024-09-21T06:14:54.027371+0000","last_clean":"2024-09-21T06:14:54.027371+0000","last_became_active":"2024-09-21T06:13:10.513657+0000","last_became_peered":"2024-09-21T06:13:10.513657+0000","last_unstale":"2024-09-21T06:14:54.027371+0000","last_undegraded":"2024-09-21T06:14:54.027371+0000","last_fullsized":"2024-09-21T06:14:54.027371+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:05.536921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'6","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:07.765185+0000","last_change":"2024-09-21T06:13:08.560811+0000","last_active":"2024-09-21T06:15:07.765185+0000","last_peered":"2024-09-21T06:15:07.765185+0000","last_clean":"2024-09-21T06:15:07.765185+0000","last_became_active":"2024-09-21T06:13:08.560145+0000","last_became_peered":"2024-09-21T06:13:08.560145+0000","last_unstale":"2024-09-21T06:15:07.765185+0000","last_undegraded":"2024-09-21T06:15:07.765185+0000","last_fullsized":"2024-09-21T06:15:07.765185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:24.550008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.036397+0000","last_change":"2024-09-21T06:13:24.613495+0000","last_active":"2024-09-21T06:14:54.036397+0000","last_peered":"2024-09-21T06:14:54.036397+0000","last_clean":"2024-09-21T06:14:54.036397+0000","last_became_active":"2024-09-21T06:13:24.612634+0000","last_became_peered":"2024-09-21T06:13:24.612634+0000","last_unstale":"2024-09-21T06:14:54.036397+0000","last_undegraded":"2024-09-21T06:14:54.036397+0000","last_fullsized":"2024-09-21T06:14:54.036397+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:22.863141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'5","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:07.768565+0000","last_change":"2024-09-21T06:13:08.486973+0000","last_active":"2024-09-21T06:15:07.768565+0000","last_peered":"2024-09-21T06:15:07.768565+0000","last_clean":"2024-09-21T06:15:07.768565+0000","last_became_active":"2024-09-21T06:13:08.486602+0000","last_became_peered":"2024-09-21T06:13:08.486602+0000","last_unstale":"2024-09-21T06:15:07.768565+0000","last_undegraded":"2024-09-21T06:15:07.768565+0000","last_fullsized":"2024-09-21T06:15:07.768565+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:41.804955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.029271+0000","last_change":"2024-09-21T06:13:10.509499+0000","last_active":"2024-09-21T06:14:54.029271+0000","last_peered":"2024-09-21T06:14:54.029271+0000","last_clean":"2024-09-21T06:14:54.029271+0000","last_became_active":"2024-09-21T06:13:10.508566+0000","last_became_peered":"2024-09-21T06:13:10.508566+0000","last_unstale":"2024-09-21T06:14:54.029271+0000","last_undegraded":"2024-09-21T06:14:54.029271+0000","last_fullsized":"2024-09-21T06:14:54.029271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:12.701631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.600737+0000","last_change":"2024-09-21T06:13:08.561378+0000","last_active":"2024-09-21T06:13:24.600737+0000","last_peered":"2024-09-21T06:13:24.600737+0000","last_clean":"2024-09-21T06:13:24.600737+0000","last_became_active":"2024-09-21T06:13:08.561002+0000","last_became_peered":"2024-09-21T06:13:08.561002+0000","last_unstale":"2024-09-21T06:13:24.600737+0000","last_undegraded":"2024-09-21T06:13:24.600737+0000","last_fullsized":"2024-09-21T06:13:24.600737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:11.325377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.030695+0000","last_change":"2024-09-21T06:13:24.602347+0000","last_active":"2024-09-21T06:14:54.030695+0000","last_peered":"2024-09-21T06:14:54.030695+0000","last_clean":"2024-09-21T06:14:54.030695+0000","last_became_active":"2024-09-21T06:13:24.601978+0000","last_became_peered":"2024-09-21T06:13:24.601978+0000","last_unstale":"2024-09-21T06:14:54.030695+0000","last_undegraded":"2024-09-21T06:14:54.030695+0000","last_fullsized":"2024-09-21T06:14:54.030695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:34.860096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608569+0000","last_change":"2024-09-21T06:13:08.561937+0000","last_active":"2024-09-21T06:13:24.608569+0000","last_peered":"2024-09-21T06:13:24.608569+0000","last_clean":"2024-09-21T06:13:24.608569+0000","last_became_active":"2024-09-21T06:13:08.560547+0000","last_became_peered":"2024-09-21T06:13:08.560547+0000","last_unstale":"2024-09-21T06:13:24.608569+0000","last_undegraded":"2024-09-21T06:13:24.608569+0000","last_fullsized":"2024-09-21T06:13:24.608569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:29.702291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.027723+0000","last_change":"2024-09-21T06:13:10.516833+0000","last_active":"2024-09-21T06:14:54.027723+0000","last_peered":"2024-09-21T06:14:54.027723+0000","last_clean":"2024-09-21T06:14:54.027723+0000","last_became_active":"2024-09-21T06:13:10.516486+0000","last_became_peered":"2024-09-21T06:13:10.516486+0000","last_unstale":"2024-09-21T06:14:54.027723+0000","last_undegraded":"2024-09-21T06:14:54.027723+0000","last_fullsized":"2024-09-21T06:14:54.027723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:05.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'19","reported_seq":42,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:08.513647+0000","last_change":"2024-09-21T06:13:08.483271+0000","last_active":"2024-09-21T06:15:08.513647+0000","last_peered":"2024-09-21T06:15:08.513647+0000","last_clean":"2024-09-21T06:15:08.513647+0000","last_became_active":"2024-09-21T06:13:08.482951+0000","last_became_peered":"2024-09-21T06:13:08.482951+0000","last_unstale":"2024-09-21T06:15:08.513647+0000","last_undegraded":"2024-09-21T06:15:08.513647+0000","last_fullsized":"2024-09-21T06:15:08.513647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:17.537988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.043217+0000","last_change":"2024-09-21T06:13:10.509125+0000","last_active":"2024-09-21T06:14:54.043217+0000","last_peered":"2024-09-21T06:14:54.043217+0000","last_clean":"2024-09-21T06:14:54.043217+0000","last_became_active":"2024-09-21T06:13:10.508773+0000","last_became_peered":"2024-09-21T06:13:10.508773+0000","last_unstale":"2024-09-21T06:14:54.043217+0000","last_undegraded":"2024-09-21T06:14:54.043217+0000","last_fullsized":"2024-09-21T06:14:54.043217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:44.570477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.616932+0000","last_change":"2024-09-21T06:13:08.497885+0000","last_active":"2024-09-21T06:13:24.616932+0000","last_peered":"2024-09-21T06:13:24.616932+0000","last_clean":"2024-09-21T06:13:24.616932+0000","last_became_active":"2024-09-21T06:13:08.497593+0000","last_became_peered":"2024-09-21T06:13:08.497593+0000","last_unstale":"2024-09-21T06:13:24.616932+0000","last_undegraded":"2024-09-21T06:13:24.616932+0000","last_fullsized":"2024-09-21T06:13:24.616932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:48.361175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.037327+0000","last_change":"2024-09-21T06:13:10.514430+0000","last_active":"2024-09-21T06:14:54.037327+0000","last_peered":"2024-09-21T06:14:54.037327+0000","last_clean":"2024-09-21T06:14:54.037327+0000","last_became_active":"2024-09-21T06:13:10.514063+0000","last_became_peered":"2024-09-21T06:13:10.514063+0000","last_unstale":"2024-09-21T06:14:54.037327+0000","last_undegraded":"2024-09-21T06:14:54.037327+0000","last_fullsized":"2024-09-21T06:14:54.037327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:37.036497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.598415+0000","last_change":"2024-09-21T06:13:08.560564+0000","last_active":"2024-09-21T06:13:24.598415+0000","last_peered":"2024-09-21T06:13:24.598415+0000","last_clean":"2024-09-21T06:13:24.598415+0000","last_became_active":"2024-09-21T06:13:08.560339+0000","last_became_peered":"2024-09-21T06:13:08.560339+0000","last_unstale":"2024-09-21T06:13:24.598415+0000","last_undegraded":"2024-09-21T06:13:24.598415+0000","last_fullsized":"2024-09-21T06:13:24.598415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:55.378911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.022629+0000","last_change":"2024-09-21T06:13:10.511354+0000","last_active":"2024-09-21T06:14:54.022629+0000","last_peered":"2024-09-21T06:14:54.022629+0000","last_clean":"2024-09-21T06:14:54.022629+0000","last_became_active":"2024-09-21T06:13:10.510946+0000","last_became_peered":"2024-09-21T06:13:10.510946+0000","last_unstale":"2024-09-21T06:14:54.022629+0000","last_undegraded":"2024-09-21T06:14:54.022629+0000","last_fullsized":"2024-09-21T06:14:54.022629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:11.535461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610883+0000","last_change":"2024-09-21T06:13:08.485578+0000","last_active":"2024-09-21T06:13:24.610883+0000","last_peered":"2024-09-21T06:13:24.610883+0000","last_clean":"2024-09-21T06:13:24.610883+0000","last_became_active":"2024-09-21T06:13:08.485170+0000","last_became_peered":"2024-09-21T06:13:08.485170+0000","last_unstale":"2024-09-21T06:13:24.610883+0000","last_undegraded":"2024-09-21T06:13:24.610883+0000","last_fullsized":"2024-09-21T06:13:24.610883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:39.000114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035886+0000","last_change":"2024-09-21T06:13:10.510360+0000","last_active":"2024-09-21T06:14:54.035886+0000","last_peered":"2024-09-21T06:14:54.035886+0000","last_clean":"2024-09-21T06:14:54.035886+0000","last_became_active":"2024-09-21T06:13:10.509887+0000","last_became_peered":"2024-09-21T06:13:10.509887+0000","last_unstale":"2024-09-21T06:14:54.035886+0000","last_undegraded":"2024-09-21T06:14:54.035886+0000","last_fullsized":"2024-09-21T06:14:54.035886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:58.135706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.625797+0000","last_change":"2024-09-21T06:13:24.625284+0000","last_active":"2024-09-21T06:13:24.625797+0000","last_peered":"2024-09-21T06:13:24.625797+0000","last_clean":"2024-09-21T06:13:24.625797+0000","last_became_active":"2024-09-21T06:13:24.612704+0000","last_became_peered":"2024-09-21T06:13:24.612704+0000","last_unstale":"2024-09-21T06:13:24.625797+0000","last_undegraded":"2024-09-21T06:13:24.625797+0000","last_fullsized":"2024-09-21T06:13:24.625797+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:29.385258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.029990+0000","last_change":"2024-09-21T06:13:10.514128+0000","last_active":"2024-09-21T06:14:54.029990+0000","last_peered":"2024-09-21T06:14:54.029990+0000","last_clean":"2024-09-21T06:14:54.029990+0000","last_became_active":"2024-09-21T06:13:10.513790+0000","last_became_peered":"2024-09-21T06:13:10.513790+0000","last_unstale":"2024-09-21T06:14:54.029990+0000","last_undegraded":"2024-09-21T06:14:54.029990+0000","last_fullsized":"2024-09-21T06:14:54.029990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:33.407189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.607762+0000","last_change":"2024-09-21T06:13:08.489105+0000","last_active":"2024-09-21T06:13:24.607762+0000","last_peered":"2024-09-21T06:13:24.607762+0000","last_clean":"2024-09-21T06:13:24.607762+0000","last_became_active":"2024-09-21T06:13:08.487110+0000","last_became_peered":"2024-09-21T06:13:08.487110+0000","last_unstale":"2024-09-21T06:13:24.607762+0000","last_undegraded":"2024-09-21T06:13:24.607762+0000","last_fullsized":"2024-09-21T06:13:24.607762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:24.849365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.044351+0000","last_change":"2024-09-21T06:13:10.513751+0000","last_active":"2024-09-21T06:14:54.044351+0000","last_peered":"2024-09-21T06:14:54.044351+0000","last_clean":"2024-09-21T06:14:54.044351+0000","last_became_active":"2024-09-21T06:13:10.513376+0000","last_became_peered":"2024-09-21T06:13:10.513376+0000","last_unstale":"2024-09-21T06:14:54.044351+0000","last_undegraded":"2024-09-21T06:14:54.044351+0000","last_fullsized":"2024-09-21T06:14:54.044351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.364959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.039103+0000","last_change":"2024-09-21T06:13:10.519359+0000","last_active":"2024-09-21T06:14:54.039103+0000","last_peered":"2024-09-21T06:14:54.039103+0000","last_clean":"2024-09-21T06:14:54.039103+0000","last_became_active":"2024-09-21T06:13:10.518518+0000","last_became_peered":"2024-09-21T06:13:10.518518+0000","last_unstale":"2024-09-21T06:14:54.039103+0000","last_undegraded":"2024-09-21T06:14:54.039103+0000","last_fullsized":"2024-09-21T06:14:54.039103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:55.554710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.608166+0000","last_change":"2024-09-21T06:13:08.488122+0000","last_active":"2024-09-21T06:13:24.608166+0000","last_peered":"2024-09-21T06:13:24.608166+0000","last_clean":"2024-09-21T06:13:24.608166+0000","last_became_active":"2024-09-21T06:13:08.487679+0000","last_became_peered":"2024-09-21T06:13:08.487679+0000","last_unstale":"2024-09-21T06:13:24.608166+0000","last_undegraded":"2024-09-21T06:13:24.608166+0000","last_fullsized":"2024-09-21T06:13:24.608166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:12.266985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.043317+0000","last_change":"2024-09-21T06:13:10.516794+0000","last_active":"2024-09-21T06:14:54.043317+0000","last_peered":"2024-09-21T06:14:54.043317+0000","last_clean":"2024-09-21T06:14:54.043317+0000","last_became_active":"2024-09-21T06:13:10.516297+0000","last_became_peered":"2024-09-21T06:13:10.516297+0000","last_unstale":"2024-09-21T06:14:54.043317+0000","last_undegraded":"2024-09-21T06:14:54.043317+0000","last_fullsized":"2024-09-21T06:14:54.043317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:19.873742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.621488+0000","last_change":"2024-09-21T06:13:24.621358+0000","last_active":"2024-09-21T06:13:24.621488+0000","last_peered":"2024-09-21T06:13:24.621488+0000","last_clean":"2024-09-21T06:13:24.621488+0000","last_became_active":"2024-09-21T06:13:24.611058+0000","last_became_peered":"2024-09-21T06:13:24.611058+0000","last_unstale":"2024-09-21T06:13:24.621488+0000","last_undegraded":"2024-09-21T06:13:24.621488+0000","last_fullsized":"2024-09-21T06:13:24.621488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:28.759029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.040559+0000","last_change":"2024-09-21T06:13:10.516339+0000","last_active":"2024-09-21T06:14:54.040559+0000","last_peered":"2024-09-21T06:14:54.040559+0000","last_clean":"2024-09-21T06:14:54.040559+0000","last_became_active":"2024-09-21T06:13:10.515397+0000","last_became_peered":"2024-09-21T06:13:10.515397+0000","last_unstale":"2024-09-21T06:14:54.040559+0000","last_undegraded":"2024-09-21T06:14:54.040559+0000","last_fullsized":"2024-09-21T06:14:54.040559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:56.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:13:24.610445+0000","last_change":"2024-09-21T06:13:08.560494+0000","last_active":"2024-09-21T06:13:24.610445+0000","last_peered":"2024-09-21T06:13:24.610445+0000","last_clean":"2024-09-21T06:13:24.610445+0000","last_became_active":"2024-09-21T06:13:08.560118+0000","last_became_peered":"2024-09-21T06:13:08.560118+0000","last_unstale":"2024-09-21T06:13:24.610445+0000","last_undegraded":"2024-09-21T06:13:24.610445+0000","last_fullsized":"2024-09-21T06:13:24.610445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:05.597233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:14:54.035172+0000","last_change":"2024-09-21T06:13:10.515780+0000","last_active":"2024-09-21T06:14:54.035172+0000","last_peered":"2024-09-21T06:14:54.035172+0000","last_clean":"2024-09-21T06:14:54.035172+0000","last_became_active":"2024-09-21T06:13:10.515437+0000","last_became_peered":"2024-09-21T06:13:10.515437+0000","last_unstale":"2024-09-21T06:14:54.035172+0000","last_undegraded":"2024-09-21T06:14:54.035172+0000","last_fullsized":"2024-09-21T06:14:54.035172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:09.472713+0000","last_clean_scrub_stamp":"2024-09-21T06:13:09.472713+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:04.848003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'14","reported_seq":146,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T06:15:09.770212+0000","last_change":"2024-09-21T06:13:08.483300+0000","last_active":"2024-09-21T06:15:09.770212+0000","last_peered":"2024-09-21T06:15:09.770212+0000","last_clean":"2024-09-21T06:15:09.770212+0000","last_became_active":"2024-09-21T06:13:08.482908+0000","last_became_peered":"2024-09-21T06:13:08.482908+0000","last_unstale":"2024-09-21T06:15:09.770212+0000","last_undegraded":"2024-09-21T06:15:09.770212+0000","last_fullsized":"2024-09-21T06:15:09.770212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:13:07.464372+0000","last_clean_scrub_stamp":"2024-09-21T06:13:07.464372+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:09.202075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82891,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1121,"num_read_kb":648,"num_write":274,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1724,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":598016,"data_stored":240240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282438,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678053,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":427664,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874283,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1092867,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037809,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":427179,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234038,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":427440,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":1020939,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593791,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":426779,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1020476,"data_compressed":41742,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":501346,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":426487,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378709,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":429172,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542234,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":494327,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428650,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34291,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35913,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":75406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":76442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:15:13.880 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:15:14.172 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:14.563 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:14 smithi003 ceph-mon[31227]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:15:14.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:14 smithi137 ceph-mon[34464]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:15:14.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:14 smithi204 ceph-mon[34659]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:15:14.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.655+0000 7f65c0f57640 1 -- 172.21.15.3:0/387550121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc102a60 msgr2=0x7f65bc102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:14.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.655+0000 7f65c0f57640 1 --2- 172.21.15.3:0/387550121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc102a60 0x7f65bc102e70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009ca0 tx=0x7f65b402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:15:14.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.655+0000 7f65c0f57640 1 -- 172.21.15.3:0/387550121 shutdown_connections 2024-09-21T06:15:14.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.655+0000 7f65c0f57640 1 --2- 172.21.15.3:0/387550121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65bc108cf0 0x7f65bc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:14.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.655+0000 7f65c0f57640 1 --2- 172.21.15.3:0/387550121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65bc1083a0 0x7f65bc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:14.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.655+0000 7f65c0f57640 1 --2- 172.21.15.3:0/387550121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc102a60 0x7f65bc102e70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:14.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.655+0000 7f65c0f57640 1 -- 172.21.15.3:0/387550121 >> 172.21.15.3:0/387550121 conn(0x7f65bc079730 msgr2=0x7f65bc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:14.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.655+0000 7f65c0f57640 1 -- 172.21.15.3:0/387550121 shutdown_connections 2024-09-21T06:15:14.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.655+0000 7f65c0f57640 1 -- 172.21.15.3:0/387550121 wait complete. 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.656+0000 7f65c0f57640 1 Processor -- start 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.656+0000 7f65c0f57640 1 -- start start 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.656+0000 7f65c0f57640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65bc102a60 0x7f65bc19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.656+0000 7f65c0f57640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65bc1083a0 0x7f65bc19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.656+0000 7f65c0f57640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc108cf0 0x7f65bc19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.656+0000 7f65c0f57640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc112960 con 0x7f65bc108cf0 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.656+0000 7f65c0f57640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1127e0 con 0x7f65bc102a60 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.656+0000 7f65c0f57640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc112ae0 con 0x7f65bc1083a0 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.656+0000 7f65bad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc108cf0 0x7f65bc19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.656+0000 7f65bad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc108cf0 0x7f65bc19bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48380/0 (socket says 172.21.15.3:48380) 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.656+0000 7f65bad9d640 1 -- 172.21.15.3:0/3293635341 learned_addr learned my addr 172.21.15.3:0/3293635341 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.656+0000 7f65ba59c640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65bc102a60 0x7f65bc19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.657+0000 7f65b9d9b640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65bc1083a0 0x7f65bc19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:14.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.657+0000 7f65bad9d640 1 -- 172.21.15.3:0/3293635341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65bc1083a0 msgr2=0x7f65bc19b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.657+0000 7f65bad9d640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65bc1083a0 0x7f65bc19b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.657+0000 7f65bad9d640 1 -- 172.21.15.3:0/3293635341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65bc102a60 msgr2=0x7f65bc19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.657+0000 7f65bad9d640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65bc102a60 0x7f65bc19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.657+0000 7f65bad9d640 1 -- 172.21.15.3:0/3293635341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc1a6540 con 0x7f65bc108cf0 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.657+0000 7f65bad9d640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc108cf0 0x7f65bc19bd30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00d750 tx=0x7f65ac00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.657+0000 7f65ab7fe640 1 -- 172.21.15.3:0/3293635341 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac017070 con 0x7f65bc108cf0 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.657+0000 7f65ab7fe640 1 -- 172.21.15.3:0/3293635341 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65ac0054d0 con 0x7f65bc108cf0 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.657+0000 7f65c0f57640 1 -- 172.21.15.3:0/3293635341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65bc1a6830 con 0x7f65bc108cf0 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.657+0000 7f65ab7fe640 1 -- 172.21.15.3:0/3293635341 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac004c60 con 0x7f65bc108cf0 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.657+0000 7f65c0f57640 1 -- 172.21.15.3:0/3293635341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65bc1a6d20 con 0x7f65bc108cf0 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.658+0000 7f65c0f57640 1 -- 172.21.15.3:0/3293635341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65bc102e70 con 0x7f65bc108cf0 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.660+0000 7f65ab7fe640 1 -- 172.21.15.3:0/3293635341 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 154579+0+0 (secure 0 0 0) 0x7f65ac009bd0 con 0x7f65bc108cf0 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.661+0000 7f65ab7fe640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f65a40b6620 0x7f65a40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:14.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.661+0000 7f65ba59c640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f65a40b6620 0x7f65a40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:14.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.661+0000 7f65ab7fe640 1 -- 172.21.15.3:0/3293635341 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f65ac0e73b0 con 0x7f65bc108cf0 2024-09-21T06:15:14.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.662+0000 7f65ba59c640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f65a40b6620 0x7f65a40b8b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f65b4005e30 tx=0x7f65b4009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:14.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.664+0000 7f65ab7fe640 1 -- 172.21.15.3:0/3293635341 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f65ac0b1d80 con 0x7f65bc108cf0 2024-09-21T06:15:14.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.891+0000 7f65c0f57640 1 -- 172.21.15.3:0/3293635341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65bc02e640 con 0x7f65bc108cf0 2024-09-21T06:15:14.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.892+0000 7f65ab7fe640 1 -- 172.21.15.3:0/3293635341 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11962 (secure 0 0 0) 0x7f65ac0b6cc0 con 0x7f65bc108cf0 2024-09-21T06:15:14.894 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:15:14.894 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"btime":"2024-09-21T06:15:09:322787+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:15:08.461776+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24670,24674,24679,24691,24704]},"id":1}]} 2024-09-21T06:15:14.895 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 28 2024-09-21T06:15:14.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.894+0000 7f65c0f57640 1 -- 172.21.15.3:0/3293635341 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f65a40b6620 msgr2=0x7f65a40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:14.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.894+0000 7f65c0f57640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f65a40b6620 0x7f65a40b8b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f65b4005e30 tx=0x7f65b4009360 comp rx=0 tx=0).stop 2024-09-21T06:15:14.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.894+0000 7f65c0f57640 1 -- 172.21.15.3:0/3293635341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc108cf0 msgr2=0x7f65bc19bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.894+0000 7f65c0f57640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc108cf0 0x7f65bc19bd30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00d750 tx=0x7f65ac00dc10 comp rx=0 tx=0).stop 2024-09-21T06:15:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.894+0000 7f65c0f57640 1 -- 172.21.15.3:0/3293635341 shutdown_connections 2024-09-21T06:15:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.894+0000 7f65c0f57640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f65a40b6620 0x7f65a40b8b30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.894+0000 7f65c0f57640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65bc1083a0 0x7f65bc19b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.894+0000 7f65c0f57640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc108cf0 0x7f65bc19bd30 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.894+0000 7f65c0f57640 1 --2- 172.21.15.3:0/3293635341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65bc102a60 0x7f65bc19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.894+0000 7f65c0f57640 1 -- 172.21.15.3:0/3293635341 >> 172.21.15.3:0/3293635341 conn(0x7f65bc079730 msgr2=0x7f65bc101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.895+0000 7f65c0f57640 1 -- 172.21.15.3:0/3293635341 shutdown_connections 2024-09-21T06:15:14.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:14.895+0000 7f65c0f57640 1 -- 172.21.15.3:0/3293635341 wait complete. 2024-09-21T06:15:15.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:15:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:15 smithi003 ceph-mon[31227]: from='client.15519 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:15:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:15 smithi003 ceph-mon[31227]: pgmap v332: 97 pgs: 97 active+clean; 658 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-21T06:15:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:15 smithi003 ceph-mon[31227]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:15:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:15 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3293635341' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:15.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:15 smithi137 ceph-mon[34464]: from='client.15519 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:15:15.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:15 smithi137 ceph-mon[34464]: pgmap v332: 97 pgs: 97 active+clean; 658 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-21T06:15:15.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:15 smithi137 ceph-mon[34464]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:15:15.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:15 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3293635341' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:15.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:15 smithi204 ceph-mon[34659]: from='client.15519 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:15:15.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:15 smithi204 ceph-mon[34659]: pgmap v332: 97 pgs: 97 active+clean; 658 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-21T06:15:15.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:15 smithi204 ceph-mon[34659]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:15:15.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:15 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3293635341' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:15.801 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:16.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.263+0000 7ff749e2c640 1 -- 172.21.15.3:0/222207682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff744102f20 msgr2=0x7ff74410fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:16.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.263+0000 7ff749e2c640 1 --2- 172.21.15.3:0/222207682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff744102f20 0x7ff74410fc40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff738009900 tx=0x7ff73802a480 comp rx=0 tx=0).stop 2024-09-21T06:15:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.263+0000 7ff749e2c640 1 -- 172.21.15.3:0/222207682 shutdown_connections 2024-09-21T06:15:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.263+0000 7ff749e2c640 1 --2- 172.21.15.3:0/222207682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff744110180 0x7ff7441125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.263+0000 7ff749e2c640 1 --2- 172.21.15.3:0/222207682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff744102f20 0x7ff74410fc40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.263+0000 7ff749e2c640 1 --2- 172.21.15.3:0/222207682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7441025d0 0x7ff7441029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.263+0000 7ff749e2c640 1 -- 172.21.15.3:0/222207682 >> 172.21.15.3:0/222207682 conn(0x7ff744079730 msgr2=0x7ff744079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.263+0000 7ff749e2c640 1 -- 172.21.15.3:0/222207682 shutdown_connections 2024-09-21T06:15:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.263+0000 7ff749e2c640 1 -- 172.21.15.3:0/222207682 wait complete. 2024-09-21T06:15:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.264+0000 7ff749e2c640 1 Processor -- start 2024-09-21T06:15:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.264+0000 7ff749e2c640 1 -- start start 2024-09-21T06:15:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.264+0000 7ff749e2c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7441025d0 0x7ff74419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.264+0000 7ff749e2c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff744102f20 0x7ff74419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.264+0000 7ff749e2c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff744110180 0x7ff7441a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:16.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.264+0000 7ff749e2c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff744115c70 con 0x7ff7441025d0 2024-09-21T06:15:16.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.264+0000 7ff749e2c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff744115af0 con 0x7ff744110180 2024-09-21T06:15:16.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.264+0000 7ff749e2c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff744115df0 con 0x7ff744102f20 2024-09-21T06:15:16.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.264+0000 7ff7437fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7441025d0 0x7ff74419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:16.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.264+0000 7ff7437fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7441025d0 0x7ff74419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48136/0 (socket says 172.21.15.3:48136) 2024-09-21T06:15:16.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.264+0000 7ff7437fe640 1 -- 172.21.15.3:0/2998351390 learned_addr learned my addr 172.21.15.3:0/2998351390 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:16.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff742ffd640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff744102f20 0x7ff74419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:16.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff743fff640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff744110180 0x7ff7441a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:16.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff7437fe640 1 -- 172.21.15.3:0/2998351390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff744102f20 msgr2=0x7ff74419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:16.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff7437fe640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff744102f20 0x7ff74419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:16.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff7437fe640 1 -- 172.21.15.3:0/2998351390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff744110180 msgr2=0x7ff7441a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:16.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff7437fe640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff744110180 0x7ff7441a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:16.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff7437fe640 1 -- 172.21.15.3:0/2998351390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7441a25b0 con 0x7ff7441025d0 2024-09-21T06:15:16.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff743fff640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff744110180 0x7ff7441a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:16.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff7437fe640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7441025d0 0x7ff74419c640 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff73400d950 tx=0x7ff73400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:16.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff740ff9640 1 -- 172.21.15.3:0/2998351390 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734017030 con 0x7ff7441025d0 2024-09-21T06:15:16.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff749e2c640 1 -- 172.21.15.3:0/2998351390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7441a28a0 con 0x7ff7441025d0 2024-09-21T06:15:16.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff740ff9640 1 -- 172.21.15.3:0/2998351390 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff734004020 con 0x7ff7441025d0 2024-09-21T06:15:16.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff740ff9640 1 -- 172.21.15.3:0/2998351390 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7340047a0 con 0x7ff7441025d0 2024-09-21T06:15:16.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.265+0000 7ff749e2c640 1 -- 172.21.15.3:0/2998351390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7441a9000 con 0x7ff7441025d0 2024-09-21T06:15:16.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.266+0000 7ff749e2c640 1 -- 172.21.15.3:0/2998351390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7441029e0 con 0x7ff7441025d0 2024-09-21T06:15:16.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.269+0000 7ff740ff9640 1 -- 172.21.15.3:0/2998351390 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 154579+0+0 (secure 0 0 0) 0x7ff734009a30 con 0x7ff7441025d0 2024-09-21T06:15:16.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.270+0000 7ff740ff9640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff72c0b65f0 0x7ff72c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:16.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.270+0000 7ff740ff9640 1 -- 172.21.15.3:0/2998351390 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff7340e7830 con 0x7ff7441025d0 2024-09-21T06:15:16.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.270+0000 7ff742ffd640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff72c0b65f0 0x7ff72c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:16.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.271+0000 7ff742ffd640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff72c0b65f0 0x7ff72c0b8b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff738031440 tx=0x7ff738005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:16.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.274+0000 7ff740ff9640 1 -- 172.21.15.3:0/2998351390 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff7340ade20 con 0x7ff7441025d0 2024-09-21T06:15:16.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.467+0000 7ff749e2c640 1 -- 172.21.15.3:0/2998351390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff74410ad80 con 0x7ff7441025d0 2024-09-21T06:15:16.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.470+0000 7ff740ff9640 1 -- 172.21.15.3:0/2998351390 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23048 (secure 0 0 0) 0x7ff7340a7e20 con 0x7ff7441025d0 2024-09-21T06:15:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:15:16.472 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","created":"2024-09-21T06:04:24.293786+0000","modified":"2024-09-21T06:13:24.594225+0000","last_up_change":"2024-09-21T06:12:17.781243+0000","last_in_change":"2024-09-21T06:11:54.727435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:08:26.015803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:13:07.084406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:13:09.149891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51736f81-f410-48ee-a8f3-121289607309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6803","nonce":615312328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6805","nonce":615312328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6809","nonce":615312328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6807","nonce":615312328}]},"public_addr":"172.21.15.3:6803/615312328","cluster_addr":"172.21.15.3:6805/615312328","heartbeat_back_addr":"172.21.15.3:6809/615312328","heartbeat_front_addr":"172.21.15.3:6807/615312328","state":["exists","up"]},{"osd":1,"uuid":"55696a37-34c1-482a-8561-7d08f0aa29fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6801","nonce":4150747115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6803","nonce":4150747115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6807","nonce":4150747115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6805","nonce":4150747115}]},"public_addr":"172.21.15.137:6801/4150747115","cluster_addr":"172.21.15.137:6803/4150747115","heartbeat_back_addr":"172.21.15.137:6807/4150747115","heartbeat_front_addr":"172.21.15.137:6805/4150747115","state":["exists","up"]},{"osd":2,"uuid":"8a81d34b-464b-46a2-8ef2-5223b5947004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6801","nonce":3392577077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6803","nonce":3392577077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6807","nonce":3392577077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6805","nonce":3392577077}]},"public_addr":"172.21.15.204:6801/3392577077","cluster_addr":"172.21.15.204:6803/3392577077","heartbeat_back_addr":"172.21.15.204:6807/3392577077","heartbeat_front_addr":"172.21.15.204:6805/3392577077","state":["exists","up"]},{"osd":3,"uuid":"94ab3938-7a70-44cb-99bf-9d2dedc4eb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6811","nonce":4293741635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6813","nonce":4293741635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6817","nonce":4293741635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6815","nonce":4293741635}]},"public_addr":"172.21.15.3:6811/4293741635","cluster_addr":"172.21.15.3:6813/4293741635","heartbeat_back_addr":"172.21.15.3:6817/4293741635","heartbeat_front_addr":"172.21.15.3:6815/4293741635","state":["exists","up"]},{"osd":4,"uuid":"38551535-d692-401c-aad0-bee38f3026d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6809","nonce":2608988785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6811","nonce":2608988785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6815","nonce":2608988785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6813","nonce":2608988785}]},"public_addr":"172.21.15.137:6809/2608988785","cluster_addr":"172.21.15.137:6811/2608988785","heartbeat_back_addr":"172.21.15.137:6815/2608988785","heartbeat_front_addr":"172.21.15.137:6813/2608988785","state":["exists","up"]},{"osd":5,"uuid":"3d4b785b-28d0-4654-a9b1-4c5c905f4bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6809","nonce":3709074044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6811","nonce":3709074044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6815","nonce":3709074044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6813","nonce":3709074044}]},"public_addr":"172.21.15.204:6809/3709074044","cluster_addr":"172.21.15.204:6811/3709074044","heartbeat_back_addr":"172.21.15.204:6815/3709074044","heartbeat_front_addr":"172.21.15.204:6813/3709074044","state":["exists","up"]},{"osd":6,"uuid":"ba21dbe9-ab43-4b7e-9446-2929ee874c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6819","nonce":3862646594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6821","nonce":3862646594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6825","nonce":3862646594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6823","nonce":3862646594}]},"public_addr":"172.21.15.3:6819/3862646594","cluster_addr":"172.21.15.3:6821/3862646594","heartbeat_back_addr":"172.21.15.3:6825/3862646594","heartbeat_front_addr":"172.21.15.3:6823/3862646594","state":["exists","up"]},{"osd":7,"uuid":"9610fe5c-93df-49f9-b8bd-5cb480e0871b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6817","nonce":2086822496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6819","nonce":2086822496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6823","nonce":2086822496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6821","nonce":2086822496}]},"public_addr":"172.21.15.137:6817/2086822496","cluster_addr":"172.21.15.137:6819/2086822496","heartbeat_back_addr":"172.21.15.137:6823/2086822496","heartbeat_front_addr":"172.21.15.137:6821/2086822496","state":["exists","up"]},{"osd":8,"uuid":"49f6e9b4-6d14-434e-8b42-8b7ffeb2a072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6817","nonce":153476798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6819","nonce":153476798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6823","nonce":153476798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6821","nonce":153476798}]},"public_addr":"172.21.15.204:6817/153476798","cluster_addr":"172.21.15.204:6819/153476798","heartbeat_back_addr":"172.21.15.204:6823/153476798","heartbeat_front_addr":"172.21.15.204:6821/153476798","state":["exists","up"]},{"osd":9,"uuid":"633f67a7-b759-4ea6-82ed-3c42b3fb70ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6827","nonce":1238491549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6829","nonce":1238491549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6833","nonce":1238491549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6831","nonce":1238491549}]},"public_addr":"172.21.15.3:6827/1238491549","cluster_addr":"172.21.15.3:6829/1238491549","heartbeat_back_addr":"172.21.15.3:6833/1238491549","heartbeat_front_addr":"172.21.15.3:6831/1238491549","state":["exists","up"]},{"osd":10,"uuid":"94b65a48-ee87-4e59-9296-a72a22787f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6825","nonce":3486552683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6827","nonce":3486552683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6831","nonce":3486552683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6829","nonce":3486552683}]},"public_addr":"172.21.15.137:6825/3486552683","cluster_addr":"172.21.15.137:6827/3486552683","heartbeat_back_addr":"172.21.15.137:6831/3486552683","heartbeat_front_addr":"172.21.15.137:6829/3486552683","state":["exists","up"]},{"osd":11,"uuid":"c5cc2850-9062-4b8e-beb3-bef0db30a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6825","nonce":1084308610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6827","nonce":1084308610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6831","nonce":1084308610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6829","nonce":1084308610}]},"public_addr":"172.21.15.204:6825/1084308610","cluster_addr":"172.21.15.204:6827/1084308610","heartbeat_back_addr":"172.21.15.204:6831/1084308610","heartbeat_front_addr":"172.21.15.204:6829/1084308610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:31.898745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.089965+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:22.367464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:48.543248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:14.687331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:41.243579+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:07.290227+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:32.930046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:59.197638+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:24.315219+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:50.333639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:12:16.453640+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1573374138":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1595702358":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/547897695":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/3673347975":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/141972779":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6800/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:6801/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/2363985440":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/2138752581":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1820734047":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/2929116087":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/45453677":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/2929116087":"2024-09-22T06:05:21.689191+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:15:16.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.472+0000 7ff749e2c640 1 -- 172.21.15.3:0/2998351390 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff72c0b65f0 msgr2=0x7ff72c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:16.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.472+0000 7ff749e2c640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff72c0b65f0 0x7ff72c0b8b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff738031440 tx=0x7ff738005d50 comp rx=0 tx=0).stop 2024-09-21T06:15:16.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.472+0000 7ff749e2c640 1 -- 172.21.15.3:0/2998351390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7441025d0 msgr2=0x7ff74419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:16.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.472+0000 7ff749e2c640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7441025d0 0x7ff74419c640 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff73400d950 tx=0x7ff73400de10 comp rx=0 tx=0).stop 2024-09-21T06:15:16.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.472+0000 7ff749e2c640 1 -- 172.21.15.3:0/2998351390 shutdown_connections 2024-09-21T06:15:16.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.472+0000 7ff749e2c640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff72c0b65f0 0x7ff72c0b8b00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:16.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.472+0000 7ff749e2c640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff744102f20 0x7ff74419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:16.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.472+0000 7ff749e2c640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff744110180 0x7ff7441a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:16.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.472+0000 7ff749e2c640 1 --2- 172.21.15.3:0/2998351390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7441025d0 0x7ff74419c640 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:16.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.472+0000 7ff749e2c640 1 -- 172.21.15.3:0/2998351390 >> 172.21.15.3:0/2998351390 conn(0x7ff744079730 msgr2=0x7ff744111600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:16.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.472+0000 7ff749e2c640 1 -- 172.21.15.3:0/2998351390 shutdown_connections 2024-09-21T06:15:16.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:16.473+0000 7ff749e2c640 1 -- 172.21.15.3:0/2998351390 wait complete. 2024-09-21T06:15:17.173 DEBUG:tasks.fwd_scrub:Yielding 2024-09-21T06:15:17.174 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T06:15:17.182 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T06:15:17.182 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-09-21T06:15:17.182 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-21T06:15:17.183 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-21T06:15:17.183 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:15:17.183 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:15:17.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.301+0000 7f715a694640 1 -- 172.21.15.3:0/3366827201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715410dd60 msgr2=0x7f7154110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:17.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.301+0000 7f715a694640 1 --2- 172.21.15.3:0/3366827201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715410dd60 0x7f7154110250 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f714c009ca0 tx=0x7f714c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.302+0000 7f715a694640 1 -- 172.21.15.3:0/3366827201 shutdown_connections 2024-09-21T06:15:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.302+0000 7f715a694640 1 --2- 172.21.15.3:0/3366827201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715410dd60 0x7f7154110250 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.302+0000 7f715a694640 1 --2- 172.21.15.3:0/3366827201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7154105250 0x7f715410d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.302+0000 7f715a694640 1 --2- 172.21.15.3:0/3366827201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7154104900 0x7f7154104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.302+0000 7f715a694640 1 -- 172.21.15.3:0/3366827201 >> 172.21.15.3:0/3366827201 conn(0x7f715407a740 msgr2=0x7f715407ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.302+0000 7f715a694640 1 -- 172.21.15.3:0/3366827201 shutdown_connections 2024-09-21T06:15:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.302+0000 7f715a694640 1 -- 172.21.15.3:0/3366827201 wait complete. 2024-09-21T06:15:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.302+0000 7f715a694640 1 Processor -- start 2024-09-21T06:15:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.302+0000 7f715a694640 1 -- start start 2024-09-21T06:15:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f715a694640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7154104900 0x7f715419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f715a694640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7154105250 0x7f715419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f715a694640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715410dd60 0x7f715419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f715a694640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f71541139e0 con 0x7f715410dd60 2024-09-21T06:15:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f715a694640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7154113860 con 0x7f7154105250 2024-09-21T06:15:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f715a694640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7154113b60 con 0x7f7154104900 2024-09-21T06:15:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f7158c31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715410dd60 0x7f715419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f7158c31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715410dd60 0x7f715419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48170/0 (socket says 172.21.15.3:48170) 2024-09-21T06:15:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f7158c31640 1 -- 172.21.15.3:0/3023941286 learned_addr learned my addr 172.21.15.3:0/3023941286 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f71537fe640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7154105250 0x7f715419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f7153fff640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7154104900 0x7f715419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f7158c31640 1 -- 172.21.15.3:0/3023941286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7154104900 msgr2=0x7f715419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f7158c31640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7154104900 0x7f715419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f7158c31640 1 -- 172.21.15.3:0/3023941286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7154105250 msgr2=0x7f715419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f7158c31640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7154105250 0x7f715419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.303+0000 7f7158c31640 1 -- 172.21.15.3:0/3023941286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71541a0390 con 0x7f715410dd60 2024-09-21T06:15:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.304+0000 7f7158c31640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715410dd60 0x7f715419fc90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f714c0098d0 tx=0x7f714c0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.304+0000 7f71517fa640 1 -- 172.21.15.3:0/3023941286 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c003b70 con 0x7f715410dd60 2024-09-21T06:15:17.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.304+0000 7f715a694640 1 -- 172.21.15.3:0/3023941286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71541a0620 con 0x7f715410dd60 2024-09-21T06:15:17.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.304+0000 7f71517fa640 1 -- 172.21.15.3:0/3023941286 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f714c003d10 con 0x7f715410dd60 2024-09-21T06:15:17.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.304+0000 7f71517fa640 1 -- 172.21.15.3:0/3023941286 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c02fa20 con 0x7f715410dd60 2024-09-21T06:15:17.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.304+0000 7f715a694640 1 -- 172.21.15.3:0/3023941286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71541a6e10 con 0x7f715410dd60 2024-09-21T06:15:17.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.305+0000 7f715a694640 1 -- 172.21.15.3:0/3023941286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7154104d10 con 0x7f715410dd60 2024-09-21T06:15:17.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.307+0000 7f71517fa640 1 -- 172.21.15.3:0/3023941286 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 154579+0+0 (secure 0 0 0) 0x7f714c02fbe0 con 0x7f715410dd60 2024-09-21T06:15:17.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.308+0000 7f71517fa640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f713c0b65f0 0x7f713c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.308+0000 7f71517fa640 1 -- 172.21.15.3:0/3023941286 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f714c108b00 con 0x7f715410dd60 2024-09-21T06:15:17.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.308+0000 7f7153fff640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f713c0b65f0 0x7f713c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.309+0000 7f7153fff640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f713c0b65f0 0x7f713c0b8b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7144009b60 tx=0x7f7144005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:17.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.312+0000 7f71517fa640 1 -- 172.21.15.3:0/3023941286 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f714c0cf1e0 con 0x7f715410dd60 2024-09-21T06:15:17.453 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.531+0000 7f715a694640 1 -- 172.21.15.3:0/3023941286 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7154100b50 con 0x7f713c0b65f0 2024-09-21T06:15:17.573 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:17 smithi003 ceph-mon[31227]: pgmap v333: 97 pgs: 97 active+clean; 658 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-09-21T06:15:17.574 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:17 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2998351390' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:15:17.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:17 smithi137 ceph-mon[34464]: pgmap v333: 97 pgs: 97 active+clean; 658 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-09-21T06:15:17.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:17 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2998351390' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:15:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:17 smithi204 ceph-mon[34659]: pgmap v333: 97 pgs: 97 active+clean; 658 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-09-21T06:15:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:17 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2998351390' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:15:17.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.693+0000 7f71517fa640 1 -- 172.21.15.3:0/3023941286 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7154100b50 con 0x7f713c0b65f0 2024-09-21T06:15:17.695 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path / 2024-09-21T06:15:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.695+0000 7f715a694640 1 -- 172.21.15.3:0/3023941286 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f713c0b65f0 msgr2=0x7f713c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.695+0000 7f715a694640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f713c0b65f0 0x7f713c0b8b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7144009b60 tx=0x7f7144005f90 comp rx=0 tx=0).stop 2024-09-21T06:15:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.695+0000 7f715a694640 1 -- 172.21.15.3:0/3023941286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715410dd60 msgr2=0x7f715419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.695+0000 7f715a694640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715410dd60 0x7f715419fc90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f714c0098d0 tx=0x7f714c0317b0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.695+0000 7f715a694640 1 -- 172.21.15.3:0/3023941286 shutdown_connections 2024-09-21T06:15:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.695+0000 7f715a694640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f713c0b65f0 0x7f713c0b8b00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.695+0000 7f715a694640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715410dd60 0x7f715419fc90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.695+0000 7f715a694640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7154105250 0x7f715419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.695+0000 7f715a694640 1 --2- 172.21.15.3:0/3023941286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7154104900 0x7f715419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.695+0000 7f715a694640 1 -- 172.21.15.3:0/3023941286 >> 172.21.15.3:0/3023941286 conn(0x7f715407a740 msgr2=0x7f71540ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.695+0000 7f715a694640 1 -- 172.21.15.3:0/3023941286 shutdown_connections 2024-09-21T06:15:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.695+0000 7f715a694640 1 -- 172.21.15.3:0/3023941286 wait complete. 2024-09-21T06:15:17.708 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-21T06:15:17.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.843+0000 7fd02a7a1640 1 -- 172.21.15.3:0/2395429529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd024108310 msgr2=0x7fd024108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:17.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.844+0000 7fd02a7a1640 1 --2- 172.21.15.3:0/2395429529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd024108310 0x7fd024108720 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd018009900 tx=0x7fd01802a410 comp rx=0 tx=0).stop 2024-09-21T06:15:17.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.844+0000 7fd02a7a1640 1 -- 172.21.15.3:0/2395429529 shutdown_connections 2024-09-21T06:15:17.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.844+0000 7fd02a7a1640 1 --2- 172.21.15.3:0/2395429529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd024108c60 0x7fd02410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.844+0000 7fd02a7a1640 1 --2- 172.21.15.3:0/2395429529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd024108310 0x7fd024108720 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.844+0000 7fd02a7a1640 1 --2- 172.21.15.3:0/2395429529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0241029e0 0x7fd024102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.844+0000 7fd02a7a1640 1 -- 172.21.15.3:0/2395429529 >> 172.21.15.3:0/2395429529 conn(0x7fd0240fe3c0 msgr2=0x7fd024100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:17.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.844+0000 7fd02a7a1640 1 -- 172.21.15.3:0/2395429529 shutdown_connections 2024-09-21T06:15:17.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.844+0000 7fd02a7a1640 1 -- 172.21.15.3:0/2395429529 wait complete. 2024-09-21T06:15:17.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.844+0000 7fd02a7a1640 1 Processor -- start 2024-09-21T06:15:17.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.844+0000 7fd02a7a1640 1 -- start start 2024-09-21T06:15:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd02a7a1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0241029e0 0x7fd02419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd02a7a1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024108310 0x7fd02419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd02a7a1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd024108c60 0x7fd02419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd02a7a1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd024113a40 con 0x7fd024108c60 2024-09-21T06:15:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd02a7a1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0241138c0 con 0x7fd024108310 2024-09-21T06:15:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd02a7a1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd024113bc0 con 0x7fd0241029e0 2024-09-21T06:15:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd028d3e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd024108c60 0x7fd02419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd028d3e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd024108c60 0x7fd02419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48208/0 (socket says 172.21.15.3:48208) 2024-09-21T06:15:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd028d3e640 1 -- 172.21.15.3:0/3316425669 learned_addr learned my addr 172.21.15.3:0/3316425669 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd023fff640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0241029e0 0x7fd02419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd0237fe640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024108310 0x7fd02419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd028d3e640 1 -- 172.21.15.3:0/3316425669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0241029e0 msgr2=0x7fd02419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd028d3e640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0241029e0 0x7fd02419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd028d3e640 1 -- 172.21.15.3:0/3316425669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024108310 msgr2=0x7fd02419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd028d3e640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024108310 0x7fd02419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd028d3e640 1 -- 172.21.15.3:0/3316425669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0241a0390 con 0x7fd024108c60 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.845+0000 7fd0237fe640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024108310 0x7fd02419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.846+0000 7fd028d3e640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd024108c60 0x7fd02419fc90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00eb10 tx=0x7fd01c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.846+0000 7fd0217fa640 1 -- 172.21.15.3:0/3316425669 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c004b20 con 0x7fd024108c60 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.846+0000 7fd0217fa640 1 -- 172.21.15.3:0/3316425669 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd01c004cc0 con 0x7fd024108c60 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.846+0000 7fd0217fa640 1 -- 172.21.15.3:0/3316425669 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c010670 con 0x7fd024108c60 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.846+0000 7fd02a7a1640 1 -- 172.21.15.3:0/3316425669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0241a0680 con 0x7fd024108c60 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.846+0000 7fd02a7a1640 1 -- 172.21.15.3:0/3316425669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0241a6e10 con 0x7fd024108c60 2024-09-21T06:15:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.847+0000 7fd02a7a1640 1 -- 172.21.15.3:0/3316425669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd024103c50 con 0x7fd024108c60 2024-09-21T06:15:17.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.848+0000 7fd0217fa640 1 -- 172.21.15.3:0/3316425669 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 154579+0+0 (secure 0 0 0) 0x7fd01c005000 con 0x7fd024108c60 2024-09-21T06:15:17.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.849+0000 7fd0217fa640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd00c0b6670 0x7fd00c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.849+0000 7fd023fff640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd00c0b6670 0x7fd00c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.849+0000 7fd0217fa640 1 -- 172.21.15.3:0/3316425669 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fd01c0e7d80 con 0x7fd024108c60 2024-09-21T06:15:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.850+0000 7fd023fff640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd00c0b6670 0x7fd00c0b8b80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd014009bd0 tx=0x7fd014005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.852+0000 7fd0217fa640 1 -- 172.21.15.3:0/3316425669 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd01c056710 con 0x7fd024108c60 2024-09-21T06:15:17.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.980+0000 7faf3c5bd640 1 -- 172.21.15.3:0/1628189556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3410de10 msgr2=0x7faf34110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:17.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.980+0000 7faf3c5bd640 1 --2- 172.21.15.3:0/1628189556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3410de10 0x7faf34110300 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009ca0 tx=0x7faf2c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.980+0000 7faf3c5bd640 1 -- 172.21.15.3:0/1628189556 shutdown_connections 2024-09-21T06:15:17.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.980+0000 7faf3c5bd640 1 --2- 172.21.15.3:0/1628189556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3410de10 0x7faf34110300 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.980+0000 7faf3c5bd640 1 --2- 172.21.15.3:0/1628189556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf34105300 0x7faf3410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.980+0000 7faf3c5bd640 1 --2- 172.21.15.3:0/1628189556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf341049b0 0x7faf34104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.980+0000 7faf3c5bd640 1 -- 172.21.15.3:0/1628189556 >> 172.21.15.3:0/1628189556 conn(0x7faf34079730 msgr2=0x7faf34079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:17.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.980+0000 7faf3c5bd640 1 -- 172.21.15.3:0/1628189556 shutdown_connections 2024-09-21T06:15:17.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.980+0000 7faf3c5bd640 1 -- 172.21.15.3:0/1628189556 wait complete. 2024-09-21T06:15:17.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.981+0000 7faf3c5bd640 1 Processor -- start 2024-09-21T06:15:17.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.982+0000 7faf3c5bd640 1 -- start start 2024-09-21T06:15:17.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.982+0000 7faf3c5bd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf341049b0 0x7faf3419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.982+0000 7faf3c5bd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf34105300 0x7faf3419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.982+0000 7faf3c5bd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf3410de10 0x7faf3419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.982+0000 7faf3c5bd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faf34113a70 con 0x7faf34105300 2024-09-21T06:15:17.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.982+0000 7faf3c5bd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faf341138f0 con 0x7faf341049b0 2024-09-21T06:15:17.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.982+0000 7faf3c5bd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faf34113bf0 con 0x7faf3410de10 2024-09-21T06:15:17.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.982+0000 7faf39b58640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf34105300 0x7faf3419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.982+0000 7faf39b58640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf34105300 0x7faf3419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48218/0 (socket says 172.21.15.3:48218) 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.982+0000 7faf39b58640 1 -- 172.21.15.3:0/1550243245 learned_addr learned my addr 172.21.15.3:0/1550243245 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.982+0000 7faf3a359640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf341049b0 0x7faf3419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf3ab5a640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf3410de10 0x7faf3419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf39b58640 1 -- 172.21.15.3:0/1550243245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf3410de10 msgr2=0x7faf3419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf39b58640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf3410de10 0x7faf3419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf39b58640 1 -- 172.21.15.3:0/1550243245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf341049b0 msgr2=0x7faf3419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf39b58640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf341049b0 0x7faf3419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf39b58640 1 -- 172.21.15.3:0/1550243245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf341a02f0 con 0x7faf34105300 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf3a359640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf341049b0 0x7faf3419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf3ab5a640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf3410de10 0x7faf3419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf39b58640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf34105300 0x7faf3419a9e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faf2400c8c0 tx=0x7faf2400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf2b7fe640 1 -- 172.21.15.3:0/1550243245 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24007bb0 con 0x7faf34105300 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf2b7fe640 1 -- 172.21.15.3:0/1550243245 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf24007d50 con 0x7faf34105300 2024-09-21T06:15:17.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf2b7fe640 1 -- 172.21.15.3:0/1550243245 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf240187c0 con 0x7faf34105300 2024-09-21T06:15:17.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf3c5bd640 1 -- 172.21.15.3:0/1550243245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf341a05e0 con 0x7faf34105300 2024-09-21T06:15:17.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.983+0000 7faf3c5bd640 1 -- 172.21.15.3:0/1550243245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf341a6e00 con 0x7faf34105300 2024-09-21T06:15:17.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.986+0000 7faf42b31740 1 Processor -- start 2024-09-21T06:15:17.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.987+0000 7faf42b31740 1 -- start start 2024-09-21T06:15:17.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.987+0000 7faf42b31740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 0x55937db21430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.987+0000 7faf42b31740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d751490 0x55937db21970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.987+0000 7faf42b31740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d759670 0x55937db21eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.987+0000 7faf42b31740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55937da97e60 con 0x55937d77a350 2024-09-21T06:15:17.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.987+0000 7faf42b31740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55937da87290 con 0x55937d759670 2024-09-21T06:15:17.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.987+0000 7faf42b31740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55937da8b0a0 con 0x55937d751490 2024-09-21T06:15:17.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.987+0000 7faf3ab5a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 0x55937db21430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.987+0000 7faf3ab5a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 0x55937db21430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48222/0 (socket says 172.21.15.3:48222) 2024-09-21T06:15:17.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.987+0000 7faf3ab5a640 1 -- 172.21.15.3:0/3770523541 learned_addr learned my addr 172.21.15.3:0/3770523541 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:17.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.987+0000 7faf2b7fe640 1 -- 172.21.15.3:0/1550243245 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 154579+0+0 (secure 0 0 0) 0x7faf24018960 con 0x7faf34105300 2024-09-21T06:15:17.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.988+0000 7faf3a359640 1 --2- 172.21.15.3:0/3770523541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d751490 0x55937db21970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.988+0000 7faf3ab5a640 1 -- 172.21.15.3:0/3770523541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d751490 msgr2=0x55937db21970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:17.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.988+0000 7faf3ab5a640 1 --2- 172.21.15.3:0/3770523541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d751490 0x55937db21970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.988+0000 7faf3ab5a640 1 -- 172.21.15.3:0/3770523541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d759670 msgr2=0x55937db21eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:17.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.988+0000 7faf3ab5a640 1 --2- 172.21.15.3:0/3770523541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d759670 0x55937db21eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.988+0000 7faf3ab5a640 1 -- 172.21.15.3:0/3770523541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937d7e5930 con 0x55937d77a350 2024-09-21T06:15:17.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.988+0000 7faf3ab5a640 1 --2- 172.21.15.3:0/3770523541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 0x55937db21430 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faf2c02f500 tx=0x7faf2c009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb079b7834fdb7d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:17.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.988+0000 7faf28ff9640 1 -- 172.21.15.3:0/3770523541 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c031450 con 0x55937d77a350 2024-09-21T06:15:17.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.988+0000 7faf28ff9640 1 -- 172.21.15.3:0/3770523541 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf2c036040 con 0x55937d77a350 2024-09-21T06:15:17.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.988+0000 7faf28ff9640 1 -- 172.21.15.3:0/3770523541 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c004350 con 0x55937d77a350 2024-09-21T06:15:17.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.988+0000 7faf2b7fe640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faf200b6620 0x7faf200b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.989+0000 7faf3a359640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faf200b6620 0x7faf200b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.989+0000 7faf2b7fe640 1 -- 172.21.15.3:0/1550243245 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7faf240e7780 con 0x7faf34105300 2024-09-21T06:15:17.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.989+0000 7faf42b31740 1 -- 172.21.15.3:0/3770523541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 msgr2=0x55937db21430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:17.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.989+0000 7faf3a359640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faf200b6620 0x7faf200b8b30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55937db200b0 tx=0x7faf30005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:17.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.989+0000 7faf42b31740 1 --2- 172.21.15.3:0/3770523541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 0x55937db21430 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faf2c02f500 tx=0x7faf2c009910 comp rx=0 tx=0).stop 2024-09-21T06:15:17.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.990+0000 7faf42b31740 1 -- 172.21.15.3:0/3770523541 shutdown_connections 2024-09-21T06:15:17.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.990+0000 7faf42b31740 1 --2- 172.21.15.3:0/3770523541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d751490 0x55937db21970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.990+0000 7faf42b31740 1 --2- 172.21.15.3:0/3770523541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d759670 0x55937db21eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.990+0000 7faf42b31740 1 --2- 172.21.15.3:0/3770523541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 0x55937db21430 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.990+0000 7faf42b31740 1 -- 172.21.15.3:0/3770523541 >> 172.21.15.3:0/3770523541 conn(0x55937d76e740 msgr2=0x55937db1f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:17.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.990+0000 7faf42b31740 1 -- 172.21.15.3:0/3770523541 shutdown_connections 2024-09-21T06:15:17.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.990+0000 7faf42b31740 1 -- 172.21.15.3:0/3770523541 wait complete. 2024-09-21T06:15:17.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.990+0000 7faf42b31740 1 -- start start 2024-09-21T06:15:17.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.990+0000 7faf42b31740 1 Processor -- start 2024-09-21T06:15:17.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf42b31740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d751490 0x55937db2d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf42b31740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d759670 0x55937db2d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf42b31740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d77a350 0x55937db37290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf42b31740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55937d8e8890 con 0x55937d751490 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf42b31740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55937da21d40 con 0x55937d77a350 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf42b31740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55937db33fc0 con 0x55937d759670 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf42b31740 10 client.0 _ll_register_callbacks cb 0x55937da85c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf3a359640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d751490 0x55937db2d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf3a359640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d751490 0x55937db2d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48230/0 (socket says 172.21.15.3:48230) 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf3a359640 1 -- 172.21.15.3:0/4161433019 learned_addr learned my addr 172.21.15.3:0/4161433019 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf39b58640 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d759670 0x55937db2d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf3ab5a640 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d77a350 0x55937db37290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf39b58640 1 -- 172.21.15.3:0/4161433019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d77a350 msgr2=0x55937db37290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf39b58640 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d77a350 0x55937db37290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf39b58640 1 -- 172.21.15.3:0/4161433019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d751490 msgr2=0x55937db2d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf39b58640 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d751490 0x55937db2d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf39b58640 1 -- 172.21.15.3:0/4161433019 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937db37900 con 0x55937d759670 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf3a359640 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d751490 0x55937db2d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:17.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf39b58640 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d759670 0x55937db2d710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faf240e5050 tx=0x7faf240a9640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5170439bfd0db93a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.991+0000 7faf1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.992+0000 7faf1dffb640 1 -- 172.21.15.3:0/4161433019 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24018e60 con 0x55937d759670 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.992+0000 7faf1dffb640 1 -- 172.21.15.3:0/4161433019 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7faf240acb30 con 0x55937d759670 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.992+0000 7faf42b31740 1 -- 172.21.15.3:0/4161433019 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf2c036040 con 0x55937d759670 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.992+0000 7faf1dffb640 1 -- 172.21.15.3:0/4161433019 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf240ad1e0 con 0x55937d759670 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.992+0000 7faf42b31740 10 client.25177 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.992+0000 7faf42b31740 1 -- 172.21.15.3:0/4161433019 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55937db402d0 con 0x55937d759670 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.992+0000 7faf1dffb640 1 -- 172.21.15.3:0/4161433019 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf2c036040 con 0x55937d759670 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.992+0000 7faf1dffb640 1 -- 172.21.15.3:0/4161433019 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faf240ac5a0 con 0x55937d759670 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.993+0000 7faf42b31740 10 client.25177 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.993+0000 7faf42b31740 10 client.25177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.993+0000 7faf42b31740 20 client.25177 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.993+0000 7faf42b31740 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55937d738cd0 0x55937db43980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.993+0000 7faf42b31740 4 client.25177 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.993+0000 7faf42b31740 1 -- 172.21.15.3:0/4161433019 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55937da99db0 con 0x55937d738cd0 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.993+0000 7faf3a359640 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55937d738cd0 0x55937db43980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.993+0000 7faf1dffb640 10 client.25177 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.993+0000 7faf3a359640 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55937d738cd0 0x55937db43980 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0a74fd80e53ecb server_cookie=713695faa8884648 in_seq=0 out_seq=0 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf1dffb640 1 -- 172.21.15.3:0/4161433019 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55937da99db0 con 0x55937d738cd0 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf1dffb640 10 client.25177 handle_command_reply: tid=0 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf42b31740 1 client.25177 shutdown 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf42b31740 1 -- 172.21.15.3:0/4161433019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d759670 msgr2=0x55937db2d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf42b31740 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d759670 0x55937db2d710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faf240e5050 tx=0x7faf240a9640 comp rx=0 tx=0).stop 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf42b31740 1 -- 172.21.15.3:0/4161433019 shutdown_connections 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf42b31740 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55937d738cd0 0x55937db43980 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf42b31740 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d759670 0x55937db2d710 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf42b31740 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d77a350 0x55937db37290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf42b31740 1 --2- 172.21.15.3:0/4161433019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d751490 0x55937db2d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf1dffb640 0 client.25177 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:18.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf42b31740 1 -- 172.21.15.3:0/4161433019 >> 172.21.15.3:0/4161433019 conn(0x55937d76e740 msgr2=0x55937db22460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:18.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf42b31740 1 -- 172.21.15.3:0/4161433019 shutdown_connections 2024-09-21T06:15:18.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.994+0000 7faf42b31740 1 -- 172.21.15.3:0/4161433019 wait complete. 2024-09-21T06:15:18.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:17.995+0000 7faf42b31740 20 client.25177 trim_cache size 0 max 16384 2024-09-21T06:15:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf42b31740 1 Processor -- start 2024-09-21T06:15:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf42b31740 1 -- start start 2024-09-21T06:15:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf42b31740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d751490 0x55937db22460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf42b31740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d759670 0x55937db229a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf42b31740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 0x55937db1ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf42b31740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faf341ad440 con 0x55937d77a350 2024-09-21T06:15:18.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf42b31740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faf341aef80 con 0x55937d759670 2024-09-21T06:15:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf42b31740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faf341adb10 con 0x55937d751490 2024-09-21T06:15:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf3ab5a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 0x55937db1ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf3ab5a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 0x55937db1ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48232/0 (socket says 172.21.15.3:48232) 2024-09-21T06:15:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf3ab5a640 1 -- 172.21.15.3:0/149684451 learned_addr learned my addr 172.21.15.3:0/149684451 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf3a359640 1 --2- 172.21.15.3:0/149684451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d751490 0x55937db22460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf39b58640 1 --2- 172.21.15.3:0/149684451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d759670 0x55937db229a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf3ab5a640 1 -- 172.21.15.3:0/149684451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d751490 msgr2=0x55937db22460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf3ab5a640 1 --2- 172.21.15.3:0/149684451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d751490 0x55937db22460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf3ab5a640 1 -- 172.21.15.3:0/149684451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d759670 msgr2=0x55937db229a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf3ab5a640 1 --2- 172.21.15.3:0/149684451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d759670 0x55937db229a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.013+0000 7faf3ab5a640 1 -- 172.21.15.3:0/149684451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937db23c20 con 0x55937d77a350 2024-09-21T06:15:18.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.014+0000 7faf3ab5a640 1 --2- 172.21.15.3:0/149684451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 0x55937db1ec40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009980 tx=0x7faf2c02afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40362cdc48cbf63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:18.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.014+0000 7faf1ffff640 1 -- 172.21.15.3:0/149684451 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c04a040 con 0x55937d77a350 2024-09-21T06:15:18.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.014+0000 7faf1ffff640 1 -- 172.21.15.3:0/149684451 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf2c036040 con 0x55937d77a350 2024-09-21T06:15:18.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.014+0000 7faf1ffff640 1 -- 172.21.15.3:0/149684451 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c004560 con 0x55937d77a350 2024-09-21T06:15:18.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.015+0000 7faf42b31740 1 -- 172.21.15.3:0/149684451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 msgr2=0x55937db1ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.015+0000 7faf42b31740 1 --2- 172.21.15.3:0/149684451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 0x55937db1ec40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009980 tx=0x7faf2c02afc0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.015+0000 7faf42b31740 1 -- 172.21.15.3:0/149684451 shutdown_connections 2024-09-21T06:15:18.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.015+0000 7faf42b31740 1 --2- 172.21.15.3:0/149684451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d77a350 0x55937db1ec40 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.015+0000 7faf42b31740 1 --2- 172.21.15.3:0/149684451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d759670 0x55937db229a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.015+0000 7faf42b31740 1 --2- 172.21.15.3:0/149684451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d751490 0x55937db22460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.015+0000 7faf42b31740 1 -- 172.21.15.3:0/149684451 >> 172.21.15.3:0/149684451 conn(0x55937d76e740 msgr2=0x55937db2b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.015+0000 7faf42b31740 1 -- 172.21.15.3:0/149684451 shutdown_connections 2024-09-21T06:15:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.015+0000 7faf42b31740 1 -- 172.21.15.3:0/149684451 wait complete. 2024-09-21T06:15:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.015+0000 7faf42b31740 1 -- start start 2024-09-21T06:15:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.015+0000 7faf42b31740 1 Processor -- start 2024-09-21T06:15:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf42b31740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d751490 0x55937db39820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf42b31740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d759670 0x55937db39d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf42b31740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d77a350 0x55937db3a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf42b31740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faf341ae8b0 con 0x55937d759670 2024-09-21T06:15:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf42b31740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faf341ae1e0 con 0x55937d751490 2024-09-21T06:15:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf42b31740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faf341ab440 con 0x55937d77a350 2024-09-21T06:15:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf42b31740 10 client.0 _ll_register_callbacks cb 0x7faf240a9570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf39b58640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d759670 0x55937db39d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf39b58640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d759670 0x55937db39d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48246/0 (socket says 172.21.15.3:48246) 2024-09-21T06:15:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf39b58640 1 -- 172.21.15.3:0/101823051 learned_addr learned my addr 172.21.15.3:0/101823051 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf3ab5a640 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d77a350 0x55937db3a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf3a359640 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d751490 0x55937db39820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf39b58640 1 -- 172.21.15.3:0/101823051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d77a350 msgr2=0x55937db3a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf39b58640 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d77a350 0x55937db3a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf39b58640 1 -- 172.21.15.3:0/101823051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d751490 msgr2=0x55937db39820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf39b58640 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d751490 0x55937db39820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf39b58640 1 -- 172.21.15.3:0/101823051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937dbad1f0 con 0x55937d759670 2024-09-21T06:15:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.016+0000 7faf3a359640 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d751490 0x55937db39820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:18.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf39b58640 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d759670 0x55937db39d60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7faf240e5080 tx=0x7faf240a9350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d24aa8e901d0ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:18.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:18.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf1dffb640 1 -- 172.21.15.3:0/101823051 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2405c070 con 0x55937d759670 2024-09-21T06:15:18.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf1dffb640 1 -- 172.21.15.3:0/101823051 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7faf240b6040 con 0x55937d759670 2024-09-21T06:15:18.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf42b31740 1 -- 172.21.15.3:0/101823051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf2c036040 con 0x55937d759670 2024-09-21T06:15:18.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf1dffb640 1 -- 172.21.15.3:0/101823051 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf24018e60 con 0x55937d759670 2024-09-21T06:15:18.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf42b31740 10 client.15573 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:18.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf42b31740 1 -- 172.21.15.3:0/101823051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55937dbae4b0 con 0x55937d759670 2024-09-21T06:15:18.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf1dffb640 1 -- 172.21.15.3:0/101823051 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf2c036040 con 0x55937d759670 2024-09-21T06:15:18.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf1dffb640 1 -- 172.21.15.3:0/101823051 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faf240b6240 con 0x55937d759670 2024-09-21T06:15:18.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf42b31740 10 client.15573 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:18.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf42b31740 10 client.15573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:18.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf42b31740 20 client.15573 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:18.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf42b31740 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55937d70e080 0x55937dbb0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf42b31740 4 client.15573 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:15:18.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.017+0000 7faf42b31740 1 -- 172.21.15.3:0/101823051 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55937da95860 con 0x55937d70e080 2024-09-21T06:15:18.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.018+0000 7faf3a359640 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55937d70e080 0x55937dbb0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:18.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.018+0000 7faf1dffb640 10 client.15573 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:18.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.018+0000 7faf3a359640 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55937d70e080 0x55937dbb0bb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5055f4e53562330 server_cookie=d17184d8a1ce9768 in_seq=0 out_seq=0 2024-09-21T06:15:18.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf1dffb640 1 -- 172.21.15.3:0/101823051 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55937da95860 con 0x55937d70e080 2024-09-21T06:15:18.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf1dffb640 10 client.15573 handle_command_reply: tid=0 2024-09-21T06:15:18.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf42b31740 1 client.15573 shutdown 2024-09-21T06:15:18.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf42b31740 1 -- 172.21.15.3:0/101823051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d759670 msgr2=0x55937db39d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf42b31740 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d759670 0x55937db39d60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7faf240e5080 tx=0x7faf240a9350 comp rx=0 tx=0).stop 2024-09-21T06:15:18.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf42b31740 1 -- 172.21.15.3:0/101823051 shutdown_connections 2024-09-21T06:15:18.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf42b31740 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55937d70e080 0x55937dbb0bb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf42b31740 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55937d77a350 0x55937db3a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf42b31740 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55937d759670 0x55937db39d60 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf42b31740 1 --2- 172.21.15.3:0/101823051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937d751490 0x55937db39820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf1dffb640 0 client.15573 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:18.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf42b31740 1 -- 172.21.15.3:0/101823051 >> 172.21.15.3:0/101823051 conn(0x55937d76e740 msgr2=0x55937db22460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:18.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf42b31740 1 -- 172.21.15.3:0/101823051 shutdown_connections 2024-09-21T06:15:18.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.020+0000 7faf42b31740 1 -- 172.21.15.3:0/101823051 wait complete. 2024-09-21T06:15:18.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.021+0000 7faf42b31740 20 client.15573 trim_cache size 0 max 16384 2024-09-21T06:15:18.025 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:15:18.025 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:15:18.025 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "4957cde1-7a9e-430b-a809-39863daca10a", 2024-09-21T06:15:18.025 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:15:18.025 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:15:18.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.021+0000 7faf3c5bd640 1 -- 172.21.15.3:0/1550243245 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faf200b6620 msgr2=0x7faf200b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.021+0000 7faf3c5bd640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faf200b6620 0x7faf200b8b30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55937db200b0 tx=0x7faf30005f90 comp rx=0 tx=0).stop 2024-09-21T06:15:18.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.021+0000 7faf3c5bd640 1 -- 172.21.15.3:0/1550243245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf34105300 msgr2=0x7faf3419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.021+0000 7faf3c5bd640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf34105300 0x7faf3419a9e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faf2400c8c0 tx=0x7faf2400cd80 comp rx=0 tx=0).stop 2024-09-21T06:15:18.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.021+0000 7faf3c5bd640 1 -- 172.21.15.3:0/1550243245 shutdown_connections 2024-09-21T06:15:18.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.021+0000 7faf3c5bd640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faf200b6620 0x7faf200b8b30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.021+0000 7faf3c5bd640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf3410de10 0x7faf3419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.021+0000 7faf3c5bd640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf34105300 0x7faf3419a9e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.021+0000 7faf3c5bd640 1 --2- 172.21.15.3:0/1550243245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf341049b0 0x7faf3419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.021+0000 7faf3c5bd640 1 -- 172.21.15.3:0/1550243245 >> 172.21.15.3:0/1550243245 conn(0x7faf34079730 msgr2=0x7faf340ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:18.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.021+0000 7faf3c5bd640 1 -- 172.21.15.3:0/1550243245 shutdown_connections 2024-09-21T06:15:18.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.021+0000 7faf3c5bd640 1 -- 172.21.15.3:0/1550243245 wait complete. 2024-09-21T06:15:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.073+0000 7fd02a7a1640 1 -- 172.21.15.3:0/3316425669 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fd024063800 con 0x7fd00c0b6670 2024-09-21T06:15:18.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.139+0000 7fd0217fa640 1 -- 172.21.15.3:0/3316425669 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fd024063800 con 0x7fd00c0b6670 2024-09-21T06:15:18.141 INFO:teuthology.orchestra.run.smithi003.stdout:Retention added to path / 2024-09-21T06:15:18.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.141+0000 7fd02a7a1640 1 -- 172.21.15.3:0/3316425669 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd00c0b6670 msgr2=0x7fd00c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.141+0000 7fd02a7a1640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd00c0b6670 0x7fd00c0b8b80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd014009bd0 tx=0x7fd014005f90 comp rx=0 tx=0).stop 2024-09-21T06:15:18.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.141+0000 7fd02a7a1640 1 -- 172.21.15.3:0/3316425669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd024108c60 msgr2=0x7fd02419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.141+0000 7fd02a7a1640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd024108c60 0x7fd02419fc90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00eb10 tx=0x7fd01c00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.141+0000 7fd02a7a1640 1 -- 172.21.15.3:0/3316425669 shutdown_connections 2024-09-21T06:15:18.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.141+0000 7fd02a7a1640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd00c0b6670 0x7fd00c0b8b80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.141+0000 7fd02a7a1640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd024108c60 0x7fd02419fc90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.141+0000 7fd02a7a1640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024108310 0x7fd02419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.141+0000 7fd02a7a1640 1 --2- 172.21.15.3:0/3316425669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0241029e0 0x7fd02419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.141+0000 7fd02a7a1640 1 -- 172.21.15.3:0/3316425669 >> 172.21.15.3:0/3316425669 conn(0x7fd0240fe3c0 msgr2=0x7fd024101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:18.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.141+0000 7fd02a7a1640 1 -- 172.21.15.3:0/3316425669 shutdown_connections 2024-09-21T06:15:18.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.141+0000 7fd02a7a1640 1 -- 172.21.15.3:0/3316425669 wait complete. 2024-09-21T06:15:18.155 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T06:15:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.269+0000 7f235af28640 1 -- 172.21.15.3:0/1144936779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2354108c30 msgr2=0x7f235410f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.269+0000 7f235af28640 1 --2- 172.21.15.3:0/1144936779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2354108c30 0x7f235410f460 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f234c009ca0 tx=0x7f234c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.269+0000 7f235af28640 1 -- 172.21.15.3:0/1144936779 shutdown_connections 2024-09-21T06:15:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.269+0000 7f235af28640 1 --2- 172.21.15.3:0/1144936779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23541082e0 0x7f23541086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.269+0000 7f235af28640 1 --2- 172.21.15.3:0/1144936779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2354108c30 0x7f235410f460 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.269+0000 7f235af28640 1 --2- 172.21.15.3:0/1144936779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23541029b0 0x7f2354102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.269+0000 7f235af28640 1 -- 172.21.15.3:0/1144936779 >> 172.21.15.3:0/1144936779 conn(0x7f235407a740 msgr2=0x7f235407ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:18.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.269+0000 7f235af28640 1 -- 172.21.15.3:0/1144936779 shutdown_connections 2024-09-21T06:15:18.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.269+0000 7f235af28640 1 -- 172.21.15.3:0/1144936779 wait complete. 2024-09-21T06:15:18.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.269+0000 7f235af28640 1 Processor -- start 2024-09-21T06:15:18.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f235af28640 1 -- start start 2024-09-21T06:15:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f235af28640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23541029b0 0x7f23540ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f235af28640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23541082e0 0x7f23540ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f235af28640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2354108c30 0x7f23540fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f235af28640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f23541128f0 con 0x7f23541082e0 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f235af28640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2354112770 con 0x7f23541029b0 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f235af28640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2354112a70 con 0x7f2354108c30 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f2353fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23541082e0 0x7f23540ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f2353fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23541082e0 0x7f23540ffa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48264/0 (socket says 172.21.15.3:48264) 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f2353fff640 1 -- 172.21.15.3:0/2696394686 learned_addr learned my addr 172.21.15.3:0/2696394686 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f2353fff640 1 -- 172.21.15.3:0/2696394686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2354108c30 msgr2=0x7f23540fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f2353fff640 1 --2- 172.21.15.3:0/2696394686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2354108c30 0x7f23540fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f2353fff640 1 -- 172.21.15.3:0/2696394686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23541029b0 msgr2=0x7f23540ff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f2353fff640 1 --2- 172.21.15.3:0/2696394686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23541029b0 0x7f23540ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f2353fff640 1 -- 172.21.15.3:0/2696394686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23541a86b0 con 0x7f23541082e0 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.270+0000 7f2353fff640 1 --2- 172.21.15.3:0/2696394686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23541082e0 0x7f23540ffa90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f234000e9c0 tx=0x7f234000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.271+0000 7f2351ffb640 1 -- 172.21.15.3:0/2696394686 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234000ccc0 con 0x7f23541082e0 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.271+0000 7f2351ffb640 1 -- 172.21.15.3:0/2696394686 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23400054d0 con 0x7f23541082e0 2024-09-21T06:15:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.271+0000 7f235af28640 1 -- 172.21.15.3:0/2696394686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23541a89a0 con 0x7f23541082e0 2024-09-21T06:15:18.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.271+0000 7f2351ffb640 1 -- 172.21.15.3:0/2696394686 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340010430 con 0x7f23541082e0 2024-09-21T06:15:18.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.271+0000 7f235af28640 1 -- 172.21.15.3:0/2696394686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23541a8ea0 con 0x7f23541082e0 2024-09-21T06:15:18.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.272+0000 7f235af28640 1 -- 172.21.15.3:0/2696394686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2354102dc0 con 0x7f23541082e0 2024-09-21T06:15:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.274+0000 7f2351ffb640 1 -- 172.21.15.3:0/2696394686 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 154579+0+0 (secure 0 0 0) 0x7f2340005000 con 0x7f23541082e0 2024-09-21T06:15:18.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.275+0000 7f2351ffb640 1 --2- 172.21.15.3:0/2696394686 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f23440b6620 0x7f23440b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.275+0000 7f2351ffb640 1 -- 172.21.15.3:0/2696394686 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f23400e7ca0 con 0x7f23541082e0 2024-09-21T06:15:18.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.275+0000 7f2358cc4640 1 --2- 172.21.15.3:0/2696394686 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f23440b6620 0x7f23440b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:18.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.276+0000 7f2358cc4640 1 --2- 172.21.15.3:0/2696394686 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f23440b6620 0x7f23440b8b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f23480058c0 tx=0x7f2348005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.278+0000 7f2351ffb640 1 -- 172.21.15.3:0/2696394686 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2340056710 con 0x7f23541082e0 2024-09-21T06:15:18.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.375+0000 7f2351ffb640 1 -- 172.21.15.3:0/2696394686 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 25) ==== 154636+0+0 (secure 0 0 0) 0x7f23400a9e00 con 0x7f23541082e0 2024-09-21T06:15:18.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.495+0000 7f235af28640 1 -- 172.21.15.3:0/2696394686 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2354101640 con 0x7f23440b6620 2024-09-21T06:15:18.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.502+0000 7f2351ffb640 1 -- 172.21.15.3:0/2696394686 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f2354101640 con 0x7f23440b6620 2024-09-21T06:15:18.504 INFO:teuthology.orchestra.run.smithi003.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T06:15:17", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-21T06:15:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.505+0000 7f235af28640 1 -- 172.21.15.3:0/2696394686 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f23440b6620 msgr2=0x7f23440b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.505+0000 7f235af28640 1 --2- 172.21.15.3:0/2696394686 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f23440b6620 0x7f23440b8b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f23480058c0 tx=0x7f2348005850 comp rx=0 tx=0).stop 2024-09-21T06:15:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.505+0000 7f235af28640 1 -- 172.21.15.3:0/2696394686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23541082e0 msgr2=0x7f23540ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.505+0000 7f235af28640 1 --2- 172.21.15.3:0/2696394686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23541082e0 0x7f23540ffa90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f234000e9c0 tx=0x7f234000ee80 comp rx=0 tx=0).stop 2024-09-21T06:15:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.505+0000 7f235af28640 1 -- 172.21.15.3:0/2696394686 shutdown_connections 2024-09-21T06:15:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.505+0000 7f235af28640 1 --2- 172.21.15.3:0/2696394686 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f23440b6620 0x7f23440b8b30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.505+0000 7f235af28640 1 --2- 172.21.15.3:0/2696394686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2354108c30 0x7f23540fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.505+0000 7f235af28640 1 --2- 172.21.15.3:0/2696394686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23541082e0 0x7f23540ffa90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.505+0000 7f235af28640 1 --2- 172.21.15.3:0/2696394686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23541029b0 0x7f23540ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.505+0000 7f235af28640 1 -- 172.21.15.3:0/2696394686 >> 172.21.15.3:0/2696394686 conn(0x7f235407a740 msgr2=0x7f235406adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.505+0000 7f235af28640 1 -- 172.21.15.3:0/2696394686 shutdown_connections 2024-09-21T06:15:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.505+0000 7f235af28640 1 -- 172.21.15.3:0/2696394686 wait complete. 2024-09-21T06:15:18.520 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T06:15:18.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.634+0000 7ff3ba569640 1 -- 172.21.15.3:0/444001922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3b4103ad0 msgr2=0x7ff3b4103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.634+0000 7ff3ba569640 1 --2- 172.21.15.3:0/444001922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3b4103ad0 0x7ff3b4103ee0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009900 tx=0x7ff3a402a410 comp rx=0 tx=0).stop 2024-09-21T06:15:18.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.635+0000 7ff3ba569640 1 -- 172.21.15.3:0/444001922 shutdown_connections 2024-09-21T06:15:18.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.635+0000 7ff3ba569640 1 --2- 172.21.15.3:0/444001922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b4109d50 0x7ff3b4110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.635+0000 7ff3ba569640 1 --2- 172.21.15.3:0/444001922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b4109400 0x7ff3b4109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.635+0000 7ff3ba569640 1 --2- 172.21.15.3:0/444001922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3b4103ad0 0x7ff3b4103ee0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.635+0000 7ff3ba569640 1 -- 172.21.15.3:0/444001922 >> 172.21.15.3:0/444001922 conn(0x7ff3b40ff510 msgr2=0x7ff3b4101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:18.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.635+0000 7ff3ba569640 1 -- 172.21.15.3:0/444001922 shutdown_connections 2024-09-21T06:15:18.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.635+0000 7ff3ba569640 1 -- 172.21.15.3:0/444001922 wait complete. 2024-09-21T06:15:18.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.635+0000 7ff3ba569640 1 Processor -- start 2024-09-21T06:15:18.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.635+0000 7ff3ba569640 1 -- start start 2024-09-21T06:15:18.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3ba569640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b4103ad0 0x7ff3b419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3ba569640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3b4109400 0x7ff3b419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3ba569640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b4109d50 0x7ff3b419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3ba569640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b4113a10 con 0x7ff3b4109400 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3ba569640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b4113890 con 0x7ff3b4103ad0 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3ba569640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b4113b90 con 0x7ff3b4109d50 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3b37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3b4109400 0x7ff3b419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3b37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3b4109400 0x7ff3b419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48284/0 (socket says 172.21.15.3:48284) 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3b37fe640 1 -- 172.21.15.3:0/3376743438 learned_addr learned my addr 172.21.15.3:0/3376743438 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3b3fff640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b4103ad0 0x7ff3b419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3b8b06640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b4109d50 0x7ff3b419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3b37fe640 1 -- 172.21.15.3:0/3376743438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b4109d50 msgr2=0x7ff3b419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3b37fe640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b4109d50 0x7ff3b419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3b37fe640 1 -- 172.21.15.3:0/3376743438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b4103ad0 msgr2=0x7ff3b419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3b37fe640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b4103ad0 0x7ff3b419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.636+0000 7ff3b37fe640 1 -- 172.21.15.3:0/3376743438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b41a02f0 con 0x7ff3b4109400 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.637+0000 7ff3b37fe640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3b4109400 0x7ff3b419a9f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800c8c0 tx=0x7ff3a800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.637+0000 7ff3b17fa640 1 -- 172.21.15.3:0/3376743438 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8007bb0 con 0x7ff3b4109400 2024-09-21T06:15:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.637+0000 7ff3ba569640 1 -- 172.21.15.3:0/3376743438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b41a05e0 con 0x7ff3b4109400 2024-09-21T06:15:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.637+0000 7ff3b17fa640 1 -- 172.21.15.3:0/3376743438 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a8007d50 con 0x7ff3b4109400 2024-09-21T06:15:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.637+0000 7ff3b17fa640 1 -- 172.21.15.3:0/3376743438 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80187c0 con 0x7ff3b4109400 2024-09-21T06:15:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.637+0000 7ff3ba569640 1 -- 172.21.15.3:0/3376743438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b41a6e00 con 0x7ff3b4109400 2024-09-21T06:15:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.638+0000 7ff3ba569640 1 -- 172.21.15.3:0/3376743438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b4103ee0 con 0x7ff3b4109400 2024-09-21T06:15:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.640+0000 7ff3b17fa640 1 -- 172.21.15.3:0/3376743438 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 154636+0+0 (secure 0 0 0) 0x7ff3a800fa20 con 0x7ff3b4109400 2024-09-21T06:15:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.640+0000 7ff3b17fa640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff39c0b6680 0x7ff39c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.641+0000 7ff3b17fa640 1 -- 172.21.15.3:0/3376743438 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff3a80e7b60 con 0x7ff3b4109400 2024-09-21T06:15:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.641+0000 7ff3b3fff640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff39c0b6680 0x7ff39c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.641+0000 7ff3b3fff640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff39c0b6680 0x7ff39c0b8b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4031440 tx=0x7ff3a4005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.643+0000 7ff3b17fa640 1 -- 172.21.15.3:0/3376743438 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff3a8014030 con 0x7ff3b4109400 2024-09-21T06:15:18.833 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:15:18.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.856+0000 7ff3ba569640 1 -- 172.21.15.3:0/3376743438 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ff3b4063800 con 0x7ff39c0b6680 2024-09-21T06:15:18.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.865+0000 7ff3b17fa640 1 -- 172.21.15.3:0/3376743438 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ff3b4063800 con 0x7ff39c0b6680 2024-09-21T06:15:18.867 INFO:teuthology.orchestra.run.smithi003.stdout:/ 1m 6m3h 2024-09-21T06:15:18.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.867+0000 7ff3ba569640 1 -- 172.21.15.3:0/3376743438 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff39c0b6680 msgr2=0x7ff39c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.867+0000 7ff3ba569640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff39c0b6680 0x7ff39c0b8b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4031440 tx=0x7ff3a4005d50 comp rx=0 tx=0).stop 2024-09-21T06:15:18.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.867+0000 7ff3ba569640 1 -- 172.21.15.3:0/3376743438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3b4109400 msgr2=0x7ff3b419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:18.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.867+0000 7ff3ba569640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3b4109400 0x7ff3b419a9f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800c8c0 tx=0x7ff3a800cd80 comp rx=0 tx=0).stop 2024-09-21T06:15:18.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.868+0000 7ff3ba569640 1 -- 172.21.15.3:0/3376743438 shutdown_connections 2024-09-21T06:15:18.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.868+0000 7ff3ba569640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff39c0b6680 0x7ff39c0b8b90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.868+0000 7ff3ba569640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b4109d50 0x7ff3b419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.868+0000 7ff3ba569640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3b4109400 0x7ff3b419a9f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.868+0000 7ff3ba569640 1 --2- 172.21.15.3:0/3376743438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b4103ad0 0x7ff3b419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:18.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.868+0000 7ff3ba569640 1 -- 172.21.15.3:0/3376743438 >> 172.21.15.3:0/3376743438 conn(0x7ff3b40ff510 msgr2=0x7ff3b4101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:18.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.868+0000 7ff3ba569640 1 -- 172.21.15.3:0/3376743438 shutdown_connections 2024-09-21T06:15:18.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:18.868+0000 7ff3ba569640 1 -- 172.21.15.3:0/3376743438 wait complete. 2024-09-21T06:15:18.881 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-21T06:15:18.949 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-21T06:15:18.960 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-21T06:15:19.024 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-09-21T06:15:19.024 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-21T06:15:19.026 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T06:15:19.055 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:15:19.055 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T06:15:19.055 INFO:tasks.workunit:timeout=3h 2024-09-21T06:15:19.055 INFO:tasks.workunit:cleanup=True 2024-09-21T06:15:19.055 DEBUG:teuthology.orchestra.run.smithi003:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T06:15:19.072 INFO:teuthology.orchestra.run.smithi003.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T06:15:19.072 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T06:15:19.072 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-21T06:15:19.072 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:15:19.072 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T06:15:19.072 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-21 06:13:30.960106753 +0000 2024-09-21T06:15:19.072 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-21 06:13:30.960106753 +0000 2024-09-21T06:15:19.073 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-21 06:15:07.750979799 +0000 2024-09-21T06:15:19.073 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-21 06:13:30.960106753 +0000 2024-09-21T06:15:19.073 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T06:15:19.073 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T06:15:19.099 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:19.133 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:15:19.190 INFO:tasks.workunit.client.0.smithi003.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T06:15:19.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:19 smithi003 ceph-mon[31227]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:19.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:19 smithi003 ceph-mon[31227]: pgmap v334: 97 pgs: 97 active+clean; 658 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-21T06:15:19.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:19 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:15:19.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:19 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:19.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:19 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:15:19.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:19 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:19.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:19 smithi003 ceph-mon[31227]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:19.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:19 smithi003 ceph-mon[31227]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-09-21T06:15:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.595+0000 7f88576a6640 1 -- 172.21.15.3:0/2548081592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8850105380 msgr2=0x7f885010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.595+0000 7f88576a6640 1 --2- 172.21.15.3:0/2548081592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8850105380 0x7f885010d8d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8840009900 tx=0x7f884002a410 comp rx=0 tx=0).stop 2024-09-21T06:15:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.595+0000 7f88576a6640 1 -- 172.21.15.3:0/2548081592 shutdown_connections 2024-09-21T06:15:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.595+0000 7f88576a6640 1 --2- 172.21.15.3:0/2548081592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f885010de60 0x7f8850110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.595+0000 7f88576a6640 1 --2- 172.21.15.3:0/2548081592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8850105380 0x7f885010d8d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.595+0000 7f88576a6640 1 --2- 172.21.15.3:0/2548081592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8850104a30 0x7f8850104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.595+0000 7f88576a6640 1 -- 172.21.15.3:0/2548081592 >> 172.21.15.3:0/2548081592 conn(0x7f8850079760 msgr2=0x7f8850079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.595+0000 7f88576a6640 1 -- 172.21.15.3:0/2548081592 shutdown_connections 2024-09-21T06:15:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.595+0000 7f88576a6640 1 -- 172.21.15.3:0/2548081592 wait complete. 2024-09-21T06:15:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.596+0000 7f88576a6640 1 Processor -- start 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.596+0000 7f88576a6640 1 -- start start 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.596+0000 7f88576a6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8850104a30 0x7f885019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.596+0000 7f88576a6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8850105380 0x7f885019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.596+0000 7f88576a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f885010de60 0x7f885019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.596+0000 7f88576a6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8850113aa0 con 0x7f8850104a30 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.596+0000 7f88576a6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8850113920 con 0x7f885010de60 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.596+0000 7f88576a6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8850113c20 con 0x7f8850105380 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.596+0000 7f8855442640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8850104a30 0x7f885019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.596+0000 7f8855442640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8850104a30 0x7f885019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48314/0 (socket says 172.21.15.3:48314) 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.596+0000 7f8855442640 1 -- 172.21.15.3:0/2223885184 learned_addr learned my addr 172.21.15.3:0/2223885184 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.597+0000 7f8854c41640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8850105380 0x7f885019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.597+0000 7f8855c43640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f885010de60 0x7f885019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.597+0000 7f8855442640 1 -- 172.21.15.3:0/2223885184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8850105380 msgr2=0x7f885019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.597+0000 7f8855442640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8850105380 0x7f885019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.597+0000 7f8855442640 1 -- 172.21.15.3:0/2223885184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f885010de60 msgr2=0x7f885019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.597+0000 7f8855442640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f885010de60 0x7f885019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.597+0000 7f8855442640 1 -- 172.21.15.3:0/2223885184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88501a0380 con 0x7f8850104a30 2024-09-21T06:15:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.597+0000 7f8855442640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8850104a30 0x7f885019a4d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f884c00c970 tx=0x7f884c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.597+0000 7f88467fc640 1 -- 172.21.15.3:0/2223885184 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c004220 con 0x7f8850104a30 2024-09-21T06:15:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.597+0000 7f88467fc640 1 -- 172.21.15.3:0/2223885184 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f884c0043c0 con 0x7f8850104a30 2024-09-21T06:15:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.597+0000 7f88576a6640 1 -- 172.21.15.3:0/2223885184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88501a0670 con 0x7f8850104a30 2024-09-21T06:15:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.597+0000 7f88467fc640 1 -- 172.21.15.3:0/2223885184 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c004a00 con 0x7f8850104a30 2024-09-21T06:15:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.597+0000 7f88576a6640 1 -- 172.21.15.3:0/2223885184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88501a6e90 con 0x7f8850104a30 2024-09-21T06:15:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.600+0000 7f885dc1a740 1 Processor -- start 2024-09-21T06:15:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.600+0000 7f885dc1a740 1 -- start start 2024-09-21T06:15:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.600+0000 7f885dc1a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf167f070 0x555bf1a26270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.600+0000 7f885dc1a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 0x555bf1a267b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.600+0000 7f885dc1a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 0x555bf1a26cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.600+0000 7f88467fc640 1 -- 172.21.15.3:0/2223885184 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 154636+0+0 (secure 0 0 0) 0x7f884c0079b0 con 0x7f8850104a30 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.600+0000 7f885dc1a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555bf199ce50 con 0x555bf16561b0 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.600+0000 7f885dc1a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555bf198c280 con 0x555bf165e390 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.600+0000 7f885dc1a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555bf1990090 con 0x555bf167f070 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f8855442640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 0x555bf1a267b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f8854c41640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 0x555bf1a26cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f8855442640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 0x555bf1a267b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48326/0 (socket says 172.21.15.3:48326) 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f8855442640 1 -- 172.21.15.3:0/2638633190 learned_addr learned my addr 172.21.15.3:0/2638633190 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f8854c41640 1 --2- 172.21.15.3:0/2638633190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 0x555bf1a26cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:57804/0 (socket says 172.21.15.3:57804) 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f8855c43640 1 --2- 172.21.15.3:0/2638633190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf167f070 0x555bf1a26270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f8855442640 1 -- 172.21.15.3:0/2638633190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf167f070 msgr2=0x555bf1a26270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f8855442640 1 --2- 172.21.15.3:0/2638633190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf167f070 0x555bf1a26270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f8855442640 1 -- 172.21.15.3:0/2638633190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 msgr2=0x555bf1a26cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f8855442640 1 --2- 172.21.15.3:0/2638633190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 0x555bf1a26cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f8855442640 1 -- 172.21.15.3:0/2638633190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf16ea650 con 0x555bf16561b0 2024-09-21T06:15:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f8855442640 1 --2- 172.21.15.3:0/2638633190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 0x555bf1a267b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f884c0e4bb0 tx=0x7f884c0edb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2881db1028a32f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f88467fc640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f883c0b6730 0x7f883c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.601+0000 7f883b7fe640 1 -- 172.21.15.3:0/2638633190 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c0fa070 con 0x555bf16561b0 2024-09-21T06:15:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.602+0000 7f883b7fe640 1 -- 172.21.15.3:0/2638633190 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f884c0ace50 con 0x555bf16561b0 2024-09-21T06:15:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.602+0000 7f883b7fe640 1 -- 172.21.15.3:0/2638633190 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c0acab0 con 0x555bf16561b0 2024-09-21T06:15:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.602+0000 7f8855c43640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f883c0b6730 0x7f883c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.602+0000 7f88467fc640 1 -- 172.21.15.3:0/2223885184 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f884c0e74b0 con 0x7f8850104a30 2024-09-21T06:15:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.603+0000 7f885dc1a740 1 -- 172.21.15.3:0/2638633190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 msgr2=0x555bf1a267b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.603+0000 7f8855c43640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f883c0b6730 0x7f883c0b8c40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8848007ed0 tx=0x7f8848007680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.603+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2638633190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 0x555bf1a267b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f884c0e4bb0 tx=0x7f884c0edb20 comp rx=0 tx=0).stop 2024-09-21T06:15:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.603+0000 7f885dc1a740 1 -- 172.21.15.3:0/2638633190 shutdown_connections 2024-09-21T06:15:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.603+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2638633190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 0x555bf1a26cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.603+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2638633190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 0x555bf1a267b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.603+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2638633190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf167f070 0x555bf1a26270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.603+0000 7f885dc1a740 1 -- 172.21.15.3:0/2638633190 >> 172.21.15.3:0/2638633190 conn(0x555bf1673460 msgr2=0x555bf1a24160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.603+0000 7f885dc1a740 1 -- 172.21.15.3:0/2638633190 shutdown_connections 2024-09-21T06:15:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.603+0000 7f885dc1a740 1 -- 172.21.15.3:0/2638633190 wait complete. 2024-09-21T06:15:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.603+0000 7f885dc1a740 1 -- start start 2024-09-21T06:15:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.603+0000 7f885dc1a740 1 Processor -- start 2024-09-21T06:15:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f885dc1a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 0x555bf1a35d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f885dc1a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf165e390 0x555bf1a36290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f885dc1a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf167f070 0x555bf1a420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f885dc1a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555bf17ed700 con 0x555bf16561b0 2024-09-21T06:15:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f885dc1a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555bf1927110 con 0x555bf167f070 2024-09-21T06:15:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f885dc1a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555bf1a38db0 con 0x555bf165e390 2024-09-21T06:15:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f885dc1a740 10 client.0 _ll_register_callbacks cb 0x555bf1940990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f8855442640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 0x555bf1a35d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f8855442640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 0x555bf1a35d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48340/0 (socket says 172.21.15.3:48340) 2024-09-21T06:15:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f8855442640 1 -- 172.21.15.3:0/2471380871 learned_addr learned my addr 172.21.15.3:0/2471380871 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f8854c41640 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf165e390 0x555bf1a36290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f8855442640 1 -- 172.21.15.3:0/2471380871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf165e390 msgr2=0x555bf1a36290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f8855c43640 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf167f070 0x555bf1a420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f8855442640 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf165e390 0x555bf1a36290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f8855442640 1 -- 172.21.15.3:0/2471380871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf167f070 msgr2=0x555bf1a420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f8855442640 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf167f070 0x555bf1a420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f8855442640 1 -- 172.21.15.3:0/2471380871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf1954c40 con 0x555bf16561b0 2024-09-21T06:15:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f8854c41640 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf165e390 0x555bf1a36290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.604+0000 7f8855c43640 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf167f070 0x555bf1a420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f8855442640 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 0x555bf1a35d50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f884c0aa5b0 tx=0x7f884c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c144328043916fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f8838ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f8838ff9640 1 -- 172.21.15.3:0/2471380871 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c0c8f20 con 0x555bf16561b0 2024-09-21T06:15:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f8838ff9640 1 -- 172.21.15.3:0/2471380871 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f884c0c80e0 con 0x555bf16561b0 2024-09-21T06:15:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f885dc1a740 1 -- 172.21.15.3:0/2471380871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f884c0ace50 con 0x555bf16561b0 2024-09-21T06:15:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f8838ff9640 1 -- 172.21.15.3:0/2471380871 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f884c0c9110 con 0x555bf16561b0 2024-09-21T06:15:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f885dc1a740 10 client.15597 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f885dc1a740 1 -- 172.21.15.3:0/2471380871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555bf1a44ae0 con 0x555bf16561b0 2024-09-21T06:15:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f8838ff9640 1 -- 172.21.15.3:0/2471380871 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f884c0ace50 con 0x555bf16561b0 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f8838ff9640 1 -- 172.21.15.3:0/2471380871 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f884c0a95b0 con 0x555bf16561b0 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f885dc1a740 10 client.15597 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f885dc1a740 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f885dc1a740 20 client.15597 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.605+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555bf163d9f0 0x555bf1a47e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.606+0000 7f885dc1a740 4 client.15597 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.606+0000 7f885dc1a740 1 -- 172.21.15.3:0/2471380871 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bf199eda0 con 0x555bf163d9f0 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.606+0000 7f8854c41640 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555bf163d9f0 0x555bf1a47e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.606+0000 7f8838ff9640 10 client.15597 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.606+0000 7f8854c41640 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555bf163d9f0 0x555bf1a47e40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d53bbf0f360d9432 server_cookie=f716a08100b037e0 in_seq=0 out_seq=0 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.607+0000 7f8838ff9640 1 -- 172.21.15.3:0/2471380871 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555bf199eda0 con 0x555bf163d9f0 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.607+0000 7f8838ff9640 10 client.15597 handle_command_reply: tid=0 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.607+0000 7f885dc1a740 1 client.15597 shutdown 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.608+0000 7f885dc1a740 1 -- 172.21.15.3:0/2471380871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 msgr2=0x555bf1a35d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.608+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 0x555bf1a35d50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f884c0aa5b0 tx=0x7f884c0046c0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.608+0000 7f885dc1a740 1 -- 172.21.15.3:0/2471380871 shutdown_connections 2024-09-21T06:15:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.608+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555bf163d9f0 0x555bf1a47e40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.608+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf165e390 0x555bf1a36290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.608+0000 7f8838ff9640 0 client.15597 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.608+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf167f070 0x555bf1a420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.608+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2471380871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf16561b0 0x555bf1a35d50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.608+0000 7f885dc1a740 1 -- 172.21.15.3:0/2471380871 >> 172.21.15.3:0/2471380871 conn(0x555bf1673460 msgr2=0x555bf1a22700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.608+0000 7f885dc1a740 1 -- 172.21.15.3:0/2471380871 shutdown_connections 2024-09-21T06:15:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.608+0000 7f885dc1a740 1 -- 172.21.15.3:0/2471380871 wait complete. 2024-09-21T06:15:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.608+0000 7f885dc1a740 20 client.15597 trim_cache size 0 max 16384 2024-09-21T06:15:19.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.625+0000 7f885dc1a740 1 Processor -- start 2024-09-21T06:15:19.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.625+0000 7f885dc1a740 1 -- start start 2024-09-21T06:15:19.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f885dc1a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 0x555bf1a27c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f885dc1a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf16561b0 0x555bf1a42050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f885dc1a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 0x555bf1a37980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f885dc1a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f88501ad490 con 0x555bf163d9f0 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f885dc1a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88501aefd0 con 0x555bf165e390 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f885dc1a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88501adb60 con 0x555bf16561b0 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f8855442640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 0x555bf1a27c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f8855442640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 0x555bf1a27c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48350/0 (socket says 172.21.15.3:48350) 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f8855442640 1 -- 172.21.15.3:0/4266917529 learned_addr learned my addr 172.21.15.3:0/4266917529 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f8854c41640 1 --2- 172.21.15.3:0/4266917529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf16561b0 0x555bf1a42050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f8855c43640 1 --2- 172.21.15.3:0/4266917529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 0x555bf1a37980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f8855442640 1 -- 172.21.15.3:0/4266917529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf16561b0 msgr2=0x555bf1a42050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f8855442640 1 --2- 172.21.15.3:0/4266917529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf16561b0 0x555bf1a42050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f8855442640 1 -- 172.21.15.3:0/4266917529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 msgr2=0x555bf1a37980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f8855442640 1 --2- 172.21.15.3:0/4266917529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 0x555bf1a37980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f8855442640 1 -- 172.21.15.3:0/4266917529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf1a28560 con 0x555bf163d9f0 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f8855442640 1 --2- 172.21.15.3:0/4266917529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 0x555bf1a27c60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f884c0ad2d0 tx=0x7f884c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bbfe7c3b91ffe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.626+0000 7f883affd640 1 -- 172.21.15.3:0/4266917529 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c01d030 con 0x555bf163d9f0 2024-09-21T06:15:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.627+0000 7f883affd640 1 -- 172.21.15.3:0/4266917529 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f884c0ace50 con 0x555bf163d9f0 2024-09-21T06:15:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.627+0000 7f883affd640 1 -- 172.21.15.3:0/4266917529 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c0c6990 con 0x555bf163d9f0 2024-09-21T06:15:19.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.627+0000 7f885dc1a740 1 -- 172.21.15.3:0/4266917529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 msgr2=0x555bf1a27c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:19.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.627+0000 7f885dc1a740 1 --2- 172.21.15.3:0/4266917529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 0x555bf1a27c60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f884c0ad2d0 tx=0x7f884c004730 comp rx=0 tx=0).stop 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.627+0000 7f885dc1a740 1 -- 172.21.15.3:0/4266917529 shutdown_connections 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.627+0000 7f885dc1a740 1 --2- 172.21.15.3:0/4266917529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf16561b0 0x555bf1a42050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.627+0000 7f885dc1a740 1 --2- 172.21.15.3:0/4266917529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 0x555bf1a37980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.627+0000 7f885dc1a740 1 --2- 172.21.15.3:0/4266917529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 0x555bf1a27c60 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.627+0000 7f885dc1a740 1 -- 172.21.15.3:0/4266917529 >> 172.21.15.3:0/4266917529 conn(0x555bf1673460 msgr2=0x555bf1a37270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.627+0000 7f885dc1a740 1 -- 172.21.15.3:0/4266917529 shutdown_connections 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.627+0000 7f885dc1a740 1 -- 172.21.15.3:0/4266917529 wait complete. 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.628+0000 7f885dc1a740 1 -- start start 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.628+0000 7f885dc1a740 1 Processor -- start 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.628+0000 7f885dc1a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 0x555bf1a48dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.628+0000 7f885dc1a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf16561b0 0x555bf1a49300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.628+0000 7f885dc1a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 0x555bf1a49840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.628+0000 7f885dc1a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f88501ae900 con 0x555bf163d9f0 2024-09-21T06:15:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.628+0000 7f885dc1a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88501ae230 con 0x555bf165e390 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.628+0000 7f885dc1a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88501ab470 con 0x555bf16561b0 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.629+0000 7f885dc1a740 10 client.0 _ll_register_callbacks cb 0x7f884c0f8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.629+0000 7f8855442640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 0x555bf1a48dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.629+0000 7f8854c41640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf16561b0 0x555bf1a49300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.629+0000 7f8855442640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 0x555bf1a48dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48356/0 (socket says 172.21.15.3:48356) 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.629+0000 7f8855442640 1 -- 172.21.15.3:0/2603643896 learned_addr learned my addr 172.21.15.3:0/2603643896 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.629+0000 7f8855c43640 1 --2- 172.21.15.3:0/2603643896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 0x555bf1a49840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.629+0000 7f8855442640 1 -- 172.21.15.3:0/2603643896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf16561b0 msgr2=0x555bf1a49300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.629+0000 7f8855442640 1 --2- 172.21.15.3:0/2603643896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf16561b0 0x555bf1a49300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.629+0000 7f8855442640 1 -- 172.21.15.3:0/2603643896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 msgr2=0x555bf1a49840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.629+0000 7f8855442640 1 --2- 172.21.15.3:0/2603643896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 0x555bf1a49840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.629+0000 7f8855442640 1 -- 172.21.15.3:0/2603643896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf1a1f580 con 0x555bf163d9f0 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.629+0000 7f8855442640 1 --2- 172.21.15.3:0/2603643896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 0x555bf1a48dc0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f884c0ad130 tx=0x7f884c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4ab15cda5b32ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.629+0000 7f8838ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f8838ff9640 1 -- 172.21.15.3:0/2603643896 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c01d030 con 0x555bf163d9f0 2024-09-21T06:15:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f8838ff9640 1 -- 172.21.15.3:0/2603643896 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f884c0b68d0 con 0x555bf163d9f0 2024-09-21T06:15:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f885dc1a740 1 -- 172.21.15.3:0/2603643896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f884c0ace50 con 0x555bf163d9f0 2024-09-21T06:15:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f8838ff9640 1 -- 172.21.15.3:0/2603643896 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f884c0c6bd0 con 0x555bf163d9f0 2024-09-21T06:15:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f885dc1a740 10 client.15603 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f885dc1a740 1 -- 172.21.15.3:0/2603643896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555bf1a1f580 con 0x555bf163d9f0 2024-09-21T06:15:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f8838ff9640 1 -- 172.21.15.3:0/2603643896 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f884c0ace50 con 0x555bf163d9f0 2024-09-21T06:15:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f8838ff9640 1 -- 172.21.15.3:0/2603643896 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f884c0e7040 con 0x555bf163d9f0 2024-09-21T06:15:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f885dc1a740 10 client.15603 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f885dc1a740 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f885dc1a740 20 client.15603 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2603643896 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555bf167f070 0x555bf1ab48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f885dc1a740 4 client.15603 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:15:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.630+0000 7f885dc1a740 1 -- 172.21.15.3:0/2603643896 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555bf1a36640 con 0x555bf167f070 2024-09-21T06:15:19.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.631+0000 7f8854c41640 1 --2- 172.21.15.3:0/2603643896 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555bf167f070 0x555bf1ab48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:19.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.631+0000 7f8854c41640 1 --2- 172.21.15.3:0/2603643896 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555bf167f070 0x555bf1ab48a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c1780c085af0dcb server_cookie=1da027fb84c1a13d in_seq=0 out_seq=0 2024-09-21T06:15:19.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.631+0000 7f8838ff9640 10 client.15603 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.631+0000 7f8838ff9640 1 -- 172.21.15.3:0/2603643896 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x555bf1a36640 con 0x555bf167f070 2024-09-21T06:15:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.631+0000 7f8838ff9640 10 client.15603 handle_command_reply: tid=0 2024-09-21T06:15:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.631+0000 7f885dc1a740 1 client.15603 shutdown 2024-09-21T06:15:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.631+0000 7f885dc1a740 1 -- 172.21.15.3:0/2603643896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 msgr2=0x555bf1a48dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.631+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2603643896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 0x555bf1a48dc0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f884c0ad130 tx=0x7f884c00cf60 comp rx=0 tx=0).stop 2024-09-21T06:15:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.631+0000 7f885dc1a740 1 -- 172.21.15.3:0/2603643896 shutdown_connections 2024-09-21T06:15:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.631+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2603643896 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555bf167f070 0x555bf1ab48a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.631+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2603643896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf16561b0 0x555bf1a49300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2603643896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf165e390 0x555bf1a49840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f885dc1a740 1 --2- 172.21.15.3:0/2603643896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555bf163d9f0 0x555bf1a48dc0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f885dc1a740 1 -- 172.21.15.3:0/2603643896 >> 172.21.15.3:0/2603643896 conn(0x555bf1673460 msgr2=0x555bf1a27a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f8838ff9640 0 client.15603 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f885dc1a740 1 -- 172.21.15.3:0/2603643896 shutdown_connections 2024-09-21T06:15:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f885dc1a740 1 -- 172.21.15.3:0/2603643896 wait complete. 2024-09-21T06:15:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f885dc1a740 20 client.15603 trim_cache size 0 max 16384 2024-09-21T06:15:19.638 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:15:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:15:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-21T06:15:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:15:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:15:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "4957cde1-7a9e-430b-a809-39863daca10a", 2024-09-21T06:15:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-21T06:15:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:15:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:15:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:15:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:15:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:15:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:15:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:15:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:15:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:15:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:15:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:15:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:15:19.639 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:15:19.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f88576a6640 1 -- 172.21.15.3:0/2223885184 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f883c0b6730 msgr2=0x7f883c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:19.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f88576a6640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f883c0b6730 0x7f883c0b8c40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8848007ed0 tx=0x7f8848007680 comp rx=0 tx=0).stop 2024-09-21T06:15:19.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f88576a6640 1 -- 172.21.15.3:0/2223885184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8850104a30 msgr2=0x7f885019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:19.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f88576a6640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8850104a30 0x7f885019a4d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f884c00c970 tx=0x7f884c00ce30 comp rx=0 tx=0).stop 2024-09-21T06:15:19.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f88576a6640 1 -- 172.21.15.3:0/2223885184 shutdown_connections 2024-09-21T06:15:19.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f88576a6640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f883c0b6730 0x7f883c0b8c40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f88576a6640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8850105380 0x7f885019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f88576a6640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f885010de60 0x7f885019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f88576a6640 1 --2- 172.21.15.3:0/2223885184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8850104a30 0x7f885019a4d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:19.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.632+0000 7f88576a6640 1 -- 172.21.15.3:0/2223885184 >> 172.21.15.3:0/2223885184 conn(0x7f8850079760 msgr2=0x7f88501008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:19.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.633+0000 7f88576a6640 1 -- 172.21.15.3:0/2223885184 shutdown_connections 2024-09-21T06:15:19.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:19.633+0000 7f88576a6640 1 -- 172.21.15.3:0/2223885184 wait complete. 2024-09-21T06:15:19.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:19 smithi137 ceph-mon[34464]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:19.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:19 smithi137 ceph-mon[34464]: pgmap v334: 97 pgs: 97 active+clean; 658 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-21T06:15:19.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:19 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:15:19.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:19 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:19.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:19 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:15:19.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:19 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:19.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:19 smithi137 ceph-mon[34464]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:19.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:19 smithi137 ceph-mon[34464]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-09-21T06:15:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:19 smithi204 ceph-mon[34659]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:19 smithi204 ceph-mon[34659]: pgmap v334: 97 pgs: 97 active+clean; 658 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-21T06:15:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:19 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:15:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:19 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:19 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:15:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:19 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:19 smithi204 ceph-mon[34659]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:19 smithi204 ceph-mon[34659]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-09-21T06:15:20.319 INFO:tasks.cephfs.filesystem:scrub status for tag:4957cde1-7a9e-430b-a809-39863daca10a - {'path': '/', 'tag': '4957cde1-7a9e-430b-a809-39863daca10a', 'options': 'recursive,force'} 2024-09-21T06:15:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:20 smithi003 ceph-mon[31227]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:20 smithi003 ceph-mon[31227]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:20.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:20 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:15:20.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:20 smithi137 ceph-mon[34464]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:20.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:20 smithi137 ceph-mon[34464]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:20.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:20 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:15:20.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:20 smithi204 ceph-mon[34659]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:20.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:20 smithi204 ceph-mon[34659]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:15:20.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:20 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:15:21.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:21 smithi137 ceph-mon[34464]: pgmap v335: 97 pgs: 97 active+clean; 1.0 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 75 KiB/s wr, 20 op/s 2024-09-21T06:15:21.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:21 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:21.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:21 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:21 smithi204 ceph-mon[34659]: pgmap v335: 97 pgs: 97 active+clean; 1.0 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 75 KiB/s wr, 20 op/s 2024-09-21T06:15:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:21 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:21 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:21 smithi003 ceph-mon[31227]: pgmap v335: 97 pgs: 97 active+clean; 1.0 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 75 KiB/s wr, 20 op/s 2024-09-21T06:15:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:21 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:21 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:22 smithi003 ceph-mon[31227]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 176 KiB/s wr, 59 op/s 2024-09-21T06:15:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:15:22.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:15:22.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:22 smithi137 ceph-mon[34464]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 176 KiB/s wr, 59 op/s 2024-09-21T06:15:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:15:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:15:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:22 smithi204 ceph-mon[34659]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 176 KiB/s wr, 59 op/s 2024-09-21T06:15:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:15:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:15:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:15:25.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:24 smithi003 ceph-mon[31227]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T06:15:25.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:24 smithi137 ceph-mon[34464]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T06:15:25.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:24 smithi204 ceph-mon[34659]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T06:15:27.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:26 smithi003 ceph-mon[31227]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T06:15:27.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:26 smithi137 ceph-mon[34464]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T06:15:27.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:26 smithi204 ceph-mon[34659]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T06:15:29.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:28 smithi003 ceph-mon[31227]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-21T06:15:29.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:28 smithi137 ceph-mon[34464]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-21T06:15:29.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:28 smithi204 ceph-mon[34659]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-21T06:15:31.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:30 smithi003 ceph-mon[31227]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T06:15:31.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:30 smithi137 ceph-mon[34464]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T06:15:31.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:30 smithi204 ceph-mon[34659]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T06:15:32.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:31 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:15:32.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:31 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:15:32.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:31 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:15:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.186+0000 7f2e4a84d640 1 -- 172.21.15.3:0/993071248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e44108c30 msgr2=0x7f2e4410f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.186+0000 7f2e4a84d640 1 --2- 172.21.15.3:0/993071248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e44108c30 0x7f2e4410f460 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009ca0 tx=0x7f2e3c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.186+0000 7f2e4a84d640 1 -- 172.21.15.3:0/993071248 shutdown_connections 2024-09-21T06:15:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.186+0000 7f2e4a84d640 1 --2- 172.21.15.3:0/993071248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e441082e0 0x7f2e441086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.186+0000 7f2e4a84d640 1 --2- 172.21.15.3:0/993071248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e44108c30 0x7f2e4410f460 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.186+0000 7f2e4a84d640 1 --2- 172.21.15.3:0/993071248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e441029b0 0x7f2e44102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.186+0000 7f2e4a84d640 1 -- 172.21.15.3:0/993071248 >> 172.21.15.3:0/993071248 conn(0x7f2e4407a740 msgr2=0x7f2e4407ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.186+0000 7f2e4a84d640 1 -- 172.21.15.3:0/993071248 shutdown_connections 2024-09-21T06:15:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.186+0000 7f2e4a84d640 1 -- 172.21.15.3:0/993071248 wait complete. 2024-09-21T06:15:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.187+0000 7f2e4a84d640 1 Processor -- start 2024-09-21T06:15:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.187+0000 7f2e4a84d640 1 -- start start 2024-09-21T06:15:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.187+0000 7f2e4a84d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e441029b0 0x7f2e4419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.187+0000 7f2e4a84d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e441082e0 0x7f2e4419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.187+0000 7f2e4a84d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e44108c30 0x7f2e4419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.187+0000 7f2e4a84d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441128f0 con 0x7f2e441029b0 2024-09-21T06:15:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.187+0000 7f2e4a84d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44112770 con 0x7f2e441082e0 2024-09-21T06:15:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.187+0000 7f2e4a84d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44112a70 con 0x7f2e44108c30 2024-09-21T06:15:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.187+0000 7f2e43fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e441029b0 0x7f2e4419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.187+0000 7f2e43fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e441029b0 0x7f2e4419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58540/0 (socket says 172.21.15.3:58540) 2024-09-21T06:15:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.187+0000 7f2e43fff640 1 -- 172.21.15.3:0/770746104 learned_addr learned my addr 172.21.15.3:0/770746104 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.188+0000 7f2e437fe640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e441082e0 0x7f2e4419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.188+0000 7f2e48dea640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e44108c30 0x7f2e4419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.188+0000 7f2e43fff640 1 -- 172.21.15.3:0/770746104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e44108c30 msgr2=0x7f2e4419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.188+0000 7f2e43fff640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e44108c30 0x7f2e4419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.188+0000 7f2e43fff640 1 -- 172.21.15.3:0/770746104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e441082e0 msgr2=0x7f2e4419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.188+0000 7f2e43fff640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e441082e0 0x7f2e4419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.188+0000 7f2e43fff640 1 -- 172.21.15.3:0/770746104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e441a0370 con 0x7f2e441029b0 2024-09-21T06:15:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.188+0000 7f2e43fff640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e441029b0 0x7f2e4419a440 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400e9f0 tx=0x7f2e3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.188+0000 7f2e417fa640 1 -- 172.21.15.3:0/770746104 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400cd80 con 0x7f2e441029b0 2024-09-21T06:15:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.188+0000 7f2e417fa640 1 -- 172.21.15.3:0/770746104 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e340108e0 con 0x7f2e441029b0 2024-09-21T06:15:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.188+0000 7f2e4a84d640 1 -- 172.21.15.3:0/770746104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e441a0660 con 0x7f2e441029b0 2024-09-21T06:15:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.188+0000 7f2e417fa640 1 -- 172.21.15.3:0/770746104 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34018980 con 0x7f2e441029b0 2024-09-21T06:15:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.188+0000 7f2e4a84d640 1 -- 172.21.15.3:0/770746104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441a6df0 con 0x7f2e441029b0 2024-09-21T06:15:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.191+0000 7f2e517bd740 1 Processor -- start 2024-09-21T06:15:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.191+0000 7f2e517bd740 1 -- start start 2024-09-21T06:15:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.191+0000 7f2e417fa640 1 -- 172.21.15.3:0/770746104 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 154636+0+0 (secure 0 0 0) 0x7f2e34010430 con 0x7f2e441029b0 2024-09-21T06:15:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.191+0000 7f2e517bd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76881b0 0x5603b7a2d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.191+0000 7f2e517bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a2def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.191+0000 7f2e517bd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76674d0 0x5603b7a2e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.191+0000 7f2e517bd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5603b79a26f0 con 0x5603b76674d0 2024-09-21T06:15:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.191+0000 7f2e517bd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5603b7991b20 con 0x5603b76881b0 2024-09-21T06:15:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.191+0000 7f2e517bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5603b7995930 con 0x5603b765f2f0 2024-09-21T06:15:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.191+0000 7f2e437fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76674d0 0x5603b7a2e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.191+0000 7f2e437fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76674d0 0x5603b7a2e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58556/0 (socket says 172.21.15.3:58556) 2024-09-21T06:15:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.191+0000 7f2e437fe640 1 -- 172.21.15.3:0/3103691301 learned_addr learned my addr 172.21.15.3:0/3103691301 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.191+0000 7f2e48dea640 1 --2- 172.21.15.3:0/3103691301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76881b0 0x5603b7a2d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e43fff640 1 --2- 172.21.15.3:0/3103691301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a2def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e437fe640 1 -- 172.21.15.3:0/3103691301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 msgr2=0x5603b7a2def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e437fe640 1 --2- 172.21.15.3:0/3103691301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a2def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e437fe640 1 -- 172.21.15.3:0/3103691301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76881b0 msgr2=0x5603b7a2d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e437fe640 1 --2- 172.21.15.3:0/3103691301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76881b0 0x5603b7a2d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e437fe640 1 -- 172.21.15.3:0/3103691301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603b7957530 con 0x5603b76674d0 2024-09-21T06:15:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e43fff640 1 --2- 172.21.15.3:0/3103691301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a2def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e437fe640 1 --2- 172.21.15.3:0/3103691301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76674d0 0x5603b7a2e430 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800bba0 tx=0x7f2e3800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0c39c7511b20d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e417fa640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e2c0b6680 0x7f2e2c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e2a7fc640 1 -- 172.21.15.3:0/3103691301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38015070 con 0x5603b76674d0 2024-09-21T06:15:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e48dea640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e2c0b6680 0x7f2e2c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e2a7fc640 1 -- 172.21.15.3:0/3103691301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e3801a910 con 0x5603b76674d0 2024-09-21T06:15:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e2a7fc640 1 -- 172.21.15.3:0/3103691301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38011900 con 0x5603b76674d0 2024-09-21T06:15:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.192+0000 7f2e417fa640 1 -- 172.21.15.3:0/770746104 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2e340e7b60 con 0x7f2e441029b0 2024-09-21T06:15:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.193+0000 7f2e517bd740 1 -- 172.21.15.3:0/3103691301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76674d0 msgr2=0x5603b7a2e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.193+0000 7f2e48dea640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e2c0b6680 0x7f2e2c0b8b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c006010 tx=0x7f2e3c009440 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.193+0000 7f2e517bd740 1 --2- 172.21.15.3:0/3103691301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76674d0 0x5603b7a2e430 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800bba0 tx=0x7f2e3800f730 comp rx=0 tx=0).stop 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.193+0000 7f2e517bd740 1 -- 172.21.15.3:0/3103691301 shutdown_connections 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.193+0000 7f2e517bd740 1 --2- 172.21.15.3:0/3103691301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a2def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.193+0000 7f2e517bd740 1 --2- 172.21.15.3:0/3103691301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76674d0 0x5603b7a2e430 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.193+0000 7f2e517bd740 1 --2- 172.21.15.3:0/3103691301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76881b0 0x5603b7a2d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.193+0000 7f2e517bd740 1 -- 172.21.15.3:0/3103691301 >> 172.21.15.3:0/3103691301 conn(0x5603b767c300 msgr2=0x5603b7a2b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.193+0000 7f2e517bd740 1 -- 172.21.15.3:0/3103691301 shutdown_connections 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.193+0000 7f2e517bd740 1 -- 172.21.15.3:0/3103691301 wait complete. 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.194+0000 7f2e517bd740 1 -- start start 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.194+0000 7f2e517bd740 1 Processor -- start 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.194+0000 7f2e517bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a39070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.194+0000 7f2e517bd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a395b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.194+0000 7f2e517bd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 0x5603b7a45420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.194+0000 7f2e517bd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5603b78c9ab0 con 0x5603b76881b0 2024-09-21T06:15:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.194+0000 7f2e517bd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5603b7937730 con 0x5603b76674d0 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.194+0000 7f2e517bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5603b7a3c130 con 0x5603b765f2f0 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.194+0000 7f2e517bd740 10 client.0 _ll_register_callbacks cb 0x5603b798b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.194+0000 7f2e48dea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 0x5603b7a45420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e48dea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 0x5603b7a45420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58562/0 (socket says 172.21.15.3:58562) 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e48dea640 1 -- 172.21.15.3:0/3469581217 learned_addr learned my addr 172.21.15.3:0/3469581217 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e43fff640 1 --2- 172.21.15.3:0/3469581217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a39070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e437fe640 1 --2- 172.21.15.3:0/3469581217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e48dea640 1 -- 172.21.15.3:0/3469581217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 msgr2=0x5603b7a39070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e48dea640 1 --2- 172.21.15.3:0/3469581217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a39070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e48dea640 1 -- 172.21.15.3:0/3469581217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 msgr2=0x5603b7a395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e48dea640 1 --2- 172.21.15.3:0/3469581217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a395b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e48dea640 1 -- 172.21.15.3:0/3469581217 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603b76f3790 con 0x5603b76881b0 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e48dea640 1 --2- 172.21.15.3:0/3469581217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 0x5603b7a45420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c02f480 tx=0x7f2e3c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cf5500f470755f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e237fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e237fe640 1 -- 172.21.15.3:0/3469581217 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0098b0 con 0x5603b76881b0 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e237fe640 1 -- 172.21.15.3:0/3469581217 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2e3c004250 con 0x5603b76881b0 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.195+0000 7f2e517bd740 1 -- 172.21.15.3:0/3469581217 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e3801a910 con 0x5603b76881b0 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.196+0000 7f2e237fe640 1 -- 172.21.15.3:0/3469581217 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e3c0318e0 con 0x5603b76881b0 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.196+0000 7f2e517bd740 10 client.15615 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.196+0000 7f2e517bd740 1 -- 172.21.15.3:0/3469581217 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5603b776c620 con 0x5603b76881b0 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.196+0000 7f2e237fe640 1 -- 172.21.15.3:0/3469581217 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e3801a910 con 0x5603b76881b0 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.196+0000 7f2e237fe640 1 -- 172.21.15.3:0/3469581217 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2e3c050070 con 0x5603b76881b0 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.196+0000 7f2e517bd740 10 client.15615 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.196+0000 7f2e517bd740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.196+0000 7f2e517bd740 20 client.15615 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.196+0000 7f2e517bd740 1 --2- 172.21.15.3:0/3469581217 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603b7646d80 0x5603b7a4b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.196+0000 7f2e517bd740 4 client.15615 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.196+0000 7f2e517bd740 1 -- 172.21.15.3:0/3469581217 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603b79a4640 con 0x5603b7646d80 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.196+0000 7f2e43fff640 1 --2- 172.21.15.3:0/3469581217 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603b7646d80 0x5603b7a4b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.197+0000 7f2e237fe640 10 client.15615 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.197+0000 7f2e43fff640 1 --2- 172.21.15.3:0/3469581217 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603b7646d80 0x5603b7a4b210 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acaf25cb8afacda5 server_cookie=a325d58e853a5b3b in_seq=0 out_seq=0 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e237fe640 1 -- 172.21.15.3:0/3469581217 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5603b79a4640 con 0x5603b7646d80 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e237fe640 10 client.15615 handle_command_reply: tid=0 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e517bd740 1 client.15615 shutdown 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e517bd740 1 -- 172.21.15.3:0/3469581217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 msgr2=0x5603b7a45420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e517bd740 1 --2- 172.21.15.3:0/3469581217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 0x5603b7a45420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c02f480 tx=0x7f2e3c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e517bd740 1 -- 172.21.15.3:0/3469581217 shutdown_connections 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e517bd740 1 --2- 172.21.15.3:0/3469581217 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603b7646d80 0x5603b7a4b210 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e517bd740 1 --2- 172.21.15.3:0/3469581217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 0x5603b7a45420 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e517bd740 1 --2- 172.21.15.3:0/3469581217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a395b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e517bd740 1 --2- 172.21.15.3:0/3469581217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a39070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e517bd740 1 -- 172.21.15.3:0/3469581217 >> 172.21.15.3:0/3469581217 conn(0x5603b767c300 msgr2=0x5603b7a2e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e237fe640 0 client.15615 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e517bd740 1 -- 172.21.15.3:0/3469581217 shutdown_connections 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e517bd740 1 -- 172.21.15.3:0/3469581217 wait complete. 2024-09-21T06:15:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.198+0000 7f2e517bd740 20 client.15615 trim_cache size 0 max 16384 2024-09-21T06:15:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.213+0000 7f2e517bd740 1 Processor -- start 2024-09-21T06:15:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.213+0000 7f2e517bd740 1 -- start start 2024-09-21T06:15:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.213+0000 7f2e517bd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b765f2f0 0x5603b7a39550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.213+0000 7f2e517bd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a2b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.213+0000 7f2e517bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b76881b0 0x5603b7a45320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.213+0000 7f2e517bd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441af660 con 0x5603b765f2f0 2024-09-21T06:15:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.213+0000 7f2e517bd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b11a0 con 0x5603b76674d0 2024-09-21T06:15:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.213+0000 7f2e517bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441afd30 con 0x5603b76881b0 2024-09-21T06:15:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.213+0000 7f2e43fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b765f2f0 0x5603b7a39550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.213+0000 7f2e43fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b765f2f0 0x5603b7a39550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58574/0 (socket says 172.21.15.3:58574) 2024-09-21T06:15:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.213+0000 7f2e43fff640 1 -- 172.21.15.3:0/565723092 learned_addr learned my addr 172.21.15.3:0/565723092 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.213+0000 7f2e437fe640 1 --2- 172.21.15.3:0/565723092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a2b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.214+0000 7f2e43fff640 1 -- 172.21.15.3:0/565723092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b76881b0 msgr2=0x5603b7a45320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.214+0000 7f2e48dea640 1 --2- 172.21.15.3:0/565723092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b76881b0 0x5603b7a45320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.214+0000 7f2e43fff640 1 --2- 172.21.15.3:0/565723092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b76881b0 0x5603b7a45320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.214+0000 7f2e43fff640 1 -- 172.21.15.3:0/565723092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 msgr2=0x5603b7a2b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.214+0000 7f2e43fff640 1 --2- 172.21.15.3:0/565723092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a2b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.214+0000 7f2e43fff640 1 -- 172.21.15.3:0/565723092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603b7a307c0 con 0x5603b765f2f0 2024-09-21T06:15:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.214+0000 7f2e437fe640 1 --2- 172.21.15.3:0/565723092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a2b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.214+0000 7f2e43fff640 1 --2- 172.21.15.3:0/565723092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b765f2f0 0x5603b7a39550 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2e340e58f0 tx=0x7f2e34000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b4ccbb530937d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.214+0000 7f2e29ffb640 1 -- 172.21.15.3:0/565723092 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e340a9160 con 0x5603b765f2f0 2024-09-21T06:15:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.214+0000 7f2e29ffb640 1 -- 172.21.15.3:0/565723092 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e34010720 con 0x5603b765f2f0 2024-09-21T06:15:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.214+0000 7f2e29ffb640 1 -- 172.21.15.3:0/565723092 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e340a9c60 con 0x5603b765f2f0 2024-09-21T06:15:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.215+0000 7f2e517bd740 1 -- 172.21.15.3:0/565723092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b765f2f0 msgr2=0x5603b7a39550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.215+0000 7f2e517bd740 1 --2- 172.21.15.3:0/565723092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b765f2f0 0x5603b7a39550 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2e340e58f0 tx=0x7f2e34000bc0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.215+0000 7f2e517bd740 1 -- 172.21.15.3:0/565723092 shutdown_connections 2024-09-21T06:15:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.215+0000 7f2e517bd740 1 --2- 172.21.15.3:0/565723092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b76881b0 0x5603b7a45320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.215+0000 7f2e517bd740 1 --2- 172.21.15.3:0/565723092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a2b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.215+0000 7f2e517bd740 1 --2- 172.21.15.3:0/565723092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b765f2f0 0x5603b7a39550 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.215+0000 7f2e517bd740 1 -- 172.21.15.3:0/565723092 >> 172.21.15.3:0/565723092 conn(0x5603b767c300 msgr2=0x5603b7a2b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.215+0000 7f2e517bd740 1 -- 172.21.15.3:0/565723092 shutdown_connections 2024-09-21T06:15:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.215+0000 7f2e517bd740 1 -- 172.21.15.3:0/565723092 wait complete. 2024-09-21T06:15:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.215+0000 7f2e517bd740 1 -- start start 2024-09-21T06:15:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.215+0000 7f2e517bd740 1 Processor -- start 2024-09-21T06:15:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e517bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a3e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e517bd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a3e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e517bd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 0x5603b7aadb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e517bd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b0ad0 con 0x5603b76881b0 2024-09-21T06:15:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e517bd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b0400 con 0x5603b76674d0 2024-09-21T06:15:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e517bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441ad640 con 0x5603b765f2f0 2024-09-21T06:15:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e517bd740 10 client.0 _ll_register_callbacks cb 0x7f2e3c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e48dea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 0x5603b7aadb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e48dea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 0x5603b7aadb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58578/0 (socket says 172.21.15.3:58578) 2024-09-21T06:15:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e48dea640 1 -- 172.21.15.3:0/1439240307 learned_addr learned my addr 172.21.15.3:0/1439240307 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e43fff640 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a3e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e437fe640 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a3e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e48dea640 1 -- 172.21.15.3:0/1439240307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 msgr2=0x5603b7a3e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e48dea640 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a3e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e48dea640 1 -- 172.21.15.3:0/1439240307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 msgr2=0x5603b7a3e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e48dea640 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a3e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e48dea640 1 -- 172.21.15.3:0/1439240307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603b7aae200 con 0x5603b76881b0 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e43fff640 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a3e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e437fe640 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a3e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e48dea640 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 0x5603b7aadb90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009ca0 tx=0x7f2e3c02af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616cf3e56a4e5b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.216+0000 7f2e237fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e237fe640 1 -- 172.21.15.3:0/1439240307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c05f020 con 0x5603b76881b0 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e237fe640 1 -- 172.21.15.3:0/1439240307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2e3c004450 con 0x5603b76881b0 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e517bd740 1 -- 172.21.15.3:0/1439240307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603b79a00f0 con 0x5603b76881b0 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e237fe640 1 -- 172.21.15.3:0/1439240307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e3c06a050 con 0x5603b76881b0 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e517bd740 10 client.15621 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e517bd740 1 -- 172.21.15.3:0/1439240307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5603b7ab5530 con 0x5603b76881b0 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e237fe640 1 -- 172.21.15.3:0/1439240307 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5603b79a00f0 con 0x5603b76881b0 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e237fe640 1 -- 172.21.15.3:0/1439240307 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2e3c050030 con 0x5603b76881b0 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e517bd740 10 client.15621 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e517bd740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e517bd740 20 client.15621 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e517bd740 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603b761c1c0 0x5603b7ab7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e517bd740 4 client.15621 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:15:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.217+0000 7f2e517bd740 1 -- 172.21.15.3:0/1439240307 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603b799d520 con 0x5603b761c1c0 2024-09-21T06:15:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.218+0000 7f2e43fff640 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603b761c1c0 0x5603b7ab7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.218+0000 7f2e43fff640 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603b761c1c0 0x5603b7ab7ce0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37516010a1aef24f server_cookie=cd9073bd56fed0a0 in_seq=0 out_seq=0 2024-09-21T06:15:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.218+0000 7f2e237fe640 10 client.15621 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e237fe640 1 -- 172.21.15.3:0/1439240307 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603b799d520 con 0x5603b761c1c0 2024-09-21T06:15:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e237fe640 10 client.15621 handle_command_reply: tid=0 2024-09-21T06:15:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e517bd740 1 client.15621 shutdown 2024-09-21T06:15:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e517bd740 1 -- 172.21.15.3:0/1439240307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 msgr2=0x5603b7aadb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e517bd740 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 0x5603b7aadb90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009ca0 tx=0x7f2e3c02af40 comp rx=0 tx=0).stop 2024-09-21T06:15:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e517bd740 1 -- 172.21.15.3:0/1439240307 shutdown_connections 2024-09-21T06:15:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e517bd740 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603b761c1c0 0x5603b7ab7ce0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e517bd740 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603b76881b0 0x5603b7aadb90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e517bd740 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603b76674d0 0x5603b7a3e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e237fe640 0 client.15621 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e517bd740 1 --2- 172.21.15.3:0/1439240307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603b765f2f0 0x5603b7a3e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e517bd740 1 -- 172.21.15.3:0/1439240307 >> 172.21.15.3:0/1439240307 conn(0x5603b767c300 msgr2=0x5603b7a29ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e517bd740 1 -- 172.21.15.3:0/1439240307 shutdown_connections 2024-09-21T06:15:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e517bd740 1 -- 172.21.15.3:0/1439240307 wait complete. 2024-09-21T06:15:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.230+0000 7f2e517bd740 20 client.15621 trim_cache size 0 max 16384 2024-09-21T06:15:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:15:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:15:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:15:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:15:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.231+0000 7f2e4a84d640 1 -- 172.21.15.3:0/770746104 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e2c0b6680 msgr2=0x7f2e2c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.231+0000 7f2e4a84d640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e2c0b6680 0x7f2e2c0b8b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c006010 tx=0x7f2e3c009440 comp rx=0 tx=0).stop 2024-09-21T06:15:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.231+0000 7f2e4a84d640 1 -- 172.21.15.3:0/770746104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e441029b0 msgr2=0x7f2e4419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.231+0000 7f2e4a84d640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e441029b0 0x7f2e4419a440 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400e9f0 tx=0x7f2e3400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.231+0000 7f2e4a84d640 1 -- 172.21.15.3:0/770746104 shutdown_connections 2024-09-21T06:15:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.231+0000 7f2e4a84d640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e2c0b6680 0x7f2e2c0b8b90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.231+0000 7f2e4a84d640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e44108c30 0x7f2e4419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.231+0000 7f2e4a84d640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e441082e0 0x7f2e4419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.231+0000 7f2e4a84d640 1 --2- 172.21.15.3:0/770746104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e441029b0 0x7f2e4419a440 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.231+0000 7f2e4a84d640 1 -- 172.21.15.3:0/770746104 >> 172.21.15.3:0/770746104 conn(0x7f2e4407a740 msgr2=0x7f2e441019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.231+0000 7f2e4a84d640 1 -- 172.21.15.3:0/770746104 shutdown_connections 2024-09-21T06:15:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:32.231+0000 7f2e4a84d640 1 -- 172.21.15.3:0/770746104 wait complete. 2024-09-21T06:15:33.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:32 smithi003 ceph-mon[31227]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 102 KiB/s wr, 49 op/s 2024-09-21T06:15:33.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:32 smithi137 ceph-mon[34464]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 102 KiB/s wr, 49 op/s 2024-09-21T06:15:33.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:32 smithi204 ceph-mon[34659]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 102 KiB/s wr, 49 op/s 2024-09-21T06:15:35.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:34 smithi003 ceph-mon[31227]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T06:15:35.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:34 smithi137 ceph-mon[34464]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T06:15:35.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:34 smithi204 ceph-mon[34659]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T06:15:37.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:36 smithi003 ceph-mon[31227]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T06:15:37.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:36 smithi137 ceph-mon[34464]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T06:15:37.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:36 smithi204 ceph-mon[34659]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T06:15:39.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:38 smithi003 ceph-mon[31227]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-09-21T06:15:39.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:38 smithi137 ceph-mon[34464]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-09-21T06:15:39.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:38 smithi204 ceph-mon[34659]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-09-21T06:15:41.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:40 smithi003 ceph-mon[31227]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-21T06:15:41.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:40 smithi137 ceph-mon[34464]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-21T06:15:41.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:40 smithi204 ceph-mon[34659]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-21T06:15:43.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:42 smithi003 ceph-mon[31227]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-09-21T06:15:43.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:42 smithi137 ceph-mon[34464]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-09-21T06:15:43.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:42 smithi204 ceph-mon[34659]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-09-21T06:15:45.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:44 smithi003 ceph-mon[31227]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-21T06:15:45.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:44 smithi137 ceph-mon[34464]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-21T06:15:45.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:44 smithi204 ceph-mon[34659]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-21T06:15:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.341+0000 7fde38a63640 1 -- 172.21.15.3:0/617029522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde3410dd60 msgr2=0x7fde34110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.341+0000 7fde38a63640 1 --2- 172.21.15.3:0/617029522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde3410dd60 0x7fde34110250 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009ca0 tx=0x7fde2c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.341+0000 7fde38a63640 1 -- 172.21.15.3:0/617029522 shutdown_connections 2024-09-21T06:15:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.341+0000 7fde38a63640 1 --2- 172.21.15.3:0/617029522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde34105250 0x7fde3410d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.341+0000 7fde38a63640 1 --2- 172.21.15.3:0/617029522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde3410dd60 0x7fde34110250 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.341+0000 7fde38a63640 1 --2- 172.21.15.3:0/617029522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34104900 0x7fde34104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.341+0000 7fde38a63640 1 -- 172.21.15.3:0/617029522 >> 172.21.15.3:0/617029522 conn(0x7fde3407a740 msgr2=0x7fde3407ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.341+0000 7fde38a63640 1 -- 172.21.15.3:0/617029522 shutdown_connections 2024-09-21T06:15:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.341+0000 7fde38a63640 1 -- 172.21.15.3:0/617029522 wait complete. 2024-09-21T06:15:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.342+0000 7fde38a63640 1 Processor -- start 2024-09-21T06:15:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.342+0000 7fde38a63640 1 -- start start 2024-09-21T06:15:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.342+0000 7fde38a63640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde34104900 0x7fde34102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.342+0000 7fde38a63640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105250 0x7fde34102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.342+0000 7fde38a63640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde3410dd60 0x7fde341030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.342+0000 7fde38a63640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fde341139c0 con 0x7fde3410dd60 2024-09-21T06:15:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.342+0000 7fde38a63640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fde34113840 con 0x7fde34105250 2024-09-21T06:15:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.342+0000 7fde38a63640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fde34113b40 con 0x7fde34104900 2024-09-21T06:15:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.342+0000 7fde32d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde3410dd60 0x7fde341030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.342+0000 7fde32d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde3410dd60 0x7fde341030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36622/0 (socket says 172.21.15.3:36622) 2024-09-21T06:15:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.342+0000 7fde32d9d640 1 -- 172.21.15.3:0/2140248213 learned_addr learned my addr 172.21.15.3:0/2140248213 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.342+0000 7fde31d9b640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105250 0x7fde34102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.342+0000 7fde3259c640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde34104900 0x7fde34102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.343+0000 7fde32d9d640 1 -- 172.21.15.3:0/2140248213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde34104900 msgr2=0x7fde34102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.343+0000 7fde32d9d640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde34104900 0x7fde34102630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.343+0000 7fde32d9d640 1 -- 172.21.15.3:0/2140248213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105250 msgr2=0x7fde34102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.343+0000 7fde32d9d640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105250 0x7fde34102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.343+0000 7fde32d9d640 1 -- 172.21.15.3:0/2140248213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde341a86f0 con 0x7fde3410dd60 2024-09-21T06:15:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.343+0000 7fde32d9d640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde3410dd60 0x7fde341030b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fde2c005cc0 tx=0x7fde2c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.343+0000 7fde237fe640 1 -- 172.21.15.3:0/2140248213 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c003bf0 con 0x7fde3410dd60 2024-09-21T06:15:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.343+0000 7fde237fe640 1 -- 172.21.15.3:0/2140248213 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde2c003d90 con 0x7fde3410dd60 2024-09-21T06:15:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.343+0000 7fde237fe640 1 -- 172.21.15.3:0/2140248213 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c02fbc0 con 0x7fde3410dd60 2024-09-21T06:15:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.343+0000 7fde38a63640 1 -- 172.21.15.3:0/2140248213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde341a8980 con 0x7fde3410dd60 2024-09-21T06:15:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.343+0000 7fde38a63640 1 -- 172.21.15.3:0/2140248213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde341a8f20 con 0x7fde3410dd60 2024-09-21T06:15:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.346+0000 7fde3f9d2740 1 Processor -- start 2024-09-21T06:15:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.347+0000 7fde237fe640 1 -- 172.21.15.3:0/2140248213 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 154636+0+0 (secure 0 0 0) 0x7fde2c02fd60 con 0x7fde3410dd60 2024-09-21T06:15:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.347+0000 7fde3f9d2740 1 -- start start 2024-09-21T06:15:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde3f9d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91c8f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde3f9d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c12f0 0x55de91c8fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde3f9d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 0x55de91c90430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde3f9d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55de91c046f0 con 0x55de918ea1b0 2024-09-21T06:15:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde3f9d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55de91bf3b20 con 0x55de918c94d0 2024-09-21T06:15:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde3f9d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55de91bf7930 con 0x55de918c12f0 2024-09-21T06:15:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91c8f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91c8f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36634/0 (socket says 172.21.15.3:36634) 2024-09-21T06:15:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde31d9b640 1 -- 172.21.15.3:0/4260303820 learned_addr learned my addr 172.21.15.3:0/4260303820 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde31d9b640 1 --2- 172.21.15.3:0/4260303820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 0x55de91c90430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde3259c640 1 --2- 172.21.15.3:0/4260303820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c12f0 0x55de91c8fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde31d9b640 1 -- 172.21.15.3:0/4260303820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c12f0 msgr2=0x55de91c8fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde31d9b640 1 --2- 172.21.15.3:0/4260303820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c12f0 0x55de91c8fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde237fe640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fde1c0b6730 0x7fde1c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde31d9b640 1 -- 172.21.15.3:0/4260303820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 msgr2=0x55de91c90430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde31d9b640 1 --2- 172.21.15.3:0/4260303820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 0x55de91c90430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.348+0000 7fde31d9b640 1 -- 172.21.15.3:0/4260303820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de91bb9530 con 0x55de918ea1b0 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.349+0000 7fde32d9d640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fde1c0b6730 0x7fde1c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.349+0000 7fde31d9b640 1 --2- 172.21.15.3:0/4260303820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91c8f9b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fde280124c0 tx=0x7fde2800b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24ecbbbfd0b7025 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.349+0000 7fde237fe640 1 -- 172.21.15.3:0/2140248213 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fde2c108950 con 0x7fde3410dd60 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.349+0000 7fde20ff9640 1 -- 172.21.15.3:0/4260303820 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28004030 con 0x55de918ea1b0 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.349+0000 7fde20ff9640 1 -- 172.21.15.3:0/4260303820 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde280054d0 con 0x55de918ea1b0 2024-09-21T06:15:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.349+0000 7fde20ff9640 1 -- 172.21.15.3:0/4260303820 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28004b60 con 0x55de918ea1b0 2024-09-21T06:15:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.350+0000 7fde3f9d2740 1 -- 172.21.15.3:0/4260303820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 msgr2=0x55de91c8f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.350+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/4260303820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91c8f9b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fde280124c0 tx=0x7fde2800b740 comp rx=0 tx=0).stop 2024-09-21T06:15:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.350+0000 7fde32d9d640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fde1c0b6730 0x7fde1c0b8c40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fde2c1066e0 tx=0x7fde2c09a4a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.350+0000 7fde3f9d2740 1 -- 172.21.15.3:0/4260303820 shutdown_connections 2024-09-21T06:15:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.350+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/4260303820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c12f0 0x55de91c8fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.350+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/4260303820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 0x55de91c90430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.350+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/4260303820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91c8f9b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.350+0000 7fde3f9d2740 1 -- 172.21.15.3:0/4260303820 >> 172.21.15.3:0/4260303820 conn(0x55de918de300 msgr2=0x55de91c8d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.350+0000 7fde3f9d2740 1 -- 172.21.15.3:0/4260303820 shutdown_connections 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.350+0000 7fde3f9d2740 1 -- 172.21.15.3:0/4260303820 wait complete. 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.350+0000 7fde3f9d2740 1 -- start start 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.351+0000 7fde3f9d2740 1 Processor -- start 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.351+0000 7fde3f9d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c12f0 0x55de91c9b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.351+0000 7fde3f9d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 0x55de91c9b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.351+0000 7fde3f9d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91ca7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.351+0000 7fde3f9d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55de91b2bab0 con 0x55de918ea1b0 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.351+0000 7fde3f9d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55de91b99730 con 0x55de918c94d0 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.351+0000 7fde3f9d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55de91c9e130 con 0x55de918c12f0 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.351+0000 7fde3f9d2740 10 client.0 _ll_register_callbacks cb 0x55de91bed6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.351+0000 7fde31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91ca7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.351+0000 7fde31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91ca7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36644/0 (socket says 172.21.15.3:36644) 2024-09-21T06:15:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.351+0000 7fde31d9b640 1 -- 172.21.15.3:0/2404343595 learned_addr learned my addr 172.21.15.3:0/2404343595 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.351+0000 7fde3259c640 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 0x55de91c9b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.351+0000 7fde31d9b640 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c12f0 0x55de91c9b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde3259c640 1 -- 172.21.15.3:0/2404343595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c12f0 msgr2=0x55de91c9b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde3259c640 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c12f0 0x55de91c9b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde3259c640 1 -- 172.21.15.3:0/2404343595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 msgr2=0x55de91ca7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde3259c640 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91ca7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde3259c640 1 -- 172.21.15.3:0/2404343595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de91955790 con 0x55de918c94d0 2024-09-21T06:15:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde31d9b640 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91ca7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde3259c640 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 0x55de91c9b5c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fde34104080 tx=0x7fde24012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dcf5f36dbac63c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:15:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde19ffb640 1 -- 172.21.15.3:0/2404343595 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24005220 con 0x55de918c94d0 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde3f9d2740 1 -- 172.21.15.3:0/2404343595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde280054d0 con 0x55de918c94d0 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde19ffb640 1 -- 172.21.15.3:0/2404343595 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fde240053c0 con 0x55de918c94d0 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde19ffb640 1 -- 172.21.15.3:0/2404343595 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde240055c0 con 0x55de918c94d0 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde3f9d2740 10 client.25247 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.352+0000 7fde3f9d2740 1 -- 172.21.15.3:0/2404343595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de919ce620 con 0x55de918c94d0 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.353+0000 7fde19ffb640 1 -- 172.21.15.3:0/2404343595 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde280054d0 con 0x55de918c94d0 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.353+0000 7fde19ffb640 1 -- 172.21.15.3:0/2404343595 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fde24019070 con 0x55de918c94d0 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.353+0000 7fde3f9d2740 10 client.25247 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.353+0000 7fde3f9d2740 10 client.25247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.353+0000 7fde3f9d2740 20 client.25247 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.353+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de918a8d80 0x55de91cad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.353+0000 7fde3f9d2740 4 client.25247 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.353+0000 7fde3f9d2740 1 -- 172.21.15.3:0/2404343595 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de91c06640 con 0x55de918a8d80 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.353+0000 7fde31d9b640 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de918a8d80 0x55de91cad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.354+0000 7fde19ffb640 10 client.25247 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.354+0000 7fde31d9b640 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de918a8d80 0x55de91cad2a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d438dcb87165a102 server_cookie=18959eccab832921 in_seq=0 out_seq=0 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.354+0000 7fde19ffb640 1 -- 172.21.15.3:0/2404343595 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de91c06640 con 0x55de918a8d80 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.354+0000 7fde19ffb640 10 client.25247 handle_command_reply: tid=0 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.354+0000 7fde3f9d2740 1 client.25247 shutdown 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.355+0000 7fde3f9d2740 1 -- 172.21.15.3:0/2404343595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 msgr2=0x55de91c9b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.355+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 0x55de91c9b5c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fde34104080 tx=0x7fde24012820 comp rx=0 tx=0).stop 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.355+0000 7fde3f9d2740 1 -- 172.21.15.3:0/2404343595 shutdown_connections 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.355+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de918a8d80 0x55de91cad2a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.355+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91ca7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.355+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 0x55de91c9b5c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.355+0000 7fde19ffb640 0 client.25247 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.355+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/2404343595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c12f0 0x55de91c9b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.355+0000 7fde3f9d2740 1 -- 172.21.15.3:0/2404343595 >> 172.21.15.3:0/2404343595 conn(0x55de918de300 msgr2=0x55de91c909e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.355+0000 7fde3f9d2740 1 -- 172.21.15.3:0/2404343595 shutdown_connections 2024-09-21T06:15:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.355+0000 7fde3f9d2740 1 -- 172.21.15.3:0/2404343595 wait complete. 2024-09-21T06:15:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.355+0000 7fde3f9d2740 20 client.25247 trim_cache size 0 max 16384 2024-09-21T06:15:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.370+0000 7fde3f9d2740 1 Processor -- start 2024-09-21T06:15:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.370+0000 7fde3f9d2740 1 -- start start 2024-09-21T06:15:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.370+0000 7fde3f9d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c12f0 0x55de91c9ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.370+0000 7fde3f9d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c94d0 0x55de91ca7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.370+0000 7fde3f9d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91c98250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.370+0000 7fde3f9d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fde341b1800 con 0x55de918ea1b0 2024-09-21T06:15:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.370+0000 7fde3f9d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fde341b3340 con 0x55de918c12f0 2024-09-21T06:15:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.370+0000 7fde3f9d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fde341b1ed0 con 0x55de918c94d0 2024-09-21T06:15:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.370+0000 7fde31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91c98250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.370+0000 7fde31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91c98250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36650/0 (socket says 172.21.15.3:36650) 2024-09-21T06:15:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.370+0000 7fde31d9b640 1 -- 172.21.15.3:0/3222779207 learned_addr learned my addr 172.21.15.3:0/3222779207 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.370+0000 7fde31d9b640 1 --2- 172.21.15.3:0/3222779207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c12f0 0x55de91c9ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.370+0000 7fde3259c640 1 --2- 172.21.15.3:0/3222779207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c94d0 0x55de91ca7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.371+0000 7fde31d9b640 1 -- 172.21.15.3:0/3222779207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c94d0 msgr2=0x55de91ca7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.371+0000 7fde31d9b640 1 --2- 172.21.15.3:0/3222779207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c94d0 0x55de91ca7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.371+0000 7fde31d9b640 1 -- 172.21.15.3:0/3222779207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c12f0 msgr2=0x55de91c9ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.371+0000 7fde31d9b640 1 --2- 172.21.15.3:0/3222779207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c12f0 0x55de91c9ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.371+0000 7fde31d9b640 1 -- 172.21.15.3:0/3222779207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de91cad610 con 0x55de918ea1b0 2024-09-21T06:15:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.371+0000 7fde31d9b640 1 --2- 172.21.15.3:0/3222779207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91c98250 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fde28005940 tx=0x7fde2800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0e48fe577dbea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.371+0000 7fde1bfff640 1 -- 172.21.15.3:0/3222779207 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28028040 con 0x55de918ea1b0 2024-09-21T06:15:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.371+0000 7fde1bfff640 1 -- 172.21.15.3:0/3222779207 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde2801eb30 con 0x55de918ea1b0 2024-09-21T06:15:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.371+0000 7fde1bfff640 1 -- 172.21.15.3:0/3222779207 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28030ac0 con 0x55de918ea1b0 2024-09-21T06:15:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.372+0000 7fde3f9d2740 1 -- 172.21.15.3:0/3222779207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 msgr2=0x55de91c98250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.372+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/3222779207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91c98250 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fde28005940 tx=0x7fde2800bea0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.372+0000 7fde3f9d2740 1 -- 172.21.15.3:0/3222779207 shutdown_connections 2024-09-21T06:15:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.372+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/3222779207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918c94d0 0x55de91ca7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.372+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/3222779207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918ea1b0 0x55de91c98250 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.372+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/3222779207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c12f0 0x55de91c9ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.372+0000 7fde3f9d2740 1 -- 172.21.15.3:0/3222779207 >> 172.21.15.3:0/3222779207 conn(0x55de918de300 msgr2=0x55de91c9c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.372+0000 7fde3f9d2740 1 -- 172.21.15.3:0/3222779207 shutdown_connections 2024-09-21T06:15:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.372+0000 7fde3f9d2740 1 -- 172.21.15.3:0/3222779207 wait complete. 2024-09-21T06:15:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.372+0000 7fde3f9d2740 1 -- start start 2024-09-21T06:15:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde3f9d2740 1 Processor -- start 2024-09-21T06:15:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde3f9d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918c12f0 0x55de91ca0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde3f9d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 0x55de91ca1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde3f9d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918ea1b0 0x55de91d0fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde3f9d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fde341b2c70 con 0x55de918c12f0 2024-09-21T06:15:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde3f9d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fde341b25a0 con 0x55de918c94d0 2024-09-21T06:15:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde3f9d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fde341af7e0 con 0x55de918ea1b0 2024-09-21T06:15:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde3f9d2740 10 client.0 _ll_register_callbacks cb 0x7fde2400bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918c12f0 0x55de91ca0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918c12f0 0x55de91ca0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36656/0 (socket says 172.21.15.3:36656) 2024-09-21T06:15:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde31d9b640 1 -- 172.21.15.3:0/3344658622 learned_addr learned my addr 172.21.15.3:0/3344658622 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde3259c640 1 --2- 172.21.15.3:0/3344658622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 0x55de91ca1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde31d9b640 1 -- 172.21.15.3:0/3344658622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918ea1b0 msgr2=0x55de91d0fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde31d9b640 1 --2- 172.21.15.3:0/3344658622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918ea1b0 0x55de91d0fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde31d9b640 1 -- 172.21.15.3:0/3344658622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 msgr2=0x55de91ca1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde31d9b640 1 --2- 172.21.15.3:0/3344658622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 0x55de91ca1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.373+0000 7fde31d9b640 1 -- 172.21.15.3:0/3344658622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de91d10280 con 0x55de918c12f0 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.374+0000 7fde31d9b640 1 --2- 172.21.15.3:0/3344658622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918c12f0 0x55de91ca0d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fde2800b500 tx=0x7fde280046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1cc497acee72bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.374+0000 7fde19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.374+0000 7fde19ffb640 1 -- 172.21.15.3:0/3344658622 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28023040 con 0x55de918c12f0 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.374+0000 7fde19ffb640 1 -- 172.21.15.3:0/3344658622 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fde2801e480 con 0x55de918c12f0 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.374+0000 7fde3f9d2740 1 -- 172.21.15.3:0/3344658622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde2801eb30 con 0x55de918c12f0 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.374+0000 7fde19ffb640 1 -- 172.21.15.3:0/3344658622 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde2801e680 con 0x55de918c12f0 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.374+0000 7fde3f9d2740 10 client.15639 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.374+0000 7fde3f9d2740 1 -- 172.21.15.3:0/3344658622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de91d17670 con 0x55de918c12f0 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.374+0000 7fde19ffb640 1 -- 172.21.15.3:0/3344658622 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde2801eb30 con 0x55de918c12f0 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.374+0000 7fde19ffb640 1 -- 172.21.15.3:0/3344658622 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fde2801ece0 con 0x55de918c12f0 2024-09-21T06:15:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.374+0000 7fde3f9d2740 10 client.15639 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.374+0000 7fde3f9d2740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.374+0000 7fde3f9d2740 20 client.15639 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.375+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/3344658622 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de9187e1c0 0x55de91d19c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.375+0000 7fde3f9d2740 4 client.15639 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:15:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.375+0000 7fde3f9d2740 1 -- 172.21.15.3:0/3344658622 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de91c020f0 con 0x55de9187e1c0 2024-09-21T06:15:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.375+0000 7fde3259c640 1 --2- 172.21.15.3:0/3344658622 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de9187e1c0 0x55de91d19c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.375+0000 7fde3259c640 1 --2- 172.21.15.3:0/3344658622 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de9187e1c0 0x55de91d19c10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c814d2147d889526 server_cookie=84ffac634720d195 in_seq=0 out_seq=0 2024-09-21T06:15:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.375+0000 7fde19ffb640 10 client.15639 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde19ffb640 1 -- 172.21.15.3:0/3344658622 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de91c020f0 con 0x55de9187e1c0 2024-09-21T06:15:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde19ffb640 10 client.15639 handle_command_reply: tid=0 2024-09-21T06:15:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde3f9d2740 1 client.15639 shutdown 2024-09-21T06:15:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde3f9d2740 1 -- 172.21.15.3:0/3344658622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918c12f0 msgr2=0x55de91ca0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/3344658622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918c12f0 0x55de91ca0d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fde2800b500 tx=0x7fde280046f0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde3f9d2740 1 -- 172.21.15.3:0/3344658622 shutdown_connections 2024-09-21T06:15:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/3344658622 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de9187e1c0 0x55de91d19c10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/3344658622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de918ea1b0 0x55de91d0fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde19ffb640 0 client.15639 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/3344658622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de918c94d0 0x55de91ca1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde3f9d2740 1 --2- 172.21.15.3:0/3344658622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de918c12f0 0x55de91ca0d90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde3f9d2740 1 -- 172.21.15.3:0/3344658622 >> 172.21.15.3:0/3344658622 conn(0x55de918de300 msgr2=0x55de91c8d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde3f9d2740 1 -- 172.21.15.3:0/3344658622 shutdown_connections 2024-09-21T06:15:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.381+0000 7fde3f9d2740 1 -- 172.21.15.3:0/3344658622 wait complete. 2024-09-21T06:15:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.382+0000 7fde3f9d2740 20 client.15639 trim_cache size 0 max 16384 2024-09-21T06:15:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:15:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:15:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:15:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:15:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.382+0000 7fde38a63640 1 -- 172.21.15.3:0/2140248213 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fde1c0b6730 msgr2=0x7fde1c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.382+0000 7fde38a63640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fde1c0b6730 0x7fde1c0b8c40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fde2c1066e0 tx=0x7fde2c09a4a0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.382+0000 7fde38a63640 1 -- 172.21.15.3:0/2140248213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde3410dd60 msgr2=0x7fde341030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.382+0000 7fde38a63640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde3410dd60 0x7fde341030b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fde2c005cc0 tx=0x7fde2c02f380 comp rx=0 tx=0).stop 2024-09-21T06:15:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.382+0000 7fde38a63640 1 -- 172.21.15.3:0/2140248213 shutdown_connections 2024-09-21T06:15:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.382+0000 7fde38a63640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fde1c0b6730 0x7fde1c0b8c40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.382+0000 7fde38a63640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde3410dd60 0x7fde341030b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.382+0000 7fde38a63640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105250 0x7fde34102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.382+0000 7fde38a63640 1 --2- 172.21.15.3:0/2140248213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde34104900 0x7fde34102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.382+0000 7fde38a63640 1 -- 172.21.15.3:0/2140248213 >> 172.21.15.3:0/2140248213 conn(0x7fde3407a740 msgr2=0x7fde340ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.382+0000 7fde38a63640 1 -- 172.21.15.3:0/2140248213 shutdown_connections 2024-09-21T06:15:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:45.382+0000 7fde38a63640 1 -- 172.21.15.3:0/2140248213 wait complete. 2024-09-21T06:15:47.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:46 smithi003 ceph-mon[31227]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T06:15:47.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:46 smithi137 ceph-mon[34464]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T06:15:47.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:46 smithi204 ceph-mon[34659]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T06:15:49.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:48 smithi003 ceph-mon[31227]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-21T06:15:49.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:48 smithi137 ceph-mon[34464]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-21T06:15:49.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:48 smithi204 ceph-mon[34659]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-21T06:15:50.320 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:15:50.663 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:51.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:50 smithi003 ceph-mon[31227]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:15:51.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:50 smithi137 ceph-mon[34464]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:15:51.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:50 smithi204 ceph-mon[34659]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:15:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.230+0000 7f073ca19640 1 -- 172.21.15.3:0/2386248312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07381083d0 msgr2=0x7f07381087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.230+0000 7f073ca19640 1 --2- 172.21.15.3:0/2386248312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07381083d0 0x7f07381087e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0720009900 tx=0x7f072002a410 comp rx=0 tx=0).stop 2024-09-21T06:15:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.231+0000 7f073ca19640 1 -- 172.21.15.3:0/2386248312 shutdown_connections 2024-09-21T06:15:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.231+0000 7f073ca19640 1 --2- 172.21.15.3:0/2386248312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0738108d20 0x7f073810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.231+0000 7f073ca19640 1 --2- 172.21.15.3:0/2386248312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07381083d0 0x7f07381087e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.231+0000 7f073ca19640 1 --2- 172.21.15.3:0/2386248312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0738102a90 0x7f0738102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.231+0000 7f073ca19640 1 -- 172.21.15.3:0/2386248312 >> 172.21.15.3:0/2386248312 conn(0x7f0738079730 msgr2=0x7f0738079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.231+0000 7f073ca19640 1 -- 172.21.15.3:0/2386248312 shutdown_connections 2024-09-21T06:15:51.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.231+0000 7f073ca19640 1 -- 172.21.15.3:0/2386248312 wait complete. 2024-09-21T06:15:51.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.232+0000 7f073ca19640 1 Processor -- start 2024-09-21T06:15:51.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.232+0000 7f073ca19640 1 -- start start 2024-09-21T06:15:51.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.232+0000 7f073ca19640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0738102a90 0x7f073819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.232+0000 7f073ca19640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07381083d0 0x7f073819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.232+0000 7f073ca19640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0738108d20 0x7f073819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.232+0000 7f073ca19640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f07381129e0 con 0x7f0738108d20 2024-09-21T06:15:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.232+0000 7f073ca19640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0738112860 con 0x7f07381083d0 2024-09-21T06:15:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.232+0000 7f073ca19640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0738112b60 con 0x7f0738102a90 2024-09-21T06:15:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.232+0000 7f0736d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0738108d20 0x7f073819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.232+0000 7f0736d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0738108d20 0x7f073819bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36672/0 (socket says 172.21.15.3:36672) 2024-09-21T06:15:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.232+0000 7f0736d9d640 1 -- 172.21.15.3:0/2394237107 learned_addr learned my addr 172.21.15.3:0/2394237107 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.232+0000 7f0735d9b640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07381083d0 0x7f073819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.232+0000 7f073659c640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0738102a90 0x7f073819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.233+0000 7f0736d9d640 1 -- 172.21.15.3:0/2394237107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0738102a90 msgr2=0x7f073819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.233+0000 7f0736d9d640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0738102a90 0x7f073819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.233+0000 7f0736d9d640 1 -- 172.21.15.3:0/2394237107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07381083d0 msgr2=0x7f073819b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:51.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.233+0000 7f0736d9d640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07381083d0 0x7f073819b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.233+0000 7f0736d9d640 1 -- 172.21.15.3:0/2394237107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07381a6580 con 0x7f0738108d20 2024-09-21T06:15:51.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.233+0000 7f0735d9b640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07381083d0 0x7f073819b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:51.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.233+0000 7f0736d9d640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0738108d20 0x7f073819bcf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f073000eaa0 tx=0x7f073000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:51.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.233+0000 7f072f7fe640 1 -- 172.21.15.3:0/2394237107 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073000cdb0 con 0x7f0738108d20 2024-09-21T06:15:51.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.233+0000 7f072f7fe640 1 -- 172.21.15.3:0/2394237107 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0730005020 con 0x7f0738108d20 2024-09-21T06:15:51.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.233+0000 7f072f7fe640 1 -- 172.21.15.3:0/2394237107 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07300116d0 con 0x7f0738108d20 2024-09-21T06:15:51.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.233+0000 7f073ca19640 1 -- 172.21.15.3:0/2394237107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07381a6870 con 0x7f0738108d20 2024-09-21T06:15:51.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.233+0000 7f073ca19640 1 -- 172.21.15.3:0/2394237107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07381a6d30 con 0x7f0738108d20 2024-09-21T06:15:51.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.236+0000 7f0742f8d740 1 Processor -- start 2024-09-21T06:15:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.236+0000 7f072f7fe640 1 -- 172.21.15.3:0/2394237107 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 154636+0+0 (secure 0 0 0) 0x7f07300051d0 con 0x7f0738108d20 2024-09-21T06:15:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.237+0000 7f0742f8d740 1 -- start start 2024-09-21T06:15:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.237+0000 7f0742f8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b6008070 0x5624b63af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.237+0000 7f0742f8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b5fdf1b0 0x5624b63afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.237+0000 7f0742f8d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.237+0000 7f0742f8d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5624b6325e50 con 0x5624b5fe7390 2024-09-21T06:15:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.237+0000 7f0742f8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5624b6315280 con 0x5624b5fdf1b0 2024-09-21T06:15:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.237+0000 7f0742f8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5624b6319090 con 0x5624b6008070 2024-09-21T06:15:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.237+0000 7f0735d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.237+0000 7f073659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b5fdf1b0 0x5624b63afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.237+0000 7f0735d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36680/0 (socket says 172.21.15.3:36680) 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.237+0000 7f0735d9b640 1 -- 172.21.15.3:0/4100423561 learned_addr learned my addr 172.21.15.3:0/4100423561 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.237+0000 7f0735d9b640 1 --2- 172.21.15.3:0/4100423561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b6008070 0x5624b63af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.238+0000 7f072f7fe640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f07240b6680 0x7f07240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.238+0000 7f0735d9b640 1 -- 172.21.15.3:0/4100423561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b6008070 msgr2=0x5624b63af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.238+0000 7f0735d9b640 1 --2- 172.21.15.3:0/4100423561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b6008070 0x5624b63af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.238+0000 7f0735d9b640 1 -- 172.21.15.3:0/4100423561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b5fdf1b0 msgr2=0x5624b63afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.238+0000 7f0735d9b640 1 --2- 172.21.15.3:0/4100423561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b5fdf1b0 0x5624b63afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.238+0000 7f0735d9b640 1 -- 172.21.15.3:0/4100423561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b6073650 con 0x5624b5fe7390 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.238+0000 7f073659c640 1 --2- 172.21.15.3:0/4100423561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b5fdf1b0 0x5624b63afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.238+0000 7f072f7fe640 1 -- 172.21.15.3:0/2394237107 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f07300e7850 con 0x7f0738108d20 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.238+0000 7f0735d9b640 1 --2- 172.21.15.3:0/4100423561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63b00d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0720009900 tx=0x7f0720000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f69a51c8409ae63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.238+0000 7f0736d9d640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f07240b6680 0x7f07240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.238+0000 7f072cff9640 1 -- 172.21.15.3:0/4100423561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720035040 con 0x5624b5fe7390 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.238+0000 7f072cff9640 1 -- 172.21.15.3:0/4100423561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0720004030 con 0x5624b5fe7390 2024-09-21T06:15:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.238+0000 7f072cff9640 1 -- 172.21.15.3:0/4100423561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720003a90 con 0x5624b5fe7390 2024-09-21T06:15:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.239+0000 7f0742f8d740 1 -- 172.21.15.3:0/4100423561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 msgr2=0x5624b63b00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.239+0000 7f0736d9d640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f07240b6680 0x7f07240b8b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f07300ad3e0 tx=0x7f073000cf50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.239+0000 7f0742f8d740 1 --2- 172.21.15.3:0/4100423561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63b00d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0720009900 tx=0x7f0720000bc0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.239+0000 7f0742f8d740 1 -- 172.21.15.3:0/4100423561 shutdown_connections 2024-09-21T06:15:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.239+0000 7f0742f8d740 1 --2- 172.21.15.3:0/4100423561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63b00d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.239+0000 7f0742f8d740 1 --2- 172.21.15.3:0/4100423561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b5fdf1b0 0x5624b63afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.239+0000 7f0742f8d740 1 --2- 172.21.15.3:0/4100423561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b6008070 0x5624b63af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.239+0000 7f0742f8d740 1 -- 172.21.15.3:0/4100423561 >> 172.21.15.3:0/4100423561 conn(0x5624b5ffc460 msgr2=0x5624b63ad540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.239+0000 7f0742f8d740 1 -- 172.21.15.3:0/4100423561 shutdown_connections 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.239+0000 7f0742f8d740 1 -- 172.21.15.3:0/4100423561 wait complete. 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.240+0000 7f0742f8d740 1 -- start start 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.240+0000 7f0742f8d740 1 Processor -- start 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.240+0000 7f0742f8d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fdf1b0 0x5624b63bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.240+0000 7f0742f8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b5fe7390 0x5624b63bb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.240+0000 7f0742f8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b6008070 0x5624b63c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.240+0000 7f0742f8d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5624b6176700 con 0x5624b5fdf1b0 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.240+0000 7f0742f8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5624b62b0110 con 0x5624b6008070 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.240+0000 7f0742f8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5624b63bddd0 con 0x5624b5fe7390 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.240+0000 7f0742f8d740 10 client.0 _ll_register_callbacks cb 0x5624b62c9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.240+0000 7f0735d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fdf1b0 0x5624b63bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.240+0000 7f0735d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fdf1b0 0x5624b63bad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36694/0 (socket says 172.21.15.3:36694) 2024-09-21T06:15:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.240+0000 7f0735d9b640 1 -- 172.21.15.3:0/1886020259 learned_addr learned my addr 172.21.15.3:0/1886020259 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f073659c640 1 --2- 172.21.15.3:0/1886020259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b5fe7390 0x5624b63bb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f0735d9b640 1 --2- 172.21.15.3:0/1886020259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b6008070 0x5624b63c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f0735d9b640 1 -- 172.21.15.3:0/1886020259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b5fe7390 msgr2=0x5624b63bb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f0735d9b640 1 --2- 172.21.15.3:0/1886020259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b5fe7390 0x5624b63bb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f0735d9b640 1 -- 172.21.15.3:0/1886020259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b6008070 msgr2=0x5624b63c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f0735d9b640 1 --2- 172.21.15.3:0/1886020259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b6008070 0x5624b63c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f0735d9b640 1 -- 172.21.15.3:0/1886020259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b62ddc40 con 0x5624b5fdf1b0 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f0735d9b640 1 --2- 172.21.15.3:0/1886020259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fdf1b0 0x5624b63bad40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0720001eb0 tx=0x7f0720003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc289d33cc42553 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f071dffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f071dffb640 1 -- 172.21.15.3:0/1886020259 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720035050 con 0x5624b5fdf1b0 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f071dffb640 1 -- 172.21.15.3:0/1886020259 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f0720005030 con 0x5624b5fdf1b0 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f0742f8d740 1 -- 172.21.15.3:0/1886020259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0720004030 con 0x5624b5fdf1b0 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f071dffb640 1 -- 172.21.15.3:0/1886020259 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0720005440 con 0x5624b5fdf1b0 2024-09-21T06:15:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f0742f8d740 10 client.15651 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.241+0000 7f0742f8d740 1 -- 172.21.15.3:0/1886020259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5624b63c9b10 con 0x5624b5fdf1b0 2024-09-21T06:15:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.242+0000 7f071dffb640 1 -- 172.21.15.3:0/1886020259 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0720004030 con 0x5624b5fdf1b0 2024-09-21T06:15:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.242+0000 7f071dffb640 1 -- 172.21.15.3:0/1886020259 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0720054030 con 0x5624b5fdf1b0 2024-09-21T06:15:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.242+0000 7f0742f8d740 10 client.15651 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.242+0000 7f0742f8d740 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.242+0000 7f0742f8d740 20 client.15651 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.242+0000 7f0742f8d740 1 --2- 172.21.15.3:0/1886020259 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5624b5fc69f0 0x5624b63cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.242+0000 7f0742f8d740 4 client.15651 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.242+0000 7f0742f8d740 1 -- 172.21.15.3:0/1886020259 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624b6327da0 con 0x5624b5fc69f0 2024-09-21T06:15:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.242+0000 7f073659c640 1 --2- 172.21.15.3:0/1886020259 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5624b5fc69f0 0x5624b63cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.243+0000 7f071dffb640 10 client.15651 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.243+0000 7f073659c640 1 --2- 172.21.15.3:0/1886020259 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5624b5fc69f0 0x5624b63cce70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc31fc727aa45ed4 server_cookie=ce8ccdaa41f4d57f in_seq=0 out_seq=0 2024-09-21T06:15:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f071dffb640 1 -- 172.21.15.3:0/1886020259 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5624b6327da0 con 0x5624b5fc69f0 2024-09-21T06:15:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f071dffb640 10 client.15651 handle_command_reply: tid=0 2024-09-21T06:15:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f0742f8d740 1 client.15651 shutdown 2024-09-21T06:15:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f0742f8d740 1 -- 172.21.15.3:0/1886020259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fdf1b0 msgr2=0x5624b63bad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f0742f8d740 1 --2- 172.21.15.3:0/1886020259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fdf1b0 0x5624b63bad40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0720001eb0 tx=0x7f0720003730 comp rx=0 tx=0).stop 2024-09-21T06:15:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f0742f8d740 1 -- 172.21.15.3:0/1886020259 shutdown_connections 2024-09-21T06:15:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f0742f8d740 1 --2- 172.21.15.3:0/1886020259 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5624b5fc69f0 0x5624b63cce70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f0742f8d740 1 --2- 172.21.15.3:0/1886020259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b5fe7390 0x5624b63bb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f0742f8d740 1 --2- 172.21.15.3:0/1886020259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b6008070 0x5624b63c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f0742f8d740 1 --2- 172.21.15.3:0/1886020259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fdf1b0 0x5624b63bad40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f0742f8d740 1 -- 172.21.15.3:0/1886020259 >> 172.21.15.3:0/1886020259 conn(0x5624b5ffc460 msgr2=0x5624b63b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f071dffb640 0 client.15651 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f0742f8d740 1 -- 172.21.15.3:0/1886020259 shutdown_connections 2024-09-21T06:15:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f0742f8d740 1 -- 172.21.15.3:0/1886020259 wait complete. 2024-09-21T06:15:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.244+0000 7f0742f8d740 20 client.15651 trim_cache size 0 max 16384 2024-09-21T06:15:51.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.262+0000 7f0742f8d740 1 Processor -- start 2024-09-21T06:15:51.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.262+0000 7f0742f8d740 1 -- start start 2024-09-21T06:15:51.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.262+0000 7f0742f8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b5fdf1b0 0x5624b63bc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.262+0000 7f0742f8d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.262+0000 7f0742f8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b6008070 0x5624b63b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.262+0000 7f0742f8d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f07381af630 con 0x5624b5fe7390 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.262+0000 7f0742f8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f07381b1170 con 0x5624b5fdf1b0 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.262+0000 7f0742f8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f07381afd00 con 0x5624b6008070 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.262+0000 7f073659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.262+0000 7f073659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36704/0 (socket says 172.21.15.3:36704) 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.262+0000 7f073659c640 1 -- 172.21.15.3:0/4057843241 learned_addr learned my addr 172.21.15.3:0/4057843241 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.262+0000 7f0735d9b640 1 --2- 172.21.15.3:0/4057843241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b6008070 0x5624b63b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.262+0000 7f0735d9b640 1 --2- 172.21.15.3:0/4057843241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b5fdf1b0 0x5624b63bc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.263+0000 7f073659c640 1 -- 172.21.15.3:0/4057843241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b6008070 msgr2=0x5624b63b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.263+0000 7f073659c640 1 --2- 172.21.15.3:0/4057843241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b6008070 0x5624b63b7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.263+0000 7f073659c640 1 -- 172.21.15.3:0/4057843241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b5fdf1b0 msgr2=0x5624b63bc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.263+0000 7f073659c640 1 --2- 172.21.15.3:0/4057843241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b5fdf1b0 0x5624b63bc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.263+0000 7f073659c640 1 -- 172.21.15.3:0/4057843241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b63b2550 con 0x5624b5fe7390 2024-09-21T06:15:51.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.263+0000 7f073659c640 1 --2- 172.21.15.3:0/4057843241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63c7030 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0728000c00 tx=0x7f072800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203b7dd14294f5c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:51.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.263+0000 7f071ffff640 1 -- 172.21.15.3:0/4057843241 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728029070 con 0x5624b5fe7390 2024-09-21T06:15:51.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.263+0000 7f071ffff640 1 -- 172.21.15.3:0/4057843241 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0728003e10 con 0x5624b5fe7390 2024-09-21T06:15:51.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.263+0000 7f071ffff640 1 -- 172.21.15.3:0/4057843241 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728011770 con 0x5624b5fe7390 2024-09-21T06:15:51.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.264+0000 7f0742f8d740 1 -- 172.21.15.3:0/4057843241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 msgr2=0x5624b63c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:51.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.264+0000 7f0742f8d740 1 --2- 172.21.15.3:0/4057843241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63c7030 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0728000c00 tx=0x7f072800b390 comp rx=0 tx=0).stop 2024-09-21T06:15:51.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.264+0000 7f0742f8d740 1 -- 172.21.15.3:0/4057843241 shutdown_connections 2024-09-21T06:15:51.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.264+0000 7f0742f8d740 1 --2- 172.21.15.3:0/4057843241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b6008070 0x5624b63b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.264+0000 7f0742f8d740 1 --2- 172.21.15.3:0/4057843241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63c7030 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.264+0000 7f0742f8d740 1 --2- 172.21.15.3:0/4057843241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b5fdf1b0 0x5624b63bc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.264+0000 7f0742f8d740 1 -- 172.21.15.3:0/4057843241 >> 172.21.15.3:0/4057843241 conn(0x5624b5ffc460 msgr2=0x5624b63bc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.264+0000 7f0742f8d740 1 -- 172.21.15.3:0/4057843241 shutdown_connections 2024-09-21T06:15:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.264+0000 7f0742f8d740 1 -- 172.21.15.3:0/4057843241 wait complete. 2024-09-21T06:15:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.265+0000 7f0742f8d740 1 -- start start 2024-09-21T06:15:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.265+0000 7f0742f8d740 1 Processor -- start 2024-09-21T06:15:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.265+0000 7f0742f8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b5fdf1b0 0x5624b63cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.265+0000 7f0742f8d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.265+0000 7f0742f8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b6008070 0x5624b63ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.265+0000 7f0742f8d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f07381b0aa0 con 0x5624b5fe7390 2024-09-21T06:15:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.265+0000 7f0742f8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f07381b03d0 con 0x5624b6008070 2024-09-21T06:15:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.265+0000 7f0742f8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f07381ad610 con 0x5624b5fdf1b0 2024-09-21T06:15:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.265+0000 7f0742f8d740 10 client.0 _ll_register_callbacks cb 0x7f072800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.265+0000 7f073659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.265+0000 7f073659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36720/0 (socket says 172.21.15.3:36720) 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.265+0000 7f073659c640 1 -- 172.21.15.3:0/2670407967 learned_addr learned my addr 172.21.15.3:0/2670407967 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.265+0000 7f0735d9b640 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b5fdf1b0 0x5624b63cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f0735d9b640 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b6008070 0x5624b63ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f073659c640 1 -- 172.21.15.3:0/2670407967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b5fdf1b0 msgr2=0x5624b63cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f073659c640 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b5fdf1b0 0x5624b63cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f073659c640 1 -- 172.21.15.3:0/2670407967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b6008070 msgr2=0x5624b63ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f073659c640 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b6008070 0x5624b63ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f073659c640 1 -- 172.21.15.3:0/2670407967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b64366c0 con 0x5624b5fe7390 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f0735d9b640 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b6008070 0x5624b63ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f073659c640 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63ce4a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f07280041c0 tx=0x7f0728024d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4999dd7502ce34ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f071dffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f071dffb640 1 -- 172.21.15.3:0/2670407967 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728005450 con 0x5624b5fe7390 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f071dffb640 1 -- 172.21.15.3:0/2670407967 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f0728024470 con 0x5624b5fe7390 2024-09-21T06:15:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f0742f8d740 1 -- 172.21.15.3:0/2670407967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0728003e10 con 0x5624b5fe7390 2024-09-21T06:15:51.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f071dffb640 1 -- 172.21.15.3:0/2670407967 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f07280055f0 con 0x5624b5fe7390 2024-09-21T06:15:51.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f0742f8d740 10 client.15657 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:51.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.266+0000 7f0742f8d740 1 -- 172.21.15.3:0/2670407967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5624b6437950 con 0x5624b5fe7390 2024-09-21T06:15:51.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.267+0000 7f071dffb640 1 -- 172.21.15.3:0/2670407967 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0728003e10 con 0x5624b5fe7390 2024-09-21T06:15:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.267+0000 7f071dffb640 1 -- 172.21.15.3:0/2670407967 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f072802c580 con 0x5624b5fe7390 2024-09-21T06:15:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.267+0000 7f0742f8d740 10 client.15657 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.267+0000 7f0742f8d740 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.267+0000 7f0742f8d740 20 client.15657 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.267+0000 7f0742f8d740 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5624b5f9bda0 0x5624b643a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.267+0000 7f0742f8d740 4 client.15657 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:15:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.267+0000 7f0742f8d740 1 -- 172.21.15.3:0/2670407967 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5624b6323850 con 0x5624b5f9bda0 2024-09-21T06:15:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.267+0000 7f0735d9b640 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5624b5f9bda0 0x5624b643a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f0735d9b640 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5624b5f9bda0 0x5624b643a000 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69f07f6bb0ba8c7c server_cookie=73b6ebd18d8e1dff in_seq=0 out_seq=0 2024-09-21T06:15:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f071dffb640 10 client.15657 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f071dffb640 1 -- 172.21.15.3:0/2670407967 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x5624b6323850 con 0x5624b5f9bda0 2024-09-21T06:15:51.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f071dffb640 10 client.15657 handle_command_reply: tid=0 2024-09-21T06:15:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f0742f8d740 1 client.15657 shutdown 2024-09-21T06:15:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f0742f8d740 1 -- 172.21.15.3:0/2670407967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 msgr2=0x5624b63ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f0742f8d740 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63ce4a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f07280041c0 tx=0x7f0728024d10 comp rx=0 tx=0).stop 2024-09-21T06:15:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f0742f8d740 1 -- 172.21.15.3:0/2670407967 shutdown_connections 2024-09-21T06:15:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f0742f8d740 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5624b5f9bda0 0x5624b643a000 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f0742f8d740 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624b6008070 0x5624b63ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f0742f8d740 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5624b5fe7390 0x5624b63ce4a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f0742f8d740 1 --2- 172.21.15.3:0/2670407967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624b5fdf1b0 0x5624b63cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f0742f8d740 1 -- 172.21.15.3:0/2670407967 >> 172.21.15.3:0/2670407967 conn(0x5624b5ffc460 msgr2=0x5624b63b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f071dffb640 0 client.15657 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.268+0000 7f0742f8d740 1 -- 172.21.15.3:0/2670407967 shutdown_connections 2024-09-21T06:15:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.269+0000 7f0742f8d740 1 -- 172.21.15.3:0/2670407967 wait complete. 2024-09-21T06:15:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.269+0000 7f0742f8d740 20 client.15657 trim_cache size 0 max 16384 2024-09-21T06:15:51.275 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:15:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:15:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:15:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:15:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:15:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:15:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:15:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:15:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:15:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:15:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:15:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:15:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:15:51.276 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:15:51.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.269+0000 7f073ca19640 1 -- 172.21.15.3:0/2394237107 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f07240b6680 msgr2=0x7f07240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:51.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.269+0000 7f073ca19640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f07240b6680 0x7f07240b8b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f07300ad3e0 tx=0x7f073000cf50 comp rx=0 tx=0).stop 2024-09-21T06:15:51.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.269+0000 7f073ca19640 1 -- 172.21.15.3:0/2394237107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0738108d20 msgr2=0x7f073819bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:51.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.269+0000 7f073ca19640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0738108d20 0x7f073819bcf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f073000eaa0 tx=0x7f073000ef60 comp rx=0 tx=0).stop 2024-09-21T06:15:51.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.269+0000 7f073ca19640 1 -- 172.21.15.3:0/2394237107 shutdown_connections 2024-09-21T06:15:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.269+0000 7f073ca19640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f07240b6680 0x7f07240b8b90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.269+0000 7f073ca19640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0738108d20 0x7f073819bcf0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.269+0000 7f073ca19640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07381083d0 0x7f073819b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.269+0000 7f073ca19640 1 --2- 172.21.15.3:0/2394237107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0738102a90 0x7f073819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.269+0000 7f073ca19640 1 -- 172.21.15.3:0/2394237107 >> 172.21.15.3:0/2394237107 conn(0x7f0738079730 msgr2=0x7f0738106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.270+0000 7f073ca19640 1 -- 172.21.15.3:0/2394237107 shutdown_connections 2024-09-21T06:15:51.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:51.270+0000 7f073ca19640 1 -- 172.21.15.3:0/2394237107 wait complete. 2024-09-21T06:15:51.949 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:15:51.950 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:15:52.303 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.799+0000 7f3288970640 1 -- 172.21.15.3:0/257263005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3284108cf0 msgr2=0x7f328410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.799+0000 7f3288970640 1 --2- 172.21.15.3:0/257263005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3284108cf0 0x7f328410f520 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f327c009ca0 tx=0x7f327c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.799+0000 7f3288970640 1 -- 172.21.15.3:0/257263005 shutdown_connections 2024-09-21T06:15:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.799+0000 7f3288970640 1 --2- 172.21.15.3:0/257263005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3284108cf0 0x7f328410f520 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.799+0000 7f3288970640 1 --2- 172.21.15.3:0/257263005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32841083a0 0x7f32841087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.799+0000 7f3288970640 1 --2- 172.21.15.3:0/257263005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3284102a60 0x7f3284102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.799+0000 7f3288970640 1 -- 172.21.15.3:0/257263005 >> 172.21.15.3:0/257263005 conn(0x7f3284079730 msgr2=0x7f3284079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.799+0000 7f3288970640 1 -- 172.21.15.3:0/257263005 shutdown_connections 2024-09-21T06:15:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.799+0000 7f3288970640 1 -- 172.21.15.3:0/257263005 wait complete. 2024-09-21T06:15:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.800+0000 7f3288970640 1 Processor -- start 2024-09-21T06:15:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.800+0000 7f3288970640 1 -- start start 2024-09-21T06:15:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f3288970640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3284102a60 0x7f3284199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f3288970640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32841083a0 0x7f3284199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f3288970640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3284108cf0 0x7f328419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f3288970640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3284112960 con 0x7f3284102a60 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f3288970640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f32841127e0 con 0x7f3284108cf0 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f3288970640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3284112ae0 con 0x7f32841083a0 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f328259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3284102a60 0x7f3284199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f328259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3284102a60 0x7f3284199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36732/0 (socket says 172.21.15.3:36732) 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f328259c640 1 -- 172.21.15.3:0/2680608304 learned_addr learned my addr 172.21.15.3:0/2680608304 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f3281d9b640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32841083a0 0x7f3284199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f3282d9d640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3284108cf0 0x7f328419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f328259c640 1 -- 172.21.15.3:0/2680608304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32841083a0 msgr2=0x7f3284199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f328259c640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32841083a0 0x7f3284199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f328259c640 1 -- 172.21.15.3:0/2680608304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3284108cf0 msgr2=0x7f328419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f328259c640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3284108cf0 0x7f328419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f328259c640 1 -- 172.21.15.3:0/2680608304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328419f2e0 con 0x7f3284102a60 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.801+0000 7f3282d9d640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3284108cf0 0x7f328419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.802+0000 7f328259c640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3284102a60 0x7f3284199370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3274009860 tx=0x7f3274009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.802+0000 7f32737fe640 1 -- 172.21.15.3:0/2680608304 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274005020 con 0x7f3284102a60 2024-09-21T06:15:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.802+0000 7f32737fe640 1 -- 172.21.15.3:0/2680608304 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32740047a0 con 0x7f3284102a60 2024-09-21T06:15:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.802+0000 7f3288970640 1 -- 172.21.15.3:0/2680608304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328419f5d0 con 0x7f3284102a60 2024-09-21T06:15:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.802+0000 7f32737fe640 1 -- 172.21.15.3:0/2680608304 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274005020 con 0x7f3284102a60 2024-09-21T06:15:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.802+0000 7f3288970640 1 -- 172.21.15.3:0/2680608304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f328406ae60 con 0x7f3284102a60 2024-09-21T06:15:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.804+0000 7f3288970640 1 -- 172.21.15.3:0/2680608304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3284102e70 con 0x7f3284102a60 2024-09-21T06:15:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.805+0000 7f32737fe640 1 -- 172.21.15.3:0/2680608304 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 154636+0+0 (secure 0 0 0) 0x7f32740051c0 con 0x7f3284102a60 2024-09-21T06:15:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.806+0000 7f32737fe640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f326c0b6880 0x7f326c0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.807+0000 7f32737fe640 1 -- 172.21.15.3:0/2680608304 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f32740e8850 con 0x7f3284102a60 2024-09-21T06:15:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.807+0000 7f3281d9b640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f326c0b6880 0x7f326c0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.808+0000 7f3281d9b640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f326c0b6880 0x7f326c0b8d90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3284105eb0 tx=0x7f327800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:52.811+0000 7f32737fe640 1 -- 172.21.15.3:0/2680608304 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f32740aed80 con 0x7f3284102a60 2024-09-21T06:15:53.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:52 smithi003 ceph-mon[31227]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:15:53.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:52 smithi137 ceph-mon[34464]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:15:53.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:52 smithi204 ceph-mon[34659]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:15:53.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.158+0000 7f3288970640 1 -- 172.21.15.3:0/2680608304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f328406b880 con 0x7f3284102a60 2024-09-21T06:15:53.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.159+0000 7f32737fe640 1 -- 172.21.15.3:0/2680608304 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11962 (secure 0 0 0) 0x7f32740b3160 con 0x7f3284102a60 2024-09-21T06:15:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:15:53.162 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"btime":"2024-09-21T06:15:09:322787+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:15:08.461776+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24670,24674,24679,24691,24704]},"id":1}]} 2024-09-21T06:15:53.162 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 28 2024-09-21T06:15:53.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.162+0000 7f3288970640 1 -- 172.21.15.3:0/2680608304 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f326c0b6880 msgr2=0x7f326c0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:53.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.162+0000 7f3288970640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f326c0b6880 0x7f326c0b8d90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3284105eb0 tx=0x7f327800b390 comp rx=0 tx=0).stop 2024-09-21T06:15:53.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.162+0000 7f3288970640 1 -- 172.21.15.3:0/2680608304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3284102a60 msgr2=0x7f3284199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:53.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.162+0000 7f3288970640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3284102a60 0x7f3284199370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3274009860 tx=0x7f3274009d20 comp rx=0 tx=0).stop 2024-09-21T06:15:53.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.162+0000 7f3288970640 1 -- 172.21.15.3:0/2680608304 shutdown_connections 2024-09-21T06:15:53.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.162+0000 7f3288970640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f326c0b6880 0x7f326c0b8d90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:53.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.162+0000 7f3288970640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32841083a0 0x7f3284199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:53.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.162+0000 7f3288970640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3284108cf0 0x7f328419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:53.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.162+0000 7f3288970640 1 --2- 172.21.15.3:0/2680608304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3284102a60 0x7f3284199370 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:53.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.162+0000 7f3288970640 1 -- 172.21.15.3:0/2680608304 >> 172.21.15.3:0/2680608304 conn(0x7f3284079730 msgr2=0x7f3284104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:53.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.162+0000 7f3288970640 1 -- 172.21.15.3:0/2680608304 shutdown_connections 2024-09-21T06:15:53.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:53.162+0000 7f3288970640 1 -- 172.21.15.3:0/2680608304 wait complete. 2024-09-21T06:15:53.820 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:15:54.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:53 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2680608304' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:54.096 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:54.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:53 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2680608304' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:54.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:53 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2680608304' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:54.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.644+0000 7f008e164640 1 -- 172.21.15.3:0/1058900676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0088102f80 msgr2=0x7f008810fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:54.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.644+0000 7f008e164640 1 --2- 172.21.15.3:0/1058900676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0088102f80 0x7f008810fca0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f007c009900 tx=0x7f007c02a480 comp rx=0 tx=0).stop 2024-09-21T06:15:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.645+0000 7f008e164640 1 -- 172.21.15.3:0/1058900676 shutdown_connections 2024-09-21T06:15:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.645+0000 7f008e164640 1 --2- 172.21.15.3:0/1058900676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00881101e0 0x7f0088112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.645+0000 7f008e164640 1 --2- 172.21.15.3:0/1058900676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0088102f80 0x7f008810fca0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.645+0000 7f008e164640 1 --2- 172.21.15.3:0/1058900676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0088102630 0x7f0088102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.645+0000 7f008e164640 1 -- 172.21.15.3:0/1058900676 >> 172.21.15.3:0/1058900676 conn(0x7f0088079730 msgr2=0x7f0088079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.645+0000 7f008e164640 1 -- 172.21.15.3:0/1058900676 shutdown_connections 2024-09-21T06:15:54.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.645+0000 7f008e164640 1 -- 172.21.15.3:0/1058900676 wait complete. 2024-09-21T06:15:54.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.646+0000 7f008e164640 1 Processor -- start 2024-09-21T06:15:54.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.646+0000 7f008e164640 1 -- start start 2024-09-21T06:15:54.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f008e164640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0088102630 0x7f008819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f008e164640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0088102f80 0x7f008819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f008e164640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00881101e0 0x7f00881a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f008e164640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0088115ca0 con 0x7f00881101e0 2024-09-21T06:15:54.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f008e164640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0088115b20 con 0x7f0088102f80 2024-09-21T06:15:54.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f008e164640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0088115e20 con 0x7f0088102630 2024-09-21T06:15:54.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f0087fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00881101e0 0x7f00881a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f0087fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00881101e0 0x7f00881a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36746/0 (socket says 172.21.15.3:36746) 2024-09-21T06:15:54.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f0087fff640 1 -- 172.21.15.3:0/4188254205 learned_addr learned my addr 172.21.15.3:0/4188254205 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:54.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f00877fe640 1 --2- 172.21.15.3:0/4188254205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0088102630 0x7f008819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f0087fff640 1 -- 172.21.15.3:0/4188254205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0088102630 msgr2=0x7f008819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:54.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f0087fff640 1 --2- 172.21.15.3:0/4188254205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0088102630 0x7f008819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f0087fff640 1 -- 172.21.15.3:0/4188254205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0088102f80 msgr2=0x7f008819cc90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:15:54.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f0087fff640 1 --2- 172.21.15.3:0/4188254205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0088102f80 0x7f008819cc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.647+0000 7f0087fff640 1 -- 172.21.15.3:0/4188254205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00881a2660 con 0x7f00881101e0 2024-09-21T06:15:54.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.648+0000 7f0087fff640 1 --2- 172.21.15.3:0/4188254205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00881101e0 0x7f00881a1f30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0080009fd0 tx=0x7f008000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:54.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.648+0000 7f0084ff9640 1 -- 172.21.15.3:0/4188254205 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080017070 con 0x7f00881101e0 2024-09-21T06:15:54.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.648+0000 7f0084ff9640 1 -- 172.21.15.3:0/4188254205 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0080003e20 con 0x7f00881101e0 2024-09-21T06:15:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.648+0000 7f0084ff9640 1 -- 172.21.15.3:0/4188254205 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080010460 con 0x7f00881101e0 2024-09-21T06:15:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.648+0000 7f008e164640 1 -- 172.21.15.3:0/4188254205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00881a2950 con 0x7f00881101e0 2024-09-21T06:15:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.648+0000 7f008e164640 1 -- 172.21.15.3:0/4188254205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00881a90b0 con 0x7f00881101e0 2024-09-21T06:15:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.652+0000 7f00946d8740 1 Processor -- start 2024-09-21T06:15:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.652+0000 7f0084ff9640 1 -- 172.21.15.3:0/4188254205 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 154636+0+0 (secure 0 0 0) 0x7f0080010640 con 0x7f00881101e0 2024-09-21T06:15:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.652+0000 7f00946d8740 1 -- start start 2024-09-21T06:15:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.653+0000 7f00946d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb029070 0x55f8eb3d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.653+0000 7f00946d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb0001b0 0x55f8eb3d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.653+0000 7f00946d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb008390 0x55f8eb3d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.653+0000 7f00946d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f8eb346ea0 con 0x55f8eb0001b0 2024-09-21T06:15:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.653+0000 7f00946d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f8eb3362d0 con 0x55f8eb029070 2024-09-21T06:15:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.653+0000 7f00946d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f8eb33a0e0 con 0x55f8eb008390 2024-09-21T06:15:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.653+0000 7f00877fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb0001b0 0x55f8eb3d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.653+0000 7f0086ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb008390 0x55f8eb3d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.653+0000 7f00877fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb0001b0 0x55f8eb3d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36750/0 (socket says 172.21.15.3:36750) 2024-09-21T06:15:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.653+0000 7f00877fe640 1 -- 172.21.15.3:0/4070179556 learned_addr learned my addr 172.21.15.3:0/4070179556 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.653+0000 7f0086ffd640 1 --2- 172.21.15.3:0/4070179556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb029070 0x55f8eb3d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.654+0000 7f0084ff9640 1 --2- 172.21.15.3:0/4188254205 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f00700b6700 0x7f00700b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.654+0000 7f00877fe640 1 -- 172.21.15.3:0/4070179556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb008390 msgr2=0x55f8eb3d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.654+0000 7f00877fe640 1 --2- 172.21.15.3:0/4070179556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb008390 0x55f8eb3d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.654+0000 7f00877fe640 1 -- 172.21.15.3:0/4070179556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb029070 msgr2=0x55f8eb3d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.654+0000 7f00877fe640 1 --2- 172.21.15.3:0/4070179556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb029070 0x55f8eb3d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.654+0000 7f00877fe640 1 -- 172.21.15.3:0/4070179556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8eb094650 con 0x55f8eb0001b0 2024-09-21T06:15:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.654+0000 7f0086ffd640 1 --2- 172.21.15.3:0/4070179556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb008390 0x55f8eb3d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.654+0000 7f0084ff9640 1 -- 172.21.15.3:0/4188254205 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f00800e7470 con 0x7f00881101e0 2024-09-21T06:15:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.654+0000 7f0087fff640 1 --2- 172.21.15.3:0/4188254205 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f00700b6700 0x7f00700b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.654+0000 7f00877fe640 1 --2- 172.21.15.3:0/4070179556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb0001b0 0x55f8eb3d0be0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f007800bcd0 tx=0x7f00780107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8481513479f1aa23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.654+0000 7f006dffb640 1 -- 172.21.15.3:0/4070179556 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078005220 con 0x55f8eb0001b0 2024-09-21T06:15:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.654+0000 7f006dffb640 1 -- 172.21.15.3:0/4070179556 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0078004c50 con 0x55f8eb0001b0 2024-09-21T06:15:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.654+0000 7f006dffb640 1 -- 172.21.15.3:0/4070179556 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00780116a0 con 0x55f8eb0001b0 2024-09-21T06:15:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.656+0000 7f00946d8740 1 -- 172.21.15.3:0/4070179556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb0001b0 msgr2=0x55f8eb3d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.656+0000 7f00946d8740 1 --2- 172.21.15.3:0/4070179556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb0001b0 0x55f8eb3d0be0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f007800bcd0 tx=0x7f00780107a0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.656+0000 7f0087fff640 1 --2- 172.21.15.3:0/4188254205 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f00700b6700 0x7f00700b8c10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f00800a9870 tx=0x7f0080004020 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.656+0000 7f00946d8740 1 -- 172.21.15.3:0/4070179556 shutdown_connections 2024-09-21T06:15:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.656+0000 7f00946d8740 1 --2- 172.21.15.3:0/4070179556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb008390 0x55f8eb3d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.656+0000 7f00946d8740 1 --2- 172.21.15.3:0/4070179556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb0001b0 0x55f8eb3d0be0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.656+0000 7f00946d8740 1 --2- 172.21.15.3:0/4070179556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb029070 0x55f8eb3d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.656+0000 7f00946d8740 1 -- 172.21.15.3:0/4070179556 >> 172.21.15.3:0/4070179556 conn(0x55f8eb01d460 msgr2=0x55f8eb3ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:54.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.656+0000 7f00946d8740 1 -- 172.21.15.3:0/4070179556 shutdown_connections 2024-09-21T06:15:54.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.656+0000 7f00946d8740 1 -- 172.21.15.3:0/4070179556 wait complete. 2024-09-21T06:15:54.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.657+0000 7f00946d8740 1 -- start start 2024-09-21T06:15:54.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.657+0000 7f00946d8740 1 Processor -- start 2024-09-21T06:15:54.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.657+0000 7f00946d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb0001b0 0x55f8eb3dbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.657+0000 7f00946d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb008390 0x55f8eb3dc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f00946d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb029070 0x55f8eb3e81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f00946d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f8eb197700 con 0x55f8eb029070 2024-09-21T06:15:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f00946d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f8eb2d1160 con 0x55f8eb008390 2024-09-21T06:15:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f00946d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f8eb3dee80 con 0x55f8eb0001b0 2024-09-21T06:15:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f00946d8740 10 client.0 _ll_register_callbacks cb 0x55f8eb2ea9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f0086ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb029070 0x55f8eb3e81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f0086ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb029070 0x55f8eb3e81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36758/0 (socket says 172.21.15.3:36758) 2024-09-21T06:15:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f0086ffd640 1 -- 172.21.15.3:0/1580018439 learned_addr learned my addr 172.21.15.3:0/1580018439 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f00877fe640 1 --2- 172.21.15.3:0/1580018439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb008390 0x55f8eb3dc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f0086ffd640 1 --2- 172.21.15.3:0/1580018439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb0001b0 0x55f8eb3dbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f0086ffd640 1 -- 172.21.15.3:0/1580018439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb0001b0 msgr2=0x55f8eb3dbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f0086ffd640 1 --2- 172.21.15.3:0/1580018439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb0001b0 0x55f8eb3dbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f0086ffd640 1 -- 172.21.15.3:0/1580018439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb008390 msgr2=0x55f8eb3dc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f0086ffd640 1 --2- 172.21.15.3:0/1580018439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb008390 0x55f8eb3dc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f0086ffd640 1 -- 172.21.15.3:0/1580018439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8eb2fec90 con 0x55f8eb029070 2024-09-21T06:15:54.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f0086ffd640 1 --2- 172.21.15.3:0/1580018439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb029070 0x55f8eb3e81a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0088103f70 tx=0x7f007c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a584ab16be3c65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:54.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.658+0000 7f0066ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:54.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.659+0000 7f0066ffd640 1 -- 172.21.15.3:0/1580018439 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c035070 con 0x55f8eb029070 2024-09-21T06:15:54.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.659+0000 7f0066ffd640 1 -- 172.21.15.3:0/1580018439 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f007c03c670 con 0x55f8eb029070 2024-09-21T06:15:54.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.659+0000 7f00946d8740 1 -- 172.21.15.3:0/1580018439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0078004c50 con 0x55f8eb029070 2024-09-21T06:15:54.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.659+0000 7f0066ffd640 1 -- 172.21.15.3:0/1580018439 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f007c061050 con 0x55f8eb029070 2024-09-21T06:15:54.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.659+0000 7f00946d8740 10 client.15675 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:54.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.659+0000 7f00946d8740 1 -- 172.21.15.3:0/1580018439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f8eb3f3400 con 0x55f8eb029070 2024-09-21T06:15:54.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.659+0000 7f0066ffd640 1 -- 172.21.15.3:0/1580018439 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0078004c50 con 0x55f8eb029070 2024-09-21T06:15:54.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.660+0000 7f0066ffd640 1 -- 172.21.15.3:0/1580018439 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f007c03a040 con 0x55f8eb029070 2024-09-21T06:15:54.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.660+0000 7f00946d8740 10 client.15675 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:54.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.660+0000 7f00946d8740 10 client.15675 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:15:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.660+0000 7f00946d8740 20 client.15675 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.660+0000 7f00946d8740 1 --2- 172.21.15.3:0/1580018439 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f8eafe79f0 0x55f8eb3f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.660+0000 7f00946d8740 4 client.15675 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.660+0000 7f00946d8740 1 -- 172.21.15.3:0/1580018439 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8eb348df0 con 0x55f8eafe79f0 2024-09-21T06:15:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.660+0000 7f00877fe640 1 --2- 172.21.15.3:0/1580018439 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f8eafe79f0 0x55f8eb3f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.661+0000 7f0066ffd640 10 client.15675 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:15:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.661+0000 7f00877fe640 1 --2- 172.21.15.3:0/1580018439 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f8eafe79f0 0x55f8eb3f68a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c884860946e8b7c5 server_cookie=fec15c6e16351ae5 in_seq=0 out_seq=0 2024-09-21T06:15:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.661+0000 7f0066ffd640 1 -- 172.21.15.3:0/1580018439 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8eb348df0 con 0x55f8eafe79f0 2024-09-21T06:15:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.661+0000 7f0066ffd640 10 client.15675 handle_command_reply: tid=0 2024-09-21T06:15:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.661+0000 7f00946d8740 1 client.15675 shutdown 2024-09-21T06:15:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.662+0000 7f00946d8740 1 -- 172.21.15.3:0/1580018439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb029070 msgr2=0x55f8eb3e81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.662+0000 7f00946d8740 1 --2- 172.21.15.3:0/1580018439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb029070 0x55f8eb3e81a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0088103f70 tx=0x7f007c009260 comp rx=0 tx=0).stop 2024-09-21T06:15:54.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.662+0000 7f00946d8740 1 -- 172.21.15.3:0/1580018439 shutdown_connections 2024-09-21T06:15:54.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.662+0000 7f00946d8740 1 --2- 172.21.15.3:0/1580018439 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f8eafe79f0 0x55f8eb3f68a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.662+0000 7f0066ffd640 0 client.15675 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:15:54.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.662+0000 7f00946d8740 1 --2- 172.21.15.3:0/1580018439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb029070 0x55f8eb3e81a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.662+0000 7f00946d8740 1 --2- 172.21.15.3:0/1580018439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb008390 0x55f8eb3dc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.662+0000 7f00946d8740 1 --2- 172.21.15.3:0/1580018439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb0001b0 0x55f8eb3dbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.662+0000 7f00946d8740 1 -- 172.21.15.3:0/1580018439 >> 172.21.15.3:0/1580018439 conn(0x55f8eb01d460 msgr2=0x55f8eb3d3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:54.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.662+0000 7f00946d8740 1 -- 172.21.15.3:0/1580018439 shutdown_connections 2024-09-21T06:15:54.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.662+0000 7f00946d8740 1 -- 172.21.15.3:0/1580018439 wait complete. 2024-09-21T06:15:54.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.662+0000 7f00946d8740 20 client.15675 trim_cache size 0 max 16384 2024-09-21T06:15:54.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.688+0000 7f00946d8740 1 Processor -- start 2024-09-21T06:15:54.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.688+0000 7f00946d8740 1 -- start start 2024-09-21T06:15:54.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.688+0000 7f00946d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb0001b0 0x55f8eb3dc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.688+0000 7f00946d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb008390 0x55f8eb3ddcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.688+0000 7f00946d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eafbcda0 0x55f8eb3d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.688+0000 7f00946d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0088071120 con 0x55f8eafbcda0 2024-09-21T06:15:54.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.688+0000 7f00946d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0088072c60 con 0x55f8eb0001b0 2024-09-21T06:15:54.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.688+0000 7f00946d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00880717f0 con 0x55f8eb008390 2024-09-21T06:15:54.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.689+0000 7f0086ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eafbcda0 0x55f8eb3d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.689+0000 7f0086ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eafbcda0 0x55f8eb3d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36766/0 (socket says 172.21.15.3:36766) 2024-09-21T06:15:54.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.689+0000 7f0086ffd640 1 -- 172.21.15.3:0/351065935 learned_addr learned my addr 172.21.15.3:0/351065935 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:54.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.689+0000 7f0086ffd640 1 --2- 172.21.15.3:0/351065935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb0001b0 0x55f8eb3dc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.689+0000 7f00877fe640 1 --2- 172.21.15.3:0/351065935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb008390 0x55f8eb3ddcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.689+0000 7f0086ffd640 1 -- 172.21.15.3:0/351065935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb008390 msgr2=0x55f8eb3ddcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:54.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.689+0000 7f0086ffd640 1 --2- 172.21.15.3:0/351065935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb008390 0x55f8eb3ddcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.689+0000 7f0086ffd640 1 -- 172.21.15.3:0/351065935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb0001b0 msgr2=0x55f8eb3dc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:54.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.689+0000 7f0086ffd640 1 --2- 172.21.15.3:0/351065935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb0001b0 0x55f8eb3dc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.689+0000 7f0086ffd640 1 -- 172.21.15.3:0/351065935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8eb3ce7c0 con 0x55f8eafbcda0 2024-09-21T06:15:54.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.689+0000 7f0086ffd640 1 --2- 172.21.15.3:0/351065935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eafbcda0 0x55f8eb3d1de0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f007c005f70 tx=0x7f007c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ef14af4cb286ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:54.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.689+0000 7f006d7fa640 1 -- 172.21.15.3:0/351065935 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c035040 con 0x55f8eafbcda0 2024-09-21T06:15:54.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.690+0000 7f006d7fa640 1 -- 172.21.15.3:0/351065935 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f007c031590 con 0x55f8eafbcda0 2024-09-21T06:15:54.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.690+0000 7f006d7fa640 1 -- 172.21.15.3:0/351065935 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c003850 con 0x55f8eafbcda0 2024-09-21T06:15:54.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.690+0000 7f00946d8740 1 -- 172.21.15.3:0/351065935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eafbcda0 msgr2=0x55f8eb3d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:54.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.690+0000 7f00946d8740 1 --2- 172.21.15.3:0/351065935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eafbcda0 0x55f8eb3d1de0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f007c005f70 tx=0x7f007c02a480 comp rx=0 tx=0).stop 2024-09-21T06:15:54.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.690+0000 7f00946d8740 1 -- 172.21.15.3:0/351065935 shutdown_connections 2024-09-21T06:15:54.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.690+0000 7f00946d8740 1 --2- 172.21.15.3:0/351065935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb008390 0x55f8eb3ddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.690+0000 7f00946d8740 1 --2- 172.21.15.3:0/351065935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eafbcda0 0x55f8eb3d1de0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.690+0000 7f00946d8740 1 --2- 172.21.15.3:0/351065935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eb0001b0 0x55f8eb3dc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.691+0000 7f00946d8740 1 -- 172.21.15.3:0/351065935 >> 172.21.15.3:0/351065935 conn(0x55f8eb01d460 msgr2=0x55f8eb3d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:54.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.691+0000 7f00946d8740 1 -- 172.21.15.3:0/351065935 shutdown_connections 2024-09-21T06:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.691+0000 7f00946d8740 1 -- 172.21.15.3:0/351065935 wait complete. 2024-09-21T06:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.691+0000 7f00946d8740 1 -- start start 2024-09-21T06:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.691+0000 7f00946d8740 1 Processor -- start 2024-09-21T06:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f00946d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eafbcda0 0x55f8eb3e0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f00946d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb0001b0 0x55f8eb3e0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f00946d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb008390 0x55f8eb3e0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f00946d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0088072590 con 0x55f8eb008390 2024-09-21T06:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f00946d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0088071ec0 con 0x55f8eafbcda0 2024-09-21T06:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f00946d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f008806f120 con 0x55f8eb0001b0 2024-09-21T06:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f00946d8740 10 client.0 _ll_register_callbacks cb 0x7f007c031950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f0086ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb008390 0x55f8eb3e0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f0086ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb008390 0x55f8eb3e0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36782/0 (socket says 172.21.15.3:36782) 2024-09-21T06:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f0086ffd640 1 -- 172.21.15.3:0/749259377 learned_addr learned my addr 172.21.15.3:0/749259377 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f00877fe640 1 --2- 172.21.15.3:0/749259377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb0001b0 0x55f8eb3e0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f0086ffd640 1 --2- 172.21.15.3:0/749259377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eafbcda0 0x55f8eb3e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f0086ffd640 1 -- 172.21.15.3:0/749259377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb0001b0 msgr2=0x55f8eb3e0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f0086ffd640 1 --2- 172.21.15.3:0/749259377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb0001b0 0x55f8eb3e0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f0086ffd640 1 -- 172.21.15.3:0/749259377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eafbcda0 msgr2=0x55f8eb3e0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f0086ffd640 1 --2- 172.21.15.3:0/749259377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eafbcda0 0x55f8eb3e0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.692+0000 7f0086ffd640 1 -- 172.21.15.3:0/749259377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8eb460260 con 0x55f8eb008390 2024-09-21T06:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.693+0000 7f0086ffd640 1 --2- 172.21.15.3:0/749259377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb008390 0x55f8eb3e0aa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f007c004500 tx=0x7f007c004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2900cfeec4a3d3cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.693+0000 7f0066ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.693+0000 7f0066ffd640 1 -- 172.21.15.3:0/749259377 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c035040 con 0x55f8eb008390 2024-09-21T06:15:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.693+0000 7f00946d8740 1 -- 172.21.15.3:0/749259377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f007c031590 con 0x55f8eb008390 2024-09-21T06:15:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.693+0000 7f0066ffd640 1 -- 172.21.15.3:0/749259377 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f007c03d5f0 con 0x55f8eb008390 2024-09-21T06:15:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.693+0000 7f0066ffd640 1 -- 172.21.15.3:0/749259377 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f007c074050 con 0x55f8eb008390 2024-09-21T06:15:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.693+0000 7f00946d8740 10 client.15681 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.693+0000 7f00946d8740 1 -- 172.21.15.3:0/749259377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f8eb461400 con 0x55f8eb008390 2024-09-21T06:15:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.694+0000 7f0066ffd640 1 -- 172.21.15.3:0/749259377 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f007c031590 con 0x55f8eb008390 2024-09-21T06:15:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.694+0000 7f0066ffd640 1 -- 172.21.15.3:0/749259377 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f007c077020 con 0x55f8eb008390 2024-09-21T06:15:54.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.694+0000 7f00946d8740 10 client.15681 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:54.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.694+0000 7f00946d8740 10 client.15681 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:15:54.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.694+0000 7f00946d8740 20 client.15681 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:54.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.694+0000 7f00946d8740 1 --2- 172.21.15.3:0/749259377 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f8eb461b10 0x55f8eb463f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.694+0000 7f00946d8740 4 client.15681 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:15:54.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.694+0000 7f00946d8740 1 -- 172.21.15.3:0/749259377 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8eb3448a0 con 0x55f8eb461b10 2024-09-21T06:15:54.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.694+0000 7f00877fe640 1 --2- 172.21.15.3:0/749259377 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f8eb461b10 0x55f8eb463f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f0066ffd640 10 client.15681 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:15:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f00877fe640 1 --2- 172.21.15.3:0/749259377 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f8eb461b10 0x55f8eb463f30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=acba75bd582c152 server_cookie=736fe41f20158c9d in_seq=0 out_seq=0 2024-09-21T06:15:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f0066ffd640 1 -- 172.21.15.3:0/749259377 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8eb3448a0 con 0x55f8eb461b10 2024-09-21T06:15:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f0066ffd640 10 client.15681 handle_command_reply: tid=0 2024-09-21T06:15:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f00946d8740 1 client.15681 shutdown 2024-09-21T06:15:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f00946d8740 1 -- 172.21.15.3:0/749259377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb008390 msgr2=0x55f8eb3e0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f00946d8740 1 --2- 172.21.15.3:0/749259377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb008390 0x55f8eb3e0aa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f007c004500 tx=0x7f007c004400 comp rx=0 tx=0).stop 2024-09-21T06:15:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f00946d8740 1 -- 172.21.15.3:0/749259377 shutdown_connections 2024-09-21T06:15:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f00946d8740 1 --2- 172.21.15.3:0/749259377 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f8eb461b10 0x55f8eb463f30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f00946d8740 1 --2- 172.21.15.3:0/749259377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8eb0001b0 0x55f8eb3e0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f0066ffd640 0 client.15681 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:15:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f00946d8740 1 --2- 172.21.15.3:0/749259377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8eb008390 0x55f8eb3e0aa0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f00946d8740 1 --2- 172.21.15.3:0/749259377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8eafbcda0 0x55f8eb3e0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.695+0000 7f00946d8740 1 -- 172.21.15.3:0/749259377 >> 172.21.15.3:0/749259377 conn(0x55f8eb01d460 msgr2=0x55f8eb3dda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.696+0000 7f00946d8740 1 -- 172.21.15.3:0/749259377 shutdown_connections 2024-09-21T06:15:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.696+0000 7f00946d8740 1 -- 172.21.15.3:0/749259377 wait complete. 2024-09-21T06:15:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.696+0000 7f00946d8740 20 client.15681 trim_cache size 0 max 16384 2024-09-21T06:15:54.710 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:15:54.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.696+0000 7f008e164640 1 -- 172.21.15.3:0/4188254205 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f00700b6700 msgr2=0x7f00700b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:54.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.696+0000 7f008e164640 1 --2- 172.21.15.3:0/4188254205 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f00700b6700 0x7f00700b8c10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f00800a9870 tx=0x7f0080004020 comp rx=0 tx=0).stop 2024-09-21T06:15:54.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.696+0000 7f008e164640 1 -- 172.21.15.3:0/4188254205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00881101e0 msgr2=0x7f00881a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:54.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.696+0000 7f008e164640 1 --2- 172.21.15.3:0/4188254205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00881101e0 0x7f00881a1f30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0080009fd0 tx=0x7f008000b260 comp rx=0 tx=0).stop 2024-09-21T06:15:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.696+0000 7f008e164640 1 -- 172.21.15.3:0/4188254205 shutdown_connections 2024-09-21T06:15:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.696+0000 7f008e164640 1 --2- 172.21.15.3:0/4188254205 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f00700b6700 0x7f00700b8c10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.696+0000 7f008e164640 1 --2- 172.21.15.3:0/4188254205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00881101e0 0x7f00881a1f30 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.696+0000 7f008e164640 1 --2- 172.21.15.3:0/4188254205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0088102f80 0x7f008819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.696+0000 7f008e164640 1 --2- 172.21.15.3:0/4188254205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0088102630 0x7f008819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.696+0000 7f008e164640 1 -- 172.21.15.3:0/4188254205 >> 172.21.15.3:0/4188254205 conn(0x7f0088079730 msgr2=0x7f0088109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.697+0000 7f008e164640 1 -- 172.21.15.3:0/4188254205 shutdown_connections 2024-09-21T06:15:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:54.697+0000 7f008e164640 1 -- 172.21.15.3:0/4188254205 wait complete. 2024-09-21T06:15:54.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:54 smithi003 ceph-mon[31227]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:15:55.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:54 smithi137 ceph-mon[34464]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:15:55.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:54 smithi204 ceph-mon[34659]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:15:55.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:15:55.689 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:56.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.195+0000 7fc712d68640 1 -- 172.21.15.3:0/3997025139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c102fe0 msgr2=0x7fc70c10fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:56.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.195+0000 7fc712d68640 1 --2- 172.21.15.3:0/3997025139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c102fe0 0x7fc70c10fd00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009900 tx=0x7fc6f802a480 comp rx=0 tx=0).stop 2024-09-21T06:15:56.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.195+0000 7fc712d68640 1 -- 172.21.15.3:0/3997025139 shutdown_connections 2024-09-21T06:15:56.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.195+0000 7fc712d68640 1 --2- 172.21.15.3:0/3997025139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc70c110240 0x7fc70c112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.195+0000 7fc712d68640 1 --2- 172.21.15.3:0/3997025139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c102fe0 0x7fc70c10fd00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.195+0000 7fc712d68640 1 --2- 172.21.15.3:0/3997025139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc70c102690 0x7fc70c102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.195+0000 7fc712d68640 1 -- 172.21.15.3:0/3997025139 >> 172.21.15.3:0/3997025139 conn(0x7fc70c079750 msgr2=0x7fc70c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:56.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.195+0000 7fc712d68640 1 -- 172.21.15.3:0/3997025139 shutdown_connections 2024-09-21T06:15:56.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.195+0000 7fc712d68640 1 -- 172.21.15.3:0/3997025139 wait complete. 2024-09-21T06:15:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.196+0000 7fc712d68640 1 Processor -- start 2024-09-21T06:15:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.196+0000 7fc712d68640 1 -- start start 2024-09-21T06:15:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.196+0000 7fc712d68640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc70c102690 0x7fc70c19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.196+0000 7fc712d68640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc70c102fe0 0x7fc70c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.196+0000 7fc712d68640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c110240 0x7fc70c1a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.196+0000 7fc712d68640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c115d00 con 0x7fc70c110240 2024-09-21T06:15:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.196+0000 7fc712d68640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c115b80 con 0x7fc70c102fe0 2024-09-21T06:15:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.196+0000 7fc712d68640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c115e80 con 0x7fc70c102690 2024-09-21T06:15:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.197+0000 7fc711305640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c110240 0x7fc70c1a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.197+0000 7fc711305640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c110240 0x7fc70c1a1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48688/0 (socket says 172.21.15.3:48688) 2024-09-21T06:15:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.197+0000 7fc711305640 1 -- 172.21.15.3:0/1999092302 learned_addr learned my addr 172.21.15.3:0/1999092302 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.197+0000 7fc70bfff640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc70c102fe0 0x7fc70c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.197+0000 7fc710b04640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc70c102690 0x7fc70c19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.197+0000 7fc711305640 1 -- 172.21.15.3:0/1999092302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc70c102690 msgr2=0x7fc70c19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.197+0000 7fc711305640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc70c102690 0x7fc70c19c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.197+0000 7fc711305640 1 -- 172.21.15.3:0/1999092302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc70c102fe0 msgr2=0x7fc70c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.197+0000 7fc711305640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc70c102fe0 0x7fc70c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.197+0000 7fc711305640 1 -- 172.21.15.3:0/1999092302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c1a2620 con 0x7fc70c110240 2024-09-21T06:15:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.197+0000 7fc711305640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c110240 0x7fc70c1a1ef0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc70400eb10 tx=0x7fc70400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.198+0000 7fc709ffb640 1 -- 172.21.15.3:0/1999092302 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704004b20 con 0x7fc70c110240 2024-09-21T06:15:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.198+0000 7fc709ffb640 1 -- 172.21.15.3:0/1999092302 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc704004cc0 con 0x7fc70c110240 2024-09-21T06:15:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.198+0000 7fc712d68640 1 -- 172.21.15.3:0/1999092302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c1a2910 con 0x7fc70c110240 2024-09-21T06:15:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.198+0000 7fc709ffb640 1 -- 172.21.15.3:0/1999092302 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704010670 con 0x7fc70c110240 2024-09-21T06:15:56.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.198+0000 7fc712d68640 1 -- 172.21.15.3:0/1999092302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c1a9070 con 0x7fc70c110240 2024-09-21T06:15:56.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.201+0000 7fc709ffb640 1 -- 172.21.15.3:0/1999092302 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 154636+0+0 (secure 0 0 0) 0x7fc704005020 con 0x7fc70c110240 2024-09-21T06:15:56.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.202+0000 7fc7192dc740 1 Processor -- start 2024-09-21T06:15:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.203+0000 7fc7192dc740 1 -- start start 2024-09-21T06:15:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.203+0000 7fc709ffb640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6fc0b6700 0x7fc6fc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.203+0000 7fc7192dc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 0x56012a0066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.203+0000 7fc709ffb640 1 -- 172.21.15.3:0/1999092302 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fc7040e7c20 con 0x7fc70c110240 2024-09-21T06:15:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.203+0000 7fc7192dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c361b0 0x56012a006be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.203+0000 7fc7192dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c3e390 0x56012a007120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.203+0000 7fc7192dc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560129f7cea0 con 0x560129c5f070 2024-09-21T06:15:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.203+0000 7fc7192dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560129f6c2d0 con 0x560129c361b0 2024-09-21T06:15:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.203+0000 7fc7192dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560129f700e0 con 0x560129c3e390 2024-09-21T06:15:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.203+0000 7fc70bfff640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6fc0b6700 0x7fc6fc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.203+0000 7fc710b04640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 0x56012a0066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.203+0000 7fc710b04640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 0x56012a0066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48700/0 (socket says 172.21.15.3:48700) 2024-09-21T06:15:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.203+0000 7fc710b04640 1 -- 172.21.15.3:0/986924553 learned_addr learned my addr 172.21.15.3:0/986924553 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.204+0000 7fc70bfff640 1 --2- 172.21.15.3:0/986924553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c361b0 0x56012a006be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.204+0000 7fc711305640 1 --2- 172.21.15.3:0/986924553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c3e390 0x56012a007120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.204+0000 7fc710b04640 1 -- 172.21.15.3:0/986924553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c3e390 msgr2=0x56012a007120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.204+0000 7fc710b04640 1 --2- 172.21.15.3:0/986924553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c3e390 0x56012a007120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.204+0000 7fc710b04640 1 -- 172.21.15.3:0/986924553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c361b0 msgr2=0x56012a006be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.204+0000 7fc710b04640 1 --2- 172.21.15.3:0/986924553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c361b0 0x56012a006be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.204+0000 7fc710b04640 1 -- 172.21.15.3:0/986924553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560129cca650 con 0x560129c5f070 2024-09-21T06:15:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.204+0000 7fc70bfff640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6fc0b6700 0x7fc6fc0b8c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8005ab0 tx=0x7fc6f80044d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.204+0000 7fc710b04640 1 --2- 172.21.15.3:0/986924553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 0x56012a0066a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc700009d20 tx=0x7fc700011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bf0448de6c133a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.205+0000 7fc6f6ffd640 1 -- 172.21.15.3:0/986924553 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70001e070 con 0x560129c5f070 2024-09-21T06:15:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.205+0000 7fc6f6ffd640 1 -- 172.21.15.3:0/986924553 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc700009e10 con 0x560129c5f070 2024-09-21T06:15:56.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.205+0000 7fc6f6ffd640 1 -- 172.21.15.3:0/986924553 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700018870 con 0x560129c5f070 2024-09-21T06:15:56.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.206+0000 7fc7192dc740 1 -- 172.21.15.3:0/986924553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 msgr2=0x56012a0066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:56.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.206+0000 7fc7192dc740 1 --2- 172.21.15.3:0/986924553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 0x56012a0066a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc700009d20 tx=0x7fc700011810 comp rx=0 tx=0).stop 2024-09-21T06:15:56.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.206+0000 7fc7192dc740 1 -- 172.21.15.3:0/986924553 shutdown_connections 2024-09-21T06:15:56.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.206+0000 7fc7192dc740 1 --2- 172.21.15.3:0/986924553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c3e390 0x56012a007120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.206+0000 7fc7192dc740 1 --2- 172.21.15.3:0/986924553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c361b0 0x56012a006be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.206+0000 7fc7192dc740 1 --2- 172.21.15.3:0/986924553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 0x56012a0066a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.206+0000 7fc7192dc740 1 -- 172.21.15.3:0/986924553 >> 172.21.15.3:0/986924553 conn(0x560129c53460 msgr2=0x56012a004590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:56.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.206+0000 7fc7192dc740 1 -- 172.21.15.3:0/986924553 shutdown_connections 2024-09-21T06:15:56.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.206+0000 7fc7192dc740 1 -- 172.21.15.3:0/986924553 wait complete. 2024-09-21T06:15:56.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.207+0000 7fc7192dc740 1 -- start start 2024-09-21T06:15:56.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.207+0000 7fc7192dc740 1 Processor -- start 2024-09-21T06:15:56.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.208+0000 7fc7192dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c361b0 0x56012a01a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.208+0000 7fc7192dc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c3e390 0x56012a01aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.208+0000 7fc7192dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c5f070 0x56012a026910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.209+0000 7fc7192dc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560129dcd700 con 0x560129c3e390 2024-09-21T06:15:56.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.209+0000 7fc70bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c3e390 0x56012a01aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.209+0000 7fc710b04640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c361b0 0x56012a01a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.209+0000 7fc70bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c3e390 0x56012a01aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48716/0 (socket says 172.21.15.3:48716) 2024-09-21T06:15:56.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.209+0000 7fc70bfff640 1 -- 172.21.15.3:0/1207995831 learned_addr learned my addr 172.21.15.3:0/1207995831 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:56.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.209+0000 7fc711305640 1 --2- 172.21.15.3:0/1207995831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c5f070 0x56012a026910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.209+0000 7fc7192dc740 1 -- 172.21.15.3:0/1207995831 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560129f07160 con 0x560129c5f070 2024-09-21T06:15:56.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.209+0000 7fc7192dc740 1 -- 172.21.15.3:0/1207995831 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56012a01d5b0 con 0x560129c361b0 2024-09-21T06:15:56.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.209+0000 7fc7192dc740 10 client.0 _ll_register_callbacks cb 0x560129f209e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:56.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc711305640 1 -- 172.21.15.3:0/1207995831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c361b0 msgr2=0x56012a01a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:56.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc711305640 1 --2- 172.21.15.3:0/1207995831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c361b0 0x56012a01a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc711305640 1 -- 172.21.15.3:0/1207995831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c3e390 msgr2=0x56012a01aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:56.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc711305640 1 --2- 172.21.15.3:0/1207995831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c3e390 0x56012a01aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc711305640 1 -- 172.21.15.3:0/1207995831 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560129f34c90 con 0x560129c5f070 2024-09-21T06:15:56.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc70bfff640 1 --2- 172.21.15.3:0/1207995831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c3e390 0x56012a01aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:56.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc711305640 1 --2- 172.21.15.3:0/1207995831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c5f070 0x56012a026910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc7040e53b0 tx=0x7fc7040a9950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=210c1d72c092bfcb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:56.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc6ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:15:56.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc6ebfff640 1 -- 172.21.15.3:0/1207995831 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7040ad220 con 0x560129c5f070 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc6ebfff640 1 -- 172.21.15.3:0/1207995831 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fc7040b2070 con 0x560129c5f070 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc7192dc740 1 -- 172.21.15.3:0/1207995831 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc700009e10 con 0x560129c5f070 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc6ebfff640 1 -- 172.21.15.3:0/1207995831 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc7040a9220 con 0x560129c5f070 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc7192dc740 10 client.25304 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.210+0000 7fc7192dc740 1 -- 172.21.15.3:0/1207995831 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56012a029310 con 0x560129c5f070 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.211+0000 7fc6ebfff640 1 -- 172.21.15.3:0/1207995831 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7040ad5c0 con 0x560129c5f070 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.211+0000 7fc6ebfff640 1 -- 172.21.15.3:0/1207995831 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc7040e5b00 con 0x560129c5f070 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.211+0000 7fc7192dc740 10 client.25304 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.211+0000 7fc7192dc740 10 client.25304 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.211+0000 7fc7192dc740 20 client.25304 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.211+0000 7fc7192dc740 1 --2- 172.21.15.3:0/1207995831 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x560129c1d9f0 0x56012a02c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.211+0000 7fc7192dc740 4 client.25304 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.211+0000 7fc7192dc740 1 -- 172.21.15.3:0/1207995831 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc700009e10 con 0x560129c1d9f0 2024-09-21T06:15:56.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.211+0000 7fc710b04640 1 --2- 172.21.15.3:0/1207995831 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x560129c1d9f0 0x56012a02c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.212+0000 7fc6ebfff640 10 client.25304 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:15:56.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.212+0000 7fc710b04640 1 --2- 172.21.15.3:0/1207995831 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x560129c1d9f0 0x56012a02c810 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d5bca9dbd3781953 server_cookie=f1cb3588e12723c in_seq=0 out_seq=0 2024-09-21T06:15:56.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc6ebfff640 1 -- 172.21.15.3:0/1207995831 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc700009e10 con 0x560129c1d9f0 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc6ebfff640 10 client.25304 handle_command_reply: tid=0 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc7192dc740 1 client.25304 shutdown 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc7192dc740 1 -- 172.21.15.3:0/1207995831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c5f070 msgr2=0x56012a026910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc7192dc740 1 --2- 172.21.15.3:0/1207995831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c5f070 0x56012a026910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc7040e53b0 tx=0x7fc7040a9950 comp rx=0 tx=0).stop 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc7192dc740 1 -- 172.21.15.3:0/1207995831 shutdown_connections 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc7192dc740 1 --2- 172.21.15.3:0/1207995831 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x560129c1d9f0 0x56012a02c810 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc7192dc740 1 --2- 172.21.15.3:0/1207995831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c5f070 0x56012a026910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc7192dc740 1 --2- 172.21.15.3:0/1207995831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c3e390 0x56012a01aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc7192dc740 1 --2- 172.21.15.3:0/1207995831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c361b0 0x56012a01a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc7192dc740 1 -- 172.21.15.3:0/1207995831 >> 172.21.15.3:0/1207995831 conn(0x560129c53460 msgr2=0x56012a002bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc6ebfff640 0 client.25304 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc7192dc740 1 -- 172.21.15.3:0/1207995831 shutdown_connections 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc7192dc740 1 -- 172.21.15.3:0/1207995831 wait complete. 2024-09-21T06:15:56.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.213+0000 7fc7192dc740 20 client.25304 trim_cache size 0 max 16384 2024-09-21T06:15:56.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.240+0000 7fc7192dc740 1 Processor -- start 2024-09-21T06:15:56.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.240+0000 7fc7192dc740 1 -- start start 2024-09-21T06:15:56.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.241+0000 7fc7192dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c361b0 0x56012a01a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.241+0000 7fc7192dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c3e390 0x56012a026810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.241+0000 7fc7192dc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 0x56012a0176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.241+0000 7fc7192dc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1ad4b0 con 0x560129c5f070 2024-09-21T06:15:56.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.241+0000 7fc7192dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1aeff0 con 0x560129c3e390 2024-09-21T06:15:56.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.241+0000 7fc7192dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1adb80 con 0x560129c361b0 2024-09-21T06:15:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.241+0000 7fc711305640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 0x56012a0176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.241+0000 7fc70bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c3e390 0x56012a026810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.241+0000 7fc711305640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 0x56012a0176f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48718/0 (socket says 172.21.15.3:48718) 2024-09-21T06:15:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.241+0000 7fc711305640 1 -- 172.21.15.3:0/3348743006 learned_addr learned my addr 172.21.15.3:0/3348743006 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.241+0000 7fc710b04640 1 --2- 172.21.15.3:0/3348743006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c361b0 0x56012a01a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.242+0000 7fc711305640 1 -- 172.21.15.3:0/3348743006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c361b0 msgr2=0x56012a01a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.242+0000 7fc711305640 1 --2- 172.21.15.3:0/3348743006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c361b0 0x56012a01a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.242+0000 7fc711305640 1 -- 172.21.15.3:0/3348743006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c3e390 msgr2=0x56012a026810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:56.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.242+0000 7fc711305640 1 --2- 172.21.15.3:0/3348743006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c3e390 0x56012a026810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.242+0000 7fc711305640 1 -- 172.21.15.3:0/3348743006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012a004ab0 con 0x560129c5f070 2024-09-21T06:15:56.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.242+0000 7fc711305640 1 --2- 172.21.15.3:0/3348743006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 0x56012a0176f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc7040049d0 tx=0x7fc704004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cf0e1ea41674eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:56.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.242+0000 7fc6f67fc640 1 -- 172.21.15.3:0/3348743006 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7040ad220 con 0x560129c5f070 2024-09-21T06:15:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.242+0000 7fc6f67fc640 1 -- 172.21.15.3:0/3348743006 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7040ad5c0 con 0x560129c5f070 2024-09-21T06:15:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.242+0000 7fc6f67fc640 1 -- 172.21.15.3:0/3348743006 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7040ad880 con 0x560129c5f070 2024-09-21T06:15:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.244+0000 7fc7192dc740 1 -- 172.21.15.3:0/3348743006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 msgr2=0x56012a0176f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.244+0000 7fc7192dc740 1 --2- 172.21.15.3:0/3348743006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 0x56012a0176f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc7040049d0 tx=0x7fc704004ab0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.244+0000 7fc7192dc740 1 -- 172.21.15.3:0/3348743006 shutdown_connections 2024-09-21T06:15:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.244+0000 7fc7192dc740 1 --2- 172.21.15.3:0/3348743006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c5f070 0x56012a0176f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.244+0000 7fc7192dc740 1 --2- 172.21.15.3:0/3348743006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c3e390 0x56012a026810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.244+0000 7fc7192dc740 1 --2- 172.21.15.3:0/3348743006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c361b0 0x56012a01a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.244+0000 7fc7192dc740 1 -- 172.21.15.3:0/3348743006 >> 172.21.15.3:0/3348743006 conn(0x560129c53460 msgr2=0x56012a01a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.244+0000 7fc7192dc740 1 -- 172.21.15.3:0/3348743006 shutdown_connections 2024-09-21T06:15:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.244+0000 7fc7192dc740 1 -- 172.21.15.3:0/3348743006 wait complete. 2024-09-21T06:15:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.244+0000 7fc7192dc740 1 -- start start 2024-09-21T06:15:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.245+0000 7fc7192dc740 1 Processor -- start 2024-09-21T06:15:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.245+0000 7fc7192dc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c361b0 0x56012a01eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.245+0000 7fc7192dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c3e390 0x56012a01efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.245+0000 7fc7192dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c5f070 0x56012a096180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.245+0000 7fc7192dc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1ae920 con 0x560129c361b0 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.245+0000 7fc7192dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1ae250 con 0x560129c5f070 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.245+0000 7fc7192dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1ab4d0 con 0x560129c3e390 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.245+0000 7fc710b04640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c361b0 0x56012a01eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.246+0000 7fc710b04640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c361b0 0x56012a01eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48722/0 (socket says 172.21.15.3:48722) 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.246+0000 7fc710b04640 1 -- 172.21.15.3:0/4144064101 learned_addr learned my addr 172.21.15.3:0/4144064101 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.246+0000 7fc7192dc740 10 client.0 _ll_register_callbacks cb 0x7fc7040a9f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.246+0000 7fc70bfff640 1 --2- 172.21.15.3:0/4144064101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c3e390 0x56012a01efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.246+0000 7fc711305640 1 --2- 172.21.15.3:0/4144064101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c5f070 0x56012a096180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.246+0000 7fc710b04640 1 -- 172.21.15.3:0/4144064101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c3e390 msgr2=0x56012a01efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.246+0000 7fc710b04640 1 --2- 172.21.15.3:0/4144064101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c3e390 0x56012a01efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.246+0000 7fc710b04640 1 -- 172.21.15.3:0/4144064101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c5f070 msgr2=0x56012a096180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.246+0000 7fc710b04640 1 --2- 172.21.15.3:0/4144064101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c5f070 0x56012a096180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.246+0000 7fc710b04640 1 -- 172.21.15.3:0/4144064101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56012a01f7d0 con 0x560129c361b0 2024-09-21T06:15:56.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.247+0000 7fc710b04640 1 --2- 172.21.15.3:0/4144064101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c361b0 0x56012a01eaa0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc700011d10 tx=0x7fc700009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b549a1971515e9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:56.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.247+0000 7fc6ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:56.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.247+0000 7fc6ebfff640 1 -- 172.21.15.3:0/4144064101 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700031060 con 0x560129c361b0 2024-09-21T06:15:56.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.247+0000 7fc6ebfff640 1 -- 172.21.15.3:0/4144064101 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fc700005470 con 0x560129c361b0 2024-09-21T06:15:56.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.247+0000 7fc7192dc740 1 -- 172.21.15.3:0/4144064101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7040ad5c0 con 0x560129c361b0 2024-09-21T06:15:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.247+0000 7fc6ebfff640 1 -- 172.21.15.3:0/4144064101 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc70001e040 con 0x560129c361b0 2024-09-21T06:15:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.247+0000 7fc7192dc740 10 client.15699 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.247+0000 7fc7192dc740 1 -- 172.21.15.3:0/4144064101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56012a00c9d0 con 0x560129c361b0 2024-09-21T06:15:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.248+0000 7fc6ebfff640 1 -- 172.21.15.3:0/4144064101 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc700018a30 con 0x560129c361b0 2024-09-21T06:15:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.248+0000 7fc6ebfff640 1 -- 172.21.15.3:0/4144064101 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc700036670 con 0x560129c361b0 2024-09-21T06:15:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.248+0000 7fc7192dc740 10 client.15699 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.248+0000 7fc7192dc740 10 client.15699 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:15:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.248+0000 7fc7192dc740 20 client.15699 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.248+0000 7fc7192dc740 1 --2- 172.21.15.3:0/4144064101 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x560129bf2da0 0x56012a00efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.248+0000 7fc7192dc740 4 client.15699 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:15:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.248+0000 7fc7192dc740 1 -- 172.21.15.3:0/4144064101 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc7040ad5c0 con 0x560129bf2da0 2024-09-21T06:15:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.248+0000 7fc70bfff640 1 --2- 172.21.15.3:0/4144064101 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x560129bf2da0 0x56012a00efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.249+0000 7fc6ebfff640 10 client.15699 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:15:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.249+0000 7fc70bfff640 1 --2- 172.21.15.3:0/4144064101 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x560129bf2da0 0x56012a00efa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=915822c39d4677a4 server_cookie=70ddd01d219ebb66 in_seq=0 out_seq=0 2024-09-21T06:15:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.249+0000 7fc6ebfff640 1 -- 172.21.15.3:0/4144064101 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc7040ad5c0 con 0x560129bf2da0 2024-09-21T06:15:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.249+0000 7fc6ebfff640 10 client.15699 handle_command_reply: tid=0 2024-09-21T06:15:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.249+0000 7fc7192dc740 1 client.15699 shutdown 2024-09-21T06:15:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.249+0000 7fc7192dc740 1 -- 172.21.15.3:0/4144064101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c361b0 msgr2=0x56012a01eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.249+0000 7fc7192dc740 1 --2- 172.21.15.3:0/4144064101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c361b0 0x56012a01eaa0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc700011d10 tx=0x7fc700009e10 comp rx=0 tx=0).stop 2024-09-21T06:15:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.250+0000 7fc7192dc740 1 -- 172.21.15.3:0/4144064101 shutdown_connections 2024-09-21T06:15:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.250+0000 7fc7192dc740 1 --2- 172.21.15.3:0/4144064101 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x560129bf2da0 0x56012a00efa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.250+0000 7fc6ebfff640 0 client.15699 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:15:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.250+0000 7fc7192dc740 1 --2- 172.21.15.3:0/4144064101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560129c3e390 0x56012a01efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.250+0000 7fc7192dc740 1 --2- 172.21.15.3:0/4144064101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560129c5f070 0x56012a096180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.250+0000 7fc7192dc740 1 --2- 172.21.15.3:0/4144064101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560129c361b0 0x56012a01eaa0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.250+0000 7fc7192dc740 1 -- 172.21.15.3:0/4144064101 >> 172.21.15.3:0/4144064101 conn(0x560129c53460 msgr2=0x56012a018980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.250+0000 7fc7192dc740 1 -- 172.21.15.3:0/4144064101 shutdown_connections 2024-09-21T06:15:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.250+0000 7fc7192dc740 1 -- 172.21.15.3:0/4144064101 wait complete. 2024-09-21T06:15:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.250+0000 7fc7192dc740 20 client.15699 trim_cache size 0 max 16384 2024-09-21T06:15:56.255 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:15:56.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.251+0000 7fc712d68640 1 -- 172.21.15.3:0/1999092302 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6fc0b6700 msgr2=0x7fc6fc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:56.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.251+0000 7fc712d68640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6fc0b6700 0x7fc6fc0b8c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8005ab0 tx=0x7fc6f80044d0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.251+0000 7fc712d68640 1 -- 172.21.15.3:0/1999092302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c110240 msgr2=0x7fc70c1a1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:56.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.251+0000 7fc712d68640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c110240 0x7fc70c1a1ef0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc70400eb10 tx=0x7fc70400c2d0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.251+0000 7fc712d68640 1 -- 172.21.15.3:0/1999092302 shutdown_connections 2024-09-21T06:15:56.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.251+0000 7fc712d68640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6fc0b6700 0x7fc6fc0b8c10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.251+0000 7fc712d68640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c110240 0x7fc70c1a1ef0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.251+0000 7fc712d68640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc70c102fe0 0x7fc70c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.251+0000 7fc712d68640 1 --2- 172.21.15.3:0/1999092302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc70c102690 0x7fc70c19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:56.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.251+0000 7fc712d68640 1 -- 172.21.15.3:0/1999092302 >> 172.21.15.3:0/1999092302 conn(0x7fc70c079750 msgr2=0x7fc70c108cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:56.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.251+0000 7fc712d68640 1 -- 172.21.15.3:0/1999092302 shutdown_connections 2024-09-21T06:15:56.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:56.251+0000 7fc712d68640 1 -- 172.21.15.3:0/1999092302 wait complete. 2024-09-21T06:15:57.019 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:56 smithi003 ceph-mon[31227]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:15:57.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:15:57.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:56 smithi137 ceph-mon[34464]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:15:57.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:56 smithi204 ceph-mon[34659]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:15:57.331 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:57.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.924+0000 7f1533118640 1 -- 172.21.15.3:0/2956229246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152c1083d0 msgr2=0x7f152c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.924+0000 7f1533118640 1 --2- 172.21.15.3:0/2956229246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152c1083d0 0x7f152c1087e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f151c009900 tx=0x7f151c02a410 comp rx=0 tx=0).stop 2024-09-21T06:15:57.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.924+0000 7f1533118640 1 -- 172.21.15.3:0/2956229246 shutdown_connections 2024-09-21T06:15:57.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.924+0000 7f1533118640 1 --2- 172.21.15.3:0/2956229246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c108d20 0x7f152c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.924+0000 7f1533118640 1 --2- 172.21.15.3:0/2956229246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152c1083d0 0x7f152c1087e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.924+0000 7f1533118640 1 --2- 172.21.15.3:0/2956229246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f152c102ae0 0x7f152c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.924+0000 7f1533118640 1 -- 172.21.15.3:0/2956229246 >> 172.21.15.3:0/2956229246 conn(0x7f152c079790 msgr2=0x7f152c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:57.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.924+0000 7f1533118640 1 -- 172.21.15.3:0/2956229246 shutdown_connections 2024-09-21T06:15:57.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.924+0000 7f1533118640 1 -- 172.21.15.3:0/2956229246 wait complete. 2024-09-21T06:15:57.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.925+0000 7f1533118640 1 Processor -- start 2024-09-21T06:15:57.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.925+0000 7f1533118640 1 -- start start 2024-09-21T06:15:57.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.925+0000 7f1533118640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f152c102ae0 0x7f152c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1533118640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c1083d0 0x7f152c1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1533118640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152c108d20 0x7f152c1a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1533118640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1118e0 con 0x7f152c108d20 2024-09-21T06:15:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1533118640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f152c111760 con 0x7f152c102ae0 2024-09-21T06:15:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1533118640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f152c111a60 con 0x7f152c1083d0 2024-09-21T06:15:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1530eb4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f152c102ae0 0x7f152c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1530eb4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f152c102ae0 0x7f152c1a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49270/0 (socket says 172.21.15.3:49270) 2024-09-21T06:15:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1530eb4640 1 -- 172.21.15.3:0/1897922116 learned_addr learned my addr 172.21.15.3:0/1897922116 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f15316b5640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152c108d20 0x7f152c1a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1523fff640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c1083d0 0x7f152c1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1530eb4640 1 -- 172.21.15.3:0/1897922116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c1083d0 msgr2=0x7f152c1a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1530eb4640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c1083d0 0x7f152c1a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1530eb4640 1 -- 172.21.15.3:0/1897922116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152c108d20 msgr2=0x7f152c1a6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1530eb4640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152c108d20 0x7f152c1a6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f1530eb4640 1 -- 172.21.15.3:0/1897922116 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152c1a6ab0 con 0x7f152c102ae0 2024-09-21T06:15:57.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.926+0000 7f15316b5640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152c108d20 0x7f152c1a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:57.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.927+0000 7f1530eb4640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f152c102ae0 0x7f152c1a0b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1528007860 tx=0x7f1528007d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:57.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.927+0000 7f1521ffb640 1 -- 172.21.15.3:0/1897922116 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528019020 con 0x7f152c102ae0 2024-09-21T06:15:57.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.927+0000 7f1521ffb640 1 -- 172.21.15.3:0/1897922116 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1528015470 con 0x7f152c102ae0 2024-09-21T06:15:57.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.927+0000 7f1533118640 1 -- 172.21.15.3:0/1897922116 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152c1a6da0 con 0x7f152c102ae0 2024-09-21T06:15:57.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.928+0000 7f1521ffb640 1 -- 172.21.15.3:0/1897922116 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528014630 con 0x7f152c102ae0 2024-09-21T06:15:57.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.928+0000 7f1533118640 1 -- 172.21.15.3:0/1897922116 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152c1ad500 con 0x7f152c102ae0 2024-09-21T06:15:57.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.932+0000 7f153968c740 1 Processor -- start 2024-09-21T06:15:57.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.932+0000 7f153968c740 1 -- start start 2024-09-21T06:15:57.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f153968c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b1b3070 0x55883b55a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f153968c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b18a1b0 0x55883b55abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f153968c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b192390 0x55883b55b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f153968c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55883b4d0ea0 con 0x55883b18a1b0 2024-09-21T06:15:57.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f153968c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55883b4c02d0 con 0x55883b1b3070 2024-09-21T06:15:57.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f153968c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55883b4c40e0 con 0x55883b192390 2024-09-21T06:15:57.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f15316b5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b1b3070 0x55883b55a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f15316b5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b1b3070 0x55883b55a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49286/0 (socket says 172.21.15.3:49286) 2024-09-21T06:15:57.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f15316b5640 1 -- 172.21.15.3:0/2634013821 learned_addr learned my addr 172.21.15.3:0/2634013821 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:57.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f1523fff640 1 --2- 172.21.15.3:0/2634013821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b192390 0x55883b55b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f1521ffb640 1 -- 172.21.15.3:0/1897922116 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 154636+0+0 (secure 0 0 0) 0x7f152801c020 con 0x7f152c102ae0 2024-09-21T06:15:57.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f15316b5640 1 -- 172.21.15.3:0/2634013821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b192390 msgr2=0x55883b55b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f15316b5640 1 --2- 172.21.15.3:0/2634013821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b192390 0x55883b55b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.933+0000 7f15316b5640 1 -- 172.21.15.3:0/2634013821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b18a1b0 msgr2=0x55883b55abe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:15:57.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.934+0000 7f1530eb4640 1 --2- 172.21.15.3:0/2634013821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b18a1b0 0x55883b55abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.934+0000 7f15316b5640 1 --2- 172.21.15.3:0/2634013821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b18a1b0 0x55883b55abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.934+0000 7f15316b5640 1 -- 172.21.15.3:0/2634013821 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883b21e650 con 0x55883b1b3070 2024-09-21T06:15:57.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.934+0000 7f1530eb4640 1 --2- 172.21.15.3:0/2634013821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b18a1b0 0x55883b55abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:57.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.934+0000 7f15316b5640 1 --2- 172.21.15.3:0/2634013821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b1b3070 0x55883b55a6a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1524003b00 tx=0x7f152400d910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9db802f840b872a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:57.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.934+0000 7f1512ffd640 1 -- 172.21.15.3:0/2634013821 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152401e070 con 0x55883b1b3070 2024-09-21T06:15:57.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.935+0000 7f1512ffd640 1 -- 172.21.15.3:0/2634013821 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1524004700 con 0x55883b1b3070 2024-09-21T06:15:57.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.935+0000 7f1512ffd640 1 -- 172.21.15.3:0/2634013821 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524019690 con 0x55883b1b3070 2024-09-21T06:15:57.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.936+0000 7f1521ffb640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f15140b6660 0x7f15140b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.936+0000 7f153968c740 1 -- 172.21.15.3:0/2634013821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b1b3070 msgr2=0x55883b55a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.936+0000 7f153968c740 1 --2- 172.21.15.3:0/2634013821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b1b3070 0x55883b55a6a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1524003b00 tx=0x7f152400d910 comp rx=0 tx=0).stop 2024-09-21T06:15:57.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.936+0000 7f153968c740 1 -- 172.21.15.3:0/2634013821 shutdown_connections 2024-09-21T06:15:57.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.936+0000 7f153968c740 1 --2- 172.21.15.3:0/2634013821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b192390 0x55883b55b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.936+0000 7f153968c740 1 --2- 172.21.15.3:0/2634013821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b18a1b0 0x55883b55abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.936+0000 7f1530eb4640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f15140b6660 0x7f15140b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.936+0000 7f153968c740 1 --2- 172.21.15.3:0/2634013821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b1b3070 0x55883b55a6a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.936+0000 7f153968c740 1 -- 172.21.15.3:0/2634013821 >> 172.21.15.3:0/2634013821 conn(0x55883b1a7460 msgr2=0x55883b558590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:57.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.937+0000 7f1521ffb640 1 -- 172.21.15.3:0/1897922116 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f15280e7340 con 0x7f152c102ae0 2024-09-21T06:15:57.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.937+0000 7f1530eb4640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f15140b6660 0x7f15140b8b70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55883b559320 tx=0x7f1528009cf0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:57.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.937+0000 7f153968c740 1 -- 172.21.15.3:0/2634013821 shutdown_connections 2024-09-21T06:15:57.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.937+0000 7f153968c740 1 -- 172.21.15.3:0/2634013821 wait complete. 2024-09-21T06:15:57.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.938+0000 7f153968c740 1 -- start start 2024-09-21T06:15:57.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.938+0000 7f153968c740 1 Processor -- start 2024-09-21T06:15:57.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.938+0000 7f153968c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b18a1b0 0x55883b565de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f153968c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 0x55883b566320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f153968c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 0x55883b583180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f1523fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 0x55883b566320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f15316b5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b18a1b0 0x55883b565de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f1523fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 0x55883b566320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48752/0 (socket says 172.21.15.3:48752) 2024-09-21T06:15:57.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f1523fff640 1 -- 172.21.15.3:0/1226679244 learned_addr learned my addr 172.21.15.3:0/1226679244 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:57.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f153968c740 1 -- 172.21.15.3:0/1226679244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55883b321700 con 0x55883b192390 2024-09-21T06:15:57.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f153968c740 1 -- 172.21.15.3:0/1226679244 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55883b45b160 con 0x55883b18a1b0 2024-09-21T06:15:57.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f153968c740 1 -- 172.21.15.3:0/1226679244 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55883b579e90 con 0x55883b1b3070 2024-09-21T06:15:57.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f15316b5640 1 --2- 172.21.15.3:0/1226679244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 0x55883b583180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f15316b5640 1 -- 172.21.15.3:0/1226679244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 msgr2=0x55883b583180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f15316b5640 1 --2- 172.21.15.3:0/1226679244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 0x55883b583180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f15316b5640 1 -- 172.21.15.3:0/1226679244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 msgr2=0x55883b566320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f15316b5640 1 --2- 172.21.15.3:0/1226679244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 0x55883b566320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f15316b5640 1 -- 172.21.15.3:0/1226679244 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883b488c90 con 0x55883b18a1b0 2024-09-21T06:15:57.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.939+0000 7f1523fff640 1 --2- 172.21.15.3:0/1226679244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 0x55883b566320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:57.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.940+0000 7f153968c740 10 client.0 _ll_register_callbacks cb 0x55883b4749e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.940+0000 7f153968c740 1 -- 172.21.15.3:0/1226679244 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1524004700 con 0x55883b18a1b0 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.940+0000 7f15316b5640 1 --2- 172.21.15.3:0/1226679244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b18a1b0 0x55883b565de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f15240049d0 tx=0x7f1524019510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac803cad7c4c676 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.940+0000 7f150bfff640 10 client.25322 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.940+0000 7f150bfff640 1 -- 172.21.15.3:0/1226679244 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152401e040 con 0x55883b18a1b0 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.940+0000 7f150bfff640 1 -- 172.21.15.3:0/1226679244 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f152400bd00 con 0x55883b18a1b0 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.940+0000 7f150bfff640 1 -- 172.21.15.3:0/1226679244 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1524019610 con 0x55883b18a1b0 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.940+0000 7f153968c740 10 client.25322 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.940+0000 7f153968c740 1 -- 172.21.15.3:0/1226679244 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55883b5858a0 con 0x55883b18a1b0 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.941+0000 7f150bfff640 1 -- 172.21.15.3:0/1226679244 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f15240197b0 con 0x55883b18a1b0 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.941+0000 7f153968c740 10 client.25322 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.941+0000 7f153968c740 10 client.25322 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.941+0000 7f153968c740 20 client.25322 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.941+0000 7f153968c740 1 --2- 172.21.15.3:0/1226679244 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55883b1719f0 0x55883b5891d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.941+0000 7f153968c740 4 client.25322 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:57.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.941+0000 7f153968c740 1 -- 172.21.15.3:0/1226679244 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55883b4d2df0 con 0x55883b1719f0 2024-09-21T06:15:57.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.941+0000 7f1523fff640 1 --2- 172.21.15.3:0/1226679244 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55883b1719f0 0x55883b5891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.942+0000 7f150bfff640 10 client.25322 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:15:57.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.942+0000 7f1523fff640 1 --2- 172.21.15.3:0/1226679244 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55883b1719f0 0x55883b5891d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9004140d3772d206 server_cookie=bc44a4b68c23ce8f in_seq=0 out_seq=0 2024-09-21T06:15:57.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.943+0000 7f150bfff640 1 -- 172.21.15.3:0/1226679244 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55883b4d2df0 con 0x55883b1719f0 2024-09-21T06:15:57.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.943+0000 7f150bfff640 10 client.25322 handle_command_reply: tid=0 2024-09-21T06:15:57.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.943+0000 7f153968c740 1 client.25322 shutdown 2024-09-21T06:15:57.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.943+0000 7f153968c740 1 -- 172.21.15.3:0/1226679244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b18a1b0 msgr2=0x55883b565de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.943+0000 7f153968c740 1 --2- 172.21.15.3:0/1226679244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b18a1b0 0x55883b565de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f15240049d0 tx=0x7f1524019510 comp rx=0 tx=0).stop 2024-09-21T06:15:57.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.943+0000 7f153968c740 1 -- 172.21.15.3:0/1226679244 shutdown_connections 2024-09-21T06:15:57.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.943+0000 7f153968c740 1 --2- 172.21.15.3:0/1226679244 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55883b1719f0 0x55883b5891d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.943+0000 7f153968c740 1 --2- 172.21.15.3:0/1226679244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 0x55883b583180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.943+0000 7f153968c740 1 --2- 172.21.15.3:0/1226679244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 0x55883b566320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.943+0000 7f153968c740 1 --2- 172.21.15.3:0/1226679244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b18a1b0 0x55883b565de0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.943+0000 7f153968c740 1 -- 172.21.15.3:0/1226679244 >> 172.21.15.3:0/1226679244 conn(0x55883b1a7460 msgr2=0x55883b55d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:57.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.943+0000 7f150bfff640 0 client.25322 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:15:57.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.943+0000 7f153968c740 1 -- 172.21.15.3:0/1226679244 shutdown_connections 2024-09-21T06:15:57.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.944+0000 7f153968c740 1 -- 172.21.15.3:0/1226679244 wait complete. 2024-09-21T06:15:57.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.944+0000 7f153968c740 20 client.25322 trim_cache size 0 max 16384 2024-09-21T06:15:57.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.970+0000 7f153968c740 1 Processor -- start 2024-09-21T06:15:57.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.970+0000 7f153968c740 1 -- start start 2024-09-21T06:15:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.971+0000 7f153968c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b18a1b0 0x55883b55bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.971+0000 7f153968c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 0x55883b557bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.971+0000 7f153968c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 0x55883b558100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.971+0000 7f153968c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f152c19f070 con 0x55883b192390 2024-09-21T06:15:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.971+0000 7f153968c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1aed00 con 0x55883b18a1b0 2024-09-21T06:15:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.971+0000 7f153968c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1ad890 con 0x55883b1b3070 2024-09-21T06:15:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.971+0000 7f1523fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 0x55883b557bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.971+0000 7f15316b5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b18a1b0 0x55883b55bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.971+0000 7f1523fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 0x55883b557bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48766/0 (socket says 172.21.15.3:48766) 2024-09-21T06:15:57.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.971+0000 7f1523fff640 1 -- 172.21.15.3:0/3467226534 learned_addr learned my addr 172.21.15.3:0/3467226534 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:57.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.971+0000 7f15316b5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b18a1b0 0x55883b55bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49298/0 (socket says 172.21.15.3:49298) 2024-09-21T06:15:57.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.972+0000 7f1523fff640 1 -- 172.21.15.3:0/3467226534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 msgr2=0x55883b558100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.972+0000 7f1523fff640 1 --2- 172.21.15.3:0/3467226534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 0x55883b558100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.972+0000 7f1523fff640 1 -- 172.21.15.3:0/3467226534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b18a1b0 msgr2=0x55883b55bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.972+0000 7f1523fff640 1 --2- 172.21.15.3:0/3467226534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b18a1b0 0x55883b55bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.972+0000 7f1523fff640 1 -- 172.21.15.3:0/3467226534 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883b55d500 con 0x55883b192390 2024-09-21T06:15:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.972+0000 7f1523fff640 1 --2- 172.21.15.3:0/3467226534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 0x55883b557bc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f151c02a930 tx=0x7f151c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6231549da090bafe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.972+0000 7f15127fc640 1 -- 172.21.15.3:0/3467226534 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c035040 con 0x55883b192390 2024-09-21T06:15:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.972+0000 7f15127fc640 1 -- 172.21.15.3:0/3467226534 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c0054f0 con 0x55883b192390 2024-09-21T06:15:57.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.972+0000 7f15127fc640 1 -- 172.21.15.3:0/3467226534 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c004d80 con 0x55883b192390 2024-09-21T06:15:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.974+0000 7f153968c740 1 -- 172.21.15.3:0/3467226534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 msgr2=0x55883b557bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.974+0000 7f153968c740 1 --2- 172.21.15.3:0/3467226534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 0x55883b557bc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f151c02a930 tx=0x7f151c031400 comp rx=0 tx=0).stop 2024-09-21T06:15:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.974+0000 7f153968c740 1 -- 172.21.15.3:0/3467226534 shutdown_connections 2024-09-21T06:15:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.974+0000 7f153968c740 1 --2- 172.21.15.3:0/3467226534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 0x55883b558100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.974+0000 7f153968c740 1 --2- 172.21.15.3:0/3467226534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b192390 0x55883b557bc0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.974+0000 7f153968c740 1 --2- 172.21.15.3:0/3467226534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b18a1b0 0x55883b55bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.974+0000 7f153968c740 1 -- 172.21.15.3:0/3467226534 >> 172.21.15.3:0/3467226534 conn(0x55883b1a7460 msgr2=0x55883b55b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.974+0000 7f153968c740 1 -- 172.21.15.3:0/3467226534 shutdown_connections 2024-09-21T06:15:57.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.974+0000 7f153968c740 1 -- 172.21.15.3:0/3467226534 wait complete. 2024-09-21T06:15:57.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.975+0000 7f153968c740 1 -- start start 2024-09-21T06:15:57.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.975+0000 7f153968c740 1 Processor -- start 2024-09-21T06:15:57.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f153968c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b18a1b0 0x55883b57f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f153968c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 0x55883b57ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f153968c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b146da0 0x55883b57b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f153968c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1ae630 con 0x55883b18a1b0 2024-09-21T06:15:57.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f153968c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1adf60 con 0x55883b146da0 2024-09-21T06:15:57.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f153968c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f152c19d050 con 0x55883b1b3070 2024-09-21T06:15:57.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f153968c740 10 client.0 _ll_register_callbacks cb 0x7f152401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:57.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f1523fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 0x55883b57ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f1523fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 0x55883b57ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41554/0 (socket says 172.21.15.3:41554) 2024-09-21T06:15:57.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f1523fff640 1 -- 172.21.15.3:0/2269892529 learned_addr learned my addr 172.21.15.3:0/2269892529 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:57.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f15316b5640 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b18a1b0 0x55883b57f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f15316b5640 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b146da0 0x55883b57b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f1523fff640 1 -- 172.21.15.3:0/2269892529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b146da0 msgr2=0x55883b57b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f1523fff640 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b146da0 0x55883b57b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f1523fff640 1 -- 172.21.15.3:0/2269892529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b18a1b0 msgr2=0x55883b57f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.976+0000 7f1523fff640 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b18a1b0 0x55883b57f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.977+0000 7f1523fff640 1 -- 172.21.15.3:0/2269892529 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883b57b7b0 con 0x55883b1b3070 2024-09-21T06:15:57.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.977+0000 7f15316b5640 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b146da0 0x55883b57b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.977+0000 7f1523fff640 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 0x55883b57ac30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f151c009850 tx=0x7f151c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9da69a81d33ce52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.977+0000 7f150bfff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.977+0000 7f150bfff640 1 -- 172.21.15.3:0/2269892529 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c035070 con 0x55883b1b3070 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.977+0000 7f153968c740 1 -- 172.21.15.3:0/2269892529 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f151c0054f0 con 0x55883b1b3070 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.977+0000 7f150bfff640 1 -- 172.21.15.3:0/2269892529 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f151c004100 con 0x55883b1b3070 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.977+0000 7f150bfff640 1 -- 172.21.15.3:0/2269892529 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f151c004300 con 0x55883b1b3070 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.977+0000 7f153968c740 10 client.25306 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.977+0000 7f153968c740 1 -- 172.21.15.3:0/2269892529 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55883b5f39c0 con 0x55883b1b3070 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.978+0000 7f150bfff640 1 -- 172.21.15.3:0/2269892529 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c0054f0 con 0x55883b1b3070 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.978+0000 7f150bfff640 1 -- 172.21.15.3:0/2269892529 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f151c003b60 con 0x55883b1b3070 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.978+0000 7f153968c740 10 client.25306 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.978+0000 7f153968c740 10 client.25306 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.978+0000 7f153968c740 20 client.25306 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:57.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.978+0000 7f153968c740 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55883b5f3f60 0x55883b5f6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.978+0000 7f153968c740 4 client.25306 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:15:57.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.978+0000 7f153968c740 1 -- 172.21.15.3:0/2269892529 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55883b4ce8a0 con 0x55883b5f3f60 2024-09-21T06:15:57.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.978+0000 7f15316b5640 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55883b5f3f60 0x55883b5f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.979+0000 7f150bfff640 10 client.25306 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:15:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.979+0000 7f15316b5640 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55883b5f3f60 0x55883b5f6380 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16a3f6b472a5908 server_cookie=e29d0fe2608af7e9 in_seq=0 out_seq=0 2024-09-21T06:15:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f150bfff640 1 -- 172.21.15.3:0/2269892529 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55883b4ce8a0 con 0x55883b5f3f60 2024-09-21T06:15:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f150bfff640 10 client.25306 handle_command_reply: tid=0 2024-09-21T06:15:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f153968c740 1 client.25306 shutdown 2024-09-21T06:15:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f153968c740 1 -- 172.21.15.3:0/2269892529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 msgr2=0x55883b57ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f153968c740 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 0x55883b57ac30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f151c009850 tx=0x7f151c004770 comp rx=0 tx=0).stop 2024-09-21T06:15:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f153968c740 1 -- 172.21.15.3:0/2269892529 shutdown_connections 2024-09-21T06:15:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f153968c740 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55883b5f3f60 0x55883b5f6380 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f153968c740 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883b1b3070 0x55883b57ac30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f153968c740 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55883b146da0 0x55883b57b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f153968c740 1 --2- 172.21.15.3:0/2269892529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55883b18a1b0 0x55883b57f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f150bfff640 0 client.25306 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:15:57.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f153968c740 1 -- 172.21.15.3:0/2269892529 >> 172.21.15.3:0/2269892529 conn(0x55883b1a7460 msgr2=0x55883b55b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:57.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f153968c740 1 -- 172.21.15.3:0/2269892529 shutdown_connections 2024-09-21T06:15:57.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.980+0000 7f153968c740 1 -- 172.21.15.3:0/2269892529 wait complete. 2024-09-21T06:15:57.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.981+0000 7f153968c740 20 client.25306 trim_cache size 0 max 16384 2024-09-21T06:15:57.984 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:15:57.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.981+0000 7f1533118640 1 -- 172.21.15.3:0/1897922116 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f15140b6660 msgr2=0x7f15140b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.981+0000 7f1533118640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f15140b6660 0x7f15140b8b70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55883b559320 tx=0x7f1528009cf0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.981+0000 7f1533118640 1 -- 172.21.15.3:0/1897922116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f152c102ae0 msgr2=0x7f152c1a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.981+0000 7f1533118640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f152c102ae0 0x7f152c1a0b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1528007860 tx=0x7f1528007d20 comp rx=0 tx=0).stop 2024-09-21T06:15:57.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.982+0000 7f1533118640 1 -- 172.21.15.3:0/1897922116 shutdown_connections 2024-09-21T06:15:57.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.982+0000 7f1533118640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f15140b6660 0x7f15140b8b70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.982+0000 7f1533118640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c1083d0 0x7f152c1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.982+0000 7f1533118640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152c108d20 0x7f152c1a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.982+0000 7f1533118640 1 --2- 172.21.15.3:0/1897922116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f152c102ae0 0x7f152c1a0b40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.982+0000 7f1533118640 1 -- 172.21.15.3:0/1897922116 >> 172.21.15.3:0/1897922116 conn(0x7f152c079790 msgr2=0x7f152c1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:57.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.982+0000 7f1533118640 1 -- 172.21.15.3:0/1897922116 shutdown_connections 2024-09-21T06:15:57.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:57.982+0000 7f1533118640 1 -- 172.21.15.3:0/1897922116 wait complete. 2024-09-21T06:15:58.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.485+0000 7fd0baf20640 1 -- 172.21.15.3:0/3109170188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0b4109420 msgr2=0x7fd0b4109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:58.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.485+0000 7fd0baf20640 1 --2- 172.21.15.3:0/3109170188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0b4109420 0x7fd0b4109830 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009900 tx=0x7fd0a002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.485+0000 7fd0baf20640 1 -- 172.21.15.3:0/3109170188 shutdown_connections 2024-09-21T06:15:58.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.485+0000 7fd0baf20640 1 --2- 172.21.15.3:0/3109170188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b4109d70 0x7fd0b41105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.485+0000 7fd0baf20640 1 --2- 172.21.15.3:0/3109170188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0b4109420 0x7fd0b4109830 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.485+0000 7fd0baf20640 1 --2- 172.21.15.3:0/3109170188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0b4103af0 0x7fd0b4103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.485+0000 7fd0baf20640 1 -- 172.21.15.3:0/3109170188 >> 172.21.15.3:0/3109170188 conn(0x7fd0b40ff570 msgr2=0x7fd0b41019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:58.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.485+0000 7fd0baf20640 1 -- 172.21.15.3:0/3109170188 shutdown_connections 2024-09-21T06:15:58.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.485+0000 7fd0baf20640 1 -- 172.21.15.3:0/3109170188 wait complete. 2024-09-21T06:15:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.486+0000 7fd0baf20640 1 Processor -- start 2024-09-21T06:15:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.486+0000 7fd0baf20640 1 -- start start 2024-09-21T06:15:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.486+0000 7fd0baf20640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b4103af0 0x7fd0b419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.486+0000 7fd0baf20640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0b4109420 0x7fd0b419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.486+0000 7fd0baf20640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0b4109d70 0x7fd0b419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.486+0000 7fd0baf20640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4113a30 con 0x7fd0b4109d70 2024-09-21T06:15:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.486+0000 7fd0baf20640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41138b0 con 0x7fd0b4109420 2024-09-21T06:15:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.486+0000 7fd0baf20640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4113bb0 con 0x7fd0b4103af0 2024-09-21T06:15:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.486+0000 7fd0b94bd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0b4109d70 0x7fd0b419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.487+0000 7fd0b94bd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0b4109d70 0x7fd0b419fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48788/0 (socket says 172.21.15.3:48788) 2024-09-21T06:15:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.487+0000 7fd0b94bd640 1 -- 172.21.15.3:0/2949987074 learned_addr learned my addr 172.21.15.3:0/2949987074 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.487+0000 7fd0b8cbc640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b4103af0 0x7fd0b419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.487+0000 7fd0b3fff640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0b4109420 0x7fd0b419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.487+0000 7fd0b94bd640 1 -- 172.21.15.3:0/2949987074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b4103af0 msgr2=0x7fd0b419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.487+0000 7fd0b94bd640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b4103af0 0x7fd0b419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.487+0000 7fd0b94bd640 1 -- 172.21.15.3:0/2949987074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0b4109420 msgr2=0x7fd0b419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.487+0000 7fd0b94bd640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0b4109420 0x7fd0b419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.487+0000 7fd0b94bd640 1 -- 172.21.15.3:0/2949987074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b41a03a0 con 0x7fd0b4109d70 2024-09-21T06:15:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.487+0000 7fd0b94bd640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0b4109d70 0x7fd0b419fca0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00eb10 tx=0x7fd0ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.487+0000 7fd0b1ffb640 1 -- 172.21.15.3:0/2949987074 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac004b20 con 0x7fd0b4109d70 2024-09-21T06:15:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.487+0000 7fd0baf20640 1 -- 172.21.15.3:0/2949987074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b41a0690 con 0x7fd0b4109d70 2024-09-21T06:15:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.487+0000 7fd0b1ffb640 1 -- 172.21.15.3:0/2949987074 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0ac004cc0 con 0x7fd0b4109d70 2024-09-21T06:15:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.488+0000 7fd0b1ffb640 1 -- 172.21.15.3:0/2949987074 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac010690 con 0x7fd0b4109d70 2024-09-21T06:15:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.488+0000 7fd0baf20640 1 -- 172.21.15.3:0/2949987074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b41a6e20 con 0x7fd0b4109d70 2024-09-21T06:15:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.491+0000 7fd0c1e8a740 1 Processor -- start 2024-09-21T06:15:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.492+0000 7fd0b1ffb640 1 -- 172.21.15.3:0/2949987074 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 154636+0+0 (secure 0 0 0) 0x7fd0ac005000 con 0x7fd0b4109d70 2024-09-21T06:15:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.492+0000 7fd0c1e8a740 1 -- start start 2024-09-21T06:15:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.492+0000 7fd0c1e8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960de41b0 0x5619611899b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.492+0000 7fd0c1e8a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x561961189ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.492+0000 7fd0c1e8a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960dc34d0 0x56196118a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.492+0000 7fd0c1e8a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5619610fe6f0 con 0x561960dbb2f0 2024-09-21T06:15:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.492+0000 7fd0c1e8a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5619610edb20 con 0x561960de41b0 2024-09-21T06:15:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.492+0000 7fd0c1e8a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5619610f1930 con 0x561960dc34d0 2024-09-21T06:15:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.492+0000 7fd0b8cbc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x561961189ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.492+0000 7fd0b8cbc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x561961189ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48798/0 (socket says 172.21.15.3:48798) 2024-09-21T06:15:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.492+0000 7fd0b8cbc640 1 -- 172.21.15.3:0/865754870 learned_addr learned my addr 172.21.15.3:0/865754870 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.493+0000 7fd0b8cbc640 1 -- 172.21.15.3:0/865754870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960dc34d0 msgr2=0x56196118a430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:15:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.493+0000 7fd0b8cbc640 1 --2- 172.21.15.3:0/865754870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960dc34d0 0x56196118a430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.493+0000 7fd0b8cbc640 1 -- 172.21.15.3:0/865754870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960de41b0 msgr2=0x5619611899b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:15:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.493+0000 7fd0b8cbc640 1 --2- 172.21.15.3:0/865754870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960de41b0 0x5619611899b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.493+0000 7fd0b8cbc640 1 -- 172.21.15.3:0/865754870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619610b3530 con 0x561960dbb2f0 2024-09-21T06:15:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.493+0000 7fd0b1ffb640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0a40b6680 0x7fd0a40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.493+0000 7fd0b3fff640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0a40b6680 0x7fd0a40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.493+0000 7fd0b8cbc640 1 --2- 172.21.15.3:0/865754870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x561961189ef0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009cf0 tx=0x7fd0a8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b63d73401fdfa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.493+0000 7fd0b1ffb640 1 -- 172.21.15.3:0/2949987074 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fd0ac0e7c50 con 0x7fd0b4109d70 2024-09-21T06:15:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.493+0000 7fd09effd640 1 -- 172.21.15.3:0/865754870 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a801e070 con 0x561960dbb2f0 2024-09-21T06:15:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.493+0000 7fd09effd640 1 -- 172.21.15.3:0/865754870 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0a8019ae0 con 0x561960dbb2f0 2024-09-21T06:15:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.493+0000 7fd09effd640 1 -- 172.21.15.3:0/865754870 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a800e8b0 con 0x561960dbb2f0 2024-09-21T06:15:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.495+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/865754870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 msgr2=0x561961189ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.495+0000 7fd0b3fff640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0a40b6680 0x7fd0a40b8b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009230 tx=0x7fd0a0031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.495+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/865754870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x561961189ef0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009cf0 tx=0x7fd0a8011810 comp rx=0 tx=0).stop 2024-09-21T06:15:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.495+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/865754870 shutdown_connections 2024-09-21T06:15:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.495+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/865754870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960dc34d0 0x56196118a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.495+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/865754870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x561961189ef0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.495+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/865754870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960de41b0 0x5619611899b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.495+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/865754870 >> 172.21.15.3:0/865754870 conn(0x561960dd8300 msgr2=0x5619611878a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.495+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/865754870 shutdown_connections 2024-09-21T06:15:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.495+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/865754870 wait complete. 2024-09-21T06:15:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.495+0000 7fd0c1e8a740 1 -- start start 2024-09-21T06:15:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.495+0000 7fd0c1e8a740 1 Processor -- start 2024-09-21T06:15:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.496+0000 7fd0c1e8a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x561961199470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.496+0000 7fd0c1e8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960dc34d0 0x5619611999b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.496+0000 7fd0c1e8a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960de41b0 0x5619611a5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.496+0000 7fd0c1e8a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561961025ab0 con 0x561960dbb2f0 2024-09-21T06:15:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.496+0000 7fd0c1e8a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561961093730 con 0x561960dc34d0 2024-09-21T06:15:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.496+0000 7fd0c1e8a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56196119c4f0 con 0x561960de41b0 2024-09-21T06:15:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.496+0000 7fd0b8cbc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x561961199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.496+0000 7fd0c1e8a740 10 client.0 _ll_register_callbacks cb 0x5619610e76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.496+0000 7fd0b8cbc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x561961199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48814/0 (socket says 172.21.15.3:48814) 2024-09-21T06:15:58.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.496+0000 7fd0b8cbc640 1 -- 172.21.15.3:0/4245700727 learned_addr learned my addr 172.21.15.3:0/4245700727 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:58.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.496+0000 7fd0b3fff640 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960dc34d0 0x5619611999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.497+0000 7fd0b94bd640 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960de41b0 0x5619611a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.497+0000 7fd0b8cbc640 1 -- 172.21.15.3:0/4245700727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960de41b0 msgr2=0x5619611a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.497+0000 7fd0b8cbc640 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960de41b0 0x5619611a5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.497+0000 7fd0b8cbc640 1 -- 172.21.15.3:0/4245700727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960dc34d0 msgr2=0x5619611999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.497+0000 7fd0b8cbc640 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960dc34d0 0x5619611999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.497+0000 7fd0b8cbc640 1 -- 172.21.15.3:0/4245700727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561960e4f790 con 0x561960dbb2f0 2024-09-21T06:15:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.497+0000 7fd0b3fff640 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960dc34d0 0x5619611999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.497+0000 7fd0b8cbc640 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x561961199470 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800e4d0 tx=0x7fd0a8009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17509878fb8d111 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.497+0000 7fd093fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.497+0000 7fd093fff640 1 -- 172.21.15.3:0/4245700727 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a801e040 con 0x561960dbb2f0 2024-09-21T06:15:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.497+0000 7fd093fff640 1 -- 172.21.15.3:0/4245700727 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fd0a80198c0 con 0x561960dbb2f0 2024-09-21T06:15:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.497+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/4245700727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0a8019ae0 con 0x561960dbb2f0 2024-09-21T06:15:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.498+0000 7fd093fff640 1 -- 172.21.15.3:0/4245700727 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0a8032050 con 0x561960dbb2f0 2024-09-21T06:15:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.498+0000 7fd0c1e8a740 10 client.15726 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.498+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/4245700727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561960ec8620 con 0x561960dbb2f0 2024-09-21T06:15:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.498+0000 7fd093fff640 1 -- 172.21.15.3:0/4245700727 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0a8019ae0 con 0x561960dbb2f0 2024-09-21T06:15:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.499+0000 7fd093fff640 1 -- 172.21.15.3:0/4245700727 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd0a80046e0 con 0x561960dbb2f0 2024-09-21T06:15:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.499+0000 7fd0c1e8a740 10 client.15726 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.499+0000 7fd0c1e8a740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.499+0000 7fd0c1e8a740 20 client.15726 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.499+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561960da2d80 0x5619611ab450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.499+0000 7fd0c1e8a740 4 client.15726 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.499+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/4245700727 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561961100640 con 0x561960da2d80 2024-09-21T06:15:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.499+0000 7fd0b3fff640 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561960da2d80 0x5619611ab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.500+0000 7fd0b3fff640 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561960da2d80 0x5619611ab450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b47e95eaf3515d43 server_cookie=687b935770201136 in_seq=0 out_seq=0 2024-09-21T06:15:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.500+0000 7fd093fff640 10 client.15726 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd093fff640 1 -- 172.21.15.3:0/4245700727 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561961100640 con 0x561960da2d80 2024-09-21T06:15:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd093fff640 10 client.15726 handle_command_reply: tid=0 2024-09-21T06:15:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd0c1e8a740 1 client.15726 shutdown 2024-09-21T06:15:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/4245700727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 msgr2=0x561961199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x561961199470 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800e4d0 tx=0x7fd0a8009910 comp rx=0 tx=0).stop 2024-09-21T06:15:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/4245700727 shutdown_connections 2024-09-21T06:15:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561960da2d80 0x5619611ab450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd093fff640 0 client.15726 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960de41b0 0x5619611a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960dc34d0 0x5619611999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/4245700727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x561961199470 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/4245700727 >> 172.21.15.3:0/4245700727 conn(0x561960dd8300 msgr2=0x561961185ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/4245700727 shutdown_connections 2024-09-21T06:15:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/4245700727 wait complete. 2024-09-21T06:15:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.501+0000 7fd0c1e8a740 20 client.15726 trim_cache size 0 max 16384 2024-09-21T06:15:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.522+0000 7fd0c1e8a740 1 Processor -- start 2024-09-21T06:15:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.522+0000 7fd0c1e8a740 1 -- start start 2024-09-21T06:15:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.522+0000 7fd0c1e8a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960da2d80 0x5619611865b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.522+0000 7fd0c1e8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960dbb2f0 0x5619611a5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.522+0000 7fd0c1e8a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dc34d0 0x561961196640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.522+0000 7fd0c1e8a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41ad5a0 con 0x561960dc34d0 2024-09-21T06:15:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.522+0000 7fd0c1e8a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41af0e0 con 0x561960dbb2f0 2024-09-21T06:15:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.522+0000 7fd0c1e8a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41adc70 con 0x561960da2d80 2024-09-21T06:15:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.523+0000 7fd0b3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960dbb2f0 0x5619611a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.523+0000 7fd0b3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960dbb2f0 0x5619611a5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49368/0 (socket says 172.21.15.3:49368) 2024-09-21T06:15:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.523+0000 7fd0b3fff640 1 -- 172.21.15.3:0/1601569586 learned_addr learned my addr 172.21.15.3:0/1601569586 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.523+0000 7fd0b8cbc640 1 --2- 172.21.15.3:0/1601569586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960da2d80 0x5619611865b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.523+0000 7fd0b94bd640 1 --2- 172.21.15.3:0/1601569586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dc34d0 0x561961196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.523+0000 7fd0b3fff640 1 -- 172.21.15.3:0/1601569586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960da2d80 msgr2=0x5619611865b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.523+0000 7fd0b3fff640 1 --2- 172.21.15.3:0/1601569586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960da2d80 0x5619611865b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.523+0000 7fd0b3fff640 1 -- 172.21.15.3:0/1601569586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dc34d0 msgr2=0x561961196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.523+0000 7fd0b3fff640 1 --2- 172.21.15.3:0/1601569586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dc34d0 0x561961196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.523+0000 7fd0b3fff640 1 -- 172.21.15.3:0/1601569586 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561961198340 con 0x561960dbb2f0 2024-09-21T06:15:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.523+0000 7fd0b8cbc640 1 --2- 172.21.15.3:0/1601569586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960da2d80 0x5619611865b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.523+0000 7fd0b3fff640 1 --2- 172.21.15.3:0/1601569586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960dbb2f0 0x5619611a5720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009900 tx=0x7fd0a0040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c312f6b717d6ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.524+0000 7fd09e7fc640 1 -- 172.21.15.3:0/1601569586 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0035040 con 0x561960dbb2f0 2024-09-21T06:15:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.524+0000 7fd09e7fc640 1 -- 172.21.15.3:0/1601569586 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0a0057070 con 0x561960dbb2f0 2024-09-21T06:15:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.524+0000 7fd09e7fc640 1 -- 172.21.15.3:0/1601569586 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0004a80 con 0x561960dbb2f0 2024-09-21T06:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.525+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/1601569586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960dbb2f0 msgr2=0x5619611a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.525+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/1601569586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960dbb2f0 0x5619611a5720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009900 tx=0x7fd0a0040000 comp rx=0 tx=0).stop 2024-09-21T06:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.525+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/1601569586 shutdown_connections 2024-09-21T06:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.525+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/1601569586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dc34d0 0x561961196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.525+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/1601569586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960dbb2f0 0x5619611a5720 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.525+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/1601569586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960da2d80 0x5619611865b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.525+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/1601569586 >> 172.21.15.3:0/1601569586 conn(0x561960dd8300 msgr2=0x561961185ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.525+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/1601569586 shutdown_connections 2024-09-21T06:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.525+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/1601569586 wait complete. 2024-09-21T06:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.526+0000 7fd0c1e8a740 1 -- start start 2024-09-21T06:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.526+0000 7fd0c1e8a740 1 Processor -- start 2024-09-21T06:15:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.526+0000 7fd0c1e8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960da2d80 0x5619611a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.526+0000 7fd0c1e8a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x56196119d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.526+0000 7fd0c1e8a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960dc34d0 0x56196119d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.526+0000 7fd0c1e8a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41aea10 con 0x561960dbb2f0 2024-09-21T06:15:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.526+0000 7fd0c1e8a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41ae340 con 0x561960da2d80 2024-09-21T06:15:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.526+0000 7fd0c1e8a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41ab580 con 0x561960dc34d0 2024-09-21T06:15:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.526+0000 7fd0c1e8a740 10 client.0 _ll_register_callbacks cb 0x7fd0a801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.527+0000 7fd0b8cbc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960da2d80 0x5619611a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.527+0000 7fd0b3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x56196119d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.527+0000 7fd0b8cbc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960da2d80 0x5619611a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49382/0 (socket says 172.21.15.3:49382) 2024-09-21T06:15:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.527+0000 7fd0b8cbc640 1 -- 172.21.15.3:0/3846222876 learned_addr learned my addr 172.21.15.3:0/3846222876 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.527+0000 7fd0b94bd640 1 --2- 172.21.15.3:0/3846222876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960dc34d0 0x56196119d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.527+0000 7fd0b3fff640 1 -- 172.21.15.3:0/3846222876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960dc34d0 msgr2=0x56196119d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.527+0000 7fd0b3fff640 1 --2- 172.21.15.3:0/3846222876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960dc34d0 0x56196119d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.527+0000 7fd0b3fff640 1 -- 172.21.15.3:0/3846222876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960da2d80 msgr2=0x5619611a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.527+0000 7fd0b3fff640 1 --2- 172.21.15.3:0/3846222876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960da2d80 0x5619611a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.527+0000 7fd0b3fff640 1 -- 172.21.15.3:0/3846222876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56196119dc10 con 0x561960dbb2f0 2024-09-21T06:15:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.527+0000 7fd0b3fff640 1 --2- 172.21.15.3:0/3846222876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x56196119d190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd0a002a910 tx=0x7fd0a0031520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51547861bc2f542 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.527+0000 7fd093fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:15:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.527+0000 7fd093fff640 1 -- 172.21.15.3:0/3846222876 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0035030 con 0x561960dbb2f0 2024-09-21T06:15:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.528+0000 7fd093fff640 1 -- 172.21.15.3:0/3846222876 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fd0a0005240 con 0x561960dbb2f0 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.528+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/3846222876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0a0057070 con 0x561960dbb2f0 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.528+0000 7fd093fff640 1 -- 172.21.15.3:0/3846222876 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0a00318d0 con 0x561960dbb2f0 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.528+0000 7fd0c1e8a740 10 client.15732 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.528+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/3846222876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5619612159f0 con 0x561960dbb2f0 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.528+0000 7fd093fff640 1 -- 172.21.15.3:0/3846222876 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0a0005440 con 0x561960dbb2f0 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.528+0000 7fd093fff640 1 -- 172.21.15.3:0/3846222876 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd0a00039c0 con 0x561960dbb2f0 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.528+0000 7fd0c1e8a740 10 client.15732 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.528+0000 7fd0c1e8a740 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.528+0000 7fd0c1e8a740 20 client.15732 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.528+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/3846222876 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561960de41b0 0x561961218100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.529+0000 7fd0c1e8a740 4 client.15732 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.529+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/3846222876 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fd0a0057070 con 0x561960de41b0 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.529+0000 7fd0b8cbc640 1 --2- 172.21.15.3:0/3846222876 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561960de41b0 0x561961218100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.529+0000 7fd093fff640 10 client.15732 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.529+0000 7fd0b8cbc640 1 --2- 172.21.15.3:0/3846222876 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561960de41b0 0x561961218100 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46605cbfa7521ca4 server_cookie=f8fa650e1a1b3192 in_seq=0 out_seq=0 2024-09-21T06:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.536+0000 7fd093fff640 1 -- 172.21.15.3:0/3846222876 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fd0a0057070 con 0x561960de41b0 2024-09-21T06:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.536+0000 7fd093fff640 10 client.15732 handle_command_reply: tid=0 2024-09-21T06:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.536+0000 7fd0c1e8a740 1 client.15732 shutdown 2024-09-21T06:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.536+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/3846222876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 msgr2=0x56196119d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.536+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/3846222876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x56196119d190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd0a002a910 tx=0x7fd0a0031520 comp rx=0 tx=0).stop 2024-09-21T06:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.536+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/3846222876 shutdown_connections 2024-09-21T06:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.536+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/3846222876 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561960de41b0 0x561961218100 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.536+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/3846222876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561960dc34d0 0x56196119d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.536+0000 7fd093fff640 0 client.15732 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.536+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/3846222876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561960dbb2f0 0x56196119d190 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.536+0000 7fd0c1e8a740 1 --2- 172.21.15.3:0/3846222876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561960da2d80 0x5619611a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.536+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/3846222876 >> 172.21.15.3:0/3846222876 conn(0x561960dd8300 msgr2=0x56196118a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.536+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/3846222876 shutdown_connections 2024-09-21T06:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.537+0000 7fd0c1e8a740 1 -- 172.21.15.3:0/3846222876 wait complete. 2024-09-21T06:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.537+0000 7fd0c1e8a740 20 client.15732 trim_cache size 0 max 16384 2024-09-21T06:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.537+0000 7fd0baf20640 1 -- 172.21.15.3:0/2949987074 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0a40b6680 msgr2=0x7fd0a40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.537+0000 7fd0baf20640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0a40b6680 0x7fd0a40b8b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009230 tx=0x7fd0a0031400 comp rx=0 tx=0).stop 2024-09-21T06:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.537+0000 7fd0baf20640 1 -- 172.21.15.3:0/2949987074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0b4109d70 msgr2=0x7fd0b419fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.537+0000 7fd0baf20640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0b4109d70 0x7fd0b419fca0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00eb10 tx=0x7fd0ac00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.538+0000 7fd0baf20640 1 -- 172.21.15.3:0/2949987074 shutdown_connections 2024-09-21T06:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.538+0000 7fd0baf20640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0a40b6680 0x7fd0a40b8b90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.538+0000 7fd0baf20640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0b4109d70 0x7fd0b419fca0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.538+0000 7fd0baf20640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0b4109420 0x7fd0b419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.538+0000 7fd0baf20640 1 --2- 172.21.15.3:0/2949987074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b4103af0 0x7fd0b419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.538+0000 7fd0baf20640 1 -- 172.21.15.3:0/2949987074 >> 172.21.15.3:0/2949987074 conn(0x7fd0b40ff570 msgr2=0x7fd0b4101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.538+0000 7fd0baf20640 1 -- 172.21.15.3:0/2949987074 shutdown_connections 2024-09-21T06:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:15:58.538+0000 7fd0baf20640 1 -- 172.21.15.3:0/2949987074 wait complete. 2024-09-21T06:15:58.889 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:15:58 smithi003 ceph-mon[31227]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T06:15:58.918 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:15:59.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:15:58 smithi137 ceph-mon[34464]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T06:15:59.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:15:58 smithi204 ceph-mon[34659]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T06:15:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.770+0000 7f8cad74c640 1 -- 172.21.15.3:0/479276268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca8104a30 msgr2=0x7f8ca8104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.770+0000 7f8cad74c640 1 --2- 172.21.15.3:0/479276268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca8104a30 0x7f8ca8104e40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009900 tx=0x7f8c9802a410 comp rx=0 tx=0).stop 2024-09-21T06:15:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.770+0000 7f8cad74c640 1 -- 172.21.15.3:0/479276268 shutdown_connections 2024-09-21T06:15:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.770+0000 7f8cad74c640 1 --2- 172.21.15.3:0/479276268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ca810de60 0x7f8ca8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.770+0000 7f8cad74c640 1 --2- 172.21.15.3:0/479276268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca8105380 0x7f8ca810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.770+0000 7f8cad74c640 1 --2- 172.21.15.3:0/479276268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca8104a30 0x7f8ca8104e40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.770+0000 7f8cad74c640 1 -- 172.21.15.3:0/479276268 >> 172.21.15.3:0/479276268 conn(0x7f8ca8079760 msgr2=0x7f8ca8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.770+0000 7f8cad74c640 1 -- 172.21.15.3:0/479276268 shutdown_connections 2024-09-21T06:15:59.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.770+0000 7f8cad74c640 1 -- 172.21.15.3:0/479276268 wait complete. 2024-09-21T06:15:59.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.771+0000 7f8cad74c640 1 Processor -- start 2024-09-21T06:15:59.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.771+0000 7f8cad74c640 1 -- start start 2024-09-21T06:15:59.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.772+0000 7f8cad74c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca8104a30 0x7f8ca819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.772+0000 7f8cad74c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ca8105380 0x7f8ca819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.772+0000 7f8cad74c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca810de60 0x7f8ca819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.772+0000 7f8cad74c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca8113aa0 con 0x7f8ca8104a30 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.772+0000 7f8cad74c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca8113920 con 0x7f8ca810de60 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.772+0000 7f8cad74c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca8113c20 con 0x7f8ca8105380 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.772+0000 7f8ca67fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ca8105380 0x7f8ca819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.772+0000 7f8ca77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca810de60 0x7f8ca819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.772+0000 7f8ca67fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ca8105380 0x7f8ca819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41604/0 (socket says 172.21.15.3:41604) 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.772+0000 7f8ca67fc640 1 -- 172.21.15.3:0/2939778761 learned_addr learned my addr 172.21.15.3:0/2939778761 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.772+0000 7f8ca77fe640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca810de60 0x7f8ca819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49400/0 (socket says 172.21.15.3:49400) 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.772+0000 7f8ca6ffd640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca8104a30 0x7f8ca819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.773+0000 7f8ca77fe640 1 -- 172.21.15.3:0/2939778761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ca8105380 msgr2=0x7f8ca819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.773+0000 7f8ca77fe640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ca8105380 0x7f8ca819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.773+0000 7f8ca77fe640 1 -- 172.21.15.3:0/2939778761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca8104a30 msgr2=0x7f8ca819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.773+0000 7f8ca77fe640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca8104a30 0x7f8ca819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.773+0000 7f8ca77fe640 1 -- 172.21.15.3:0/2939778761 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca81a0480 con 0x7f8ca810de60 2024-09-21T06:15:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.773+0000 7f8ca77fe640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca810de60 0x7f8ca819fd50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000e940 tx=0x7f8ca000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.773+0000 7f8c8ffff640 1 -- 172.21.15.3:0/2939778761 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0004d20 con 0x7f8ca810de60 2024-09-21T06:15:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.773+0000 7f8c8ffff640 1 -- 172.21.15.3:0/2939778761 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca0005020 con 0x7f8ca810de60 2024-09-21T06:15:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.773+0000 7f8cad74c640 1 -- 172.21.15.3:0/2939778761 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca81a0770 con 0x7f8ca810de60 2024-09-21T06:15:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.773+0000 7f8c8ffff640 1 -- 172.21.15.3:0/2939778761 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca00116b0 con 0x7f8ca810de60 2024-09-21T06:15:59.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.774+0000 7f8cad74c640 1 -- 172.21.15.3:0/2939778761 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca81a6ed0 con 0x7f8ca810de60 2024-09-21T06:15:59.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.776+0000 7f8c8ffff640 1 -- 172.21.15.3:0/2939778761 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 154636+0+0 (secure 0 0 0) 0x7f8ca0005460 con 0x7f8ca810de60 2024-09-21T06:15:59.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.777+0000 7f8cb3cc0740 1 Processor -- start 2024-09-21T06:15:59.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.777+0000 7f8cb3cc0740 1 -- start start 2024-09-21T06:15:59.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.777+0000 7f8c8ffff640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8c900b6700 0x7f8c900b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.778+0000 7f8cb3cc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babe96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.778+0000 7f8cb3cc0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7babe9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.778+0000 7f8cb3cc0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7babea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.778+0000 7f8ca67fc640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8c900b6700 0x7f8c900b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.778+0000 7f8cb3cc0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d7bab5fea0 con 0x55d7ba8191b0 2024-09-21T06:15:59.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.778+0000 7f8cb3cc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d7bab4f2d0 con 0x55d7ba842070 2024-09-21T06:15:59.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.778+0000 7f8cb3cc0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d7bab530e0 con 0x55d7ba821390 2024-09-21T06:15:59.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.778+0000 7f8c8ffff640 1 -- 172.21.15.3:0/2939778761 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8ca00e76c0 con 0x7f8ca810de60 2024-09-21T06:15:59.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.778+0000 7f8ca6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babe96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.778+0000 7f8ca67fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7babe9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.778+0000 7f8ca77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7babea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.778+0000 7f8ca77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7babea120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41620/0 (socket says 172.21.15.3:41620) 2024-09-21T06:15:59.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.778+0000 7f8ca77fe640 1 -- 172.21.15.3:0/2314626361 learned_addr learned my addr 172.21.15.3:0/2314626361 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:59.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.779+0000 7f8ca67fc640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8c900b6700 0x7f8c900b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c004650 tx=0x7f8c9c00bf40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:59.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.779+0000 7f8ca77fe640 1 -- 172.21.15.3:0/2314626361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 msgr2=0x55d7babe96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:59.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.779+0000 7f8ca77fe640 1 --2- 172.21.15.3:0/2314626361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babe96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.779+0000 7f8ca77fe640 1 -- 172.21.15.3:0/2314626361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 msgr2=0x55d7babe9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:59.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.779+0000 7f8ca77fe640 1 --2- 172.21.15.3:0/2314626361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7babe9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.779+0000 7f8ca77fe640 1 -- 172.21.15.3:0/2314626361 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ba8ad650 con 0x55d7ba821390 2024-09-21T06:15:59.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.779+0000 7f8ca67fc640 1 --2- 172.21.15.3:0/2314626361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7babe9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:59.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.780+0000 7f8ca77fe640 1 --2- 172.21.15.3:0/2314626361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7babea120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00a9b70 tx=0x7f8ca00acd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f96def985b661d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:59.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.780+0000 7f8c8d7fa640 1 -- 172.21.15.3:0/2314626361 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca001f040 con 0x55d7ba821390 2024-09-21T06:15:59.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.780+0000 7f8c8d7fa640 1 -- 172.21.15.3:0/2314626361 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca00b1fd0 con 0x55d7ba821390 2024-09-21T06:15:59.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.780+0000 7f8c8d7fa640 1 -- 172.21.15.3:0/2314626361 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca00a91e0 con 0x55d7ba821390 2024-09-21T06:15:59.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.781+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/2314626361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 msgr2=0x55d7babea120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:59.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.781+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/2314626361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7babea120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00a9b70 tx=0x7f8ca00acd10 comp rx=0 tx=0).stop 2024-09-21T06:15:59.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.782+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/2314626361 shutdown_connections 2024-09-21T06:15:59.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.782+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/2314626361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7babea120 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.782+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/2314626361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7babe9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.782+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/2314626361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babe96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.782+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/2314626361 >> 172.21.15.3:0/2314626361 conn(0x55d7ba836460 msgr2=0x55d7babe7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:59.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.782+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/2314626361 shutdown_connections 2024-09-21T06:15:59.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.782+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/2314626361 wait complete. 2024-09-21T06:15:59.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.782+0000 7f8cb3cc0740 1 -- start start 2024-09-21T06:15:59.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.783+0000 7f8cb3cc0740 1 Processor -- start 2024-09-21T06:15:59.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.783+0000 7f8cb3cc0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba8191b0 0x55d7babf4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.783+0000 7f8cb3cc0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba821390 0x55d7babf5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8cb3cc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babf57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8cb3cc0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d7ba9b0700 con 0x55d7ba821390 2024-09-21T06:15:59.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8ca67fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba821390 0x55d7babf5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8ca77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babf57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8ca77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babf57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49426/0 (socket says 172.21.15.3:49426) 2024-09-21T06:15:59.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8ca77fe640 1 -- 172.21.15.3:0/1617724457 learned_addr learned my addr 172.21.15.3:0/1617724457 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:59.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/1617724457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d7baaea160 con 0x55d7ba842070 2024-09-21T06:15:59.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/1617724457 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d7babf7e20 con 0x55d7ba8191b0 2024-09-21T06:15:59.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8cb3cc0740 10 client.0 _ll_register_callbacks cb 0x55d7bab039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:59.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8ca77fe640 1 -- 172.21.15.3:0/1617724457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba8191b0 msgr2=0x55d7babf4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:59.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8ca6ffd640 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba8191b0 0x55d7babf4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8ca77fe640 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba8191b0 0x55d7babf4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8ca77fe640 1 -- 172.21.15.3:0/1617724457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba821390 msgr2=0x55d7babf5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:59.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8ca77fe640 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba821390 0x55d7babf5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.784+0000 7f8ca77fe640 1 -- 172.21.15.3:0/1617724457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7bab17c90 con 0x55d7ba842070 2024-09-21T06:15:59.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.785+0000 7f8ca67fc640 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba821390 0x55d7babf5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:59.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.785+0000 7f8ca6ffd640 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba8191b0 0x55d7babf4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:59.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.785+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/1617724457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ca00b1fd0 con 0x55d7ba842070 2024-09-21T06:15:59.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.785+0000 7f8ca77fe640 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babf57b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00ad2a0 tx=0x7f8ca00aca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9841275259f6fd11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:59.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.785+0000 7f8c8aff5640 10 client.25352 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:15:59.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.785+0000 7f8c8aff5640 1 -- 172.21.15.3:0/1617724457 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0058e60 con 0x55d7ba842070 2024-09-21T06:15:59.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.785+0000 7f8c8aff5640 1 -- 172.21.15.3:0/1617724457 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8ca00c8b40 con 0x55d7ba842070 2024-09-21T06:15:59.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.785+0000 7f8c8aff5640 1 -- 172.21.15.3:0/1617724457 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8ca001f070 con 0x55d7ba842070 2024-09-21T06:15:59.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.785+0000 7f8cb3cc0740 10 client.25352 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:59.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.785+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/1617724457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55d7bac039d0 con 0x55d7ba842070 2024-09-21T06:15:59.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.786+0000 7f8c8aff5640 1 -- 172.21.15.3:0/1617724457 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8ca0094ce0 con 0x55d7ba842070 2024-09-21T06:15:59.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.786+0000 7f8cb3cc0740 10 client.25352 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:59.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.786+0000 7f8cb3cc0740 10 client.25352 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:15:59.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.786+0000 7f8cb3cc0740 20 client.25352 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:59.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.786+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55d7ba8009f0 0x55d7bac06d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.786+0000 7f8cb3cc0740 4 client.25352 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:59.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.786+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/1617724457 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7bab61df0 con 0x55d7ba8009f0 2024-09-21T06:15:59.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.787+0000 7f8ca6ffd640 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55d7ba8009f0 0x55d7bac06d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.787+0000 7f8ca6ffd640 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55d7ba8009f0 0x55d7bac06d30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39ce48b844d82d6d server_cookie=4a72980e28f07102 in_seq=0 out_seq=0 2024-09-21T06:15:59.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.787+0000 7f8c8aff5640 10 client.25352 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:15:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.788+0000 7f8c8aff5640 1 -- 172.21.15.3:0/1617724457 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d7bab61df0 con 0x55d7ba8009f0 2024-09-21T06:15:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.788+0000 7f8c8aff5640 10 client.25352 handle_command_reply: tid=0 2024-09-21T06:15:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.788+0000 7f8cb3cc0740 1 client.25352 shutdown 2024-09-21T06:15:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.788+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/1617724457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 msgr2=0x55d7babf57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.788+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babf57b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00ad2a0 tx=0x7f8ca00aca30 comp rx=0 tx=0).stop 2024-09-21T06:15:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.788+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/1617724457 shutdown_connections 2024-09-21T06:15:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.788+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55d7ba8009f0 0x55d7bac06d30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.788+0000 7f8c8aff5640 0 client.25352 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:15:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.788+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babf57b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.788+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba821390 0x55d7babf5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.788+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/1617724457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba8191b0 0x55d7babf4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.788+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/1617724457 >> 172.21.15.3:0/1617724457 conn(0x55d7ba836460 msgr2=0x55d7babec830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:59.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.788+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/1617724457 shutdown_connections 2024-09-21T06:15:59.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.789+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/1617724457 wait complete. 2024-09-21T06:15:59.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.789+0000 7f8cb3cc0740 20 client.25352 trim_cache size 0 max 16384 2024-09-21T06:15:59.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.805+0000 7f8cb3cc0740 1 Processor -- start 2024-09-21T06:15:59.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.805+0000 7f8cb3cc0740 1 -- start start 2024-09-21T06:15:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.805+0000 7f8cb3cc0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7babeb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.805+0000 7f8cb3cc0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7babf6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.805+0000 7f8cb3cc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babe6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.805+0000 7f8cb3cc0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81ad500 con 0x55d7ba8191b0 2024-09-21T06:15:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.805+0000 7f8cb3cc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81af040 con 0x55d7ba842070 2024-09-21T06:15:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.805+0000 7f8cb3cc0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81adbd0 con 0x55d7ba821390 2024-09-21T06:15:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.806+0000 7f8ca6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7babeb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.806+0000 7f8ca6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7babeb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48890/0 (socket says 172.21.15.3:48890) 2024-09-21T06:15:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.806+0000 7f8ca6ffd640 1 -- 172.21.15.3:0/2620338814 learned_addr learned my addr 172.21.15.3:0/2620338814 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.806+0000 7f8ca77fe640 1 --2- 172.21.15.3:0/2620338814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babe6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.806+0000 7f8ca6ffd640 1 -- 172.21.15.3:0/2620338814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 msgr2=0x55d7babf6940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:15:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.806+0000 7f8ca6ffd640 1 --2- 172.21.15.3:0/2620338814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7babf6940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.806+0000 7f8ca6ffd640 1 -- 172.21.15.3:0/2620338814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 msgr2=0x55d7babe6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:59.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.806+0000 7f8ca6ffd640 1 --2- 172.21.15.3:0/2620338814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babe6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.806+0000 7f8ca6ffd640 1 -- 172.21.15.3:0/2620338814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7babf5fd0 con 0x55d7ba8191b0 2024-09-21T06:15:59.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.806+0000 7f8ca6ffd640 1 --2- 172.21.15.3:0/2620338814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7babeb090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009900 tx=0x7f8c98031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e180048fb17ddcf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:59.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.807+0000 7f8c8cff9640 1 -- 172.21.15.3:0/2620338814 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98035040 con 0x55d7ba8191b0 2024-09-21T06:15:59.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.807+0000 7f8c8cff9640 1 -- 172.21.15.3:0/2620338814 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c98048070 con 0x55d7ba8191b0 2024-09-21T06:15:59.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.807+0000 7f8c8cff9640 1 -- 172.21.15.3:0/2620338814 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98004a40 con 0x55d7ba8191b0 2024-09-21T06:15:59.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.808+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/2620338814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 msgr2=0x55d7babeb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:59.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.808+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/2620338814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7babeb090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009900 tx=0x7f8c98031400 comp rx=0 tx=0).stop 2024-09-21T06:15:59.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.808+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/2620338814 shutdown_connections 2024-09-21T06:15:59.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.808+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/2620338814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7babf6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.808+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/2620338814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7babe6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.808+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/2620338814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7babeb090 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.808+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/2620338814 >> 172.21.15.3:0/2620338814 conn(0x55d7ba836460 msgr2=0x55d7babf6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:59.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.809+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/2620338814 shutdown_connections 2024-09-21T06:15:59.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.809+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/2620338814 wait complete. 2024-09-21T06:15:59.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.809+0000 7f8cb3cc0740 1 -- start start 2024-09-21T06:15:59.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.809+0000 7f8cb3cc0740 1 Processor -- start 2024-09-21T06:15:59.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8cb3cc0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7bac0a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8cb3cc0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7bac07c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8ca6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7bac0a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8cb3cc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7bac08370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8ca6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7bac0a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48900/0 (socket says 172.21.15.3:48900) 2024-09-21T06:15:59.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8cb3cc0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81ae970 con 0x55d7ba8191b0 2024-09-21T06:15:59.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8cb3cc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81ae2a0 con 0x55d7ba842070 2024-09-21T06:15:59.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8cb3cc0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81ab4e0 con 0x55d7ba821390 2024-09-21T06:15:59.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8cb3cc0740 10 client.0 _ll_register_callbacks cb 0x7f8c9c004280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:59.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8ca67fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7bac07c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8ca67fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7bac07c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41640/0 (socket says 172.21.15.3:41640) 2024-09-21T06:15:59.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8ca67fc640 1 -- 172.21.15.3:0/621243971 learned_addr learned my addr 172.21.15.3:0/621243971 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:15:59.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8ca77fe640 1 --2- 172.21.15.3:0/621243971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7bac08370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8ca67fc640 1 -- 172.21.15.3:0/621243971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 msgr2=0x55d7bac08370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:59.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8ca67fc640 1 --2- 172.21.15.3:0/621243971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7bac08370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8ca67fc640 1 -- 172.21.15.3:0/621243971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 msgr2=0x55d7bac0a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8ca67fc640 1 --2- 172.21.15.3:0/621243971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7bac0a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.810+0000 7f8ca67fc640 1 -- 172.21.15.3:0/621243971 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7bac704b0 con 0x55d7ba821390 2024-09-21T06:15:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.811+0000 7f8ca6ffd640 1 --2- 172.21.15.3:0/621243971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7bac0a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T06:15:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.811+0000 7f8ca67fc640 1 --2- 172.21.15.3:0/621243971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7bac07c60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c005fd0 tx=0x7f8c9c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11695280edaa2fcd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.811+0000 7f8c8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:15:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.811+0000 7f8c8aff5640 1 -- 172.21.15.3:0/621243971 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c011d90 con 0x55d7ba821390 2024-09-21T06:15:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.811+0000 7f8c8aff5640 1 -- 172.21.15.3:0/621243971 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8c9c02a480 con 0x55d7ba821390 2024-09-21T06:15:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.811+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/621243971 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c98048070 con 0x55d7ba821390 2024-09-21T06:15:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.812+0000 7f8c8aff5640 1 -- 172.21.15.3:0/621243971 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8c9c02a680 con 0x55d7ba821390 2024-09-21T06:15:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.812+0000 7f8cb3cc0740 10 client.25333 fetch_fsmap learned FSMap version 28 2024-09-21T06:15:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.812+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/621243971 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7bac71600 con 0x55d7ba821390 2024-09-21T06:15:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.812+0000 7f8c8aff5640 1 -- 172.21.15.3:0/621243971 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c02acd0 con 0x55d7ba821390 2024-09-21T06:15:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.813+0000 7f8c8aff5640 1 -- 172.21.15.3:0/621243971 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8c9c01f940 con 0x55d7ba821390 2024-09-21T06:15:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.813+0000 7f8cb3cc0740 10 client.25333 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:15:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.813+0000 7f8cb3cc0740 10 client.25333 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.813+0000 7f8cb3cc0740 20 client.25333 populate_metadata read hostname 'smithi003' 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.813+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/621243971 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55d7ba7d5da0 0x55d7bac73d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.813+0000 7f8cb3cc0740 4 client.25333 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.813+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/621243971 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8c98048070 con 0x55d7ba7d5da0 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.813+0000 7f8ca6ffd640 1 --2- 172.21.15.3:0/621243971 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55d7ba7d5da0 0x55d7bac73d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8c8aff5640 10 client.25333 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8ca6ffd640 1 --2- 172.21.15.3:0/621243971 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55d7ba7d5da0 0x55d7bac73d70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80b002db445933d9 server_cookie=bb9e737cd0e926c6 in_seq=0 out_seq=0 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8c8aff5640 1 -- 172.21.15.3:0/621243971 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8c98048070 con 0x55d7ba7d5da0 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8c8aff5640 10 client.25333 handle_command_reply: tid=0 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8cb3cc0740 1 client.25333 shutdown 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/621243971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 msgr2=0x55d7bac07c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/621243971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7bac07c60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c005fd0 tx=0x7f8c9c004730 comp rx=0 tx=0).stop 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/621243971 shutdown_connections 2024-09-21T06:15:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/621243971 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55d7ba7d5da0 0x55d7bac73d70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/621243971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7ba821390 0x55d7bac07c60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/621243971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba842070 0x55d7bac08370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8cb3cc0740 1 --2- 172.21.15.3:0/621243971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ba8191b0 0x55d7bac0a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/621243971 >> 172.21.15.3:0/621243971 conn(0x55d7ba836460 msgr2=0x55d7babead70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:59.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.814+0000 7f8c8aff5640 0 client.25333 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:15:59.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.816+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/621243971 shutdown_connections 2024-09-21T06:15:59.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.816+0000 7f8cb3cc0740 1 -- 172.21.15.3:0/621243971 wait complete. 2024-09-21T06:15:59.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.818+0000 7f8cb3cc0740 20 client.25333 trim_cache size 0 max 16384 2024-09-21T06:15:59.819 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:15:59.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.818+0000 7f8cad74c640 1 -- 172.21.15.3:0/2939778761 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8c900b6700 msgr2=0x7f8c900b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:59.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.818+0000 7f8cad74c640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8c900b6700 0x7f8c900b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c004650 tx=0x7f8c9c00bf40 comp rx=0 tx=0).stop 2024-09-21T06:15:59.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.818+0000 7f8cad74c640 1 -- 172.21.15.3:0/2939778761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca810de60 msgr2=0x7f8ca819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:59.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.818+0000 7f8cad74c640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca810de60 0x7f8ca819fd50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000e940 tx=0x7f8ca000ee00 comp rx=0 tx=0).stop 2024-09-21T06:15:59.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.819+0000 7f8cad74c640 1 -- 172.21.15.3:0/2939778761 shutdown_connections 2024-09-21T06:15:59.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.819+0000 7f8cad74c640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8c900b6700 0x7f8c900b8c10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.819+0000 7f8cad74c640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ca8105380 0x7f8ca819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.819+0000 7f8cad74c640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca810de60 0x7f8ca819fd50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.819+0000 7f8cad74c640 1 --2- 172.21.15.3:0/2939778761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca8104a30 0x7f8ca819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:59.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.819+0000 7f8cad74c640 1 -- 172.21.15.3:0/2939778761 >> 172.21.15.3:0/2939778761 conn(0x7f8ca8079760 msgr2=0x7f8ca8100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:59.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.819+0000 7f8cad74c640 1 -- 172.21.15.3:0/2939778761 shutdown_connections 2024-09-21T06:15:59.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:15:59.819+0000 7f8cad74c640 1 -- 172.21.15.3:0/2939778761 wait complete. 2024-09-21T06:16:00.590 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:16:00 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:16:00.152+0000 7fc4be0bd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:16:00.590 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:16:00 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:16:00.153+0000 7fc4be0bd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:16:00.590 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:16:00 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:16:00.153+0000 7fc4be0bd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:16:00.590 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:16:00 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:16:00.153+0000 7fc4be0bd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:16:00.590 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:16:00 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:16:00.153+0000 7fc4be0bd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:16:00.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:16:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:01.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:00 smithi003 ceph-mon[31227]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T06:16:01.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:00 smithi137 ceph-mon[34464]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T06:16:01.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:00 smithi204 ceph-mon[34659]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T06:16:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.539+0000 7ff8d5313640 1 -- 172.21.15.3:0/628142755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8d0109500 msgr2=0x7ff8d0109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.539+0000 7ff8d5313640 1 --2- 172.21.15.3:0/628142755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8d0109500 0x7ff8d0109910 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009900 tx=0x7ff8c402a410 comp rx=0 tx=0).stop 2024-09-21T06:16:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.540+0000 7ff8d5313640 1 -- 172.21.15.3:0/628142755 shutdown_connections 2024-09-21T06:16:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.540+0000 7ff8d5313640 1 --2- 172.21.15.3:0/628142755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8d0109e50 0x7ff8d0110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.540+0000 7ff8d5313640 1 --2- 172.21.15.3:0/628142755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8d0109500 0x7ff8d0109910 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.540+0000 7ff8d5313640 1 --2- 172.21.15.3:0/628142755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8d0102050 0x7ff8d0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.540+0000 7ff8d5313640 1 -- 172.21.15.3:0/628142755 >> 172.21.15.3:0/628142755 conn(0x7ff8d0079e10 msgr2=0x7ff8d007a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.540+0000 7ff8d5313640 1 -- 172.21.15.3:0/628142755 shutdown_connections 2024-09-21T06:16:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.540+0000 7ff8d5313640 1 -- 172.21.15.3:0/628142755 wait complete. 2024-09-21T06:16:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.541+0000 7ff8d5313640 1 Processor -- start 2024-09-21T06:16:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.541+0000 7ff8d5313640 1 -- start start 2024-09-21T06:16:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.541+0000 7ff8d5313640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8d0102050 0x7ff8d019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.541+0000 7ff8d5313640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8d0109500 0x7ff8d019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.541+0000 7ff8d5313640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8d0109e50 0x7ff8d019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.541+0000 7ff8d5313640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d0113b00 con 0x7ff8d0109500 2024-09-21T06:16:01.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.542+0000 7ff8d5313640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d0113980 con 0x7ff8d0109e50 2024-09-21T06:16:01.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.542+0000 7ff8d5313640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d0113c80 con 0x7ff8d0102050 2024-09-21T06:16:01.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.542+0000 7ff8ce7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8d0109500 0x7ff8d019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.542+0000 7ff8ce7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8d0109500 0x7ff8d019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48928/0 (socket says 172.21.15.3:48928) 2024-09-21T06:16:01.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.542+0000 7ff8ce7fc640 1 -- 172.21.15.3:0/44335868 learned_addr learned my addr 172.21.15.3:0/44335868 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:01.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.542+0000 7ff8cf7fe640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8d0109e50 0x7ff8d019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.542+0000 7ff8ceffd640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8d0102050 0x7ff8d019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.542+0000 7ff8ce7fc640 1 -- 172.21.15.3:0/44335868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8d0102050 msgr2=0x7ff8d019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.542+0000 7ff8ce7fc640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8d0102050 0x7ff8d019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.542+0000 7ff8ce7fc640 1 -- 172.21.15.3:0/44335868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8d0109e50 msgr2=0x7ff8d019fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.542+0000 7ff8ce7fc640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8d0109e50 0x7ff8d019fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.542+0000 7ff8ce7fc640 1 -- 172.21.15.3:0/44335868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d01a0460 con 0x7ff8d0109500 2024-09-21T06:16:01.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.542+0000 7ff8ce7fc640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8d0109500 0x7ff8d019ab50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff8c402a930 tx=0x7ff8c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:01.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.543+0000 7ff8b7fff640 1 -- 172.21.15.3:0/44335868 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c403f030 con 0x7ff8d0109500 2024-09-21T06:16:01.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.543+0000 7ff8b7fff640 1 -- 172.21.15.3:0/44335868 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c4003aa0 con 0x7ff8d0109500 2024-09-21T06:16:01.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.543+0000 7ff8d5313640 1 -- 172.21.15.3:0/44335868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d01a06f0 con 0x7ff8d0109500 2024-09-21T06:16:01.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.543+0000 7ff8b7fff640 1 -- 172.21.15.3:0/44335868 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4035070 con 0x7ff8d0109500 2024-09-21T06:16:01.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.543+0000 7ff8d5313640 1 -- 172.21.15.3:0/44335868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d01a0c20 con 0x7ff8d0109500 2024-09-21T06:16:01.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.547+0000 7ff8db887740 1 Processor -- start 2024-09-21T06:16:01.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.547+0000 7ff8db887740 1 -- start start 2024-09-21T06:16:01.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.547+0000 7ff8b7fff640 1 -- 172.21.15.3:0/44335868 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 154636+0+0 (secure 0 0 0) 0x7ff8c4042030 con 0x7ff8d0109500 2024-09-21T06:16:01.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.547+0000 7ff8db887740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb2a070 0x55c81ced16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.547+0000 7ff8db887740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb011b0 0x55c81ced1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.547+0000 7ff8db887740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb09390 0x55c81ced2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.547+0000 7ff8db887740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c81ce47ea0 con 0x55c81cb09390 2024-09-21T06:16:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.547+0000 7ff8db887740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c81ce372d0 con 0x55c81cb2a070 2024-09-21T06:16:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.547+0000 7ff8db887740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c81ce3b0e0 con 0x55c81cb011b0 2024-09-21T06:16:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8ceffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb011b0 0x55c81ced1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8ceffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb011b0 0x55c81ced1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41676/0 (socket says 172.21.15.3:41676) 2024-09-21T06:16:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8ceffd640 1 -- 172.21.15.3:0/4131157976 learned_addr learned my addr 172.21.15.3:0/4131157976 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8cf7fe640 1 --2- 172.21.15.3:0/4131157976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb2a070 0x55c81ced16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8ce7fc640 1 --2- 172.21.15.3:0/4131157976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb09390 0x55c81ced2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8ceffd640 1 -- 172.21.15.3:0/4131157976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb2a070 msgr2=0x55c81ced16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8ceffd640 1 --2- 172.21.15.3:0/4131157976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb2a070 0x55c81ced16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8ceffd640 1 -- 172.21.15.3:0/4131157976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb09390 msgr2=0x55c81ced2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8ceffd640 1 --2- 172.21.15.3:0/4131157976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb09390 0x55c81ced2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8ceffd640 1 -- 172.21.15.3:0/4131157976 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c81cb95650 con 0x55c81cb011b0 2024-09-21T06:16:01.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8b7fff640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff8b80b6730 0x7ff8b80b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8ceffd640 1 --2- 172.21.15.3:0/4131157976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb011b0 0x55c81ced1be0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000b760 tx=0x7ff8c000bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=105994dc91122fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:01.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8cf7fe640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff8b80b6730 0x7ff8b80b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.548+0000 7ff8b7fff640 1 -- 172.21.15.3:0/44335868 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff8c4107c90 con 0x7ff8d0109500 2024-09-21T06:16:01.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.549+0000 7ff8b57fa640 1 -- 172.21.15.3:0/4131157976 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0003dd0 con 0x55c81cb011b0 2024-09-21T06:16:01.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.549+0000 7ff8ce7fc640 1 --2- 172.21.15.3:0/4131157976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb09390 0x55c81ced2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:16:01.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.549+0000 7ff8b57fa640 1 -- 172.21.15.3:0/4131157976 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c0015070 con 0x55c81cb011b0 2024-09-21T06:16:01.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.549+0000 7ff8b57fa640 1 -- 172.21.15.3:0/4131157976 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0004e00 con 0x55c81cb011b0 2024-09-21T06:16:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.550+0000 7ff8db887740 1 -- 172.21.15.3:0/4131157976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb011b0 msgr2=0x55c81ced1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.550+0000 7ff8cf7fe640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff8b80b6730 0x7ff8b80b8c40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8007e50 tx=0x7ff8c8007520 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.550+0000 7ff8db887740 1 --2- 172.21.15.3:0/4131157976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb011b0 0x55c81ced1be0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000b760 tx=0x7ff8c000bc20 comp rx=0 tx=0).stop 2024-09-21T06:16:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.550+0000 7ff8db887740 1 -- 172.21.15.3:0/4131157976 shutdown_connections 2024-09-21T06:16:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.550+0000 7ff8db887740 1 --2- 172.21.15.3:0/4131157976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb011b0 0x55c81ced1be0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.550+0000 7ff8db887740 1 --2- 172.21.15.3:0/4131157976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb09390 0x55c81ced2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.550+0000 7ff8db887740 1 --2- 172.21.15.3:0/4131157976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb2a070 0x55c81ced16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.550+0000 7ff8db887740 1 -- 172.21.15.3:0/4131157976 >> 172.21.15.3:0/4131157976 conn(0x55c81cb1e460 msgr2=0x55c81cecf590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.550+0000 7ff8db887740 1 -- 172.21.15.3:0/4131157976 shutdown_connections 2024-09-21T06:16:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.550+0000 7ff8db887740 1 -- 172.21.15.3:0/4131157976 wait complete. 2024-09-21T06:16:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.551+0000 7ff8db887740 1 -- start start 2024-09-21T06:16:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.551+0000 7ff8db887740 1 Processor -- start 2024-09-21T06:16:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.551+0000 7ff8db887740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb011b0 0x55c81cedccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.551+0000 7ff8db887740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb09390 0x55c81cedd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8ce7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb09390 0x55c81cedd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8ce7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb09390 0x55c81cedd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48952/0 (socket says 172.21.15.3:48952) 2024-09-21T06:16:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8db887740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb2a070 0x55c81cedd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8ce7fc640 1 -- 172.21.15.3:0/868391268 learned_addr learned my addr 172.21.15.3:0/868391268 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8db887740 1 -- 172.21.15.3:0/868391268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c81cc98700 con 0x55c81cb09390 2024-09-21T06:16:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8db887740 1 -- 172.21.15.3:0/868391268 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c81cdd2160 con 0x55c81cb011b0 2024-09-21T06:16:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8db887740 1 -- 172.21.15.3:0/868391268 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c81cedfe80 con 0x55c81cb2a070 2024-09-21T06:16:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8db887740 10 client.0 _ll_register_callbacks cb 0x55c81cdeb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8ceffd640 1 --2- 172.21.15.3:0/868391268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb011b0 0x55c81cedccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8ce7fc640 1 -- 172.21.15.3:0/868391268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb2a070 msgr2=0x55c81cedd760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:16:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8ce7fc640 1 --2- 172.21.15.3:0/868391268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb2a070 0x55c81cedd760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8ce7fc640 1 -- 172.21.15.3:0/868391268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb011b0 msgr2=0x55c81cedccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8ce7fc640 1 --2- 172.21.15.3:0/868391268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb011b0 0x55c81cedccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8ce7fc640 1 -- 172.21.15.3:0/868391268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c81cdffc90 con 0x55c81cb09390 2024-09-21T06:16:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8ce7fc640 1 --2- 172.21.15.3:0/868391268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb09390 0x55c81cedd200 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40cae40 tx=0x7ff8c4003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9754976087ec8a2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.552+0000 7ff8b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.553+0000 7ff8b2ff5640 1 -- 172.21.15.3:0/868391268 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4049030 con 0x55c81cb09390 2024-09-21T06:16:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.553+0000 7ff8b2ff5640 1 -- 172.21.15.3:0/868391268 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff8c41056a0 con 0x55c81cb09390 2024-09-21T06:16:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.553+0000 7ff8db887740 1 -- 172.21.15.3:0/868391268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8c0015070 con 0x55c81cb09390 2024-09-21T06:16:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.553+0000 7ff8b2ff5640 1 -- 172.21.15.3:0/868391268 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8c40cd6e0 con 0x55c81cb09390 2024-09-21T06:16:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.553+0000 7ff8db887740 10 client.15762 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.553+0000 7ff8db887740 1 -- 172.21.15.3:0/868391268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c81ceefeb0 con 0x55c81cb09390 2024-09-21T06:16:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.554+0000 7ff8b2ff5640 1 -- 172.21.15.3:0/868391268 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c40cee50 con 0x55c81cb09390 2024-09-21T06:16:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.554+0000 7ff8b2ff5640 1 -- 172.21.15.3:0/868391268 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff8c410c040 con 0x55c81cb09390 2024-09-21T06:16:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.554+0000 7ff8db887740 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.554+0000 7ff8db887740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.554+0000 7ff8db887740 20 client.15762 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.554+0000 7ff8db887740 1 --2- 172.21.15.3:0/868391268 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55c81cae89f0 0x55c81cef3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.554+0000 7ff8db887740 4 client.15762 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.554+0000 7ff8db887740 1 -- 172.21.15.3:0/868391268 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff8c0015070 con 0x55c81cae89f0 2024-09-21T06:16:01.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.555+0000 7ff8ceffd640 1 --2- 172.21.15.3:0/868391268 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55c81cae89f0 0x55c81cef3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.555+0000 7ff8b2ff5640 10 client.15762 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:01.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.555+0000 7ff8ceffd640 1 --2- 172.21.15.3:0/868391268 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55c81cae89f0 0x55c81cef3360 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e249511aedb5bb68 server_cookie=885fda6ef87e4884 in_seq=0 out_seq=0 2024-09-21T06:16:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8b2ff5640 1 -- 172.21.15.3:0/868391268 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff8c0015070 con 0x55c81cae89f0 2024-09-21T06:16:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8b2ff5640 10 client.15762 handle_command_reply: tid=0 2024-09-21T06:16:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8db887740 1 client.15762 shutdown 2024-09-21T06:16:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8db887740 1 -- 172.21.15.3:0/868391268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb09390 msgr2=0x55c81cedd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8db887740 1 --2- 172.21.15.3:0/868391268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb09390 0x55c81cedd200 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40cae40 tx=0x7ff8c4003f50 comp rx=0 tx=0).stop 2024-09-21T06:16:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8db887740 1 -- 172.21.15.3:0/868391268 shutdown_connections 2024-09-21T06:16:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8db887740 1 --2- 172.21.15.3:0/868391268 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55c81cae89f0 0x55c81cef3360 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8db887740 1 --2- 172.21.15.3:0/868391268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb2a070 0x55c81cedd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8db887740 1 --2- 172.21.15.3:0/868391268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb09390 0x55c81cedd200 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8db887740 1 --2- 172.21.15.3:0/868391268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb011b0 0x55c81cedccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8db887740 1 -- 172.21.15.3:0/868391268 >> 172.21.15.3:0/868391268 conn(0x55c81cb1e460 msgr2=0x55c81ced4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8b2ff5640 0 client.15762 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:01.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8db887740 1 -- 172.21.15.3:0/868391268 shutdown_connections 2024-09-21T06:16:01.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.556+0000 7ff8db887740 1 -- 172.21.15.3:0/868391268 wait complete. 2024-09-21T06:16:01.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.557+0000 7ff8db887740 20 client.15762 trim_cache size 0 max 16384 2024-09-21T06:16:01.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.583+0000 7ff8db887740 1 Processor -- start 2024-09-21T06:16:01.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.583+0000 7ff8db887740 1 -- start start 2024-09-21T06:16:01.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.583+0000 7ff8db887740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb011b0 0x55c81cedbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8db887740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb2a070 0x55c81cecebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8db887740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cabdda0 0x55c81cecf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8db887740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d01ad470 con 0x55c81cb2a070 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8db887740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d01aefb0 con 0x55c81cabdda0 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8db887740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d01adb40 con 0x55c81cb011b0 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8ce7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb2a070 0x55c81cecebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8ce7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb2a070 0x55c81cecebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48968/0 (socket says 172.21.15.3:48968) 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8ce7fc640 1 -- 172.21.15.3:0/1104694835 learned_addr learned my addr 172.21.15.3:0/1104694835 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8cf7fe640 1 --2- 172.21.15.3:0/1104694835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cabdda0 0x55c81cecf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8ceffd640 1 --2- 172.21.15.3:0/1104694835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb011b0 0x55c81cedbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8ce7fc640 1 -- 172.21.15.3:0/1104694835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb011b0 msgr2=0x55c81cedbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8ce7fc640 1 --2- 172.21.15.3:0/1104694835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb011b0 0x55c81cedbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8ce7fc640 1 -- 172.21.15.3:0/1104694835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cabdda0 msgr2=0x55c81cecf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8ce7fc640 1 --2- 172.21.15.3:0/1104694835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cabdda0 0x55c81cecf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.584+0000 7ff8ce7fc640 1 -- 172.21.15.3:0/1104694835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c81cee90a0 con 0x55c81cb2a070 2024-09-21T06:16:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.585+0000 7ff8ceffd640 1 --2- 172.21.15.3:0/1104694835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb011b0 0x55c81cedbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:16:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.585+0000 7ff8cf7fe640 1 --2- 172.21.15.3:0/1104694835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cabdda0 0x55c81cecf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.585+0000 7ff8ce7fc640 1 --2- 172.21.15.3:0/1104694835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb2a070 0x55c81cecebc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4105a20 tx=0x7ff8c40092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07c09d6dce753e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.585+0000 7ff8b4ff9640 1 -- 172.21.15.3:0/1104694835 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c40ca220 con 0x55c81cb2a070 2024-09-21T06:16:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.585+0000 7ff8b4ff9640 1 -- 172.21.15.3:0/1104694835 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c40d2600 con 0x55c81cb2a070 2024-09-21T06:16:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.585+0000 7ff8b4ff9640 1 -- 172.21.15.3:0/1104694835 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c40cd520 con 0x55c81cb2a070 2024-09-21T06:16:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.586+0000 7ff8db887740 1 -- 172.21.15.3:0/1104694835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb2a070 msgr2=0x55c81cecebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.586+0000 7ff8db887740 1 --2- 172.21.15.3:0/1104694835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb2a070 0x55c81cecebc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4105a20 tx=0x7ff8c40092e0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.587+0000 7ff8db887740 1 -- 172.21.15.3:0/1104694835 shutdown_connections 2024-09-21T06:16:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.587+0000 7ff8db887740 1 --2- 172.21.15.3:0/1104694835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cabdda0 0x55c81cecf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.587+0000 7ff8db887740 1 --2- 172.21.15.3:0/1104694835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cb2a070 0x55c81cecebc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.587+0000 7ff8db887740 1 --2- 172.21.15.3:0/1104694835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb011b0 0x55c81cedbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.587+0000 7ff8db887740 1 -- 172.21.15.3:0/1104694835 >> 172.21.15.3:0/1104694835 conn(0x55c81cb1e460 msgr2=0x55c81cedb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.587+0000 7ff8db887740 1 -- 172.21.15.3:0/1104694835 shutdown_connections 2024-09-21T06:16:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.587+0000 7ff8db887740 1 -- 172.21.15.3:0/1104694835 wait complete. 2024-09-21T06:16:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.587+0000 7ff8db887740 1 -- start start 2024-09-21T06:16:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.587+0000 7ff8db887740 1 Processor -- start 2024-09-21T06:16:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8db887740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cabdda0 0x55c81cee1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8db887740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb011b0 0x55c81cee1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cabdda0 0x55c81cee1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8db887740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb2a070 0x55c81cee1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cabdda0 0x55c81cee1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48976/0 (socket says 172.21.15.3:48976) 2024-09-21T06:16:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8ceffd640 1 -- 172.21.15.3:0/2515370903 learned_addr learned my addr 172.21.15.3:0/2515370903 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8db887740 1 -- 172.21.15.3:0/2515370903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d01ae8e0 con 0x55c81cabdda0 2024-09-21T06:16:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8db887740 1 -- 172.21.15.3:0/2515370903 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d01ae210 con 0x55c81cb011b0 2024-09-21T06:16:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8db887740 1 -- 172.21.15.3:0/2515370903 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d01ab450 con 0x55c81cb2a070 2024-09-21T06:16:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8db887740 10 client.0 _ll_register_callbacks cb 0x7ff8c4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8cf7fe640 1 --2- 172.21.15.3:0/2515370903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb2a070 0x55c81cee1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8ceffd640 1 -- 172.21.15.3:0/2515370903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb2a070 msgr2=0x55c81cee1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8ceffd640 1 --2- 172.21.15.3:0/2515370903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb2a070 0x55c81cee1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8ceffd640 1 -- 172.21.15.3:0/2515370903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb011b0 msgr2=0x55c81cee1840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:16:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8ceffd640 1 --2- 172.21.15.3:0/2515370903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb011b0 0x55c81cee1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.588+0000 7ff8ceffd640 1 -- 172.21.15.3:0/2515370903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c81cf5cf50 con 0x55c81cabdda0 2024-09-21T06:16:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.589+0000 7ff8ceffd640 1 --2- 172.21.15.3:0/2515370903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cabdda0 0x55c81cee1300 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55c81ced0320 tx=0x7ff8c000b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a8c8f32f9fc893 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.589+0000 7ff8b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.589+0000 7ff8b2ff5640 1 -- 172.21.15.3:0/2515370903 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0031060 con 0x55c81cabdda0 2024-09-21T06:16:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.589+0000 7ff8b2ff5640 1 -- 172.21.15.3:0/2515370903 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff8c0011440 con 0x55c81cabdda0 2024-09-21T06:16:01.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.589+0000 7ff8db887740 1 -- 172.21.15.3:0/2515370903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8c40d2600 con 0x55c81cabdda0 2024-09-21T06:16:01.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.589+0000 7ff8b2ff5640 1 -- 172.21.15.3:0/2515370903 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8c0011640 con 0x55c81cabdda0 2024-09-21T06:16:01.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.589+0000 7ff8db887740 10 client.15768 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:01.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.589+0000 7ff8db887740 1 -- 172.21.15.3:0/2515370903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c81cf5e120 con 0x55c81cabdda0 2024-09-21T06:16:01.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.590+0000 7ff8b2ff5640 1 -- 172.21.15.3:0/2515370903 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c0015070 con 0x55c81cabdda0 2024-09-21T06:16:01.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.590+0000 7ff8b2ff5640 1 -- 172.21.15.3:0/2515370903 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff8c0036900 con 0x55c81cabdda0 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.590+0000 7ff8db887740 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.590+0000 7ff8db887740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.590+0000 7ff8db887740 20 client.15768 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.590+0000 7ff8db887740 1 --2- 172.21.15.3:0/2515370903 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55c81cf5e790 0x55c81cf60bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.590+0000 7ff8db887740 4 client.15768 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.590+0000 7ff8db887740 1 -- 172.21.15.3:0/2515370903 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff8c40d2600 con 0x55c81cf5e790 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.590+0000 7ff8ce7fc640 1 --2- 172.21.15.3:0/2515370903 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55c81cf5e790 0x55c81cf60bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.591+0000 7ff8ce7fc640 1 --2- 172.21.15.3:0/2515370903 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55c81cf5e790 0x55c81cf60bb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15a7535ae206012b server_cookie=feea2e0f8547583e in_seq=0 out_seq=0 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.591+0000 7ff8b2ff5640 10 client.15768 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.591+0000 7ff8b2ff5640 1 -- 172.21.15.3:0/2515370903 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff8c40d2600 con 0x55c81cf5e790 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.591+0000 7ff8b2ff5640 10 client.15768 handle_command_reply: tid=0 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.591+0000 7ff8db887740 1 client.15768 shutdown 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.591+0000 7ff8db887740 1 -- 172.21.15.3:0/2515370903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cabdda0 msgr2=0x55c81cee1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.591+0000 7ff8db887740 1 --2- 172.21.15.3:0/2515370903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cabdda0 0x55c81cee1300 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55c81ced0320 tx=0x7ff8c000b6e0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.592+0000 7ff8db887740 1 -- 172.21.15.3:0/2515370903 shutdown_connections 2024-09-21T06:16:01.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.592+0000 7ff8db887740 1 --2- 172.21.15.3:0/2515370903 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55c81cf5e790 0x55c81cf60bb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.592+0000 7ff8b2ff5640 0 client.15768 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:01.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.592+0000 7ff8db887740 1 --2- 172.21.15.3:0/2515370903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c81cb2a070 0x55c81cee1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.592+0000 7ff8db887740 1 --2- 172.21.15.3:0/2515370903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c81cb011b0 0x55c81cee1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.592+0000 7ff8db887740 1 --2- 172.21.15.3:0/2515370903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c81cabdda0 0x55c81cee1300 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.592+0000 7ff8db887740 1 -- 172.21.15.3:0/2515370903 >> 172.21.15.3:0/2515370903 conn(0x55c81cb1e460 msgr2=0x55c81ced26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:01.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.592+0000 7ff8db887740 1 -- 172.21.15.3:0/2515370903 shutdown_connections 2024-09-21T06:16:01.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.592+0000 7ff8db887740 1 -- 172.21.15.3:0/2515370903 wait complete. 2024-09-21T06:16:01.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.592+0000 7ff8db887740 20 client.15768 trim_cache size 0 max 16384 2024-09-21T06:16:01.596 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:16:01.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.593+0000 7ff8d5313640 1 -- 172.21.15.3:0/44335868 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff8b80b6730 msgr2=0x7ff8b80b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.593+0000 7ff8d5313640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff8b80b6730 0x7ff8b80b8c40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8007e50 tx=0x7ff8c8007520 comp rx=0 tx=0).stop 2024-09-21T06:16:01.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.593+0000 7ff8d5313640 1 -- 172.21.15.3:0/44335868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8d0109500 msgr2=0x7ff8d019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.593+0000 7ff8d5313640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8d0109500 0x7ff8d019ab50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff8c402a930 tx=0x7ff8c4004270 comp rx=0 tx=0).stop 2024-09-21T06:16:01.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.593+0000 7ff8d5313640 1 -- 172.21.15.3:0/44335868 shutdown_connections 2024-09-21T06:16:01.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.593+0000 7ff8d5313640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff8b80b6730 0x7ff8b80b8c40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.593+0000 7ff8d5313640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8d0109e50 0x7ff8d019fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.593+0000 7ff8d5313640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8d0109500 0x7ff8d019ab50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.593+0000 7ff8d5313640 1 --2- 172.21.15.3:0/44335868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8d0102050 0x7ff8d019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.593+0000 7ff8d5313640 1 -- 172.21.15.3:0/44335868 >> 172.21.15.3:0/44335868 conn(0x7ff8d0079e10 msgr2=0x7ff8d01055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:01.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.593+0000 7ff8d5313640 1 -- 172.21.15.3:0/44335868 shutdown_connections 2024-09-21T06:16:01.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:01.593+0000 7ff8d5313640 1 -- 172.21.15.3:0/44335868 wait complete. 2024-09-21T06:16:03.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:02 smithi003 ceph-mon[31227]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-09-21T06:16:03.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:02 smithi003 ceph-mon[31227]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-09-21T06:16:03.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:02 smithi137 ceph-mon[34464]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-09-21T06:16:03.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:02 smithi137 ceph-mon[34464]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-09-21T06:16:03.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:02 smithi204 ceph-mon[34659]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-09-21T06:16:03.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:02 smithi204 ceph-mon[34659]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-09-21T06:16:03.417 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:16:03.417 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:16:03.829 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.401+0000 7f6694977640 1 -- 172.21.15.3:0/3974598712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6690105300 msgr2=0x7f669010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.401+0000 7f6694977640 1 --2- 172.21.15.3:0/3974598712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6690105300 0x7f669010d850 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6678009900 tx=0x7f667802a410 comp rx=0 tx=0).stop 2024-09-21T06:16:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.401+0000 7f6694977640 1 -- 172.21.15.3:0/3974598712 shutdown_connections 2024-09-21T06:16:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.401+0000 7f6694977640 1 --2- 172.21.15.3:0/3974598712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669010de10 0x7f6690110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.401+0000 7f6694977640 1 --2- 172.21.15.3:0/3974598712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6690105300 0x7f669010d850 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.401+0000 7f6694977640 1 --2- 172.21.15.3:0/3974598712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66901049b0 0x7f6690104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.401+0000 7f6694977640 1 -- 172.21.15.3:0/3974598712 >> 172.21.15.3:0/3974598712 conn(0x7f6690079730 msgr2=0x7f6690079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.401+0000 7f6694977640 1 -- 172.21.15.3:0/3974598712 shutdown_connections 2024-09-21T06:16:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.401+0000 7f6694977640 1 -- 172.21.15.3:0/3974598712 wait complete. 2024-09-21T06:16:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.402+0000 7f6694977640 1 Processor -- start 2024-09-21T06:16:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.402+0000 7f6694977640 1 -- start start 2024-09-21T06:16:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.402+0000 7f6694977640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66901049b0 0x7f669019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.402+0000 7f6694977640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6690105300 0x7f669019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.402+0000 7f6694977640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669010de10 0x7f669019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f668e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66901049b0 0x7f669019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f6694977640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6690113ac0 con 0x7f66901049b0 2024-09-21T06:16:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f6694977640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6690113940 con 0x7f6690105300 2024-09-21T06:16:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f6694977640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6690113c40 con 0x7f669010de10 2024-09-21T06:16:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f668e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66901049b0 0x7f669019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48988/0 (socket says 172.21.15.3:48988) 2024-09-21T06:16:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f668e59c640 1 -- 172.21.15.3:0/2335360174 learned_addr learned my addr 172.21.15.3:0/2335360174 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f668ed9d640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669010de10 0x7f669019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f6687fff640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6690105300 0x7f669019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f668ed9d640 1 -- 172.21.15.3:0/2335360174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6690105300 msgr2=0x7f669019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f668ed9d640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6690105300 0x7f669019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f668ed9d640 1 -- 172.21.15.3:0/2335360174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66901049b0 msgr2=0x7f669019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f668ed9d640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66901049b0 0x7f669019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f668ed9d640 1 -- 172.21.15.3:0/2335360174 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66901a04c0 con 0x7f669010de10 2024-09-21T06:16:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.403+0000 7f668e59c640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66901049b0 0x7f669019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.404+0000 7f668ed9d640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669010de10 0x7f669019fd90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6688009fd0 tx=0x7f668800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.404+0000 7f66877fe640 1 -- 172.21.15.3:0/2335360174 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668800ce20 con 0x7f669010de10 2024-09-21T06:16:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.404+0000 7f6694977640 1 -- 172.21.15.3:0/2335360174 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66901a07b0 con 0x7f669010de10 2024-09-21T06:16:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.404+0000 7f6694977640 1 -- 172.21.15.3:0/2335360174 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66901a6f10 con 0x7f669010de10 2024-09-21T06:16:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.404+0000 7f66877fe640 1 -- 172.21.15.3:0/2335360174 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6688005020 con 0x7f669010de10 2024-09-21T06:16:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.404+0000 7f66877fe640 1 -- 172.21.15.3:0/2335360174 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688010690 con 0x7f669010de10 2024-09-21T06:16:04.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.407+0000 7f669aeeb740 1 Processor -- start 2024-09-21T06:16:04.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.407+0000 7f66877fe640 1 -- 172.21.15.3:0/2335360174 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7f6688010830 con 0x7f669010de10 2024-09-21T06:16:04.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.408+0000 7f669aeeb740 1 -- start start 2024-09-21T06:16:04.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.408+0000 7f669aeeb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ecc350 0x5598a3273430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.408+0000 7f669aeeb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 0x5598a3273970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.409+0000 7f669aeeb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2eab670 0x5598a3273eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.409+0000 7f66877fe640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f667c0b6790 0x7f667c0b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.409+0000 7f668e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 0x5598a3273970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.409+0000 7f6687fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ecc350 0x5598a3273430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.409+0000 7f6687fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ecc350 0x5598a3273430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41730/0 (socket says 172.21.15.3:41730) 2024-09-21T06:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.409+0000 7f6687fff640 1 -- 172.21.15.3:0/631854074 learned_addr learned my addr 172.21.15.3:0/631854074 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.409+0000 7f669aeeb740 1 -- 172.21.15.3:0/631854074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5598a31e9e60 con 0x5598a2eab670 2024-09-21T06:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.409+0000 7f669aeeb740 1 -- 172.21.15.3:0/631854074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5598a31d9290 con 0x5598a2ea3490 2024-09-21T06:16:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.409+0000 7f669aeeb740 1 -- 172.21.15.3:0/631854074 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5598a31dd0a0 con 0x5598a2ecc350 2024-09-21T06:16:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.409+0000 7f6687fff640 1 --2- 172.21.15.3:0/631854074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2eab670 0x5598a3273eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.409+0000 7f668ed9d640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f667c0b6790 0x7f667c0b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.410+0000 7f6687fff640 1 -- 172.21.15.3:0/631854074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ecc350 msgr2=0x5598a3273430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.410+0000 7f6687fff640 1 --2- 172.21.15.3:0/631854074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ecc350 0x5598a3273430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.410+0000 7f6687fff640 1 -- 172.21.15.3:0/631854074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 msgr2=0x5598a3273970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.410+0000 7f6687fff640 1 --2- 172.21.15.3:0/631854074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 0x5598a3273970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.410+0000 7f6687fff640 1 -- 172.21.15.3:0/631854074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a2f37930 con 0x5598a2eab670 2024-09-21T06:16:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.410+0000 7f6687fff640 1 --2- 172.21.15.3:0/631854074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2eab670 0x5598a3273eb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f66780096c0 tx=0x7f6678031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf243c06cb33c047 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.410+0000 7f66877fe640 1 -- 172.21.15.3:0/2335360174 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f66880e7d30 con 0x7f669010de10 2024-09-21T06:16:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.410+0000 7f668ed9d640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f667c0b6790 0x7f667c0b8ca0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f66880e5460 tx=0x7f66880a98f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.411+0000 7f6684ff9640 1 -- 172.21.15.3:0/631854074 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678035040 con 0x5598a2eab670 2024-09-21T06:16:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.411+0000 7f6684ff9640 1 -- 172.21.15.3:0/631854074 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66780044d0 con 0x5598a2eab670 2024-09-21T06:16:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.411+0000 7f6684ff9640 1 -- 172.21.15.3:0/631854074 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678003850 con 0x5598a2eab670 2024-09-21T06:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.412+0000 7f669aeeb740 1 -- 172.21.15.3:0/631854074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2eab670 msgr2=0x5598a3273eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.412+0000 7f669aeeb740 1 --2- 172.21.15.3:0/631854074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2eab670 0x5598a3273eb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f66780096c0 tx=0x7f6678031400 comp rx=0 tx=0).stop 2024-09-21T06:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.412+0000 7f669aeeb740 1 -- 172.21.15.3:0/631854074 shutdown_connections 2024-09-21T06:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.412+0000 7f669aeeb740 1 --2- 172.21.15.3:0/631854074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2eab670 0x5598a3273eb0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.412+0000 7f669aeeb740 1 --2- 172.21.15.3:0/631854074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 0x5598a3273970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.412+0000 7f669aeeb740 1 --2- 172.21.15.3:0/631854074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ecc350 0x5598a3273430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.412+0000 7f669aeeb740 1 -- 172.21.15.3:0/631854074 >> 172.21.15.3:0/631854074 conn(0x5598a2ec0740 msgr2=0x5598a3271320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.412+0000 7f669aeeb740 1 -- 172.21.15.3:0/631854074 shutdown_connections 2024-09-21T06:16:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.412+0000 7f669aeeb740 1 -- 172.21.15.3:0/631854074 wait complete. 2024-09-21T06:16:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.412+0000 7f669aeeb740 1 -- start start 2024-09-21T06:16:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.413+0000 7f669aeeb740 1 Processor -- start 2024-09-21T06:16:04.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.413+0000 7f669aeeb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 0x5598a327f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.413+0000 7f669aeeb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2eab670 0x5598a327f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.413+0000 7f669aeeb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ecc350 0x5598a328af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.413+0000 7f669aeeb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5598a303a890 con 0x5598a2eab670 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.413+0000 7f668e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2eab670 0x5598a327f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.413+0000 7f6687fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 0x5598a327f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.413+0000 7f668e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2eab670 0x5598a327f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48996/0 (socket says 172.21.15.3:48996) 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.413+0000 7f6687fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 0x5598a327f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49522/0 (socket says 172.21.15.3:49522) 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.413+0000 7f668e59c640 1 -- 172.21.15.3:0/1355883457 learned_addr learned my addr 172.21.15.3:0/1355883457 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.413+0000 7f669aeeb740 1 -- 172.21.15.3:0/1355883457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5598a3173d40 con 0x5598a2ea3490 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.413+0000 7f669aeeb740 1 -- 172.21.15.3:0/1355883457 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5598a3281bb0 con 0x5598a2ecc350 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.413+0000 7f669aeeb740 10 client.0 _ll_register_callbacks cb 0x5598a31d7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.414+0000 7f6687fff640 1 --2- 172.21.15.3:0/1355883457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ecc350 0x5598a328af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.414+0000 7f668e59c640 1 -- 172.21.15.3:0/1355883457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ecc350 msgr2=0x5598a328af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.414+0000 7f668e59c640 1 --2- 172.21.15.3:0/1355883457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ecc350 0x5598a328af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.414+0000 7f668e59c640 1 -- 172.21.15.3:0/1355883457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 msgr2=0x5598a327f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.414+0000 7f668e59c640 1 --2- 172.21.15.3:0/1355883457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 0x5598a327f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.414+0000 7f668e59c640 1 -- 172.21.15.3:0/1355883457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a328b570 con 0x5598a2eab670 2024-09-21T06:16:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.414+0000 7f6687fff640 1 --2- 172.21.15.3:0/1355883457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ecc350 0x5598a328af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.414+0000 7f668e59c640 1 --2- 172.21.15.3:0/1355883457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2eab670 0x5598a327f590 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6680011900 tx=0x7f6680011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6c074e2afa149b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.414+0000 7f6675ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.414+0000 7f6675ffb640 1 -- 172.21.15.3:0/1355883457 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680010c60 con 0x5598a2eab670 2024-09-21T06:16:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.414+0000 7f6675ffb640 1 -- 172.21.15.3:0/1355883457 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f6680010e00 con 0x5598a2eab670 2024-09-21T06:16:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.415+0000 7f669aeeb740 1 -- 172.21.15.3:0/1355883457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66780044d0 con 0x5598a2eab670 2024-09-21T06:16:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.415+0000 7f6675ffb640 1 -- 172.21.15.3:0/1355883457 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f668001b420 con 0x5598a2eab670 2024-09-21T06:16:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.415+0000 7f669aeeb740 10 client.15780 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.415+0000 7f669aeeb740 1 -- 172.21.15.3:0/1355883457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598a328def0 con 0x5598a2eab670 2024-09-21T06:16:04.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.415+0000 7f6675ffb640 1 -- 172.21.15.3:0/1355883457 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66780044d0 con 0x5598a2eab670 2024-09-21T06:16:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.416+0000 7f6675ffb640 1 -- 172.21.15.3:0/1355883457 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f668001a800 con 0x5598a2eab670 2024-09-21T06:16:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.416+0000 7f669aeeb740 10 client.15780 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.416+0000 7f669aeeb740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.416+0000 7f669aeeb740 20 client.15780 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.416+0000 7f669aeeb740 1 --2- 172.21.15.3:0/1355883457 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5598a2e8acd0 0x5598a3291590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.416+0000 7f669aeeb740 4 client.15780 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.416+0000 7f669aeeb740 1 -- 172.21.15.3:0/1355883457 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598a31ebdb0 con 0x5598a2e8acd0 2024-09-21T06:16:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.416+0000 7f6687fff640 1 --2- 172.21.15.3:0/1355883457 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5598a2e8acd0 0x5598a3291590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.417+0000 7f6675ffb640 10 client.15780 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.417+0000 7f6687fff640 1 --2- 172.21.15.3:0/1355883457 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5598a2e8acd0 0x5598a3291590 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=358c702057f9c51f server_cookie=2d736fd2b597bc44 in_seq=0 out_seq=0 2024-09-21T06:16:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f6675ffb640 1 -- 172.21.15.3:0/1355883457 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5598a31ebdb0 con 0x5598a2e8acd0 2024-09-21T06:16:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f6675ffb640 10 client.15780 handle_command_reply: tid=0 2024-09-21T06:16:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f669aeeb740 1 client.15780 shutdown 2024-09-21T06:16:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f669aeeb740 1 -- 172.21.15.3:0/1355883457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2eab670 msgr2=0x5598a327f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f669aeeb740 1 --2- 172.21.15.3:0/1355883457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2eab670 0x5598a327f590 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6680011900 tx=0x7f6680011dc0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f669aeeb740 1 -- 172.21.15.3:0/1355883457 shutdown_connections 2024-09-21T06:16:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f669aeeb740 1 --2- 172.21.15.3:0/1355883457 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5598a2e8acd0 0x5598a3291590 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f6675ffb640 0 client.15780 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f669aeeb740 1 --2- 172.21.15.3:0/1355883457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ecc350 0x5598a328af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f669aeeb740 1 --2- 172.21.15.3:0/1355883457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2eab670 0x5598a327f590 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f669aeeb740 1 --2- 172.21.15.3:0/1355883457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 0x5598a327f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f669aeeb740 1 -- 172.21.15.3:0/1355883457 >> 172.21.15.3:0/1355883457 conn(0x5598a2ec0740 msgr2=0x5598a32765c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:04.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f669aeeb740 1 -- 172.21.15.3:0/1355883457 shutdown_connections 2024-09-21T06:16:04.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f669aeeb740 1 -- 172.21.15.3:0/1355883457 wait complete. 2024-09-21T06:16:04.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.418+0000 7f669aeeb740 20 client.15780 trim_cache size 0 max 16384 2024-09-21T06:16:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.437+0000 7f669aeeb740 1 Processor -- start 2024-09-21T06:16:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.437+0000 7f669aeeb740 1 -- start start 2024-09-21T06:16:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.437+0000 7f669aeeb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 0x5598a3278540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.437+0000 7f669aeeb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2eab670 0x5598a3278a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.437+0000 7f669aeeb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2ecc350 0x5598a3270860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.437+0000 7f669aeeb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f667000e6f0 con 0x5598a2ecc350 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.437+0000 7f669aeeb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6670010230 con 0x5598a2ea3490 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.437+0000 7f669aeeb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f667000edc0 con 0x5598a2eab670 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.437+0000 7f6687fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2ecc350 0x5598a3270860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.437+0000 7f6687fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2ecc350 0x5598a3270860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49004/0 (socket says 172.21.15.3:49004) 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.437+0000 7f6687fff640 1 -- 172.21.15.3:0/3451809110 learned_addr learned my addr 172.21.15.3:0/3451809110 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.437+0000 7f668e59c640 1 --2- 172.21.15.3:0/3451809110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2eab670 0x5598a3278a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.437+0000 7f6687fff640 1 --2- 172.21.15.3:0/3451809110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 0x5598a3278540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.438+0000 7f6687fff640 1 -- 172.21.15.3:0/3451809110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2eab670 msgr2=0x5598a3278a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.438+0000 7f6687fff640 1 --2- 172.21.15.3:0/3451809110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2eab670 0x5598a3278a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.438+0000 7f6687fff640 1 -- 172.21.15.3:0/3451809110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 msgr2=0x5598a3278540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.438+0000 7f6687fff640 1 --2- 172.21.15.3:0/3451809110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 0x5598a3278540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.438+0000 7f6687fff640 1 -- 172.21.15.3:0/3451809110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a3281490 con 0x5598a2ecc350 2024-09-21T06:16:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.438+0000 7f6687fff640 1 --2- 172.21.15.3:0/3451809110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2ecc350 0x5598a3270860 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6678000c00 tx=0x7f6678005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b6b7cd12e05450 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.438+0000 7f6677fff640 1 -- 172.21.15.3:0/3451809110 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678062070 con 0x5598a2ecc350 2024-09-21T06:16:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.438+0000 7f6677fff640 1 -- 172.21.15.3:0/3451809110 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66780048c0 con 0x5598a2ecc350 2024-09-21T06:16:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.438+0000 7f6677fff640 1 -- 172.21.15.3:0/3451809110 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678041040 con 0x5598a2ecc350 2024-09-21T06:16:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.439+0000 7f669aeeb740 1 -- 172.21.15.3:0/3451809110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2ecc350 msgr2=0x5598a3270860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.439+0000 7f669aeeb740 1 --2- 172.21.15.3:0/3451809110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2ecc350 0x5598a3270860 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6678000c00 tx=0x7f6678005eb0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.439+0000 7f669aeeb740 1 -- 172.21.15.3:0/3451809110 shutdown_connections 2024-09-21T06:16:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.439+0000 7f669aeeb740 1 --2- 172.21.15.3:0/3451809110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2eab670 0x5598a3278a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.439+0000 7f669aeeb740 1 --2- 172.21.15.3:0/3451809110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2ecc350 0x5598a3270860 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.439+0000 7f669aeeb740 1 --2- 172.21.15.3:0/3451809110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2ea3490 0x5598a3278540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.439+0000 7f669aeeb740 1 -- 172.21.15.3:0/3451809110 >> 172.21.15.3:0/3451809110 conn(0x5598a2ec0740 msgr2=0x5598a327d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.440+0000 7f669aeeb740 1 -- 172.21.15.3:0/3451809110 shutdown_connections 2024-09-21T06:16:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.440+0000 7f669aeeb740 1 -- 172.21.15.3:0/3451809110 wait complete. 2024-09-21T06:16:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.440+0000 7f669aeeb740 1 -- start start 2024-09-21T06:16:04.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.440+0000 7f669aeeb740 1 Processor -- start 2024-09-21T06:16:04.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.440+0000 7f669aeeb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ea3490 0x5598a32839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f669aeeb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2eab670 0x5598a3283f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f669aeeb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2ecc350 0x5598a32f5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f669aeeb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f667000fb60 con 0x5598a2ecc350 2024-09-21T06:16:04.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f669aeeb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f667000f490 con 0x5598a2eab670 2024-09-21T06:16:04.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f669aeeb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f667000c6d0 con 0x5598a2ea3490 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f669aeeb740 10 client.0 _ll_register_callbacks cb 0x7f668000bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f668e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2eab670 0x5598a3283f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f668e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2eab670 0x5598a3283f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49548/0 (socket says 172.21.15.3:49548) 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f668e59c640 1 -- 172.21.15.3:0/3018593962 learned_addr learned my addr 172.21.15.3:0/3018593962 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f6687fff640 1 --2- 172.21.15.3:0/3018593962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2ecc350 0x5598a32f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f668e59c640 1 -- 172.21.15.3:0/3018593962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ea3490 msgr2=0x5598a32839f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f668e59c640 1 --2- 172.21.15.3:0/3018593962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ea3490 0x5598a32839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f668e59c640 1 -- 172.21.15.3:0/3018593962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2ecc350 msgr2=0x5598a32f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f668e59c640 1 --2- 172.21.15.3:0/3018593962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2ecc350 0x5598a32f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f668e59c640 1 -- 172.21.15.3:0/3018593962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a32f5700 con 0x5598a2eab670 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.441+0000 7f6687fff640 1 --2- 172.21.15.3:0/3018593962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2ecc350 0x5598a32f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.442+0000 7f668e59c640 1 --2- 172.21.15.3:0/3018593962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2eab670 0x5598a3283f90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6680001eb0 tx=0x7f668001ae60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89052eed4284ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.442+0000 7f6675ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:16:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.442+0000 7f6675ffb640 1 -- 172.21.15.3:0/3018593962 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680032060 con 0x5598a2eab670 2024-09-21T06:16:04.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.442+0000 7f6675ffb640 1 -- 172.21.15.3:0/3018593962 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f66800052d0 con 0x5598a2eab670 2024-09-21T06:16:04.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.442+0000 7f669aeeb740 1 -- 172.21.15.3:0/3018593962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66780048c0 con 0x5598a2eab670 2024-09-21T06:16:04.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.442+0000 7f6675ffb640 1 -- 172.21.15.3:0/3018593962 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6680028070 con 0x5598a2eab670 2024-09-21T06:16:04.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.442+0000 7f669aeeb740 10 client.25394 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:04.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.443+0000 7f669aeeb740 1 -- 172.21.15.3:0/3018593962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598a32fcb60 con 0x5598a2eab670 2024-09-21T06:16:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.443+0000 7f6675ffb640 1 -- 172.21.15.3:0/3018593962 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66780048c0 con 0x5598a2eab670 2024-09-21T06:16:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.443+0000 7f6675ffb640 1 -- 172.21.15.3:0/3018593962 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6680004920 con 0x5598a2eab670 2024-09-21T06:16:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.443+0000 7f669aeeb740 10 client.25394 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.443+0000 7f669aeeb740 10 client.25394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.443+0000 7f669aeeb740 20 client.25394 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.443+0000 7f669aeeb740 1 --2- 172.21.15.3:0/3018593962 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5598a2e60080 0x5598a32ff370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.443+0000 7f669aeeb740 4 client.25394 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:16:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.443+0000 7f669aeeb740 1 -- 172.21.15.3:0/3018593962 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5598a31e7860 con 0x5598a2e60080 2024-09-21T06:16:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.443+0000 7f6687fff640 1 --2- 172.21.15.3:0/3018593962 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5598a2e60080 0x5598a32ff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.444+0000 7f6675ffb640 10 client.25394 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:04.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.444+0000 7f6687fff640 1 --2- 172.21.15.3:0/3018593962 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5598a2e60080 0x5598a32ff370 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f37838dce6f30d server_cookie=f8843088fb0ff3a7 in_seq=0 out_seq=0 2024-09-21T06:16:04.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.446+0000 7f6675ffb640 1 -- 172.21.15.3:0/3018593962 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5598a31e7860 con 0x5598a2e60080 2024-09-21T06:16:04.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.446+0000 7f6675ffb640 10 client.25394 handle_command_reply: tid=0 2024-09-21T06:16:04.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.446+0000 7f669aeeb740 1 client.25394 shutdown 2024-09-21T06:16:04.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.446+0000 7f669aeeb740 1 -- 172.21.15.3:0/3018593962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2eab670 msgr2=0x5598a3283f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.446+0000 7f669aeeb740 1 --2- 172.21.15.3:0/3018593962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2eab670 0x5598a3283f90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6680001eb0 tx=0x7f668001ae60 comp rx=0 tx=0).stop 2024-09-21T06:16:04.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.446+0000 7f669aeeb740 1 -- 172.21.15.3:0/3018593962 shutdown_connections 2024-09-21T06:16:04.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.446+0000 7f669aeeb740 1 --2- 172.21.15.3:0/3018593962 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5598a2e60080 0x5598a32ff370 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.446+0000 7f669aeeb740 1 --2- 172.21.15.3:0/3018593962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5598a2ecc350 0x5598a32f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.446+0000 7f6675ffb640 0 client.25394 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:04.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.446+0000 7f669aeeb740 1 --2- 172.21.15.3:0/3018593962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598a2eab670 0x5598a3283f90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.446+0000 7f669aeeb740 1 --2- 172.21.15.3:0/3018593962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598a2ea3490 0x5598a32839f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.446+0000 7f669aeeb740 1 -- 172.21.15.3:0/3018593962 >> 172.21.15.3:0/3018593962 conn(0x5598a2ec0740 msgr2=0x5598a327d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:04.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.446+0000 7f669aeeb740 1 -- 172.21.15.3:0/3018593962 shutdown_connections 2024-09-21T06:16:04.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.447+0000 7f669aeeb740 1 -- 172.21.15.3:0/3018593962 wait complete. 2024-09-21T06:16:04.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.447+0000 7f669aeeb740 20 client.25394 trim_cache size 0 max 16384 2024-09-21T06:16:04.450 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:16:04.450 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:16:04.450 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "c367f31f-a684-41c6-88ee-4b4e42f0d4e2", 2024-09-21T06:16:04.450 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:16:04.450 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:16:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.447+0000 7f6694977640 1 -- 172.21.15.3:0/2335360174 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f667c0b6790 msgr2=0x7f667c0b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.447+0000 7f6694977640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f667c0b6790 0x7f667c0b8ca0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f66880e5460 tx=0x7f66880a98f0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.447+0000 7f6694977640 1 -- 172.21.15.3:0/2335360174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669010de10 msgr2=0x7f669019fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.447+0000 7f6694977640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669010de10 0x7f669019fd90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6688009fd0 tx=0x7f668800ef10 comp rx=0 tx=0).stop 2024-09-21T06:16:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.448+0000 7f6694977640 1 -- 172.21.15.3:0/2335360174 shutdown_connections 2024-09-21T06:16:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.448+0000 7f6694977640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f667c0b6790 0x7f667c0b8ca0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.448+0000 7f6694977640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f669010de10 0x7f669019fd90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.448+0000 7f6694977640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6690105300 0x7f669019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.448+0000 7f6694977640 1 --2- 172.21.15.3:0/2335360174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66901049b0 0x7f669019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.448+0000 7f6694977640 1 -- 172.21.15.3:0/2335360174 >> 172.21.15.3:0/2335360174 conn(0x7f6690079730 msgr2=0x7f6690106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.448+0000 7f6694977640 1 -- 172.21.15.3:0/2335360174 shutdown_connections 2024-09-21T06:16:04.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:04.448+0000 7f6694977640 1 -- 172.21.15.3:0/2335360174 wait complete. 2024-09-21T06:16:05.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:04 smithi003 ceph-mon[31227]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-09-21T06:16:05.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:04 smithi137 ceph-mon[34464]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-09-21T06:16:05.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:04 smithi204 ceph-mon[34659]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-09-21T06:16:05.253 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:16:05.651 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:06.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:05 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:16:06.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:05 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:06.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:05 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:16:06.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:05 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:06.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:05 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:16:06.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:05 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:06.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:05 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:16:06.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:05 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:06.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:05 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:16:06.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:05 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:06.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:05 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:16:06.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:05 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:06.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.232+0000 7fd9cd24a640 1 -- 172.21.15.3:0/1690472490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c8102a60 msgr2=0x7fd9c8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.232+0000 7fd9cd24a640 1 --2- 172.21.15.3:0/1690472490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c8102a60 0x7fd9c8102e70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009900 tx=0x7fd9b802a410 comp rx=0 tx=0).stop 2024-09-21T06:16:06.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.232+0000 7fd9cd24a640 1 -- 172.21.15.3:0/1690472490 shutdown_connections 2024-09-21T06:16:06.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.232+0000 7fd9cd24a640 1 --2- 172.21.15.3:0/1690472490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c8108cf0 0x7fd9c810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.232+0000 7fd9cd24a640 1 --2- 172.21.15.3:0/1690472490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9c81083a0 0x7fd9c81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.232+0000 7fd9cd24a640 1 --2- 172.21.15.3:0/1690472490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c8102a60 0x7fd9c8102e70 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.232+0000 7fd9cd24a640 1 -- 172.21.15.3:0/1690472490 >> 172.21.15.3:0/1690472490 conn(0x7fd9c8079730 msgr2=0x7fd9c8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:06.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.232+0000 7fd9cd24a640 1 -- 172.21.15.3:0/1690472490 shutdown_connections 2024-09-21T06:16:06.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.232+0000 7fd9cd24a640 1 -- 172.21.15.3:0/1690472490 wait complete. 2024-09-21T06:16:06.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.233+0000 7fd9cd24a640 1 Processor -- start 2024-09-21T06:16:06.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.234+0000 7fd9cd24a640 1 -- start start 2024-09-21T06:16:06.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.234+0000 7fd9cd24a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c81083a0 0x7fd9c8113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.234+0000 7fd9cd24a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9c8108cf0 0x7fd9c8113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.234+0000 7fd9cd24a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c8113cc0 0x7fd9c81acec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.234+0000 7fd9cd24a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8112960 con 0x7fd9c8113cc0 2024-09-21T06:16:06.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.234+0000 7fd9cd24a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81127e0 con 0x7fd9c8108cf0 2024-09-21T06:16:06.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.234+0000 7fd9cd24a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8112ae0 con 0x7fd9c81083a0 2024-09-21T06:16:06.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.234+0000 7fd9c759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c8113cc0 0x7fd9c81acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.234+0000 7fd9c659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9c8108cf0 0x7fd9c8113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.234+0000 7fd9c759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c8113cc0 0x7fd9c81acec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41664/0 (socket says 172.21.15.3:41664) 2024-09-21T06:16:06.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.234+0000 7fd9c659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9c8108cf0 0x7fd9c8113780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:44076/0 (socket says 172.21.15.3:44076) 2024-09-21T06:16:06.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.234+0000 7fd9c6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c81083a0 0x7fd9c8113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.234+0000 7fd9c759e640 1 -- 172.21.15.3:0/195203015 learned_addr learned my addr 172.21.15.3:0/195203015 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:06.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.235+0000 7fd9c659c640 1 -- 172.21.15.3:0/195203015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c81083a0 msgr2=0x7fd9c8113240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.235+0000 7fd9c659c640 1 --2- 172.21.15.3:0/195203015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c81083a0 0x7fd9c8113240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.235+0000 7fd9c659c640 1 -- 172.21.15.3:0/195203015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c8113cc0 msgr2=0x7fd9c81acec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.235+0000 7fd9c659c640 1 --2- 172.21.15.3:0/195203015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c8113cc0 0x7fd9c81acec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.235+0000 7fd9c659c640 1 -- 172.21.15.3:0/195203015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c81a2360 con 0x7fd9c8108cf0 2024-09-21T06:16:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.235+0000 7fd9c6d9d640 1 --2- 172.21.15.3:0/195203015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c81083a0 0x7fd9c8113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.235+0000 7fd9c659c640 1 --2- 172.21.15.3:0/195203015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9c8108cf0 0x7fd9c8113780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd9c806aaa0 tx=0x7fd9bc00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.235+0000 7fd9affff640 1 -- 172.21.15.3:0/195203015 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc009c30 con 0x7fd9c8108cf0 2024-09-21T06:16:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.235+0000 7fd9cd24a640 1 -- 172.21.15.3:0/195203015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c81ad400 con 0x7fd9c8108cf0 2024-09-21T06:16:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.235+0000 7fd9affff640 1 -- 172.21.15.3:0/195203015 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9bc009dd0 con 0x7fd9c8108cf0 2024-09-21T06:16:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.235+0000 7fd9affff640 1 -- 172.21.15.3:0/195203015 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc003e50 con 0x7fd9c8108cf0 2024-09-21T06:16:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.235+0000 7fd9cd24a640 1 -- 172.21.15.3:0/195203015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c81ad910 con 0x7fd9c8108cf0 2024-09-21T06:16:06.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.239+0000 7fd9affff640 1 -- 172.21.15.3:0/195203015 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7fd9bc004000 con 0x7fd9c8108cf0 2024-09-21T06:16:06.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.239+0000 7fd9d37be740 1 Processor -- start 2024-09-21T06:16:06.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.239+0000 7fd9d37be740 1 -- start start 2024-09-21T06:16:06.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.239+0000 7fd9d37be740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3bc070 0x56145d763270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.239+0000 7fd9d37be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 0x56145d7637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.239+0000 7fd9d37be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d39b390 0x56145d763cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.239+0000 7fd9d37be740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56145d6d9e50 con 0x56145d3bc070 2024-09-21T06:16:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.239+0000 7fd9d37be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56145d6c9280 con 0x56145d3931b0 2024-09-21T06:16:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9d37be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56145d6cd090 con 0x56145d39b390 2024-09-21T06:16:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9affff640 1 --2- 172.21.15.3:0/195203015 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd9b00b6930 0x7fd9b00b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9c759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3bc070 0x56145d763270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9c759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3bc070 0x56145d763270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41668/0 (socket says 172.21.15.3:41668) 2024-09-21T06:16:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9c759e640 1 -- 172.21.15.3:0/326066055 learned_addr learned my addr 172.21.15.3:0/326066055 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9c6d9d640 1 --2- 172.21.15.3:0/326066055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 0x56145d7637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9c659c640 1 --2- 172.21.15.3:0/326066055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d39b390 0x56145d763cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9c759e640 1 --2- 172.21.15.3:0/195203015 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd9b00b6930 0x7fd9b00b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9c759e640 1 -- 172.21.15.3:0/326066055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d39b390 msgr2=0x56145d763cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9c759e640 1 --2- 172.21.15.3:0/326066055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d39b390 0x56145d763cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9c759e640 1 -- 172.21.15.3:0/326066055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 msgr2=0x56145d7637b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9c759e640 1 --2- 172.21.15.3:0/326066055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 0x56145d7637b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9c759e640 1 -- 172.21.15.3:0/326066055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145d427650 con 0x56145d3bc070 2024-09-21T06:16:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9affff640 1 -- 172.21.15.3:0/195203015 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fd9bc0e7780 con 0x7fd9c8108cf0 2024-09-21T06:16:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.240+0000 7fd9c759e640 1 --2- 172.21.15.3:0/326066055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3bc070 0x56145d763270 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0005f40 tx=0x7fd9c0013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284fcf2d7e16d84d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.241+0000 7fd9ad7fa640 1 -- 172.21.15.3:0/326066055 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0022020 con 0x56145d3bc070 2024-09-21T06:16:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.241+0000 7fd9ad7fa640 1 -- 172.21.15.3:0/326066055 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9c000fd60 con 0x56145d3bc070 2024-09-21T06:16:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.241+0000 7fd9c759e640 1 --2- 172.21.15.3:0/195203015 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd9b00b6930 0x7fd9b00b8e40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0023000 tx=0x7fd9c0021000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.241+0000 7fd9ad7fa640 1 -- 172.21.15.3:0/326066055 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0010d40 con 0x56145d3bc070 2024-09-21T06:16:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.242+0000 7fd9d37be740 1 -- 172.21.15.3:0/326066055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3bc070 msgr2=0x56145d763270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.242+0000 7fd9d37be740 1 --2- 172.21.15.3:0/326066055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3bc070 0x56145d763270 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0005f40 tx=0x7fd9c0013f50 comp rx=0 tx=0).stop 2024-09-21T06:16:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.242+0000 7fd9d37be740 1 -- 172.21.15.3:0/326066055 shutdown_connections 2024-09-21T06:16:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.242+0000 7fd9d37be740 1 --2- 172.21.15.3:0/326066055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d39b390 0x56145d763cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.242+0000 7fd9d37be740 1 --2- 172.21.15.3:0/326066055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 0x56145d7637b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.242+0000 7fd9d37be740 1 --2- 172.21.15.3:0/326066055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3bc070 0x56145d763270 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.242+0000 7fd9d37be740 1 -- 172.21.15.3:0/326066055 >> 172.21.15.3:0/326066055 conn(0x56145d3b0460 msgr2=0x56145d761160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.242+0000 7fd9d37be740 1 -- 172.21.15.3:0/326066055 shutdown_connections 2024-09-21T06:16:06.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.242+0000 7fd9d37be740 1 -- 172.21.15.3:0/326066055 wait complete. 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.243+0000 7fd9d37be740 1 -- start start 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.243+0000 7fd9d37be740 1 Processor -- start 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.243+0000 7fd9d37be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 0x56145d76e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.243+0000 7fd9d37be740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d39b390 0x56145d76eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.243+0000 7fd9d37be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d3bc070 0x56145d77ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.243+0000 7fd9c659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d39b390 0x56145d76eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.243+0000 7fd9d37be740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56145d52a700 con 0x56145d39b390 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.243+0000 7fd9d37be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56145d664110 con 0x56145d3931b0 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.243+0000 7fd9d37be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56145d771a50 con 0x56145d3bc070 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.243+0000 7fd9c659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d39b390 0x56145d76eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41672/0 (socket says 172.21.15.3:41672) 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.243+0000 7fd9c659c640 1 -- 172.21.15.3:0/1647360083 learned_addr learned my addr 172.21.15.3:0/1647360083 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.244+0000 7fd9d37be740 10 client.0 _ll_register_callbacks cb 0x56145d67d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.244+0000 7fd9c659c640 1 -- 172.21.15.3:0/1647360083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d3bc070 msgr2=0x56145d77ad40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.244+0000 7fd9c659c640 1 --2- 172.21.15.3:0/1647360083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d3bc070 0x56145d77ad40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.244+0000 7fd9c659c640 1 -- 172.21.15.3:0/1647360083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 msgr2=0x56145d76e9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.244+0000 7fd9c659c640 1 --2- 172.21.15.3:0/1647360083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 0x56145d76e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.244+0000 7fd9c659c640 1 -- 172.21.15.3:0/1647360083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145d691c40 con 0x56145d39b390 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.244+0000 7fd9c659c640 1 --2- 172.21.15.3:0/1647360083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d39b390 0x56145d76eef0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0e4f30 tx=0x7fd9bc0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c8ac03b20922b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.244+0000 7fd9aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.244+0000 7fd9aaff5640 1 -- 172.21.15.3:0/1647360083 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc020050 con 0x56145d39b390 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.244+0000 7fd9aaff5640 1 -- 172.21.15.3:0/1647360083 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fd9bc0ad7b0 con 0x56145d39b390 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.244+0000 7fd9d37be740 1 -- 172.21.15.3:0/1647360083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9c000fd60 con 0x56145d39b390 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.245+0000 7fd9aaff5640 1 -- 172.21.15.3:0/1647360083 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9bc0e5300 con 0x56145d39b390 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.245+0000 7fd9d37be740 10 client.15798 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.245+0000 7fd9d37be740 1 -- 172.21.15.3:0/1647360083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56145d77d820 con 0x56145d39b390 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.245+0000 7fd9aaff5640 1 -- 172.21.15.3:0/1647360083 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9c000fd60 con 0x56145d39b390 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.246+0000 7fd9aaff5640 1 -- 172.21.15.3:0/1647360083 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd9bc0b6a30 con 0x56145d39b390 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.246+0000 7fd9d37be740 10 client.15798 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.246+0000 7fd9d37be740 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.246+0000 7fd9d37be740 20 client.15798 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.246+0000 7fd9d37be740 1 --2- 172.21.15.3:0/1647360083 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56145d37a9f0 0x56145d780be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.246+0000 7fd9d37be740 4 client.15798 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.246+0000 7fd9d37be740 1 -- 172.21.15.3:0/1647360083 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56145d6dbda0 con 0x56145d37a9f0 2024-09-21T06:16:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.246+0000 7fd9c6d9d640 1 --2- 172.21.15.3:0/1647360083 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56145d37a9f0 0x56145d780be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.247+0000 7fd9c6d9d640 1 --2- 172.21.15.3:0/1647360083 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56145d37a9f0 0x56145d780be0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2c4e6c66f07dfa server_cookie=1a5227ca261c7cb0 in_seq=0 out_seq=0 2024-09-21T06:16:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.247+0000 7fd9aaff5640 10 client.15798 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9aaff5640 1 -- 172.21.15.3:0/1647360083 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56145d6dbda0 con 0x56145d37a9f0 2024-09-21T06:16:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9aaff5640 10 client.15798 handle_command_reply: tid=0 2024-09-21T06:16:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9d37be740 1 client.15798 shutdown 2024-09-21T06:16:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9d37be740 1 -- 172.21.15.3:0/1647360083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d39b390 msgr2=0x56145d76eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9d37be740 1 --2- 172.21.15.3:0/1647360083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d39b390 0x56145d76eef0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0e4f30 tx=0x7fd9bc0035f0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9d37be740 1 -- 172.21.15.3:0/1647360083 shutdown_connections 2024-09-21T06:16:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9d37be740 1 --2- 172.21.15.3:0/1647360083 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56145d37a9f0 0x56145d780be0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9aaff5640 0 client.15798 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9d37be740 1 --2- 172.21.15.3:0/1647360083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d3bc070 0x56145d77ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9d37be740 1 --2- 172.21.15.3:0/1647360083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d39b390 0x56145d76eef0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9d37be740 1 --2- 172.21.15.3:0/1647360083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 0x56145d76e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9d37be740 1 -- 172.21.15.3:0/1647360083 >> 172.21.15.3:0/1647360083 conn(0x56145d3b0460 msgr2=0x56145d766450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9d37be740 1 -- 172.21.15.3:0/1647360083 shutdown_connections 2024-09-21T06:16:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.248+0000 7fd9d37be740 1 -- 172.21.15.3:0/1647360083 wait complete. 2024-09-21T06:16:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.249+0000 7fd9d37be740 20 client.15798 trim_cache size 0 max 16384 2024-09-21T06:16:06.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.284+0000 7fd9d37be740 1 Processor -- start 2024-09-21T06:16:06.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.284+0000 7fd9d37be740 1 -- start start 2024-09-21T06:16:06.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.284+0000 7fd9d37be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d37a9f0 0x56145d766a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.284+0000 7fd9d37be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 0x56145d77ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.284+0000 7fd9d37be740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3bc070 0x56145d76bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.284+0000 7fd9c6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d37a9f0 0x56145d766a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.284+0000 7fd9c6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d37a9f0 0x56145d766a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:48112/0 (socket says 172.21.15.3:48112) 2024-09-21T06:16:06.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.284+0000 7fd9c6d9d640 1 -- 172.21.15.3:0/4288985322 learned_addr learned my addr 172.21.15.3:0/4288985322 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:06.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.284+0000 7fd9d37be740 1 -- 172.21.15.3:0/4288985322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81a25b0 con 0x56145d3bc070 2024-09-21T06:16:06.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.284+0000 7fd9c759e640 1 --2- 172.21.15.3:0/4288985322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3bc070 0x56145d76bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.284+0000 7fd9d37be740 1 -- 172.21.15.3:0/4288985322 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81a40f0 con 0x56145d3931b0 2024-09-21T06:16:06.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.285+0000 7fd9d37be740 1 -- 172.21.15.3:0/4288985322 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81a2c80 con 0x56145d37a9f0 2024-09-21T06:16:06.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.285+0000 7fd9c659c640 1 --2- 172.21.15.3:0/4288985322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 0x56145d77ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.285+0000 7fd9c759e640 1 -- 172.21.15.3:0/4288985322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d37a9f0 msgr2=0x56145d766a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:06.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.285+0000 7fd9c759e640 1 --2- 172.21.15.3:0/4288985322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d37a9f0 0x56145d766a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.285+0000 7fd9c759e640 1 -- 172.21.15.3:0/4288985322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 msgr2=0x56145d77ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:06.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.285+0000 7fd9c759e640 1 --2- 172.21.15.3:0/4288985322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 0x56145d77ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.285+0000 7fd9c759e640 1 -- 172.21.15.3:0/4288985322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145d76d910 con 0x56145d3bc070 2024-09-21T06:16:06.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.285+0000 7fd9c759e640 1 --2- 172.21.15.3:0/4288985322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3bc070 0x56145d76bba0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000f7f0 tx=0x7fd9c000f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb213459f58e2f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:06.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.286+0000 7fd9acff9640 1 -- 172.21.15.3:0/4288985322 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0034070 con 0x56145d3bc070 2024-09-21T06:16:06.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.286+0000 7fd9acff9640 1 -- 172.21.15.3:0/4288985322 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9c002b040 con 0x56145d3bc070 2024-09-21T06:16:06.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.286+0000 7fd9acff9640 1 -- 172.21.15.3:0/4288985322 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0022070 con 0x56145d3bc070 2024-09-21T06:16:06.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.287+0000 7fd9d37be740 1 -- 172.21.15.3:0/4288985322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3bc070 msgr2=0x56145d76bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.287+0000 7fd9d37be740 1 --2- 172.21.15.3:0/4288985322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3bc070 0x56145d76bba0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000f7f0 tx=0x7fd9c000f8a0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.287+0000 7fd9d37be740 1 -- 172.21.15.3:0/4288985322 shutdown_connections 2024-09-21T06:16:06.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.287+0000 7fd9d37be740 1 --2- 172.21.15.3:0/4288985322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3bc070 0x56145d76bba0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.287+0000 7fd9d37be740 1 --2- 172.21.15.3:0/4288985322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d3931b0 0x56145d77ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.287+0000 7fd9d37be740 1 --2- 172.21.15.3:0/4288985322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d37a9f0 0x56145d766a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.287+0000 7fd9d37be740 1 -- 172.21.15.3:0/4288985322 >> 172.21.15.3:0/4288985322 conn(0x56145d3b0460 msgr2=0x56145d766370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:06.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.287+0000 7fd9d37be740 1 -- 172.21.15.3:0/4288985322 shutdown_connections 2024-09-21T06:16:06.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.287+0000 7fd9d37be740 1 -- 172.21.15.3:0/4288985322 wait complete. 2024-09-21T06:16:06.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.288+0000 7fd9d37be740 1 -- start start 2024-09-21T06:16:06.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.288+0000 7fd9d37be740 1 Processor -- start 2024-09-21T06:16:06.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.289+0000 7fd9d37be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d37a9f0 0x56145d773c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.289+0000 7fd9d37be740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3931b0 0x56145d7741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.289+0000 7fd9c659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3931b0 0x56145d7741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.289+0000 7fd9c659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3931b0 0x56145d7741c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41690/0 (socket says 172.21.15.3:41690) 2024-09-21T06:16:06.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.289+0000 7fd9c659c640 1 -- 172.21.15.3:0/1783143899 learned_addr learned my addr 172.21.15.3:0/1783143899 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:06.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.289+0000 7fd9d37be740 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d3bc070 0x56145d774700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.289+0000 7fd9c6d9d640 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d37a9f0 0x56145d773c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.289+0000 7fd9d37be740 1 -- 172.21.15.3:0/1783143899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81a3a20 con 0x56145d3931b0 2024-09-21T06:16:06.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.289+0000 7fd9d37be740 1 -- 172.21.15.3:0/1783143899 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81a3350 con 0x56145d37a9f0 2024-09-21T06:16:06.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.289+0000 7fd9d37be740 1 -- 172.21.15.3:0/1783143899 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8114540 con 0x56145d3bc070 2024-09-21T06:16:06.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.290+0000 7fd9c759e640 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d3bc070 0x56145d774700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.290+0000 7fd9c659c640 1 -- 172.21.15.3:0/1783143899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d3bc070 msgr2=0x56145d774700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:06.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.290+0000 7fd9c659c640 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d3bc070 0x56145d774700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.290+0000 7fd9c659c640 1 -- 172.21.15.3:0/1783143899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d37a9f0 msgr2=0x56145d773c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:06.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.290+0000 7fd9c659c640 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d37a9f0 0x56145d773c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.290+0000 7fd9c659c640 1 -- 172.21.15.3:0/1783143899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145d80cf30 con 0x56145d3931b0 2024-09-21T06:16:06.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.290+0000 7fd9c659c640 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3931b0 0x56145d7741c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0a9b20 tx=0x7fd9bc0a9220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64076aa2b5f5b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:06.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.290+0000 7fd9aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:06.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.290+0000 7fd9d37be740 10 client.0 _ll_register_callbacks cb 0x7fd9bc003770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:06.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.290+0000 7fd9aaff5640 1 -- 172.21.15.3:0/1783143899 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc020070 con 0x56145d3931b0 2024-09-21T06:16:06.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.290+0000 7fd9d37be740 1 -- 172.21.15.3:0/1783143899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9c002b040 con 0x56145d3931b0 2024-09-21T06:16:06.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.291+0000 7fd9aaff5640 1 -- 172.21.15.3:0/1783143899 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fd9bc0ac9c0 con 0x56145d3931b0 2024-09-21T06:16:06.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.291+0000 7fd9aaff5640 1 -- 172.21.15.3:0/1783143899 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9bc0ad320 con 0x56145d3931b0 2024-09-21T06:16:06.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.291+0000 7fd9d37be740 10 client.15804 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:06.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.291+0000 7fd9d37be740 1 -- 172.21.15.3:0/1783143899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56145d80df50 con 0x56145d3931b0 2024-09-21T06:16:06.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.291+0000 7fd9aaff5640 1 -- 172.21.15.3:0/1783143899 <== mon.0 v2:172.21.15.3:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd9bc0ac220 con 0x56145d3931b0 2024-09-21T06:16:06.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.291+0000 7fd9d37be740 10 client.15804 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.291+0000 7fd9d37be740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.291+0000 7fd9d37be740 20 client.15804 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.291+0000 7fd9d37be740 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56145d34fda0 0x56145d8103d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.291+0000 7fd9d37be740 4 client.15804 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.291+0000 7fd9d37be740 1 -- 172.21.15.3:0/1783143899 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fd9c002b040 con 0x56145d34fda0 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.292+0000 7fd9c6d9d640 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56145d34fda0 0x56145d8103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.292+0000 7fd9aaff5640 10 client.15804 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.292+0000 7fd9c6d9d640 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56145d34fda0 0x56145d8103d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a666178c57addc6 server_cookie=a86081e60e60f863 in_seq=0 out_seq=0 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.292+0000 7fd9aaff5640 1 -- 172.21.15.3:0/1783143899 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x7fd9c002b040 con 0x56145d34fda0 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.292+0000 7fd9aaff5640 10 client.15804 handle_command_reply: tid=0 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.292+0000 7fd9d37be740 1 client.15804 shutdown 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.293+0000 7fd9d37be740 1 -- 172.21.15.3:0/1783143899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3931b0 msgr2=0x56145d7741c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.293+0000 7fd9d37be740 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3931b0 0x56145d7741c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0a9b20 tx=0x7fd9bc0a9220 comp rx=0 tx=0).stop 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.293+0000 7fd9d37be740 1 -- 172.21.15.3:0/1783143899 shutdown_connections 2024-09-21T06:16:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.293+0000 7fd9d37be740 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56145d34fda0 0x56145d8103d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.293+0000 7fd9d37be740 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56145d3bc070 0x56145d774700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.293+0000 7fd9aaff5640 0 client.15804 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.293+0000 7fd9d37be740 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56145d3931b0 0x56145d7741c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.293+0000 7fd9d37be740 1 --2- 172.21.15.3:0/1783143899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56145d37a9f0 0x56145d773c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.293+0000 7fd9d37be740 1 -- 172.21.15.3:0/1783143899 >> 172.21.15.3:0/1783143899 conn(0x56145d3b0460 msgr2=0x56145d7642a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.293+0000 7fd9d37be740 1 -- 172.21.15.3:0/1783143899 shutdown_connections 2024-09-21T06:16:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.293+0000 7fd9d37be740 1 -- 172.21.15.3:0/1783143899 wait complete. 2024-09-21T06:16:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.293+0000 7fd9d37be740 20 client.15804 trim_cache size 0 max 16384 2024-09-21T06:16:06.297 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:16:06.298 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:16:06.298 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:16:06.298 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:16:06.298 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:16:06.298 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:16:06.298 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:16:06.298 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:16:06.298 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:16:06.298 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:16:06.298 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:16:06.298 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:16:06.298 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:16:06.298 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:16:06.299 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:16:06.299 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:16:06.299 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:16:06.299 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:16:06.299 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:16:06.299 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:16:06.299 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:16:06.299 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:16:06.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.294+0000 7fd9cd24a640 1 -- 172.21.15.3:0/195203015 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd9b00b6930 msgr2=0x7fd9b00b8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.294+0000 7fd9cd24a640 1 --2- 172.21.15.3:0/195203015 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd9b00b6930 0x7fd9b00b8e40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0023000 tx=0x7fd9c0021000 comp rx=0 tx=0).stop 2024-09-21T06:16:06.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.294+0000 7fd9cd24a640 1 -- 172.21.15.3:0/195203015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9c8108cf0 msgr2=0x7fd9c8113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.294+0000 7fd9cd24a640 1 --2- 172.21.15.3:0/195203015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9c8108cf0 0x7fd9c8113780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd9c806aaa0 tx=0x7fd9bc00dde0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.294+0000 7fd9cd24a640 1 -- 172.21.15.3:0/195203015 shutdown_connections 2024-09-21T06:16:06.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.294+0000 7fd9cd24a640 1 --2- 172.21.15.3:0/195203015 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd9b00b6930 0x7fd9b00b8e40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.294+0000 7fd9cd24a640 1 --2- 172.21.15.3:0/195203015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c8113cc0 0x7fd9c81acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.294+0000 7fd9cd24a640 1 --2- 172.21.15.3:0/195203015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9c8108cf0 0x7fd9c8113780 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.294+0000 7fd9cd24a640 1 --2- 172.21.15.3:0/195203015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c81083a0 0x7fd9c8113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.294+0000 7fd9cd24a640 1 -- 172.21.15.3:0/195203015 >> 172.21.15.3:0/195203015 conn(0x7fd9c8079730 msgr2=0x7fd9c81006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:06.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.294+0000 7fd9cd24a640 1 -- 172.21.15.3:0/195203015 shutdown_connections 2024-09-21T06:16:06.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:06.294+0000 7fd9cd24a640 1 -- 172.21.15.3:0/195203015 wait complete. 2024-09-21T06:16:06.997 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:06 smithi003 ceph-mon[31227]: pgmap v358: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-09-21T06:16:06.997 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:06 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:16:07.028 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:16:07.028 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:16:07.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:06 smithi137 ceph-mon[34464]: pgmap v358: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-09-21T06:16:07.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:06 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:16:07.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:06 smithi204 ceph-mon[34659]: pgmap v358: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-09-21T06:16:07.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:06 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:16:07.308 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.942+0000 7f28b7c4c640 1 -- 172.21.15.3:0/1693706070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b0102630 msgr2=0x7f28b0102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.942+0000 7f28b7c4c640 1 --2- 172.21.15.3:0/1693706070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b0102630 0x7f28b0102a40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009900 tx=0x7f28ac02a480 comp rx=0 tx=0).stop 2024-09-21T06:16:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.943+0000 7f28b7c4c640 1 -- 172.21.15.3:0/1693706070 shutdown_connections 2024-09-21T06:16:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.943+0000 7f28b7c4c640 1 --2- 172.21.15.3:0/1693706070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28b01101e0 0x7f28b0112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.943+0000 7f28b7c4c640 1 --2- 172.21.15.3:0/1693706070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28b0102f80 0x7f28b010fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.943+0000 7f28b7c4c640 1 --2- 172.21.15.3:0/1693706070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b0102630 0x7f28b0102a40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.943+0000 7f28b7c4c640 1 -- 172.21.15.3:0/1693706070 >> 172.21.15.3:0/1693706070 conn(0x7f28b0079730 msgr2=0x7f28b0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.943+0000 7f28b7c4c640 1 -- 172.21.15.3:0/1693706070 shutdown_connections 2024-09-21T06:16:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.943+0000 7f28b7c4c640 1 -- 172.21.15.3:0/1693706070 wait complete. 2024-09-21T06:16:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.944+0000 7f28b7c4c640 1 Processor -- start 2024-09-21T06:16:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.944+0000 7f28b7c4c640 1 -- start start 2024-09-21T06:16:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.944+0000 7f28b7c4c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28b0102630 0x7f28b019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.944+0000 7f28b7c4c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b0102f80 0x7f28b019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:07.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.944+0000 7f28b7c4c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28b01101e0 0x7f28b01a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:07.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b7c4c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f28b0115c50 con 0x7f28b0102f80 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b7c4c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f28b0115ad0 con 0x7f28b01101e0 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b7c4c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f28b0115dd0 con 0x7f28b0102630 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b51e7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b0102f80 0x7f28b019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b51e7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b0102f80 0x7f28b019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41710/0 (socket says 172.21.15.3:41710) 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b51e7640 1 -- 172.21.15.3:0/2560662870 learned_addr learned my addr 172.21.15.3:0/2560662870 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b59e8640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28b0102630 0x7f28b019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b61e9640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28b01101e0 0x7f28b01a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b51e7640 1 -- 172.21.15.3:0/2560662870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28b0102630 msgr2=0x7f28b019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b51e7640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28b0102630 0x7f28b019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b51e7640 1 -- 172.21.15.3:0/2560662870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28b01101e0 msgr2=0x7f28b01a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b51e7640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28b01101e0 0x7f28b01a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b51e7640 1 -- 172.21.15.3:0/2560662870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b01a2590 con 0x7f28b0102f80 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.945+0000 7f28b51e7640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b0102f80 0x7f28b019cbc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f28a000e9f0 tx=0x7f28a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.946+0000 7f28a6ffd640 1 -- 172.21.15.3:0/2560662870 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a000cd80 con 0x7f28b0102f80 2024-09-21T06:16:07.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.946+0000 7f28a6ffd640 1 -- 172.21.15.3:0/2560662870 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28a00108e0 con 0x7f28b0102f80 2024-09-21T06:16:07.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.946+0000 7f28b7c4c640 1 -- 172.21.15.3:0/2560662870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b01a2880 con 0x7f28b0102f80 2024-09-21T06:16:07.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.947+0000 7f28a6ffd640 1 -- 172.21.15.3:0/2560662870 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a0018980 con 0x7f28b0102f80 2024-09-21T06:16:07.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.947+0000 7f28b7c4c640 1 -- 172.21.15.3:0/2560662870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b01a8fe0 con 0x7f28b0102f80 2024-09-21T06:16:07.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.948+0000 7f28b7c4c640 1 -- 172.21.15.3:0/2560662870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b0102a40 con 0x7f28b0102f80 2024-09-21T06:16:07.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.949+0000 7f28a6ffd640 1 -- 172.21.15.3:0/2560662870 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7f28a0010430 con 0x7f28b0102f80 2024-09-21T06:16:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.950+0000 7f28a6ffd640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f289c0b6790 0x7f289c0b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.950+0000 7f28a6ffd640 1 -- 172.21.15.3:0/2560662870 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f28a00e7c50 con 0x7f28b0102f80 2024-09-21T06:16:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.950+0000 7f28b59e8640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f289c0b6790 0x7f289c0b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.951+0000 7f28b59e8640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f289c0b6790 0x7f289c0b8ca0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f28ac031440 tx=0x7f28ac009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:07.959+0000 7f28a6ffd640 1 -- 172.21.15.3:0/2560662870 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f28a00ed020 con 0x7f28b0102f80 2024-09-21T06:16:08.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.206+0000 7f28b7c4c640 1 -- 172.21.15.3:0/2560662870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28b01a9210 con 0x7f28b0102f80 2024-09-21T06:16:08.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.207+0000 7f28a6ffd640 1 -- 172.21.15.3:0/2560662870 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11962 (secure 0 0 0) 0x7f28a00b2450 con 0x7f28b0102f80 2024-09-21T06:16:08.209 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:16:08.210 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 28 2024-09-21T06:16:08.210 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"btime":"2024-09-21T06:15:09:322787+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:15:08.461776+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24670,24674,24679,24691,24704]},"id":1}]} 2024-09-21T06:16:08.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.210+0000 7f28b7c4c640 1 -- 172.21.15.3:0/2560662870 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f289c0b6790 msgr2=0x7f289c0b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:08.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.210+0000 7f28b7c4c640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f289c0b6790 0x7f289c0b8ca0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f28ac031440 tx=0x7f28ac009560 comp rx=0 tx=0).stop 2024-09-21T06:16:08.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.210+0000 7f28b7c4c640 1 -- 172.21.15.3:0/2560662870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b0102f80 msgr2=0x7f28b019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:08.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.210+0000 7f28b7c4c640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b0102f80 0x7f28b019cbc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f28a000e9f0 tx=0x7f28a000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.210+0000 7f28b7c4c640 1 -- 172.21.15.3:0/2560662870 shutdown_connections 2024-09-21T06:16:08.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.210+0000 7f28b7c4c640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f289c0b6790 0x7f289c0b8ca0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.210+0000 7f28b7c4c640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28b01101e0 0x7f28b01a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.210+0000 7f28b7c4c640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b0102f80 0x7f28b019cbc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.210+0000 7f28b7c4c640 1 --2- 172.21.15.3:0/2560662870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28b0102630 0x7f28b019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.210+0000 7f28b7c4c640 1 -- 172.21.15.3:0/2560662870 >> 172.21.15.3:0/2560662870 conn(0x7f28b0079730 msgr2=0x7f28b0109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:08.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.210+0000 7f28b7c4c640 1 -- 172.21.15.3:0/2560662870 shutdown_connections 2024-09-21T06:16:08.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:08.210+0000 7f28b7c4c640 1 -- 172.21.15.3:0/2560662870 wait complete. 2024-09-21T06:16:09.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:08 smithi003 ceph-mon[31227]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-21T06:16:09.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:08 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2560662870' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:09.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:16:09.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:08 smithi137 ceph-mon[34464]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-21T06:16:09.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:08 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2560662870' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:09.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:08 smithi204 ceph-mon[34659]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-21T06:16:09.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:08 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2560662870' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.959+0000 7f474c4a5640 1 -- 172.21.15.3:0/1718137453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f474410de60 msgr2=0x7f4744110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.959+0000 7f474c4a5640 1 --2- 172.21.15.3:0/1718137453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f474410de60 0x7f4744110300 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f473c009ca0 tx=0x7f473c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.959+0000 7f474c4a5640 1 -- 172.21.15.3:0/1718137453 shutdown_connections 2024-09-21T06:16:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.959+0000 7f474c4a5640 1 --2- 172.21.15.3:0/1718137453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f474410de60 0x7f4744110300 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.959+0000 7f474c4a5640 1 --2- 172.21.15.3:0/1718137453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47441052e0 0x7f474410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.959+0000 7f474c4a5640 1 --2- 172.21.15.3:0/1718137453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47441048b0 0x7f4744104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.959+0000 7f474c4a5640 1 -- 172.21.15.3:0/1718137453 >> 172.21.15.3:0/1718137453 conn(0x7f4744079730 msgr2=0x7f4744079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.959+0000 7f474c4a5640 1 -- 172.21.15.3:0/1718137453 shutdown_connections 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.960+0000 7f474c4a5640 1 -- 172.21.15.3:0/1718137453 wait complete. 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.960+0000 7f474c4a5640 1 Processor -- start 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.961+0000 7f474c4a5640 1 -- start start 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.961+0000 7f474c4a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47441048b0 0x7f474419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.961+0000 7f474c4a5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47441052e0 0x7f474419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.961+0000 7f474c4a5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474410de60 0x7f474419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.961+0000 7f474a241640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47441048b0 0x7f474419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.961+0000 7f474a241640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47441048b0 0x7f474419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41726/0 (socket says 172.21.15.3:41726) 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.961+0000 7f474a241640 1 -- 172.21.15.3:0/3039417205 learned_addr learned my addr 172.21.15.3:0/3039417205 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.961+0000 7f474c4a5640 1 -- 172.21.15.3:0/3039417205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4744113a70 con 0x7f47441048b0 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.961+0000 7f474c4a5640 1 -- 172.21.15.3:0/3039417205 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47441138f0 con 0x7f474410de60 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.961+0000 7f474c4a5640 1 -- 172.21.15.3:0/3039417205 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4744113bf0 con 0x7f47441052e0 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.961+0000 7f4749a40640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47441052e0 0x7f474419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.961+0000 7f474aa42640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474410de60 0x7f474419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.962+0000 7f474a241640 1 -- 172.21.15.3:0/3039417205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47441052e0 msgr2=0x7f474419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.962+0000 7f474a241640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47441052e0 0x7f474419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.962+0000 7f474a241640 1 -- 172.21.15.3:0/3039417205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474410de60 msgr2=0x7f474419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.962+0000 7f474a241640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474410de60 0x7f474419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.962+0000 7f474a241640 1 -- 172.21.15.3:0/3039417205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47441a0350 con 0x7f47441048b0 2024-09-21T06:16:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.962+0000 7f474a241640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47441048b0 0x7f474419a3e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f474000e9c0 tx=0x7f474000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.962+0000 7f473b7fe640 1 -- 172.21.15.3:0/3039417205 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474000cd80 con 0x7f47441048b0 2024-09-21T06:16:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.962+0000 7f473b7fe640 1 -- 172.21.15.3:0/3039417205 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47400035f0 con 0x7f47441048b0 2024-09-21T06:16:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.962+0000 7f473b7fe640 1 -- 172.21.15.3:0/3039417205 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740010690 con 0x7f47441048b0 2024-09-21T06:16:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.962+0000 7f474c4a5640 1 -- 172.21.15.3:0/3039417205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47441a0640 con 0x7f47441048b0 2024-09-21T06:16:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.963+0000 7f474c4a5640 1 -- 172.21.15.3:0/3039417205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47441a6da0 con 0x7f47441048b0 2024-09-21T06:16:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.965+0000 7f4752a19740 1 Processor -- start 2024-09-21T06:16:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.966+0000 7f4752a19740 1 -- start start 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.966+0000 7f4752a19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee423070 0x55f4ee7ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.966+0000 7f4752a19740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.966+0000 7f4752a19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee402390 0x55f4ee7cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.966+0000 7f4752a19740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f4ee740ea0 con 0x55f4ee3fa1b0 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.966+0000 7f4752a19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f4ee7302d0 con 0x55f4ee402390 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.966+0000 7f4752a19740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f4ee7340e0 con 0x55f4ee423070 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.967+0000 7f474aa42640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee423070 0x55f4ee7ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.967+0000 7f4749a40640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee402390 0x55f4ee7cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.967+0000 7f474aa42640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee423070 0x55f4ee7ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:48174/0 (socket says 172.21.15.3:48174) 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.967+0000 7f474aa42640 1 -- 172.21.15.3:0/931510445 learned_addr learned my addr 172.21.15.3:0/931510445 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.967+0000 7f473b7fe640 1 -- 172.21.15.3:0/3039417205 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7f4740004000 con 0x7f47441048b0 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.967+0000 7f474aa42640 1 -- 172.21.15.3:0/931510445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee402390 msgr2=0x55f4ee7cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.967+0000 7f474aa42640 1 --2- 172.21.15.3:0/931510445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee402390 0x55f4ee7cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.967+0000 7f474aa42640 1 -- 172.21.15.3:0/931510445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3fa1b0 msgr2=0x55f4ee7ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.967+0000 7f474aa42640 1 --2- 172.21.15.3:0/931510445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.967+0000 7f474aa42640 1 -- 172.21.15.3:0/931510445 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4ee48e650 con 0x55f4ee423070 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.967+0000 7f474aa42640 1 --2- 172.21.15.3:0/931510445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee423070 0x55f4ee7ca2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f473c02f4c0 tx=0x7f473c009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9b06b0f13cafeb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.968+0000 7f4738d9a640 1 -- 172.21.15.3:0/931510445 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c031cf0 con 0x55f4ee423070 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.968+0000 7f4738d9a640 1 -- 172.21.15.3:0/931510445 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f473c036030 con 0x55f4ee423070 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.968+0000 7f4738d9a640 1 -- 172.21.15.3:0/931510445 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c0042a0 con 0x55f4ee423070 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.969+0000 7f473b7fe640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f47300b6790 0x7f47300b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.969+0000 7f474a241640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f47300b6790 0x7f47300b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.969+0000 7f4752a19740 1 -- 172.21.15.3:0/931510445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee423070 msgr2=0x55f4ee7ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.969+0000 7f4752a19740 1 --2- 172.21.15.3:0/931510445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee423070 0x55f4ee7ca2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f473c02f4c0 tx=0x7f473c009800 comp rx=0 tx=0).stop 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.969+0000 7f473b7fe640 1 -- 172.21.15.3:0/3039417205 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f47400e7d50 con 0x7f47441048b0 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.970+0000 7f474a241640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f47300b6790 0x7f47300b8ca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f47400e5420 tx=0x7f474007a6c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.970+0000 7f4752a19740 1 -- 172.21.15.3:0/931510445 shutdown_connections 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.970+0000 7f4752a19740 1 --2- 172.21.15.3:0/931510445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee402390 0x55f4ee7cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.970+0000 7f4752a19740 1 --2- 172.21.15.3:0/931510445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.970+0000 7f4752a19740 1 --2- 172.21.15.3:0/931510445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee423070 0x55f4ee7ca2c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.970+0000 7f4752a19740 1 -- 172.21.15.3:0/931510445 >> 172.21.15.3:0/931510445 conn(0x55f4ee417460 msgr2=0x55f4ee7c81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.970+0000 7f4752a19740 1 -- 172.21.15.3:0/931510445 shutdown_connections 2024-09-21T06:16:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.970+0000 7f4752a19740 1 -- 172.21.15.3:0/931510445 wait complete. 2024-09-21T06:16:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.971+0000 7f4752a19740 1 -- start start 2024-09-21T06:16:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.971+0000 7f4752a19740 1 Processor -- start 2024-09-21T06:16:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.972+0000 7f4752a19740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7d59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.972+0000 7f4752a19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee402390 0x55f4ee7d5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.972+0000 7f474aa42640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7d59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.972+0000 7f474aa42640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7d59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41752/0 (socket says 172.21.15.3:41752) 2024-09-21T06:16:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.972+0000 7f474aa42640 1 -- 172.21.15.3:0/1311264757 learned_addr learned my addr 172.21.15.3:0/1311264757 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.972+0000 7f4752a19740 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee423070 0x55f4ee7e5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.972+0000 7f4752a19740 1 -- 172.21.15.3:0/1311264757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f4ee591700 con 0x55f4ee3fa1b0 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.972+0000 7f4752a19740 1 -- 172.21.15.3:0/1311264757 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f4ee6cb160 con 0x55f4ee423070 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.972+0000 7f4752a19740 1 -- 172.21.15.3:0/1311264757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f4ee7dccd0 con 0x55f4ee402390 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.972+0000 7f4749a40640 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee402390 0x55f4ee7d5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.973+0000 7f4749a40640 1 -- 172.21.15.3:0/1311264757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee423070 msgr2=0x55f4ee7e5fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.973+0000 7f4749a40640 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee423070 0x55f4ee7e5fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.973+0000 7f4749a40640 1 -- 172.21.15.3:0/1311264757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3fa1b0 msgr2=0x55f4ee7d59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.973+0000 7f4749a40640 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7d59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.973+0000 7f4749a40640 1 -- 172.21.15.3:0/1311264757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4ee6f8c90 con 0x55f4ee402390 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.973+0000 7f474aa42640 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7d59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.973+0000 7f4752a19740 10 client.0 _ll_register_callbacks cb 0x55f4ee6e49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.973+0000 7f4752a19740 1 -- 172.21.15.3:0/1311264757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473c036030 con 0x55f4ee402390 2024-09-21T06:16:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.973+0000 7f4749a40640 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee402390 0x55f4ee7d5f00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4734005d00 tx=0x7f47340118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef9e93836956b670 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.973+0000 7f472dffb640 10 client.25399 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.974+0000 7f472dffb640 1 -- 172.21.15.3:0/1311264757 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734004020 con 0x55f4ee402390 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.974+0000 7f472dffb640 1 -- 172.21.15.3:0/1311264757 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f47340041c0 con 0x55f4ee402390 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.974+0000 7f472dffb640 1 -- 172.21.15.3:0/1311264757 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47340043c0 con 0x55f4ee402390 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.974+0000 7f4752a19740 10 client.25399 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.974+0000 7f4752a19740 1 -- 172.21.15.3:0/1311264757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55f4ee7e88e0 con 0x55f4ee402390 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.974+0000 7f472dffb640 1 -- 172.21.15.3:0/1311264757 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f47340107c0 con 0x55f4ee402390 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.974+0000 7f4752a19740 10 client.25399 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.974+0000 7f4752a19740 10 client.25399 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.974+0000 7f4752a19740 20 client.25399 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.974+0000 7f4752a19740 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f4ee3e19f0 0x55f4ee7ec020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.974+0000 7f4752a19740 4 client.25399 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.974+0000 7f4752a19740 1 -- 172.21.15.3:0/1311264757 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4ee742df0 con 0x55f4ee3e19f0 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.975+0000 7f474aa42640 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f4ee3e19f0 0x55f4ee7ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.975+0000 7f472dffb640 10 client.25399 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.975+0000 7f474aa42640 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f4ee3e19f0 0x55f4ee7ec020 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4d10f95f768549c1 server_cookie=824af355043126c3 in_seq=0 out_seq=0 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f472dffb640 1 -- 172.21.15.3:0/1311264757 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f4ee742df0 con 0x55f4ee3e19f0 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f472dffb640 10 client.25399 handle_command_reply: tid=0 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f4752a19740 1 client.25399 shutdown 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f4752a19740 1 -- 172.21.15.3:0/1311264757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee402390 msgr2=0x55f4ee7d5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f4752a19740 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee402390 0x55f4ee7d5f00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4734005d00 tx=0x7f47340118d0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f4752a19740 1 -- 172.21.15.3:0/1311264757 shutdown_connections 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f4752a19740 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f4ee3e19f0 0x55f4ee7ec020 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f472dffb640 0 client.25399 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f4752a19740 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee402390 0x55f4ee7d5f00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f4752a19740 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee423070 0x55f4ee7e5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f4752a19740 1 --2- 172.21.15.3:0/1311264757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7d59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f4752a19740 1 -- 172.21.15.3:0/1311264757 >> 172.21.15.3:0/1311264757 conn(0x55f4ee417460 msgr2=0x55f4ee7cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f4752a19740 1 -- 172.21.15.3:0/1311264757 shutdown_connections 2024-09-21T06:16:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.976+0000 7f4752a19740 1 -- 172.21.15.3:0/1311264757 wait complete. 2024-09-21T06:16:09.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:09.977+0000 7f4752a19740 20 client.25399 trim_cache size 0 max 16384 2024-09-21T06:16:10.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.003+0000 7f4752a19740 1 Processor -- start 2024-09-21T06:16:10.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.003+0000 7f4752a19740 1 -- start start 2024-09-21T06:16:10.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f4752a19740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3e19f0 0x55f4ee7cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f4752a19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f4752a19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee402390 0x55f4ee7c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f4752a19740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f47441ad230 con 0x55f4ee3e19f0 2024-09-21T06:16:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f4752a19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47441aed70 con 0x55f4ee402390 2024-09-21T06:16:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f4752a19740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f47441ad900 con 0x55f4ee3fa1b0 2024-09-21T06:16:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f474aa42640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3e19f0 0x55f4ee7cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f474aa42640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3e19f0 0x55f4ee7cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41760/0 (socket says 172.21.15.3:41760) 2024-09-21T06:16:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f474aa42640 1 -- 172.21.15.3:0/3618008104 learned_addr learned my addr 172.21.15.3:0/3618008104 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f4749a40640 1 --2- 172.21.15.3:0/3618008104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f474aa42640 1 -- 172.21.15.3:0/3618008104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee3fa1b0 msgr2=0x55f4ee7c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f474aa42640 1 --2- 172.21.15.3:0/3618008104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f474aa42640 1 -- 172.21.15.3:0/3618008104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee402390 msgr2=0x55f4ee7c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f474aa42640 1 --2- 172.21.15.3:0/3618008104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee402390 0x55f4ee7c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.004+0000 7f474aa42640 1 -- 172.21.15.3:0/3618008104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4ee7d8100 con 0x55f4ee3e19f0 2024-09-21T06:16:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.005+0000 7f474aa42640 1 --2- 172.21.15.3:0/3618008104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3e19f0 0x55f4ee7cba00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f473c009a60 tx=0x7f473c02f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275734b97f30d5c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.005+0000 7f472ffff640 1 -- 172.21.15.3:0/3618008104 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c05c020 con 0x55f4ee3e19f0 2024-09-21T06:16:10.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.005+0000 7f472ffff640 1 -- 172.21.15.3:0/3618008104 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f473c031900 con 0x55f4ee3e19f0 2024-09-21T06:16:10.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.005+0000 7f472ffff640 1 -- 172.21.15.3:0/3618008104 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c04a040 con 0x55f4ee3e19f0 2024-09-21T06:16:10.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.006+0000 7f4752a19740 1 -- 172.21.15.3:0/3618008104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3e19f0 msgr2=0x55f4ee7cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:10.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.006+0000 7f4752a19740 1 --2- 172.21.15.3:0/3618008104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3e19f0 0x55f4ee7cba00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f473c009a60 tx=0x7f473c02f3a0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.006+0000 7f4752a19740 1 -- 172.21.15.3:0/3618008104 shutdown_connections 2024-09-21T06:16:10.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.006+0000 7f4752a19740 1 --2- 172.21.15.3:0/3618008104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.006+0000 7f4752a19740 1 --2- 172.21.15.3:0/3618008104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee402390 0x55f4ee7c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.006+0000 7f4752a19740 1 --2- 172.21.15.3:0/3618008104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee3e19f0 0x55f4ee7cba00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.006+0000 7f4752a19740 1 -- 172.21.15.3:0/3618008104 >> 172.21.15.3:0/3618008104 conn(0x55f4ee417460 msgr2=0x55f4ee7cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:10.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.006+0000 7f4752a19740 1 -- 172.21.15.3:0/3618008104 shutdown_connections 2024-09-21T06:16:10.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.007+0000 7f4752a19740 1 -- 172.21.15.3:0/3618008104 wait complete. 2024-09-21T06:16:10.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.007+0000 7f4752a19740 1 -- start start 2024-09-21T06:16:10.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.007+0000 7f4752a19740 1 Processor -- start 2024-09-21T06:16:10.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.008+0000 7f4752a19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee3e19f0 0x55f4ee7e25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.008+0000 7f4752a19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7ddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.008+0000 7f4752a19740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee402390 0x55f4ee7de030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.008+0000 7f4752a19740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f47441ae6a0 con 0x55f4ee402390 2024-09-21T06:16:10.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.008+0000 7f4752a19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47441adfd0 con 0x55f4ee3fa1b0 2024-09-21T06:16:10.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.008+0000 7f474aa42640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee3e19f0 0x55f4ee7e25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.008+0000 7f474aa42640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee3e19f0 0x55f4ee7e25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:48196/0 (socket says 172.21.15.3:48196) 2024-09-21T06:16:10.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.008+0000 7f474aa42640 1 -- 172.21.15.3:0/1751278809 learned_addr learned my addr 172.21.15.3:0/1751278809 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:10.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.008+0000 7f4752a19740 1 -- 172.21.15.3:0/1751278809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f47441ab210 con 0x55f4ee3e19f0 2024-09-21T06:16:10.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.008+0000 7f474aa42640 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee402390 0x55f4ee7de030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.008+0000 7f4752a19740 10 client.0 _ll_register_callbacks cb 0x7f473400bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:10.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.009+0000 7f4749a40640 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7ddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.009+0000 7f474aa42640 1 -- 172.21.15.3:0/1751278809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee3fa1b0 msgr2=0x55f4ee7ddaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.009+0000 7f474aa42640 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7ddaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.009+0000 7f474aa42640 1 -- 172.21.15.3:0/1751278809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee402390 msgr2=0x55f4ee7de030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.009+0000 7f474aa42640 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee402390 0x55f4ee7de030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.009+0000 7f474aa42640 1 -- 172.21.15.3:0/1751278809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4ee7de670 con 0x55f4ee3e19f0 2024-09-21T06:16:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.009+0000 7f4749a40640 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7ddaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.009+0000 7f474aa42640 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee3e19f0 0x55f4ee7e25a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f473c009620 tx=0x7f473c055040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d41dfd0aa930514 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.009+0000 7f472dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:16:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.009+0000 7f472dffb640 1 -- 172.21.15.3:0/1751278809 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c069060 con 0x55f4ee3e19f0 2024-09-21T06:16:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.010+0000 7f472dffb640 1 -- 172.21.15.3:0/1751278809 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f473c031650 con 0x55f4ee3e19f0 2024-09-21T06:16:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.010+0000 7f4752a19740 1 -- 172.21.15.3:0/1751278809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f4ee73e8a0 con 0x55f4ee3e19f0 2024-09-21T06:16:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.010+0000 7f472dffb640 1 -- 172.21.15.3:0/1751278809 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f473c05c030 con 0x55f4ee3e19f0 2024-09-21T06:16:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.010+0000 7f4752a19740 10 client.25405 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:10.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.010+0000 7f4752a19740 1 -- 172.21.15.3:0/1751278809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f4ee8565f0 con 0x55f4ee3e19f0 2024-09-21T06:16:10.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.010+0000 7f472dffb640 1 -- 172.21.15.3:0/1751278809 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f4ee73e8a0 con 0x55f4ee3e19f0 2024-09-21T06:16:10.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.011+0000 7f472dffb640 1 -- 172.21.15.3:0/1751278809 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f473c043070 con 0x55f4ee3e19f0 2024-09-21T06:16:10.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.011+0000 7f4752a19740 10 client.25405 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:10.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.011+0000 7f4752a19740 10 client.25405 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:16:10.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.011+0000 7f4752a19740 20 client.25405 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:10.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.011+0000 7f4752a19740 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f4ee423070 0x55f4ee8588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.011+0000 7f4752a19740 4 client.25405 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:10.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.011+0000 7f4752a19740 1 -- 172.21.15.3:0/1751278809 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4ee73bcd0 con 0x55f4ee423070 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.011+0000 7f4749a40640 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f4ee423070 0x55f4ee8588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f472dffb640 10 client.25405 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f4749a40640 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f4ee423070 0x55f4ee8588f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3488e211bb414864 server_cookie=d3be699aa517ffd8 in_seq=0 out_seq=0 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f472dffb640 1 -- 172.21.15.3:0/1751278809 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4ee73bcd0 con 0x55f4ee423070 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f472dffb640 10 client.25405 handle_command_reply: tid=0 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f4752a19740 1 client.25405 shutdown 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f4752a19740 1 -- 172.21.15.3:0/1751278809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee3e19f0 msgr2=0x55f4ee7e25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f4752a19740 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee3e19f0 0x55f4ee7e25a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f473c009620 tx=0x7f473c055040 comp rx=0 tx=0).stop 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f4752a19740 1 -- 172.21.15.3:0/1751278809 shutdown_connections 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f4752a19740 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55f4ee423070 0x55f4ee8588f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f4752a19740 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4ee402390 0x55f4ee7de030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f4752a19740 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4ee3fa1b0 0x55f4ee7ddaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f4752a19740 1 --2- 172.21.15.3:0/1751278809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4ee3e19f0 0x55f4ee7e25a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f4752a19740 1 -- 172.21.15.3:0/1751278809 >> 172.21.15.3:0/1751278809 conn(0x55f4ee417460 msgr2=0x55f4ee7cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:10.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.012+0000 7f472dffb640 0 client.25405 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:16:10.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.013+0000 7f4752a19740 1 -- 172.21.15.3:0/1751278809 shutdown_connections 2024-09-21T06:16:10.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.013+0000 7f4752a19740 1 -- 172.21.15.3:0/1751278809 wait complete. 2024-09-21T06:16:10.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.013+0000 7f4752a19740 20 client.25405 trim_cache size 0 max 16384 2024-09-21T06:16:10.017 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:16:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.014+0000 7f474c4a5640 1 -- 172.21.15.3:0/3039417205 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f47300b6790 msgr2=0x7f47300b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.014+0000 7f474c4a5640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f47300b6790 0x7f47300b8ca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f47400e5420 tx=0x7f474007a6c0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.014+0000 7f474c4a5640 1 -- 172.21.15.3:0/3039417205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47441048b0 msgr2=0x7f474419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.014+0000 7f474c4a5640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47441048b0 0x7f474419a3e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f474000e9c0 tx=0x7f474000ee80 comp rx=0 tx=0).stop 2024-09-21T06:16:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.014+0000 7f474c4a5640 1 -- 172.21.15.3:0/3039417205 shutdown_connections 2024-09-21T06:16:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.014+0000 7f474c4a5640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f47300b6790 0x7f47300b8ca0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.014+0000 7f474c4a5640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47441052e0 0x7f474419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.014+0000 7f474c4a5640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474410de60 0x7f474419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.014+0000 7f474c4a5640 1 --2- 172.21.15.3:0/3039417205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47441048b0 0x7f474419a3e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.014+0000 7f474c4a5640 1 -- 172.21.15.3:0/3039417205 >> 172.21.15.3:0/3039417205 conn(0x7f4744079730 msgr2=0x7f47441007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:10.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.014+0000 7f474c4a5640 1 -- 172.21.15.3:0/3039417205 shutdown_connections 2024-09-21T06:16:10.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:10.015+0000 7f474c4a5640 1 -- 172.21.15.3:0/3039417205 wait complete. 2024-09-21T06:16:10.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:10 smithi003 ceph-mon[31227]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-21T06:16:10.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:16:11.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:10 smithi137 ceph-mon[34464]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-21T06:16:11.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:10 smithi204 ceph-mon[34659]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-21T06:16:11.172 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.646+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/3724511648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf041082e0 msgr2=0x7fbf041086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.646+0000 7fbf0bc4d640 1 --2- 172.21.15.3:0/3724511648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf041082e0 0x7fbf041086f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef402a410 comp rx=0 tx=0).stop 2024-09-21T06:16:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.646+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/3724511648 shutdown_connections 2024-09-21T06:16:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.646+0000 7fbf0bc4d640 1 --2- 172.21.15.3:0/3724511648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf041082e0 0x7fbf041086f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.646+0000 7fbf0bc4d640 1 --2- 172.21.15.3:0/3724511648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf04108c30 0x7fbf0410f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.646+0000 7fbf0bc4d640 1 --2- 172.21.15.3:0/3724511648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf041029b0 0x7fbf04102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.646+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/3724511648 >> 172.21.15.3:0/3724511648 conn(0x7fbf0407a740 msgr2=0x7fbf0407ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.647+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/3724511648 shutdown_connections 2024-09-21T06:16:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.647+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/3724511648 wait complete. 2024-09-21T06:16:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.647+0000 7fbf0bc4d640 1 Processor -- start 2024-09-21T06:16:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.648+0000 7fbf0bc4d640 1 -- start start 2024-09-21T06:16:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.648+0000 7fbf0bc4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf041029b0 0x7fbf0419b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.648+0000 7fbf0bc4d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf041082e0 0x7fbf0419b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.648+0000 7fbf091e8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf041082e0 0x7fbf0419b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.648+0000 7fbf091e8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf041082e0 0x7fbf0419b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41800/0 (socket says 172.21.15.3:41800) 2024-09-21T06:16:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.648+0000 7fbf091e8640 1 -- 172.21.15.3:0/1189632991 learned_addr learned my addr 172.21.15.3:0/1189632991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.648+0000 7fbf0bc4d640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf04108c30 0x7fbf0419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.648+0000 7fbf099e9640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf041029b0 0x7fbf0419b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.648+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/1189632991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041128f0 con 0x7fbf041082e0 2024-09-21T06:16:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.649+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/1189632991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04112770 con 0x7fbf04108c30 2024-09-21T06:16:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.649+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/1189632991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04112a70 con 0x7fbf041029b0 2024-09-21T06:16:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.649+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf04108c30 0x7fbf0419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.649+0000 7fbf091e8640 1 -- 172.21.15.3:0/1189632991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf041029b0 msgr2=0x7fbf0419b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.649+0000 7fbf091e8640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf041029b0 0x7fbf0419b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.649+0000 7fbf091e8640 1 -- 172.21.15.3:0/1189632991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf04108c30 msgr2=0x7fbf0419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.649+0000 7fbf091e8640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf04108c30 0x7fbf0419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.649+0000 7fbf091e8640 1 -- 172.21.15.3:0/1189632991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf041a6390 con 0x7fbf041082e0 2024-09-21T06:16:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.649+0000 7fbf091e8640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf041082e0 0x7fbf0419b640 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009850 tx=0x7fbef4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.650+0000 7fbefaffd640 1 -- 172.21.15.3:0/1189632991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4035030 con 0x7fbf041082e0 2024-09-21T06:16:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.650+0000 7fbefaffd640 1 -- 172.21.15.3:0/1189632991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef4041070 con 0x7fbf041082e0 2024-09-21T06:16:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.650+0000 7fbefaffd640 1 -- 172.21.15.3:0/1189632991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4030d00 con 0x7fbf041082e0 2024-09-21T06:16:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.650+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/1189632991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf04198ef0 con 0x7fbf041082e0 2024-09-21T06:16:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.650+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/1189632991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf04199390 con 0x7fbf041082e0 2024-09-21T06:16:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.654+0000 7fbf12bbe740 1 Processor -- start 2024-09-21T06:16:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.654+0000 7fbf12bbe740 1 -- start start 2024-09-21T06:16:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.654+0000 7fbf12bbe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b651b0 0x5572d9f0a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.654+0000 7fbefaffd640 1 -- 172.21.15.3:0/1189632991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7fbef4031a60 con 0x7fbf041082e0 2024-09-21T06:16:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.655+0000 7fbf12bbe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 0x5572d9f0aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.655+0000 7fbf0a1ea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b651b0 0x5572d9f0a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.655+0000 7fbf0a1ea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b651b0 0x5572d9f0a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:48228/0 (socket says 172.21.15.3:48228) 2024-09-21T06:16:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.655+0000 7fbf099e9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 0x5572d9f0aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.655+0000 7fbf099e9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 0x5572d9f0aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41804/0 (socket says 172.21.15.3:41804) 2024-09-21T06:16:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.655+0000 7fbf12bbe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 0x5572d9f0b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.655+0000 7fbf0a1ea640 1 -- 172.21.15.3:0/3287446545 learned_addr learned my addr 172.21.15.3:0/3287446545 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.655+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3287446545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5572d9e7f6f0 con 0x5572d9b3c2f0 2024-09-21T06:16:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.655+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3287446545 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5572d9e6eb20 con 0x5572d9b444d0 2024-09-21T06:16:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.655+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3287446545 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5572d9e72930 con 0x5572d9b651b0 2024-09-21T06:16:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.655+0000 7fbefaffd640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbef00b6790 0x7fbef00b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.656+0000 7fbf091e8640 1 --2- 172.21.15.3:0/3287446545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 0x5572d9f0b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.656+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbef00b6790 0x7fbef00b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.656+0000 7fbefaffd640 1 -- 172.21.15.3:0/1189632991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fbef410c350 con 0x7fbf041082e0 2024-09-21T06:16:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.656+0000 7fbf0a1ea640 1 -- 172.21.15.3:0/3287446545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 msgr2=0x5572d9f0b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.656+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/3287446545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 0x5572d9f0b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.656+0000 7fbf0a1ea640 1 -- 172.21.15.3:0/3287446545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 msgr2=0x5572d9f0aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.656+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/3287446545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 0x5572d9f0aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.656+0000 7fbf0a1ea640 1 -- 172.21.15.3:0/3287446545 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d9e34530 con 0x5572d9b651b0 2024-09-21T06:16:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.656+0000 7fbf091e8640 1 --2- 172.21.15.3:0/3287446545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 0x5572d9f0b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.656+0000 7fbf099e9640 1 --2- 172.21.15.3:0/3287446545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 0x5572d9f0aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.656+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/3287446545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b651b0 0x5572d9f0a9b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbefc013ac0 tx=0x7fbefc013f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3284deea98afb86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.657+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbef00b6790 0x7fbef00b8ca0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbefc01d000 tx=0x7fbefc01f000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.657+0000 7fbeeffff640 1 -- 172.21.15.3:0/3287446545 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0120d0 con 0x5572d9b651b0 2024-09-21T06:16:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.657+0000 7fbeeffff640 1 -- 172.21.15.3:0/3287446545 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbefc022070 con 0x5572d9b651b0 2024-09-21T06:16:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.657+0000 7fbeeffff640 1 -- 172.21.15.3:0/3287446545 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc010830 con 0x5572d9b651b0 2024-09-21T06:16:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.658+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3287446545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b651b0 msgr2=0x5572d9f0a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.658+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/3287446545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b651b0 0x5572d9f0a9b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbefc013ac0 tx=0x7fbefc013f80 comp rx=0 tx=0).stop 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.658+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3287446545 shutdown_connections 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.658+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/3287446545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 0x5572d9f0b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.658+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/3287446545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 0x5572d9f0aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.658+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/3287446545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b651b0 0x5572d9f0a9b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.658+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3287446545 >> 172.21.15.3:0/3287446545 conn(0x5572d9b59300 msgr2=0x5572d9f088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.659+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3287446545 shutdown_connections 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.659+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3287446545 wait complete. 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.659+0000 7fbf12bbe740 1 -- start start 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.660+0000 7fbf12bbe740 1 Processor -- start 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.661+0000 7fbf12bbe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b3c2f0 0x5572d9f16150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.661+0000 7fbf12bbe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 0x5572d9f16690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.661+0000 7fbf12bbe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b651b0 0x5572d9f223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.661+0000 7fbf099e9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b3c2f0 0x5572d9f16150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.661+0000 7fbf099e9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b3c2f0 0x5572d9f16150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:48230/0 (socket says 172.21.15.3:48230) 2024-09-21T06:16:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.661+0000 7fbf099e9640 1 -- 172.21.15.3:0/3220353526 learned_addr learned my addr 172.21.15.3:0/3220353526 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.661+0000 7fbf091e8640 1 --2- 172.21.15.3:0/3220353526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 0x5572d9f16690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.661+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/3220353526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b651b0 0x5572d9f223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.661+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3220353526 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5572d9da6ab0 con 0x5572d9b651b0 2024-09-21T06:16:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.661+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3220353526 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5572d9e14730 con 0x5572d9b444d0 2024-09-21T06:16:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.661+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3220353526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5572d9f190d0 con 0x5572d9b3c2f0 2024-09-21T06:16:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.662+0000 7fbf12bbe740 10 client.0 _ll_register_callbacks cb 0x5572d9e686b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.662+0000 7fbf0a1ea640 1 -- 172.21.15.3:0/3220353526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b3c2f0 msgr2=0x5572d9f16150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.662+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/3220353526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b3c2f0 0x5572d9f16150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.662+0000 7fbf0a1ea640 1 -- 172.21.15.3:0/3220353526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 msgr2=0x5572d9f16690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.662+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/3220353526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 0x5572d9f16690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.662+0000 7fbf0a1ea640 1 -- 172.21.15.3:0/3220353526 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d9bd0790 con 0x5572d9b651b0 2024-09-21T06:16:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.662+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/3220353526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b651b0 0x5572d9f223c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbefc013020 tx=0x7fbefc009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcf61145ca1464f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.662+0000 7fbeed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.662+0000 7fbeed7fa640 1 -- 172.21.15.3:0/3220353526 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc035060 con 0x5572d9b651b0 2024-09-21T06:16:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.662+0000 7fbeed7fa640 1 -- 172.21.15.3:0/3220353526 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fbefc00fde0 con 0x5572d9b651b0 2024-09-21T06:16:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.662+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3220353526 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbefc022070 con 0x5572d9b651b0 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.663+0000 7fbeed7fa640 1 -- 172.21.15.3:0/3220353526 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbefc013680 con 0x5572d9b651b0 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.663+0000 7fbf12bbe740 10 client.15837 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.663+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3220353526 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5572d9c49620 con 0x5572d9b651b0 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.663+0000 7fbeed7fa640 1 -- 172.21.15.3:0/3220353526 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbefc012cd0 con 0x5572d9b651b0 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.663+0000 7fbeed7fa640 1 -- 172.21.15.3:0/3220353526 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbefc010bc0 con 0x5572d9b651b0 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.663+0000 7fbf12bbe740 10 client.15837 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.663+0000 7fbf12bbe740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.663+0000 7fbf12bbe740 20 client.15837 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.663+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/3220353526 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5572d9b23d80 0x5572d9f28110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.664+0000 7fbf12bbe740 4 client.15837 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.664+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3220353526 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbefc022070 con 0x5572d9b23d80 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.664+0000 7fbf099e9640 1 --2- 172.21.15.3:0/3220353526 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5572d9b23d80 0x5572d9f28110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.664+0000 7fbf099e9640 1 --2- 172.21.15.3:0/3220353526 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5572d9b23d80 0x5572d9f28110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebbaf0694d12fd49 server_cookie=ee97561be8d2fef7 in_seq=0 out_seq=0 2024-09-21T06:16:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.664+0000 7fbeed7fa640 10 client.15837 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.665+0000 7fbeed7fa640 1 -- 172.21.15.3:0/3220353526 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fbefc022070 con 0x5572d9b23d80 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.665+0000 7fbeed7fa640 10 client.15837 handle_command_reply: tid=0 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.665+0000 7fbf12bbe740 1 client.15837 shutdown 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.665+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3220353526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b651b0 msgr2=0x5572d9f223c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.665+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/3220353526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b651b0 0x5572d9f223c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbefc013020 tx=0x7fbefc009be0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.666+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3220353526 shutdown_connections 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.666+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/3220353526 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5572d9b23d80 0x5572d9f28110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.666+0000 7fbeed7fa640 0 client.15837 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.666+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/3220353526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b651b0 0x5572d9f223c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.666+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/3220353526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 0x5572d9f16690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.666+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/3220353526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b3c2f0 0x5572d9f16150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.666+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3220353526 >> 172.21.15.3:0/3220353526 conn(0x5572d9b59300 msgr2=0x5572d9f0b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.666+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3220353526 shutdown_connections 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.666+0000 7fbf12bbe740 1 -- 172.21.15.3:0/3220353526 wait complete. 2024-09-21T06:16:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.666+0000 7fbf12bbe740 20 client.15837 trim_cache size 0 max 16384 2024-09-21T06:16:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.697+0000 7fbf12bbe740 1 Processor -- start 2024-09-21T06:16:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.697+0000 7fbf12bbe740 1 -- start start 2024-09-21T06:16:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.698+0000 7fbf12bbe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b23d80 0x5572d9f16630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.698+0000 7fbf12bbe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 0x5572d9f18030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.698+0000 7fbf099e9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b23d80 0x5572d9f16630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.698+0000 7fbf091e8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 0x5572d9f18030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.698+0000 7fbf099e9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b23d80 0x5572d9f16630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:48244/0 (socket says 172.21.15.3:48244) 2024-09-21T06:16:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.698+0000 7fbf091e8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 0x5572d9f18030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41826/0 (socket says 172.21.15.3:41826) 2024-09-21T06:16:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.698+0000 7fbf099e9640 1 -- 172.21.15.3:0/2804407610 learned_addr learned my addr 172.21.15.3:0/2804407610 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.699+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/2804407610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 0x5572d9f0bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.699+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2804407610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041ad4f0 con 0x5572d9b3c2f0 2024-09-21T06:16:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.699+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2804407610 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041af030 con 0x5572d9b444d0 2024-09-21T06:16:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.699+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2804407610 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041adbc0 con 0x5572d9b23d80 2024-09-21T06:16:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.699+0000 7fbf099e9640 1 -- 172.21.15.3:0/2804407610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 msgr2=0x5572d9f0bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.699+0000 7fbf099e9640 1 --2- 172.21.15.3:0/2804407610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 0x5572d9f0bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.699+0000 7fbf099e9640 1 -- 172.21.15.3:0/2804407610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 msgr2=0x5572d9f18030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.699+0000 7fbf099e9640 1 --2- 172.21.15.3:0/2804407610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 0x5572d9f18030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.699+0000 7fbf099e9640 1 -- 172.21.15.3:0/2804407610 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d9f09240 con 0x5572d9b23d80 2024-09-21T06:16:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.699+0000 7fbf091e8640 1 --2- 172.21.15.3:0/2804407610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 0x5572d9f18030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.699+0000 7fbf099e9640 1 --2- 172.21.15.3:0/2804407610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b23d80 0x5572d9f16630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009fc0 tx=0x7fbf00005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ce588158f2f5ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.700+0000 7fbeef7fe640 1 -- 172.21.15.3:0/2804407610 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00025440 con 0x5572d9b23d80 2024-09-21T06:16:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.700+0000 7fbeef7fe640 1 -- 172.21.15.3:0/2804407610 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf000108e0 con 0x5572d9b23d80 2024-09-21T06:16:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.700+0000 7fbeef7fe640 1 -- 172.21.15.3:0/2804407610 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0001f7c0 con 0x5572d9b23d80 2024-09-21T06:16:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.701+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2804407610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b23d80 msgr2=0x5572d9f16630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.701+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/2804407610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b23d80 0x5572d9f16630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009fc0 tx=0x7fbf00005f90 comp rx=0 tx=0).stop 2024-09-21T06:16:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.701+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2804407610 shutdown_connections 2024-09-21T06:16:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.701+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/2804407610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b444d0 0x5572d9f0bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.701+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/2804407610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b3c2f0 0x5572d9f18030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.701+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/2804407610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b23d80 0x5572d9f16630 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.701+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2804407610 >> 172.21.15.3:0/2804407610 conn(0x5572d9b59300 msgr2=0x5572d9f0b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.701+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2804407610 shutdown_connections 2024-09-21T06:16:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.701+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2804407610 wait complete. 2024-09-21T06:16:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.702+0000 7fbf12bbe740 1 -- start start 2024-09-21T06:16:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.702+0000 7fbf12bbe740 1 Processor -- start 2024-09-21T06:16:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.702+0000 7fbf12bbe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b23d80 0x5572d9f19be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.703+0000 7fbf099e9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b23d80 0x5572d9f19be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.703+0000 7fbf099e9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b23d80 0x5572d9f19be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41842/0 (socket says 172.21.15.3:41842) 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.703+0000 7fbf099e9640 1 -- 172.21.15.3:0/2596398879 learned_addr learned my addr 172.21.15.3:0/2596398879 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.703+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b3c2f0 0x5572d9f1a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.703+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b444d0 0x5572d9f8a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.703+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2596398879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041ae960 con 0x5572d9b23d80 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.703+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2596398879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041ae290 con 0x5572d9b3c2f0 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.703+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2596398879 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041ab4d0 con 0x5572d9b444d0 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.703+0000 7fbf091e8640 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b3c2f0 0x5572d9f1a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.703+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b444d0 0x5572d9f8a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbf0a1ea640 1 -- 172.21.15.3:0/2596398879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b3c2f0 msgr2=0x5572d9f1a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b3c2f0 0x5572d9f1a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbf0a1ea640 1 -- 172.21.15.3:0/2596398879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b23d80 msgr2=0x5572d9f19be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b23d80 0x5572d9f19be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbf12bbe740 10 client.0 _ll_register_callbacks cb 0x7fbefc0138c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbf0a1ea640 1 -- 172.21.15.3:0/2596398879 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d9f8afa0 con 0x5572d9b444d0 2024-09-21T06:16:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbf099e9640 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b23d80 0x5572d9f19be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2596398879 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf000108e0 con 0x5572d9b444d0 2024-09-21T06:16:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbf0a1ea640 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b444d0 0x5572d9f8a960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0050a0 tx=0x7fbefc013500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c546ae9831c99ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbeed7fa640 10 client.25423 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:16:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbeed7fa640 1 -- 172.21.15.3:0/2596398879 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc005bd0 con 0x5572d9b444d0 2024-09-21T06:16:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbeed7fa640 1 -- 172.21.15.3:0/2596398879 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fbefc012b30 con 0x5572d9b444d0 2024-09-21T06:16:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbeed7fa640 1 -- 172.21.15.3:0/2596398879 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbefc012db0 con 0x5572d9b444d0 2024-09-21T06:16:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbf12bbe740 10 client.25423 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.704+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2596398879 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5572d9f92110 con 0x5572d9b444d0 2024-09-21T06:16:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.705+0000 7fbeed7fa640 1 -- 172.21.15.3:0/2596398879 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbefc004be0 con 0x5572d9b444d0 2024-09-21T06:16:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.705+0000 7fbf12bbe740 10 client.25423 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.705+0000 7fbf12bbe740 10 client.25423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.705+0000 7fbf12bbe740 20 client.25423 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.705+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5572d9b651b0 0x5572d9f947d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.705+0000 7fbf12bbe740 4 client.25423 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:16:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.705+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2596398879 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5572d9e81640 con 0x5572d9b651b0 2024-09-21T06:16:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.705+0000 7fbf099e9640 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5572d9b651b0 0x5572d9f947d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.706+0000 7fbf099e9640 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5572d9b651b0 0x5572d9f947d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77b1b3608864f99 server_cookie=8260e388c472f03c in_seq=0 out_seq=0 2024-09-21T06:16:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.706+0000 7fbeed7fa640 10 client.25423 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.712+0000 7fbeed7fa640 1 -- 172.21.15.3:0/2596398879 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5572d9e81640 con 0x5572d9b651b0 2024-09-21T06:16:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.712+0000 7fbeed7fa640 10 client.25423 handle_command_reply: tid=0 2024-09-21T06:16:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.712+0000 7fbf12bbe740 1 client.25423 shutdown 2024-09-21T06:16:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.712+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2596398879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b444d0 msgr2=0x5572d9f8a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.712+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b444d0 0x5572d9f8a960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0050a0 tx=0x7fbefc013500 comp rx=0 tx=0).stop 2024-09-21T06:16:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2596398879 shutdown_connections 2024-09-21T06:16:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5572d9b651b0 0x5572d9f947d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572d9b444d0 0x5572d9f8a960 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9b3c2f0 0x5572d9f1a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbeed7fa640 0 client.25423 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbf12bbe740 1 --2- 172.21.15.3:0/2596398879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572d9b23d80 0x5572d9f19be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2596398879 >> 172.21.15.3:0/2596398879 conn(0x5572d9b59300 msgr2=0x5572d9f0c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2596398879 shutdown_connections 2024-09-21T06:16:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbf12bbe740 1 -- 172.21.15.3:0/2596398879 wait complete. 2024-09-21T06:16:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbf12bbe740 20 client.25423 trim_cache size 0 max 16384 2024-09-21T06:16:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:16:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:16:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:16:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:16:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/1189632991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbef00b6790 msgr2=0x7fbef00b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbf0bc4d640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbef00b6790 0x7fbef00b8ca0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbefc01d000 tx=0x7fbefc01f000 comp rx=0 tx=0).stop 2024-09-21T06:16:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/1189632991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf041082e0 msgr2=0x7fbf0419b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.713+0000 7fbf0bc4d640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf041082e0 0x7fbf0419b640 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009850 tx=0x7fbef4005d50 comp rx=0 tx=0).stop 2024-09-21T06:16:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.714+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/1189632991 shutdown_connections 2024-09-21T06:16:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.714+0000 7fbf0bc4d640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbef00b6790 0x7fbef00b8ca0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.714+0000 7fbf0bc4d640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf04108c30 0x7fbf0419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.714+0000 7fbf0bc4d640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf041082e0 0x7fbf0419b640 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.714+0000 7fbf0bc4d640 1 --2- 172.21.15.3:0/1189632991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf041029b0 0x7fbf0419b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.714+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/1189632991 >> 172.21.15.3:0/1189632991 conn(0x7fbf0407a740 msgr2=0x7fbf041062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.714+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/1189632991 shutdown_connections 2024-09-21T06:16:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:11.714+0000 7fbf0bc4d640 1 -- 172.21.15.3:0/1189632991 wait complete. 2024-09-21T06:16:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.755+0000 7f507066d640 1 -- 172.21.15.3:0/129646898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5068101f70 msgr2=0x7f5068102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.755+0000 7f507066d640 1 --2- 172.21.15.3:0/129646898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5068101f70 0x7f5068102380 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5064009900 tx=0x7f506402a410 comp rx=0 tx=0).stop 2024-09-21T06:16:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.755+0000 7f507066d640 1 -- 172.21.15.3:0/129646898 shutdown_connections 2024-09-21T06:16:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.755+0000 7f507066d640 1 --2- 172.21.15.3:0/129646898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5068109e00 0x7f5068110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.755+0000 7f507066d640 1 --2- 172.21.15.3:0/129646898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50681094b0 0x7f50681098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.755+0000 7f507066d640 1 --2- 172.21.15.3:0/129646898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5068101f70 0x7f5068102380 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.755+0000 7f507066d640 1 -- 172.21.15.3:0/129646898 >> 172.21.15.3:0/129646898 conn(0x7f5068077c60 msgr2=0x7f50680780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.756+0000 7f507066d640 1 -- 172.21.15.3:0/129646898 shutdown_connections 2024-09-21T06:16:11.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.756+0000 7f507066d640 1 -- 172.21.15.3:0/129646898 wait complete. 2024-09-21T06:16:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.757+0000 7f507066d640 1 Processor -- start 2024-09-21T06:16:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.757+0000 7f507066d640 1 -- start start 2024-09-21T06:16:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.757+0000 7f507066d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5068101f70 0x7f5068104920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.757+0000 7f507066d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50681094b0 0x7f5068102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.757+0000 7f507066d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5068109e00 0x7f5068103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.757+0000 7f507066d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5068113b10 con 0x7f5068101f70 2024-09-21T06:16:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.757+0000 7f507066d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5068113990 con 0x7f5068109e00 2024-09-21T06:16:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.757+0000 7f507066d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5068113c90 con 0x7f50681094b0 2024-09-21T06:16:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.758+0000 7f506dc08640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50681094b0 0x7f5068102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.758+0000 7f506dc08640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50681094b0 0x7f5068102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:48274/0 (socket says 172.21.15.3:48274) 2024-09-21T06:16:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.758+0000 7f506dc08640 1 -- 172.21.15.3:0/1194864422 learned_addr learned my addr 172.21.15.3:0/1194864422 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.758+0000 7f506ec0a640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5068109e00 0x7f5068103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.758+0000 7f506e409640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5068101f70 0x7f5068104920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.758+0000 7f506dc08640 1 -- 172.21.15.3:0/1194864422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5068109e00 msgr2=0x7f5068103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.758+0000 7f506dc08640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5068109e00 0x7f5068103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.758+0000 7f506dc08640 1 -- 172.21.15.3:0/1194864422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5068101f70 msgr2=0x7f5068104920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.758+0000 7f506dc08640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5068101f70 0x7f5068104920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.758+0000 7f506dc08640 1 -- 172.21.15.3:0/1194864422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5068103c70 con 0x7f50681094b0 2024-09-21T06:16:11.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.758+0000 7f506e409640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5068101f70 0x7f5068104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:11.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.758+0000 7f506dc08640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50681094b0 0x7f5068102f10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f505800e9f0 tx=0x7f505800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.759+0000 7f505f7fe640 1 -- 172.21.15.3:0/1194864422 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505800ccc0 con 0x7f50681094b0 2024-09-21T06:16:11.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.759+0000 7f505f7fe640 1 -- 172.21.15.3:0/1194864422 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50580108e0 con 0x7f50681094b0 2024-09-21T06:16:11.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.759+0000 7f505f7fe640 1 -- 172.21.15.3:0/1194864422 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058018860 con 0x7f50681094b0 2024-09-21T06:16:11.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.759+0000 7f507066d640 1 -- 172.21.15.3:0/1194864422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50681ace20 con 0x7f50681094b0 2024-09-21T06:16:11.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.759+0000 7f507066d640 1 -- 172.21.15.3:0/1194864422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50681ad1c0 con 0x7f50681094b0 2024-09-21T06:16:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.763+0000 7f5076be1740 1 Processor -- start 2024-09-21T06:16:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.763+0000 7f5076be1740 1 -- start start 2024-09-21T06:16:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.763+0000 7f505f7fe640 1 -- 172.21.15.3:0/1194864422 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7f5058010430 con 0x7f50681094b0 2024-09-21T06:16:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.763+0000 7f5076be1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706be070 0x561370a656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.764+0000 7f506ec0a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706be070 0x561370a656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.764+0000 7f5076be1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613706951b0 0x561370a65be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.764+0000 7f506ec0a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706be070 0x561370a656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41870/0 (socket says 172.21.15.3:41870) 2024-09-21T06:16:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.764+0000 7f506ec0a640 1 -- 172.21.15.3:0/2691861882 learned_addr learned my addr 172.21.15.3:0/2691861882 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.764+0000 7f5076be1740 1 --2- 172.21.15.3:0/2691861882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137069d390 0x561370a66120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.764+0000 7f5076be1740 1 -- 172.21.15.3:0/2691861882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5613709dbea0 con 0x5613706be070 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.764+0000 7f5076be1740 1 -- 172.21.15.3:0/2691861882 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5613709cb2d0 con 0x56137069d390 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.764+0000 7f5076be1740 1 -- 172.21.15.3:0/2691861882 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5613709cf0e0 con 0x5613706951b0 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.764+0000 7f506e409640 1 --2- 172.21.15.3:0/2691861882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613706951b0 0x561370a65be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.764+0000 7f506dc08640 1 --2- 172.21.15.3:0/2691861882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137069d390 0x561370a66120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.764+0000 7f506e409640 1 -- 172.21.15.3:0/2691861882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137069d390 msgr2=0x561370a66120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.764+0000 7f506e409640 1 --2- 172.21.15.3:0/2691861882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137069d390 0x561370a66120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.764+0000 7f506e409640 1 -- 172.21.15.3:0/2691861882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706be070 msgr2=0x561370a656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.765+0000 7f506e409640 1 --2- 172.21.15.3:0/2691861882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706be070 0x561370a656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.765+0000 7f506e409640 1 -- 172.21.15.3:0/2691861882 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561370729650 con 0x5613706951b0 2024-09-21T06:16:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.765+0000 7f506ec0a640 1 --2- 172.21.15.3:0/2691861882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706be070 0x561370a656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.765+0000 7f505f7fe640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50540b6710 0x7f50540b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.765+0000 7f506ec0a640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50540b6710 0x7f50540b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.765+0000 7f506e409640 1 --2- 172.21.15.3:0/2691861882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613706951b0 0x561370a65be0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f506402aa00 tx=0x7f5064004300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6d0c75d2e61423e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.765+0000 7f505f7fe640 1 -- 172.21.15.3:0/1194864422 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f50580e7bb0 con 0x7f50681094b0 2024-09-21T06:16:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.765+0000 7f505cff9640 1 -- 172.21.15.3:0/2691861882 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064035030 con 0x5613706951b0 2024-09-21T06:16:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.766+0000 7f505cff9640 1 -- 172.21.15.3:0/2691861882 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5064005de0 con 0x5613706951b0 2024-09-21T06:16:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.766+0000 7f505cff9640 1 -- 172.21.15.3:0/2691861882 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640035f0 con 0x5613706951b0 2024-09-21T06:16:11.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.767+0000 7f506ec0a640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50540b6710 0x7f50540b8c20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5060009fd0 tx=0x7f5060009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.767+0000 7f5076be1740 1 -- 172.21.15.3:0/2691861882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613706951b0 msgr2=0x561370a65be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.767+0000 7f5076be1740 1 --2- 172.21.15.3:0/2691861882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613706951b0 0x561370a65be0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f506402aa00 tx=0x7f5064004300 comp rx=0 tx=0).stop 2024-09-21T06:16:11.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.767+0000 7f5076be1740 1 -- 172.21.15.3:0/2691861882 shutdown_connections 2024-09-21T06:16:11.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.767+0000 7f5076be1740 1 --2- 172.21.15.3:0/2691861882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613706951b0 0x561370a65be0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.767+0000 7f5076be1740 1 --2- 172.21.15.3:0/2691861882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137069d390 0x561370a66120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.767+0000 7f5076be1740 1 --2- 172.21.15.3:0/2691861882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706be070 0x561370a656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.767+0000 7f5076be1740 1 -- 172.21.15.3:0/2691861882 >> 172.21.15.3:0/2691861882 conn(0x5613706b2460 msgr2=0x561370a63590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.767+0000 7f5076be1740 1 -- 172.21.15.3:0/2691861882 shutdown_connections 2024-09-21T06:16:11.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.767+0000 7f5076be1740 1 -- 172.21.15.3:0/2691861882 wait complete. 2024-09-21T06:16:11.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.768+0000 7f5076be1740 1 -- start start 2024-09-21T06:16:11.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.768+0000 7f5076be1740 1 Processor -- start 2024-09-21T06:16:11.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.769+0000 7f5076be1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613706951b0 0x561370a70d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.769+0000 7f5076be1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 0x561370a712d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.769+0000 7f506e409640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613706951b0 0x561370a70d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.769+0000 7f506e409640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613706951b0 0x561370a70d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:48290/0 (socket says 172.21.15.3:48290) 2024-09-21T06:16:11.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.769+0000 7f506dc08640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 0x561370a712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.769+0000 7f506dc08640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 0x561370a712d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41874/0 (socket says 172.21.15.3:41874) 2024-09-21T06:16:11.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.769+0000 7f5076be1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613706be070 0x561370a7d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.769+0000 7f506dc08640 1 -- 172.21.15.3:0/781472011 learned_addr learned my addr 172.21.15.3:0/781472011 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:11.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.769+0000 7f5076be1740 1 -- 172.21.15.3:0/781472011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56137082c700 con 0x56137069d390 2024-09-21T06:16:11.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.769+0000 7f5076be1740 1 -- 172.21.15.3:0/781472011 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561370966160 con 0x5613706be070 2024-09-21T06:16:11.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.769+0000 7f5076be1740 1 -- 172.21.15.3:0/781472011 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561370a73e20 con 0x5613706951b0 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.770+0000 7f506e409640 1 -- 172.21.15.3:0/781472011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613706be070 msgr2=0x561370a7d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.770+0000 7f506e409640 1 --2- 172.21.15.3:0/781472011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613706be070 0x561370a7d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.770+0000 7f506e409640 1 -- 172.21.15.3:0/781472011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 msgr2=0x561370a712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.770+0000 7f506e409640 1 --2- 172.21.15.3:0/781472011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 0x561370a712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.770+0000 7f506e409640 1 -- 172.21.15.3:0/781472011 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561370993c90 con 0x5613706951b0 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.770+0000 7f506dc08640 1 --2- 172.21.15.3:0/781472011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 0x561370a712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.770+0000 7f5076be1740 10 client.0 _ll_register_callbacks cb 0x56137097f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.770+0000 7f506e409640 1 --2- 172.21.15.3:0/781472011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613706951b0 0x561370a70d90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5064000c00 tx=0x7f5064004f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b35410b1dae0d63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.770+0000 7f5051ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.770+0000 7f5076be1740 1 -- 172.21.15.3:0/781472011 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5613709dddf0 con 0x5613706951b0 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.770+0000 7f5051ffb640 1 -- 172.21.15.3:0/781472011 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064035040 con 0x5613706951b0 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.770+0000 7f5051ffb640 1 -- 172.21.15.3:0/781472011 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5064005500 con 0x5613706951b0 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.771+0000 7f5051ffb640 1 -- 172.21.15.3:0/781472011 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50640314a0 con 0x5613706951b0 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.771+0000 7f5076be1740 10 client.25435 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.771+0000 7f5076be1740 1 -- 172.21.15.3:0/781472011 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=28}) -- 0x561370a7f840 con 0x5613706951b0 2024-09-21T06:16:11.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.771+0000 7f5051ffb640 1 -- 172.21.15.3:0/781472011 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5064031640 con 0x5613706951b0 2024-09-21T06:16:11.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.771+0000 7f5076be1740 10 client.25435 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.771+0000 7f5076be1740 10 client.25435 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:16:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.771+0000 7f5076be1740 20 client.25435 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.771+0000 7f5076be1740 1 --2- 172.21.15.3:0/781472011 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56137067c9f0 0x561370a82ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.771+0000 7f5076be1740 4 client.25435 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.771+0000 7f5076be1740 1 -- 172.21.15.3:0/781472011 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613709dddf0 con 0x56137067c9f0 2024-09-21T06:16:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.772+0000 7f506dc08640 1 --2- 172.21.15.3:0/781472011 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56137067c9f0 0x561370a82ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.772+0000 7f5051ffb640 10 client.25435 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:16:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.772+0000 7f506dc08640 1 --2- 172.21.15.3:0/781472011 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56137067c9f0 0x561370a82ba0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ecb9999b6744be68 server_cookie=62e01e14442e9b72 in_seq=0 out_seq=0 2024-09-21T06:16:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.773+0000 7f5051ffb640 1 -- 172.21.15.3:0/781472011 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5613709dddf0 con 0x56137067c9f0 2024-09-21T06:16:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.773+0000 7f5051ffb640 10 client.25435 handle_command_reply: tid=0 2024-09-21T06:16:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.773+0000 7f5076be1740 1 client.25435 shutdown 2024-09-21T06:16:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.773+0000 7f5076be1740 1 -- 172.21.15.3:0/781472011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613706951b0 msgr2=0x561370a70d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.773+0000 7f5076be1740 1 --2- 172.21.15.3:0/781472011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613706951b0 0x561370a70d90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5064000c00 tx=0x7f5064004f00 comp rx=0 tx=0).stop 2024-09-21T06:16:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.773+0000 7f5076be1740 1 -- 172.21.15.3:0/781472011 shutdown_connections 2024-09-21T06:16:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.773+0000 7f5076be1740 1 --2- 172.21.15.3:0/781472011 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56137067c9f0 0x561370a82ba0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.773+0000 7f5076be1740 1 --2- 172.21.15.3:0/781472011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613706be070 0x561370a7d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.773+0000 7f5076be1740 1 --2- 172.21.15.3:0/781472011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 0x561370a712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.773+0000 7f5051ffb640 0 client.25435 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:16:11.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.773+0000 7f5076be1740 1 --2- 172.21.15.3:0/781472011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613706951b0 0x561370a70d90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.773+0000 7f5076be1740 1 -- 172.21.15.3:0/781472011 >> 172.21.15.3:0/781472011 conn(0x5613706b2460 msgr2=0x561370a68880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.773+0000 7f5076be1740 1 -- 172.21.15.3:0/781472011 shutdown_connections 2024-09-21T06:16:11.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.774+0000 7f5076be1740 1 -- 172.21.15.3:0/781472011 wait complete. 2024-09-21T06:16:11.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.774+0000 7f5076be1740 20 client.25435 trim_cache size 0 max 16384 2024-09-21T06:16:11.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.800+0000 7f5076be1740 1 Processor -- start 2024-09-21T06:16:11.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.800+0000 7f5076be1740 1 -- start start 2024-09-21T06:16:11.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.801+0000 7f5076be1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137067c9f0 0x561370a687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.801+0000 7f5076be1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706951b0 0x561370a68ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.801+0000 7f506dc08640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706951b0 0x561370a68ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.801+0000 7f506dc08640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706951b0 0x561370a68ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41888/0 (socket says 172.21.15.3:41888) 2024-09-21T06:16:11.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.801+0000 7f5076be1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137069d390 0x561370a6df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.801+0000 7f506dc08640 1 -- 172.21.15.3:0/1841698420 learned_addr learned my addr 172.21.15.3:0/1841698420 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.801+0000 7f5076be1740 1 -- 172.21.15.3:0/1841698420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f50681b5b40 con 0x5613706951b0 2024-09-21T06:16:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.801+0000 7f5076be1740 1 -- 172.21.15.3:0/1841698420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f50681b7680 con 0x56137069d390 2024-09-21T06:16:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.801+0000 7f5076be1740 1 -- 172.21.15.3:0/1841698420 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50681b6210 con 0x56137067c9f0 2024-09-21T06:16:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.801+0000 7f506e409640 1 --2- 172.21.15.3:0/1841698420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137067c9f0 0x561370a687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.801+0000 7f506ec0a640 1 --2- 172.21.15.3:0/1841698420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137069d390 0x561370a6df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.802+0000 7f506e409640 1 -- 172.21.15.3:0/1841698420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137069d390 msgr2=0x561370a6df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.802+0000 7f506e409640 1 --2- 172.21.15.3:0/1841698420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137069d390 0x561370a6df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.802+0000 7f506e409640 1 -- 172.21.15.3:0/1841698420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706951b0 msgr2=0x561370a68ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.802+0000 7f506e409640 1 --2- 172.21.15.3:0/1841698420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706951b0 0x561370a68ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.802+0000 7f506e409640 1 -- 172.21.15.3:0/1841698420 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561370a72050 con 0x56137067c9f0 2024-09-21T06:16:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.802+0000 7f506dc08640 1 --2- 172.21.15.3:0/1841698420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706951b0 0x561370a68ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.802+0000 7f506e409640 1 --2- 172.21.15.3:0/1841698420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137067c9f0 0x561370a687a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f506402a930 tx=0x7f5064044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=773c24973e1ddcd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.802+0000 7f5053fff640 1 -- 172.21.15.3:0/1841698420 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064035070 con 0x56137067c9f0 2024-09-21T06:16:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.802+0000 7f5053fff640 1 -- 172.21.15.3:0/1841698420 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5064031d40 con 0x56137067c9f0 2024-09-21T06:16:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.802+0000 7f5053fff640 1 -- 172.21.15.3:0/1841698420 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506403bd00 con 0x56137067c9f0 2024-09-21T06:16:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.804+0000 7f5076be1740 1 -- 172.21.15.3:0/1841698420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137067c9f0 msgr2=0x561370a687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.804+0000 7f5076be1740 1 --2- 172.21.15.3:0/1841698420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137067c9f0 0x561370a687a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f506402a930 tx=0x7f5064044000 comp rx=0 tx=0).stop 2024-09-21T06:16:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.804+0000 7f5076be1740 1 -- 172.21.15.3:0/1841698420 shutdown_connections 2024-09-21T06:16:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.804+0000 7f5076be1740 1 --2- 172.21.15.3:0/1841698420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137069d390 0x561370a6df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.804+0000 7f5076be1740 1 --2- 172.21.15.3:0/1841698420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613706951b0 0x561370a68ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.804+0000 7f5076be1740 1 --2- 172.21.15.3:0/1841698420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137067c9f0 0x561370a687a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.804+0000 7f5076be1740 1 -- 172.21.15.3:0/1841698420 >> 172.21.15.3:0/1841698420 conn(0x5613706b2460 msgr2=0x561370a6f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.804+0000 7f5076be1740 1 -- 172.21.15.3:0/1841698420 shutdown_connections 2024-09-21T06:16:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.804+0000 7f5076be1740 1 -- 172.21.15.3:0/1841698420 wait complete. 2024-09-21T06:16:11.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.804+0000 7f5076be1740 1 -- start start 2024-09-21T06:16:11.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.804+0000 7f5076be1740 1 Processor -- start 2024-09-21T06:16:11.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.805+0000 7f5076be1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137067c9f0 0x561370a753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.805+0000 7f5076be1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613706951b0 0x561370a75900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.805+0000 7f5076be1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 0x561370aec3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.805+0000 7f506e409640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137067c9f0 0x561370a753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.805+0000 7f506dc08640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613706951b0 0x561370a75900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.805+0000 7f506e409640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137067c9f0 0x561370a753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:48304/0 (socket says 172.21.15.3:48304) 2024-09-21T06:16:11.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.805+0000 7f506dc08640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613706951b0 0x561370a75900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:44272/0 (socket says 172.21.15.3:44272) 2024-09-21T06:16:11.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.805+0000 7f506ec0a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 0x561370aec3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.805+0000 7f506ec0a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 0x561370aec3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41904/0 (socket says 172.21.15.3:41904) 2024-09-21T06:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.805+0000 7f5076be1740 1 -- 172.21.15.3:0/3642355163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f50681b6fb0 con 0x56137069d390 2024-09-21T06:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.806+0000 7f506e409640 1 -- 172.21.15.3:0/3642355163 learned_addr learned my addr 172.21.15.3:0/3642355163 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.806+0000 7f5076be1740 1 -- 172.21.15.3:0/3642355163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f50681b68e0 con 0x5613706951b0 2024-09-21T06:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.806+0000 7f5076be1740 1 -- 172.21.15.3:0/3642355163 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50681b3b20 con 0x56137067c9f0 2024-09-21T06:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.806+0000 7f5076be1740 10 client.0 _ll_register_callbacks cb 0x7f5060009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.806+0000 7f506e409640 1 -- 172.21.15.3:0/3642355163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613706951b0 msgr2=0x561370a75900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.806+0000 7f506e409640 1 --2- 172.21.15.3:0/3642355163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613706951b0 0x561370a75900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.806+0000 7f506e409640 1 -- 172.21.15.3:0/3642355163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 msgr2=0x561370aec3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.806+0000 7f506e409640 1 --2- 172.21.15.3:0/3642355163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 0x561370aec3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.806+0000 7f506e409640 1 -- 172.21.15.3:0/3642355163 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561370a761d0 con 0x56137067c9f0 2024-09-21T06:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.806+0000 7f506ec0a640 1 --2- 172.21.15.3:0/3642355163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 0x561370aec3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.806+0000 7f5076be1740 1 -- 172.21.15.3:0/3642355163 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5064031d40 con 0x56137067c9f0 2024-09-21T06:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.806+0000 7f506e409640 1 --2- 172.21.15.3:0/3642355163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137067c9f0 0x561370a753c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5064031440 tx=0x7f5064004300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75a97e48be9f9d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.806+0000 7f5051ffb640 10 client.25441 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:16:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.807+0000 7f5051ffb640 1 -- 172.21.15.3:0/3642355163 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064035040 con 0x56137067c9f0 2024-09-21T06:16:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.807+0000 7f5051ffb640 1 -- 172.21.15.3:0/3642355163 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f506403bb90 con 0x56137067c9f0 2024-09-21T06:16:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.807+0000 7f5051ffb640 1 -- 172.21.15.3:0/3642355163 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f506403bdd0 con 0x56137067c9f0 2024-09-21T06:16:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.807+0000 7f5076be1740 10 client.25441 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.807+0000 7f5076be1740 1 -- 172.21.15.3:0/3642355163 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=28}) -- 0x561370a761d0 con 0x56137067c9f0 2024-09-21T06:16:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.807+0000 7f5051ffb640 1 -- 172.21.15.3:0/3642355163 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f506406b030 con 0x56137067c9f0 2024-09-21T06:16:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.808+0000 7f5076be1740 10 client.25441 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.808+0000 7f5076be1740 10 client.25441 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:16:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.808+0000 7f5076be1740 20 client.25441 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.808+0000 7f5076be1740 1 --2- 172.21.15.3:0/3642355163 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5613706be070 0x561370aefa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.808+0000 7f5076be1740 4 client.25441 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.808+0000 7f5076be1740 1 -- 172.21.15.3:0/3642355163 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613709d98a0 con 0x5613706be070 2024-09-21T06:16:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.808+0000 7f506dc08640 1 --2- 172.21.15.3:0/3642355163 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5613706be070 0x561370aefa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.809+0000 7f5051ffb640 10 client.25441 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:16:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.809+0000 7f506dc08640 1 --2- 172.21.15.3:0/3642355163 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5613706be070 0x561370aefa90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1269371da99e5b90 server_cookie=1c4d5474c092a781 in_seq=0 out_seq=0 2024-09-21T06:16:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.809+0000 7f5051ffb640 1 -- 172.21.15.3:0/3642355163 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613709d98a0 con 0x5613706be070 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.809+0000 7f5051ffb640 10 client.25441 handle_command_reply: tid=0 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.809+0000 7f5076be1740 1 client.25441 shutdown 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.809+0000 7f5076be1740 1 -- 172.21.15.3:0/3642355163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137067c9f0 msgr2=0x561370a753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.809+0000 7f5076be1740 1 --2- 172.21.15.3:0/3642355163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137067c9f0 0x561370a753c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5064031440 tx=0x7f5064004300 comp rx=0 tx=0).stop 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.809+0000 7f5076be1740 1 -- 172.21.15.3:0/3642355163 shutdown_connections 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.809+0000 7f5076be1740 1 --2- 172.21.15.3:0/3642355163 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5613706be070 0x561370aefa90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.809+0000 7f5076be1740 1 --2- 172.21.15.3:0/3642355163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56137069d390 0x561370aec3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.809+0000 7f5076be1740 1 --2- 172.21.15.3:0/3642355163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613706951b0 0x561370a75900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.809+0000 7f5076be1740 1 --2- 172.21.15.3:0/3642355163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137067c9f0 0x561370a753c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.809+0000 7f5076be1740 1 -- 172.21.15.3:0/3642355163 >> 172.21.15.3:0/3642355163 conn(0x5613706b2460 msgr2=0x561370a687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.810+0000 7f5051ffb640 0 client.25441 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.810+0000 7f5076be1740 1 -- 172.21.15.3:0/3642355163 shutdown_connections 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.810+0000 7f5076be1740 1 -- 172.21.15.3:0/3642355163 wait complete. 2024-09-21T06:16:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.810+0000 7f5076be1740 20 client.25441 trim_cache size 0 max 16384 2024-09-21T06:16:11.814 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:16:11.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.811+0000 7f507066d640 1 -- 172.21.15.3:0/1194864422 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50540b6710 msgr2=0x7f50540b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.811+0000 7f507066d640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50540b6710 0x7f50540b8c20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5060009fd0 tx=0x7f5060009500 comp rx=0 tx=0).stop 2024-09-21T06:16:11.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.811+0000 7f507066d640 1 -- 172.21.15.3:0/1194864422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50681094b0 msgr2=0x7f5068102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.811+0000 7f507066d640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50681094b0 0x7f5068102f10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f505800e9f0 tx=0x7f505800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.811+0000 7f507066d640 1 -- 172.21.15.3:0/1194864422 shutdown_connections 2024-09-21T06:16:11.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.811+0000 7f507066d640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f50540b6710 0x7f50540b8c20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.811+0000 7f507066d640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50681094b0 0x7f5068102f10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.811+0000 7f507066d640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5068109e00 0x7f5068103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.811+0000 7f507066d640 1 --2- 172.21.15.3:0/1194864422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5068101f70 0x7f5068104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.811+0000 7f507066d640 1 -- 172.21.15.3:0/1194864422 >> 172.21.15.3:0/1194864422 conn(0x7f5068077c60 msgr2=0x7f50681074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.811+0000 7f507066d640 1 -- 172.21.15.3:0/1194864422 shutdown_connections 2024-09-21T06:16:11.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:11.811+0000 7f507066d640 1 -- 172.21.15.3:0/1194864422 wait complete. 2024-09-21T06:16:12.664 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:16:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:13.085 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:12 smithi003 ceph-mon[31227]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-21T06:16:13.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:12 smithi137 ceph-mon[34464]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-21T06:16:13.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:12 smithi204 ceph-mon[34659]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-21T06:16:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.543+0000 7f27ffd44640 1 -- 172.21.15.3:0/769706549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27f81083e0 msgr2=0x7f27f81087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.543+0000 7f27ffd44640 1 --2- 172.21.15.3:0/769706549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27f81083e0 0x7f27f81087f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009900 tx=0x7f27e802a410 comp rx=0 tx=0).stop 2024-09-21T06:16:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.543+0000 7f27ffd44640 1 -- 172.21.15.3:0/769706549 shutdown_connections 2024-09-21T06:16:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.543+0000 7f27ffd44640 1 --2- 172.21.15.3:0/769706549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27f8108d30 0x7f27f810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.543+0000 7f27ffd44640 1 --2- 172.21.15.3:0/769706549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27f81083e0 0x7f27f81087f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.543+0000 7f27ffd44640 1 --2- 172.21.15.3:0/769706549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f8102aa0 0x7f27f8102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.543+0000 7f27ffd44640 1 -- 172.21.15.3:0/769706549 >> 172.21.15.3:0/769706549 conn(0x7f27f80797e0 msgr2=0x7f27f8079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.543+0000 7f27ffd44640 1 -- 172.21.15.3:0/769706549 shutdown_connections 2024-09-21T06:16:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.543+0000 7f27ffd44640 1 -- 172.21.15.3:0/769706549 wait complete. 2024-09-21T06:16:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.545+0000 7f27ffd44640 1 Processor -- start 2024-09-21T06:16:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.545+0000 7f27ffd44640 1 -- start start 2024-09-21T06:16:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.545+0000 7f27ffd44640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f8102aa0 0x7f27f819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.545+0000 7f27ffd44640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27f8108d30 0x7f27f819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.546+0000 7f27fd2df640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27f8108d30 0x7f27f819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.546+0000 7f27fdae0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f8102aa0 0x7f27f819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.546+0000 7f27fdae0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f8102aa0 0x7f27f819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:48310/0 (socket says 172.21.15.3:48310) 2024-09-21T06:16:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.546+0000 7f27fdae0640 1 -- 172.21.15.3:0/1473079655 learned_addr learned my addr 172.21.15.3:0/1473079655 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.546+0000 7f27ffd44640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27f819fdc0 0x7f27f81a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.546+0000 7f27ffd44640 1 -- 172.21.15.3:0/1473079655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8113a90 con 0x7f27f8108d30 2024-09-21T06:16:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.546+0000 7f27ffd44640 1 -- 172.21.15.3:0/1473079655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8113910 con 0x7f27f819fdc0 2024-09-21T06:16:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.546+0000 7f27ffd44640 1 -- 172.21.15.3:0/1473079655 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8113c10 con 0x7f27f8102aa0 2024-09-21T06:16:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.546+0000 7f27fe2e1640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27f819fdc0 0x7f27f81a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.547+0000 7f27fd2df640 1 -- 172.21.15.3:0/1473079655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f8102aa0 msgr2=0x7f27f819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.547+0000 7f27fd2df640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f8102aa0 0x7f27f819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.547+0000 7f27fd2df640 1 -- 172.21.15.3:0/1473079655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27f819fdc0 msgr2=0x7f27f81a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.547+0000 7f27fd2df640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27f819fdc0 0x7f27f81a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.547+0000 7f27fd2df640 1 -- 172.21.15.3:0/1473079655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f81a0910 con 0x7f27f8108d30 2024-09-21T06:16:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.547+0000 7f27fdae0640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f8102aa0 0x7f27f819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.547+0000 7f27fd2df640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27f8108d30 0x7f27f819abe0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f27f8103aa0 tx=0x7f27e80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.547+0000 7f27eeffd640 1 -- 172.21.15.3:0/1473079655 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e803f070 con 0x7f27f8108d30 2024-09-21T06:16:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.547+0000 7f27eeffd640 1 -- 172.21.15.3:0/1473079655 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27e80044c0 con 0x7f27f8108d30 2024-09-21T06:16:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.547+0000 7f27eeffd640 1 -- 172.21.15.3:0/1473079655 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8035070 con 0x7f27f8108d30 2024-09-21T06:16:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.548+0000 7f27ffd44640 1 -- 172.21.15.3:0/1473079655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f81a0ba0 con 0x7f27f8108d30 2024-09-21T06:16:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.550+0000 7f27ffd44640 1 -- 172.21.15.3:0/1473079655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f8110330 con 0x7f27f8108d30 2024-09-21T06:16:13.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.553+0000 7f27eeffd640 1 -- 172.21.15.3:0/1473079655 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7f27e80037f0 con 0x7f27f8108d30 2024-09-21T06:16:13.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.553+0000 7f28062b8740 1 Processor -- start 2024-09-21T06:16:13.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.554+0000 7f28062b8740 1 -- start start 2024-09-21T06:16:13.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.554+0000 7f28062b8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a84070 0x55b1a3e33eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.554+0000 7f28062b8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.554+0000 7f28062b8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a63390 0x55b1a3e34930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.554+0000 7f28062b8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b1a3da1ea0 con 0x55b1a3a63390 2024-09-21T06:16:13.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.554+0000 7f28062b8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b1a3d912d0 con 0x55b1a3a84070 2024-09-21T06:16:13.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.554+0000 7f28062b8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b1a3d950e0 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.554+0000 7f27fd2df640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a63390 0x55b1a3e34930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.554+0000 7f27fdae0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.554+0000 7f27fe2e1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a84070 0x55b1a3e33eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.554+0000 7f27fe2e1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a84070 0x55b1a3e33eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:44284/0 (socket says 172.21.15.3:44284) 2024-09-21T06:16:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.554+0000 7f27fe2e1640 1 -- 172.21.15.3:0/3891493322 learned_addr learned my addr 172.21.15.3:0/3891493322 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.554+0000 7f27fdae0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e343f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:48324/0 (socket says 172.21.15.3:48324) 2024-09-21T06:16:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.555+0000 7f27fe2e1640 1 -- 172.21.15.3:0/3891493322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a5b1b0 msgr2=0x55b1a3e343f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.555+0000 7f27fe2e1640 1 --2- 172.21.15.3:0/3891493322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e343f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.555+0000 7f27fe2e1640 1 -- 172.21.15.3:0/3891493322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a63390 msgr2=0x55b1a3e34930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.555+0000 7f27fe2e1640 1 --2- 172.21.15.3:0/3891493322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a63390 0x55b1a3e34930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.555+0000 7f27fe2e1640 1 -- 172.21.15.3:0/3891493322 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a3aef650 con 0x55b1a3a84070 2024-09-21T06:16:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.555+0000 7f27fe2e1640 1 --2- 172.21.15.3:0/3891493322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a84070 0x55b1a3e33eb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f27f0007f40 tx=0x7f27f000e7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8ff7c57967d7255 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:13.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.555+0000 7f27e3fff640 1 -- 172.21.15.3:0/3891493322 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0019630 con 0x55b1a3a84070 2024-09-21T06:16:13.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.556+0000 7f27e3fff640 1 -- 172.21.15.3:0/3891493322 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f00108e0 con 0x55b1a3a84070 2024-09-21T06:16:13.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.556+0000 7f27e3fff640 1 -- 172.21.15.3:0/3891493322 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f00188b0 con 0x55b1a3a84070 2024-09-21T06:16:13.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.557+0000 7f27eeffd640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f27e40b6790 0x7f27e40b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.557+0000 7f28062b8740 1 -- 172.21.15.3:0/3891493322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a84070 msgr2=0x55b1a3e33eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.557+0000 7f28062b8740 1 --2- 172.21.15.3:0/3891493322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a84070 0x55b1a3e33eb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f27f0007f40 tx=0x7f27f000e7e0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.557+0000 7f28062b8740 1 -- 172.21.15.3:0/3891493322 shutdown_connections 2024-09-21T06:16:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.557+0000 7f28062b8740 1 --2- 172.21.15.3:0/3891493322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e343f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.557+0000 7f28062b8740 1 --2- 172.21.15.3:0/3891493322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a63390 0x55b1a3e34930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.557+0000 7f28062b8740 1 --2- 172.21.15.3:0/3891493322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a84070 0x55b1a3e33eb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.557+0000 7f28062b8740 1 -- 172.21.15.3:0/3891493322 >> 172.21.15.3:0/3891493322 conn(0x55b1a3a78460 msgr2=0x55b1a3e31da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.557+0000 7f27fdae0640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f27e40b6790 0x7f27e40b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.557+0000 7f28062b8740 1 -- 172.21.15.3:0/3891493322 shutdown_connections 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.557+0000 7f28062b8740 1 -- 172.21.15.3:0/3891493322 wait complete. 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.557+0000 7f27eeffd640 1 -- 172.21.15.3:0/1473079655 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f27e8107ea0 con 0x7f27f8108d30 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.558+0000 7f28062b8740 1 -- start start 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.558+0000 7f27fdae0640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f27e40b6790 0x7f27e40b8ca0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55b1a3e32b30 tx=0x7f27f4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.558+0000 7f28062b8740 1 Processor -- start 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.558+0000 7f28062b8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e3f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.558+0000 7f28062b8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a63390 0x55b1a3e3fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.558+0000 7f28062b8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a84070 0x55b1a3e2fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f28062b8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b1a3bf2700 con 0x55b1a3a84070 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f28062b8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b1a3d2c160 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f28062b8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b1a3e42690 con 0x55b1a3a63390 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f27fe2e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a84070 0x55b1a3e2fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f27fd2df640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a63390 0x55b1a3e3fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f27fe2e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a84070 0x55b1a3e2fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41936/0 (socket says 172.21.15.3:41936) 2024-09-21T06:16:13.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f27fe2e1640 1 -- 172.21.15.3:0/215592177 learned_addr learned my addr 172.21.15.3:0/215592177 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:13.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f27fdae0640 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e3f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f27fd2df640 1 -- 172.21.15.3:0/215592177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a5b1b0 msgr2=0x55b1a3e3f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:13.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f27fd2df640 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e3f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f27fd2df640 1 -- 172.21.15.3:0/215592177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a84070 msgr2=0x55b1a3e2fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f27fd2df640 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a84070 0x55b1a3e2fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f27fd2df640 1 -- 172.21.15.3:0/215592177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a3d59c90 con 0x55b1a3a63390 2024-09-21T06:16:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f27fdae0640 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e3f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:16:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f27fe2e1640 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a84070 0x55b1a3e2fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.559+0000 7f28062b8740 10 client.0 _ll_register_callbacks cb 0x55b1a3d459e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.560+0000 7f28062b8740 1 -- 172.21.15.3:0/215592177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27f00108e0 con 0x55b1a3a63390 2024-09-21T06:16:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.560+0000 7f27fd2df640 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a63390 0x55b1a3e3fac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f27e8105570 tx=0x7f27e80057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22ee3149fc13c485 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.560+0000 7f27e17fa640 10 client.25453 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:16:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.560+0000 7f27e17fa640 1 -- 172.21.15.3:0/215592177 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e80ca210 con 0x55b1a3a63390 2024-09-21T06:16:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.560+0000 7f27e17fa640 1 -- 172.21.15.3:0/215592177 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f27e80efbd0 con 0x55b1a3a63390 2024-09-21T06:16:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.560+0000 7f27e17fa640 1 -- 172.21.15.3:0/215592177 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27e80f1280 con 0x55b1a3a63390 2024-09-21T06:16:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.560+0000 7f28062b8740 10 client.25453 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.560+0000 7f28062b8740 1 -- 172.21.15.3:0/215592177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55b1a3e56ac0 con 0x55b1a3a63390 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.560+0000 7f27e17fa640 1 -- 172.21.15.3:0/215592177 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27e80d7150 con 0x55b1a3a63390 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.561+0000 7f28062b8740 10 client.25453 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.561+0000 7f28062b8740 10 client.25453 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.561+0000 7f28062b8740 20 client.25453 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.561+0000 7f28062b8740 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b1a3a429f0 0x55b1a3e59ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.561+0000 7f28062b8740 4 client.25453 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.561+0000 7f28062b8740 1 -- 172.21.15.3:0/215592177 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1a3da3df0 con 0x55b1a3a429f0 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.561+0000 7f27fdae0640 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b1a3a429f0 0x55b1a3e59ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.562+0000 7f27fdae0640 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b1a3a429f0 0x55b1a3e59ed0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a51b02198be45e6b server_cookie=3d2cb5de8ffe7183 in_seq=0 out_seq=0 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.562+0000 7f27e17fa640 10 client.25453 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.562+0000 7f27e17fa640 1 -- 172.21.15.3:0/215592177 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b1a3da3df0 con 0x55b1a3a429f0 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.562+0000 7f27e17fa640 10 client.25453 handle_command_reply: tid=0 2024-09-21T06:16:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.562+0000 7f28062b8740 1 client.25453 shutdown 2024-09-21T06:16:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.562+0000 7f28062b8740 1 -- 172.21.15.3:0/215592177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a63390 msgr2=0x55b1a3e3fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.562+0000 7f28062b8740 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a63390 0x55b1a3e3fac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f27e8105570 tx=0x7f27e80057d0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.563+0000 7f28062b8740 1 -- 172.21.15.3:0/215592177 shutdown_connections 2024-09-21T06:16:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.563+0000 7f28062b8740 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b1a3a429f0 0x55b1a3e59ed0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.563+0000 7f27e17fa640 0 client.25453 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:16:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.563+0000 7f28062b8740 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a63390 0x55b1a3e3fac0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.563+0000 7f28062b8740 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a84070 0x55b1a3e2fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.563+0000 7f28062b8740 1 --2- 172.21.15.3:0/215592177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e3f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.563+0000 7f28062b8740 1 -- 172.21.15.3:0/215592177 >> 172.21.15.3:0/215592177 conn(0x55b1a3a78460 msgr2=0x55b1a3e37090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.563+0000 7f28062b8740 1 -- 172.21.15.3:0/215592177 shutdown_connections 2024-09-21T06:16:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.563+0000 7f28062b8740 1 -- 172.21.15.3:0/215592177 wait complete. 2024-09-21T06:16:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.563+0000 7f28062b8740 20 client.25453 trim_cache size 0 max 16384 2024-09-21T06:16:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.589+0000 7f28062b8740 1 Processor -- start 2024-09-21T06:16:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.589+0000 7f28062b8740 1 -- start start 2024-09-21T06:16:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.590+0000 7f28062b8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a429f0 0x55b1a3e355f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.590+0000 7f28062b8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e3c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.590+0000 7f28062b8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a63390 0x55b1a3e3ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.590+0000 7f28062b8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8073810 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.590+0000 7f27fd2df640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e3c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.590+0000 7f28062b8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8075350 con 0x55b1a3a63390 2024-09-21T06:16:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.590+0000 7f28062b8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8073ee0 con 0x55b1a3a429f0 2024-09-21T06:16:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.590+0000 7f27fd2df640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e3c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41944/0 (socket says 172.21.15.3:41944) 2024-09-21T06:16:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.590+0000 7f27fd2df640 1 -- 172.21.15.3:0/1036012544 learned_addr learned my addr 172.21.15.3:0/1036012544 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.590+0000 7f27fdae0640 1 --2- 172.21.15.3:0/1036012544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a429f0 0x55b1a3e355f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.590+0000 7f27fe2e1640 1 --2- 172.21.15.3:0/1036012544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a63390 0x55b1a3e3ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.591+0000 7f27fd2df640 1 -- 172.21.15.3:0/1036012544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a429f0 msgr2=0x55b1a3e355f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.591+0000 7f27fd2df640 1 --2- 172.21.15.3:0/1036012544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a429f0 0x55b1a3e355f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.591+0000 7f27fd2df640 1 -- 172.21.15.3:0/1036012544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a63390 msgr2=0x55b1a3e3ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.591+0000 7f27fd2df640 1 --2- 172.21.15.3:0/1036012544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a63390 0x55b1a3e3ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.591+0000 7f27fd2df640 1 -- 172.21.15.3:0/1036012544 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a3e40840 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.591+0000 7f27fd2df640 1 --2- 172.21.15.3:0/1036012544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e3c770 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f27e8000c00 tx=0x7f27e8109000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3785a3896f856d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.592+0000 7f27e37fe640 1 -- 172.21.15.3:0/1036012544 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e80c9330 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.592+0000 7f27e37fe640 1 -- 172.21.15.3:0/1036012544 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f00108e0 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.592+0000 7f27e37fe640 1 -- 172.21.15.3:0/1036012544 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e80cd330 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.593+0000 7f28062b8740 1 -- 172.21.15.3:0/1036012544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a5b1b0 msgr2=0x55b1a3e3c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.593+0000 7f28062b8740 1 --2- 172.21.15.3:0/1036012544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e3c770 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f27e8000c00 tx=0x7f27e8109000 comp rx=0 tx=0).stop 2024-09-21T06:16:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.593+0000 7f28062b8740 1 -- 172.21.15.3:0/1036012544 shutdown_connections 2024-09-21T06:16:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.593+0000 7f28062b8740 1 --2- 172.21.15.3:0/1036012544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a63390 0x55b1a3e3ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.593+0000 7f28062b8740 1 --2- 172.21.15.3:0/1036012544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e3c770 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.593+0000 7f28062b8740 1 --2- 172.21.15.3:0/1036012544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a429f0 0x55b1a3e355f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.593+0000 7f28062b8740 1 -- 172.21.15.3:0/1036012544 >> 172.21.15.3:0/1036012544 conn(0x55b1a3a78460 msgr2=0x55b1a3e34ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.593+0000 7f28062b8740 1 -- 172.21.15.3:0/1036012544 shutdown_connections 2024-09-21T06:16:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.593+0000 7f28062b8740 1 -- 172.21.15.3:0/1036012544 wait complete. 2024-09-21T06:16:13.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.594+0000 7f28062b8740 1 -- start start 2024-09-21T06:16:13.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.594+0000 7f28062b8740 1 Processor -- start 2024-09-21T06:16:13.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.595+0000 7f28062b8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a429f0 0x55b1a3e2aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.595+0000 7f28062b8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e2b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.595+0000 7f28062b8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a63390 0x55b1a3ec38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.595+0000 7f28062b8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8074c80 con 0x55b1a3a63390 2024-09-21T06:16:13.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.595+0000 7f28062b8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f27f80745b0 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.595+0000 7f28062b8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f27f80717f0 con 0x55b1a3a429f0 2024-09-21T06:16:13.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.596+0000 7f27fe2e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a63390 0x55b1a3ec38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.596+0000 7f27fd2df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e2b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.596+0000 7f27fdae0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a429f0 0x55b1a3e2aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.596+0000 7f27fd2df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e2b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:44312/0 (socket says 172.21.15.3:44312) 2024-09-21T06:16:13.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.596+0000 7f27fd2df640 1 -- 172.21.15.3:0/3952229067 learned_addr learned my addr 172.21.15.3:0/3952229067 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:13.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.596+0000 7f28062b8740 10 client.0 _ll_register_callbacks cb 0x7f27e8105400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:13.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.596+0000 7f27fd2df640 1 -- 172.21.15.3:0/3952229067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a429f0 msgr2=0x55b1a3e2aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:13.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.596+0000 7f27fd2df640 1 --2- 172.21.15.3:0/3952229067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a429f0 0x55b1a3e2aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.596+0000 7f27fd2df640 1 -- 172.21.15.3:0/3952229067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a63390 msgr2=0x55b1a3ec38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:13.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.596+0000 7f27fd2df640 1 --2- 172.21.15.3:0/3952229067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a63390 0x55b1a3ec38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.596+0000 7f27fd2df640 1 -- 172.21.15.3:0/3952229067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a3e2bc90 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.596+0000 7f27fe2e1640 1 --2- 172.21.15.3:0/3952229067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a63390 0x55b1a3ec38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:13.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.597+0000 7f28062b8740 1 -- 172.21.15.3:0/3952229067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27f00108e0 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.597+0000 7f27fd2df640 1 --2- 172.21.15.3:0/3952229067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e2b410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f27e80cae30 tx=0x7f27e81081a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751caa85058d19d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:13.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.597+0000 7f27e17fa640 10 client.25469 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:16:13.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.597+0000 7f27e17fa640 1 -- 172.21.15.3:0/3952229067 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e80cd510 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.597+0000 7f27e17fa640 1 -- 172.21.15.3:0/3952229067 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f27e810b2f0 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.597+0000 7f27e17fa640 1 -- 172.21.15.3:0/3952229067 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27e80cd990 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.597+0000 7f28062b8740 10 client.25469 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:13.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.597+0000 7f28062b8740 1 -- 172.21.15.3:0/3952229067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55b1a3e2bc90 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.598+0000 7f27e17fa640 1 -- 172.21.15.3:0/3952229067 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27e80d21c0 con 0x55b1a3a5b1b0 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.598+0000 7f28062b8740 10 client.25469 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.598+0000 7f28062b8740 10 client.25469 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.598+0000 7f28062b8740 20 client.25469 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.598+0000 7f28062b8740 1 --2- 172.21.15.3:0/3952229067 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b1a3a84070 0x55b1a3ec7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.598+0000 7f28062b8740 4 client.25469 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.598+0000 7f28062b8740 1 -- 172.21.15.3:0/3952229067 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1a3d9f8a0 con 0x55b1a3a84070 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.599+0000 7f27fdae0640 1 --2- 172.21.15.3:0/3952229067 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b1a3a84070 0x55b1a3ec7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.599+0000 7f27e17fa640 10 client.25469 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.599+0000 7f27fdae0640 1 --2- 172.21.15.3:0/3952229067 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b1a3a84070 0x55b1a3ec7040 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24a541cb28708e91 server_cookie=1671fbcf50ce5128 in_seq=0 out_seq=0 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.599+0000 7f27e17fa640 1 -- 172.21.15.3:0/3952229067 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1a3d9f8a0 con 0x55b1a3a84070 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.599+0000 7f27e17fa640 10 client.25469 handle_command_reply: tid=0 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.600+0000 7f28062b8740 1 client.25469 shutdown 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.600+0000 7f28062b8740 1 -- 172.21.15.3:0/3952229067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a5b1b0 msgr2=0x55b1a3e2b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.600+0000 7f28062b8740 1 --2- 172.21.15.3:0/3952229067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e2b410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f27e80cae30 tx=0x7f27e81081a0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.600+0000 7f28062b8740 1 -- 172.21.15.3:0/3952229067 shutdown_connections 2024-09-21T06:16:13.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.600+0000 7f28062b8740 1 --2- 172.21.15.3:0/3952229067 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b1a3a84070 0x55b1a3ec7040 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.600+0000 7f28062b8740 1 --2- 172.21.15.3:0/3952229067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1a3a63390 0x55b1a3ec38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.600+0000 7f28062b8740 1 --2- 172.21.15.3:0/3952229067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1a3a5b1b0 0x55b1a3e2b410 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.600+0000 7f28062b8740 1 --2- 172.21.15.3:0/3952229067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1a3a429f0 0x55b1a3e2aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.600+0000 7f28062b8740 1 -- 172.21.15.3:0/3952229067 >> 172.21.15.3:0/3952229067 conn(0x55b1a3a78460 msgr2=0x55b1a3e40a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:13.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.600+0000 7f27e17fa640 0 client.25469 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:16:13.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.600+0000 7f28062b8740 1 -- 172.21.15.3:0/3952229067 shutdown_connections 2024-09-21T06:16:13.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.600+0000 7f28062b8740 1 -- 172.21.15.3:0/3952229067 wait complete. 2024-09-21T06:16:13.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.600+0000 7f28062b8740 20 client.25469 trim_cache size 0 max 16384 2024-09-21T06:16:13.604 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:16:13.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.601+0000 7f27ffd44640 1 -- 172.21.15.3:0/1473079655 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f27e40b6790 msgr2=0x7f27e40b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:13.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.601+0000 7f27ffd44640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f27e40b6790 0x7f27e40b8ca0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55b1a3e32b30 tx=0x7f27f4005f90 comp rx=0 tx=0).stop 2024-09-21T06:16:13.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.601+0000 7f27ffd44640 1 -- 172.21.15.3:0/1473079655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27f8108d30 msgr2=0x7f27f819abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:13.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.601+0000 7f27ffd44640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27f8108d30 0x7f27f819abe0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f27f8103aa0 tx=0x7f27e80040c0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.602+0000 7f27ffd44640 1 -- 172.21.15.3:0/1473079655 shutdown_connections 2024-09-21T06:16:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.602+0000 7f27ffd44640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f27e40b6790 0x7f27e40b8ca0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.602+0000 7f27ffd44640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27f819fdc0 0x7f27f81a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.602+0000 7f27ffd44640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27f8108d30 0x7f27f819abe0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.602+0000 7f27ffd44640 1 --2- 172.21.15.3:0/1473079655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f8102aa0 0x7f27f819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.602+0000 7f27ffd44640 1 -- 172.21.15.3:0/1473079655 >> 172.21.15.3:0/1473079655 conn(0x7f27f80797e0 msgr2=0x7f27f8104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.602+0000 7f27ffd44640 1 -- 172.21.15.3:0/1473079655 shutdown_connections 2024-09-21T06:16:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:13.602+0000 7f27ffd44640 1 -- 172.21.15.3:0/1473079655 wait complete. 2024-09-21T06:16:14.439 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:16:14.829 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:14.830 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:14 smithi003 ceph-mon[31227]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T06:16:15.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:14 smithi137 ceph-mon[34464]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T06:16:15.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:14 smithi204 ceph-mon[34659]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T06:16:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.422+0000 7fa26efb1640 1 -- 172.21.15.3:0/135606360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268108cf0 msgr2=0x7fa26810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.422+0000 7fa26efb1640 1 --2- 172.21.15.3:0/135606360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268108cf0 0x7fa26810f520 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa260009ca0 tx=0x7fa26002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.422+0000 7fa26efb1640 1 -- 172.21.15.3:0/135606360 shutdown_connections 2024-09-21T06:16:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.422+0000 7fa26efb1640 1 --2- 172.21.15.3:0/135606360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2681083a0 0x7fa2681087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.422+0000 7fa26efb1640 1 --2- 172.21.15.3:0/135606360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268108cf0 0x7fa26810f520 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.422+0000 7fa26efb1640 1 --2- 172.21.15.3:0/135606360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa268102a60 0x7fa268102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.422+0000 7fa26efb1640 1 -- 172.21.15.3:0/135606360 >> 172.21.15.3:0/135606360 conn(0x7fa268079730 msgr2=0x7fa268079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:15.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.423+0000 7fa26efb1640 1 -- 172.21.15.3:0/135606360 shutdown_connections 2024-09-21T06:16:15.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.423+0000 7fa26efb1640 1 -- 172.21.15.3:0/135606360 wait complete. 2024-09-21T06:16:15.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.424+0000 7fa26efb1640 1 Processor -- start 2024-09-21T06:16:15.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.424+0000 7fa26efb1640 1 -- start start 2024-09-21T06:16:15.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.425+0000 7fa26efb1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268102a60 0x7fa268199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.425+0000 7fa26efb1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2681083a0 0x7fa268199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.425+0000 7fa26efb1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa268108cf0 0x7fa268199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.425+0000 7fa26efb1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa268112960 con 0x7fa268102a60 2024-09-21T06:16:15.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.425+0000 7fa26efb1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681127e0 con 0x7fa2681083a0 2024-09-21T06:16:15.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.425+0000 7fa26efb1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa268112ae0 con 0x7fa268108cf0 2024-09-21T06:16:15.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.425+0000 7fa26cd4d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268102a60 0x7fa268199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.425+0000 7fa26cd4d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268102a60 0x7fa268199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46800/0 (socket says 172.21.15.3:46800) 2024-09-21T06:16:15.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.425+0000 7fa26cd4d640 1 -- 172.21.15.3:0/1205745615 learned_addr learned my addr 172.21.15.3:0/1205745615 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:15.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.425+0000 7fa267fff640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2681083a0 0x7fa268199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.425+0000 7fa26d54e640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa268108cf0 0x7fa268199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.426+0000 7fa26cd4d640 1 -- 172.21.15.3:0/1205745615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa268108cf0 msgr2=0x7fa268199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:15.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.426+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa268108cf0 0x7fa268199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.426+0000 7fa26cd4d640 1 -- 172.21.15.3:0/1205745615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2681083a0 msgr2=0x7fa268199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:15.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.426+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2681083a0 0x7fa268199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.426+0000 7fa26cd4d640 1 -- 172.21.15.3:0/1205745615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2681a8740 con 0x7fa268102a60 2024-09-21T06:16:15.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.426+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268102a60 0x7fa268199040 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00d980 tx=0x7fa25c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:15.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.426+0000 7fa265ffb640 1 -- 172.21.15.3:0/1205745615 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c017070 con 0x7fa268102a60 2024-09-21T06:16:15.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.426+0000 7fa265ffb640 1 -- 172.21.15.3:0/1205745615 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa25c009c30 con 0x7fa268102a60 2024-09-21T06:16:15.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.426+0000 7fa26efb1640 1 -- 172.21.15.3:0/1205745615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26819b2b0 con 0x7fa268102a60 2024-09-21T06:16:15.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.426+0000 7fa265ffb640 1 -- 172.21.15.3:0/1205745615 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c00f620 con 0x7fa268102a60 2024-09-21T06:16:15.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.427+0000 7fa26efb1640 1 -- 172.21.15.3:0/1205745615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa26819b840 con 0x7fa268102a60 2024-09-21T06:16:15.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.431+0000 7fa265ffb640 1 -- 172.21.15.3:0/1205745615 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7fa25c010460 con 0x7fa268102a60 2024-09-21T06:16:15.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.431+0000 7fa275525740 1 Processor -- start 2024-09-21T06:16:15.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.431+0000 7fa275525740 1 -- start start 2024-09-21T06:16:15.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.431+0000 7fa275525740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e2938132c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.431+0000 7fa275525740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e293813800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa275525740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29344b390 0x55e293813d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa26cd4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e293813800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa275525740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e293789ea0 con 0x55e29346c070 2024-09-21T06:16:15.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa275525740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e2937792d0 con 0x55e29344b390 2024-09-21T06:16:15.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa275525740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e29377d0e0 con 0x55e2934431b0 2024-09-21T06:16:15.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa26d54e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e2938132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa26cd4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e293813800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:56758/0 (socket says 172.21.15.3:56758) 2024-09-21T06:16:15.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa26cd4d640 1 -- 172.21.15.3:0/3321273985 learned_addr learned my addr 172.21.15.3:0/3321273985 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:15.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa267fff640 1 --2- 172.21.15.3:0/3321273985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29344b390 0x55e293813d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa26cd4d640 1 -- 172.21.15.3:0/3321273985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29344b390 msgr2=0x55e293813d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/3321273985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29344b390 0x55e293813d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa26cd4d640 1 -- 172.21.15.3:0/3321273985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 msgr2=0x55e2938132c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/3321273985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e2938132c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa26cd4d640 1 -- 172.21.15.3:0/3321273985 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2934d7650 con 0x55e2934431b0 2024-09-21T06:16:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.432+0000 7fa26d54e640 1 --2- 172.21.15.3:0/3321273985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e2938132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.433+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/3321273985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e293813800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0e4aa0 tx=0x7fa25c0efa10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=287c325441a8351b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.433+0000 7fa252ffd640 1 -- 172.21.15.3:0/3321273985 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0fb070 con 0x55e2934431b0 2024-09-21T06:16:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.433+0000 7fa252ffd640 1 -- 172.21.15.3:0/3321273985 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa25c0e7cf0 con 0x55e2934431b0 2024-09-21T06:16:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.433+0000 7fa252ffd640 1 -- 172.21.15.3:0/3321273985 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0f39f0 con 0x55e2934431b0 2024-09-21T06:16:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.433+0000 7fa265ffb640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa2580b6790 0x7fa2580b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.434+0000 7fa265ffb640 1 -- 172.21.15.3:0/1205745615 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fa25c0e7430 con 0x7fa268102a60 2024-09-21T06:16:15.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.434+0000 7fa26d54e640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa2580b6790 0x7fa2580b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.434+0000 7fa275525740 1 -- 172.21.15.3:0/3321273985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 msgr2=0x55e293813800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:15.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.434+0000 7fa275525740 1 --2- 172.21.15.3:0/3321273985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e293813800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0e4aa0 tx=0x7fa25c0efa10 comp rx=0 tx=0).stop 2024-09-21T06:16:15.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.434+0000 7fa275525740 1 -- 172.21.15.3:0/3321273985 shutdown_connections 2024-09-21T06:16:15.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.434+0000 7fa275525740 1 --2- 172.21.15.3:0/3321273985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e293813800 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.434+0000 7fa275525740 1 --2- 172.21.15.3:0/3321273985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29344b390 0x55e293813d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.434+0000 7fa275525740 1 --2- 172.21.15.3:0/3321273985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e2938132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.434+0000 7fa275525740 1 -- 172.21.15.3:0/3321273985 >> 172.21.15.3:0/3321273985 conn(0x55e293460460 msgr2=0x55e2938111b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:15.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.435+0000 7fa275525740 1 -- 172.21.15.3:0/3321273985 shutdown_connections 2024-09-21T06:16:15.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.435+0000 7fa26d54e640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa2580b6790 0x7fa2580b8ca0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa260005cc0 tx=0x7fa260043040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:15.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.435+0000 7fa275525740 1 -- 172.21.15.3:0/3321273985 wait complete. 2024-09-21T06:16:15.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.435+0000 7fa275525740 1 -- start start 2024-09-21T06:16:15.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.435+0000 7fa275525740 1 Processor -- start 2024-09-21T06:16:15.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.436+0000 7fa275525740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e29381e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.436+0000 7fa275525740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29344b390 0x55e29381ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.436+0000 7fa275525740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29346c070 0x55e29383bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.436+0000 7fa275525740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e2935da700 con 0x55e29344b390 2024-09-21T06:16:15.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.436+0000 7fa275525740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e293714160 con 0x55e29346c070 2024-09-21T06:16:15.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.436+0000 7fa275525740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e293832a50 con 0x55e2934431b0 2024-09-21T06:16:15.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.436+0000 7fa275525740 10 client.0 _ll_register_callbacks cb 0x55e29372d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:15.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.436+0000 7fa267fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29344b390 0x55e29381ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.436+0000 7fa267fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29344b390 0x55e29381ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46818/0 (socket says 172.21.15.3:46818) 2024-09-21T06:16:15.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.436+0000 7fa267fff640 1 -- 172.21.15.3:0/2099924905 learned_addr learned my addr 172.21.15.3:0/2099924905 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:15.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.437+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/2099924905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e29381e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.437+0000 7fa26d54e640 1 --2- 172.21.15.3:0/2099924905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29346c070 0x55e29383bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.437+0000 7fa267fff640 1 -- 172.21.15.3:0/2099924905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 msgr2=0x55e29381e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.437+0000 7fa267fff640 1 --2- 172.21.15.3:0/2099924905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e29381e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.437+0000 7fa267fff640 1 -- 172.21.15.3:0/2099924905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29346c070 msgr2=0x55e29383bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.437+0000 7fa267fff640 1 --2- 172.21.15.3:0/2099924905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29346c070 0x55e29383bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.437+0000 7fa267fff640 1 -- 172.21.15.3:0/2099924905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e293741c90 con 0x55e29344b390 2024-09-21T06:16:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.437+0000 7fa267fff640 1 --2- 172.21.15.3:0/2099924905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29344b390 0x55e29381ef20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa25400be30 tx=0x7fa254012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44811b19de312ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.437+0000 7fa247fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.437+0000 7fa247fff640 1 -- 172.21.15.3:0/2099924905 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254004220 con 0x55e29344b390 2024-09-21T06:16:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.437+0000 7fa247fff640 1 -- 172.21.15.3:0/2099924905 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fa2540043c0 con 0x55e29344b390 2024-09-21T06:16:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.437+0000 7fa275525740 1 -- 172.21.15.3:0/2099924905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa25c0e7cf0 con 0x55e29344b390 2024-09-21T06:16:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.437+0000 7fa247fff640 1 -- 172.21.15.3:0/2099924905 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa254005020 con 0x55e29344b390 2024-09-21T06:16:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.438+0000 7fa275525740 10 client.15891 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.438+0000 7fa275525740 1 -- 172.21.15.3:0/2099924905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e29383e7b0 con 0x55e29344b390 2024-09-21T06:16:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.438+0000 7fa247fff640 1 -- 172.21.15.3:0/2099924905 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa25c0e7cf0 con 0x55e29344b390 2024-09-21T06:16:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.438+0000 7fa247fff640 1 -- 172.21.15.3:0/2099924905 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa254019070 con 0x55e29344b390 2024-09-21T06:16:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.438+0000 7fa275525740 10 client.15891 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.438+0000 7fa275525740 10 client.15891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:16:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.438+0000 7fa275525740 20 client.15891 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.438+0000 7fa275525740 1 --2- 172.21.15.3:0/2099924905 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55e29342a9f0 0x55e293841d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.438+0000 7fa275525740 4 client.15891 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.438+0000 7fa275525740 1 -- 172.21.15.3:0/2099924905 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e29378bdf0 con 0x55e29342a9f0 2024-09-21T06:16:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.438+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/2099924905 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55e29342a9f0 0x55e293841d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.439+0000 7fa247fff640 10 client.15891 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:16:15.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.439+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/2099924905 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55e29342a9f0 0x55e293841d80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc720400d105805f server_cookie=ceadeb7c3dc9ea55 in_seq=0 out_seq=0 2024-09-21T06:16:15.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.440+0000 7fa247fff640 1 -- 172.21.15.3:0/2099924905 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e29378bdf0 con 0x55e29342a9f0 2024-09-21T06:16:15.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.440+0000 7fa247fff640 10 client.15891 handle_command_reply: tid=0 2024-09-21T06:16:15.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.440+0000 7fa275525740 1 client.15891 shutdown 2024-09-21T06:16:15.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.444+0000 7fa275525740 1 -- 172.21.15.3:0/2099924905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29344b390 msgr2=0x55e29381ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:15.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.444+0000 7fa275525740 1 --2- 172.21.15.3:0/2099924905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29344b390 0x55e29381ef20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa25400be30 tx=0x7fa254012820 comp rx=0 tx=0).stop 2024-09-21T06:16:15.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.444+0000 7fa275525740 1 -- 172.21.15.3:0/2099924905 shutdown_connections 2024-09-21T06:16:15.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.444+0000 7fa275525740 1 --2- 172.21.15.3:0/2099924905 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55e29342a9f0 0x55e293841d80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.444+0000 7fa247fff640 0 client.15891 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:16:15.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.444+0000 7fa275525740 1 --2- 172.21.15.3:0/2099924905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29346c070 0x55e29383bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.444+0000 7fa275525740 1 --2- 172.21.15.3:0/2099924905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29344b390 0x55e29381ef20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.444+0000 7fa275525740 1 --2- 172.21.15.3:0/2099924905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e29381e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.444+0000 7fa275525740 1 -- 172.21.15.3:0/2099924905 >> 172.21.15.3:0/2099924905 conn(0x55e293460460 msgr2=0x55e2938164a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:15.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.444+0000 7fa275525740 1 -- 172.21.15.3:0/2099924905 shutdown_connections 2024-09-21T06:16:15.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.444+0000 7fa275525740 1 -- 172.21.15.3:0/2099924905 wait complete. 2024-09-21T06:16:15.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.445+0000 7fa275525740 20 client.15891 trim_cache size 0 max 16384 2024-09-21T06:16:15.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.470+0000 7fa275525740 1 Processor -- start 2024-09-21T06:16:15.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.470+0000 7fa275525740 1 -- start start 2024-09-21T06:16:15.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.470+0000 7fa275525740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e29381eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.470+0000 7fa275525740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29344b390 0x55e2938163c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.470+0000 7fa275525740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e293816900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.470+0000 7fa275525740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681ad590 con 0x55e29346c070 2024-09-21T06:16:15.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.470+0000 7fa275525740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681af0d0 con 0x55e29344b390 2024-09-21T06:16:15.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.470+0000 7fa275525740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681adc60 con 0x55e2934431b0 2024-09-21T06:16:15.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.471+0000 7fa26d54e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e293816900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.471+0000 7fa267fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29344b390 0x55e2938163c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.471+0000 7fa26d54e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e293816900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46834/0 (socket says 172.21.15.3:46834) 2024-09-21T06:16:15.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.471+0000 7fa267fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29344b390 0x55e2938163c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40548/0 (socket says 172.21.15.3:40548) 2024-09-21T06:16:15.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.471+0000 7fa26d54e640 1 -- 172.21.15.3:0/2445258653 learned_addr learned my addr 172.21.15.3:0/2445258653 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:15.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.471+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/2445258653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e29381eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.471+0000 7fa267fff640 1 -- 172.21.15.3:0/2445258653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 msgr2=0x55e29381eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.471+0000 7fa267fff640 1 --2- 172.21.15.3:0/2445258653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e29381eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.471+0000 7fa267fff640 1 -- 172.21.15.3:0/2445258653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 msgr2=0x55e293816900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.471+0000 7fa267fff640 1 --2- 172.21.15.3:0/2445258653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e293816900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.471+0000 7fa267fff640 1 -- 172.21.15.3:0/2445258653 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2938155b0 con 0x55e29344b390 2024-09-21T06:16:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.471+0000 7fa26d54e640 1 --2- 172.21.15.3:0/2445258653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e293816900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.472+0000 7fa267fff640 1 --2- 172.21.15.3:0/2445258653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29344b390 0x55e2938163c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa254010860 tx=0x7fa254010790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad27af4ddc30555a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.472+0000 7fa2527fc640 1 -- 172.21.15.3:0/2445258653 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254026070 con 0x55e29344b390 2024-09-21T06:16:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.472+0000 7fa2527fc640 1 -- 172.21.15.3:0/2445258653 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2540053c0 con 0x55e29344b390 2024-09-21T06:16:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.472+0000 7fa2527fc640 1 -- 172.21.15.3:0/2445258653 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254004c40 con 0x55e29344b390 2024-09-21T06:16:15.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.473+0000 7fa275525740 1 -- 172.21.15.3:0/2445258653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29344b390 msgr2=0x55e2938163c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:15.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.473+0000 7fa275525740 1 --2- 172.21.15.3:0/2445258653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29344b390 0x55e2938163c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa254010860 tx=0x7fa254010790 comp rx=0 tx=0).stop 2024-09-21T06:16:15.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.473+0000 7fa275525740 1 -- 172.21.15.3:0/2445258653 shutdown_connections 2024-09-21T06:16:15.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.473+0000 7fa275525740 1 --2- 172.21.15.3:0/2445258653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e293816900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.473+0000 7fa275525740 1 --2- 172.21.15.3:0/2445258653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e29344b390 0x55e2938163c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.473+0000 7fa275525740 1 --2- 172.21.15.3:0/2445258653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2934431b0 0x55e29381eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.473+0000 7fa275525740 1 -- 172.21.15.3:0/2445258653 >> 172.21.15.3:0/2445258653 conn(0x55e293460460 msgr2=0x55e29381e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:15.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.473+0000 7fa275525740 1 -- 172.21.15.3:0/2445258653 shutdown_connections 2024-09-21T06:16:15.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.474+0000 7fa275525740 1 -- 172.21.15.3:0/2445258653 wait complete. 2024-09-21T06:16:15.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.474+0000 7fa275525740 1 -- start start 2024-09-21T06:16:15.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.474+0000 7fa275525740 1 Processor -- start 2024-09-21T06:16:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.475+0000 7fa275525740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2934431b0 0x55e293833ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.475+0000 7fa275525740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29344b390 0x55e2938341e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.475+0000 7fa275525740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e293834720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.475+0000 7fa275525740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681aea00 con 0x55e29346c070 2024-09-21T06:16:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.475+0000 7fa275525740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681ae330 con 0x55e2934431b0 2024-09-21T06:16:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.475+0000 7fa275525740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681ab570 con 0x55e29344b390 2024-09-21T06:16:15.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.475+0000 7fa275525740 10 client.0 _ll_register_callbacks cb 0x7fa25400ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:15.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.475+0000 7fa267fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29344b390 0x55e2938341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.475+0000 7fa26d54e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e293834720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.476+0000 7fa26d54e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e293834720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46840/0 (socket says 172.21.15.3:46840) 2024-09-21T06:16:15.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.476+0000 7fa26d54e640 1 -- 172.21.15.3:0/2182225905 learned_addr learned my addr 172.21.15.3:0/2182225905 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:15.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.476+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/2182225905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2934431b0 0x55e293833ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.476+0000 7fa26d54e640 1 -- 172.21.15.3:0/2182225905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29344b390 msgr2=0x55e2938341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:15.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.476+0000 7fa26d54e640 1 --2- 172.21.15.3:0/2182225905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29344b390 0x55e2938341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.476+0000 7fa26d54e640 1 -- 172.21.15.3:0/2182225905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2934431b0 msgr2=0x55e293833ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:15.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.476+0000 7fa26d54e640 1 --2- 172.21.15.3:0/2182225905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2934431b0 0x55e293833ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.476+0000 7fa26d54e640 1 -- 172.21.15.3:0/2182225905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2938ab780 con 0x55e29346c070 2024-09-21T06:16:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.476+0000 7fa26d54e640 1 --2- 172.21.15.3:0/2182225905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e293834720 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa26002ade0 tx=0x7fa26002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98548b4a4bb9cad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.476+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/2182225905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2934431b0 0x55e293833ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:16:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.476+0000 7fa247fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.477+0000 7fa247fff640 1 -- 172.21.15.3:0/2182225905 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260054070 con 0x55e29346c070 2024-09-21T06:16:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.477+0000 7fa247fff640 1 -- 172.21.15.3:0/2182225905 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fa2600043d0 con 0x55e29346c070 2024-09-21T06:16:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.477+0000 7fa275525740 1 -- 172.21.15.3:0/2182225905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2540053c0 con 0x55e29346c070 2024-09-21T06:16:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.477+0000 7fa247fff640 1 -- 172.21.15.3:0/2182225905 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa260060050 con 0x55e29346c070 2024-09-21T06:16:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.477+0000 7fa275525740 10 client.15897 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.477+0000 7fa275525740 1 -- 172.21.15.3:0/2182225905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e2938ac9a0 con 0x55e29346c070 2024-09-21T06:16:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.477+0000 7fa247fff640 1 -- 172.21.15.3:0/2182225905 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa260040070 con 0x55e29346c070 2024-09-21T06:16:15.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.478+0000 7fa247fff640 1 -- 172.21.15.3:0/2182225905 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa2600305c0 con 0x55e29346c070 2024-09-21T06:16:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.478+0000 7fa275525740 10 client.15897 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.478+0000 7fa275525740 10 client.15897 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:16:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.478+0000 7fa275525740 20 client.15897 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.478+0000 7fa275525740 1 --2- 172.21.15.3:0/2182225905 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55e2933ffda0 0x55e2938af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.478+0000 7fa275525740 4 client.15897 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.478+0000 7fa275525740 1 -- 172.21.15.3:0/2182225905 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa2540053c0 con 0x55e2933ffda0 2024-09-21T06:16:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.479+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/2182225905 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55e2933ffda0 0x55e2938af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.479+0000 7fa247fff640 10 client.15897 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:16:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.479+0000 7fa26cd4d640 1 --2- 172.21.15.3:0/2182225905 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55e2933ffda0 0x55e2938af1c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6bf0a6a75903a84 server_cookie=210950b8d1006404 in_seq=0 out_seq=0 2024-09-21T06:16:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.480+0000 7fa247fff640 1 -- 172.21.15.3:0/2182225905 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa2540053c0 con 0x55e2933ffda0 2024-09-21T06:16:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.480+0000 7fa247fff640 10 client.15897 handle_command_reply: tid=0 2024-09-21T06:16:15.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.480+0000 7fa275525740 1 client.15897 shutdown 2024-09-21T06:16:15.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.481+0000 7fa275525740 1 -- 172.21.15.3:0/2182225905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 msgr2=0x55e293834720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:15.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.481+0000 7fa275525740 1 --2- 172.21.15.3:0/2182225905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e293834720 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa26002ade0 tx=0x7fa26002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.481+0000 7fa275525740 1 -- 172.21.15.3:0/2182225905 shutdown_connections 2024-09-21T06:16:15.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.481+0000 7fa275525740 1 --2- 172.21.15.3:0/2182225905 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55e2933ffda0 0x55e2938af1c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.481+0000 7fa275525740 1 --2- 172.21.15.3:0/2182225905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29344b390 0x55e2938341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.481+0000 7fa275525740 1 --2- 172.21.15.3:0/2182225905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e29346c070 0x55e293834720 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.481+0000 7fa275525740 1 --2- 172.21.15.3:0/2182225905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2934431b0 0x55e293833ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.481+0000 7fa275525740 1 -- 172.21.15.3:0/2182225905 >> 172.21.15.3:0/2182225905 conn(0x55e293460460 msgr2=0x55e2938163c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:15.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.481+0000 7fa247fff640 0 client.15897 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:16:15.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.481+0000 7fa275525740 1 -- 172.21.15.3:0/2182225905 shutdown_connections 2024-09-21T06:16:15.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.481+0000 7fa275525740 1 -- 172.21.15.3:0/2182225905 wait complete. 2024-09-21T06:16:15.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.482+0000 7fa275525740 20 client.15897 trim_cache size 0 max 16384 2024-09-21T06:16:15.486 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:16:15.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.482+0000 7fa26efb1640 1 -- 172.21.15.3:0/1205745615 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa2580b6790 msgr2=0x7fa2580b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:15.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.482+0000 7fa26efb1640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa2580b6790 0x7fa2580b8ca0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa260005cc0 tx=0x7fa260043040 comp rx=0 tx=0).stop 2024-09-21T06:16:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.482+0000 7fa26efb1640 1 -- 172.21.15.3:0/1205745615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268102a60 msgr2=0x7fa268199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.482+0000 7fa26efb1640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268102a60 0x7fa268199040 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00d980 tx=0x7fa25c00de40 comp rx=0 tx=0).stop 2024-09-21T06:16:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.483+0000 7fa26efb1640 1 -- 172.21.15.3:0/1205745615 shutdown_connections 2024-09-21T06:16:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.483+0000 7fa26efb1640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa2580b6790 0x7fa2580b8ca0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.483+0000 7fa26efb1640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa268108cf0 0x7fa268199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.483+0000 7fa26efb1640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2681083a0 0x7fa268199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.483+0000 7fa26efb1640 1 --2- 172.21.15.3:0/1205745615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa268102a60 0x7fa268199040 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.483+0000 7fa26efb1640 1 -- 172.21.15.3:0/1205745615 >> 172.21.15.3:0/1205745615 conn(0x7fa268079730 msgr2=0x7fa268104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.483+0000 7fa26efb1640 1 -- 172.21.15.3:0/1205745615 shutdown_connections 2024-09-21T06:16:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:15.483+0000 7fa26efb1640 1 -- 172.21.15.3:0/1205745615 wait complete. 2024-09-21T06:16:16.452 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:16:16.852 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:17.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:16 smithi003 ceph-mon[31227]: pgmap v363: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T06:16:17.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:16 smithi137 ceph-mon[34464]: pgmap v363: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T06:16:17.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:16 smithi204 ceph-mon[34659]: pgmap v363: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T06:16:17.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.390+0000 7f49369b5640 1 -- 172.21.15.3:0/4091899587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4930102a60 msgr2=0x7f4930102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.390+0000 7f49369b5640 1 --2- 172.21.15.3:0/4091899587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4930102a60 0x7f4930102e70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f492000fb10 tx=0x7f492002e570 comp rx=0 tx=0).stop 2024-09-21T06:16:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.390+0000 7f49369b5640 1 -- 172.21.15.3:0/4091899587 shutdown_connections 2024-09-21T06:16:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.390+0000 7f49369b5640 1 --2- 172.21.15.3:0/4091899587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49301083a0 0x7f49301087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.390+0000 7f49369b5640 1 --2- 172.21.15.3:0/4091899587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4930108cf0 0x7f493010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.390+0000 7f49369b5640 1 --2- 172.21.15.3:0/4091899587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4930102a60 0x7f4930102e70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.390+0000 7f49369b5640 1 -- 172.21.15.3:0/4091899587 >> 172.21.15.3:0/4091899587 conn(0x7f4930079730 msgr2=0x7f4930079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.391+0000 7f49369b5640 1 -- 172.21.15.3:0/4091899587 shutdown_connections 2024-09-21T06:16:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.391+0000 7f49369b5640 1 -- 172.21.15.3:0/4091899587 wait complete. 2024-09-21T06:16:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.393+0000 7f49369b5640 1 Processor -- start 2024-09-21T06:16:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.393+0000 7f49369b5640 1 -- start start 2024-09-21T06:16:17.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.394+0000 7f49369b5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49301083a0 0x7f4930198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.394+0000 7f49369b5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4930108cf0 0x7f49301989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.394+0000 7f49369b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f493019db90 0x7f493019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.394+0000 7f492f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4930108cf0 0x7f49301989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.394+0000 7f492f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4930108cf0 0x7f49301989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46868/0 (socket says 172.21.15.3:46868) 2024-09-21T06:16:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.394+0000 7f492ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49301083a0 0x7f4930198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.394+0000 7f492ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49301083a0 0x7f4930198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40570/0 (socket says 172.21.15.3:40570) 2024-09-21T06:16:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.394+0000 7f492f7fe640 1 -- 172.21.15.3:0/1198386993 learned_addr learned my addr 172.21.15.3:0/1198386993 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.394+0000 7f49369b5640 1 -- 172.21.15.3:0/1198386993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f49301118a0 con 0x7f4930108cf0 2024-09-21T06:16:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.394+0000 7f49369b5640 1 -- 172.21.15.3:0/1198386993 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4930111720 con 0x7f49301083a0 2024-09-21T06:16:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.394+0000 7f49369b5640 1 -- 172.21.15.3:0/1198386993 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4930111a20 con 0x7f493019db90 2024-09-21T06:16:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.395+0000 7f4934f52640 1 --2- 172.21.15.3:0/1198386993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f493019db90 0x7f493019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.395+0000 7f492f7fe640 1 -- 172.21.15.3:0/1198386993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f493019db90 msgr2=0x7f493019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.395+0000 7f492f7fe640 1 --2- 172.21.15.3:0/1198386993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f493019db90 0x7f493019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.395+0000 7f492f7fe640 1 -- 172.21.15.3:0/1198386993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49301083a0 msgr2=0x7f4930198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.395+0000 7f492f7fe640 1 --2- 172.21.15.3:0/1198386993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49301083a0 0x7f4930198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.395+0000 7f492f7fe640 1 -- 172.21.15.3:0/1198386993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493019e770 con 0x7f4930108cf0 2024-09-21T06:16:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.395+0000 7f492f7fe640 1 --2- 172.21.15.3:0/1198386993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4930108cf0 0x7f49301989b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f492400e9f0 tx=0x7f492400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.396+0000 7f492d7fa640 1 -- 172.21.15.3:0/1198386993 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492400ccc0 con 0x7f4930108cf0 2024-09-21T06:16:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.396+0000 7f492d7fa640 1 -- 172.21.15.3:0/1198386993 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49240054d0 con 0x7f4930108cf0 2024-09-21T06:16:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.396+0000 7f492d7fa640 1 -- 172.21.15.3:0/1198386993 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4924010430 con 0x7f4930108cf0 2024-09-21T06:16:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.396+0000 7f49369b5640 1 -- 172.21.15.3:0/1198386993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49301a4d10 con 0x7f4930108cf0 2024-09-21T06:16:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.397+0000 7f49369b5640 1 -- 172.21.15.3:0/1198386993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49301a5110 con 0x7f4930108cf0 2024-09-21T06:16:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.400+0000 7f493cf29740 1 Processor -- start 2024-09-21T06:16:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.400+0000 7f493cf29740 1 -- start start 2024-09-21T06:16:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.400+0000 7f493cf29740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cf26070 0x55fb4d2cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.400+0000 7f493cf29740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.400+0000 7f493cf29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf05390 0x55fb4d2cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.400+0000 7f493cf29740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fb4d243ea0 con 0x55fb4cefd1b0 2024-09-21T06:16:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.400+0000 7f493cf29740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fb4d2332d0 con 0x55fb4cf26070 2024-09-21T06:16:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.400+0000 7f493cf29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fb4d2370e0 con 0x55fb4cf05390 2024-09-21T06:16:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.400+0000 7f492ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.400+0000 7f492ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46882/0 (socket says 172.21.15.3:46882) 2024-09-21T06:16:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.400+0000 7f492ffff640 1 -- 172.21.15.3:0/2082735492 learned_addr learned my addr 172.21.15.3:0/2082735492 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.401+0000 7f4934f52640 1 --2- 172.21.15.3:0/2082735492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cf26070 0x55fb4d2cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.401+0000 7f492ffff640 1 -- 172.21.15.3:0/2082735492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf05390 msgr2=0x55fb4d2cdd40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:16:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.401+0000 7f492ffff640 1 --2- 172.21.15.3:0/2082735492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf05390 0x55fb4d2cdd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.401+0000 7f492ffff640 1 -- 172.21.15.3:0/2082735492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cf26070 msgr2=0x55fb4d2cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.401+0000 7f492ffff640 1 --2- 172.21.15.3:0/2082735492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cf26070 0x55fb4d2cd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.401+0000 7f492ffff640 1 -- 172.21.15.3:0/2082735492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb4cf91650 con 0x55fb4cefd1b0 2024-09-21T06:16:17.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.401+0000 7f492ffff640 1 --2- 172.21.15.3:0/2082735492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2cd800 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f492002eb60 tx=0x7f492000e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738e943edd14d5fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:17.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.402+0000 7f492d7fa640 1 -- 172.21.15.3:0/1198386993 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7f4924005000 con 0x7f4930108cf0 2024-09-21T06:16:17.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.402+0000 7f49167fc640 1 -- 172.21.15.3:0/2082735492 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49200048c0 con 0x55fb4cefd1b0 2024-09-21T06:16:17.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.402+0000 7f49167fc640 1 -- 172.21.15.3:0/2082735492 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4920045070 con 0x55fb4cefd1b0 2024-09-21T06:16:17.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.402+0000 7f49167fc640 1 -- 172.21.15.3:0/2082735492 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492000eb60 con 0x55fb4cefd1b0 2024-09-21T06:16:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.403+0000 7f492d7fa640 1 --2- 172.21.15.3:0/1198386993 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f49180b6810 0x7f49180b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.403+0000 7f492d7fa640 1 -- 172.21.15.3:0/1198386993 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f49240e7cc0 con 0x7f4930108cf0 2024-09-21T06:16:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.403+0000 7f4934f52640 1 --2- 172.21.15.3:0/1198386993 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f49180b6810 0x7f49180b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.403+0000 7f493cf29740 1 -- 172.21.15.3:0/2082735492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cefd1b0 msgr2=0x55fb4d2cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.403+0000 7f493cf29740 1 --2- 172.21.15.3:0/2082735492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2cd800 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f492002eb60 tx=0x7f492000e710 comp rx=0 tx=0).stop 2024-09-21T06:16:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.404+0000 7f493cf29740 1 -- 172.21.15.3:0/2082735492 shutdown_connections 2024-09-21T06:16:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.404+0000 7f493cf29740 1 --2- 172.21.15.3:0/2082735492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf05390 0x55fb4d2cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.404+0000 7f493cf29740 1 --2- 172.21.15.3:0/2082735492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2cd800 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.404+0000 7f493cf29740 1 --2- 172.21.15.3:0/2082735492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cf26070 0x55fb4d2cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.404+0000 7f493cf29740 1 -- 172.21.15.3:0/2082735492 >> 172.21.15.3:0/2082735492 conn(0x55fb4cf1a460 msgr2=0x55fb4d2cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:17.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.404+0000 7f4934f52640 1 --2- 172.21.15.3:0/1198386993 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f49180b6810 0x7f49180b8d20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f492800b3e0 tx=0x7f4928005bd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:17.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.404+0000 7f493cf29740 1 -- 172.21.15.3:0/2082735492 shutdown_connections 2024-09-21T06:16:17.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.404+0000 7f493cf29740 1 -- 172.21.15.3:0/2082735492 wait complete. 2024-09-21T06:16:17.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.405+0000 7f493cf29740 1 -- start start 2024-09-21T06:16:17.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.405+0000 7f493cf29740 1 Processor -- start 2024-09-21T06:16:17.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.405+0000 7f493cf29740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2d8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.406+0000 7f493cf29740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2d8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.406+0000 7f493cf29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf26070 0x55fb4d2e4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.406+0000 7f492f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2d8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.406+0000 7f492f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2d8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46890/0 (socket says 172.21.15.3:46890) 2024-09-21T06:16:17.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.406+0000 7f492f7fe640 1 -- 172.21.15.3:0/1458663560 learned_addr learned my addr 172.21.15.3:0/1458663560 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:17.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.406+0000 7f493cf29740 1 -- 172.21.15.3:0/1458663560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fb4d094700 con 0x55fb4cf05390 2024-09-21T06:16:17.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.406+0000 7f493cf29740 1 -- 172.21.15.3:0/1458663560 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fb4d1ce160 con 0x55fb4cefd1b0 2024-09-21T06:16:17.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.406+0000 7f493cf29740 1 -- 172.21.15.3:0/1458663560 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fb4d2dba40 con 0x55fb4cf26070 2024-09-21T06:16:17.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.406+0000 7f4934f52640 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf26070 0x55fb4d2e4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.406+0000 7f493cf29740 10 client.0 _ll_register_callbacks cb 0x55fb4d1e79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.406+0000 7f492ffff640 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2d8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.407+0000 7f4934f52640 1 -- 172.21.15.3:0/1458663560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cefd1b0 msgr2=0x55fb4d2d8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.407+0000 7f4934f52640 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2d8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.407+0000 7f4934f52640 1 -- 172.21.15.3:0/1458663560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 msgr2=0x55fb4d2d8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.407+0000 7f4934f52640 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2d8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.407+0000 7f4934f52640 1 -- 172.21.15.3:0/1458663560 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb4d1fbc90 con 0x55fb4cf26070 2024-09-21T06:16:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.407+0000 7f492f7fe640 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2d8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.407+0000 7f4934f52640 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf26070 0x55fb4d2e4d60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4928005fd0 tx=0x7f4928004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5440407d3b947b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.407+0000 7f490f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:16:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.408+0000 7f490f7fe640 1 -- 172.21.15.3:0/1458663560 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928028060 con 0x55fb4cf26070 2024-09-21T06:16:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.408+0000 7f493cf29740 1 -- 172.21.15.3:0/1458663560 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4920045070 con 0x55fb4cf26070 2024-09-21T06:16:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.408+0000 7f490f7fe640 1 -- 172.21.15.3:0/1458663560 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f4928017a00 con 0x55fb4cf26070 2024-09-21T06:16:17.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.408+0000 7f490f7fe640 1 -- 172.21.15.3:0/1458663560 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4928017c00 con 0x55fb4cf26070 2024-09-21T06:16:17.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.408+0000 7f493cf29740 10 client.25483 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.408+0000 7f493cf29740 1 -- 172.21.15.3:0/1458663560 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb4d2e7810 con 0x55fb4cf26070 2024-09-21T06:16:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.408+0000 7f490f7fe640 1 -- 172.21.15.3:0/1458663560 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4920045070 con 0x55fb4cf26070 2024-09-21T06:16:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.409+0000 7f490f7fe640 1 -- 172.21.15.3:0/1458663560 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4928018420 con 0x55fb4cf26070 2024-09-21T06:16:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.409+0000 7f493cf29740 10 client.25483 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.409+0000 7f493cf29740 10 client.25483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.409+0000 7f493cf29740 20 client.25483 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.409+0000 7f493cf29740 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55fb4cee49f0 0x55fb4d2eac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.409+0000 7f493cf29740 4 client.25483 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.409+0000 7f493cf29740 1 -- 172.21.15.3:0/1458663560 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb4d245df0 con 0x55fb4cee49f0 2024-09-21T06:16:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.410+0000 7f492ffff640 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55fb4cee49f0 0x55fb4d2eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.410+0000 7f492ffff640 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55fb4cee49f0 0x55fb4d2eac90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae79fac75a995454 server_cookie=3b1ebdfb521b5d24 in_seq=0 out_seq=0 2024-09-21T06:16:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.410+0000 7f490f7fe640 10 client.25483 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.411+0000 7f490f7fe640 1 -- 172.21.15.3:0/1458663560 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fb4d245df0 con 0x55fb4cee49f0 2024-09-21T06:16:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.411+0000 7f490f7fe640 10 client.25483 handle_command_reply: tid=0 2024-09-21T06:16:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.411+0000 7f493cf29740 1 client.25483 shutdown 2024-09-21T06:16:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.411+0000 7f493cf29740 1 -- 172.21.15.3:0/1458663560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf26070 msgr2=0x55fb4d2e4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.411+0000 7f493cf29740 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf26070 0x55fb4d2e4d60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4928005fd0 tx=0x7f4928004ab0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.412+0000 7f493cf29740 1 -- 172.21.15.3:0/1458663560 shutdown_connections 2024-09-21T06:16:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.412+0000 7f493cf29740 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55fb4cee49f0 0x55fb4d2eac90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.412+0000 7f493cf29740 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf26070 0x55fb4d2e4d60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.412+0000 7f490f7fe640 0 client.25483 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.412+0000 7f493cf29740 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2d8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.412+0000 7f493cf29740 1 --2- 172.21.15.3:0/1458663560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2d8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.412+0000 7f493cf29740 1 -- 172.21.15.3:0/1458663560 >> 172.21.15.3:0/1458663560 conn(0x55fb4cf1a460 msgr2=0x55fb4d2d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.412+0000 7f493cf29740 1 -- 172.21.15.3:0/1458663560 shutdown_connections 2024-09-21T06:16:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.412+0000 7f493cf29740 1 -- 172.21.15.3:0/1458663560 wait complete. 2024-09-21T06:16:17.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.412+0000 7f493cf29740 20 client.25483 trim_cache size 0 max 16384 2024-09-21T06:16:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.425+0000 7f493cf29740 1 Processor -- start 2024-09-21T06:16:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.426+0000 7f493cf29740 1 -- start start 2024-09-21T06:16:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.426+0000 7f493cf29740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.426+0000 7f493cf29740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2da670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.426+0000 7f493cf29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf26070 0x55fb4d2e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.426+0000 7f493cf29740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f49301ad9c0 con 0x55fb4cf05390 2024-09-21T06:16:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.426+0000 7f492f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2da670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.426+0000 7f493cf29740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49301af500 con 0x55fb4cefd1b0 2024-09-21T06:16:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.426+0000 7f493cf29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f49301ae090 con 0x55fb4cf26070 2024-09-21T06:16:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.426+0000 7f492f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2da670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46900/0 (socket says 172.21.15.3:46900) 2024-09-21T06:16:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.426+0000 7f492f7fe640 1 -- 172.21.15.3:0/1491045677 learned_addr learned my addr 172.21.15.3:0/1491045677 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.426+0000 7f492ffff640 1 --2- 172.21.15.3:0/1491045677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.427+0000 7f4934f52640 1 --2- 172.21.15.3:0/1491045677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf26070 0x55fb4d2e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.427+0000 7f492ffff640 1 -- 172.21.15.3:0/1491045677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf26070 msgr2=0x55fb4d2e4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.427+0000 7f492ffff640 1 --2- 172.21.15.3:0/1491045677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf26070 0x55fb4d2e4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.427+0000 7f492ffff640 1 -- 172.21.15.3:0/1491045677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 msgr2=0x55fb4d2da670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.427+0000 7f492ffff640 1 --2- 172.21.15.3:0/1491045677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2da670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.427+0000 7f492ffff640 1 -- 172.21.15.3:0/1491045677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb4d2d3670 con 0x55fb4cefd1b0 2024-09-21T06:16:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.427+0000 7f492ffff640 1 --2- 172.21.15.3:0/1491045677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2cecb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4920037040 tx=0x7f4920043040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5253a5c19a01497a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.428+0000 7f4915ffb640 1 -- 172.21.15.3:0/1491045677 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492005a020 con 0x55fb4cefd1b0 2024-09-21T06:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.428+0000 7f4915ffb640 1 -- 172.21.15.3:0/1491045677 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4920045070 con 0x55fb4cefd1b0 2024-09-21T06:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.428+0000 7f4915ffb640 1 -- 172.21.15.3:0/1491045677 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49200035f0 con 0x55fb4cefd1b0 2024-09-21T06:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.428+0000 7f493cf29740 1 -- 172.21.15.3:0/1491045677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cefd1b0 msgr2=0x55fb4d2cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.429+0000 7f493cf29740 1 --2- 172.21.15.3:0/1491045677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2cecb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4920037040 tx=0x7f4920043040 comp rx=0 tx=0).stop 2024-09-21T06:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.429+0000 7f493cf29740 1 -- 172.21.15.3:0/1491045677 shutdown_connections 2024-09-21T06:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.429+0000 7f493cf29740 1 --2- 172.21.15.3:0/1491045677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4cf26070 0x55fb4d2e4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.429+0000 7f493cf29740 1 --2- 172.21.15.3:0/1491045677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2da670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.429+0000 7f493cf29740 1 --2- 172.21.15.3:0/1491045677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cefd1b0 0x55fb4d2cecb0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4920037040 tx=0x7f4920043040 comp rx=0 tx=0).stop 2024-09-21T06:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.429+0000 7f493cf29740 1 -- 172.21.15.3:0/1491045677 >> 172.21.15.3:0/1491045677 conn(0x55fb4cf1a460 msgr2=0x55fb4d2d9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.429+0000 7f493cf29740 1 -- 172.21.15.3:0/1491045677 shutdown_connections 2024-09-21T06:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.429+0000 7f493cf29740 1 -- 172.21.15.3:0/1491045677 wait complete. 2024-09-21T06:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.429+0000 7f493cf29740 1 -- start start 2024-09-21T06:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.430+0000 7f493cf29740 1 Processor -- start 2024-09-21T06:16:17.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.430+0000 7f493cf29740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.430+0000 7f493cf29740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cf26070 0x55fb4d2df890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.430+0000 7f493cf29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4ceb9da0 0x55fb4d35ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.430+0000 7f492ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.430+0000 7f492ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2df350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46914/0 (socket says 172.21.15.3:46914) 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.430+0000 7f492ffff640 1 -- 172.21.15.3:0/4102296102 learned_addr learned my addr 172.21.15.3:0/4102296102 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.430+0000 7f493cf29740 1 -- 172.21.15.3:0/4102296102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f49301aee30 con 0x55fb4cf05390 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.430+0000 7f493cf29740 1 -- 172.21.15.3:0/4102296102 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49301ae760 con 0x55fb4cf26070 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.430+0000 7f493cf29740 1 -- 172.21.15.3:0/4102296102 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f49301ab9a0 con 0x55fb4ceb9da0 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.430+0000 7f492f7fe640 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cf26070 0x55fb4d2df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.430+0000 7f493cf29740 10 client.0 _ll_register_callbacks cb 0x7f492800bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.431+0000 7f4934f52640 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4ceb9da0 0x55fb4d35ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.431+0000 7f492f7fe640 1 -- 172.21.15.3:0/4102296102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4ceb9da0 msgr2=0x55fb4d35ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.431+0000 7f492f7fe640 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4ceb9da0 0x55fb4d35ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.431+0000 7f492f7fe640 1 -- 172.21.15.3:0/4102296102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 msgr2=0x55fb4d2df350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.431+0000 7f492f7fe640 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2df350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.431+0000 7f492f7fe640 1 -- 172.21.15.3:0/4102296102 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb4d35d3b0 con 0x55fb4cf26070 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.431+0000 7f492ffff640 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2df350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.431+0000 7f492f7fe640 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cf26070 0x55fb4d2df890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49240aabf0 tx=0x7f49240e5f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfebc1241dfe541b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.431+0000 7f490f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:16:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.431+0000 7f490f7fe640 1 -- 172.21.15.3:0/4102296102 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49240ad750 con 0x55fb4cf26070 2024-09-21T06:16:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.431+0000 7f490f7fe640 1 -- 172.21.15.3:0/4102296102 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f49240a92d0 con 0x55fb4cf26070 2024-09-21T06:16:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.431+0000 7f493cf29740 1 -- 172.21.15.3:0/4102296102 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4920045070 con 0x55fb4cf26070 2024-09-21T06:16:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.432+0000 7f490f7fe640 1 -- 172.21.15.3:0/4102296102 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f49240a90f0 con 0x55fb4cf26070 2024-09-21T06:16:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.432+0000 7f493cf29740 10 client.25499 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.432+0000 7f493cf29740 1 -- 172.21.15.3:0/4102296102 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb4d35e5d0 con 0x55fb4cf26070 2024-09-21T06:16:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.432+0000 7f490f7fe640 1 -- 172.21.15.3:0/4102296102 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4924005680 con 0x55fb4cf26070 2024-09-21T06:16:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.433+0000 7f490f7fe640 1 -- 172.21.15.3:0/4102296102 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f49240a94d0 con 0x55fb4cf26070 2024-09-21T06:16:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.433+0000 7f493cf29740 10 client.25499 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.433+0000 7f493cf29740 10 client.25499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:17.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.433+0000 7f493cf29740 20 client.25499 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:17.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.433+0000 7f493cf29740 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55fb4d35ed80 0x55fb4d3611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:17.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.433+0000 7f493cf29740 4 client.25499 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:17.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.433+0000 7f493cf29740 1 -- 172.21.15.3:0/4102296102 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4920045070 con 0x55fb4d35ed80 2024-09-21T06:16:17.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.433+0000 7f492ffff640 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55fb4d35ed80 0x55fb4d3611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:17.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.434+0000 7f490f7fe640 10 client.25499 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:17.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.434+0000 7f492ffff640 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55fb4d35ed80 0x55fb4d3611a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e31db09ad7f7d742 server_cookie=e3fc52dc05597eb in_seq=0 out_seq=0 2024-09-21T06:16:17.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.434+0000 7f490f7fe640 1 -- 172.21.15.3:0/4102296102 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4920045070 con 0x55fb4d35ed80 2024-09-21T06:16:17.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.434+0000 7f490f7fe640 10 client.25499 handle_command_reply: tid=0 2024-09-21T06:16:17.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.434+0000 7f493cf29740 1 client.25499 shutdown 2024-09-21T06:16:17.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.435+0000 7f493cf29740 1 -- 172.21.15.3:0/4102296102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cf26070 msgr2=0x55fb4d2df890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:17.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.435+0000 7f493cf29740 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cf26070 0x55fb4d2df890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49240aabf0 tx=0x7f49240e5f10 comp rx=0 tx=0).stop 2024-09-21T06:16:17.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.435+0000 7f493cf29740 1 -- 172.21.15.3:0/4102296102 shutdown_connections 2024-09-21T06:16:17.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.435+0000 7f493cf29740 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55fb4d35ed80 0x55fb4d3611a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.435+0000 7f493cf29740 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb4ceb9da0 0x55fb4d35ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.435+0000 7f493cf29740 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb4cf26070 0x55fb4d2df890 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.435+0000 7f493cf29740 1 --2- 172.21.15.3:0/4102296102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb4cf05390 0x55fb4d2df350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.435+0000 7f493cf29740 1 -- 172.21.15.3:0/4102296102 >> 172.21.15.3:0/4102296102 conn(0x55fb4cf1a460 msgr2=0x55fb4d2ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:17.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.435+0000 7f490f7fe640 0 client.25499 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:17.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.435+0000 7f493cf29740 1 -- 172.21.15.3:0/4102296102 shutdown_connections 2024-09-21T06:16:17.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.435+0000 7f493cf29740 1 -- 172.21.15.3:0/4102296102 wait complete. 2024-09-21T06:16:17.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.435+0000 7f493cf29740 20 client.25499 trim_cache size 0 max 16384 2024-09-21T06:16:17.439 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:16:17.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.436+0000 7f49369b5640 1 -- 172.21.15.3:0/1198386993 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f49180b6810 msgr2=0x7f49180b8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:17.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.436+0000 7f49369b5640 1 --2- 172.21.15.3:0/1198386993 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f49180b6810 0x7f49180b8d20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f492800b3e0 tx=0x7f4928005bd0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.436+0000 7f49369b5640 1 -- 172.21.15.3:0/1198386993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4930108cf0 msgr2=0x7f49301989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:17.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.436+0000 7f49369b5640 1 --2- 172.21.15.3:0/1198386993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4930108cf0 0x7f49301989b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f492400e9f0 tx=0x7f492400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.436+0000 7f49369b5640 1 -- 172.21.15.3:0/1198386993 shutdown_connections 2024-09-21T06:16:17.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.436+0000 7f49369b5640 1 --2- 172.21.15.3:0/1198386993 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f49180b6810 0x7f49180b8d20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.437+0000 7f49369b5640 1 --2- 172.21.15.3:0/1198386993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f493019db90 0x7f493019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.437+0000 7f49369b5640 1 --2- 172.21.15.3:0/1198386993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4930108cf0 0x7f49301989b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.437+0000 7f49369b5640 1 --2- 172.21.15.3:0/1198386993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49301083a0 0x7f4930198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:17.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.437+0000 7f49369b5640 1 -- 172.21.15.3:0/1198386993 >> 172.21.15.3:0/1198386993 conn(0x7f4930079730 msgr2=0x7f4930101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:17.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.437+0000 7f49369b5640 1 -- 172.21.15.3:0/1198386993 shutdown_connections 2024-09-21T06:16:17.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:17.437+0000 7f49369b5640 1 -- 172.21.15.3:0/1198386993 wait complete. 2024-09-21T06:16:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:18 smithi003 ceph-mon[31227]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-21T06:16:19.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:18 smithi137 ceph-mon[34464]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-21T06:16:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:18 smithi204 ceph-mon[34659]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-21T06:16:19.234 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:16:19.235 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:16:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.165+0000 7f67162a7640 1 -- 172.21.15.3:0/3950689837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6710108cf0 msgr2=0x7f671010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.165+0000 7f67162a7640 1 --2- 172.21.15.3:0/3950689837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6710108cf0 0x7f671010f520 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6708009ca0 tx=0x7f670802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.166+0000 7f67162a7640 1 -- 172.21.15.3:0/3950689837 shutdown_connections 2024-09-21T06:16:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.166+0000 7f67162a7640 1 --2- 172.21.15.3:0/3950689837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67101083a0 0x7f67101087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.166+0000 7f67162a7640 1 --2- 172.21.15.3:0/3950689837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6710108cf0 0x7f671010f520 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.166+0000 7f67162a7640 1 --2- 172.21.15.3:0/3950689837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6710102a60 0x7f6710102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.166+0000 7f67162a7640 1 -- 172.21.15.3:0/3950689837 >> 172.21.15.3:0/3950689837 conn(0x7f6710079730 msgr2=0x7f6710079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.166+0000 7f67162a7640 1 -- 172.21.15.3:0/3950689837 shutdown_connections 2024-09-21T06:16:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.166+0000 7f67162a7640 1 -- 172.21.15.3:0/3950689837 wait complete. 2024-09-21T06:16:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.167+0000 7f67162a7640 1 Processor -- start 2024-09-21T06:16:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.167+0000 7f67162a7640 1 -- start start 2024-09-21T06:16:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.167+0000 7f67162a7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6710102a60 0x7f6710199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f670ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6710102a60 0x7f6710199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f670ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6710102a60 0x7f6710199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46930/0 (socket says 172.21.15.3:46930) 2024-09-21T06:16:20.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f670ffff640 1 -- 172.21.15.3:0/336311008 learned_addr learned my addr 172.21.15.3:0/336311008 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:20.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f67162a7640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67101083a0 0x7f67101998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f67162a7640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6710108cf0 0x7f671019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f67162a7640 1 -- 172.21.15.3:0/336311008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67101129b0 con 0x7f6710102a60 2024-09-21T06:16:20.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f67162a7640 1 -- 172.21.15.3:0/336311008 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6710112830 con 0x7f67101083a0 2024-09-21T06:16:20.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f67162a7640 1 -- 172.21.15.3:0/336311008 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6710112b30 con 0x7f6710108cf0 2024-09-21T06:16:20.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f670f7fe640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67101083a0 0x7f67101998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f6714844640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6710108cf0 0x7f671019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f670ffff640 1 -- 172.21.15.3:0/336311008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6710108cf0 msgr2=0x7f671019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f670ffff640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6710108cf0 0x7f671019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f670ffff640 1 -- 172.21.15.3:0/336311008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67101083a0 msgr2=0x7f67101998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f670ffff640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67101083a0 0x7f67101998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f670ffff640 1 -- 172.21.15.3:0/336311008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671019f2b0 con 0x7f6710102a60 2024-09-21T06:16:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.168+0000 7f670f7fe640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67101083a0 0x7f67101998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.169+0000 7f670ffff640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6710102a60 0x7f6710199340 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f670000bec0 tx=0x7f670000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:20.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.169+0000 7f670d7fa640 1 -- 172.21.15.3:0/336311008 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700018070 con 0x7f6710102a60 2024-09-21T06:16:20.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.169+0000 7f670d7fa640 1 -- 172.21.15.3:0/336311008 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6700016070 con 0x7f6710102a60 2024-09-21T06:16:20.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.169+0000 7f67162a7640 1 -- 172.21.15.3:0/336311008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671019f5a0 con 0x7f6710102a60 2024-09-21T06:16:20.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.169+0000 7f67162a7640 1 -- 172.21.15.3:0/336311008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671006ae60 con 0x7f6710102a60 2024-09-21T06:16:20.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.170+0000 7f670d7fa640 1 -- 172.21.15.3:0/336311008 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67000047a0 con 0x7f6710102a60 2024-09-21T06:16:20.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.173+0000 7f671c81b740 1 Processor -- start 2024-09-21T06:16:20.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.173+0000 7f671c81b740 1 -- start start 2024-09-21T06:16:20.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.174+0000 7f671c81b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd752e350 0x563dd78ddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.174+0000 7f671c81b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd7505490 0x563dd78de180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.174+0000 7f671c81b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 0x563dd78de6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.174+0000 7f670d7fa640 1 -- 172.21.15.3:0/336311008 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7f6700004940 con 0x7f6710102a60 2024-09-21T06:16:20.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.174+0000 7f671c81b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563dd784be60 con 0x563dd750d670 2024-09-21T06:16:20.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.174+0000 7f671c81b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563dd783b290 con 0x563dd752e350 2024-09-21T06:16:20.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.174+0000 7f671c81b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563dd783f0a0 con 0x563dd7505490 2024-09-21T06:16:20.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.174+0000 7f670ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd7505490 0x563dd78de180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.174+0000 7f6714844640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd752e350 0x563dd78ddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.175+0000 7f670f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 0x563dd78de6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.175+0000 7f670f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 0x563dd78de6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46936/0 (socket says 172.21.15.3:46936) 2024-09-21T06:16:20.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.175+0000 7f670f7fe640 1 -- 172.21.15.3:0/1281685850 learned_addr learned my addr 172.21.15.3:0/1281685850 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:20.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.174+0000 7f670ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd7505490 0x563dd78de180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:56862/0 (socket says 172.21.15.3:56862) 2024-09-21T06:16:20.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.175+0000 7f670f7fe640 1 -- 172.21.15.3:0/1281685850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd7505490 msgr2=0x563dd78de180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:20.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.175+0000 7f670f7fe640 1 --2- 172.21.15.3:0/1281685850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd7505490 0x563dd78de180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.175+0000 7f670f7fe640 1 -- 172.21.15.3:0/1281685850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd752e350 msgr2=0x563dd78ddc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:20.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.175+0000 7f670f7fe640 1 --2- 172.21.15.3:0/1281685850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd752e350 0x563dd78ddc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.175+0000 7f670f7fe640 1 -- 172.21.15.3:0/1281685850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dd7599930 con 0x563dd750d670 2024-09-21T06:16:20.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.175+0000 7f670f7fe640 1 --2- 172.21.15.3:0/1281685850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 0x563dd78de6c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6704009f80 tx=0x7f6704010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3f21e2cc97b7c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:20.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.176+0000 7f66f67fc640 1 -- 172.21.15.3:0/1281685850 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704005220 con 0x563dd750d670 2024-09-21T06:16:20.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.176+0000 7f670d7fa640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f66f80b68e0 0x7f66f80b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.176+0000 7f66f67fc640 1 -- 172.21.15.3:0/1281685850 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6704015070 con 0x563dd750d670 2024-09-21T06:16:20.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.176+0000 7f66f67fc640 1 -- 172.21.15.3:0/1281685850 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704011880 con 0x563dd750d670 2024-09-21T06:16:20.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.176+0000 7f6714844640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f66f80b68e0 0x7f66f80b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.177+0000 7f670d7fa640 1 -- 172.21.15.3:0/336311008 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f67000e6980 con 0x7f6710102a60 2024-09-21T06:16:20.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.177+0000 7f671c81b740 1 -- 172.21.15.3:0/1281685850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 msgr2=0x563dd78de6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:20.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.177+0000 7f6714844640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f66f80b68e0 0x7f66f80b8df0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6708006010 tx=0x7f6708009960 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:20.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.177+0000 7f671c81b740 1 --2- 172.21.15.3:0/1281685850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 0x563dd78de6c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6704009f80 tx=0x7f6704010860 comp rx=0 tx=0).stop 2024-09-21T06:16:20.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.178+0000 7f671c81b740 1 -- 172.21.15.3:0/1281685850 shutdown_connections 2024-09-21T06:16:20.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.178+0000 7f671c81b740 1 --2- 172.21.15.3:0/1281685850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd7505490 0x563dd78de180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.178+0000 7f671c81b740 1 --2- 172.21.15.3:0/1281685850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 0x563dd78de6c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.178+0000 7f671c81b740 1 --2- 172.21.15.3:0/1281685850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd752e350 0x563dd78ddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.178+0000 7f671c81b740 1 -- 172.21.15.3:0/1281685850 >> 172.21.15.3:0/1281685850 conn(0x563dd7522740 msgr2=0x563dd78dbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:20.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.178+0000 7f671c81b740 1 -- 172.21.15.3:0/1281685850 shutdown_connections 2024-09-21T06:16:20.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.178+0000 7f671c81b740 1 -- 172.21.15.3:0/1281685850 wait complete. 2024-09-21T06:16:20.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.179+0000 7f671c81b740 1 -- start start 2024-09-21T06:16:20.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.179+0000 7f671c81b740 1 Processor -- start 2024-09-21T06:16:20.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.180+0000 7f671c81b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd7505490 0x563dd78e9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.180+0000 7f671c81b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd750d670 0x563dd78e9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.180+0000 7f671c81b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd752e350 0x563dd78f5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.180+0000 7f670f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd750d670 0x563dd78e9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.180+0000 7f6714844640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd752e350 0x563dd78f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.180+0000 7f670ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd7505490 0x563dd78e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.180+0000 7f670f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd750d670 0x563dd78e9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40642/0 (socket says 172.21.15.3:40642) 2024-09-21T06:16:20.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.180+0000 7f6714844640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd752e350 0x563dd78f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46940/0 (socket says 172.21.15.3:46940) 2024-09-21T06:16:20.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.180+0000 7f670ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd7505490 0x563dd78e9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:56866/0 (socket says 172.21.15.3:56866) 2024-09-21T06:16:20.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.180+0000 7f670f7fe640 1 -- 172.21.15.3:0/3417666809 learned_addr learned my addr 172.21.15.3:0/3417666809 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:20.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.181+0000 7f671c81b740 1 -- 172.21.15.3:0/3417666809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563dd769c890 con 0x563dd752e350 2024-09-21T06:16:20.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.181+0000 7f671c81b740 1 -- 172.21.15.3:0/3417666809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563dd77d5d40 con 0x563dd750d670 2024-09-21T06:16:20.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.181+0000 7f671c81b740 1 -- 172.21.15.3:0/3417666809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563dd78ec420 con 0x563dd7505490 2024-09-21T06:16:20.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.181+0000 7f671c81b740 10 client.0 _ll_register_callbacks cb 0x563dd7839c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:20.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.181+0000 7f6714844640 1 -- 172.21.15.3:0/3417666809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd7505490 msgr2=0x563dd78e9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:20.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.181+0000 7f6714844640 1 --2- 172.21.15.3:0/3417666809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd7505490 0x563dd78e9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.181+0000 7f6714844640 1 -- 172.21.15.3:0/3417666809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd750d670 msgr2=0x563dd78e9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:20.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.181+0000 7f6714844640 1 --2- 172.21.15.3:0/3417666809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd750d670 0x563dd78e9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.181+0000 7f6714844640 1 -- 172.21.15.3:0/3417666809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dd78f5d00 con 0x563dd752e350 2024-09-21T06:16:20.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.182+0000 7f670ffff640 1 --2- 172.21.15.3:0/3417666809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd7505490 0x563dd78e9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:20.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.182+0000 7f6714844640 1 --2- 172.21.15.3:0/3417666809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd752e350 0x563dd78f5770 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6708009a60 tx=0x7f67080095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417b28f3738f7901 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:20.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.182+0000 7f66ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:20.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.182+0000 7f66ef7fe640 1 -- 172.21.15.3:0/3417666809 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670804f070 con 0x563dd752e350 2024-09-21T06:16:20.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.183+0000 7f66ef7fe640 1 -- 172.21.15.3:0/3417666809 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f670802f810 con 0x563dd752e350 2024-09-21T06:16:20.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.183+0000 7f671c81b740 1 -- 172.21.15.3:0/3417666809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6704015070 con 0x563dd752e350 2024-09-21T06:16:20.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.183+0000 7f66ef7fe640 1 -- 172.21.15.3:0/3417666809 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f670802fa10 con 0x563dd752e350 2024-09-21T06:16:20.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.183+0000 7f671c81b740 10 client.15927 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.183+0000 7f671c81b740 1 -- 172.21.15.3:0/3417666809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563dd78f8610 con 0x563dd752e350 2024-09-21T06:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.184+0000 7f66ef7fe640 1 -- 172.21.15.3:0/3417666809 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6708036040 con 0x563dd752e350 2024-09-21T06:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.185+0000 7f66ef7fe640 1 -- 172.21.15.3:0/3417666809 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6708044040 con 0x563dd752e350 2024-09-21T06:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.185+0000 7f671c81b740 10 client.15927 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.185+0000 7f671c81b740 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.185+0000 7f671c81b740 20 client.15927 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.185+0000 7f671c81b740 1 --2- 172.21.15.3:0/3417666809 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x563dd74eccd0 0x563dd78fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.185+0000 7f671c81b740 4 client.15927 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.185+0000 7f671c81b740 1 -- 172.21.15.3:0/3417666809 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6704015070 con 0x563dd74eccd0 2024-09-21T06:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.185+0000 7f670ffff640 1 --2- 172.21.15.3:0/3417666809 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x563dd74eccd0 0x563dd78fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.185+0000 7f670ffff640 1 --2- 172.21.15.3:0/3417666809 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x563dd74eccd0 0x563dd78fbe50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95604b08743d6eb9 server_cookie=6b2f524732c9d3b6 in_seq=0 out_seq=0 2024-09-21T06:16:20.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.186+0000 7f66ef7fe640 10 client.15927 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:20.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.186+0000 7f66ef7fe640 1 -- 172.21.15.3:0/3417666809 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f6704015070 con 0x563dd74eccd0 2024-09-21T06:16:20.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.186+0000 7f66ef7fe640 10 client.15927 handle_command_reply: tid=0 2024-09-21T06:16:20.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.186+0000 7f671c81b740 1 client.15927 shutdown 2024-09-21T06:16:20.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.187+0000 7f671c81b740 1 -- 172.21.15.3:0/3417666809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd752e350 msgr2=0x563dd78f5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:20.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.187+0000 7f671c81b740 1 --2- 172.21.15.3:0/3417666809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd752e350 0x563dd78f5770 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6708009a60 tx=0x7f67080095c0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.187+0000 7f671c81b740 1 -- 172.21.15.3:0/3417666809 shutdown_connections 2024-09-21T06:16:20.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.187+0000 7f671c81b740 1 --2- 172.21.15.3:0/3417666809 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x563dd74eccd0 0x563dd78fbe50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.187+0000 7f671c81b740 1 --2- 172.21.15.3:0/3417666809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd752e350 0x563dd78f5770 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.187+0000 7f671c81b740 1 --2- 172.21.15.3:0/3417666809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd750d670 0x563dd78e9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.187+0000 7f671c81b740 1 --2- 172.21.15.3:0/3417666809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd7505490 0x563dd78e9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.187+0000 7f671c81b740 1 -- 172.21.15.3:0/3417666809 >> 172.21.15.3:0/3417666809 conn(0x563dd7522740 msgr2=0x563dd78e0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:20.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.187+0000 7f66ef7fe640 0 client.15927 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:20.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.187+0000 7f671c81b740 1 -- 172.21.15.3:0/3417666809 shutdown_connections 2024-09-21T06:16:20.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.187+0000 7f671c81b740 1 -- 172.21.15.3:0/3417666809 wait complete. 2024-09-21T06:16:20.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.187+0000 7f671c81b740 20 client.15927 trim_cache size 0 max 16384 2024-09-21T06:16:20.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.223+0000 7f671c81b740 1 Processor -- start 2024-09-21T06:16:20.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.223+0000 7f671c81b740 1 -- start start 2024-09-21T06:16:20.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.224+0000 7f671c81b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd7505490 0x563dd78db400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.224+0000 7f671c81b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 0x563dd78db940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.224+0000 7f671c81b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd752e350 0x563dd78e6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.224+0000 7f671c81b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc00e700 con 0x563dd750d670 2024-09-21T06:16:20.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.224+0000 7f671c81b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc010240 con 0x563dd7505490 2024-09-21T06:16:20.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.224+0000 7f670f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 0x563dd78db940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.224+0000 7f670f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 0x563dd78db940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46946/0 (socket says 172.21.15.3:46946) 2024-09-21T06:16:20.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.224+0000 7f670f7fe640 1 -- 172.21.15.3:0/2748845496 learned_addr learned my addr 172.21.15.3:0/2748845496 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:20.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.224+0000 7f671c81b740 1 -- 172.21.15.3:0/2748845496 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc00edd0 con 0x563dd752e350 2024-09-21T06:16:20.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.224+0000 7f6714844640 1 --2- 172.21.15.3:0/2748845496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd752e350 0x563dd78e6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.225+0000 7f670ffff640 1 --2- 172.21.15.3:0/2748845496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd7505490 0x563dd78db400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.225+0000 7f670f7fe640 1 -- 172.21.15.3:0/2748845496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd752e350 msgr2=0x563dd78e6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:20.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.225+0000 7f670f7fe640 1 --2- 172.21.15.3:0/2748845496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd752e350 0x563dd78e6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.225+0000 7f670f7fe640 1 -- 172.21.15.3:0/2748845496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd7505490 msgr2=0x563dd78db400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:20.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.225+0000 7f670f7fe640 1 --2- 172.21.15.3:0/2748845496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd7505490 0x563dd78db400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.225+0000 7f670f7fe640 1 -- 172.21.15.3:0/2748845496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dd78dc890 con 0x563dd750d670 2024-09-21T06:16:20.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.225+0000 7f670f7fe640 1 --2- 172.21.15.3:0/2748845496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 0x563dd78db940 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6704005780 tx=0x7f6704009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f73684505b8cb0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:20.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.226+0000 7f66f5ffb640 1 -- 172.21.15.3:0/2748845496 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704004020 con 0x563dd750d670 2024-09-21T06:16:20.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.226+0000 7f66f5ffb640 1 -- 172.21.15.3:0/2748845496 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6704015030 con 0x563dd750d670 2024-09-21T06:16:20.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.226+0000 7f66f5ffb640 1 -- 172.21.15.3:0/2748845496 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704003e30 con 0x563dd750d670 2024-09-21T06:16:20.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.227+0000 7f671c81b740 1 -- 172.21.15.3:0/2748845496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 msgr2=0x563dd78db940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:20.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.227+0000 7f671c81b740 1 --2- 172.21.15.3:0/2748845496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 0x563dd78db940 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6704005780 tx=0x7f6704009f80 comp rx=0 tx=0).stop 2024-09-21T06:16:20.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.228+0000 7f671c81b740 1 -- 172.21.15.3:0/2748845496 shutdown_connections 2024-09-21T06:16:20.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.228+0000 7f671c81b740 1 --2- 172.21.15.3:0/2748845496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd752e350 0x563dd78e6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.228+0000 7f671c81b740 1 --2- 172.21.15.3:0/2748845496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd750d670 0x563dd78db940 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.228+0000 7f671c81b740 1 --2- 172.21.15.3:0/2748845496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd7505490 0x563dd78db400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.228+0000 7f671c81b740 1 -- 172.21.15.3:0/2748845496 >> 172.21.15.3:0/2748845496 conn(0x563dd7522740 msgr2=0x563dd78e7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:20.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.228+0000 7f671c81b740 1 -- 172.21.15.3:0/2748845496 shutdown_connections 2024-09-21T06:16:20.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.228+0000 7f671c81b740 1 -- 172.21.15.3:0/2748845496 wait complete. 2024-09-21T06:16:20.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.228+0000 7f671c81b740 1 -- start start 2024-09-21T06:16:20.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.228+0000 7f671c81b740 1 Processor -- start 2024-09-21T06:16:20.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.229+0000 7f671c81b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd7505490 0x563dd78f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.229+0000 7f671c81b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd750d670 0x563dd78f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.229+0000 7f671c81b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd752e350 0x563dd78ec710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.229+0000 7f671c81b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc00fb70 con 0x563dd752e350 2024-09-21T06:16:20.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.229+0000 7f671c81b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc00f4a0 con 0x563dd7505490 2024-09-21T06:16:20.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.229+0000 7f671c81b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc00c6e0 con 0x563dd750d670 2024-09-21T06:16:20.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.229+0000 7f671c81b740 10 client.0 _ll_register_callbacks cb 0x7f6708009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:20.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.229+0000 7f670ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd7505490 0x563dd78f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.229+0000 7f670ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd7505490 0x563dd78f1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40668/0 (socket says 172.21.15.3:40668) 2024-09-21T06:16:20.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.229+0000 7f670ffff640 1 -- 172.21.15.3:0/291658329 learned_addr learned my addr 172.21.15.3:0/291658329 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:20.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.229+0000 7f6714844640 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd752e350 0x563dd78ec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.230+0000 7f670f7fe640 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd750d670 0x563dd78f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.230+0000 7f670ffff640 1 -- 172.21.15.3:0/291658329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd750d670 msgr2=0x563dd78f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:20.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.230+0000 7f670ffff640 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd750d670 0x563dd78f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.230+0000 7f670ffff640 1 -- 172.21.15.3:0/291658329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd752e350 msgr2=0x563dd78ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:20.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.230+0000 7f670ffff640 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd752e350 0x563dd78ec710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.230+0000 7f670ffff640 1 -- 172.21.15.3:0/291658329 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dd78ecea0 con 0x563dd7505490 2024-09-21T06:16:20.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.230+0000 7f6714844640 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd752e350 0x563dd78ec710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:20.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.231+0000 7f670ffff640 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd7505490 0x563dd78f1a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f67000b0750 tx=0x7f67000e8d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efd7567f85860782 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:20.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.231+0000 7f66ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:16:20.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.231+0000 7f66ef7fe640 1 -- 172.21.15.3:0/291658329 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67000b5070 con 0x563dd7505490 2024-09-21T06:16:20.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.231+0000 7f66ef7fe640 1 -- 172.21.15.3:0/291658329 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f67000afcb0 con 0x563dd7505490 2024-09-21T06:16:20.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.231+0000 7f671c81b740 1 -- 172.21.15.3:0/291658329 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6704015030 con 0x563dd7505490 2024-09-21T06:16:20.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.231+0000 7f66ef7fe640 1 -- 172.21.15.3:0/291658329 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f67000c7050 con 0x563dd7505490 2024-09-21T06:16:20.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.231+0000 7f671c81b740 10 client.25517 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:20.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.231+0000 7f671c81b740 1 -- 172.21.15.3:0/291658329 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563dd7967340 con 0x563dd7505490 2024-09-21T06:16:20.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.231+0000 7f66ef7fe640 1 -- 172.21.15.3:0/291658329 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67000e44b0 con 0x563dd7505490 2024-09-21T06:16:20.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.232+0000 7f66ef7fe640 1 -- 172.21.15.3:0/291658329 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f67000af720 con 0x563dd7505490 2024-09-21T06:16:20.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.232+0000 7f671c81b740 10 client.25517 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:20.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.232+0000 7f671c81b740 10 client.25517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:20.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.232+0000 7f671c81b740 20 client.25517 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:20.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.232+0000 7f671c81b740 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x563dd74c2080 0x563dd7969ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:20.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.232+0000 7f671c81b740 4 client.25517 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:16:20.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.232+0000 7f671c81b740 1 -- 172.21.15.3:0/291658329 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f6704015030 con 0x563dd74c2080 2024-09-21T06:16:20.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.232+0000 7f670f7fe640 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x563dd74c2080 0x563dd7969ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:20.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.232+0000 7f66ef7fe640 10 client.25517 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:20.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.233+0000 7f670f7fe640 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x563dd74c2080 0x563dd7969ad0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60056d7a0898be01 server_cookie=59edee673fc74343 in_seq=0 out_seq=0 2024-09-21T06:16:20.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.235+0000 7f66ef7fe640 1 -- 172.21.15.3:0/291658329 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f6704015030 con 0x563dd74c2080 2024-09-21T06:16:20.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.235+0000 7f66ef7fe640 10 client.25517 handle_command_reply: tid=0 2024-09-21T06:16:20.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.235+0000 7f671c81b740 1 client.25517 shutdown 2024-09-21T06:16:20.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.235+0000 7f671c81b740 1 -- 172.21.15.3:0/291658329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd7505490 msgr2=0x563dd78f1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:20.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.235+0000 7f671c81b740 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd7505490 0x563dd78f1a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f67000b0750 tx=0x7f67000e8d50 comp rx=0 tx=0).stop 2024-09-21T06:16:20.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.236+0000 7f671c81b740 1 -- 172.21.15.3:0/291658329 shutdown_connections 2024-09-21T06:16:20.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.236+0000 7f671c81b740 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x563dd74c2080 0x563dd7969ad0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.236+0000 7f66ef7fe640 0 client.25517 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:20.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.236+0000 7f671c81b740 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dd750d670 0x563dd78f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.236+0000 7f671c81b740 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563dd752e350 0x563dd78ec710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.240 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:16:20.240 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:16:20.240 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "640ef6d2-da94-4f3b-b441-02e50117a2f6", 2024-09-21T06:16:20.240 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:16:20.240 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:16:20.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.236+0000 7f671c81b740 1 --2- 172.21.15.3:0/291658329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563dd7505490 0x563dd78f1a20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.236+0000 7f671c81b740 1 -- 172.21.15.3:0/291658329 >> 172.21.15.3:0/291658329 conn(0x563dd7522740 msgr2=0x563dd78e7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:20.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.236+0000 7f671c81b740 1 -- 172.21.15.3:0/291658329 shutdown_connections 2024-09-21T06:16:20.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.236+0000 7f671c81b740 1 -- 172.21.15.3:0/291658329 wait complete. 2024-09-21T06:16:20.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.236+0000 7f671c81b740 20 client.25517 trim_cache size 0 max 16384 2024-09-21T06:16:20.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.237+0000 7f67162a7640 1 -- 172.21.15.3:0/336311008 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f66f80b68e0 msgr2=0x7f66f80b8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:20.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.237+0000 7f67162a7640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f66f80b68e0 0x7f66f80b8df0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6708006010 tx=0x7f6708009960 comp rx=0 tx=0).stop 2024-09-21T06:16:20.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.237+0000 7f67162a7640 1 -- 172.21.15.3:0/336311008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6710102a60 msgr2=0x7f6710199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:20.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.237+0000 7f67162a7640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6710102a60 0x7f6710199340 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f670000bec0 tx=0x7f670000dd20 comp rx=0 tx=0).stop 2024-09-21T06:16:20.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.237+0000 7f67162a7640 1 -- 172.21.15.3:0/336311008 shutdown_connections 2024-09-21T06:16:20.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.237+0000 7f67162a7640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f66f80b68e0 0x7f66f80b8df0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.237+0000 7f67162a7640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6710108cf0 0x7f671019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.237+0000 7f67162a7640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67101083a0 0x7f67101998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.237+0000 7f67162a7640 1 --2- 172.21.15.3:0/336311008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6710102a60 0x7f6710199340 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:20.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.237+0000 7f67162a7640 1 -- 172.21.15.3:0/336311008 >> 172.21.15.3:0/336311008 conn(0x7f6710079730 msgr2=0x7f6710104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:20.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.237+0000 7f67162a7640 1 -- 172.21.15.3:0/336311008 shutdown_connections 2024-09-21T06:16:20.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:20.237+0000 7f67162a7640 1 -- 172.21.15.3:0/336311008 wait complete. 2024-09-21T06:16:21.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:20 smithi003 ceph-mon[31227]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T06:16:21.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:16:21.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:20 smithi137 ceph-mon[34464]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T06:16:21.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:20 smithi204 ceph-mon[34659]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T06:16:21.443 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.018+0000 7fa5b29a9640 1 -- 172.21.15.3:0/1823026704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5ac102ac0 msgr2=0x7fa5ac102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.018+0000 7fa5b29a9640 1 --2- 172.21.15.3:0/1823026704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5ac102ac0 0x7fa5ac102ed0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009900 tx=0x7fa59c02a410 comp rx=0 tx=0).stop 2024-09-21T06:16:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.018+0000 7fa5b29a9640 1 -- 172.21.15.3:0/1823026704 shutdown_connections 2024-09-21T06:16:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.018+0000 7fa5b29a9640 1 --2- 172.21.15.3:0/1823026704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac1083b0 0x7fa5ac1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.018+0000 7fa5b29a9640 1 --2- 172.21.15.3:0/1823026704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5ac108d00 0x7fa5ac10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.018+0000 7fa5b29a9640 1 --2- 172.21.15.3:0/1823026704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5ac102ac0 0x7fa5ac102ed0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.018+0000 7fa5b29a9640 1 -- 172.21.15.3:0/1823026704 >> 172.21.15.3:0/1823026704 conn(0x7fa5ac079790 msgr2=0x7fa5ac079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.018+0000 7fa5b29a9640 1 -- 172.21.15.3:0/1823026704 shutdown_connections 2024-09-21T06:16:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.018+0000 7fa5b29a9640 1 -- 172.21.15.3:0/1823026704 wait complete. 2024-09-21T06:16:22.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.019+0000 7fa5b29a9640 1 Processor -- start 2024-09-21T06:16:22.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.019+0000 7fa5b29a9640 1 -- start start 2024-09-21T06:16:22.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.020+0000 7fa5b29a9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5ac102ac0 0x7fa5ac19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.020+0000 7fa5b29a9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5ac1083b0 0x7fa5ac19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.020+0000 7fa5b29a9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac108d00 0x7fa5ac19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.020+0000 7fa5b29a9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1129c0 con 0x7fa5ac1083b0 2024-09-21T06:16:22.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.020+0000 7fa5b29a9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac112840 con 0x7fa5ac102ac0 2024-09-21T06:16:22.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.020+0000 7fa5b29a9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac112b40 con 0x7fa5ac108d00 2024-09-21T06:16:22.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.020+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5ac1083b0 0x7fa5ac19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.020+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5ac1083b0 0x7fa5ac19b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46982/0 (socket says 172.21.15.3:46982) 2024-09-21T06:16:22.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.020+0000 7fa5a3fff640 1 -- 172.21.15.3:0/3199785528 learned_addr learned my addr 172.21.15.3:0/3199785528 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:22.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.020+0000 7fa5b0f46640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac108d00 0x7fa5ac19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.020+0000 7fa5abfff640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5ac102ac0 0x7fa5ac19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.021+0000 7fa5b0f46640 1 -- 172.21.15.3:0/3199785528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5ac102ac0 msgr2=0x7fa5ac19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:22.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.021+0000 7fa5b0f46640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5ac102ac0 0x7fa5ac19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.021+0000 7fa5b0f46640 1 -- 172.21.15.3:0/3199785528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5ac1083b0 msgr2=0x7fa5ac19b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:22.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.021+0000 7fa5b0f46640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5ac1083b0 0x7fa5ac19b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.021+0000 7fa5b0f46640 1 -- 172.21.15.3:0/3199785528 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac1a65b0 con 0x7fa5ac108d00 2024-09-21T06:16:22.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.021+0000 7fa5a3fff640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5ac1083b0 0x7fa5ac19b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:22.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.021+0000 7fa5b0f46640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac108d00 0x7fa5ac19bd70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400da10 tx=0x7fa5a400ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:22.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.021+0000 7fa5a9ffb640 1 -- 172.21.15.3:0/3199785528 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a400bd70 con 0x7fa5ac108d00 2024-09-21T06:16:22.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.021+0000 7fa5a9ffb640 1 -- 172.21.15.3:0/3199785528 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5a4003e20 con 0x7fa5ac108d00 2024-09-21T06:16:22.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.021+0000 7fa5b29a9640 1 -- 172.21.15.3:0/3199785528 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ac1a68a0 con 0x7fa5ac108d00 2024-09-21T06:16:22.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.021+0000 7fa5a9ffb640 1 -- 172.21.15.3:0/3199785528 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40106c0 con 0x7fa5ac108d00 2024-09-21T06:16:22.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.021+0000 7fa5b29a9640 1 -- 172.21.15.3:0/3199785528 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ac1a6d90 con 0x7fa5ac108d00 2024-09-21T06:16:22.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.024+0000 7fa5a9ffb640 1 -- 172.21.15.3:0/3199785528 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7fa5a4004270 con 0x7fa5ac108d00 2024-09-21T06:16:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.026+0000 7fa5a9ffb640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa5980b6710 0x7fa5980b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.026+0000 7fa5abfff640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa5980b6710 0x7fa5980b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.026+0000 7fa5a9ffb640 1 -- 172.21.15.3:0/3199785528 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fa5a40e7c30 con 0x7fa5ac108d00 2024-09-21T06:16:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.026+0000 7fa5b8f1d740 1 Processor -- start 2024-09-21T06:16:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.026+0000 7fa5abfff640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa5980b6710 0x7fa5980b8c20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa59c000c00 tx=0x7fa59c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:22.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5b8f1d740 1 -- start start 2024-09-21T06:16:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5b8f1d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25ae070 0x5555b2955270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5b8f1d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25851b0 0x5555b29557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5b8f1d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 0x5555b2955cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5b8f1d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5555b28cbe50 con 0x5555b25851b0 2024-09-21T06:16:22.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5b8f1d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555b28bb280 con 0x5555b25ae070 2024-09-21T06:16:22.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5b8f1d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5555b28bf090 con 0x5555b258d390 2024-09-21T06:16:22.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25851b0 0x5555b29557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25851b0 0x5555b29557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46984/0 (socket says 172.21.15.3:46984) 2024-09-21T06:16:22.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5a3fff640 1 -- 172.21.15.3:0/3739210958 learned_addr learned my addr 172.21.15.3:0/3739210958 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:22.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5abfff640 1 --2- 172.21.15.3:0/3739210958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25ae070 0x5555b2955270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5a3fff640 1 -- 172.21.15.3:0/3739210958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 msgr2=0x5555b2955cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:22.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5b0f46640 1 --2- 172.21.15.3:0/3739210958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 0x5555b2955cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5a3fff640 1 --2- 172.21.15.3:0/3739210958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 0x5555b2955cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5a3fff640 1 -- 172.21.15.3:0/3739210958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25ae070 msgr2=0x5555b2955270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:22.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5a3fff640 1 --2- 172.21.15.3:0/3739210958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25ae070 0x5555b2955270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.027+0000 7fa5a3fff640 1 -- 172.21.15.3:0/3739210958 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555b2619650 con 0x5555b25851b0 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.028+0000 7fa5a3fff640 1 --2- 172.21.15.3:0/3739210958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25851b0 0x5555b29557b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa594011540 tx=0x7fa594011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e00c295b008bc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.028+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3739210958 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594005020 con 0x5555b25851b0 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.028+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3739210958 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5940047a0 con 0x5555b25851b0 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.028+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3739210958 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59400f880 con 0x5555b25851b0 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.029+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/3739210958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25851b0 msgr2=0x5555b29557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.029+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/3739210958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25851b0 0x5555b29557b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa594011540 tx=0x7fa594011a00 comp rx=0 tx=0).stop 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.029+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/3739210958 shutdown_connections 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.029+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/3739210958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 0x5555b2955cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.029+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/3739210958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25851b0 0x5555b29557b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.029+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/3739210958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25ae070 0x5555b2955270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.029+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/3739210958 >> 172.21.15.3:0/3739210958 conn(0x5555b25a2460 msgr2=0x5555b2953160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.029+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/3739210958 shutdown_connections 2024-09-21T06:16:22.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.029+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/3739210958 wait complete. 2024-09-21T06:16:22.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.029+0000 7fa5b8f1d740 1 -- start start 2024-09-21T06:16:22.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.030+0000 7fa5b8f1d740 1 Processor -- start 2024-09-21T06:16:22.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.030+0000 7fa5b8f1d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25851b0 0x5555b2964c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.030+0000 7fa5b8f1d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 0x5555b2965190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.030+0000 7fa5b8f1d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25ae070 0x5555b29656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.030+0000 7fa5b8f1d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5555b271c700 con 0x5555b25ae070 2024-09-21T06:16:22.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.030+0000 7fa5b8f1d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555b2856110 con 0x5555b25851b0 2024-09-21T06:16:22.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.030+0000 7fa5b8f1d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5555b2967db0 con 0x5555b258d390 2024-09-21T06:16:22.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.030+0000 7fa5b8f1d740 10 client.0 _ll_register_callbacks cb 0x5555b286f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:22.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.030+0000 7fa5b0f46640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25ae070 0x5555b29656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.030+0000 7fa5b0f46640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25ae070 0x5555b29656d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46988/0 (socket says 172.21.15.3:46988) 2024-09-21T06:16:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.030+0000 7fa5b0f46640 1 -- 172.21.15.3:0/827965772 learned_addr learned my addr 172.21.15.3:0/827965772 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.030+0000 7fa5abfff640 1 --2- 172.21.15.3:0/827965772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25851b0 0x5555b2964c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.030+0000 7fa5a3fff640 1 --2- 172.21.15.3:0/827965772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 0x5555b2965190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa5b0f46640 1 -- 172.21.15.3:0/827965772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 msgr2=0x5555b2965190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa5b0f46640 1 --2- 172.21.15.3:0/827965772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 0x5555b2965190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa5b0f46640 1 -- 172.21.15.3:0/827965772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25851b0 msgr2=0x5555b2964c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa5b0f46640 1 --2- 172.21.15.3:0/827965772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25851b0 0x5555b2964c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa5b0f46640 1 -- 172.21.15.3:0/827965772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555b2883c40 con 0x5555b25ae070 2024-09-21T06:16:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa5b0f46640 1 --2- 172.21.15.3:0/827965772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25ae070 0x5555b29656d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40e5390 tx=0x7fa5a40ad3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a245237f4928f359 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa58b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa58b7fe640 1 -- 172.21.15.3:0/827965772 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40b2020 con 0x5555b25ae070 2024-09-21T06:16:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa58b7fe640 1 -- 172.21.15.3:0/827965772 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fa5a40e55b0 con 0x5555b25ae070 2024-09-21T06:16:22.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/827965772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5940047a0 con 0x5555b25ae070 2024-09-21T06:16:22.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa58b7fe640 1 -- 172.21.15.3:0/827965772 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa5a40e57b0 con 0x5555b25ae070 2024-09-21T06:16:22.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa5b8f1d740 10 client.15945 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:22.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/827965772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555b2973a50 con 0x5555b25ae070 2024-09-21T06:16:22.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.031+0000 7fa58b7fe640 1 -- 172.21.15.3:0/827965772 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5940047a0 con 0x5555b25ae070 2024-09-21T06:16:22.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.032+0000 7fa58b7fe640 1 -- 172.21.15.3:0/827965772 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa5a4024060 con 0x5555b25ae070 2024-09-21T06:16:22.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.032+0000 7fa5b8f1d740 10 client.15945 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:22.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.032+0000 7fa5b8f1d740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:22.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.032+0000 7fa5b8f1d740 20 client.15945 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.032+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/827965772 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5555b256c9f0 0x5555b2976f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.032+0000 7fa5b8f1d740 4 client.15945 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.032+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/827965772 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555b28cdda0 con 0x5555b256c9f0 2024-09-21T06:16:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.032+0000 7fa5abfff640 1 --2- 172.21.15.3:0/827965772 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5555b256c9f0 0x5555b2976f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.033+0000 7fa5abfff640 1 --2- 172.21.15.3:0/827965772 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5555b256c9f0 0x5555b2976f80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6885f4cec33b1dd server_cookie=183bf7d73129454e in_seq=0 out_seq=0 2024-09-21T06:16:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.033+0000 7fa58b7fe640 10 client.15945 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.033+0000 7fa58b7fe640 1 -- 172.21.15.3:0/827965772 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5555b28cdda0 con 0x5555b256c9f0 2024-09-21T06:16:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.033+0000 7fa58b7fe640 10 client.15945 handle_command_reply: tid=0 2024-09-21T06:16:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.033+0000 7fa5b8f1d740 1 client.15945 shutdown 2024-09-21T06:16:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.034+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/827965772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25ae070 msgr2=0x5555b29656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.034+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/827965772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25ae070 0x5555b29656d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40e5390 tx=0x7fa5a40ad3d0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.034+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/827965772 shutdown_connections 2024-09-21T06:16:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.034+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/827965772 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5555b256c9f0 0x5555b2976f80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.034+0000 7fa58b7fe640 0 client.15945 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:22.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.034+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/827965772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 0x5555b2965190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.034+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/827965772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b25ae070 0x5555b29656d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.034+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/827965772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25851b0 0x5555b2964c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.034+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/827965772 >> 172.21.15.3:0/827965772 conn(0x5555b25a2460 msgr2=0x5555b2956c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:22.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.034+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/827965772 shutdown_connections 2024-09-21T06:16:22.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.034+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/827965772 wait complete. 2024-09-21T06:16:22.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.034+0000 7fa5b8f1d740 20 client.15945 trim_cache size 0 max 16384 2024-09-21T06:16:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.052+0000 7fa5b8f1d740 1 Processor -- start 2024-09-21T06:16:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.052+0000 7fa5b8f1d740 1 -- start start 2024-09-21T06:16:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.052+0000 7fa5b8f1d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b256c9f0 0x5555b2961e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.052+0000 7fa5b8f1d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25851b0 0x5555b2962380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.052+0000 7fa5b8f1d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 0x5555b297c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.052+0000 7fa5b8f1d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1ad480 con 0x5555b256c9f0 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.052+0000 7fa5b8f1d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1aefc0 con 0x5555b25851b0 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.052+0000 7fa5b8f1d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1adb50 con 0x5555b258d390 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.052+0000 7fa5abfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b256c9f0 0x5555b2961e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.052+0000 7fa5abfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b256c9f0 0x5555b2961e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47000/0 (socket says 172.21.15.3:47000) 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.052+0000 7fa5abfff640 1 -- 172.21.15.3:0/2877724545 learned_addr learned my addr 172.21.15.3:0/2877724545 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.052+0000 7fa5b0f46640 1 --2- 172.21.15.3:0/2877724545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 0x5555b297c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.052+0000 7fa5a3fff640 1 --2- 172.21.15.3:0/2877724545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25851b0 0x5555b2962380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.053+0000 7fa5abfff640 1 -- 172.21.15.3:0/2877724545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 msgr2=0x5555b297c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.053+0000 7fa5abfff640 1 --2- 172.21.15.3:0/2877724545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 0x5555b297c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.053+0000 7fa5abfff640 1 -- 172.21.15.3:0/2877724545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25851b0 msgr2=0x5555b2962380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.053+0000 7fa5abfff640 1 --2- 172.21.15.3:0/2877724545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25851b0 0x5555b2962380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.053+0000 7fa5abfff640 1 -- 172.21.15.3:0/2877724545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555b2965a10 con 0x5555b256c9f0 2024-09-21T06:16:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.053+0000 7fa5a3fff640 1 --2- 172.21.15.3:0/2877724545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25851b0 0x5555b2962380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:22.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.053+0000 7fa5abfff640 1 --2- 172.21.15.3:0/2877724545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b256c9f0 0x5555b2961e40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa59c02a930 tx=0x7fa59c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01fe6b43c4a9479 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:22.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.053+0000 7fa5a1ffb640 1 -- 172.21.15.3:0/2877724545 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c035040 con 0x5555b256c9f0 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.053+0000 7fa5a1ffb640 1 -- 172.21.15.3:0/2877724545 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa59c005040 con 0x5555b256c9f0 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.053+0000 7fa5a1ffb640 1 -- 172.21.15.3:0/2877724545 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0049b0 con 0x5555b256c9f0 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.054+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/2877724545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b256c9f0 msgr2=0x5555b2961e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.054+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/2877724545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b256c9f0 0x5555b2961e40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa59c02a930 tx=0x7fa59c040000 comp rx=0 tx=0).stop 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.054+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/2877724545 shutdown_connections 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.054+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/2877724545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b258d390 0x5555b297c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.054+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/2877724545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b25851b0 0x5555b2962380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.054+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/2877724545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b256c9f0 0x5555b2961e40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.054+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/2877724545 >> 172.21.15.3:0/2877724545 conn(0x5555b25a2460 msgr2=0x5555b2956c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.054+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/2877724545 shutdown_connections 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.054+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/2877724545 wait complete. 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.054+0000 7fa5b8f1d740 1 -- start start 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.055+0000 7fa5b8f1d740 1 Processor -- start 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.055+0000 7fa5b8f1d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b256c9f0 0x5555b296b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.055+0000 7fa5b8f1d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b25851b0 0x5555b296b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.055+0000 7fa5b8f1d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b258d390 0x5555b29e0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.055+0000 7fa5b8f1d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1ae8f0 con 0x5555b258d390 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.055+0000 7fa5b8f1d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1ae220 con 0x5555b256c9f0 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.055+0000 7fa5b8f1d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1ab460 con 0x5555b25851b0 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.055+0000 7fa5b8f1d740 10 client.0 _ll_register_callbacks cb 0x7fa5a40ad790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.055+0000 7fa5b0f46640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b258d390 0x5555b29e0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.055+0000 7fa5b0f46640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b258d390 0x5555b29e0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47006/0 (socket says 172.21.15.3:47006) 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.055+0000 7fa5b0f46640 1 -- 172.21.15.3:0/1458572103 learned_addr learned my addr 172.21.15.3:0/1458572103 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.055+0000 7fa5a3fff640 1 --2- 172.21.15.3:0/1458572103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b25851b0 0x5555b296b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.055+0000 7fa5abfff640 1 --2- 172.21.15.3:0/1458572103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b256c9f0 0x5555b296b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa5b0f46640 1 -- 172.21.15.3:0/1458572103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b25851b0 msgr2=0x5555b296b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa5b0f46640 1 --2- 172.21.15.3:0/1458572103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b25851b0 0x5555b296b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa5b0f46640 1 -- 172.21.15.3:0/1458572103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b256c9f0 msgr2=0x5555b296b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa5b0f46640 1 --2- 172.21.15.3:0/1458572103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b256c9f0 0x5555b296b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa5b0f46640 1 -- 172.21.15.3:0/1458572103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555b296c190 con 0x5555b258d390 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa5b0f46640 1 --2- 172.21.15.3:0/1458572103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b258d390 0x5555b29e0920 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40e53e0 tx=0x7fa5a40ad000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f31174d031f81ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa58b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa58b7fe640 1 -- 172.21.15.3:0/1458572103 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40b2040 con 0x5555b258d390 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa58b7fe640 1 -- 172.21.15.3:0/1458572103 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fa5a40e5b60 con 0x5555b258d390 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/1458572103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa59c005040 con 0x5555b258d390 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa58b7fe640 1 -- 172.21.15.3:0/1458572103 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa5a40a9fa0 con 0x5555b258d390 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa5b8f1d740 10 client.15951 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/1458572103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555b296c190 con 0x5555b258d390 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.056+0000 7fa58b7fe640 1 -- 172.21.15.3:0/1458572103 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa59c005040 con 0x5555b258d390 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.057+0000 7fa58b7fe640 1 -- 172.21.15.3:0/1458572103 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa5a40c3b00 con 0x5555b258d390 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.057+0000 7fa5b8f1d740 10 client.15951 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.057+0000 7fa5b8f1d740 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.057+0000 7fa5b8f1d740 20 client.15951 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.057+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/1458572103 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5555b25ae070 0x5555b29e4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.057+0000 7fa5b8f1d740 4 client.15951 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:16:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.057+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/1458572103 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5555b28c9850 con 0x5555b25ae070 2024-09-21T06:16:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.057+0000 7fa5abfff640 1 --2- 172.21.15.3:0/1458572103 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5555b25ae070 0x5555b29e4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.057+0000 7fa58b7fe640 10 client.15951 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.057+0000 7fa5abfff640 1 --2- 172.21.15.3:0/1458572103 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5555b25ae070 0x5555b29e4280 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae1a3422bc43793f server_cookie=5311f66d344c9ff6 in_seq=0 out_seq=0 2024-09-21T06:16:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa58b7fe640 1 -- 172.21.15.3:0/1458572103 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x5555b28c9850 con 0x5555b25ae070 2024-09-21T06:16:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa58b7fe640 10 client.15951 handle_command_reply: tid=0 2024-09-21T06:16:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa5b8f1d740 1 client.15951 shutdown 2024-09-21T06:16:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/1458572103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b258d390 msgr2=0x5555b29e0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/1458572103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b258d390 0x5555b29e0920 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40e53e0 tx=0x7fa5a40ad000 comp rx=0 tx=0).stop 2024-09-21T06:16:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/1458572103 shutdown_connections 2024-09-21T06:16:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/1458572103 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5555b25ae070 0x5555b29e4280 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/1458572103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b25851b0 0x5555b296b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/1458572103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555b258d390 0x5555b29e0920 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa5b8f1d740 1 --2- 172.21.15.3:0/1458572103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b256c9f0 0x5555b296b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/1458572103 >> 172.21.15.3:0/1458572103 conn(0x5555b25a2460 msgr2=0x5555b2965190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa58b7fe640 0 client.15951 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/1458572103 shutdown_connections 2024-09-21T06:16:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa5b8f1d740 1 -- 172.21.15.3:0/1458572103 wait complete. 2024-09-21T06:16:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.058+0000 7fa5b8f1d740 20 client.15951 trim_cache size 0 max 16384 2024-09-21T06:16:22.065 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:16:22.065 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:16:22.065 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:16:22.065 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:16:22.065 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:16:22.065 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:16:22.066 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:16:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:16:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:16:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:16:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:16:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:16:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:16:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:16:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:16:22.067 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:16:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.059+0000 7fa5b29a9640 1 -- 172.21.15.3:0/3199785528 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa5980b6710 msgr2=0x7fa5980b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.059+0000 7fa5b29a9640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa5980b6710 0x7fa5980b8c20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa59c000c00 tx=0x7fa59c031400 comp rx=0 tx=0).stop 2024-09-21T06:16:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.059+0000 7fa5b29a9640 1 -- 172.21.15.3:0/3199785528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac108d00 msgr2=0x7fa5ac19bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.059+0000 7fa5b29a9640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac108d00 0x7fa5ac19bd70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400da10 tx=0x7fa5a400ded0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.059+0000 7fa5b29a9640 1 -- 172.21.15.3:0/3199785528 shutdown_connections 2024-09-21T06:16:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.059+0000 7fa5b29a9640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa5980b6710 0x7fa5980b8c20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.059+0000 7fa5b29a9640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac108d00 0x7fa5ac19bd70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.059+0000 7fa5b29a9640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5ac1083b0 0x7fa5ac19b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.059+0000 7fa5b29a9640 1 --2- 172.21.15.3:0/3199785528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5ac102ac0 0x7fa5ac19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.059+0000 7fa5b29a9640 1 -- 172.21.15.3:0/3199785528 >> 172.21.15.3:0/3199785528 conn(0x7fa5ac079790 msgr2=0x7fa5ac1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.059+0000 7fa5b29a9640 1 -- 172.21.15.3:0/3199785528 shutdown_connections 2024-09-21T06:16:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:22.059+0000 7fa5b29a9640 1 -- 172.21.15.3:0/3199785528 wait complete. 2024-09-21T06:16:22.095 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:21 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:16:22.095 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:21 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:22.095 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:21 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:16:22.095 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:21 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:22.095 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:21 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:16:22.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:21 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:16:22.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:21 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:22.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:21 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:16:22.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:21 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:22.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:21 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:16:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:21 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:16:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:21 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:21 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:16:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:21 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:21 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:16:22.996 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:16:22.996 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:16:23.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:22 smithi137 ceph-mon[34464]: pgmap v366: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T06:16:23.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:16:23.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:23.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:23.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:23.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:22 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:23.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:22 smithi204 ceph-mon[34659]: pgmap v366: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T06:16:23.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:16:23.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:23.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:23.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:23.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:22 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:23.263 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:23.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:22 smithi003 ceph-mon[31227]: pgmap v366: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T06:16:23.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:16:23.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:23.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:23.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:23.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:22 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.806+0000 7f5135269640 1 -- 172.21.15.3:0/1863414815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5130108cf0 msgr2=0x7f513010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.806+0000 7f5135269640 1 --2- 172.21.15.3:0/1863414815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5130108cf0 0x7f513010f520 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5128009ca0 tx=0x7f512802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:16:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.806+0000 7f5135269640 1 -- 172.21.15.3:0/1863414815 shutdown_connections 2024-09-21T06:16:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.806+0000 7f5135269640 1 --2- 172.21.15.3:0/1863414815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5130108cf0 0x7f513010f520 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.806+0000 7f5135269640 1 --2- 172.21.15.3:0/1863414815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51301083a0 0x7f51301087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.806+0000 7f5135269640 1 --2- 172.21.15.3:0/1863414815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5130102a60 0x7f5130102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.806+0000 7f5135269640 1 -- 172.21.15.3:0/1863414815 >> 172.21.15.3:0/1863414815 conn(0x7f5130079730 msgr2=0x7f5130079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.806+0000 7f5135269640 1 -- 172.21.15.3:0/1863414815 shutdown_connections 2024-09-21T06:16:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.806+0000 7f5135269640 1 -- 172.21.15.3:0/1863414815 wait complete. 2024-09-21T06:16:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.807+0000 7f5135269640 1 Processor -- start 2024-09-21T06:16:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.807+0000 7f5135269640 1 -- start start 2024-09-21T06:16:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.807+0000 7f5135269640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5130102a60 0x7f513019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.807+0000 7f5135269640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51301083a0 0x7f513019b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.807+0000 7f5135269640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5130108cf0 0x7f513019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.807+0000 7f5135269640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5130112960 con 0x7f5130108cf0 2024-09-21T06:16:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.807+0000 7f5135269640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51301127e0 con 0x7f51301083a0 2024-09-21T06:16:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.807+0000 7f5135269640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5130112ae0 con 0x7f5130102a60 2024-09-21T06:16:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f512f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5130108cf0 0x7f513019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f512f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5130108cf0 0x7f513019bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47024/0 (socket says 172.21.15.3:47024) 2024-09-21T06:16:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f512f7fe640 1 -- 172.21.15.3:0/771837174 learned_addr learned my addr 172.21.15.3:0/771837174 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f512e7fc640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51301083a0 0x7f513019b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f512effd640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5130102a60 0x7f513019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f512f7fe640 1 -- 172.21.15.3:0/771837174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5130102a60 msgr2=0x7f513019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f512f7fe640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5130102a60 0x7f513019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f512f7fe640 1 -- 172.21.15.3:0/771837174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51301083a0 msgr2=0x7f513019b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f512f7fe640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51301083a0 0x7f513019b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f512f7fe640 1 -- 172.21.15.3:0/771837174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51301a6510 con 0x7f5130108cf0 2024-09-21T06:16:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f512e7fc640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51301083a0 0x7f513019b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f512f7fe640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5130108cf0 0x7f513019bd90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5128005cc0 tx=0x7f512802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f5117fff640 1 -- 172.21.15.3:0/771837174 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128003bf0 con 0x7f5130108cf0 2024-09-21T06:16:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f5135269640 1 -- 172.21.15.3:0/771837174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51301a67a0 con 0x7f5130108cf0 2024-09-21T06:16:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f5117fff640 1 -- 172.21.15.3:0/771837174 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5128003d90 con 0x7f5130108cf0 2024-09-21T06:16:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f5117fff640 1 -- 172.21.15.3:0/771837174 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512802fb20 con 0x7f5130108cf0 2024-09-21T06:16:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.808+0000 7f5135269640 1 -- 172.21.15.3:0/771837174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51301a6ce0 con 0x7f5130108cf0 2024-09-21T06:16:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.809+0000 7f5135269640 1 -- 172.21.15.3:0/771837174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5130102e70 con 0x7f5130108cf0 2024-09-21T06:16:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.811+0000 7f5117fff640 1 -- 172.21.15.3:0/771837174 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7f5128004580 con 0x7f5130108cf0 2024-09-21T06:16:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.812+0000 7f5117fff640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51180b6740 0x7f51180b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.812+0000 7f512effd640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51180b6740 0x7f51180b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.812+0000 7f5117fff640 1 -- 172.21.15.3:0/771837174 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5128108cb0 con 0x7f5130108cf0 2024-09-21T06:16:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.813+0000 7f512effd640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51180b6740 0x7f51180b8c50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f513006b9c0 tx=0x7f512000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:23.816+0000 7f5117fff640 1 -- 172.21.15.3:0/771837174 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51280098b0 con 0x7f5130108cf0 2024-09-21T06:16:24.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.050+0000 7f5135269640 1 -- 172.21.15.3:0/771837174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f513010aae0 con 0x7f5130108cf0 2024-09-21T06:16:24.052 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:16:24.053 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:16:24.053 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:24.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.051+0000 7f5117fff640 1 -- 172.21.15.3:0/771837174 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11962 (secure 0 0 0) 0x7f51280cf0d0 con 0x7f5130108cf0 2024-09-21T06:16:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:16:24.053 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"btime":"2024-09-21T06:15:09:322787+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:15:08.461776+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24670,24674,24679,24691,24704]},"id":1}]} 2024-09-21T06:16:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 28 2024-09-21T06:16:24.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.053+0000 7f5135269640 1 -- 172.21.15.3:0/771837174 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51180b6740 msgr2=0x7f51180b8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:24.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.053+0000 7f5135269640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51180b6740 0x7f51180b8c50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f513006b9c0 tx=0x7f512000a2d0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.053+0000 7f5135269640 1 -- 172.21.15.3:0/771837174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5130108cf0 msgr2=0x7f513019bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.053+0000 7f5135269640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5130108cf0 0x7f513019bd90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5128005cc0 tx=0x7f512802f380 comp rx=0 tx=0).stop 2024-09-21T06:16:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.053+0000 7f5135269640 1 -- 172.21.15.3:0/771837174 shutdown_connections 2024-09-21T06:16:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.053+0000 7f5135269640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51180b6740 0x7f51180b8c50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.053+0000 7f5135269640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5130108cf0 0x7f513019bd90 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.053+0000 7f5135269640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51301083a0 0x7f513019b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.053+0000 7f5135269640 1 --2- 172.21.15.3:0/771837174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5130102a60 0x7f513019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.053+0000 7f5135269640 1 -- 172.21.15.3:0/771837174 >> 172.21.15.3:0/771837174 conn(0x7f5130079730 msgr2=0x7f5130104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.054+0000 7f5135269640 1 -- 172.21.15.3:0/771837174 shutdown_connections 2024-09-21T06:16:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:24.054+0000 7f5135269640 1 -- 172.21.15.3:0/771837174 wait complete. 2024-09-21T06:16:24.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:16:24.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:16:24.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:24.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:16:24.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:16:24.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:16:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.816+0000 7fb37e3df640 1 -- 172.21.15.3:0/2642350798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb378104990 msgr2=0x7fb378104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.816+0000 7fb37e3df640 1 --2- 172.21.15.3:0/2642350798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb378104990 0x7fb378104da0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb368009900 tx=0x7fb36802a410 comp rx=0 tx=0).stop 2024-09-21T06:16:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.816+0000 7fb37e3df640 1 -- 172.21.15.3:0/2642350798 shutdown_connections 2024-09-21T06:16:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.816+0000 7fb37e3df640 1 --2- 172.21.15.3:0/2642350798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb37810dd60 0x7fb378110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.816+0000 7fb37e3df640 1 --2- 172.21.15.3:0/2642350798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3781052e0 0x7fb37810d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.816+0000 7fb37e3df640 1 --2- 172.21.15.3:0/2642350798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb378104990 0x7fb378104da0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.816+0000 7fb37e3df640 1 -- 172.21.15.3:0/2642350798 >> 172.21.15.3:0/2642350798 conn(0x7fb37807a6e0 msgr2=0x7fb37807aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.816+0000 7fb37e3df640 1 -- 172.21.15.3:0/2642350798 shutdown_connections 2024-09-21T06:16:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.816+0000 7fb37e3df640 1 -- 172.21.15.3:0/2642350798 wait complete. 2024-09-21T06:16:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.817+0000 7fb37e3df640 1 Processor -- start 2024-09-21T06:16:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.817+0000 7fb37e3df640 1 -- start start 2024-09-21T06:16:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.818+0000 7fb37e3df640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb378104990 0x7fb37819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.818+0000 7fb37e3df640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3781052e0 0x7fb37819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.818+0000 7fb37e3df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb37810dd60 0x7fb37819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.818+0000 7fb37e3df640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781139e0 con 0x7fb378104990 2024-09-21T06:16:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.818+0000 7fb37e3df640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb378113860 con 0x7fb37810dd60 2024-09-21T06:16:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.818+0000 7fb37e3df640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb378113b60 con 0x7fb3781052e0 2024-09-21T06:16:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.818+0000 7fb377fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb378104990 0x7fb37819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.818+0000 7fb377fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb378104990 0x7fb37819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47046/0 (socket says 172.21.15.3:47046) 2024-09-21T06:16:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.818+0000 7fb377fff640 1 -- 172.21.15.3:0/3738797659 learned_addr learned my addr 172.21.15.3:0/3738797659 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.818+0000 7fb3777fe640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3781052e0 0x7fb37819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.818+0000 7fb37c97c640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb37810dd60 0x7fb37819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.819+0000 7fb377fff640 1 -- 172.21.15.3:0/3738797659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3781052e0 msgr2=0x7fb37819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.819+0000 7fb377fff640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3781052e0 0x7fb37819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.819+0000 7fb377fff640 1 -- 172.21.15.3:0/3738797659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb37810dd60 msgr2=0x7fb37819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.819+0000 7fb377fff640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb37810dd60 0x7fb37819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.819+0000 7fb377fff640 1 -- 172.21.15.3:0/3738797659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3781a0330 con 0x7fb378104990 2024-09-21T06:16:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.819+0000 7fb377fff640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb378104990 0x7fb37819a400 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb368005940 tx=0x7fb3680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.819+0000 7fb3757fa640 1 -- 172.21.15.3:0/3738797659 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36803f030 con 0x7fb378104990 2024-09-21T06:16:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.819+0000 7fb3757fa640 1 -- 172.21.15.3:0/3738797659 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb368003cb0 con 0x7fb378104990 2024-09-21T06:16:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.819+0000 7fb37e3df640 1 -- 172.21.15.3:0/3738797659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3781a05c0 con 0x7fb378104990 2024-09-21T06:16:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.819+0000 7fb3757fa640 1 -- 172.21.15.3:0/3738797659 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368035070 con 0x7fb378104990 2024-09-21T06:16:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.819+0000 7fb37e3df640 1 -- 172.21.15.3:0/3738797659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3781a6db0 con 0x7fb378104990 2024-09-21T06:16:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.822+0000 7fb385427740 1 Processor -- start 2024-09-21T06:16:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.822+0000 7fb3757fa640 1 -- 172.21.15.3:0/3738797659 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7fb368042030 con 0x7fb378104990 2024-09-21T06:16:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.822+0000 7fb385427740 1 -- start start 2024-09-21T06:16:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.822+0000 7fb385427740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 0x560f2c1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.822+0000 7fb385427740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd922f0 0x560f2c160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.822+0000 7fb385427740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 0x560f2c161430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.822+0000 7fb385427740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560f2c0d56f0 con 0x560f2bdbb1b0 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.822+0000 7fb385427740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560f2c0c4b20 con 0x560f2bd922f0 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.822+0000 7fb385427740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560f2c0c8930 con 0x560f2bd9a4d0 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.822+0000 7fb37c97c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 0x560f2c1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.822+0000 7fb37c97c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 0x560f2c1609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47060/0 (socket says 172.21.15.3:47060) 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.822+0000 7fb37c97c640 1 -- 172.21.15.3:0/883944783 learned_addr learned my addr 172.21.15.3:0/883944783 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb3777fe640 1 --2- 172.21.15.3:0/883944783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 0x560f2c161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb377fff640 1 --2- 172.21.15.3:0/883944783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd922f0 0x560f2c160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb37c97c640 1 -- 172.21.15.3:0/883944783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 msgr2=0x560f2c161430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb37c97c640 1 --2- 172.21.15.3:0/883944783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 0x560f2c161430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb37c97c640 1 -- 172.21.15.3:0/883944783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd922f0 msgr2=0x560f2c160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb37c97c640 1 --2- 172.21.15.3:0/883944783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd922f0 0x560f2c160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb37c97c640 1 -- 172.21.15.3:0/883944783 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f2c08a530 con 0x560f2bdbb1b0 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb377fff640 1 --2- 172.21.15.3:0/883944783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd922f0 0x560f2c160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb37c97c640 1 --2- 172.21.15.3:0/883944783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 0x560f2c1609b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb370009ee0 tx=0x7fb370010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9495f2ee2eaaa866 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb3757fa640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb3600b6710 0x7fb3600b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb35e7fc640 1 -- 172.21.15.3:0/883944783 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370016070 con 0x560f2bdbb1b0 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb377fff640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb3600b6710 0x7fb3600b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb35e7fc640 1 -- 172.21.15.3:0/883944783 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb370004d20 con 0x560f2bdbb1b0 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.823+0000 7fb35e7fc640 1 -- 172.21.15.3:0/883944783 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3700126f0 con 0x560f2bdbb1b0 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.824+0000 7fb3757fa640 1 -- 172.21.15.3:0/3738797659 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb3681081e0 con 0x7fb378104990 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.824+0000 7fb385427740 1 -- 172.21.15.3:0/883944783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 msgr2=0x560f2c1609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.824+0000 7fb377fff640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb3600b6710 0x7fb3600b8c20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb368105910 tx=0x7fb368092040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.824+0000 7fb385427740 1 --2- 172.21.15.3:0/883944783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 0x560f2c1609b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb370009ee0 tx=0x7fb370010740 comp rx=0 tx=0).stop 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.824+0000 7fb385427740 1 -- 172.21.15.3:0/883944783 shutdown_connections 2024-09-21T06:16:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.824+0000 7fb385427740 1 --2- 172.21.15.3:0/883944783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 0x560f2c161430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.824+0000 7fb385427740 1 --2- 172.21.15.3:0/883944783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd922f0 0x560f2c160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.824+0000 7fb385427740 1 --2- 172.21.15.3:0/883944783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 0x560f2c1609b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.824+0000 7fb385427740 1 -- 172.21.15.3:0/883944783 >> 172.21.15.3:0/883944783 conn(0x560f2bdaf300 msgr2=0x560f2c15e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.825+0000 7fb385427740 1 -- 172.21.15.3:0/883944783 shutdown_connections 2024-09-21T06:16:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.825+0000 7fb385427740 1 -- 172.21.15.3:0/883944783 wait complete. 2024-09-21T06:16:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.825+0000 7fb385427740 1 -- start start 2024-09-21T06:16:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.825+0000 7fb385427740 1 Processor -- start 2024-09-21T06:16:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.825+0000 7fb385427740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd922f0 0x560f2c1704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.825+0000 7fb385427740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 0x560f2c1709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.825+0000 7fb385427740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 0x560f2c17c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.825+0000 7fb385427740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560f2bffcab0 con 0x560f2bdbb1b0 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb385427740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560f2c06a730 con 0x560f2bd922f0 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb385427740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560f2c173510 con 0x560f2bd9a4d0 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb385427740 10 client.0 _ll_register_callbacks cb 0x560f2c0be6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb37c97c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 0x560f2c17c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb37c97c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 0x560f2c17c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47068/0 (socket says 172.21.15.3:47068) 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb37c97c640 1 -- 172.21.15.3:0/4258157754 learned_addr learned my addr 172.21.15.3:0/4258157754 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb3777fe640 1 --2- 172.21.15.3:0/4258157754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 0x560f2c1709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb37c97c640 1 --2- 172.21.15.3:0/4258157754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd922f0 0x560f2c1704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb37c97c640 1 -- 172.21.15.3:0/4258157754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 msgr2=0x560f2c1709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb37c97c640 1 --2- 172.21.15.3:0/4258157754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 0x560f2c1709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb37c97c640 1 -- 172.21.15.3:0/4258157754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd922f0 msgr2=0x560f2c1704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb37c97c640 1 --2- 172.21.15.3:0/4258157754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd922f0 0x560f2c1704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb37c97c640 1 -- 172.21.15.3:0/4258157754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f2be26790 con 0x560f2bdbb1b0 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb37c97c640 1 --2- 172.21.15.3:0/4258157754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 0x560f2c17c840 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb370009ca0 tx=0x7fb3700059c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91feae518708466 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb3577fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb3577fe640 1 -- 172.21.15.3:0/4258157754 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37001e030 con 0x560f2bdbb1b0 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb3577fe640 1 -- 172.21.15.3:0/4258157754 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb370004ed0 con 0x560f2bdbb1b0 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.826+0000 7fb385427740 1 -- 172.21.15.3:0/4258157754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb370004d20 con 0x560f2bdbb1b0 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.827+0000 7fb3577fe640 1 -- 172.21.15.3:0/4258157754 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb370016040 con 0x560f2bdbb1b0 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.827+0000 7fb385427740 10 client.15969 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.827+0000 7fb385427740 1 -- 172.21.15.3:0/4258157754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f2be9f620 con 0x560f2bdbb1b0 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.827+0000 7fb3577fe640 1 -- 172.21.15.3:0/4258157754 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb370004d20 con 0x560f2bdbb1b0 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.827+0000 7fb3577fe640 1 -- 172.21.15.3:0/4258157754 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb37000f040 con 0x560f2bdbb1b0 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.827+0000 7fb385427740 10 client.15969 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.827+0000 7fb385427740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.827+0000 7fb385427740 20 client.15969 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.827+0000 7fb385427740 1 --2- 172.21.15.3:0/4258157754 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560f2bd79d80 0x560f2c182480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.827+0000 7fb385427740 4 client.15969 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.827+0000 7fb385427740 1 -- 172.21.15.3:0/4258157754 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f2c0d7640 con 0x560f2bd79d80 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.827+0000 7fb3777fe640 1 --2- 172.21.15.3:0/4258157754 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560f2bd79d80 0x560f2c182480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.828+0000 7fb3777fe640 1 --2- 172.21.15.3:0/4258157754 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560f2bd79d80 0x560f2c182480 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=406bdc59d0f0c70c server_cookie=d494a698843b9c20 in_seq=0 out_seq=0 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.828+0000 7fb3577fe640 10 client.15969 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb3577fe640 1 -- 172.21.15.3:0/4258157754 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f2c0d7640 con 0x560f2bd79d80 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb3577fe640 10 client.15969 handle_command_reply: tid=0 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb385427740 1 client.15969 shutdown 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb385427740 1 -- 172.21.15.3:0/4258157754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 msgr2=0x560f2c17c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb385427740 1 --2- 172.21.15.3:0/4258157754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 0x560f2c17c840 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb370009ca0 tx=0x7fb3700059c0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb385427740 1 -- 172.21.15.3:0/4258157754 shutdown_connections 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb385427740 1 --2- 172.21.15.3:0/4258157754 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560f2bd79d80 0x560f2c182480 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb385427740 1 --2- 172.21.15.3:0/4258157754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 0x560f2c1709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb3577fe640 0 client.15969 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb385427740 1 --2- 172.21.15.3:0/4258157754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bdbb1b0 0x560f2c17c840 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb385427740 1 --2- 172.21.15.3:0/4258157754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd922f0 0x560f2c1704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb385427740 1 -- 172.21.15.3:0/4258157754 >> 172.21.15.3:0/4258157754 conn(0x560f2bdaf300 msgr2=0x560f2c15cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb385427740 1 -- 172.21.15.3:0/4258157754 shutdown_connections 2024-09-21T06:16:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb385427740 1 -- 172.21.15.3:0/4258157754 wait complete. 2024-09-21T06:16:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.829+0000 7fb385427740 20 client.15969 trim_cache size 0 max 16384 2024-09-21T06:16:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.844+0000 7fb385427740 1 Processor -- start 2024-09-21T06:16:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.844+0000 7fb385427740 1 -- start start 2024-09-21T06:16:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.844+0000 7fb385427740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd79d80 0x560f2c15e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.844+0000 7fb385427740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd922f0 0x560f2c170980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.844+0000 7fb385427740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 0x560f2c1720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.844+0000 7fb385427740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781ad3f0 con 0x560f2bd922f0 2024-09-21T06:16:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.844+0000 7fb385427740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781aef30 con 0x560f2bd79d80 2024-09-21T06:16:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.844+0000 7fb385427740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781adac0 con 0x560f2bd9a4d0 2024-09-21T06:16:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb3777fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd922f0 0x560f2c170980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb3777fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd922f0 0x560f2c170980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47078/0 (socket says 172.21.15.3:47078) 2024-09-21T06:16:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb3777fe640 1 -- 172.21.15.3:0/2961834750 learned_addr learned my addr 172.21.15.3:0/2961834750 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb37c97c640 1 --2- 172.21.15.3:0/2961834750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 0x560f2c1720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb37c97c640 1 --2- 172.21.15.3:0/2961834750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd79d80 0x560f2c15e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb3777fe640 1 -- 172.21.15.3:0/2961834750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 msgr2=0x560f2c1720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb3777fe640 1 --2- 172.21.15.3:0/2961834750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 0x560f2c1720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb3777fe640 1 -- 172.21.15.3:0/2961834750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd79d80 msgr2=0x560f2c15e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb3777fe640 1 --2- 172.21.15.3:0/2961834750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd79d80 0x560f2c15e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb3777fe640 1 -- 172.21.15.3:0/2961834750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f2c159d50 con 0x560f2bd922f0 2024-09-21T06:16:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb3777fe640 1 --2- 172.21.15.3:0/2961834750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd922f0 0x560f2c170980 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00bc10 tx=0x7fb36c0044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82a1dd3a97399ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb35dffb640 1 -- 172.21.15.3:0/2961834750 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c003e50 con 0x560f2bd922f0 2024-09-21T06:16:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb35dffb640 1 -- 172.21.15.3:0/2961834750 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb36c0118f0 con 0x560f2bd922f0 2024-09-21T06:16:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.845+0000 7fb35dffb640 1 -- 172.21.15.3:0/2961834750 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c01c8e0 con 0x560f2bd922f0 2024-09-21T06:16:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.846+0000 7fb385427740 1 -- 172.21.15.3:0/2961834750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd922f0 msgr2=0x560f2c170980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.846+0000 7fb385427740 1 --2- 172.21.15.3:0/2961834750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd922f0 0x560f2c170980 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00bc10 tx=0x7fb36c0044d0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.846+0000 7fb385427740 1 -- 172.21.15.3:0/2961834750 shutdown_connections 2024-09-21T06:16:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.846+0000 7fb385427740 1 --2- 172.21.15.3:0/2961834750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd9a4d0 0x560f2c1720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.846+0000 7fb385427740 1 --2- 172.21.15.3:0/2961834750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd922f0 0x560f2c170980 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.846+0000 7fb385427740 1 --2- 172.21.15.3:0/2961834750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd79d80 0x560f2c15e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.846+0000 7fb385427740 1 -- 172.21.15.3:0/2961834750 >> 172.21.15.3:0/2961834750 conn(0x560f2bdaf300 msgr2=0x560f2c1719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.846+0000 7fb385427740 1 -- 172.21.15.3:0/2961834750 shutdown_connections 2024-09-21T06:16:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.846+0000 7fb385427740 1 -- 172.21.15.3:0/2961834750 wait complete. 2024-09-21T06:16:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.847+0000 7fb385427740 1 -- start start 2024-09-21T06:16:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.847+0000 7fb385427740 1 Processor -- start 2024-09-21T06:16:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.847+0000 7fb385427740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd79d80 0x560f2c182b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.847+0000 7fb385427740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd922f0 0x560f2c1830b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.847+0000 7fb385427740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd9a4d0 0x560f2c1eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.847+0000 7fb385427740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781ae860 con 0x560f2bd79d80 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.847+0000 7fb385427740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781ae190 con 0x560f2bd9a4d0 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.847+0000 7fb385427740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781ab3d0 con 0x560f2bd922f0 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.847+0000 7fb385427740 10 client.0 _ll_register_callbacks cb 0x7fb36c00b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.847+0000 7fb37c97c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd79d80 0x560f2c182b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.847+0000 7fb37c97c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd79d80 0x560f2c182b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47080/0 (socket says 172.21.15.3:47080) 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.847+0000 7fb37c97c640 1 -- 172.21.15.3:0/639507589 learned_addr learned my addr 172.21.15.3:0/639507589 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.848+0000 7fb37c97c640 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd9a4d0 0x560f2c1eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.848+0000 7fb3777fe640 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd922f0 0x560f2c1830b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.848+0000 7fb37c97c640 1 -- 172.21.15.3:0/639507589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd922f0 msgr2=0x560f2c1830b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.848+0000 7fb37c97c640 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd922f0 0x560f2c1830b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.848+0000 7fb37c97c640 1 -- 172.21.15.3:0/639507589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd9a4d0 msgr2=0x560f2c1eabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.848+0000 7fb37c97c640 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd9a4d0 0x560f2c1eabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.848+0000 7fb37c97c640 1 -- 172.21.15.3:0/639507589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f2c1eb130 con 0x560f2bd79d80 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.848+0000 7fb3777fe640 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd922f0 0x560f2c1830b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.848+0000 7fb37c97c640 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd79d80 0x560f2c182b70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb370009ee0 tx=0x7fb370005820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f551b2f7ba8aab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.848+0000 7fb3577fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb3577fe640 1 -- 172.21.15.3:0/639507589 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370037020 con 0x560f2bd79d80 2024-09-21T06:16:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb3577fe640 1 -- 172.21.15.3:0/639507589 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb37002ba50 con 0x560f2bd79d80 2024-09-21T06:16:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb385427740 1 -- 172.21.15.3:0/639507589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb36c0118f0 con 0x560f2bd79d80 2024-09-21T06:16:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb3577fe640 1 -- 172.21.15.3:0/639507589 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb370045050 con 0x560f2bd79d80 2024-09-21T06:16:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb385427740 10 client.15975 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb385427740 1 -- 172.21.15.3:0/639507589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f2c1ec350 con 0x560f2bd79d80 2024-09-21T06:16:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb3577fe640 1 -- 172.21.15.3:0/639507589 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb36c0118f0 con 0x560f2bd79d80 2024-09-21T06:16:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb3577fe640 1 -- 172.21.15.3:0/639507589 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb370004ce0 con 0x560f2bd79d80 2024-09-21T06:16:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb385427740 10 client.15975 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb385427740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb385427740 20 client.15975 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb385427740 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560f2bdbb1b0 0x560f2c1eea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb385427740 4 client.15975 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:16:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.852+0000 7fb385427740 1 -- 172.21.15.3:0/639507589 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f2c0d30f0 con 0x560f2bdbb1b0 2024-09-21T06:16:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.853+0000 7fb3777fe640 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560f2bdbb1b0 0x560f2c1eea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.853+0000 7fb3777fe640 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560f2bdbb1b0 0x560f2c1eea60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6409b343e1b82d0 server_cookie=1d2a160e77742b42 in_seq=0 out_seq=0 2024-09-21T06:16:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.853+0000 7fb3577fe640 10 client.15975 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.859+0000 7fb3577fe640 1 -- 172.21.15.3:0/639507589 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f2c0d30f0 con 0x560f2bdbb1b0 2024-09-21T06:16:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.859+0000 7fb3577fe640 10 client.15975 handle_command_reply: tid=0 2024-09-21T06:16:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.859+0000 7fb385427740 1 client.15975 shutdown 2024-09-21T06:16:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.859+0000 7fb385427740 1 -- 172.21.15.3:0/639507589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd79d80 msgr2=0x560f2c182b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.859+0000 7fb385427740 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd79d80 0x560f2c182b70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb370009ee0 tx=0x7fb370005820 comp rx=0 tx=0).stop 2024-09-21T06:16:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.859+0000 7fb385427740 1 -- 172.21.15.3:0/639507589 shutdown_connections 2024-09-21T06:16:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.859+0000 7fb385427740 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560f2bdbb1b0 0x560f2c1eea60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.859+0000 7fb385427740 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f2bd922f0 0x560f2c1830b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.859+0000 7fb385427740 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f2bd9a4d0 0x560f2c1eabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.859+0000 7fb3577fe640 0 client.15975 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.859+0000 7fb385427740 1 --2- 172.21.15.3:0/639507589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f2bd79d80 0x560f2c182b70 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.859+0000 7fb385427740 1 -- 172.21.15.3:0/639507589 >> 172.21.15.3:0/639507589 conn(0x560f2bdaf300 msgr2=0x560f2c16b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb385427740 1 -- 172.21.15.3:0/639507589 shutdown_connections 2024-09-21T06:16:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb385427740 1 -- 172.21.15.3:0/639507589 wait complete. 2024-09-21T06:16:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb385427740 20 client.15975 trim_cache size 0 max 16384 2024-09-21T06:16:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:16:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:16:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:16:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:16:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb37e3df640 1 -- 172.21.15.3:0/3738797659 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb3600b6710 msgr2=0x7fb3600b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb37e3df640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb3600b6710 0x7fb3600b8c20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb368105910 tx=0x7fb368092040 comp rx=0 tx=0).stop 2024-09-21T06:16:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb37e3df640 1 -- 172.21.15.3:0/3738797659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb378104990 msgr2=0x7fb37819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb37e3df640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb378104990 0x7fb37819a400 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb368005940 tx=0x7fb3680042f0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb37e3df640 1 -- 172.21.15.3:0/3738797659 shutdown_connections 2024-09-21T06:16:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb37e3df640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb3600b6710 0x7fb3600b8c20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb37e3df640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3781052e0 0x7fb37819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb37e3df640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb37810dd60 0x7fb37819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb37e3df640 1 --2- 172.21.15.3:0/3738797659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb378104990 0x7fb37819a400 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb37e3df640 1 -- 172.21.15.3:0/3738797659 >> 172.21.15.3:0/3738797659 conn(0x7fb37807a6e0 msgr2=0x7fb378100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.860+0000 7fb37e3df640 1 -- 172.21.15.3:0/3738797659 shutdown_connections 2024-09-21T06:16:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:24.861+0000 7fb37e3df640 1 -- 172.21.15.3:0/3738797659 wait complete. 2024-09-21T06:16:24.924 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:16:25.142 INFO:tasks.workunit.client.0.smithi003.stderr:Updating files: 90% (11455/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T06:16:25.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:24 smithi137 ceph-mon[34464]: pgmap v367: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T06:16:25.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:24 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/771837174' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:25.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:24 smithi204 ceph-mon[34659]: pgmap v367: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T06:16:25.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:24 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/771837174' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:25.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:24 smithi003 ceph-mon[31227]: pgmap v367: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T06:16:25.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:24 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/771837174' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:25.604 INFO:tasks.workunit.client.0.smithi003.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T06:16:25.604 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-21T06:16:25.604 INFO:tasks.workunit.client.0.smithi003.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T06:16:25.604 INFO:tasks.workunit.client.0.smithi003.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T06:16:25.604 INFO:tasks.workunit.client.0.smithi003.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T06:16:25.604 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-21T06:16:25.604 INFO:tasks.workunit.client.0.smithi003.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T06:16:25.604 INFO:tasks.workunit.client.0.smithi003.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T06:16:25.605 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-21T06:16:25.605 INFO:tasks.workunit.client.0.smithi003.stderr: git switch -c 2024-09-21T06:16:25.605 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-21T06:16:25.605 INFO:tasks.workunit.client.0.smithi003.stderr:Or undo this operation with: 2024-09-21T06:16:25.605 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-21T06:16:25.605 INFO:tasks.workunit.client.0.smithi003.stderr: git switch - 2024-09-21T06:16:25.605 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-21T06:16:25.605 INFO:tasks.workunit.client.0.smithi003.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T06:16:25.605 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-21T06:16:25.605 INFO:tasks.workunit.client.0.smithi003.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T06:16:25.609 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T06:16:25.731 INFO:tasks.workunit.client.0.smithi003.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T06:16:25.733 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T06:16:25.733 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T06:16:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.829+0000 7ff6e2615640 1 -- 172.21.15.3:0/2460250970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6dc108480 msgr2=0x7ff6dc108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.829+0000 7ff6e2615640 1 --2- 172.21.15.3:0/2460250970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6dc108480 0x7ff6dc108890 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009900 tx=0x7ff6c402a410 comp rx=0 tx=0).stop 2024-09-21T06:16:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.829+0000 7ff6e2615640 1 -- 172.21.15.3:0/2460250970 shutdown_connections 2024-09-21T06:16:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.829+0000 7ff6e2615640 1 --2- 172.21.15.3:0/2460250970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6dc108dd0 0x7ff6dc10f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.829+0000 7ff6e2615640 1 --2- 172.21.15.3:0/2460250970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6dc108480 0x7ff6dc108890 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.829+0000 7ff6e2615640 1 --2- 172.21.15.3:0/2460250970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6dc102b40 0x7ff6dc102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.829+0000 7ff6e2615640 1 -- 172.21.15.3:0/2460250970 >> 172.21.15.3:0/2460250970 conn(0x7ff6dc079750 msgr2=0x7ff6dc079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.829+0000 7ff6e2615640 1 -- 172.21.15.3:0/2460250970 shutdown_connections 2024-09-21T06:16:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.829+0000 7ff6e2615640 1 -- 172.21.15.3:0/2460250970 wait complete. 2024-09-21T06:16:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.830+0000 7ff6e2615640 1 Processor -- start 2024-09-21T06:16:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.830+0000 7ff6e2615640 1 -- start start 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.830+0000 7ff6e2615640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6dc102b40 0x7ff6dc19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.830+0000 7ff6e2615640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6dc108480 0x7ff6dc19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.830+0000 7ff6e2615640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6dc108dd0 0x7ff6dc19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.830+0000 7ff6e2615640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc113ac0 con 0x7ff6dc108dd0 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.830+0000 7ff6e2615640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc113940 con 0x7ff6dc102b40 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.830+0000 7ff6e2615640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc113c40 con 0x7ff6dc108480 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6e0bb2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6dc108dd0 0x7ff6dc19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6e0bb2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6dc108dd0 0x7ff6dc19fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35466/0 (socket says 172.21.15.3:35466) 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6e0bb2640 1 -- 172.21.15.3:0/2768146193 learned_addr learned my addr 172.21.15.3:0/2768146193 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6dbfff640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6dc102b40 0x7ff6dc19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6dc108480 0x7ff6dc19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6e0bb2640 1 -- 172.21.15.3:0/2768146193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6dc108480 msgr2=0x7ff6dc19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6e0bb2640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6dc108480 0x7ff6dc19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6e0bb2640 1 -- 172.21.15.3:0/2768146193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6dc102b40 msgr2=0x7ff6dc19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6e0bb2640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6dc102b40 0x7ff6dc19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6e0bb2640 1 -- 172.21.15.3:0/2768146193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6dc1a04a0 con 0x7ff6dc108dd0 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6dbfff640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6dc102b40 0x7ff6dc19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6dc108480 0x7ff6dc19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6e0bb2640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6dc108dd0 0x7ff6dc19fd70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009fd0 tx=0x7ff6d400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6d9ffb640 1 -- 172.21.15.3:0/2768146193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4017070 con 0x7ff6dc108dd0 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6d9ffb640 1 -- 172.21.15.3:0/2768146193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d4003e20 con 0x7ff6dc108dd0 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6e2615640 1 -- 172.21.15.3:0/2768146193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6dc1a0790 con 0x7ff6dc108dd0 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6d9ffb640 1 -- 172.21.15.3:0/2768146193 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4010460 con 0x7ff6dc108dd0 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.831+0000 7ff6e2615640 1 -- 172.21.15.3:0/2768146193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6dc1a6ef0 con 0x7ff6dc108dd0 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.834+0000 7ff6e8b89740 1 Processor -- start 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.834+0000 7ff6d9ffb640 1 -- 172.21.15.3:0/2768146193 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 154693+0+0 (secure 0 0 0) 0x7ff6d4003970 con 0x7ff6dc108dd0 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.834+0000 7ff6e8b89740 1 -- start start 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6e8b89740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb87070 0x55a7dbf2e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6e8b89740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf2e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6e8b89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb66390 0x55a7dbf2ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6e8b89740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a7dbea4ea0 con 0x55a7dbb87070 2024-09-21T06:16:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6e8b89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7dbe942d0 con 0x55a7dbb66390 2024-09-21T06:16:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6e8b89740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a7dbe980e0 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6d3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb87070 0x55a7dbf2e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6d3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb87070 0x55a7dbf2e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35476/0 (socket says 172.21.15.3:35476) 2024-09-21T06:16:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6d3fff640 1 -- 172.21.15.3:0/2886276697 learned_addr learned my addr 172.21.15.3:0/2886276697 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6dbfff640 1 --2- 172.21.15.3:0/2886276697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf2e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/2886276697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb66390 0x55a7dbf2ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6d9ffb640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff6c80b6710 0x7ff6c80b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6d3fff640 1 -- 172.21.15.3:0/2886276697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 msgr2=0x55a7dbf2e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/2886276697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf2e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6d3fff640 1 -- 172.21.15.3:0/2886276697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb66390 msgr2=0x55a7dbf2ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/2886276697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb66390 0x55a7dbf2ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6d3fff640 1 -- 172.21.15.3:0/2886276697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7dbbf2650 con 0x55a7dbb87070 2024-09-21T06:16:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.835+0000 7ff6e0bb2640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff6c80b6710 0x7ff6c80b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.836+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/2886276697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb87070 0x55a7dbf2e2c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4005ab0 tx=0x7ff6c4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253ce2d1ebabb033 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.836+0000 7ff6d9ffb640 1 -- 172.21.15.3:0/2768146193 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff6d40e7630 con 0x7ff6dc108dd0 2024-09-21T06:16:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.836+0000 7ff6d27fc640 1 -- 172.21.15.3:0/2886276697 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4035050 con 0x55a7dbb87070 2024-09-21T06:16:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.836+0000 7ff6d27fc640 1 -- 172.21.15.3:0/2886276697 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6c40316f0 con 0x55a7dbb87070 2024-09-21T06:16:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.836+0000 7ff6d27fc640 1 -- 172.21.15.3:0/2886276697 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c40046c0 con 0x55a7dbb87070 2024-09-21T06:16:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.837+0000 7ff6e8b89740 1 -- 172.21.15.3:0/2886276697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb87070 msgr2=0x55a7dbf2e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.837+0000 7ff6e0bb2640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff6c80b6710 0x7ff6c80b8c20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40e4d60 tx=0x7ff6d407a000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:25.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.837+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/2886276697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb87070 0x55a7dbf2e2c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4005ab0 tx=0x7ff6c4009560 comp rx=0 tx=0).stop 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.837+0000 7ff6e8b89740 1 -- 172.21.15.3:0/2886276697 shutdown_connections 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.837+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/2886276697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf2e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.837+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/2886276697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb66390 0x55a7dbf2ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.837+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/2886276697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb87070 0x55a7dbf2e2c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.837+0000 7ff6e8b89740 1 -- 172.21.15.3:0/2886276697 >> 172.21.15.3:0/2886276697 conn(0x55a7dbb7b460 msgr2=0x55a7dbf2c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.837+0000 7ff6e8b89740 1 -- 172.21.15.3:0/2886276697 shutdown_connections 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.837+0000 7ff6e8b89740 1 -- 172.21.15.3:0/2886276697 wait complete. 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.837+0000 7ff6e8b89740 1 -- start start 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.837+0000 7ff6e8b89740 1 Processor -- start 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6e8b89740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf398e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6e8b89740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb66390 0x55a7dbf39e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6e8b89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb87070 0x55a7dbf3a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6e8b89740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a7dbcf5700 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6e8b89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7dbe2f160 con 0x55a7dbb87070 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6e8b89740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a7dbf3ca40 con 0x55a7dbb66390 2024-09-21T06:16:25.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6e8b89740 10 client.0 _ll_register_callbacks cb 0x55a7dbe489e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6d3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6d3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf398e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35492/0 (socket says 172.21.15.3:35492) 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6d3fff640 1 -- 172.21.15.3:0/625914264 learned_addr learned my addr 172.21.15.3:0/625914264 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6dbfff640 1 --2- 172.21.15.3:0/625914264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb66390 0x55a7dbf39e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/625914264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb87070 0x55a7dbf3a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6d3fff640 1 -- 172.21.15.3:0/625914264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb66390 msgr2=0x55a7dbf39e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/625914264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb66390 0x55a7dbf39e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6d3fff640 1 -- 172.21.15.3:0/625914264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb87070 msgr2=0x55a7dbf3a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/625914264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb87070 0x55a7dbf3a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6d3fff640 1 -- 172.21.15.3:0/625914264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7dbe5cc90 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/625914264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf398e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff6c402a930 tx=0x7ff6c4005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786902928aad799a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.838+0000 7ff6bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.839+0000 7ff6bb7fe640 1 -- 172.21.15.3:0/625914264 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4035070 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.839+0000 7ff6bb7fe640 1 -- 172.21.15.3:0/625914264 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff6c40035f0 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.839+0000 7ff6e8b89740 1 -- 172.21.15.3:0/625914264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6c40316f0 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.839+0000 7ff6bb7fe640 1 -- 172.21.15.3:0/625914264 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff6c40037f0 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.839+0000 7ff6e8b89740 10 client.15987 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.839+0000 7ff6e8b89740 1 -- 172.21.15.3:0/625914264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7dbf486b0 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.839+0000 7ff6bb7fe640 1 -- 172.21.15.3:0/625914264 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6c40316f0 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.839+0000 7ff6bb7fe640 1 -- 172.21.15.3:0/625914264 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff6c403a7f0 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.839+0000 7ff6e8b89740 10 client.15987 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.839+0000 7ff6e8b89740 10 client.15987 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.839+0000 7ff6e8b89740 20 client.15987 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.839+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/625914264 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55a7dbb459f0 0x55a7dbf4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.839+0000 7ff6e8b89740 4 client.15987 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.840+0000 7ff6e8b89740 1 -- 172.21.15.3:0/625914264 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7dbea6df0 con 0x55a7dbb459f0 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.840+0000 7ff6dbfff640 1 --2- 172.21.15.3:0/625914264 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55a7dbb459f0 0x55a7dbf4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.840+0000 7ff6dbfff640 1 --2- 172.21.15.3:0/625914264 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55a7dbb459f0 0x55a7dbf4ba10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9decc83bd117601 server_cookie=73432092ab142052 in_seq=0 out_seq=0 2024-09-21T06:16:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.840+0000 7ff6bb7fe640 10 client.15987 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6bb7fe640 1 -- 172.21.15.3:0/625914264 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a7dbea6df0 con 0x55a7dbb459f0 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6bb7fe640 10 client.15987 handle_command_reply: tid=0 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6e8b89740 1 client.15987 shutdown 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6e8b89740 1 -- 172.21.15.3:0/625914264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb5e1b0 msgr2=0x55a7dbf398e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/625914264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf398e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff6c402a930 tx=0x7ff6c4005420 comp rx=0 tx=0).stop 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6e8b89740 1 -- 172.21.15.3:0/625914264 shutdown_connections 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/625914264 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55a7dbb459f0 0x55a7dbf4ba10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/625914264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb66390 0x55a7dbf39e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/625914264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb87070 0x55a7dbf3a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/625914264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf398e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6e8b89740 1 -- 172.21.15.3:0/625914264 >> 172.21.15.3:0/625914264 conn(0x55a7dbb7b460 msgr2=0x55a7dbf314a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6bb7fe640 0 client.15987 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6e8b89740 1 -- 172.21.15.3:0/625914264 shutdown_connections 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6e8b89740 1 -- 172.21.15.3:0/625914264 wait complete. 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.841+0000 7ff6e8b89740 20 client.15987 trim_cache size 0 max 16384 2024-09-21T06:16:25.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.855+0000 7ff6e8b89740 1 Processor -- start 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.855+0000 7ff6e8b89740 1 -- start start 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.855+0000 7ff6e8b89740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf2fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.855+0000 7ff6e8b89740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb66390 0x55a7dbf3b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.855+0000 7ff6e8b89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb87070 0x55a7dbf45400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.855+0000 7ff6e8b89740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc1af640 con 0x55a7dbb66390 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.855+0000 7ff6e8b89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc1b1180 con 0x55a7dbb87070 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.855+0000 7ff6e8b89740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc1afd10 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.855+0000 7ff6dbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb66390 0x55a7dbf3b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.855+0000 7ff6dbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb66390 0x55a7dbf3b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35508/0 (socket says 172.21.15.3:35508) 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.855+0000 7ff6dbfff640 1 -- 172.21.15.3:0/3350361973 learned_addr learned my addr 172.21.15.3:0/3350361973 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.855+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/3350361973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf2fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.856+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/3350361973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb87070 0x55a7dbf45400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.856+0000 7ff6dbfff640 1 -- 172.21.15.3:0/3350361973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 msgr2=0x55a7dbf2fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.856+0000 7ff6dbfff640 1 --2- 172.21.15.3:0/3350361973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf2fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.856+0000 7ff6dbfff640 1 -- 172.21.15.3:0/3350361973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb87070 msgr2=0x55a7dbf45400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:25.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.856+0000 7ff6dbfff640 1 --2- 172.21.15.3:0/3350361973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb87070 0x55a7dbf45400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.856+0000 7ff6dbfff640 1 -- 172.21.15.3:0/3350361973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7dbf305b0 con 0x55a7dbb66390 2024-09-21T06:16:25.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.856+0000 7ff6dbfff640 1 --2- 172.21.15.3:0/3350361973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb66390 0x55a7dbf3b7c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00bc80 tx=0x7ff6cc013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441e4b1f3133e06e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:25.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.856+0000 7ff6d1ffb640 1 -- 172.21.15.3:0/3350361973 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc0130d0 con 0x55a7dbb66390 2024-09-21T06:16:25.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.856+0000 7ff6d1ffb640 1 -- 172.21.15.3:0/3350361973 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6cc013270 con 0x55a7dbb66390 2024-09-21T06:16:25.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.856+0000 7ff6d1ffb640 1 -- 172.21.15.3:0/3350361973 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc01c460 con 0x55a7dbb66390 2024-09-21T06:16:25.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.857+0000 7ff6e8b89740 1 -- 172.21.15.3:0/3350361973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb66390 msgr2=0x55a7dbf3b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:25.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.857+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/3350361973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb66390 0x55a7dbf3b7c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00bc80 tx=0x7ff6cc013830 comp rx=0 tx=0).stop 2024-09-21T06:16:25.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.857+0000 7ff6e8b89740 1 -- 172.21.15.3:0/3350361973 shutdown_connections 2024-09-21T06:16:25.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.857+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/3350361973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb87070 0x55a7dbf45400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.857+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/3350361973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb66390 0x55a7dbf3b7c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.857+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/3350361973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf2fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.857+0000 7ff6e8b89740 1 -- 172.21.15.3:0/3350361973 >> 172.21.15.3:0/3350361973 conn(0x55a7dbb7b460 msgr2=0x55a7dbf44cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.857+0000 7ff6e8b89740 1 -- 172.21.15.3:0/3350361973 shutdown_connections 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.857+0000 7ff6e8b89740 1 -- 172.21.15.3:0/3350361973 wait complete. 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.858+0000 7ff6e8b89740 1 -- start start 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.858+0000 7ff6e8b89740 1 Processor -- start 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.858+0000 7ff6e8b89740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf4c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.858+0000 7ff6e8b89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb66390 0x55a7dbf4c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.858+0000 7ff6e8b89740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb87070 0x55a7dbfae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.858+0000 7ff6e8b89740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc1b0ab0 con 0x55a7dbb87070 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.858+0000 7ff6e8b89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc1b03e0 con 0x55a7dbb66390 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.858+0000 7ff6e8b89740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc1ad620 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.858+0000 7ff6e8b89740 10 client.0 _ll_register_callbacks cb 0x7ff6cc00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.859+0000 7ff6d3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf4c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.859+0000 7ff6dbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb66390 0x55a7dbf4c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.859+0000 7ff6d3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf4c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:57630/0 (socket says 172.21.15.3:57630) 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.859+0000 7ff6d3fff640 1 -- 172.21.15.3:0/3920349677 learned_addr learned my addr 172.21.15.3:0/3920349677 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.859+0000 7ff6dbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb66390 0x55a7dbf4c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:54048/0 (socket says 172.21.15.3:54048) 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.859+0000 7ff6d3fff640 1 -- 172.21.15.3:0/3920349677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb66390 msgr2=0x55a7dbf4c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.859+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/3920349677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb66390 0x55a7dbf4c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.859+0000 7ff6d3fff640 1 -- 172.21.15.3:0/3920349677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb87070 msgr2=0x55a7dbfae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.859+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/3920349677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb87070 0x55a7dbfae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.859+0000 7ff6d3fff640 1 -- 172.21.15.3:0/3920349677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7dbfae8a0 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.859+0000 7ff6d3fff640 1 --2- 172.21.15.3:0/3920349677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf4c3f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4005e70 tx=0x7ff6c4005f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fcb4982d7a197 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.859+0000 7ff6bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.869+0000 7ff6d9ffb640 1 -- 172.21.15.3:0/2768146193 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7ff6d40a95c0 con 0x7ff6dc108dd0 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.870+0000 7ff6bb7fe640 1 -- 172.21.15.3:0/3920349677 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4063030 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.870+0000 7ff6bb7fe640 1 -- 172.21.15.3:0/3920349677 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff6c4044040 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.870+0000 7ff6e8b89740 1 -- 172.21.15.3:0/3920349677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6cc013270 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.870+0000 7ff6bb7fe640 1 -- 172.21.15.3:0/3920349677 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff6c4035040 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.870+0000 7ff6e8b89740 10 client.25561 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:25.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.870+0000 7ff6e8b89740 1 -- 172.21.15.3:0/3920349677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7dbfb5bc0 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.871+0000 7ff6bb7fe640 1 -- 172.21.15.3:0/3920349677 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6cc013270 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.871+0000 7ff6bb7fe640 1 -- 172.21.15.3:0/3920349677 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff6c403a480 con 0x55a7dbb5e1b0 2024-09-21T06:16:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.871+0000 7ff6e8b89740 10 client.25561 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.871+0000 7ff6e8b89740 10 client.25561 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:16:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.871+0000 7ff6e8b89740 20 client.25561 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.871+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/3920349677 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55a7dbb1ada0 0x55a7dbfb8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.871+0000 7ff6e8b89740 4 client.25561 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.871+0000 7ff6e8b89740 1 -- 172.21.15.3:0/3920349677 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7dbea28a0 con 0x55a7dbb1ada0 2024-09-21T06:16:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.871+0000 7ff6dbfff640 1 --2- 172.21.15.3:0/3920349677 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55a7dbb1ada0 0x55a7dbfb8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6bb7fe640 10 client.25561 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:16:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6dbfff640 1 --2- 172.21.15.3:0/3920349677 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55a7dbb1ada0 0x55a7dbfb8190 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3155120f4ad5dce2 server_cookie=4f657858dfebacc4 in_seq=0 out_seq=0 2024-09-21T06:16:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6bb7fe640 1 -- 172.21.15.3:0/3920349677 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7dbea28a0 con 0x55a7dbb1ada0 2024-09-21T06:16:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6bb7fe640 10 client.25561 handle_command_reply: tid=0 2024-09-21T06:16:25.910 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:16:25.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6e8b89740 1 client.25561 shutdown 2024-09-21T06:16:25.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6e8b89740 1 -- 172.21.15.3:0/3920349677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 msgr2=0x55a7dbf4c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:25.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/3920349677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf4c3f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4005e70 tx=0x7ff6c4005f30 comp rx=0 tx=0).stop 2024-09-21T06:16:25.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6e8b89740 1 -- 172.21.15.3:0/3920349677 shutdown_connections 2024-09-21T06:16:25.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/3920349677 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55a7dbb1ada0 0x55a7dbfb8190 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/3920349677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7dbb87070 0x55a7dbfae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6bb7fe640 0 client.25561 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:16:25.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/3920349677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dbb66390 0x55a7dbf4c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6e8b89740 1 --2- 172.21.15.3:0/3920349677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7dbb5e1b0 0x55a7dbf4c3f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.872+0000 7ff6e8b89740 1 -- 172.21.15.3:0/3920349677 >> 172.21.15.3:0/3920349677 conn(0x55a7dbb7b460 msgr2=0x55a7dbf31d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:25.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e8b89740 1 -- 172.21.15.3:0/3920349677 shutdown_connections 2024-09-21T06:16:25.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e8b89740 1 -- 172.21.15.3:0/3920349677 wait complete. 2024-09-21T06:16:25.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e8b89740 20 client.25561 trim_cache size 0 max 16384 2024-09-21T06:16:25.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e2615640 1 -- 172.21.15.3:0/2768146193 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff6c80b6710 msgr2=0x7ff6c80b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:25.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e2615640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff6c80b6710 0x7ff6c80b8c20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40e4d60 tx=0x7ff6d407a000 comp rx=0 tx=0).stop 2024-09-21T06:16:25.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e2615640 1 -- 172.21.15.3:0/2768146193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6dc108dd0 msgr2=0x7ff6dc19fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:25.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e2615640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6dc108dd0 0x7ff6dc19fd70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009fd0 tx=0x7ff6d400b260 comp rx=0 tx=0).stop 2024-09-21T06:16:25.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e2615640 1 -- 172.21.15.3:0/2768146193 shutdown_connections 2024-09-21T06:16:25.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e2615640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff6c80b6710 0x7ff6c80b8c20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e2615640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6dc108480 0x7ff6dc19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e2615640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6dc108dd0 0x7ff6dc19fd70 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e2615640 1 --2- 172.21.15.3:0/2768146193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6dc102b40 0x7ff6dc19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:25.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e2615640 1 -- 172.21.15.3:0/2768146193 >> 172.21.15.3:0/2768146193 conn(0x7ff6dc079750 msgr2=0x7ff6dc1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:25.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e2615640 1 -- 172.21.15.3:0/2768146193 shutdown_connections 2024-09-21T06:16:25.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:25.873+0000 7ff6e2615640 1 -- 172.21.15.3:0/2768146193 wait complete. 2024-09-21T06:16:26.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:16:27.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:26 smithi003 ceph-mon[31227]: pgmap v368: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T06:16:27.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:26 smithi003 ceph-mon[31227]: mgrmap e27: x(active, since 10m), standbys: z, y 2024-09-21T06:16:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:27.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:26 smithi137 ceph-mon[34464]: pgmap v368: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T06:16:27.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:26 smithi137 ceph-mon[34464]: mgrmap e27: x(active, since 10m), standbys: z, y 2024-09-21T06:16:27.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:26 smithi204 ceph-mon[34659]: pgmap v368: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T06:16:27.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:26 smithi204 ceph-mon[34659]: mgrmap e27: x(active, since 10m), standbys: z, y 2024-09-21T06:16:27.524 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T06:16:27.637 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T06:16:27.678 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T06:16:27.680 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T06:16:27.680 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T06:16:27.732 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T06:16:27.735 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:16:27.736 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T06:16:27.752 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-21T06:16:27.753 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-21T06:16:27.754 DEBUG:teuthology.orchestra.run.smithi003:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-21T06:16:27.816 INFO:tasks.workunit.client.0.smithi003.stderr:+ dbench 1 2024-09-21T06:16:27.833 INFO:tasks.workunit.client.0.smithi003.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-21T06:16:27.834 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-21T06:16:27.834 INFO:tasks.workunit.client.0.smithi003.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-21T06:16:27.844 INFO:tasks.workunit.client.0.smithi003.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-21T06:16:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.876+0000 7fa627339640 1 -- 172.21.15.3:0/4049483967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa620107b70 msgr2=0x7fa620107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.876+0000 7fa627339640 1 --2- 172.21.15.3:0/4049483967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa620107b70 0x7fa620107f80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009900 tx=0x7fa61c02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.876+0000 7fa627339640 1 -- 172.21.15.3:0/4049483967 shutdown_connections 2024-09-21T06:16:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.876+0000 7fa627339640 1 --2- 172.21.15.3:0/4049483967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6201084c0 0x7fa62010ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.876+0000 7fa627339640 1 --2- 172.21.15.3:0/4049483967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa620107b70 0x7fa620107f80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.876+0000 7fa627339640 1 --2- 172.21.15.3:0/4049483967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa620102230 0x7fa620102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.876+0000 7fa627339640 1 -- 172.21.15.3:0/4049483967 >> 172.21.15.3:0/4049483967 conn(0x7fa620079e40 msgr2=0x7fa62007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.877+0000 7fa627339640 1 -- 172.21.15.3:0/4049483967 shutdown_connections 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.877+0000 7fa627339640 1 -- 172.21.15.3:0/4049483967 wait complete. 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.877+0000 7fa627339640 1 Processor -- start 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.877+0000 7fa627339640 1 -- start start 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa627339640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa620102230 0x7fa62019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa627339640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa620107b70 0x7fa62019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa627339640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6201084c0 0x7fa62019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa627339640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa620113ae0 con 0x7fa620102230 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa627339640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa620113960 con 0x7fa620107b70 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa627339640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa620113c60 con 0x7fa6201084c0 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa6250d5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa620102230 0x7fa62019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa6250d5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa620102230 0x7fa62019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35550/0 (socket says 172.21.15.3:35550) 2024-09-21T06:16:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa6250d5640 1 -- 172.21.15.3:0/3790131444 learned_addr learned my addr 172.21.15.3:0/3790131444 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa6248d4640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa620107b70 0x7fa62019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa6258d6640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6201084c0 0x7fa62019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa6250d5640 1 -- 172.21.15.3:0/3790131444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6201084c0 msgr2=0x7fa62019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa6250d5640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6201084c0 0x7fa62019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa6250d5640 1 -- 172.21.15.3:0/3790131444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa620107b70 msgr2=0x7fa62019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa6250d5640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa620107b70 0x7fa62019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa6250d5640 1 -- 172.21.15.3:0/3790131444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6201a0420 con 0x7fa620102230 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa6258d6640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6201084c0 0x7fa62019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa6250d5640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa620102230 0x7fa62019a570 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa61400c890 tx=0x7fa61400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.878+0000 7fa6127fc640 1 -- 172.21.15.3:0/3790131444 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614004220 con 0x7fa620102230 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.879+0000 7fa627339640 1 -- 172.21.15.3:0/3790131444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6201a0710 con 0x7fa620102230 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.879+0000 7fa6127fc640 1 -- 172.21.15.3:0/3790131444 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6140043c0 con 0x7fa620102230 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.879+0000 7fa6127fc640 1 -- 172.21.15.3:0/3790131444 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614004a00 con 0x7fa620102230 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.879+0000 7fa627339640 1 -- 172.21.15.3:0/3790131444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6201a6f30 con 0x7fa620102230 2024-09-21T06:16:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.881+0000 7fa6127fc640 1 -- 172.21.15.3:0/3790131444 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fa6140079b0 con 0x7fa620102230 2024-09-21T06:16:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.881+0000 7fa62d8ad740 1 Processor -- start 2024-09-21T06:16:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa62d8ad740 1 -- start start 2024-09-21T06:16:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa62d8ad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 0x55a04b4ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa62d8ad740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4ab800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa62d8ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4abd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa62d8ad740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a04b421ea0 con 0x55a04b0db1b0 2024-09-21T06:16:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa62d8ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a04b4112d0 con 0x55a04b0e3390 2024-09-21T06:16:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa62d8ad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a04b4150e0 con 0x55a04b104070 2024-09-21T06:16:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa6250d5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa6250d5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4ab800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35562/0 (socket says 172.21.15.3:35562) 2024-09-21T06:16:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa6250d5640 1 -- 172.21.15.3:0/2069369232 learned_addr learned my addr 172.21.15.3:0/2069369232 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa6258d6640 1 --2- 172.21.15.3:0/2069369232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 0x55a04b4ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa6248d4640 1 --2- 172.21.15.3:0/2069369232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa6250d5640 1 -- 172.21.15.3:0/2069369232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 msgr2=0x55a04b4ab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa6250d5640 1 --2- 172.21.15.3:0/2069369232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 0x55a04b4ab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa6250d5640 1 -- 172.21.15.3:0/2069369232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 msgr2=0x55a04b4abd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa6250d5640 1 --2- 172.21.15.3:0/2069369232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4abd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa6250d5640 1 -- 172.21.15.3:0/2069369232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04b16f650 con 0x55a04b0db1b0 2024-09-21T06:16:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.882+0000 7fa6248d4640 1 --2- 172.21.15.3:0/2069369232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.883+0000 7fa6127fc640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa60c0b66b0 0x7fa60c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.883+0000 7fa6258d6640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa60c0b66b0 0x7fa60c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.883+0000 7fa6250d5640 1 --2- 172.21.15.3:0/2069369232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4ab800 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa6140e4b80 tx=0x7fa6140edb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd93c6126b15887f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.883+0000 7fa6127fc640 1 -- 172.21.15.3:0/3790131444 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fa6140e7480 con 0x7fa620102230 2024-09-21T06:16:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.883+0000 7fa60b7fe640 1 -- 172.21.15.3:0/2069369232 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61401d030 con 0x55a04b0db1b0 2024-09-21T06:16:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.883+0000 7fa60b7fe640 1 -- 172.21.15.3:0/2069369232 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa614005290 con 0x55a04b0db1b0 2024-09-21T06:16:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.883+0000 7fa60b7fe640 1 -- 172.21.15.3:0/2069369232 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140b6030 con 0x55a04b0db1b0 2024-09-21T06:16:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.884+0000 7fa62d8ad740 1 -- 172.21.15.3:0/2069369232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 msgr2=0x55a04b4ab800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.884+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/2069369232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4ab800 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa6140e4b80 tx=0x7fa6140edb00 comp rx=0 tx=0).stop 2024-09-21T06:16:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.884+0000 7fa6258d6640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa60c0b66b0 0x7fa60c0b8bc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa618007f10 tx=0x7fa6180076a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.884+0000 7fa62d8ad740 1 -- 172.21.15.3:0/2069369232 shutdown_connections 2024-09-21T06:16:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.884+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/2069369232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.884+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/2069369232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4ab800 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.884+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/2069369232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 0x55a04b4ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.884+0000 7fa62d8ad740 1 -- 172.21.15.3:0/2069369232 >> 172.21.15.3:0/2069369232 conn(0x55a04b0f8460 msgr2=0x55a04b4a91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.884+0000 7fa62d8ad740 1 -- 172.21.15.3:0/2069369232 shutdown_connections 2024-09-21T06:16:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.884+0000 7fa62d8ad740 1 -- 172.21.15.3:0/2069369232 wait complete. 2024-09-21T06:16:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.884+0000 7fa62d8ad740 1 -- start start 2024-09-21T06:16:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.885+0000 7fa62d8ad740 1 Processor -- start 2024-09-21T06:16:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.885+0000 7fa62d8ad740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4bad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.885+0000 7fa62d8ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4bb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.885+0000 7fa62d8ad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 0x55a04b4c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.885+0000 7fa62d8ad740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a04b272700 con 0x55a04b0db1b0 2024-09-21T06:16:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.885+0000 7fa62d8ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a04b3ac160 con 0x55a04b0e3390 2024-09-21T06:16:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.885+0000 7fa62d8ad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a04b4bddd0 con 0x55a04b104070 2024-09-21T06:16:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.885+0000 7fa62d8ad740 10 client.0 _ll_register_callbacks cb 0x55a04b3c59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.885+0000 7fa6250d5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.885+0000 7fa6250d5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4bad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35568/0 (socket says 172.21.15.3:35568) 2024-09-21T06:16:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.885+0000 7fa6250d5640 1 -- 172.21.15.3:0/1615372712 learned_addr learned my addr 172.21.15.3:0/1615372712 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.885+0000 7fa6248d4640 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4bb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa6258d6640 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 0x55a04b4c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa6250d5640 1 -- 172.21.15.3:0/1615372712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 msgr2=0x55a04b4c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa6250d5640 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 0x55a04b4c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa6250d5640 1 -- 172.21.15.3:0/1615372712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 msgr2=0x55a04b4bb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa6250d5640 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4bb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa6250d5640 1 -- 172.21.15.3:0/1615372712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04b3d9c90 con 0x55a04b0db1b0 2024-09-21T06:16:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa6258d6640 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 0x55a04b4c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa6250d5640 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4bad50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa6140e5210 tx=0x7fa614007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62b64d6dd28f9cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa608ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa608ff9640 1 -- 172.21.15.3:0/1615372712 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61401d040 con 0x55a04b0db1b0 2024-09-21T06:16:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa608ff9640 1 -- 172.21.15.3:0/1615372712 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fa6140e7030 con 0x55a04b0db1b0 2024-09-21T06:16:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa62d8ad740 1 -- 172.21.15.3:0/1615372712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a04b423df0 con 0x55a04b0db1b0 2024-09-21T06:16:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa608ff9640 1 -- 172.21.15.3:0/1615372712 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6140a91b0 con 0x55a04b0db1b0 2024-09-21T06:16:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa62d8ad740 10 client.16002 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa62d8ad740 1 -- 172.21.15.3:0/1615372712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a04b4c9b10 con 0x55a04b0db1b0 2024-09-21T06:16:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.886+0000 7fa608ff9640 1 -- 172.21.15.3:0/1615372712 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a04b423df0 con 0x55a04b0db1b0 2024-09-21T06:16:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.887+0000 7fa608ff9640 1 -- 172.21.15.3:0/1615372712 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa6140a8c60 con 0x55a04b0db1b0 2024-09-21T06:16:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.887+0000 7fa62d8ad740 10 client.16002 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.887+0000 7fa62d8ad740 10 client.16002 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:16:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.887+0000 7fa62d8ad740 20 client.16002 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.887+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55a04b0c29f0 0x55a04b4cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.887+0000 7fa62d8ad740 4 client.16002 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.887+0000 7fa62d8ad740 1 -- 172.21.15.3:0/1615372712 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a04b41f8a0 con 0x55a04b0c29f0 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.887+0000 7fa6248d4640 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55a04b0c29f0 0x55a04b4cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.888+0000 7fa608ff9640 10 client.16002 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.888+0000 7fa6248d4640 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55a04b0c29f0 0x55a04b4cce70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a0f97f0c34248fc7 server_cookie=db60326aa39ec572 in_seq=0 out_seq=0 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa608ff9640 1 -- 172.21.15.3:0/1615372712 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a04b41f8a0 con 0x55a04b0c29f0 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa608ff9640 10 client.16002 handle_command_reply: tid=0 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa62d8ad740 1 client.16002 shutdown 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa62d8ad740 1 -- 172.21.15.3:0/1615372712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 msgr2=0x55a04b4bad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4bad50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa6140e5210 tx=0x7fa614007ed0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa62d8ad740 1 -- 172.21.15.3:0/1615372712 shutdown_connections 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55a04b0c29f0 0x55a04b4cce70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 0x55a04b4c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4bb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa608ff9640 0 client.16002 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:16:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/1615372712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4bad50 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa62d8ad740 1 -- 172.21.15.3:0/1615372712 >> 172.21.15.3:0/1615372712 conn(0x55a04b0f8460 msgr2=0x55a04b4ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa62d8ad740 1 -- 172.21.15.3:0/1615372712 shutdown_connections 2024-09-21T06:16:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa62d8ad740 1 -- 172.21.15.3:0/1615372712 wait complete. 2024-09-21T06:16:27.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.889+0000 7fa62d8ad740 20 client.16002 trim_cache size 0 max 16384 2024-09-21T06:16:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.902+0000 7fa62d8ad740 1 Processor -- start 2024-09-21T06:16:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.902+0000 7fa62d8ad740 1 -- start start 2024-09-21T06:16:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa62d8ad740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4a8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa62d8ad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b0e3390 0x55a04b4a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa62d8ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b104070 0x55a04b4b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa62d8ad740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201ad500 con 0x55a04b0db1b0 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa62d8ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201af040 con 0x55a04b104070 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa62d8ad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201adbd0 con 0x55a04b0e3390 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa6250d5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa6250d5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4a8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35576/0 (socket says 172.21.15.3:35576) 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa6250d5640 1 -- 172.21.15.3:0/3660390891 learned_addr learned my addr 172.21.15.3:0/3660390891 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa6248d4640 1 --2- 172.21.15.3:0/3660390891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b0e3390 0x55a04b4a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa6258d6640 1 --2- 172.21.15.3:0/3660390891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b104070 0x55a04b4b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa6250d5640 1 -- 172.21.15.3:0/3660390891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b0e3390 msgr2=0x55a04b4a8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa6250d5640 1 --2- 172.21.15.3:0/3660390891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b0e3390 0x55a04b4a8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa6250d5640 1 -- 172.21.15.3:0/3660390891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b104070 msgr2=0x55a04b4b7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa6250d5640 1 --2- 172.21.15.3:0/3660390891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b104070 0x55a04b4b7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa6250d5640 1 -- 172.21.15.3:0/3660390891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04b4ae0c0 con 0x55a04b0db1b0 2024-09-21T06:16:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.903+0000 7fa6258d6640 1 --2- 172.21.15.3:0/3660390891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b104070 0x55a04b4b7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.904+0000 7fa6250d5640 1 --2- 172.21.15.3:0/3660390891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4a8760 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa6140058c0 tx=0x7fa6140edb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dc3e188f744bef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.904+0000 7fa60affd640 1 -- 172.21.15.3:0/3660390891 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140b6030 con 0x55a04b0db1b0 2024-09-21T06:16:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.904+0000 7fa60affd640 1 -- 172.21.15.3:0/3660390891 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6140eada0 con 0x55a04b0db1b0 2024-09-21T06:16:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.904+0000 7fa60affd640 1 -- 172.21.15.3:0/3660390891 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61401d070 con 0x55a04b0db1b0 2024-09-21T06:16:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.905+0000 7fa62d8ad740 1 -- 172.21.15.3:0/3660390891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 msgr2=0x55a04b4a8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.905+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/3660390891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4a8760 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa6140058c0 tx=0x7fa6140edb00 comp rx=0 tx=0).stop 2024-09-21T06:16:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.905+0000 7fa62d8ad740 1 -- 172.21.15.3:0/3660390891 shutdown_connections 2024-09-21T06:16:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.905+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/3660390891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b0e3390 0x55a04b4a8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.905+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/3660390891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b104070 0x55a04b4b7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.905+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/3660390891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4a8760 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.905+0000 7fa62d8ad740 1 -- 172.21.15.3:0/3660390891 >> 172.21.15.3:0/3660390891 conn(0x55a04b0f8460 msgr2=0x55a04b4b95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.905+0000 7fa62d8ad740 1 -- 172.21.15.3:0/3660390891 shutdown_connections 2024-09-21T06:16:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.905+0000 7fa62d8ad740 1 -- 172.21.15.3:0/3660390891 wait complete. 2024-09-21T06:16:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.905+0000 7fa62d8ad740 1 -- start start 2024-09-21T06:16:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.905+0000 7fa62d8ad740 1 Processor -- start 2024-09-21T06:16:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa62d8ad740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa62d8ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa62d8ad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 0x55a04b4ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa62d8ad740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201ae970 con 0x55a04b0db1b0 2024-09-21T06:16:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa62d8ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201ae2a0 con 0x55a04b0e3390 2024-09-21T06:16:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa62d8ad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201ab4e0 con 0x55a04b104070 2024-09-21T06:16:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa62d8ad740 10 client.0 _ll_register_callbacks cb 0x7fa618007540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa6250d5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa6250d5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35590/0 (socket says 172.21.15.3:35590) 2024-09-21T06:16:27.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa6250d5640 1 -- 172.21.15.3:0/252045793 learned_addr learned my addr 172.21.15.3:0/252045793 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:27.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa6248d4640 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa6258d6640 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 0x55a04b4ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa6250d5640 1 -- 172.21.15.3:0/252045793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 msgr2=0x55a04b4ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa6250d5640 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 0x55a04b4ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa6250d5640 1 -- 172.21.15.3:0/252045793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 msgr2=0x55a04b4ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa6250d5640 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa6250d5640 1 -- 172.21.15.3:0/252045793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04b5365f0 con 0x55a04b0db1b0 2024-09-21T06:16:27.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa6248d4640 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:27.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa6250d5640 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4cdf20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa6140aa580 tx=0x7fa6140ef0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2870d1b46beca875 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa608ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa608ff9640 1 -- 172.21.15.3:0/252045793 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61401d030 con 0x55a04b0db1b0 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa608ff9640 1 -- 172.21.15.3:0/252045793 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fa6140e7250 con 0x55a04b0db1b0 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.906+0000 7fa62d8ad740 1 -- 172.21.15.3:0/252045793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6140eada0 con 0x55a04b0db1b0 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.907+0000 7fa608ff9640 1 -- 172.21.15.3:0/252045793 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6140aca80 con 0x55a04b0db1b0 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.907+0000 7fa62d8ad740 10 client.16008 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.907+0000 7fa62d8ad740 1 -- 172.21.15.3:0/252045793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a04b5365f0 con 0x55a04b0db1b0 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.907+0000 7fa608ff9640 1 -- 172.21.15.3:0/252045793 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6140eada0 con 0x55a04b0db1b0 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.907+0000 7fa608ff9640 1 -- 172.21.15.3:0/252045793 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa6140ac570 con 0x55a04b0db1b0 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.907+0000 7fa62d8ad740 10 client.16008 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.907+0000 7fa62d8ad740 10 client.16008 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.907+0000 7fa62d8ad740 20 client.16008 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.907+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55a04b097da0 0x55a04b539b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.907+0000 7fa62d8ad740 4 client.16008 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.907+0000 7fa62d8ad740 1 -- 172.21.15.3:0/252045793 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a04b41ccd0 con 0x55a04b097da0 2024-09-21T06:16:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.907+0000 7fa6248d4640 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55a04b097da0 0x55a04b539b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa608ff9640 10 client.16008 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:16:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa6248d4640 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55a04b097da0 0x55a04b539b50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=16be172158c92d2a server_cookie=4a40f28832e30613 in_seq=0 out_seq=0 2024-09-21T06:16:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa608ff9640 1 -- 172.21.15.3:0/252045793 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a04b41ccd0 con 0x55a04b097da0 2024-09-21T06:16:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa608ff9640 10 client.16008 handle_command_reply: tid=0 2024-09-21T06:16:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa62d8ad740 1 client.16008 shutdown 2024-09-21T06:16:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa62d8ad740 1 -- 172.21.15.3:0/252045793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 msgr2=0x55a04b4cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4cdf20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa6140aa580 tx=0x7fa6140ef0b0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa62d8ad740 1 -- 172.21.15.3:0/252045793 shutdown_connections 2024-09-21T06:16:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55a04b097da0 0x55a04b539b50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a04b104070 0x55a04b4ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a04b0e3390 0x55a04b4ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa62d8ad740 1 --2- 172.21.15.3:0/252045793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a04b0db1b0 0x55a04b4cdf20 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa608ff9640 0 client.16008 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:16:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa62d8ad740 1 -- 172.21.15.3:0/252045793 >> 172.21.15.3:0/252045793 conn(0x55a04b0f8460 msgr2=0x55a04b4ae360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa62d8ad740 1 -- 172.21.15.3:0/252045793 shutdown_connections 2024-09-21T06:16:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.908+0000 7fa62d8ad740 1 -- 172.21.15.3:0/252045793 wait complete. 2024-09-21T06:16:27.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.909+0000 7fa62d8ad740 20 client.16008 trim_cache size 0 max 16384 2024-09-21T06:16:27.925 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:16:27.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.909+0000 7fa627339640 1 -- 172.21.15.3:0/3790131444 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa60c0b66b0 msgr2=0x7fa60c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.909+0000 7fa627339640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa60c0b66b0 0x7fa60c0b8bc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa618007f10 tx=0x7fa6180076a0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.909+0000 7fa627339640 1 -- 172.21.15.3:0/3790131444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa620102230 msgr2=0x7fa62019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.909+0000 7fa627339640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa620102230 0x7fa62019a570 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa61400c890 tx=0x7fa61400cd50 comp rx=0 tx=0).stop 2024-09-21T06:16:27.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.909+0000 7fa627339640 1 -- 172.21.15.3:0/3790131444 shutdown_connections 2024-09-21T06:16:27.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.909+0000 7fa627339640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa60c0b66b0 0x7fa60c0b8bc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.909+0000 7fa627339640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6201084c0 0x7fa62019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.909+0000 7fa627339640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa620107b70 0x7fa62019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.909+0000 7fa627339640 1 --2- 172.21.15.3:0/3790131444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa620102230 0x7fa62019a570 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.909+0000 7fa627339640 1 -- 172.21.15.3:0/3790131444 >> 172.21.15.3:0/3790131444 conn(0x7fa620079e40 msgr2=0x7fa620103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:27.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.909+0000 7fa627339640 1 -- 172.21.15.3:0/3790131444 shutdown_connections 2024-09-21T06:16:27.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:27.909+0000 7fa627339640 1 -- 172.21.15.3:0/3790131444 wait complete. 2024-09-21T06:16:27.944 INFO:tasks.workunit.client.0.smithi003.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-21T06:16:27.944 INFO:tasks.workunit.client.0.smithi003.stdout:releasing clients 2024-09-21T06:16:28.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:16:28.944 INFO:tasks.workunit.client.0.smithi003.stdout: 1 3792 39.76 MB/sec warmup 1 sec latency 7.670 ms 2024-09-21T06:16:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:29.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:28 smithi003 ceph-mon[31227]: pgmap v369: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-21T06:16:29.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:28 smithi137 ceph-mon[34464]: pgmap v369: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-21T06:16:29.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:28 smithi204 ceph-mon[34659]: pgmap v369: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-21T06:16:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.484+0000 7f339294b640 1 -- 172.21.15.3:0/1907987097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f338c1048c0 msgr2=0x7f338c104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.484+0000 7f339294b640 1 --2- 172.21.15.3:0/1907987097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f338c1048c0 0x7f338c104cd0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f337c009900 tx=0x7f337c02a410 comp rx=0 tx=0).stop 2024-09-21T06:16:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.484+0000 7f339294b640 1 -- 172.21.15.3:0/1907987097 shutdown_connections 2024-09-21T06:16:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.484+0000 7f339294b640 1 --2- 172.21.15.3:0/1907987097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f338c1052a0 0x7f338c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.484+0000 7f339294b640 1 --2- 172.21.15.3:0/1907987097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f338c10de70 0x7f338c110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.484+0000 7f339294b640 1 --2- 172.21.15.3:0/1907987097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f338c1048c0 0x7f338c104cd0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.484+0000 7f339294b640 1 -- 172.21.15.3:0/1907987097 >> 172.21.15.3:0/1907987097 conn(0x7f338c079760 msgr2=0x7f338c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.484+0000 7f339294b640 1 -- 172.21.15.3:0/1907987097 shutdown_connections 2024-09-21T06:16:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.484+0000 7f339294b640 1 -- 172.21.15.3:0/1907987097 wait complete. 2024-09-21T06:16:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.484+0000 7f339294b640 1 Processor -- start 2024-09-21T06:16:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.484+0000 7f339294b640 1 -- start start 2024-09-21T06:16:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f339294b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f338c1048c0 0x7f338c1027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f339294b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f338c1052a0 0x7f338c102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f339294b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f338c10de70 0x7f338c103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f339294b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f338c113ad0 con 0x7f338c1052a0 2024-09-21T06:16:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f339294b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f338c113950 con 0x7f338c10de70 2024-09-21T06:16:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f339294b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f338c113c50 con 0x7f338c1048c0 2024-09-21T06:16:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f338b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f338c1052a0 0x7f338c102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f338b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f338c1052a0 0x7f338c102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35600/0 (socket says 172.21.15.3:35600) 2024-09-21T06:16:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f338b7fe640 1 -- 172.21.15.3:0/3043082678 learned_addr learned my addr 172.21.15.3:0/3043082678 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f3390ee8640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f338c10de70 0x7f338c103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f338bfff640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f338c1048c0 0x7f338c1027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f338b7fe640 1 -- 172.21.15.3:0/3043082678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f338c1048c0 msgr2=0x7f338c1027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f338b7fe640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f338c1048c0 0x7f338c1027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f338b7fe640 1 -- 172.21.15.3:0/3043082678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f338c10de70 msgr2=0x7f338c103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f338b7fe640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f338c10de70 0x7f338c103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.485+0000 7f338b7fe640 1 -- 172.21.15.3:0/3043082678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338c1a87c0 con 0x7f338c1052a0 2024-09-21T06:16:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.486+0000 7f338b7fe640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f338c1052a0 0x7f338c102d10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f338000e9c0 tx=0x7f338000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.486+0000 7f33897fa640 1 -- 172.21.15.3:0/3043082678 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338000ccc0 con 0x7f338c1052a0 2024-09-21T06:16:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.486+0000 7f33897fa640 1 -- 172.21.15.3:0/3043082678 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33800054d0 con 0x7f338c1052a0 2024-09-21T06:16:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.486+0000 7f33897fa640 1 -- 172.21.15.3:0/3043082678 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380010430 con 0x7f338c1052a0 2024-09-21T06:16:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.486+0000 7f339294b640 1 -- 172.21.15.3:0/3043082678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338c1a8ab0 con 0x7f338c1052a0 2024-09-21T06:16:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.486+0000 7f339294b640 1 -- 172.21.15.3:0/3043082678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f338c1a8ff0 con 0x7f338c1052a0 2024-09-21T06:16:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f3398ebf740 1 Processor -- start 2024-09-21T06:16:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f3398ebf740 1 -- start start 2024-09-21T06:16:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f33897fa640 1 -- 172.21.15.3:0/3043082678 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f3380005020 con 0x7f338c1052a0 2024-09-21T06:16:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f3398ebf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53656070 0x555f539fd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f3398ebf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f5362d1b0 0x555f539fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f3398ebf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f53635390 0x555f539fe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f3398ebf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555f53973ea0 con 0x555f53635390 2024-09-21T06:16:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f3398ebf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555f539632d0 con 0x555f53656070 2024-09-21T06:16:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f3398ebf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555f539670e0 con 0x555f5362d1b0 2024-09-21T06:16:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f3390ee8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53656070 0x555f539fd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f338b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f53635390 0x555f539fe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f3390ee8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53656070 0x555f539fd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:54126/0 (socket says 172.21.15.3:54126) 2024-09-21T06:16:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f3390ee8640 1 -- 172.21.15.3:0/2621167505 learned_addr learned my addr 172.21.15.3:0/2621167505 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.489+0000 7f338bfff640 1 --2- 172.21.15.3:0/2621167505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f5362d1b0 0x555f539fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.490+0000 7f3390ee8640 1 -- 172.21.15.3:0/2621167505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f5362d1b0 msgr2=0x555f539fdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.490+0000 7f3390ee8640 1 --2- 172.21.15.3:0/2621167505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f5362d1b0 0x555f539fdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.490+0000 7f3390ee8640 1 -- 172.21.15.3:0/2621167505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f53635390 msgr2=0x555f539fe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.490+0000 7f3390ee8640 1 --2- 172.21.15.3:0/2621167505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f53635390 0x555f539fe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.490+0000 7f3390ee8640 1 -- 172.21.15.3:0/2621167505 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f536c1650 con 0x555f53656070 2024-09-21T06:16:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.490+0000 7f338b7fe640 1 --2- 172.21.15.3:0/2621167505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f53635390 0x555f539fe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.490+0000 7f33897fa640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f33740b6700 0x7f33740b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.490+0000 7f3390ee8640 1 --2- 172.21.15.3:0/2621167505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53656070 0x555f539fd6a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3384009fd0 tx=0x7f33840118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3bec646149de7cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.490+0000 7f338bfff640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f33740b6700 0x7f33740b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.490+0000 7f33897fa640 1 -- 172.21.15.3:0/3043082678 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f33800e7a50 con 0x7f338c1052a0 2024-09-21T06:16:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.490+0000 7f33727fc640 1 -- 172.21.15.3:0/2621167505 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384004d20 con 0x555f53656070 2024-09-21T06:16:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.490+0000 7f33727fc640 1 -- 172.21.15.3:0/2621167505 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3384004050 con 0x555f53656070 2024-09-21T06:16:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.490+0000 7f33727fc640 1 -- 172.21.15.3:0/2621167505 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384005280 con 0x555f53656070 2024-09-21T06:16:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.491+0000 7f3398ebf740 1 -- 172.21.15.3:0/2621167505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53656070 msgr2=0x555f539fd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.491+0000 7f338bfff640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f33740b6700 0x7f33740b8c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f337c009230 tx=0x7f337c005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.491+0000 7f3398ebf740 1 --2- 172.21.15.3:0/2621167505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53656070 0x555f539fd6a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3384009fd0 tx=0x7f33840118d0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.491+0000 7f3398ebf740 1 -- 172.21.15.3:0/2621167505 shutdown_connections 2024-09-21T06:16:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.491+0000 7f3398ebf740 1 --2- 172.21.15.3:0/2621167505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f5362d1b0 0x555f539fdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.491+0000 7f3398ebf740 1 --2- 172.21.15.3:0/2621167505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f53635390 0x555f539fe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.491+0000 7f3398ebf740 1 --2- 172.21.15.3:0/2621167505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53656070 0x555f539fd6a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.491+0000 7f3398ebf740 1 -- 172.21.15.3:0/2621167505 >> 172.21.15.3:0/2621167505 conn(0x555f5364a460 msgr2=0x555f539fb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.492+0000 7f3398ebf740 1 -- 172.21.15.3:0/2621167505 shutdown_connections 2024-09-21T06:16:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.492+0000 7f3398ebf740 1 -- 172.21.15.3:0/2621167505 wait complete. 2024-09-21T06:16:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.492+0000 7f3398ebf740 1 -- start start 2024-09-21T06:16:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.492+0000 7f3398ebf740 1 Processor -- start 2024-09-21T06:16:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f3398ebf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 0x555f53a08dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f3398ebf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53635390 0x555f53a09300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f3398ebf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f53656070 0x555f53a15170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f3398ebf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555f537c4700 con 0x555f5362d1b0 2024-09-21T06:16:29.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f3398ebf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555f538fe160 con 0x555f53635390 2024-09-21T06:16:29.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f3398ebf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555f53a0be80 con 0x555f53656070 2024-09-21T06:16:29.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f3398ebf740 10 client.0 _ll_register_callbacks cb 0x555f539179e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:29.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f338bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 0x555f53a08dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f338bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 0x555f53a08dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35622/0 (socket says 172.21.15.3:35622) 2024-09-21T06:16:29.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f338bfff640 1 -- 172.21.15.3:0/1356062328 learned_addr learned my addr 172.21.15.3:0/1356062328 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:29.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f338b7fe640 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53635390 0x555f53a09300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f3390ee8640 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f53656070 0x555f53a15170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f338bfff640 1 -- 172.21.15.3:0/1356062328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f53656070 msgr2=0x555f53a15170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:29.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f338bfff640 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f53656070 0x555f53a15170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f338bfff640 1 -- 172.21.15.3:0/1356062328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53635390 msgr2=0x555f53a09300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:29.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f338bfff640 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53635390 0x555f53a09300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f3390ee8640 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f53656070 0x555f53a15170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:16:29.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.493+0000 7f338bfff640 1 -- 172.21.15.3:0/1356062328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f5392bc90 con 0x555f5362d1b0 2024-09-21T06:16:29.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.494+0000 7f338bfff640 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 0x555f53a08dc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f338c06c8b0 tx=0x7f337c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268be7743be1ee5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:29.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.494+0000 7f336b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:29.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.494+0000 7f336b7fe640 1 -- 172.21.15.3:0/1356062328 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c04c030 con 0x555f5362d1b0 2024-09-21T06:16:29.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.494+0000 7f336b7fe640 1 -- 172.21.15.3:0/1356062328 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f337c0043d0 con 0x555f5362d1b0 2024-09-21T06:16:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.494+0000 7f3398ebf740 1 -- 172.21.15.3:0/1356062328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3384004050 con 0x555f5362d1b0 2024-09-21T06:16:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.494+0000 7f336b7fe640 1 -- 172.21.15.3:0/1356062328 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f337c035040 con 0x555f5362d1b0 2024-09-21T06:16:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.494+0000 7f3398ebf740 10 client.16020 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.494+0000 7f3398ebf740 1 -- 172.21.15.3:0/1356062328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f53a17bc0 con 0x555f5362d1b0 2024-09-21T06:16:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.495+0000 7f336b7fe640 1 -- 172.21.15.3:0/1356062328 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3384004050 con 0x555f5362d1b0 2024-09-21T06:16:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.495+0000 7f336b7fe640 1 -- 172.21.15.3:0/1356062328 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f337c048600 con 0x555f5362d1b0 2024-09-21T06:16:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.495+0000 7f3398ebf740 10 client.16020 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.495+0000 7f3398ebf740 10 client.16020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:16:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.495+0000 7f3398ebf740 20 client.16020 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.495+0000 7f3398ebf740 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555f536149f0 0x555f53a1b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.495+0000 7f3398ebf740 4 client.16020 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.495+0000 7f3398ebf740 1 -- 172.21.15.3:0/1356062328 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f53975df0 con 0x555f536149f0 2024-09-21T06:16:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.495+0000 7f338b7fe640 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555f536149f0 0x555f53a1b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.496+0000 7f338b7fe640 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555f536149f0 0x555f53a1b0b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5794a288604f08b server_cookie=14eff8f4d9c3acf3 in_seq=0 out_seq=0 2024-09-21T06:16:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.496+0000 7f336b7fe640 10 client.16020 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:16:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.496+0000 7f336b7fe640 1 -- 172.21.15.3:0/1356062328 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555f53975df0 con 0x555f536149f0 2024-09-21T06:16:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.496+0000 7f336b7fe640 10 client.16020 handle_command_reply: tid=0 2024-09-21T06:16:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.496+0000 7f3398ebf740 1 client.16020 shutdown 2024-09-21T06:16:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.497+0000 7f3398ebf740 1 -- 172.21.15.3:0/1356062328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 msgr2=0x555f53a08dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.497+0000 7f3398ebf740 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 0x555f53a08dc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f338c06c8b0 tx=0x7f337c009260 comp rx=0 tx=0).stop 2024-09-21T06:16:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.497+0000 7f3398ebf740 1 -- 172.21.15.3:0/1356062328 shutdown_connections 2024-09-21T06:16:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.497+0000 7f3398ebf740 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555f536149f0 0x555f53a1b0b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.497+0000 7f3398ebf740 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f53656070 0x555f53a15170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.497+0000 7f3398ebf740 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53635390 0x555f53a09300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.497+0000 7f336b7fe640 0 client.16020 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:16:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.497+0000 7f3398ebf740 1 --2- 172.21.15.3:0/1356062328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 0x555f53a08dc0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.497+0000 7f3398ebf740 1 -- 172.21.15.3:0/1356062328 >> 172.21.15.3:0/1356062328 conn(0x555f5364a460 msgr2=0x555f53a00880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.497+0000 7f3398ebf740 1 -- 172.21.15.3:0/1356062328 shutdown_connections 2024-09-21T06:16:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.497+0000 7f3398ebf740 1 -- 172.21.15.3:0/1356062328 wait complete. 2024-09-21T06:16:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.497+0000 7f3398ebf740 20 client.16020 trim_cache size 0 max 16384 2024-09-21T06:16:29.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.511+0000 7f3398ebf740 1 Processor -- start 2024-09-21T06:16:29.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.511+0000 7f3398ebf740 1 -- start start 2024-09-21T06:16:29.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.511+0000 7f3398ebf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 0x555f53a0adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.511+0000 7f3398ebf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53635390 0x555f539ff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.511+0000 7f3398ebf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f53656070 0x555f53a15070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.511+0000 7f3398ebf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1b1940 con 0x555f5362d1b0 2024-09-21T06:16:29.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.511+0000 7f3398ebf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1b3480 con 0x555f53635390 2024-09-21T06:16:29.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.511+0000 7f3398ebf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1b2010 con 0x555f53656070 2024-09-21T06:16:29.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.511+0000 7f338bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 0x555f53a0adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.511+0000 7f338bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 0x555f53a0adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35624/0 (socket says 172.21.15.3:35624) 2024-09-21T06:16:29.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.511+0000 7f338bfff640 1 -- 172.21.15.3:0/1474273985 learned_addr learned my addr 172.21.15.3:0/1474273985 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:29.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.511+0000 7f338b7fe640 1 --2- 172.21.15.3:0/1474273985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53635390 0x555f539ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.512+0000 7f3390ee8640 1 --2- 172.21.15.3:0/1474273985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f53656070 0x555f53a15070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.512+0000 7f338bfff640 1 -- 172.21.15.3:0/1474273985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f53656070 msgr2=0x555f53a15070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:29.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.512+0000 7f338bfff640 1 --2- 172.21.15.3:0/1474273985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f53656070 0x555f53a15070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.512+0000 7f338bfff640 1 -- 172.21.15.3:0/1474273985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53635390 msgr2=0x555f539ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.512+0000 7f338bfff640 1 --2- 172.21.15.3:0/1474273985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53635390 0x555f539ff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.512+0000 7f338bfff640 1 -- 172.21.15.3:0/1474273985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f53a072c0 con 0x555f5362d1b0 2024-09-21T06:16:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.512+0000 7f3390ee8640 1 --2- 172.21.15.3:0/1474273985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f53656070 0x555f53a15070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:16:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.512+0000 7f338bfff640 1 --2- 172.21.15.3:0/1474273985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 0x555f53a0adb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f337c006010 tx=0x7f337c009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195207ac4ba562f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.512+0000 7f3371ffb640 1 -- 172.21.15.3:0/1474273985 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c05f030 con 0x555f5362d1b0 2024-09-21T06:16:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.512+0000 7f3371ffb640 1 -- 172.21.15.3:0/1474273985 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f337c055020 con 0x555f5362d1b0 2024-09-21T06:16:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.512+0000 7f3371ffb640 1 -- 172.21.15.3:0/1474273985 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c04c040 con 0x555f5362d1b0 2024-09-21T06:16:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.513+0000 7f3398ebf740 1 -- 172.21.15.3:0/1474273985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 msgr2=0x555f53a0adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.513+0000 7f3398ebf740 1 --2- 172.21.15.3:0/1474273985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 0x555f53a0adb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f337c006010 tx=0x7f337c009640 comp rx=0 tx=0).stop 2024-09-21T06:16:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.513+0000 7f3398ebf740 1 -- 172.21.15.3:0/1474273985 shutdown_connections 2024-09-21T06:16:29.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.513+0000 7f3398ebf740 1 --2- 172.21.15.3:0/1474273985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f53656070 0x555f53a15070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.513+0000 7f3398ebf740 1 --2- 172.21.15.3:0/1474273985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53635390 0x555f539ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.513+0000 7f3398ebf740 1 --2- 172.21.15.3:0/1474273985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f5362d1b0 0x555f53a0adb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.513+0000 7f3398ebf740 1 -- 172.21.15.3:0/1474273985 >> 172.21.15.3:0/1474273985 conn(0x555f5364a460 msgr2=0x555f53a027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:29.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.513+0000 7f3398ebf740 1 -- 172.21.15.3:0/1474273985 shutdown_connections 2024-09-21T06:16:29.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.513+0000 7f3398ebf740 1 -- 172.21.15.3:0/1474273985 wait complete. 2024-09-21T06:16:29.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.513+0000 7f3398ebf740 1 -- start start 2024-09-21T06:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f3398ebf740 1 Processor -- start 2024-09-21T06:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f3398ebf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f5362d1b0 0x555f53a0da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f3398ebf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f53635390 0x555f53a0dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f3398ebf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53656070 0x555f53a7ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f3398ebf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1b2db0 con 0x555f53635390 2024-09-21T06:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f3398ebf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1b26e0 con 0x555f53656070 2024-09-21T06:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f3398ebf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1af920 con 0x555f5362d1b0 2024-09-21T06:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f3398ebf740 10 client.0 _ll_register_callbacks cb 0x7f3380005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f338b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f53635390 0x555f53a0dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f338b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f53635390 0x555f53a0dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35634/0 (socket says 172.21.15.3:35634) 2024-09-21T06:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f338b7fe640 1 -- 172.21.15.3:0/4045058959 learned_addr learned my addr 172.21.15.3:0/4045058959 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f3390ee8640 1 --2- 172.21.15.3:0/4045058959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53656070 0x555f53a7ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f338bfff640 1 --2- 172.21.15.3:0/4045058959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f5362d1b0 0x555f53a0da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f338b7fe640 1 -- 172.21.15.3:0/4045058959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f5362d1b0 msgr2=0x555f53a0da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f338b7fe640 1 --2- 172.21.15.3:0/4045058959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f5362d1b0 0x555f53a0da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f338b7fe640 1 -- 172.21.15.3:0/4045058959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53656070 msgr2=0x555f53a7ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:29.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f338b7fe640 1 --2- 172.21.15.3:0/4045058959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53656070 0x555f53a7ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.514+0000 7f338b7fe640 1 -- 172.21.15.3:0/4045058959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f53a7f0b0 con 0x555f53635390 2024-09-21T06:16:29.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.515+0000 7f338b7fe640 1 --2- 172.21.15.3:0/4045058959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f53635390 0x555f53a0dff0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f33800a9eb0 tx=0x7f3380004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e05a3bc90932ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:29.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.515+0000 7f336b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:29.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.515+0000 7f336b7fe640 1 -- 172.21.15.3:0/4045058959 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33800ad230 con 0x555f53635390 2024-09-21T06:16:29.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.515+0000 7f336b7fe640 1 -- 172.21.15.3:0/4045058959 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f33800acb40 con 0x555f53635390 2024-09-21T06:16:29.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.515+0000 7f3398ebf740 1 -- 172.21.15.3:0/4045058959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f337c055020 con 0x555f53635390 2024-09-21T06:16:29.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.515+0000 7f336b7fe640 1 -- 172.21.15.3:0/4045058959 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f338000ce60 con 0x555f53635390 2024-09-21T06:16:29.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.515+0000 7f3398ebf740 10 client.16026 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:29.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.515+0000 7f3398ebf740 1 -- 172.21.15.3:0/4045058959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f53a86450 con 0x555f53635390 2024-09-21T06:16:29.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.515+0000 7f336b7fe640 1 -- 172.21.15.3:0/4045058959 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f337c055020 con 0x555f53635390 2024-09-21T06:16:29.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.515+0000 7f336b7fe640 1 -- 172.21.15.3:0/4045058959 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f33800ac5b0 con 0x555f53635390 2024-09-21T06:16:29.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.516+0000 7f3398ebf740 10 client.16026 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:29.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.516+0000 7f3398ebf740 10 client.16026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:16:29.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.516+0000 7f3398ebf740 20 client.16026 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:29.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.516+0000 7f3398ebf740 1 --2- 172.21.15.3:0/4045058959 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555f535e9da0 0x555f53a88b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:29.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.516+0000 7f3398ebf740 4 client.16026 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:29.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.516+0000 7f3398ebf740 1 -- 172.21.15.3:0/4045058959 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f539718a0 con 0x555f535e9da0 2024-09-21T06:16:29.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.516+0000 7f338bfff640 1 --2- 172.21.15.3:0/4045058959 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555f535e9da0 0x555f53a88b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.516+0000 7f338bfff640 1 --2- 172.21.15.3:0/4045058959 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555f535e9da0 0x555f53a88b70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a25a49f1a1f2a808 server_cookie=11ff802cf272681f in_seq=0 out_seq=0 2024-09-21T06:16:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.516+0000 7f336b7fe640 10 client.16026 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:16:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.516+0000 7f336b7fe640 1 -- 172.21.15.3:0/4045058959 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f539718a0 con 0x555f535e9da0 2024-09-21T06:16:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.516+0000 7f336b7fe640 10 client.16026 handle_command_reply: tid=0 2024-09-21T06:16:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.517+0000 7f3398ebf740 1 client.16026 shutdown 2024-09-21T06:16:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.517+0000 7f3398ebf740 1 -- 172.21.15.3:0/4045058959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f53635390 msgr2=0x555f53a0dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.517+0000 7f3398ebf740 1 --2- 172.21.15.3:0/4045058959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f53635390 0x555f53a0dff0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f33800a9eb0 tx=0x7f3380004a20 comp rx=0 tx=0).stop 2024-09-21T06:16:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.517+0000 7f3398ebf740 1 -- 172.21.15.3:0/4045058959 shutdown_connections 2024-09-21T06:16:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.517+0000 7f3398ebf740 1 --2- 172.21.15.3:0/4045058959 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555f535e9da0 0x555f53a88b70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.517+0000 7f3398ebf740 1 --2- 172.21.15.3:0/4045058959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f53656070 0x555f53a7ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.517+0000 7f3398ebf740 1 --2- 172.21.15.3:0/4045058959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555f53635390 0x555f53a0dff0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.517+0000 7f3398ebf740 1 --2- 172.21.15.3:0/4045058959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f5362d1b0 0x555f53a0da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.517+0000 7f336b7fe640 0 client.16026 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:16:29.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.517+0000 7f3398ebf740 1 -- 172.21.15.3:0/4045058959 >> 172.21.15.3:0/4045058959 conn(0x555f5364a460 msgr2=0x555f539fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:29.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.517+0000 7f3398ebf740 1 -- 172.21.15.3:0/4045058959 shutdown_connections 2024-09-21T06:16:29.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.517+0000 7f3398ebf740 1 -- 172.21.15.3:0/4045058959 wait complete. 2024-09-21T06:16:29.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.517+0000 7f3398ebf740 20 client.16026 trim_cache size 0 max 16384 2024-09-21T06:16:29.527 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:16:29.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.518+0000 7f339294b640 1 -- 172.21.15.3:0/3043082678 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f33740b6700 msgr2=0x7f33740b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:29.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.518+0000 7f339294b640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f33740b6700 0x7f33740b8c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f337c009230 tx=0x7f337c005a20 comp rx=0 tx=0).stop 2024-09-21T06:16:29.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.518+0000 7f339294b640 1 -- 172.21.15.3:0/3043082678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f338c1052a0 msgr2=0x7f338c102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.518+0000 7f339294b640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f338c1052a0 0x7f338c102d10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f338000e9c0 tx=0x7f338000ee80 comp rx=0 tx=0).stop 2024-09-21T06:16:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.518+0000 7f339294b640 1 -- 172.21.15.3:0/3043082678 shutdown_connections 2024-09-21T06:16:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.518+0000 7f339294b640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f33740b6700 0x7f33740b8c10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.518+0000 7f339294b640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f338c10de70 0x7f338c103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.518+0000 7f339294b640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f338c1052a0 0x7f338c102d10 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.518+0000 7f339294b640 1 --2- 172.21.15.3:0/3043082678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f338c1048c0 0x7f338c1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.518+0000 7f339294b640 1 -- 172.21.15.3:0/3043082678 >> 172.21.15.3:0/3043082678 conn(0x7f338c079760 msgr2=0x7f338c100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.518+0000 7f339294b640 1 -- 172.21.15.3:0/3043082678 shutdown_connections 2024-09-21T06:16:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:29.518+0000 7f339294b640 1 -- 172.21.15.3:0/3043082678 wait complete. 2024-09-21T06:16:29.944 INFO:tasks.workunit.client.0.smithi003.stdout: 1 7260 28.95 MB/sec warmup 2 sec latency 18.284 ms 2024-09-21T06:16:30.288 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:16:30.561 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:30.944 INFO:tasks.workunit.client.0.smithi003.stdout: 1 10370 24.52 MB/sec warmup 3 sec latency 34.825 ms 2024-09-21T06:16:31.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.065+0000 7f374178c640 1 -- 172.21.15.3:0/1826209636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373c1083a0 msgr2=0x7f373c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:31.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.065+0000 7f374178c640 1 --2- 172.21.15.3:0/1826209636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373c1083a0 0x7f373c1087b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3730009900 tx=0x7f373002a410 comp rx=0 tx=0).stop 2024-09-21T06:16:31.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.065+0000 7f374178c640 1 -- 172.21.15.3:0/1826209636 shutdown_connections 2024-09-21T06:16:31.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.065+0000 7f374178c640 1 --2- 172.21.15.3:0/1826209636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c108cf0 0x7f373c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.065+0000 7f374178c640 1 --2- 172.21.15.3:0/1826209636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373c1083a0 0x7f373c1087b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.065+0000 7f374178c640 1 --2- 172.21.15.3:0/1826209636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f373c102a60 0x7f373c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.065+0000 7f374178c640 1 -- 172.21.15.3:0/1826209636 >> 172.21.15.3:0/1826209636 conn(0x7f373c079730 msgr2=0x7f373c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:31.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.065+0000 7f374178c640 1 -- 172.21.15.3:0/1826209636 shutdown_connections 2024-09-21T06:16:31.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.065+0000 7f374178c640 1 -- 172.21.15.3:0/1826209636 wait complete. 2024-09-21T06:16:31.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.065+0000 7f374178c640 1 Processor -- start 2024-09-21T06:16:31.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f374178c640 1 -- start start 2024-09-21T06:16:31.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f374178c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f373c102a60 0x7f373c199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f374178c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373c1083a0 0x7f373c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f374178c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c108cf0 0x7f373c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f374178c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f373c1129b0 con 0x7f373c1083a0 2024-09-21T06:16:31.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f374178c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f373c112830 con 0x7f373c108cf0 2024-09-21T06:16:31.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f374178c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f373c112b30 con 0x7f373c102a60 2024-09-21T06:16:31.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f373a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373c1083a0 0x7f373c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f373a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373c1083a0 0x7f373c1998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35650/0 (socket says 172.21.15.3:35650) 2024-09-21T06:16:31.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f373a7fc640 1 -- 172.21.15.3:0/1818410825 learned_addr learned my addr 172.21.15.3:0/1818410825 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f373b7fe640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c108cf0 0x7f373c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f373affd640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f373c102a60 0x7f373c199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f373a7fc640 1 -- 172.21.15.3:0/1818410825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f373c102a60 msgr2=0x7f373c199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f373a7fc640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f373c102a60 0x7f373c199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f373a7fc640 1 -- 172.21.15.3:0/1818410825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c108cf0 msgr2=0x7f373c19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f373a7fc640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c108cf0 0x7f373c19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.066+0000 7f373a7fc640 1 -- 172.21.15.3:0/1818410825 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373c19f2b0 con 0x7f373c1083a0 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.067+0000 7f373a7fc640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373c1083a0 0x7f373c1998e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3730005ab0 tx=0x7f3730003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.067+0000 7f3723fff640 1 -- 172.21.15.3:0/1818410825 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373003f070 con 0x7f373c1083a0 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.067+0000 7f374178c640 1 -- 172.21.15.3:0/1818410825 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373c19f540 con 0x7f373c1083a0 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.067+0000 7f3723fff640 1 -- 172.21.15.3:0/1818410825 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3730031b50 con 0x7f373c1083a0 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.067+0000 7f3723fff640 1 -- 172.21.15.3:0/1818410825 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730035030 con 0x7f373c1083a0 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.067+0000 7f374178c640 1 -- 172.21.15.3:0/1818410825 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f373c1a5d00 con 0x7f373c1083a0 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.070+0000 7f3747d00740 1 Processor -- start 2024-09-21T06:16:31.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.070+0000 7f3747d00740 1 -- start start 2024-09-21T06:16:31.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f3747d00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc56070 0x562a3cffd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f3747d00740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc2d1b0 0x562a3cffdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f3747d00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 0x562a3cffe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f3747d00740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562a3cf73ea0 con 0x562a3cc2d1b0 2024-09-21T06:16:31.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f3747d00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562a3cf632d0 con 0x562a3cc56070 2024-09-21T06:16:31.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f3747d00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562a3cf670e0 con 0x562a3cc35390 2024-09-21T06:16:31.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f373affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc2d1b0 0x562a3cffdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f3723fff640 1 -- 172.21.15.3:0/1818410825 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f3730042070 con 0x7f373c1083a0 2024-09-21T06:16:31.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f373b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc56070 0x562a3cffd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f373affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc2d1b0 0x562a3cffdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35660/0 (socket says 172.21.15.3:35660) 2024-09-21T06:16:31.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f373b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc56070 0x562a3cffd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:54180/0 (socket says 172.21.15.3:54180) 2024-09-21T06:16:31.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f373affd640 1 -- 172.21.15.3:0/2952306004 learned_addr learned my addr 172.21.15.3:0/2952306004 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:31.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f373b7fe640 1 -- 172.21.15.3:0/2952306004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 msgr2=0x562a3cffe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:31.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f373b7fe640 1 --2- 172.21.15.3:0/2952306004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 0x562a3cffe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f373b7fe640 1 -- 172.21.15.3:0/2952306004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc2d1b0 msgr2=0x562a3cffdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f373b7fe640 1 --2- 172.21.15.3:0/2952306004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc2d1b0 0x562a3cffdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f373b7fe640 1 -- 172.21.15.3:0/2952306004 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a3ccc1650 con 0x562a3cc56070 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f373affd640 1 --2- 172.21.15.3:0/2952306004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc2d1b0 0x562a3cffdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.071+0000 7f373b7fe640 1 --2- 172.21.15.3:0/2952306004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc56070 0x562a3cffd6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3734009f00 tx=0x7f3734011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51be53e205f23fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.072+0000 7f37217fa640 1 -- 172.21.15.3:0/2952306004 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734004d20 con 0x562a3cc56070 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.072+0000 7f37217fa640 1 -- 172.21.15.3:0/2952306004 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3734004030 con 0x562a3cc56070 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.072+0000 7f37217fa640 1 -- 172.21.15.3:0/2952306004 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734005260 con 0x562a3cc56070 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.072+0000 7f3723fff640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f37240b6680 0x7f37240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.072+0000 7f373affd640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f37240b6680 0x7f37240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.072+0000 7f3723fff640 1 -- 172.21.15.3:0/1818410825 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f3730107de0 con 0x7f373c1083a0 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.073+0000 7f3747d00740 1 -- 172.21.15.3:0/2952306004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc56070 msgr2=0x562a3cffd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.073+0000 7f373affd640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f37240b6680 0x7f37240b8b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x562a3cffc320 tx=0x7f372c00b5e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:31.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.073+0000 7f3747d00740 1 --2- 172.21.15.3:0/2952306004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc56070 0x562a3cffd6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3734009f00 tx=0x7f3734011810 comp rx=0 tx=0).stop 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.073+0000 7f3747d00740 1 -- 172.21.15.3:0/2952306004 shutdown_connections 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.073+0000 7f3747d00740 1 --2- 172.21.15.3:0/2952306004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 0x562a3cffe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.073+0000 7f3747d00740 1 --2- 172.21.15.3:0/2952306004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc2d1b0 0x562a3cffdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.073+0000 7f3747d00740 1 --2- 172.21.15.3:0/2952306004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc56070 0x562a3cffd6a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.073+0000 7f3747d00740 1 -- 172.21.15.3:0/2952306004 >> 172.21.15.3:0/2952306004 conn(0x562a3cc4a460 msgr2=0x562a3cffb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.073+0000 7f3747d00740 1 -- 172.21.15.3:0/2952306004 shutdown_connections 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.073+0000 7f3747d00740 1 -- 172.21.15.3:0/2952306004 wait complete. 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.073+0000 7f3747d00740 1 -- start start 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.073+0000 7f3747d00740 1 Processor -- start 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f3747d00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d00afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f3747d00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 0x562a3d00b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f3747d00740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc56070 0x562a3d0172e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f3747d00740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562a3cdc4700 con 0x562a3cc56070 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f3747d00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562a3cefe160 con 0x562a3cc2d1b0 2024-09-21T06:16:31.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f3747d00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562a3d00dfb0 con 0x562a3cc35390 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f3747d00740 10 client.0 _ll_register_callbacks cb 0x562a3cf179e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f373b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc56070 0x562a3d0172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f373affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d00afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f373b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc56070 0x562a3d0172e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35672/0 (socket says 172.21.15.3:35672) 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f373b7fe640 1 -- 172.21.15.3:0/126900216 learned_addr learned my addr 172.21.15.3:0/126900216 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f373a7fc640 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 0x562a3d00b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f373affd640 1 -- 172.21.15.3:0/126900216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 msgr2=0x562a3d00b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f373affd640 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 0x562a3d00b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f373affd640 1 -- 172.21.15.3:0/126900216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc56070 msgr2=0x562a3d0172e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f373affd640 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc56070 0x562a3d0172e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f373affd640 1 -- 172.21.15.3:0/126900216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a3cf2bc90 con 0x562a3cc2d1b0 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f373a7fc640 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 0x562a3d00b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f373b7fe640 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc56070 0x562a3d0172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f373affd640 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d00afe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f372c010710 tx=0x7f372c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41369b5ecb4b8e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:31.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.074+0000 7f371eff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:16:31.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f371eff5640 1 -- 172.21.15.3:0/126900216 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c027430 con 0x562a3cc2d1b0 2024-09-21T06:16:31.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f3747d00740 1 -- 172.21.15.3:0/126900216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3734004030 con 0x562a3cc2d1b0 2024-09-21T06:16:31.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f371eff5640 1 -- 172.21.15.3:0/126900216 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f372c01d410 con 0x562a3cc2d1b0 2024-09-21T06:16:31.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f371eff5640 1 -- 172.21.15.3:0/126900216 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f372c01d610 con 0x562a3cc2d1b0 2024-09-21T06:16:31.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f3747d00740 10 client.25622 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:31.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f3747d00740 1 -- 172.21.15.3:0/126900216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562a3d019e30 con 0x562a3cc2d1b0 2024-09-21T06:16:31.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f371eff5640 1 -- 172.21.15.3:0/126900216 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3734004030 con 0x562a3cc2d1b0 2024-09-21T06:16:31.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f371eff5640 1 -- 172.21.15.3:0/126900216 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f372c027a10 con 0x562a3cc2d1b0 2024-09-21T06:16:31.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f3747d00740 10 client.25622 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:31.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f3747d00740 10 client.25622 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:16:31.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f3747d00740 20 client.25622 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:31.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f3747d00740 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562a3cc149f0 0x562a3d01d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f3747d00740 4 client.25622 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.075+0000 7f3747d00740 1 -- 172.21.15.3:0/126900216 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a3cf75df0 con 0x562a3cc149f0 2024-09-21T06:16:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.076+0000 7f373a7fc640 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562a3cc149f0 0x562a3d01d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.076+0000 7f371eff5640 10 client.25622 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:16:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.076+0000 7f373a7fc640 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562a3cc149f0 0x562a3d01d300 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d55947165b3af1c server_cookie=224dd5283e93bc0c in_seq=0 out_seq=0 2024-09-21T06:16:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f371eff5640 1 -- 172.21.15.3:0/126900216 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a3cf75df0 con 0x562a3cc149f0 2024-09-21T06:16:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f371eff5640 10 client.25622 handle_command_reply: tid=0 2024-09-21T06:16:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f3747d00740 1 client.25622 shutdown 2024-09-21T06:16:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f3747d00740 1 -- 172.21.15.3:0/126900216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 msgr2=0x562a3d00afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f3747d00740 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d00afe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f372c010710 tx=0x7f372c0046c0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f3747d00740 1 -- 172.21.15.3:0/126900216 shutdown_connections 2024-09-21T06:16:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f3747d00740 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562a3cc149f0 0x562a3d01d300 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f3747d00740 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 0x562a3d00b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f3747d00740 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc56070 0x562a3d0172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f371eff5640 0 client.25622 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:16:31.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f3747d00740 1 --2- 172.21.15.3:0/126900216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d00afe0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f3747d00740 1 -- 172.21.15.3:0/126900216 >> 172.21.15.3:0/126900216 conn(0x562a3cc4a460 msgr2=0x562a3cff9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:31.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f3747d00740 1 -- 172.21.15.3:0/126900216 shutdown_connections 2024-09-21T06:16:31.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f3747d00740 1 -- 172.21.15.3:0/126900216 wait complete. 2024-09-21T06:16:31.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.077+0000 7f3747d00740 20 client.25622 trim_cache size 0 max 16384 2024-09-21T06:16:31.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.091+0000 7f3747d00740 1 Processor -- start 2024-09-21T06:16:31.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f3747d00740 1 -- start start 2024-09-21T06:16:31.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f3747d00740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc149f0 0x562a3d003230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f3747d00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d00cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f3747d00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 0x562a3cffede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f3747d00740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f373c1ad440 con 0x562a3cc149f0 2024-09-21T06:16:31.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f3747d00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f373c1aef80 con 0x562a3cc2d1b0 2024-09-21T06:16:31.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f3747d00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f373c1adb10 con 0x562a3cc35390 2024-09-21T06:16:31.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f373affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc149f0 0x562a3d003230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f373affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc149f0 0x562a3d003230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35676/0 (socket says 172.21.15.3:35676) 2024-09-21T06:16:31.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f373affd640 1 -- 172.21.15.3:0/40425956 learned_addr learned my addr 172.21.15.3:0/40425956 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:31.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f373a7fc640 1 --2- 172.21.15.3:0/40425956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d00cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f373b7fe640 1 --2- 172.21.15.3:0/40425956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 0x562a3cffede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f373affd640 1 -- 172.21.15.3:0/40425956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 msgr2=0x562a3cffede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:31.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f373affd640 1 --2- 172.21.15.3:0/40425956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 0x562a3cffede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f373affd640 1 -- 172.21.15.3:0/40425956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 msgr2=0x562a3d00cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:31.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f373affd640 1 --2- 172.21.15.3:0/40425956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d00cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.092+0000 7f373affd640 1 -- 172.21.15.3:0/40425956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a3d0094e0 con 0x562a3cc149f0 2024-09-21T06:16:31.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.093+0000 7f373affd640 1 --2- 172.21.15.3:0/40425956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc149f0 0x562a3d003230 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f372c017040 tx=0x7f372c00b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43c6f68067d3221 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:31.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.093+0000 7f3720ff9640 1 -- 172.21.15.3:0/40425956 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c034030 con 0x562a3cc149f0 2024-09-21T06:16:31.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.093+0000 7f3720ff9640 1 -- 172.21.15.3:0/40425956 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372c036070 con 0x562a3cc149f0 2024-09-21T06:16:31.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.093+0000 7f3720ff9640 1 -- 172.21.15.3:0/40425956 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c01dbd0 con 0x562a3cc149f0 2024-09-21T06:16:31.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.094+0000 7f3747d00740 1 -- 172.21.15.3:0/40425956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc149f0 msgr2=0x562a3d003230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:31.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.094+0000 7f3747d00740 1 --2- 172.21.15.3:0/40425956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc149f0 0x562a3d003230 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f372c017040 tx=0x7f372c00b980 comp rx=0 tx=0).stop 2024-09-21T06:16:31.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.094+0000 7f3747d00740 1 -- 172.21.15.3:0/40425956 shutdown_connections 2024-09-21T06:16:31.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.094+0000 7f3747d00740 1 --2- 172.21.15.3:0/40425956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc35390 0x562a3cffede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.094+0000 7f3747d00740 1 --2- 172.21.15.3:0/40425956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d00cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.094+0000 7f3747d00740 1 --2- 172.21.15.3:0/40425956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc149f0 0x562a3d003230 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.094+0000 7f3747d00740 1 -- 172.21.15.3:0/40425956 >> 172.21.15.3:0/40425956 conn(0x562a3cc4a460 msgr2=0x562a3cffe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:31.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.094+0000 7f3747d00740 1 -- 172.21.15.3:0/40425956 shutdown_connections 2024-09-21T06:16:31.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.094+0000 7f3747d00740 1 -- 172.21.15.3:0/40425956 wait complete. 2024-09-21T06:16:31.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.094+0000 7f3747d00740 1 -- start start 2024-09-21T06:16:31.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.094+0000 7f3747d00740 1 Processor -- start 2024-09-21T06:16:31.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.094+0000 7f3747d00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc149f0 0x562a3d013c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f3747d00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d011100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f3747d00740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc35390 0x562a3d011810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f3747d00740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f373c1ae8b0 con 0x562a3cc35390 2024-09-21T06:16:31.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f3747d00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f373c1ae1e0 con 0x562a3cc2d1b0 2024-09-21T06:16:31.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f3747d00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f373c1ab420 con 0x562a3cc149f0 2024-09-21T06:16:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f3747d00740 10 client.0 _ll_register_callbacks cb 0x7f372c00b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f373b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc35390 0x562a3d011810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f373a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d011100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f373b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc35390 0x562a3d011810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35688/0 (socket says 172.21.15.3:35688) 2024-09-21T06:16:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f373a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d011100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:54214/0 (socket says 172.21.15.3:54214) 2024-09-21T06:16:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f373b7fe640 1 -- 172.21.15.3:0/4240829898 learned_addr learned my addr 172.21.15.3:0/4240829898 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f373affd640 1 --2- 172.21.15.3:0/4240829898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc149f0 0x562a3d013c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f373b7fe640 1 -- 172.21.15.3:0/4240829898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc149f0 msgr2=0x562a3d013c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f373b7fe640 1 --2- 172.21.15.3:0/4240829898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc149f0 0x562a3d013c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f373b7fe640 1 -- 172.21.15.3:0/4240829898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 msgr2=0x562a3d011100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f373b7fe640 1 --2- 172.21.15.3:0/4240829898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d011100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f373b7fe640 1 -- 172.21.15.3:0/4240829898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a3d086cd0 con 0x562a3cc35390 2024-09-21T06:16:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f373a7fc640 1 --2- 172.21.15.3:0/4240829898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d011100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f373b7fe640 1 --2- 172.21.15.3:0/4240829898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc35390 0x562a3d011810 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3734000f80 tx=0x7f3734004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c279a49a77a327 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f371eff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.095+0000 7f371eff5640 1 -- 172.21.15.3:0/4240829898 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373401e040 con 0x562a3cc35390 2024-09-21T06:16:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.096+0000 7f371eff5640 1 -- 172.21.15.3:0/4240829898 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f3734003a30 con 0x562a3cc35390 2024-09-21T06:16:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.096+0000 7f3747d00740 1 -- 172.21.15.3:0/4240829898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f372c036070 con 0x562a3cc35390 2024-09-21T06:16:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.096+0000 7f371eff5640 1 -- 172.21.15.3:0/4240829898 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3734003c30 con 0x562a3cc35390 2024-09-21T06:16:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.096+0000 7f3747d00740 10 client.16044 fetch_fsmap learned FSMap version 28 2024-09-21T06:16:31.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.096+0000 7f3747d00740 1 -- 172.21.15.3:0/4240829898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562a3d0039d0 con 0x562a3cc35390 2024-09-21T06:16:31.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.096+0000 7f371eff5640 1 -- 172.21.15.3:0/4240829898 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372c036070 con 0x562a3cc35390 2024-09-21T06:16:31.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.096+0000 7f371eff5640 1 -- 172.21.15.3:0/4240829898 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3734018070 con 0x562a3cc35390 2024-09-21T06:16:31.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.096+0000 7f3747d00740 10 client.16044 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:16:31.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.096+0000 7f3747d00740 10 client.16044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:16:31.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.096+0000 7f3747d00740 20 client.16044 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:31.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.096+0000 7f3747d00740 1 --2- 172.21.15.3:0/4240829898 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562a3cc56070 0x562a3d0040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:31.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.096+0000 7f3747d00740 4 client.16044 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:31.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.096+0000 7f3747d00740 1 -- 172.21.15.3:0/4240829898 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a3cf718a0 con 0x562a3cc56070 2024-09-21T06:16:31.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.097+0000 7f373affd640 1 --2- 172.21.15.3:0/4240829898 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562a3cc56070 0x562a3d0040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:31.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.097+0000 7f373affd640 1 --2- 172.21.15.3:0/4240829898 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562a3cc56070 0x562a3d0040c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a17e3a0c4659925 server_cookie=f96bd531c748086f in_seq=0 out_seq=0 2024-09-21T06:16:31.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.097+0000 7f371eff5640 10 client.16044 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:16:31.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.097+0000 7f371eff5640 1 -- 172.21.15.3:0/4240829898 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a3cf718a0 con 0x562a3cc56070 2024-09-21T06:16:31.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.097+0000 7f371eff5640 10 client.16044 handle_command_reply: tid=0 2024-09-21T06:16:31.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.097+0000 7f3747d00740 1 client.16044 shutdown 2024-09-21T06:16:31.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.097+0000 7f3747d00740 1 -- 172.21.15.3:0/4240829898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc35390 msgr2=0x562a3d011810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:31.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.097+0000 7f3747d00740 1 --2- 172.21.15.3:0/4240829898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc35390 0x562a3d011810 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3734000f80 tx=0x7f3734004ee0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.097+0000 7f3747d00740 1 -- 172.21.15.3:0/4240829898 shutdown_connections 2024-09-21T06:16:31.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.097+0000 7f3747d00740 1 --2- 172.21.15.3:0/4240829898 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562a3cc56070 0x562a3d0040c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f3747d00740 1 --2- 172.21.15.3:0/4240829898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a3cc35390 0x562a3d011810 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f3747d00740 1 --2- 172.21.15.3:0/4240829898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a3cc2d1b0 0x562a3d011100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f3747d00740 1 --2- 172.21.15.3:0/4240829898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3cc149f0 0x562a3d013c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f3747d00740 1 -- 172.21.15.3:0/4240829898 >> 172.21.15.3:0/4240829898 conn(0x562a3cc4a460 msgr2=0x562a3cffee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:31.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f371eff5640 0 client.16044 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:16:31.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f3747d00740 1 -- 172.21.15.3:0/4240829898 shutdown_connections 2024-09-21T06:16:31.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f3747d00740 1 -- 172.21.15.3:0/4240829898 wait complete. 2024-09-21T06:16:31.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f3747d00740 20 client.16044 trim_cache size 0 max 16384 2024-09-21T06:16:31.106 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:16:31.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f374178c640 1 -- 172.21.15.3:0/1818410825 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f37240b6680 msgr2=0x7f37240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:31.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f374178c640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f37240b6680 0x7f37240b8b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x562a3cffc320 tx=0x7f372c00b5e0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f374178c640 1 -- 172.21.15.3:0/1818410825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373c1083a0 msgr2=0x7f373c1998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:31.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f374178c640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373c1083a0 0x7f373c1998e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3730005ab0 tx=0x7f3730003b90 comp rx=0 tx=0).stop 2024-09-21T06:16:31.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f374178c640 1 -- 172.21.15.3:0/1818410825 shutdown_connections 2024-09-21T06:16:31.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f374178c640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f37240b6680 0x7f37240b8b90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f374178c640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c108cf0 0x7f373c19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f374178c640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373c1083a0 0x7f373c1998e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f374178c640 1 --2- 172.21.15.3:0/1818410825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f373c102a60 0x7f373c199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:31.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.098+0000 7f374178c640 1 -- 172.21.15.3:0/1818410825 >> 172.21.15.3:0/1818410825 conn(0x7f373c079730 msgr2=0x7f373c1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:31.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.099+0000 7f374178c640 1 -- 172.21.15.3:0/1818410825 shutdown_connections 2024-09-21T06:16:31.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:31.099+0000 7f374178c640 1 -- 172.21.15.3:0/1818410825 wait complete. 2024-09-21T06:16:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:31 smithi003 ceph-mon[31227]: pgmap v370: 97 pgs: 97 active+clean; 2.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 64 KiB/s wr, 14 op/s 2024-09-21T06:16:31.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:31 smithi137 ceph-mon[34464]: pgmap v370: 97 pgs: 97 active+clean; 2.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 64 KiB/s wr, 14 op/s 2024-09-21T06:16:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:31 smithi204 ceph-mon[34659]: pgmap v370: 97 pgs: 97 active+clean; 2.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 64 KiB/s wr, 14 op/s 2024-09-21T06:16:31.788 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:16:31.944 INFO:tasks.workunit.client.0.smithi003.stdout: 1 13131 21.52 MB/sec warmup 4 sec latency 13.735 ms 2024-09-21T06:16:32.054 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:32.408 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:32 smithi003 ceph-mon[31227]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:16:32.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:32 smithi137 ceph-mon[34464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:16:32.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.645+0000 7f7daf65a640 1 -- 172.21.15.3:0/738199269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da8108cf0 msgr2=0x7f7da810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:32.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.645+0000 7f7daf65a640 1 --2- 172.21.15.3:0/738199269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da8108cf0 0x7f7da810f520 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009ca0 tx=0x7f7da002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.645+0000 7f7daf65a640 1 -- 172.21.15.3:0/738199269 shutdown_connections 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.645+0000 7f7daf65a640 1 --2- 172.21.15.3:0/738199269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da8108cf0 0x7f7da810f520 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.645+0000 7f7daf65a640 1 --2- 172.21.15.3:0/738199269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7da81083a0 0x7f7da81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.645+0000 7f7daf65a640 1 --2- 172.21.15.3:0/738199269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7da8102a60 0x7f7da8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.645+0000 7f7daf65a640 1 -- 172.21.15.3:0/738199269 >> 172.21.15.3:0/738199269 conn(0x7f7da8079730 msgr2=0x7f7da8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.645+0000 7f7daf65a640 1 -- 172.21.15.3:0/738199269 shutdown_connections 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.645+0000 7f7daf65a640 1 -- 172.21.15.3:0/738199269 wait complete. 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.646+0000 7f7daf65a640 1 Processor -- start 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.646+0000 7f7daf65a640 1 -- start start 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.646+0000 7f7daf65a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7da8102a60 0x7f7da8199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.646+0000 7f7daf65a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7da81083a0 0x7f7da81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.646+0000 7f7daf65a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da8108cf0 0x7f7da8199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.646+0000 7f7daf65a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7da8112960 con 0x7f7da8108cf0 2024-09-21T06:16:32.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.646+0000 7f7daf65a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81127e0 con 0x7f7da81083a0 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.646+0000 7f7daf65a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7da8112ae0 con 0x7f7da8102a60 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.646+0000 7f7dadbf7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da8108cf0 0x7f7da8199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.646+0000 7f7dacbf5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7da81083a0 0x7f7da81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.646+0000 7f7dadbf7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da8108cf0 0x7f7da8199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35708/0 (socket says 172.21.15.3:35708) 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.646+0000 7f7dadbf7640 1 -- 172.21.15.3:0/2116715276 learned_addr learned my addr 172.21.15.3:0/2116715276 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.646+0000 7f7dad3f6640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7da8102a60 0x7f7da8199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.647+0000 7f7dacbf5640 1 -- 172.21.15.3:0/2116715276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7da8102a60 msgr2=0x7f7da8199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.647+0000 7f7dacbf5640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7da8102a60 0x7f7da8199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.647+0000 7f7dacbf5640 1 -- 172.21.15.3:0/2116715276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da8108cf0 msgr2=0x7f7da8199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.647+0000 7f7dacbf5640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da8108cf0 0x7f7da8199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.647+0000 7f7dacbf5640 1 -- 172.21.15.3:0/2116715276 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da81a8730 con 0x7f7da81083a0 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.647+0000 7f7dadbf7640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da8108cf0 0x7f7da8199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.647+0000 7f7dacbf5640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7da81083a0 0x7f7da81995c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00e9c0 tx=0x7f7d9c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:32.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.647+0000 7f7d9a7fc640 1 -- 172.21.15.3:0/2116715276 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00ccc0 con 0x7f7da81083a0 2024-09-21T06:16:32.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.648+0000 7f7d9a7fc640 1 -- 172.21.15.3:0/2116715276 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d9c0108e0 con 0x7f7da81083a0 2024-09-21T06:16:32.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.648+0000 7f7d9a7fc640 1 -- 172.21.15.3:0/2116715276 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0187c0 con 0x7f7da81083a0 2024-09-21T06:16:32.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.648+0000 7f7daf65a640 1 -- 172.21.15.3:0/2116715276 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da81a8a20 con 0x7f7da81083a0 2024-09-21T06:16:32.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.648+0000 7f7daf65a640 1 -- 172.21.15.3:0/2116715276 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da81a8f10 con 0x7f7da81083a0 2024-09-21T06:16:32.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7db5bce740 1 Processor -- start 2024-09-21T06:16:32.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7db5bce740 1 -- start start 2024-09-21T06:16:32.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7d9a7fc640 1 -- 172.21.15.3:0/2116715276 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f7d9c010430 con 0x7f7da81083a0 2024-09-21T06:16:32.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7db5bce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d7a4070 0x557a1db4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7db5bce740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db4b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7db5bce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db4bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7db5bce740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557a1dac1ea0 con 0x557a1d77b1b0 2024-09-21T06:16:32.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7db5bce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557a1dab12d0 con 0x557a1d7a4070 2024-09-21T06:16:32.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7db5bce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557a1dab50e0 con 0x557a1d783390 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7dad3f6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db4b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7dadbf7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d7a4070 0x557a1db4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7dad3f6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db4b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35724/0 (socket says 172.21.15.3:35724) 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7dacbf5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db4bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.651+0000 7f7dad3f6640 1 -- 172.21.15.3:0/471859 learned_addr learned my addr 172.21.15.3:0/471859 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.652+0000 7f7dad3f6640 1 -- 172.21.15.3:0/471859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 msgr2=0x557a1db4bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.652+0000 7f7dad3f6640 1 --2- 172.21.15.3:0/471859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db4bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.652+0000 7f7dad3f6640 1 -- 172.21.15.3:0/471859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d7a4070 msgr2=0x557a1db4b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.652+0000 7f7dad3f6640 1 --2- 172.21.15.3:0/471859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d7a4070 0x557a1db4b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.652+0000 7f7dad3f6640 1 -- 172.21.15.3:0/471859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1d80f650 con 0x557a1d77b1b0 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.652+0000 7f7dacbf5640 1 --2- 172.21.15.3:0/471859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db4bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.652+0000 7f7d9a7fc640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7d940b6680 0x7f7d940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.652+0000 7f7dad3f6640 1 --2- 172.21.15.3:0/471859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db4b800 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009ca0 tx=0x7f7da400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db98f2d3be7c1f06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.652+0000 7f7d937fe640 1 -- 172.21.15.3:0/471859 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4009e50 con 0x557a1d77b1b0 2024-09-21T06:16:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.652+0000 7f7d937fe640 1 -- 172.21.15.3:0/471859 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da40054d0 con 0x557a1d77b1b0 2024-09-21T06:16:32.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.652+0000 7f7d937fe640 1 -- 172.21.15.3:0/471859 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4021670 con 0x557a1d77b1b0 2024-09-21T06:16:32.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.652+0000 7f7d9a7fc640 1 -- 172.21.15.3:0/2116715276 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f7d9c0e7980 con 0x7f7da81083a0 2024-09-21T06:16:32.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.652+0000 7f7dadbf7640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7d940b6680 0x7f7d940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:32 smithi204 ceph-mon[34659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:16:32.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.653+0000 7f7db5bce740 1 -- 172.21.15.3:0/471859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 msgr2=0x557a1db4b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:32.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.653+0000 7f7dadbf7640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7d940b6680 0x7f7d940b8b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7da0005e30 tx=0x7f7da0005c50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:32.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.653+0000 7f7db5bce740 1 --2- 172.21.15.3:0/471859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db4b800 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009ca0 tx=0x7f7da400f790 comp rx=0 tx=0).stop 2024-09-21T06:16:32.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.653+0000 7f7db5bce740 1 -- 172.21.15.3:0/471859 shutdown_connections 2024-09-21T06:16:32.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.653+0000 7f7db5bce740 1 --2- 172.21.15.3:0/471859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db4bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.653+0000 7f7db5bce740 1 --2- 172.21.15.3:0/471859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db4b800 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.653+0000 7f7db5bce740 1 --2- 172.21.15.3:0/471859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d7a4070 0x557a1db4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.653+0000 7f7db5bce740 1 -- 172.21.15.3:0/471859 >> 172.21.15.3:0/471859 conn(0x557a1d798460 msgr2=0x557a1db491b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.653+0000 7f7db5bce740 1 -- 172.21.15.3:0/471859 shutdown_connections 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.654+0000 7f7db5bce740 1 -- 172.21.15.3:0/471859 wait complete. 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.654+0000 7f7db5bce740 1 -- start start 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.654+0000 7f7db5bce740 1 Processor -- start 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7db5bce740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db56a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7db5bce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db56f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7db5bce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d7a4070 0x557a1db62d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7db5bce740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557a1d912700 con 0x557a1d77b1b0 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7db5bce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557a1da4c160 con 0x557a1d7a4070 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7db5bce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557a1db59aa0 con 0x557a1d783390 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7db5bce740 10 client.0 _ll_register_callbacks cb 0x557a1da659e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7dad3f6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db56a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7dacbf5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db56f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7dad3f6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db56a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35734/0 (socket says 172.21.15.3:35734) 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7dad3f6640 1 -- 172.21.15.3:0/1759378904 learned_addr learned my addr 172.21.15.3:0/1759378904 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7dadbf7640 1 --2- 172.21.15.3:0/1759378904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d7a4070 0x557a1db62d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7dacbf5640 1 -- 172.21.15.3:0/1759378904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d7a4070 msgr2=0x557a1db62d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7dacbf5640 1 --2- 172.21.15.3:0/1759378904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d7a4070 0x557a1db62d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7dacbf5640 1 -- 172.21.15.3:0/1759378904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 msgr2=0x557a1db56a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7dacbf5640 1 --2- 172.21.15.3:0/1759378904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db56a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7dacbf5640 1 -- 172.21.15.3:0/1759378904 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1da79c90 con 0x557a1d783390 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7dacbf5640 1 --2- 172.21.15.3:0/1759378904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db56f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0e5110 tx=0x7f7d9c0ad690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93b5644584c0efa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.655+0000 7f7d90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7d90ff9640 1 -- 172.21.15.3:0/1759378904 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0ad030 con 0x557a1d783390 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7d90ff9640 1 -- 172.21.15.3:0/1759378904 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f7d9c0aeb40 con 0x557a1d783390 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7db5bce740 1 -- 172.21.15.3:0/1759378904 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da40054d0 con 0x557a1d783390 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7d90ff9640 1 -- 172.21.15.3:0/1759378904 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d9c0e55a0 con 0x557a1d783390 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7db5bce740 10 client.25612 fetch_fsmap learned FSMap version 29 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7db5bce740 1 -- 172.21.15.3:0/1759378904 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a1db65810 con 0x557a1d783390 2024-09-21T06:16:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7d90ff9640 1 -- 172.21.15.3:0/1759378904 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da40054d0 con 0x557a1d783390 2024-09-21T06:16:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7d90ff9640 1 -- 172.21.15.3:0/1759378904 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7d9c0ac6b0 con 0x557a1d783390 2024-09-21T06:16:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7db5bce740 10 client.25612 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:16:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7db5bce740 10 client.25612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7db5bce740 20 client.25612 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7db5bce740 1 --2- 172.21.15.3:0/1759378904 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557a1d7629f0 0x557a1db68d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7db5bce740 4 client.25612 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.656+0000 7f7db5bce740 1 -- 172.21.15.3:0/1759378904 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a1dac3df0 con 0x557a1d7629f0 2024-09-21T06:16:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.657+0000 7f7dad3f6640 1 --2- 172.21.15.3:0/1759378904 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557a1d7629f0 0x557a1db68d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.657+0000 7f7d90ff9640 10 client.25612 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.657+0000 7f7dad3f6640 1 --2- 172.21.15.3:0/1759378904 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557a1d7629f0 0x557a1db68d50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb75bb23cecf066b server_cookie=ba4130ca32d28647 in_seq=0 out_seq=0 2024-09-21T06:16:32.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7d90ff9640 1 -- 172.21.15.3:0/1759378904 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a1dac3df0 con 0x557a1d7629f0 2024-09-21T06:16:32.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7d90ff9640 10 client.25612 handle_command_reply: tid=0 2024-09-21T06:16:32.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7db5bce740 1 client.25612 shutdown 2024-09-21T06:16:32.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7db5bce740 1 -- 172.21.15.3:0/1759378904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 msgr2=0x557a1db56f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:32.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7db5bce740 1 --2- 172.21.15.3:0/1759378904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db56f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0e5110 tx=0x7f7d9c0ad690 comp rx=0 tx=0).stop 2024-09-21T06:16:32.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7db5bce740 1 -- 172.21.15.3:0/1759378904 shutdown_connections 2024-09-21T06:16:32.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7db5bce740 1 --2- 172.21.15.3:0/1759378904 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557a1d7629f0 0x557a1db68d50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7db5bce740 1 --2- 172.21.15.3:0/1759378904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db56f60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7db5bce740 1 --2- 172.21.15.3:0/1759378904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d7a4070 0x557a1db62d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7db5bce740 1 --2- 172.21.15.3:0/1759378904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db56a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7d90ff9640 0 client.25612 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:32.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7db5bce740 1 -- 172.21.15.3:0/1759378904 >> 172.21.15.3:0/1759378904 conn(0x557a1d798460 msgr2=0x557a1db4e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:32.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7db5bce740 1 -- 172.21.15.3:0/1759378904 shutdown_connections 2024-09-21T06:16:32.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7db5bce740 1 -- 172.21.15.3:0/1759378904 wait complete. 2024-09-21T06:16:32.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.658+0000 7f7db5bce740 20 client.25612 trim_cache size 0 max 16384 2024-09-21T06:16:32.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.672+0000 7f7db5bce740 1 Processor -- start 2024-09-21T06:16:32.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.672+0000 7f7db5bce740 1 -- start start 2024-09-21T06:16:32.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.672+0000 7f7db5bce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d77b1b0 0x557a1db4ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.672+0000 7f7db5bce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db62c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.672+0000 7f7db5bce740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d7a4070 0x557a1db631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.672+0000 7f7db5bce740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81ad3f0 con 0x557a1d7a4070 2024-09-21T06:16:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.672+0000 7f7db5bce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81aef90 con 0x557a1d77b1b0 2024-09-21T06:16:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.672+0000 7f7db5bce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81adac0 con 0x557a1d783390 2024-09-21T06:16:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.672+0000 7f7dadbf7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d7a4070 0x557a1db631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.672+0000 7f7dad3f6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d77b1b0 0x557a1db4ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.672+0000 7f7dadbf7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d7a4070 0x557a1db631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35744/0 (socket says 172.21.15.3:35744) 2024-09-21T06:16:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.672+0000 7f7dacbf5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db62c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.672+0000 7f7dadbf7640 1 -- 172.21.15.3:0/758595308 learned_addr learned my addr 172.21.15.3:0/758595308 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:32.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.673+0000 7f7dacbf5640 1 -- 172.21.15.3:0/758595308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d77b1b0 msgr2=0x557a1db4ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:32.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.673+0000 7f7dacbf5640 1 --2- 172.21.15.3:0/758595308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d77b1b0 0x557a1db4ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.673+0000 7f7dacbf5640 1 -- 172.21.15.3:0/758595308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d7a4070 msgr2=0x557a1db631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.673+0000 7f7dacbf5640 1 --2- 172.21.15.3:0/758595308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d7a4070 0x557a1db631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.673+0000 7f7dacbf5640 1 -- 172.21.15.3:0/758595308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1db57ce0 con 0x557a1d783390 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.673+0000 7f7dad3f6640 1 --2- 172.21.15.3:0/758595308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d77b1b0 0x557a1db4ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.673+0000 7f7dadbf7640 1 --2- 172.21.15.3:0/758595308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d7a4070 0x557a1db631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.673+0000 7f7dacbf5640 1 --2- 172.21.15.3:0/758595308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db62c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0ad850 tx=0x7f7d9c0ad7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=532a19ee1571a4b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.673+0000 7f7d92ffd640 1 -- 172.21.15.3:0/758595308 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0a9f10 con 0x557a1d783390 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.673+0000 7f7d92ffd640 1 -- 172.21.15.3:0/758595308 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d9c010720 con 0x557a1d783390 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.673+0000 7f7d92ffd640 1 -- 172.21.15.3:0/758595308 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0ac590 con 0x557a1d783390 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.674+0000 7f7db5bce740 1 -- 172.21.15.3:0/758595308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 msgr2=0x557a1db62c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.674+0000 7f7db5bce740 1 --2- 172.21.15.3:0/758595308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db62c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0ad850 tx=0x7f7d9c0ad7c0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.674+0000 7f7db5bce740 1 -- 172.21.15.3:0/758595308 shutdown_connections 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.674+0000 7f7db5bce740 1 --2- 172.21.15.3:0/758595308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d783390 0x557a1db62c90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.674+0000 7f7db5bce740 1 --2- 172.21.15.3:0/758595308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d7a4070 0x557a1db631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.674+0000 7f7db5bce740 1 --2- 172.21.15.3:0/758595308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d77b1b0 0x557a1db4ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.674+0000 7f7db5bce740 1 -- 172.21.15.3:0/758595308 >> 172.21.15.3:0/758595308 conn(0x557a1d798460 msgr2=0x557a1db4e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:32.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.674+0000 7f7db5bce740 1 -- 172.21.15.3:0/758595308 shutdown_connections 2024-09-21T06:16:32.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.674+0000 7f7db5bce740 1 -- 172.21.15.3:0/758595308 wait complete. 2024-09-21T06:16:32.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.675+0000 7f7db5bce740 1 -- start start 2024-09-21T06:16:32.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.675+0000 7f7db5bce740 1 Processor -- start 2024-09-21T06:16:32.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.675+0000 7f7db5bce740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db5f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7db5bce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d783390 0x557a1db5aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7db5bce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d7a4070 0x557a1db5b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7db5bce740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81ae860 con 0x557a1d77b1b0 2024-09-21T06:16:32.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7db5bce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81ae190 con 0x557a1d783390 2024-09-21T06:16:32.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7db5bce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81ab3d0 con 0x557a1d7a4070 2024-09-21T06:16:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7db5bce740 10 client.0 _ll_register_callbacks cb 0x7f7d9c0056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7dad3f6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db5f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7dacbf5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d783390 0x557a1db5aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7dad3f6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db5f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35754/0 (socket says 172.21.15.3:35754) 2024-09-21T06:16:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7dad3f6640 1 -- 172.21.15.3:0/3448364439 learned_addr learned my addr 172.21.15.3:0/3448364439 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7dacbf5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d783390 0x557a1db5aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:54252/0 (socket says 172.21.15.3:54252) 2024-09-21T06:16:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7dadbf7640 1 --2- 172.21.15.3:0/3448364439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d7a4070 0x557a1db5b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7dacbf5640 1 -- 172.21.15.3:0/3448364439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d7a4070 msgr2=0x557a1db5b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7dacbf5640 1 --2- 172.21.15.3:0/3448364439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d7a4070 0x557a1db5b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7dacbf5640 1 -- 172.21.15.3:0/3448364439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 msgr2=0x557a1db5f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7dacbf5640 1 --2- 172.21.15.3:0/3448364439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db5f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7dacbf5640 1 -- 172.21.15.3:0/3448364439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1db5b830 con 0x557a1d783390 2024-09-21T06:16:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7dad3f6640 1 --2- 172.21.15.3:0/3448364439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db5f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:32.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7dacbf5640 1 --2- 172.21.15.3:0/3448364439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d783390 0x557a1db5aa70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0aab30 tx=0x7f7d9c00ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad44557c0db3255 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:32.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.676+0000 7f7d90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:16:32.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7d90ff9640 1 -- 172.21.15.3:0/3448364439 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0a9f10 con 0x557a1d783390 2024-09-21T06:16:32.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7d90ff9640 1 -- 172.21.15.3:0/3448364439 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f7d9c0a9420 con 0x557a1d783390 2024-09-21T06:16:32.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7db5bce740 1 -- 172.21.15.3:0/3448364439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557a1dabf8a0 con 0x557a1d783390 2024-09-21T06:16:32.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7d90ff9640 1 -- 172.21.15.3:0/3448364439 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d9c0c8e60 con 0x557a1d783390 2024-09-21T06:16:32.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7db5bce740 10 client.25646 fetch_fsmap learned FSMap version 29 2024-09-21T06:16:32.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7db5bce740 1 -- 172.21.15.3:0/3448364439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a1dbd3750 con 0x557a1d783390 2024-09-21T06:16:32.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7d90ff9640 1 -- 172.21.15.3:0/3448364439 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557a1dabf8a0 con 0x557a1d783390 2024-09-21T06:16:32.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7d90ff9640 1 -- 172.21.15.3:0/3448364439 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7d9c0cf310 con 0x557a1d783390 2024-09-21T06:16:32.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7db5bce740 10 client.25646 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:16:32.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7db5bce740 10 client.25646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:32.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7db5bce740 20 client.25646 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7db5bce740 1 --2- 172.21.15.3:0/3448364439 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557a1d737da0 0x557a1dbd5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7db5bce740 4 client.25646 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7db5bce740 1 -- 172.21.15.3:0/3448364439 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a1dabccd0 con 0x557a1d737da0 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.677+0000 7f7dad3f6640 1 --2- 172.21.15.3:0/3448364439 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557a1d737da0 0x557a1dbd5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.678+0000 7f7d90ff9640 10 client.25646 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.678+0000 7f7dad3f6640 1 --2- 172.21.15.3:0/3448364439 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557a1d737da0 0x557a1dbd5b80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66f4e59b14882ef server_cookie=ce66c65741373e5a in_seq=0 out_seq=0 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.678+0000 7f7d90ff9640 1 -- 172.21.15.3:0/3448364439 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a1dabccd0 con 0x557a1d737da0 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.678+0000 7f7d90ff9640 10 client.25646 handle_command_reply: tid=0 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.678+0000 7f7db5bce740 1 client.25646 shutdown 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.678+0000 7f7db5bce740 1 -- 172.21.15.3:0/3448364439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d783390 msgr2=0x557a1db5aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.678+0000 7f7db5bce740 1 --2- 172.21.15.3:0/3448364439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d783390 0x557a1db5aa70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0aab30 tx=0x7f7d9c00ce60 comp rx=0 tx=0).stop 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.678+0000 7f7db5bce740 1 -- 172.21.15.3:0/3448364439 shutdown_connections 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.678+0000 7f7db5bce740 1 --2- 172.21.15.3:0/3448364439 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557a1d737da0 0x557a1dbd5b80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.678+0000 7f7db5bce740 1 --2- 172.21.15.3:0/3448364439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a1d7a4070 0x557a1db5b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.679+0000 7f7d90ff9640 0 client.25646 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.678+0000 7f7db5bce740 1 --2- 172.21.15.3:0/3448364439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a1d783390 0x557a1db5aa70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.679+0000 7f7db5bce740 1 --2- 172.21.15.3:0/3448364439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a1d77b1b0 0x557a1db5f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.679+0000 7f7db5bce740 1 -- 172.21.15.3:0/3448364439 >> 172.21.15.3:0/3448364439 conn(0x557a1d798460 msgr2=0x557a1db54ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.679+0000 7f7db5bce740 1 -- 172.21.15.3:0/3448364439 shutdown_connections 2024-09-21T06:16:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.679+0000 7f7db5bce740 1 -- 172.21.15.3:0/3448364439 wait complete. 2024-09-21T06:16:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.679+0000 7f7db5bce740 20 client.25646 trim_cache size 0 max 16384 2024-09-21T06:16:32.685 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:16:32.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.679+0000 7f7daf65a640 1 -- 172.21.15.3:0/2116715276 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7d940b6680 msgr2=0x7f7d940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:32.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.679+0000 7f7daf65a640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7d940b6680 0x7f7d940b8b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7da0005e30 tx=0x7f7da0005c50 comp rx=0 tx=0).stop 2024-09-21T06:16:32.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.679+0000 7f7daf65a640 1 -- 172.21.15.3:0/2116715276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7da81083a0 msgr2=0x7f7da81995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:32.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.679+0000 7f7daf65a640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7da81083a0 0x7f7da81995c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00e9c0 tx=0x7f7d9c00ee80 comp rx=0 tx=0).stop 2024-09-21T06:16:32.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.680+0000 7f7daf65a640 1 -- 172.21.15.3:0/2116715276 shutdown_connections 2024-09-21T06:16:32.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.680+0000 7f7daf65a640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f7d940b6680 0x7f7d940b8b90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.680+0000 7f7daf65a640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da8108cf0 0x7f7da8199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.680+0000 7f7daf65a640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7da81083a0 0x7f7da81995c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.680+0000 7f7daf65a640 1 --2- 172.21.15.3:0/2116715276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7da8102a60 0x7f7da8199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:32.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.680+0000 7f7daf65a640 1 -- 172.21.15.3:0/2116715276 >> 172.21.15.3:0/2116715276 conn(0x7f7da8079730 msgr2=0x7f7da8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:32.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.680+0000 7f7daf65a640 1 -- 172.21.15.3:0/2116715276 shutdown_connections 2024-09-21T06:16:32.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:32.680+0000 7f7daf65a640 1 -- 172.21.15.3:0/2116715276 wait complete. 2024-09-21T06:16:32.944 INFO:tasks.workunit.client.0.smithi003.stdout: 1 16572 20.89 MB/sec warmup 5 sec latency 13.515 ms 2024-09-21T06:16:33.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:33 smithi003 ceph-mon[31227]: pgmap v371: 97 pgs: 97 active+clean; 5.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 432 KiB/s wr, 71 op/s 2024-09-21T06:16:33.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:33 smithi137 ceph-mon[34464]: pgmap v371: 97 pgs: 97 active+clean; 5.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 432 KiB/s wr, 71 op/s 2024-09-21T06:16:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:33 smithi204 ceph-mon[34659]: pgmap v371: 97 pgs: 97 active+clean; 5.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 432 KiB/s wr, 71 op/s 2024-09-21T06:16:33.944 INFO:tasks.workunit.client.0.smithi003.stdout: 1 19702 20.34 MB/sec warmup 6 sec latency 13.731 ms 2024-09-21T06:16:34.391 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:16:34.391 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:16:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:34.944 INFO:tasks.workunit.client.0.smithi003.stdout: 1 22793 20.12 MB/sec warmup 7 sec latency 30.655 ms 2024-09-21T06:16:35.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.181+0000 7fdbbe928640 1 -- 172.21.15.3:0/2692450376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb8102a60 msgr2=0x7fdbb8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:35.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.181+0000 7fdbbe928640 1 --2- 172.21.15.3:0/2692450376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb8102a60 0x7fdbb8102e70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009900 tx=0x7fdba802a410 comp rx=0 tx=0).stop 2024-09-21T06:16:35.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.181+0000 7fdbbe928640 1 -- 172.21.15.3:0/2692450376 shutdown_connections 2024-09-21T06:16:35.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.181+0000 7fdbbe928640 1 --2- 172.21.15.3:0/2692450376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbb8108cf0 0x7fdbb810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.181+0000 7fdbbe928640 1 --2- 172.21.15.3:0/2692450376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbb81083a0 0x7fdbb81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.181+0000 7fdbbe928640 1 --2- 172.21.15.3:0/2692450376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb8102a60 0x7fdbb8102e70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.181+0000 7fdbbe928640 1 -- 172.21.15.3:0/2692450376 >> 172.21.15.3:0/2692450376 conn(0x7fdbb8079730 msgr2=0x7fdbb8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.181+0000 7fdbbe928640 1 -- 172.21.15.3:0/2692450376 shutdown_connections 2024-09-21T06:16:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.181+0000 7fdbbe928640 1 -- 172.21.15.3:0/2692450376 wait complete. 2024-09-21T06:16:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.182+0000 7fdbbe928640 1 Processor -- start 2024-09-21T06:16:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.182+0000 7fdbbe928640 1 -- start start 2024-09-21T06:16:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.182+0000 7fdbbe928640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb8102a60 0x7fdbb819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.182+0000 7fdbbe928640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbb81083a0 0x7fdbb819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.182+0000 7fdbbe928640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbb8108cf0 0x7fdbb819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.182+0000 7fdbbe928640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8113a80 con 0x7fdbb8102a60 2024-09-21T06:16:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.182+0000 7fdbbe928640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8113900 con 0x7fdbb8108cf0 2024-09-21T06:16:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.182+0000 7fdbbe928640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8113c00 con 0x7fdbb81083a0 2024-09-21T06:16:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.182+0000 7fdbb7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb8102a60 0x7fdbb819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbb81083a0 0x7fdbb819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbb7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb8102a60 0x7fdbb819a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35774/0 (socket says 172.21.15.3:35774) 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbb7fff640 1 -- 172.21.15.3:0/614516531 learned_addr learned my addr 172.21.15.3:0/614516531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbbcec5640 1 --2- 172.21.15.3:0/614516531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbb8108cf0 0x7fdbb819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbaffff640 1 -- 172.21.15.3:0/614516531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbb8108cf0 msgr2=0x7fdbb819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbaffff640 1 --2- 172.21.15.3:0/614516531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbb8108cf0 0x7fdbb819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbaffff640 1 -- 172.21.15.3:0/614516531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb8102a60 msgr2=0x7fdbb819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbaffff640 1 --2- 172.21.15.3:0/614516531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb8102a60 0x7fdbb819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbaffff640 1 -- 172.21.15.3:0/614516531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb81a0380 con 0x7fdbb81083a0 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbaffff640 1 --2- 172.21.15.3:0/614516531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbb81083a0 0x7fdbb819a9b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdba000e9c0 tx=0x7fdba000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbb5ffb640 1 -- 172.21.15.3:0/614516531 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba000ccc0 con 0x7fdbb81083a0 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbbe928640 1 -- 172.21.15.3:0/614516531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb81a0670 con 0x7fdbb81083a0 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbb5ffb640 1 -- 172.21.15.3:0/614516531 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba00108e0 con 0x7fdbb81083a0 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbb5ffb640 1 -- 172.21.15.3:0/614516531 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba00187a0 con 0x7fdbb81083a0 2024-09-21T06:16:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.183+0000 7fdbbe928640 1 -- 172.21.15.3:0/614516531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb810fe90 con 0x7fdbb81083a0 2024-09-21T06:16:35.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbc4e9c740 1 Processor -- start 2024-09-21T06:16:35.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbc4e9c740 1 -- start start 2024-09-21T06:16:35.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbc4e9c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c4c350 0x55b961ff3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbc4e9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c23490 0x55b961ff3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbb5ffb640 1 -- 172.21.15.3:0/614516531 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fdba0010430 con 0x7fdbb81083a0 2024-09-21T06:16:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbc4e9c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 0x55b961ff3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbc4e9c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b961f69e60 con 0x55b961c4c350 2024-09-21T06:16:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbc4e9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b961f59290 con 0x55b961c23490 2024-09-21T06:16:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbc4e9c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b961f5d0a0 con 0x55b961c2b670 2024-09-21T06:16:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbbcec5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c4c350 0x55b961ff3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbbcec5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c4c350 0x55b961ff3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35788/0 (socket says 172.21.15.3:35788) 2024-09-21T06:16:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbbcec5640 1 -- 172.21.15.3:0/1253055037 learned_addr learned my addr 172.21.15.3:0/1253055037 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbb7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c23490 0x55b961ff3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.187+0000 7fdbaffff640 1 --2- 172.21.15.3:0/1253055037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 0x55b961ff3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.188+0000 7fdbb7fff640 1 -- 172.21.15.3:0/1253055037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 msgr2=0x55b961ff3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.188+0000 7fdbb7fff640 1 --2- 172.21.15.3:0/1253055037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 0x55b961ff3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.188+0000 7fdbb7fff640 1 -- 172.21.15.3:0/1253055037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c4c350 msgr2=0x55b961ff3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.188+0000 7fdbb7fff640 1 --2- 172.21.15.3:0/1253055037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c4c350 0x55b961ff3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.188+0000 7fdbb7fff640 1 -- 172.21.15.3:0/1253055037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b961cb7930 con 0x55b961c23490 2024-09-21T06:16:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.188+0000 7fdbaffff640 1 --2- 172.21.15.3:0/1253055037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 0x55b961ff3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.188+0000 7fdbb7fff640 1 --2- 172.21.15.3:0/1253055037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c23490 0x55b961ff3970 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdba8004770 tx=0x7fdba8031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a63aea40340e8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:35.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.188+0000 7fdbae7fc640 1 -- 172.21.15.3:0/1253055037 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8040070 con 0x55b961c23490 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.188+0000 7fdbae7fc640 1 -- 172.21.15.3:0/1253055037 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba8004030 con 0x55b961c23490 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.188+0000 7fdbae7fc640 1 -- 172.21.15.3:0/1253055037 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8035040 con 0x55b961c23490 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.188+0000 7fdbb5ffb640 1 --2- 172.21.15.3:0/614516531 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdba40b6680 0x7fdba40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.189+0000 7fdbbcec5640 1 --2- 172.21.15.3:0/614516531 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdba40b6680 0x7fdba40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.189+0000 7fdbb5ffb640 1 -- 172.21.15.3:0/614516531 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fdba00e7a60 con 0x7fdbb81083a0 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.189+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/1253055037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c23490 msgr2=0x55b961ff3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.189+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/1253055037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c23490 0x55b961ff3970 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdba8004770 tx=0x7fdba8031b10 comp rx=0 tx=0).stop 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.189+0000 7fdbbcec5640 1 --2- 172.21.15.3:0/614516531 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdba40b6680 0x7fdba40b8b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00116b0 tx=0x7fdbb0009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.189+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/1253055037 shutdown_connections 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.189+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/1253055037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 0x55b961ff3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.189+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/1253055037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c23490 0x55b961ff3970 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.189+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/1253055037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c4c350 0x55b961ff3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.189+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/1253055037 >> 172.21.15.3:0/1253055037 conn(0x55b961c40740 msgr2=0x55b961ff1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.190+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/1253055037 shutdown_connections 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.190+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/1253055037 wait complete. 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.190+0000 7fdbc4e9c740 1 -- start start 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.190+0000 7fdbc4e9c740 1 Processor -- start 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbc4e9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c23490 0x55b961fff000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbc4e9c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 0x55b961fff540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbc4e9c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c4c350 0x55b96200aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbc4e9c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b961dba890 con 0x55b961c4c350 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbc4e9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b961ef3d40 con 0x55b961c23490 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbc4e9c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b962001bb0 con 0x55b961c2b670 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbc4e9c740 10 client.0 _ll_register_callbacks cb 0x55b961f57c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbb7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c23490 0x55b961fff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbbcec5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c4c350 0x55b96200aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbb7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c23490 0x55b961fff000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:54288/0 (socket says 172.21.15.3:54288) 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbb7fff640 1 -- 172.21.15.3:0/820381225 learned_addr learned my addr 172.21.15.3:0/820381225 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbaffff640 1 --2- 172.21.15.3:0/820381225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 0x55b961fff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbb7fff640 1 -- 172.21.15.3:0/820381225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 msgr2=0x55b961fff540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:35.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbb7fff640 1 --2- 172.21.15.3:0/820381225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 0x55b961fff540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbb7fff640 1 -- 172.21.15.3:0/820381225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c4c350 msgr2=0x55b96200aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:35.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbb7fff640 1 --2- 172.21.15.3:0/820381225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c4c350 0x55b96200aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbb7fff640 1 -- 172.21.15.3:0/820381225 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b96200b490 con 0x55b961c23490 2024-09-21T06:16:35.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.191+0000 7fdbbcec5640 1 --2- 172.21.15.3:0/820381225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c4c350 0x55b96200aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.192+0000 7fdbb7fff640 1 --2- 172.21.15.3:0/820381225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c23490 0x55b961fff000 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdba802abe0 tx=0x7fdba8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbad38e264322d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.192+0000 7fdb977fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.192+0000 7fdb977fe640 1 -- 172.21.15.3:0/820381225 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba804c070 con 0x55b961c23490 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.192+0000 7fdb977fe640 1 -- 172.21.15.3:0/820381225 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fdba80314a0 con 0x55b961c23490 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.192+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/820381225 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdba8004030 con 0x55b961c23490 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.192+0000 7fdb977fe640 1 -- 172.21.15.3:0/820381225 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdba8058050 con 0x55b961c23490 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.192+0000 7fdbc4e9c740 10 client.25658 fetch_fsmap learned FSMap version 29 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.192+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/820381225 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b96200dda0 con 0x55b961c23490 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.192+0000 7fdb977fe640 1 -- 172.21.15.3:0/820381225 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba8004030 con 0x55b961c23490 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.193+0000 7fdb977fe640 1 -- 172.21.15.3:0/820381225 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdba80035f0 con 0x55b961c23490 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.193+0000 7fdbc4e9c740 10 client.25658 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.193+0000 7fdbc4e9c740 10 client.25658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.193+0000 7fdbc4e9c740 20 client.25658 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.193+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/820381225 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b961c0acd0 0x55b962011100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.193+0000 7fdbc4e9c740 4 client.25658 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:35.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.193+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/820381225 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b961f6bdb0 con 0x55b961c0acd0 2024-09-21T06:16:35.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.193+0000 7fdbaffff640 1 --2- 172.21.15.3:0/820381225 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b961c0acd0 0x55b962011100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.193+0000 7fdbaffff640 1 --2- 172.21.15.3:0/820381225 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b961c0acd0 0x55b962011100 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e6858248f4685ab server_cookie=2a1d36689f49e234 in_seq=0 out_seq=0 2024-09-21T06:16:35.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.193+0000 7fdb977fe640 10 client.25658 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:35.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.195+0000 7fdb977fe640 1 -- 172.21.15.3:0/820381225 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b961f6bdb0 con 0x55b961c0acd0 2024-09-21T06:16:35.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.195+0000 7fdb977fe640 10 client.25658 handle_command_reply: tid=0 2024-09-21T06:16:35.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.195+0000 7fdbc4e9c740 1 client.25658 shutdown 2024-09-21T06:16:35.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.196+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/820381225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c23490 msgr2=0x55b961fff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:35.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.196+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/820381225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c23490 0x55b961fff000 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdba802abe0 tx=0x7fdba8009260 comp rx=0 tx=0).stop 2024-09-21T06:16:35.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.196+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/820381225 shutdown_connections 2024-09-21T06:16:35.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.196+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/820381225 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b961c0acd0 0x55b962011100 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.196+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/820381225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 0x55b961fff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.196+0000 7fdb977fe640 0 client.25658 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:35.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.196+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/820381225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c4c350 0x55b96200aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.196+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/820381225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c23490 0x55b961fff000 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.196+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/820381225 >> 172.21.15.3:0/820381225 conn(0x55b961c40740 msgr2=0x55b961ff6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:35.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.196+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/820381225 shutdown_connections 2024-09-21T06:16:35.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.196+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/820381225 wait complete. 2024-09-21T06:16:35.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.196+0000 7fdbc4e9c740 20 client.25658 trim_cache size 0 max 16384 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbc4e9c740 1 Processor -- start 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbc4e9c740 1 -- start start 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbc4e9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c0acd0 0x55b96200ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbc4e9c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c23490 0x55b961ff9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbc4e9c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 0x55b961ffbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbc4e9c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81b1800 con 0x55b961c23490 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbc4e9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81b33a0 con 0x55b961c0acd0 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbc4e9c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81b1ed0 con 0x55b961c2b670 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c23490 0x55b961ff9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbb7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c0acd0 0x55b96200ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c23490 0x55b961ff9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35800/0 (socket says 172.21.15.3:35800) 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbaffff640 1 -- 172.21.15.3:0/559218781 learned_addr learned my addr 172.21.15.3:0/559218781 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbbcec5640 1 --2- 172.21.15.3:0/559218781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 0x55b961ffbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.214+0000 7fdbaffff640 1 -- 172.21.15.3:0/559218781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 msgr2=0x55b961ffbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.215+0000 7fdbaffff640 1 --2- 172.21.15.3:0/559218781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 0x55b961ffbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.215+0000 7fdbaffff640 1 -- 172.21.15.3:0/559218781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c0acd0 msgr2=0x55b96200ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.215+0000 7fdbaffff640 1 --2- 172.21.15.3:0/559218781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c0acd0 0x55b96200ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.215+0000 7fdbaffff640 1 -- 172.21.15.3:0/559218781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b961ff5720 con 0x55b961c23490 2024-09-21T06:16:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.215+0000 7fdbaffff640 1 --2- 172.21.15.3:0/559218781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c23490 0x55b961ff9550 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdba00aabb0 tx=0x7fdba00d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7ea071b2b7d895 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.215+0000 7fdbadffb640 1 -- 172.21.15.3:0/559218781 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0010720 con 0x55b961c23490 2024-09-21T06:16:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.215+0000 7fdbadffb640 1 -- 172.21.15.3:0/559218781 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba00e7e50 con 0x55b961c23490 2024-09-21T06:16:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.215+0000 7fdbadffb640 1 -- 172.21.15.3:0/559218781 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba00a9bb0 con 0x55b961c23490 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.216+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/559218781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c23490 msgr2=0x55b961ff9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.216+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/559218781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c23490 0x55b961ff9550 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdba00aabb0 tx=0x7fdba00d1040 comp rx=0 tx=0).stop 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.216+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/559218781 shutdown_connections 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.216+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/559218781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c2b670 0x55b961ffbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.216+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/559218781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c23490 0x55b961ff9550 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.216+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/559218781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c0acd0 0x55b96200ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.216+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/559218781 >> 172.21.15.3:0/559218781 conn(0x55b961c40740 msgr2=0x55b961ffd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.216+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/559218781 shutdown_connections 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.216+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/559218781 wait complete. 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.216+0000 7fdbc4e9c740 1 -- start start 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.216+0000 7fdbc4e9c740 1 Processor -- start 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbc4e9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c0acd0 0x55b962012870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbc4e9c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c23490 0x55b962012db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbc4e9c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c2b670 0x55b96207a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbc4e9c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81b2c70 con 0x55b961c2b670 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbc4e9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81b25a0 con 0x55b961c0acd0 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbc4e9c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81af7e0 con 0x55b961c23490 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbc4e9c740 10 client.0 _ll_register_callbacks cb 0x7fdbb0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbbcec5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c2b670 0x55b96207a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbbcec5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c2b670 0x55b96207a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35814/0 (socket says 172.21.15.3:35814) 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbb7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c0acd0 0x55b962012870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c23490 0x55b962012db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbbcec5640 1 -- 172.21.15.3:0/3091116751 learned_addr learned my addr 172.21.15.3:0/3091116751 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbaffff640 1 -- 172.21.15.3:0/3091116751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c0acd0 msgr2=0x55b962012870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbaffff640 1 --2- 172.21.15.3:0/3091116751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c0acd0 0x55b962012870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbaffff640 1 -- 172.21.15.3:0/3091116751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c2b670 msgr2=0x55b96207a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbaffff640 1 --2- 172.21.15.3:0/3091116751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c2b670 0x55b96207a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbaffff640 1 -- 172.21.15.3:0/3091116751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b96207adc0 con 0x55b961c23490 2024-09-21T06:16:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.217+0000 7fdbbcec5640 1 --2- 172.21.15.3:0/3091116751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c2b670 0x55b96207a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.218+0000 7fdbaffff640 1 --2- 172.21.15.3:0/3091116751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c23490 0x55b962012db0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdba00aabb0 tx=0x7fdba0009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bc8a875f636db53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.218+0000 7fdb977fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.218+0000 7fdb977fe640 1 -- 172.21.15.3:0/3091116751 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba00b2040 con 0x55b961c23490 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.218+0000 7fdb977fe640 1 -- 172.21.15.3:0/3091116751 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fdba00e5410 con 0x55b961c23490 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.218+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/3091116751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdba00e7e50 con 0x55b961c23490 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.218+0000 7fdb977fe640 1 -- 172.21.15.3:0/3091116751 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdba0010720 con 0x55b961c23490 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.218+0000 7fdbc4e9c740 10 client.25636 fetch_fsmap learned FSMap version 29 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.218+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/3091116751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b96207c080 con 0x55b961c23490 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.218+0000 7fdb977fe640 1 -- 172.21.15.3:0/3091116751 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba00e7e50 con 0x55b961c23490 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.218+0000 7fdb977fe640 1 -- 172.21.15.3:0/3091116751 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdba00aec50 con 0x55b961c23490 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.219+0000 7fdbc4e9c740 10 client.25636 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.219+0000 7fdbc4e9c740 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.219+0000 7fdbc4e9c740 20 client.25636 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.219+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/3091116751 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b961c4c350 0x55b96207e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.219+0000 7fdbc4e9c740 4 client.25636 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:16:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.219+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/3091116751 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b961f67860 con 0x55b961c4c350 2024-09-21T06:16:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.219+0000 7fdbb7fff640 1 --2- 172.21.15.3:0/3091116751 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b961c4c350 0x55b96207e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.219+0000 7fdbb7fff640 1 --2- 172.21.15.3:0/3091116751 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b961c4c350 0x55b96207e850 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64af7068e13da73a server_cookie=a88ed8da5244f71a in_seq=0 out_seq=0 2024-09-21T06:16:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.219+0000 7fdb977fe640 10 client.25636 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdb977fe640 1 -- 172.21.15.3:0/3091116751 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b961f67860 con 0x55b961c4c350 2024-09-21T06:16:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdb977fe640 10 client.25636 handle_command_reply: tid=0 2024-09-21T06:16:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdbc4e9c740 1 client.25636 shutdown 2024-09-21T06:16:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/3091116751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c23490 msgr2=0x55b962012db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/3091116751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c23490 0x55b962012db0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdba00aabb0 tx=0x7fdba0009700 comp rx=0 tx=0).stop 2024-09-21T06:16:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/3091116751 shutdown_connections 2024-09-21T06:16:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/3091116751 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b961c4c350 0x55b96207e850 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/3091116751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b961c23490 0x55b962012db0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/3091116751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b961c2b670 0x55b96207a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdbc4e9c740 1 --2- 172.21.15.3:0/3091116751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b961c0acd0 0x55b962012870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdb977fe640 0 client.25636 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:35.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/3091116751 >> 172.21.15.3:0/3091116751 conn(0x55b961c40740 msgr2=0x55b961ff4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:35.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/3091116751 shutdown_connections 2024-09-21T06:16:35.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.221+0000 7fdbc4e9c740 1 -- 172.21.15.3:0/3091116751 wait complete. 2024-09-21T06:16:35.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.222+0000 7fdbc4e9c740 20 client.25636 trim_cache size 0 max 16384 2024-09-21T06:16:35.226 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:16:35.226 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:16:35.226 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9", 2024-09-21T06:16:35.226 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:16:35.226 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:16:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.222+0000 7fdbbe928640 1 -- 172.21.15.3:0/614516531 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdba40b6680 msgr2=0x7fdba40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.222+0000 7fdbbe928640 1 --2- 172.21.15.3:0/614516531 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdba40b6680 0x7fdba40b8b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00116b0 tx=0x7fdbb0009500 comp rx=0 tx=0).stop 2024-09-21T06:16:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.222+0000 7fdbbe928640 1 -- 172.21.15.3:0/614516531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbb81083a0 msgr2=0x7fdbb819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.222+0000 7fdbbe928640 1 --2- 172.21.15.3:0/614516531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbb81083a0 0x7fdbb819a9b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdba000e9c0 tx=0x7fdba000ee80 comp rx=0 tx=0).stop 2024-09-21T06:16:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.222+0000 7fdbbe928640 1 -- 172.21.15.3:0/614516531 shutdown_connections 2024-09-21T06:16:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.222+0000 7fdbbe928640 1 --2- 172.21.15.3:0/614516531 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdba40b6680 0x7fdba40b8b90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.222+0000 7fdbbe928640 1 --2- 172.21.15.3:0/614516531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbb81083a0 0x7fdbb819a9b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.222+0000 7fdbbe928640 1 --2- 172.21.15.3:0/614516531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbb8108cf0 0x7fdbb819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.222+0000 7fdbbe928640 1 --2- 172.21.15.3:0/614516531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb8102a60 0x7fdbb819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.222+0000 7fdbbe928640 1 -- 172.21.15.3:0/614516531 >> 172.21.15.3:0/614516531 conn(0x7fdbb8079730 msgr2=0x7fdbb81013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.223+0000 7fdbbe928640 1 -- 172.21.15.3:0/614516531 shutdown_connections 2024-09-21T06:16:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:35.223+0000 7fdbbe928640 1 -- 172.21.15.3:0/614516531 wait complete. 2024-09-21T06:16:35.339 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:35 smithi003 ceph-mon[31227]: pgmap v372: 97 pgs: 97 active+clean; 7.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 575 KiB/s wr, 88 op/s 2024-09-21T06:16:35.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:35 smithi137 ceph-mon[34464]: pgmap v372: 97 pgs: 97 active+clean; 7.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 575 KiB/s wr, 88 op/s 2024-09-21T06:16:35.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:35 smithi204 ceph-mon[34659]: pgmap v372: 97 pgs: 97 active+clean; 7.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 575 KiB/s wr, 88 op/s 2024-09-21T06:16:35.944 INFO:tasks.workunit.client.0.smithi003.stdout: 1 25541 19.37 MB/sec warmup 8 sec latency 33.249 ms 2024-09-21T06:16:36.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:16:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:36.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:36 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:16:36.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:36 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:36.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:36 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:16:36.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:36 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:16:36.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:36 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:36.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:36 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:36.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:36 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:16:36.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:36 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:36.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:36 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:16:36.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:36 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:16:36.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:36 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:36.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:36 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:36 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:16:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:36 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:36 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:16:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:36 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:16:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:36 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:36 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.795+0000 7f0f25db6640 1 -- 172.21.15.3:0/891889720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f20105300 msgr2=0x7f0f2010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.795+0000 7f0f25db6640 1 --2- 172.21.15.3:0/891889720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f20105300 0x7f0f2010d850 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009900 tx=0x7f0f0802a410 comp rx=0 tx=0).stop 2024-09-21T06:16:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.795+0000 7f0f25db6640 1 -- 172.21.15.3:0/891889720 shutdown_connections 2024-09-21T06:16:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.795+0000 7f0f25db6640 1 --2- 172.21.15.3:0/891889720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f2010de10 0x7f0f20110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.795+0000 7f0f25db6640 1 --2- 172.21.15.3:0/891889720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f20105300 0x7f0f2010d850 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.795+0000 7f0f25db6640 1 --2- 172.21.15.3:0/891889720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f201049b0 0x7f0f20104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.795+0000 7f0f25db6640 1 -- 172.21.15.3:0/891889720 >> 172.21.15.3:0/891889720 conn(0x7f0f20079730 msgr2=0x7f0f20079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.796+0000 7f0f25db6640 1 -- 172.21.15.3:0/891889720 shutdown_connections 2024-09-21T06:16:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.796+0000 7f0f25db6640 1 -- 172.21.15.3:0/891889720 wait complete. 2024-09-21T06:16:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.796+0000 7f0f25db6640 1 Processor -- start 2024-09-21T06:16:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.797+0000 7f0f25db6640 1 -- start start 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.797+0000 7f0f25db6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f201049b0 0x7f0f2019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.797+0000 7f0f25db6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f20105300 0x7f0f2019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.797+0000 7f0f25db6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f2010de10 0x7f0f2019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.797+0000 7f0f25db6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20113a70 con 0x7f0f2010de10 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.797+0000 7f0f25db6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201138f0 con 0x7f0f201049b0 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.797+0000 7f0f25db6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20113bf0 con 0x7f0f20105300 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.797+0000 7f0f1ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f2010de10 0x7f0f2019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.797+0000 7f0f16ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f20105300 0x7f0f2019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.797+0000 7f0f1ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f2010de10 0x7f0f2019fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34198/0 (socket says 172.21.15.3:34198) 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.797+0000 7f0f1ffff640 1 -- 172.21.15.3:0/3197509226 learned_addr learned my addr 172.21.15.3:0/3197509226 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.797+0000 7f0f1f7fe640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f201049b0 0x7f0f2019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.798+0000 7f0f1ffff640 1 -- 172.21.15.3:0/3197509226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f20105300 msgr2=0x7f0f2019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.798+0000 7f0f1ffff640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f20105300 0x7f0f2019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.798+0000 7f0f1ffff640 1 -- 172.21.15.3:0/3197509226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f201049b0 msgr2=0x7f0f2019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.798+0000 7f0f1ffff640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f201049b0 0x7f0f2019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.798+0000 7f0f1ffff640 1 -- 172.21.15.3:0/3197509226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f201a0410 con 0x7f0f2010de10 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.798+0000 7f0f16ffd640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f20105300 0x7f0f2019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.798+0000 7f0f1ffff640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f2010de10 0x7f0f2019fce0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800eaa0 tx=0x7f0f1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.798+0000 7f0f1d7fa640 1 -- 172.21.15.3:0/3197509226 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1800cdb0 con 0x7f0f2010de10 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.798+0000 7f0f25db6640 1 -- 172.21.15.3:0/3197509226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f201a0700 con 0x7f0f2010de10 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.798+0000 7f0f25db6640 1 -- 172.21.15.3:0/3197509226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f201a6e60 con 0x7f0f2010de10 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.798+0000 7f0f1d7fa640 1 -- 172.21.15.3:0/3197509226 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f18005020 con 0x7f0f2010de10 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.798+0000 7f0f1d7fa640 1 -- 172.21.15.3:0/3197509226 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f180116d0 con 0x7f0f2010de10 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.801+0000 7f0f2c32a740 1 Processor -- start 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.801+0000 7f0f1d7fa640 1 -- 172.21.15.3:0/3197509226 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f0f18011870 con 0x7f0f2010de10 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.801+0000 7f0f2c32a740 1 -- start start 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f2c32a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a557070 0x559e2a8fe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f2c32a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 0x559e2a8fe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f2c32a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a536390 0x559e2a8fecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f2c32a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559e2a874e50 con 0x559e2a52e1b0 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f2c32a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559e2a864280 con 0x559e2a557070 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f2c32a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559e2a868090 con 0x559e2a536390 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f1f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 0x559e2a8fe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f16ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a557070 0x559e2a8fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f1f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 0x559e2a8fe7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34212/0 (socket says 172.21.15.3:34212) 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f1f7fe640 1 -- 172.21.15.3:0/1651843297 learned_addr learned my addr 172.21.15.3:0/1651843297 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f16ffd640 1 --2- 172.21.15.3:0/1651843297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a536390 0x559e2a8fecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f16ffd640 1 -- 172.21.15.3:0/1651843297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a536390 msgr2=0x559e2a8fecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f16ffd640 1 --2- 172.21.15.3:0/1651843297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a536390 0x559e2a8fecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f16ffd640 1 -- 172.21.15.3:0/1651843297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 msgr2=0x559e2a8fe7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f16ffd640 1 --2- 172.21.15.3:0/1651843297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 0x559e2a8fe7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f16ffd640 1 -- 172.21.15.3:0/1651843297 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e2a5c2650 con 0x559e2a557070 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f1f7fe640 1 --2- 172.21.15.3:0/1651843297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 0x559e2a8fe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f1d7fa640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0f0c0b6680 0x7f0f0c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f1f7fe640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0f0c0b6680 0x7f0f0c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.802+0000 7f0f16ffd640 1 --2- 172.21.15.3:0/1651843297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a557070 0x559e2a8fe270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f0802a930 tx=0x7f0f08000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cf8f2bac42d4105 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.803+0000 7f0f1d7fa640 1 -- 172.21.15.3:0/3197509226 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f0f180e7830 con 0x7f0f2010de10 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.803+0000 7f0f15ffb640 1 -- 172.21.15.3:0/1651843297 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08044020 con 0x559e2a557070 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.803+0000 7f0f15ffb640 1 -- 172.21.15.3:0/1651843297 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f080035f0 con 0x559e2a557070 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.803+0000 7f0f15ffb640 1 -- 172.21.15.3:0/1651843297 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08035030 con 0x559e2a557070 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.804+0000 7f0f2c32a740 1 -- 172.21.15.3:0/1651843297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a557070 msgr2=0x559e2a8fe270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.804+0000 7f0f1f7fe640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0f0c0b6680 0x7f0f0c0b8b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x559e2a8fcef0 tx=0x7f0f10005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.804+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/1651843297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a557070 0x559e2a8fe270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f0802a930 tx=0x7f0f08000bc0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.804+0000 7f0f2c32a740 1 -- 172.21.15.3:0/1651843297 shutdown_connections 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.804+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/1651843297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a536390 0x559e2a8fecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.804+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/1651843297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 0x559e2a8fe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.804+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/1651843297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a557070 0x559e2a8fe270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.804+0000 7f0f2c32a740 1 -- 172.21.15.3:0/1651843297 >> 172.21.15.3:0/1651843297 conn(0x559e2a54b460 msgr2=0x559e2a8fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.804+0000 7f0f2c32a740 1 -- 172.21.15.3:0/1651843297 shutdown_connections 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.804+0000 7f0f2c32a740 1 -- 172.21.15.3:0/1651843297 wait complete. 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.804+0000 7f0f2c32a740 1 -- start start 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.804+0000 7f0f2c32a740 1 Processor -- start 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f2c32a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 0x559e2a90dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f2c32a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a536390 0x559e2a90e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f2c32a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a557070 0x559e2a91a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f2c32a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559e2a6c5700 con 0x559e2a536390 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f2c32a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559e2a7ff110 con 0x559e2a557070 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f2c32a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559e2a910cf0 con 0x559e2a52e1b0 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f1f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 0x559e2a90dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f1f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 0x559e2a90dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41808/0 (socket says 172.21.15.3:41808) 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f1ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a557070 0x559e2a91a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f1f7fe640 1 -- 172.21.15.3:0/288150042 learned_addr learned my addr 172.21.15.3:0/288150042 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f2c32a740 10 client.0 _ll_register_callbacks cb 0x559e2a818990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f16ffd640 1 --2- 172.21.15.3:0/288150042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a536390 0x559e2a90e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f1f7fe640 1 -- 172.21.15.3:0/288150042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a557070 msgr2=0x559e2a91a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f1f7fe640 1 --2- 172.21.15.3:0/288150042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a557070 0x559e2a91a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f1f7fe640 1 -- 172.21.15.3:0/288150042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a536390 msgr2=0x559e2a90e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f1f7fe640 1 --2- 172.21.15.3:0/288150042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a536390 0x559e2a90e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.805+0000 7f0f1f7fe640 1 -- 172.21.15.3:0/288150042 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e2a82cc40 con 0x559e2a52e1b0 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.806+0000 7f0f1f7fe640 1 --2- 172.21.15.3:0/288150042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 0x559e2a90dd70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f100045f0 tx=0x7f0f10004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2fbea9d05e2dcc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.806+0000 7f0efeffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.806+0000 7f0efeffd640 1 -- 172.21.15.3:0/288150042 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1000f690 con 0x559e2a52e1b0 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.806+0000 7f0efeffd640 1 -- 172.21.15.3:0/288150042 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f0f1000f830 con 0x559e2a52e1b0 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.806+0000 7f0f2c32a740 1 -- 172.21.15.3:0/288150042 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f080035f0 con 0x559e2a52e1b0 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.806+0000 7f0efeffd640 1 -- 172.21.15.3:0/288150042 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f1000fa30 con 0x559e2a52e1b0 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.806+0000 7f0f2c32a740 10 client.25648 fetch_fsmap learned FSMap version 29 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.806+0000 7f0f2c32a740 1 -- 172.21.15.3:0/288150042 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e2a91ca80 con 0x559e2a52e1b0 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.806+0000 7f0efeffd640 1 -- 172.21.15.3:0/288150042 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f080035f0 con 0x559e2a52e1b0 2024-09-21T06:16:36.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.807+0000 7f0efeffd640 1 -- 172.21.15.3:0/288150042 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0f10027070 con 0x559e2a52e1b0 2024-09-21T06:16:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.807+0000 7f0f2c32a740 10 client.25648 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:16:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.807+0000 7f0f2c32a740 10 client.25648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.807+0000 7f0f2c32a740 20 client.25648 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:36.845 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:16:36.845 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:16:36.845 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-21T06:16:36.845 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:16:36.845 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:16:36.845 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9", 2024-09-21T06:16:36.845 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:16:36.846 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:16:36.847 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:16:36.848 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:16:36.848 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:16:36.848 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:16:36.848 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:16:36.848 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:16:36.848 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:16:36.848 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:16:36.848 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:16:36.848 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:16:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.807+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/288150042 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559e2a5159f0 0x559e2a920050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.807+0000 7f0f2c32a740 4 client.25648 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.807+0000 7f0f2c32a740 1 -- 172.21.15.3:0/288150042 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e2a876da0 con 0x559e2a5159f0 2024-09-21T06:16:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.807+0000 7f0f16ffd640 1 --2- 172.21.15.3:0/288150042 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559e2a5159f0 0x559e2a920050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.807+0000 7f0efeffd640 10 client.25648 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.807+0000 7f0f16ffd640 1 --2- 172.21.15.3:0/288150042 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559e2a5159f0 0x559e2a920050 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cda68a95ab11f326 server_cookie=20d9feac0c5646b7 in_seq=0 out_seq=0 2024-09-21T06:16:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.808+0000 7f0efeffd640 1 -- 172.21.15.3:0/288150042 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559e2a876da0 con 0x559e2a5159f0 2024-09-21T06:16:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.808+0000 7f0efeffd640 10 client.25648 handle_command_reply: tid=0 2024-09-21T06:16:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.808+0000 7f0f2c32a740 1 client.25648 shutdown 2024-09-21T06:16:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.808+0000 7f0f2c32a740 1 -- 172.21.15.3:0/288150042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 msgr2=0x559e2a90dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.808+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/288150042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 0x559e2a90dd70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f100045f0 tx=0x7f0f10004730 comp rx=0 tx=0).stop 2024-09-21T06:16:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.808+0000 7f0f2c32a740 1 -- 172.21.15.3:0/288150042 shutdown_connections 2024-09-21T06:16:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.808+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/288150042 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559e2a5159f0 0x559e2a920050 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.808+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/288150042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a557070 0x559e2a91a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.808+0000 7f0efeffd640 0 client.25648 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.808+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/288150042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a536390 0x559e2a90e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.808+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/288150042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 0x559e2a90dd70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.808+0000 7f0f2c32a740 1 -- 172.21.15.3:0/288150042 >> 172.21.15.3:0/288150042 conn(0x559e2a54b460 msgr2=0x559e2a8fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.809+0000 7f0f2c32a740 1 -- 172.21.15.3:0/288150042 shutdown_connections 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.809+0000 7f0f2c32a740 1 -- 172.21.15.3:0/288150042 wait complete. 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.809+0000 7f0f2c32a740 20 client.25648 trim_cache size 0 max 16384 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.828+0000 7f0f2c32a740 1 Processor -- start 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.828+0000 7f0f2c32a740 1 -- start start 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.828+0000 7f0f2c32a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 0x559e2a90cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.828+0000 7f0f2c32a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a536390 0x559e2a90f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.828+0000 7f0f2c32a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a557070 0x559e2a919f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.828+0000 7f0f2c32a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201ad3e0 con 0x559e2a52e1b0 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.828+0000 7f0f2c32a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201aef80 con 0x559e2a536390 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.828+0000 7f0f2c32a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201adab0 con 0x559e2a557070 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.828+0000 7f0f1f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 0x559e2a90cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f16ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a536390 0x559e2a90f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f1f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 0x559e2a90cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34234/0 (socket says 172.21.15.3:34234) 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f1f7fe640 1 -- 172.21.15.3:0/3897063733 learned_addr learned my addr 172.21.15.3:0/3897063733 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f1ffff640 1 --2- 172.21.15.3:0/3897063733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a557070 0x559e2a919f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f1f7fe640 1 -- 172.21.15.3:0/3897063733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a557070 msgr2=0x559e2a919f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f1f7fe640 1 --2- 172.21.15.3:0/3897063733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a557070 0x559e2a919f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f1f7fe640 1 -- 172.21.15.3:0/3897063733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a536390 msgr2=0x559e2a90f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f1f7fe640 1 --2- 172.21.15.3:0/3897063733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a536390 0x559e2a90f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f1f7fe640 1 -- 172.21.15.3:0/3897063733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e2a900560 con 0x559e2a52e1b0 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f16ffd640 1 --2- 172.21.15.3:0/3897063733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a536390 0x559e2a90f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:36.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f1f7fe640 1 --2- 172.21.15.3:0/3897063733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 0x559e2a90cc60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0f10012d00 tx=0x7f0f10009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a123d795ebd5b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f157fa640 1 -- 172.21.15.3:0/3897063733 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10036030 con 0x559e2a52e1b0 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f157fa640 1 -- 172.21.15.3:0/3897063733 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f100054d0 con 0x559e2a52e1b0 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.829+0000 7f0f157fa640 1 -- 172.21.15.3:0/3897063733 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10004a00 con 0x559e2a52e1b0 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.830+0000 7f0f2c32a740 1 -- 172.21.15.3:0/3897063733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 msgr2=0x559e2a90cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.830+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/3897063733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 0x559e2a90cc60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0f10012d00 tx=0x7f0f10009c80 comp rx=0 tx=0).stop 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.830+0000 7f0f2c32a740 1 -- 172.21.15.3:0/3897063733 shutdown_connections 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.830+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/3897063733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a557070 0x559e2a919f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.830+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/3897063733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a536390 0x559e2a90f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.830+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/3897063733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a52e1b0 0x559e2a90cc60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.830+0000 7f0f2c32a740 1 -- 172.21.15.3:0/3897063733 >> 172.21.15.3:0/3897063733 conn(0x559e2a54b460 msgr2=0x559e2a90f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.830+0000 7f0f2c32a740 1 -- 172.21.15.3:0/3897063733 shutdown_connections 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.830+0000 7f0f2c32a740 1 -- 172.21.15.3:0/3897063733 wait complete. 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.831+0000 7f0f2c32a740 1 -- start start 2024-09-21T06:16:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.831+0000 7f0f2c32a740 1 Processor -- start 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.831+0000 7f0f2c32a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 0x559e2a9122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.831+0000 7f0f2c32a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a536390 0x559e2a9127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.831+0000 7f0f2c32a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a557070 0x559e2a989a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.831+0000 7f0f2c32a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201ae850 con 0x559e2a557070 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.831+0000 7f0f2c32a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201ae180 con 0x559e2a536390 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.831+0000 7f0f2c32a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201ab3c0 con 0x559e2a52e1b0 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.831+0000 7f0f2c32a740 10 client.0 _ll_register_callbacks cb 0x7f0f10009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.831+0000 7f0f1f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 0x559e2a9122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0f1ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a557070 0x559e2a989a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0f16ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a536390 0x559e2a9127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0f1f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 0x559e2a9122a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41824/0 (socket says 172.21.15.3:41824) 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0f1f7fe640 1 -- 172.21.15.3:0/2835096714 learned_addr learned my addr 172.21.15.3:0/2835096714 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0f1ffff640 1 -- 172.21.15.3:0/2835096714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 msgr2=0x559e2a9122a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:36.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0f1ffff640 1 --2- 172.21.15.3:0/2835096714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 0x559e2a9122a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0f1ffff640 1 -- 172.21.15.3:0/2835096714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a536390 msgr2=0x559e2a9127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0f1ffff640 1 --2- 172.21.15.3:0/2835096714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a536390 0x559e2a9127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0f1ffff640 1 -- 172.21.15.3:0/2835096714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e2a9130b0 con 0x559e2a557070 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0f16ffd640 1 --2- 172.21.15.3:0/2835096714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a536390 0x559e2a9127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0f1f7fe640 1 --2- 172.21.15.3:0/2835096714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 0x559e2a9122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0f1ffff640 1 --2- 172.21.15.3:0/2835096714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a557070 0x559e2a989a10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0f180e4fc0 tx=0x7f0f18005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190166cd72033b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0efeffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0efeffd640 1 -- 172.21.15.3:0/2835096714 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18098070 con 0x559e2a557070 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0efeffd640 1 -- 172.21.15.3:0/2835096714 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f0f180acbd0 con 0x559e2a557070 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.832+0000 7f0f2c32a740 1 -- 172.21.15.3:0/2835096714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f100054d0 con 0x559e2a557070 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.833+0000 7f0efeffd640 1 -- 172.21.15.3:0/2835096714 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f1801f050 con 0x559e2a557070 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.833+0000 7f0f2c32a740 10 client.16098 fetch_fsmap learned FSMap version 29 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.833+0000 7f0f2c32a740 1 -- 172.21.15.3:0/2835096714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e2a98b040 con 0x559e2a557070 2024-09-21T06:16:36.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.833+0000 7f0efeffd640 1 -- 172.21.15.3:0/2835096714 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f100054d0 con 0x559e2a557070 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.833+0000 7f0efeffd640 1 -- 172.21.15.3:0/2835096714 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0f1800f040 con 0x559e2a557070 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.833+0000 7f0f2c32a740 10 client.16098 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.833+0000 7f0f2c32a740 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.833+0000 7f0f2c32a740 20 client.16098 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.833+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/2835096714 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559e2a4eada0 0x559e2a98d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.833+0000 7f0f2c32a740 4 client.16098 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.833+0000 7f0f2c32a740 1 -- 172.21.15.3:0/2835096714 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e2a8fafd0 con 0x559e2a4eada0 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.833+0000 7f0f1f7fe640 1 --2- 172.21.15.3:0/2835096714 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559e2a4eada0 0x559e2a98d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0efeffd640 10 client.16098 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0f1f7fe640 1 --2- 172.21.15.3:0/2835096714 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559e2a4eada0 0x559e2a98d810 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef88d448c2327ff1 server_cookie=1662bfad00a1479f in_seq=0 out_seq=0 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0efeffd640 1 -- 172.21.15.3:0/2835096714 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x559e2a8fafd0 con 0x559e2a4eada0 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0efeffd640 10 client.16098 handle_command_reply: tid=0 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0f2c32a740 1 client.16098 shutdown 2024-09-21T06:16:36.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0f2c32a740 1 -- 172.21.15.3:0/2835096714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a557070 msgr2=0x559e2a989a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/2835096714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a557070 0x559e2a989a10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0f180e4fc0 tx=0x7f0f18005400 comp rx=0 tx=0).stop 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0f2c32a740 1 -- 172.21.15.3:0/2835096714 shutdown_connections 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/2835096714 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559e2a4eada0 0x559e2a98d810 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/2835096714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559e2a557070 0x559e2a989a10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/2835096714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e2a536390 0x559e2a9127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0f2c32a740 1 --2- 172.21.15.3:0/2835096714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e2a52e1b0 0x559e2a9122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0efeffd640 0 client.16098 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0f2c32a740 1 -- 172.21.15.3:0/2835096714 >> 172.21.15.3:0/2835096714 conn(0x559e2a54b460 msgr2=0x559e2a90f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.834+0000 7f0f2c32a740 1 -- 172.21.15.3:0/2835096714 shutdown_connections 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.835+0000 7f0f2c32a740 1 -- 172.21.15.3:0/2835096714 wait complete. 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.835+0000 7f0f2c32a740 20 client.16098 trim_cache size 0 max 16384 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.835+0000 7f0f25db6640 1 -- 172.21.15.3:0/3197509226 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0f0c0b6680 msgr2=0x7f0f0c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.835+0000 7f0f25db6640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0f0c0b6680 0x7f0f0c0b8b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x559e2a8fcef0 tx=0x7f0f10005f90 comp rx=0 tx=0).stop 2024-09-21T06:16:36.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.835+0000 7f0f25db6640 1 -- 172.21.15.3:0/3197509226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f2010de10 msgr2=0x7f0f2019fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:36.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.835+0000 7f0f25db6640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f2010de10 0x7f0f2019fce0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800eaa0 tx=0x7f0f1800ef60 comp rx=0 tx=0).stop 2024-09-21T06:16:36.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.835+0000 7f0f25db6640 1 -- 172.21.15.3:0/3197509226 shutdown_connections 2024-09-21T06:16:36.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.835+0000 7f0f25db6640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0f0c0b6680 0x7f0f0c0b8b90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.835+0000 7f0f25db6640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f20105300 0x7f0f2019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.835+0000 7f0f25db6640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f2010de10 0x7f0f2019fce0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.835+0000 7f0f25db6640 1 --2- 172.21.15.3:0/3197509226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f201049b0 0x7f0f2019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:36.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.835+0000 7f0f25db6640 1 -- 172.21.15.3:0/3197509226 >> 172.21.15.3:0/3197509226 conn(0x7f0f20079730 msgr2=0x7f0f20100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:36.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.836+0000 7f0f25db6640 1 -- 172.21.15.3:0/3197509226 shutdown_connections 2024-09-21T06:16:36.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:16:36.836+0000 7f0f25db6640 1 -- 172.21.15.3:0/3197509226 wait complete. 2024-09-21T06:16:36.944 INFO:tasks.workunit.client.0.smithi003.stdout: 1 28758 19.33 MB/sec warmup 9 sec latency 27.454 ms 2024-09-21T06:16:37.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:37 smithi003 ceph-mon[31227]: pgmap v373: 97 pgs: 97 active+clean; 10 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 902 KiB/s wr, 101 op/s 2024-09-21T06:16:37.561 INFO:tasks.cephfs.filesystem:scrub status for tag:a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9 - {'path': '/', 'tag': 'a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9', 'options': 'recursive,force'} 2024-09-21T06:16:37.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:37 smithi137 ceph-mon[34464]: pgmap v373: 97 pgs: 97 active+clean; 10 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 902 KiB/s wr, 101 op/s 2024-09-21T06:16:37.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:37 smithi204 ceph-mon[34659]: pgmap v373: 97 pgs: 97 active+clean; 10 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 902 KiB/s wr, 101 op/s 2024-09-21T06:16:37.944 INFO:tasks.workunit.client.0.smithi003.stdout: 1 31946 19.14 MB/sec warmup 10 sec latency 14.440 ms 2024-09-21T06:16:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.965+0000 7ff58724d640 1 -- 172.21.15.3:0/1136580699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5801029b0 msgr2=0x7ff580102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.965+0000 7ff58724d640 1 --2- 172.21.15.3:0/1136580699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5801029b0 0x7ff580102dc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009900 tx=0x7ff57c02a410 comp rx=0 tx=0).stop 2024-09-21T06:16:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.965+0000 7ff58724d640 1 -- 172.21.15.3:0/1136580699 shutdown_connections 2024-09-21T06:16:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.965+0000 7ff58724d640 1 --2- 172.21.15.3:0/1136580699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5801082e0 0x7ff5801086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.965+0000 7ff58724d640 1 --2- 172.21.15.3:0/1136580699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff580108c30 0x7ff58010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.965+0000 7ff58724d640 1 --2- 172.21.15.3:0/1136580699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5801029b0 0x7ff580102dc0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.965+0000 7ff58724d640 1 -- 172.21.15.3:0/1136580699 >> 172.21.15.3:0/1136580699 conn(0x7ff58007a740 msgr2=0x7ff58007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.965+0000 7ff58724d640 1 -- 172.21.15.3:0/1136580699 shutdown_connections 2024-09-21T06:16:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.965+0000 7ff58724d640 1 -- 172.21.15.3:0/1136580699 wait complete. 2024-09-21T06:16:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.965+0000 7ff58724d640 1 Processor -- start 2024-09-21T06:16:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff58724d640 1 -- start start 2024-09-21T06:16:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff58724d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5801029b0 0x7ff58019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff58724d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5801082e0 0x7ff58019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff58724d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff580108c30 0x7ff58019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff58724d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801128f0 con 0x7ff5801029b0 2024-09-21T06:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff58724d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff580112770 con 0x7ff5801082e0 2024-09-21T06:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff58724d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff580112a70 con 0x7ff580108c30 2024-09-21T06:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff584fe9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5801029b0 0x7ff58019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff584fe9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5801029b0 0x7ff58019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34252/0 (socket says 172.21.15.3:34252) 2024-09-21T06:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff584fe9640 1 -- 172.21.15.3:0/1053393290 learned_addr learned my addr 172.21.15.3:0/1053393290 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff577fff640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5801082e0 0x7ff58019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff5857ea640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff580108c30 0x7ff58019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff584fe9640 1 -- 172.21.15.3:0/1053393290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff580108c30 msgr2=0x7ff58019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff584fe9640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff580108c30 0x7ff58019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff584fe9640 1 -- 172.21.15.3:0/1053393290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5801082e0 msgr2=0x7ff58019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff584fe9640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5801082e0 0x7ff58019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff584fe9640 1 -- 172.21.15.3:0/1053393290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5801a03b0 con 0x7ff5801029b0 2024-09-21T06:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.966+0000 7ff577fff640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5801082e0 0x7ff58019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.967+0000 7ff584fe9640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5801029b0 0x7ff58019a480 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff57c005940 tx=0x7ff57c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.967+0000 7ff575ffb640 1 -- 172.21.15.3:0/1053393290 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c03f030 con 0x7ff5801029b0 2024-09-21T06:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.967+0000 7ff58724d640 1 -- 172.21.15.3:0/1053393290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5801a0640 con 0x7ff5801029b0 2024-09-21T06:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.967+0000 7ff575ffb640 1 -- 172.21.15.3:0/1053393290 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff57c0044c0 con 0x7ff5801029b0 2024-09-21T06:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.967+0000 7ff575ffb640 1 -- 172.21.15.3:0/1053393290 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c035070 con 0x7ff5801029b0 2024-09-21T06:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.967+0000 7ff58724d640 1 -- 172.21.15.3:0/1053393290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5801a6e30 con 0x7ff5801029b0 2024-09-21T06:16:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff575ffb640 1 -- 172.21.15.3:0/1053393290 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7ff57c042030 con 0x7ff5801029b0 2024-09-21T06:16:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff58e1be740 1 Processor -- start 2024-09-21T06:16:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff58e1be740 1 -- start start 2024-09-21T06:16:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff58e1be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf1239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff58e1be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf123ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff58e1be740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf124430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff58e1be740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562fdf0986f0 con 0x562fded5d4d0 2024-09-21T06:16:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff58e1be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562fdf087b20 con 0x562fded552f0 2024-09-21T06:16:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff58e1be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562fdf08b930 con 0x562fded7e1b0 2024-09-21T06:16:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff577fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf124430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff584fe9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf123ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff5857ea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf1239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff5857ea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf1239b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41866/0 (socket says 172.21.15.3:41866) 2024-09-21T06:16:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff584fe9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf123ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:34844/0 (socket says 172.21.15.3:34844) 2024-09-21T06:16:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff577fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf124430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34262/0 (socket says 172.21.15.3:34262) 2024-09-21T06:16:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.970+0000 7ff5857ea640 1 -- 172.21.15.3:0/959525918 learned_addr learned my addr 172.21.15.3:0/959525918 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.971+0000 7ff5857ea640 1 -- 172.21.15.3:0/959525918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 msgr2=0x562fdf123ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.971+0000 7ff575ffb640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff5680b6680 0x7ff5680b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.971+0000 7ff5857ea640 1 --2- 172.21.15.3:0/959525918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf123ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.971+0000 7ff5857ea640 1 -- 172.21.15.3:0/959525918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 msgr2=0x562fdf124430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.971+0000 7ff5857ea640 1 --2- 172.21.15.3:0/959525918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf124430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.971+0000 7ff5857ea640 1 -- 172.21.15.3:0/959525918 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fdf04d530 con 0x562fded7e1b0 2024-09-21T06:16:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.971+0000 7ff575ffb640 1 -- 172.21.15.3:0/1053393290 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff57c107ab0 con 0x7ff5801029b0 2024-09-21T06:16:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.971+0000 7ff5857ea640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff5680b6680 0x7ff5680b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.971+0000 7ff5857ea640 1 --2- 172.21.15.3:0/959525918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf1239b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff578009f80 tx=0x7ff5780118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cede8df1af11507 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.971+0000 7ff566ffd640 1 -- 172.21.15.3:0/959525918 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578004250 con 0x562fded7e1b0 2024-09-21T06:16:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.971+0000 7ff566ffd640 1 -- 172.21.15.3:0/959525918 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff578003d50 con 0x562fded7e1b0 2024-09-21T06:16:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.971+0000 7ff566ffd640 1 -- 172.21.15.3:0/959525918 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57801bb30 con 0x562fded7e1b0 2024-09-21T06:16:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.972+0000 7ff58e1be740 1 -- 172.21.15.3:0/959525918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 msgr2=0x562fdf1239b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.972+0000 7ff5857ea640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff5680b6680 0x7ff5680b8b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff578020000 tx=0x7ff578021000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.972+0000 7ff58e1be740 1 --2- 172.21.15.3:0/959525918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf1239b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff578009f80 tx=0x7ff5780118d0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.972+0000 7ff58e1be740 1 -- 172.21.15.3:0/959525918 shutdown_connections 2024-09-21T06:16:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.972+0000 7ff58e1be740 1 --2- 172.21.15.3:0/959525918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf124430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.972+0000 7ff58e1be740 1 --2- 172.21.15.3:0/959525918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf123ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.972+0000 7ff58e1be740 1 --2- 172.21.15.3:0/959525918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf1239b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.972+0000 7ff58e1be740 1 -- 172.21.15.3:0/959525918 >> 172.21.15.3:0/959525918 conn(0x562fded72300 msgr2=0x562fdf1218a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.972+0000 7ff58e1be740 1 -- 172.21.15.3:0/959525918 shutdown_connections 2024-09-21T06:16:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.972+0000 7ff58e1be740 1 -- 172.21.15.3:0/959525918 wait complete. 2024-09-21T06:16:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.973+0000 7ff58e1be740 1 -- start start 2024-09-21T06:16:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.973+0000 7ff58e1be740 1 Processor -- start 2024-09-21T06:16:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.973+0000 7ff58e1be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf12f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.973+0000 7ff58e1be740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf12f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.973+0000 7ff58e1be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf13b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.973+0000 7ff58e1be740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562fdefbfab0 con 0x562fded5d4d0 2024-09-21T06:16:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.973+0000 7ff58e1be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562fdf02d730 con 0x562fded552f0 2024-09-21T06:16:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.973+0000 7ff58e1be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562fdf132130 con 0x562fded7e1b0 2024-09-21T06:16:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.973+0000 7ff58e1be740 10 client.0 _ll_register_callbacks cb 0x562fdf0816b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.973+0000 7ff577fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf12f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff584fe9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf12f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff5857ea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf13b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff577fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf12f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34276/0 (socket says 172.21.15.3:34276) 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff577fff640 1 -- 172.21.15.3:0/1913929996 learned_addr learned my addr 172.21.15.3:0/1913929996 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff577fff640 1 -- 172.21.15.3:0/1913929996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 msgr2=0x562fdf13b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff577fff640 1 --2- 172.21.15.3:0/1913929996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf13b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff577fff640 1 -- 172.21.15.3:0/1913929996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 msgr2=0x562fdf12f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff577fff640 1 --2- 172.21.15.3:0/1913929996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf12f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff577fff640 1 -- 172.21.15.3:0/1913929996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fdede9790 con 0x562fded5d4d0 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff5857ea640 1 --2- 172.21.15.3:0/1913929996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf13b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff584fe9640 1 --2- 172.21.15.3:0/1913929996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf12f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff577fff640 1 --2- 172.21.15.3:0/1913929996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf12f580 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff570012d20 tx=0x7ff57000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3b4158a595d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff55ffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff55ffff640 1 -- 172.21.15.3:0/1913929996 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570012110 con 0x562fded5d4d0 2024-09-21T06:16:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff55ffff640 1 -- 172.21.15.3:0/1913929996 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff570004050 con 0x562fded5d4d0 2024-09-21T06:16:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff58e1be740 1 -- 172.21.15.3:0/1913929996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff578003d50 con 0x562fded5d4d0 2024-09-21T06:16:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff55ffff640 1 -- 172.21.15.3:0/1913929996 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5700122b0 con 0x562fded5d4d0 2024-09-21T06:16:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff58e1be740 10 client.16110 fetch_fsmap learned FSMap version 29 2024-09-21T06:16:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.974+0000 7ff58e1be740 1 -- 172.21.15.3:0/1913929996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562fdee62620 con 0x562fded5d4d0 2024-09-21T06:16:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.975+0000 7ff55ffff640 1 -- 172.21.15.3:0/1913929996 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff578003d50 con 0x562fded5d4d0 2024-09-21T06:16:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.975+0000 7ff55ffff640 1 -- 172.21.15.3:0/1913929996 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff570010430 con 0x562fded5d4d0 2024-09-21T06:16:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.975+0000 7ff58e1be740 10 client.16110 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:16:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.975+0000 7ff58e1be740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.975+0000 7ff58e1be740 20 client.16110 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.975+0000 7ff58e1be740 1 --2- 172.21.15.3:0/1913929996 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562fded3cd80 0x562fdf141220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.975+0000 7ff58e1be740 4 client.16110 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.975+0000 7ff58e1be740 1 -- 172.21.15.3:0/1913929996 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fdf09a640 con 0x562fded3cd80 2024-09-21T06:16:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.975+0000 7ff584fe9640 1 --2- 172.21.15.3:0/1913929996 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562fded3cd80 0x562fdf141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.976+0000 7ff55ffff640 10 client.16110 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.976+0000 7ff584fe9640 1 --2- 172.21.15.3:0/1913929996 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562fded3cd80 0x562fdf141220 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a813cf8438ed41c0 server_cookie=386dbb3af6880d3c in_seq=0 out_seq=0 2024-09-21T06:16:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff55ffff640 1 -- 172.21.15.3:0/1913929996 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562fdf09a640 con 0x562fded3cd80 2024-09-21T06:16:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff55ffff640 10 client.16110 handle_command_reply: tid=0 2024-09-21T06:16:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff58e1be740 1 client.16110 shutdown 2024-09-21T06:16:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff58e1be740 1 -- 172.21.15.3:0/1913929996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 msgr2=0x562fdf12f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff58e1be740 1 --2- 172.21.15.3:0/1913929996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf12f580 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff570012d20 tx=0x7ff57000f790 comp rx=0 tx=0).stop 2024-09-21T06:16:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff58e1be740 1 -- 172.21.15.3:0/1913929996 shutdown_connections 2024-09-21T06:16:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff58e1be740 1 --2- 172.21.15.3:0/1913929996 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562fded3cd80 0x562fdf141220 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff58e1be740 1 --2- 172.21.15.3:0/1913929996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf13b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff55ffff640 0 client.16110 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff58e1be740 1 --2- 172.21.15.3:0/1913929996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf12f580 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff58e1be740 1 --2- 172.21.15.3:0/1913929996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf12f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff58e1be740 1 -- 172.21.15.3:0/1913929996 >> 172.21.15.3:0/1913929996 conn(0x562fded72300 msgr2=0x562fdf1249e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff58e1be740 1 -- 172.21.15.3:0/1913929996 shutdown_connections 2024-09-21T06:16:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff58e1be740 1 -- 172.21.15.3:0/1913929996 wait complete. 2024-09-21T06:16:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.977+0000 7ff58e1be740 20 client.16110 trim_cache size 0 max 16384 2024-09-21T06:16:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff58e1be740 1 Processor -- start 2024-09-21T06:16:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff58e1be740 1 -- start start 2024-09-21T06:16:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff58e1be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded552f0 0x562fdf12f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff58e1be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded5d4d0 0x562fdf1218d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff58e1be740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded7e1b0 0x562fdf13b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff58e1be740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801af620 con 0x562fded7e1b0 2024-09-21T06:16:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff58e1be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801b11c0 con 0x562fded5d4d0 2024-09-21T06:16:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff58e1be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801afcf0 con 0x562fded552f0 2024-09-21T06:16:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff584fe9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded552f0 0x562fdf12f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff5857ea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded7e1b0 0x562fdf13b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff5857ea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded7e1b0 0x562fdf13b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34292/0 (socket says 172.21.15.3:34292) 2024-09-21T06:16:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff584fe9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded552f0 0x562fdf12f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41886/0 (socket says 172.21.15.3:41886) 2024-09-21T06:16:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff5857ea640 1 -- 172.21.15.3:0/2044917307 learned_addr learned my addr 172.21.15.3:0/2044917307 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.992+0000 7ff577fff640 1 --2- 172.21.15.3:0/2044917307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded5d4d0 0x562fdf1218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.993+0000 7ff5857ea640 1 -- 172.21.15.3:0/2044917307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded552f0 msgr2=0x562fdf12f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.993+0000 7ff5857ea640 1 --2- 172.21.15.3:0/2044917307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded552f0 0x562fdf12f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.993+0000 7ff5857ea640 1 -- 172.21.15.3:0/2044917307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded5d4d0 msgr2=0x562fdf1218d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.993+0000 7ff5857ea640 1 --2- 172.21.15.3:0/2044917307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded5d4d0 0x562fdf1218d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.993+0000 7ff5857ea640 1 -- 172.21.15.3:0/2044917307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fdf12d520 con 0x562fded7e1b0 2024-09-21T06:16:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.993+0000 7ff5857ea640 1 --2- 172.21.15.3:0/2044917307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded7e1b0 0x562fdf13b350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff57801b5c0 tx=0x7ff578027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628eec9d206669b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.993+0000 7ff5667fc640 1 -- 172.21.15.3:0/2044917307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578036070 con 0x562fded7e1b0 2024-09-21T06:16:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.993+0000 7ff5667fc640 1 -- 172.21.15.3:0/2044917307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff578003cf0 con 0x562fded7e1b0 2024-09-21T06:16:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.993+0000 7ff5667fc640 1 -- 172.21.15.3:0/2044917307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578029040 con 0x562fded7e1b0 2024-09-21T06:16:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.994+0000 7ff58e1be740 1 -- 172.21.15.3:0/2044917307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded7e1b0 msgr2=0x562fdf13b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.994+0000 7ff58e1be740 1 --2- 172.21.15.3:0/2044917307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded7e1b0 0x562fdf13b350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff57801b5c0 tx=0x7ff578027000 comp rx=0 tx=0).stop 2024-09-21T06:16:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.994+0000 7ff58e1be740 1 -- 172.21.15.3:0/2044917307 shutdown_connections 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.994+0000 7ff58e1be740 1 --2- 172.21.15.3:0/2044917307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded7e1b0 0x562fdf13b350 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.994+0000 7ff58e1be740 1 --2- 172.21.15.3:0/2044917307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded5d4d0 0x562fdf1218d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.994+0000 7ff58e1be740 1 --2- 172.21.15.3:0/2044917307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded552f0 0x562fdf12f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.994+0000 7ff58e1be740 1 -- 172.21.15.3:0/2044917307 >> 172.21.15.3:0/2044917307 conn(0x562fded72300 msgr2=0x562fdf1211c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.994+0000 7ff58e1be740 1 -- 172.21.15.3:0/2044917307 shutdown_connections 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.994+0000 7ff58e1be740 1 -- 172.21.15.3:0/2044917307 wait complete. 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.994+0000 7ff58e1be740 1 -- start start 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.994+0000 7ff58e1be740 1 Processor -- start 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff58e1be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf1329a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff58e1be740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf132f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff58e1be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf1a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff58e1be740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801b0a90 con 0x562fded5d4d0 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff58e1be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801b03c0 con 0x562fded552f0 2024-09-21T06:16:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff58e1be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801ad600 con 0x562fded7e1b0 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff58e1be740 10 client.0 _ll_register_callbacks cb 0x7ff570009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff577fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf132f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff577fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf132f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34296/0 (socket says 172.21.15.3:34296) 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff577fff640 1 -- 172.21.15.3:0/2853355993 learned_addr learned my addr 172.21.15.3:0/2853355993 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff584fe9640 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf1329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff5857ea640 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf1a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff577fff640 1 -- 172.21.15.3:0/2853355993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 msgr2=0x562fdf1a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff577fff640 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf1a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff577fff640 1 -- 172.21.15.3:0/2853355993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 msgr2=0x562fdf1329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff577fff640 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf1329a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff577fff640 1 -- 172.21.15.3:0/2853355993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fdf1aa1a0 con 0x562fded5d4d0 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff584fe9640 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf1329a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff577fff640 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf132f80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff5700108d0 tx=0x7ff570010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456f4bab85e5efe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.995+0000 7ff55ffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff55ffff640 1 -- 172.21.15.3:0/2853355993 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570032060 con 0x562fded5d4d0 2024-09-21T06:16:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff55ffff640 1 -- 172.21.15.3:0/2853355993 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff570021020 con 0x562fded5d4d0 2024-09-21T06:16:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff58e1be740 1 -- 172.21.15.3:0/2853355993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff578003cf0 con 0x562fded5d4d0 2024-09-21T06:16:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff55ffff640 1 -- 172.21.15.3:0/2853355993 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff570029040 con 0x562fded5d4d0 2024-09-21T06:16:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff58e1be740 10 client.16116 fetch_fsmap learned FSMap version 29 2024-09-21T06:16:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff58e1be740 1 -- 172.21.15.3:0/2853355993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562fdf1ab360 con 0x562fded5d4d0 2024-09-21T06:16:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff55ffff640 1 -- 172.21.15.3:0/2853355993 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff578003cf0 con 0x562fded5d4d0 2024-09-21T06:16:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff55ffff640 1 -- 172.21.15.3:0/2853355993 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff570003990 con 0x562fded5d4d0 2024-09-21T06:16:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff58e1be740 10 client.16116 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:16:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff58e1be740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff58e1be740 20 client.16116 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff58e1be740 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562fded121c0 0x562fdf1adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff58e1be740 4 client.16116 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:16:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.996+0000 7ff58e1be740 1 -- 172.21.15.3:0/2853355993 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562fdf0960f0 con 0x562fded121c0 2024-09-21T06:16:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.997+0000 7ff584fe9640 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562fded121c0 0x562fdf1adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.997+0000 7ff55ffff640 10 client.16116 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:37.997+0000 7ff584fe9640 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562fded121c0 0x562fdf1adbb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad2139f40c455b4f server_cookie=11574b5daeccf4f7 in_seq=0 out_seq=0 2024-09-21T06:16:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff55ffff640 1 -- 172.21.15.3:0/2853355993 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562fdf0960f0 con 0x562fded121c0 2024-09-21T06:16:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff55ffff640 10 client.16116 handle_command_reply: tid=0 2024-09-21T06:16:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff58e1be740 1 client.16116 shutdown 2024-09-21T06:16:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff58e1be740 1 -- 172.21.15.3:0/2853355993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 msgr2=0x562fdf132f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff58e1be740 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf132f80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff5700108d0 tx=0x7ff570010cd0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff58e1be740 1 -- 172.21.15.3:0/2853355993 shutdown_connections 2024-09-21T06:16:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff58e1be740 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562fded121c0 0x562fdf1adbb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff58e1be740 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562fded7e1b0 0x562fdf1a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff58e1be740 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fded5d4d0 0x562fdf132f80 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff55ffff640 0 client.16116 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff58e1be740 1 --2- 172.21.15.3:0/2853355993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fded552f0 0x562fdf1329a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff58e1be740 1 -- 172.21.15.3:0/2853355993 >> 172.21.15.3:0/2853355993 conn(0x562fded72300 msgr2=0x562fdf11fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff58e1be740 1 -- 172.21.15.3:0/2853355993 shutdown_connections 2024-09-21T06:16:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff58e1be740 1 -- 172.21.15.3:0/2853355993 wait complete. 2024-09-21T06:16:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.125+0000 7ff58e1be740 20 client.16116 trim_cache size 0 max 16384 2024-09-21T06:16:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:16:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:16:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:16:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:16:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.126+0000 7ff58724d640 1 -- 172.21.15.3:0/1053393290 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff5680b6680 msgr2=0x7ff5680b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.126+0000 7ff58724d640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff5680b6680 0x7ff5680b8b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff578020000 tx=0x7ff578021000 comp rx=0 tx=0).stop 2024-09-21T06:16:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.126+0000 7ff58724d640 1 -- 172.21.15.3:0/1053393290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5801029b0 msgr2=0x7ff58019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.126+0000 7ff58724d640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5801029b0 0x7ff58019a480 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff57c005940 tx=0x7ff57c0040c0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.126+0000 7ff58724d640 1 -- 172.21.15.3:0/1053393290 shutdown_connections 2024-09-21T06:16:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.126+0000 7ff58724d640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff5680b6680 0x7ff5680b8b90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.126+0000 7ff58724d640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff580108c30 0x7ff58019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.126+0000 7ff58724d640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5801082e0 0x7ff58019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.126+0000 7ff58724d640 1 --2- 172.21.15.3:0/1053393290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5801029b0 0x7ff58019a480 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.126+0000 7ff58724d640 1 -- 172.21.15.3:0/1053393290 >> 172.21.15.3:0/1053393290 conn(0x7ff58007a740 msgr2=0x7ff580101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.126+0000 7ff58724d640 1 -- 172.21.15.3:0/1053393290 shutdown_connections 2024-09-21T06:16:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:38.126+0000 7ff58724d640 1 -- 172.21.15.3:0/1053393290 wait complete. 2024-09-21T06:16:38.945 INFO:tasks.workunit.client.0.smithi003.stdout: 1 33943 18.05 MB/sec warmup 11 sec latency 56.234 ms 2024-09-21T06:16:39.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:39 smithi003 ceph-mon[31227]: pgmap v374: 97 pgs: 97 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-09-21T06:16:39.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:39 smithi137 ceph-mon[34464]: pgmap v374: 97 pgs: 97 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-09-21T06:16:39.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:39 smithi204 ceph-mon[34659]: pgmap v374: 97 pgs: 97 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-09-21T06:16:39.945 INFO:tasks.workunit.client.0.smithi003.stdout: 1 36616 17.85 MB/sec warmup 12 sec latency 12.540 ms 2024-09-21T06:16:40.945 INFO:tasks.workunit.client.0.smithi003.stdout: 1 39598 17.80 MB/sec warmup 13 sec latency 9.423 ms 2024-09-21T06:16:41.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:41 smithi003 ceph-mon[31227]: pgmap v375: 97 pgs: 97 active+clean; 15 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-09-21T06:16:41.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:41 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:16:41.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:41 smithi137 ceph-mon[34464]: pgmap v375: 97 pgs: 97 active+clean; 15 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-09-21T06:16:41.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:41 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:16:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:41 smithi204 ceph-mon[34659]: pgmap v375: 97 pgs: 97 active+clean; 15 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-09-21T06:16:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:41 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:16:41.945 INFO:tasks.workunit.client.0.smithi003.stdout: 1 42830 17.87 MB/sec warmup 14 sec latency 15.038 ms 2024-09-21T06:16:42.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:42 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:16:42.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:42 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:16:42.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:42 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:16:42.945 INFO:tasks.workunit.client.0.smithi003.stdout: 1 44998 17.16 MB/sec warmup 15 sec latency 117.047 ms 2024-09-21T06:16:43.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:43 smithi003 ceph-mon[31227]: pgmap v376: 97 pgs: 97 active+clean; 17 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 2.3 MiB/s wr, 357 op/s 2024-09-21T06:16:43.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:43 smithi137 ceph-mon[34464]: pgmap v376: 97 pgs: 97 active+clean; 17 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 2.3 MiB/s wr, 357 op/s 2024-09-21T06:16:43.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:43 smithi204 ceph-mon[34659]: pgmap v376: 97 pgs: 97 active+clean; 17 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 2.3 MiB/s wr, 357 op/s 2024-09-21T06:16:43.945 INFO:tasks.workunit.client.0.smithi003.stdout: 1 47331 16.98 MB/sec warmup 16 sec latency 20.087 ms 2024-09-21T06:16:44.945 INFO:tasks.workunit.client.0.smithi003.stdout: 1 50222 16.98 MB/sec warmup 17 sec latency 10.361 ms 2024-09-21T06:16:45.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:45 smithi003 ceph-mon[31227]: pgmap v377: 97 pgs: 97 active+clean; 17 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.9 MiB/s wr, 304 op/s 2024-09-21T06:16:45.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:45 smithi137 ceph-mon[34464]: pgmap v377: 97 pgs: 97 active+clean; 17 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.9 MiB/s wr, 304 op/s 2024-09-21T06:16:45.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:45 smithi204 ceph-mon[34659]: pgmap v377: 97 pgs: 97 active+clean; 17 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.9 MiB/s wr, 304 op/s 2024-09-21T06:16:45.945 INFO:tasks.workunit.client.0.smithi003.stdout: 1 53530 17.08 MB/sec warmup 18 sec latency 12.828 ms 2024-09-21T06:16:46.945 INFO:tasks.workunit.client.0.smithi003.stdout: 1 56432 16.90 MB/sec warmup 19 sec latency 13.038 ms 2024-09-21T06:16:47.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:47 smithi003 ceph-mon[31227]: pgmap v378: 97 pgs: 97 active+clean; 20 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.1 MiB/s wr, 317 op/s 2024-09-21T06:16:47.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:47 smithi137 ceph-mon[34464]: pgmap v378: 97 pgs: 97 active+clean; 20 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.1 MiB/s wr, 317 op/s 2024-09-21T06:16:47.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:47 smithi204 ceph-mon[34659]: pgmap v378: 97 pgs: 97 active+clean; 20 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.1 MiB/s wr, 317 op/s 2024-09-21T06:16:47.945 INFO:tasks.workunit.client.0.smithi003.stdout: 1 59115 16.69 MB/sec warmup 20 sec latency 18.775 ms 2024-09-21T06:16:48.945 INFO:tasks.workunit.client.0.smithi003.stdout: 1 60695 16.48 MB/sec warmup 21 sec latency 199.871 ms 2024-09-21T06:16:49.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:49 smithi003 ceph-mon[31227]: pgmap v379: 97 pgs: 97 active+clean; 24 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 2.2 MiB/s wr, 373 op/s 2024-09-21T06:16:49.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:49 smithi137 ceph-mon[34464]: pgmap v379: 97 pgs: 97 active+clean; 24 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 2.2 MiB/s wr, 373 op/s 2024-09-21T06:16:49.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:49 smithi204 ceph-mon[34659]: pgmap v379: 97 pgs: 97 active+clean; 24 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 2.2 MiB/s wr, 373 op/s 2024-09-21T06:16:49.945 INFO:tasks.workunit.client.0.smithi003.stdout: 1 62719 16.05 MB/sec warmup 22 sec latency 217.402 ms 2024-09-21T06:16:50.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 64743 15.90 MB/sec warmup 23 sec latency 13.114 ms 2024-09-21T06:16:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.232+0000 7f81b057b640 1 -- 172.21.15.3:0/1452578344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81a81029b0 msgr2=0x7f81a8102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.232+0000 7f81b057b640 1 --2- 172.21.15.3:0/1452578344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81a81029b0 0x7f81a8102dc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8198009ca0 tx=0x7f819802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.232+0000 7f81b057b640 1 -- 172.21.15.3:0/1452578344 shutdown_connections 2024-09-21T06:16:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.232+0000 7f81b057b640 1 --2- 172.21.15.3:0/1452578344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81a81082e0 0x7f81a81086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.232+0000 7f81b057b640 1 --2- 172.21.15.3:0/1452578344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81a8108c30 0x7f81a810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.232+0000 7f81b057b640 1 --2- 172.21.15.3:0/1452578344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81a81029b0 0x7f81a8102dc0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.232+0000 7f81b057b640 1 -- 172.21.15.3:0/1452578344 >> 172.21.15.3:0/1452578344 conn(0x7f81a807a740 msgr2=0x7f81a807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.232+0000 7f81b057b640 1 -- 172.21.15.3:0/1452578344 shutdown_connections 2024-09-21T06:16:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.232+0000 7f81b057b640 1 -- 172.21.15.3:0/1452578344 wait complete. 2024-09-21T06:16:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.233+0000 7f81b057b640 1 Processor -- start 2024-09-21T06:16:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.233+0000 7f81b057b640 1 -- start start 2024-09-21T06:16:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.233+0000 7f81b057b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81a81029b0 0x7f81a8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.233+0000 7f81b057b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81a81082e0 0x7f81a81998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.233+0000 7f81b057b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81a8108c30 0x7f81a819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.233+0000 7f81b057b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81128f0 con 0x7f81a81029b0 2024-09-21T06:16:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.233+0000 7f81b057b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f81a8112770 con 0x7f81a81082e0 2024-09-21T06:16:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.233+0000 7f81b057b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f81a8112a70 con 0x7f81a8108c30 2024-09-21T06:16:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.233+0000 7f81ae317640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81a81029b0 0x7f81a8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.233+0000 7f81ae317640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81a81029b0 0x7f81a8199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35034/0 (socket says 172.21.15.3:35034) 2024-09-21T06:16:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.233+0000 7f81ae317640 1 -- 172.21.15.3:0/1736838870 learned_addr learned my addr 172.21.15.3:0/1736838870 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.233+0000 7f81adb16640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81a81082e0 0x7f81a81998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.233+0000 7f81aeb18640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81a8108c30 0x7f81a819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.234+0000 7f81ae317640 1 -- 172.21.15.3:0/1736838870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81a8108c30 msgr2=0x7f81a819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.234+0000 7f81ae317640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81a8108c30 0x7f81a819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.234+0000 7f81ae317640 1 -- 172.21.15.3:0/1736838870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81a81082e0 msgr2=0x7f81a81998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.234+0000 7f81ae317640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81a81082e0 0x7f81a81998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.234+0000 7f81ae317640 1 -- 172.21.15.3:0/1736838870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a819f260 con 0x7f81a81029b0 2024-09-21T06:16:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.234+0000 7f81adb16640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81a81082e0 0x7f81a81998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.234+0000 7f81ae317640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81a81029b0 0x7f81a8199330 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8198033040 tx=0x7f81980316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.234+0000 7f81a77fe640 1 -- 172.21.15.3:0/1736838870 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198003b70 con 0x7f81a81029b0 2024-09-21T06:16:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.234+0000 7f81b057b640 1 -- 172.21.15.3:0/1736838870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a819f4f0 con 0x7f81a81029b0 2024-09-21T06:16:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.234+0000 7f81a77fe640 1 -- 172.21.15.3:0/1736838870 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8198003d10 con 0x7f81a81029b0 2024-09-21T06:16:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.234+0000 7f81a77fe640 1 -- 172.21.15.3:0/1736838870 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819802f9e0 con 0x7f81a81029b0 2024-09-21T06:16:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.234+0000 7f81b057b640 1 -- 172.21.15.3:0/1736838870 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a81a5ce0 con 0x7f81a81029b0 2024-09-21T06:16:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.237+0000 7f81b75c3740 1 Processor -- start 2024-09-21T06:16:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.237+0000 7f81b75c3740 1 -- start start 2024-09-21T06:16:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.237+0000 7f81a77fe640 1 -- 172.21.15.3:0/1736838870 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f819802fba0 con 0x7f81a81029b0 2024-09-21T06:16:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.237+0000 7f81b75c3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55854135c1b0 0x5585417019b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.237+0000 7f81b75c3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585413332f0 0x558541701ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.237+0000 7f81b75c3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x558541702430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.237+0000 7f81b75c3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5585416766f0 con 0x55854135c1b0 2024-09-21T06:16:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.237+0000 7f81b75c3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558541665b20 con 0x5585413332f0 2024-09-21T06:16:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.237+0000 7f81b75c3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558541669930 con 0x55854133b4d0 2024-09-21T06:16:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81adb16640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x558541702430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81aeb18640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55854135c1b0 0x5585417019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81ae317640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585413332f0 0x558541701ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81adb16640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x558541702430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:37126/0 (socket says 172.21.15.3:37126) 2024-09-21T06:16:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81adb16640 1 -- 172.21.15.3:0/325324628 learned_addr learned my addr 172.21.15.3:0/325324628 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81aeb18640 1 -- 172.21.15.3:0/325324628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 msgr2=0x558541702430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:51.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81aeb18640 1 --2- 172.21.15.3:0/325324628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x558541702430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81aeb18640 1 -- 172.21.15.3:0/325324628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585413332f0 msgr2=0x558541701ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:51.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81aeb18640 1 --2- 172.21.15.3:0/325324628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585413332f0 0x558541701ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81aeb18640 1 -- 172.21.15.3:0/325324628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55854162b530 con 0x55854135c1b0 2024-09-21T06:16:51.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81aeb18640 1 --2- 172.21.15.3:0/325324628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55854135c1b0 0x5585417019b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009b80 tx=0x7f81a0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7baecf2434b5f40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:51.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81a77fe640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f81900b6680 0x7f81900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81a4bfe640 1 -- 172.21.15.3:0/325324628 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0004030 con 0x55854135c1b0 2024-09-21T06:16:51.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81ae317640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f81900b6680 0x7f81900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81a4bfe640 1 -- 172.21.15.3:0/325324628 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a0005020 con 0x55854135c1b0 2024-09-21T06:16:51.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81a4bfe640 1 -- 172.21.15.3:0/325324628 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0004be0 con 0x55854135c1b0 2024-09-21T06:16:51.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.238+0000 7f81a77fe640 1 -- 172.21.15.3:0/1736838870 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f81981088a0 con 0x7f81a81029b0 2024-09-21T06:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.239+0000 7f81b75c3740 1 -- 172.21.15.3:0/325324628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55854135c1b0 msgr2=0x5585417019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.239+0000 7f81ae317640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f81900b6680 0x7f81900b8b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f81981060e0 tx=0x7f819810e550 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.239+0000 7f81b75c3740 1 --2- 172.21.15.3:0/325324628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55854135c1b0 0x5585417019b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009b80 tx=0x7f81a0011810 comp rx=0 tx=0).stop 2024-09-21T06:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.239+0000 7f81b75c3740 1 -- 172.21.15.3:0/325324628 shutdown_connections 2024-09-21T06:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.239+0000 7f81b75c3740 1 --2- 172.21.15.3:0/325324628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x558541702430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.239+0000 7f81b75c3740 1 --2- 172.21.15.3:0/325324628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585413332f0 0x558541701ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.239+0000 7f81b75c3740 1 --2- 172.21.15.3:0/325324628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55854135c1b0 0x5585417019b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.239+0000 7f81b75c3740 1 -- 172.21.15.3:0/325324628 >> 172.21.15.3:0/325324628 conn(0x558541350300 msgr2=0x5585416ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.240+0000 7f81b75c3740 1 -- 172.21.15.3:0/325324628 shutdown_connections 2024-09-21T06:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.240+0000 7f81b75c3740 1 -- 172.21.15.3:0/325324628 wait complete. 2024-09-21T06:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.240+0000 7f81b75c3740 1 -- start start 2024-09-21T06:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.240+0000 7f81b75c3740 1 Processor -- start 2024-09-21T06:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.240+0000 7f81b75c3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585413332f0 0x55854170d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.240+0000 7f81b75c3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x55854170d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.240+0000 7f81b75c3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55854135c1b0 0x558541719420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.240+0000 7f81b75c3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55854159dab0 con 0x55854135c1b0 2024-09-21T06:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.240+0000 7f81b75c3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55854160b730 con 0x5585413332f0 2024-09-21T06:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.240+0000 7f81b75c3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558541710130 con 0x55854133b4d0 2024-09-21T06:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.240+0000 7f81b75c3740 10 client.0 _ll_register_callbacks cb 0x55854165f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f81aeb18640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585413332f0 0x55854170d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f81aeb18640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585413332f0 0x55854170d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:58046/0 (socket says 172.21.15.3:58046) 2024-09-21T06:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f81aeb18640 1 -- 172.21.15.3:0/3344644425 learned_addr learned my addr 172.21.15.3:0/3344644425 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f81adb16640 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x55854170d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f81aeb18640 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55854135c1b0 0x558541719420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f81aeb18640 1 -- 172.21.15.3:0/3344644425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 msgr2=0x55854170d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f81aeb18640 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x55854170d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f81aeb18640 1 -- 172.21.15.3:0/3344644425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55854135c1b0 msgr2=0x558541719420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f81aeb18640 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55854135c1b0 0x558541719420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f81aeb18640 1 -- 172.21.15.3:0/3344644425 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585413c7790 con 0x5585413332f0 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f81adb16640 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x55854170d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f81aeb18640 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585413332f0 0x55854170d030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81a00047e0 tx=0x7f81a0004840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1376de9f746f48d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f818dffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f818dffb640 1 -- 172.21.15.3:0/3344644425 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a002c060 con 0x5585413332f0 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f818dffb640 1 -- 172.21.15.3:0/3344644425 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f81a0004260 con 0x5585413332f0 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.241+0000 7f81b75c3740 1 -- 172.21.15.3:0/3344644425 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81a0005020 con 0x5585413332f0 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.242+0000 7f818dffb640 1 -- 172.21.15.3:0/3344644425 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f81a0005590 con 0x5585413332f0 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.242+0000 7f81b75c3740 10 client.25712 fetch_fsmap learned FSMap version 29 2024-09-21T06:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.242+0000 7f81b75c3740 1 -- 172.21.15.3:0/3344644425 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558541440620 con 0x5585413332f0 2024-09-21T06:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.242+0000 7f818dffb640 1 -- 172.21.15.3:0/3344644425 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a0005020 con 0x5585413332f0 2024-09-21T06:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.242+0000 7f818dffb640 1 -- 172.21.15.3:0/3344644425 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f81a0018070 con 0x5585413332f0 2024-09-21T06:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.242+0000 7f81b75c3740 10 client.25712 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.242+0000 7f81b75c3740 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.242+0000 7f81b75c3740 20 client.25712 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.242+0000 7f81b75c3740 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55854131ad80 0x5585417212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.242+0000 7f81b75c3740 4 client.25712 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.242+0000 7f81b75c3740 1 -- 172.21.15.3:0/3344644425 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558541678640 con 0x55854131ad80 2024-09-21T06:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.242+0000 7f81adb16640 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55854131ad80 0x5585417212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.243+0000 7f818dffb640 10 client.25712 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.243+0000 7f81adb16640 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55854131ad80 0x5585417212c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=627ae3c2da140afb server_cookie=636f940a57039721 in_seq=0 out_seq=0 2024-09-21T06:16:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f818dffb640 1 -- 172.21.15.3:0/3344644425 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558541678640 con 0x55854131ad80 2024-09-21T06:16:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f818dffb640 10 client.25712 handle_command_reply: tid=0 2024-09-21T06:16:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f81b75c3740 1 client.25712 shutdown 2024-09-21T06:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f81b75c3740 1 -- 172.21.15.3:0/3344644425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585413332f0 msgr2=0x55854170d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f81b75c3740 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585413332f0 0x55854170d030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81a00047e0 tx=0x7f81a0004840 comp rx=0 tx=0).stop 2024-09-21T06:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f81b75c3740 1 -- 172.21.15.3:0/3344644425 shutdown_connections 2024-09-21T06:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f81b75c3740 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55854131ad80 0x5585417212c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f81b75c3740 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x55854170d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f81b75c3740 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55854135c1b0 0x558541719420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f818dffb640 0 client.25712 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f81b75c3740 1 --2- 172.21.15.3:0/3344644425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585413332f0 0x55854170d030 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f81b75c3740 1 -- 172.21.15.3:0/3344644425 >> 172.21.15.3:0/3344644425 conn(0x558541350300 msgr2=0x5585417029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f81b75c3740 1 -- 172.21.15.3:0/3344644425 shutdown_connections 2024-09-21T06:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f81b75c3740 1 -- 172.21.15.3:0/3344644425 wait complete. 2024-09-21T06:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.245+0000 7f81b75c3740 20 client.25712 trim_cache size 0 max 16384 2024-09-21T06:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81b75c3740 1 Processor -- start 2024-09-21T06:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81b75c3740 1 -- start start 2024-09-21T06:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81b75c3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854131ad80 0x5585417033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81b75c3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 0x558541719320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81b75c3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55854133b4d0 0x55854170ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81b75c3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81ad340 con 0x5585413332f0 2024-09-21T06:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81b75c3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81aeee0 con 0x55854133b4d0 2024-09-21T06:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81b75c3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81ada10 con 0x55854131ad80 2024-09-21T06:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81adb16640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 0x558541719320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81aeb18640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854131ad80 0x5585417033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81adb16640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 0x558541719320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35066/0 (socket says 172.21.15.3:35066) 2024-09-21T06:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81adb16640 1 -- 172.21.15.3:0/4013080008 learned_addr learned my addr 172.21.15.3:0/4013080008 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81aeb18640 1 --2- 172.21.15.3:0/4013080008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55854133b4d0 0x55854170ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81adb16640 1 -- 172.21.15.3:0/4013080008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854131ad80 msgr2=0x5585417033a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.261+0000 7f81adb16640 1 --2- 172.21.15.3:0/4013080008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854131ad80 0x5585417033a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.262+0000 7f81adb16640 1 -- 172.21.15.3:0/4013080008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55854133b4d0 msgr2=0x55854170ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.262+0000 7f81adb16640 1 --2- 172.21.15.3:0/4013080008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55854133b4d0 0x55854170ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.262+0000 7f81adb16640 1 -- 172.21.15.3:0/4013080008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585416fad50 con 0x5585413332f0 2024-09-21T06:16:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.262+0000 7f81aeb18640 1 --2- 172.21.15.3:0/4013080008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854131ad80 0x5585417033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.262+0000 7f81adb16640 1 --2- 172.21.15.3:0/4013080008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 0x558541719320 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f819c012da0 tx=0x7f819c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7803b30715b0185f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.262+0000 7f818ffff640 1 -- 172.21.15.3:0/4013080008 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c00fcb0 con 0x5585413332f0 2024-09-21T06:16:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.262+0000 7f818ffff640 1 -- 172.21.15.3:0/4013080008 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f819c00fe50 con 0x5585413332f0 2024-09-21T06:16:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.262+0000 7f818ffff640 1 -- 172.21.15.3:0/4013080008 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c01f6c0 con 0x5585413332f0 2024-09-21T06:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.263+0000 7f81b75c3740 1 -- 172.21.15.3:0/4013080008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 msgr2=0x558541719320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.263+0000 7f81b75c3740 1 --2- 172.21.15.3:0/4013080008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 0x558541719320 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f819c012da0 tx=0x7f819c005f90 comp rx=0 tx=0).stop 2024-09-21T06:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.263+0000 7f81b75c3740 1 -- 172.21.15.3:0/4013080008 shutdown_connections 2024-09-21T06:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.263+0000 7f81b75c3740 1 --2- 172.21.15.3:0/4013080008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55854133b4d0 0x55854170ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.263+0000 7f81b75c3740 1 --2- 172.21.15.3:0/4013080008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 0x558541719320 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.263+0000 7f81b75c3740 1 --2- 172.21.15.3:0/4013080008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854131ad80 0x5585417033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.263+0000 7f81b75c3740 1 -- 172.21.15.3:0/4013080008 >> 172.21.15.3:0/4013080008 conn(0x558541350300 msgr2=0x55854170e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.263+0000 7f81b75c3740 1 -- 172.21.15.3:0/4013080008 shutdown_connections 2024-09-21T06:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.263+0000 7f81b75c3740 1 -- 172.21.15.3:0/4013080008 wait complete. 2024-09-21T06:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.263+0000 7f81b75c3740 1 -- start start 2024-09-21T06:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.263+0000 7f81b75c3740 1 Processor -- start 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81b75c3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55854131ad80 0x558541715900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81b75c3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 0x558541712dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81b75c3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x558541713300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81b75c3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81ae7b0 con 0x5585413332f0 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81b75c3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81ae0e0 con 0x55854131ad80 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81b75c3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81ab320 con 0x55854133b4d0 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81b75c3740 10 client.0 _ll_register_callbacks cb 0x7f819810e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81adb16640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 0x558541712dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81adb16640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 0x558541712dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35068/0 (socket says 172.21.15.3:35068) 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81adb16640 1 -- 172.21.15.3:0/3492256055 learned_addr learned my addr 172.21.15.3:0/3492256055 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81aeb18640 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55854131ad80 0x558541715900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81aeb18640 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x558541713300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81adb16640 1 -- 172.21.15.3:0/3492256055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 msgr2=0x558541713300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81adb16640 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x558541713300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81adb16640 1 -- 172.21.15.3:0/3492256055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55854131ad80 msgr2=0x558541715900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81adb16640 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55854131ad80 0x558541715900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81adb16640 1 -- 172.21.15.3:0/3492256055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558541713840 con 0x5585413332f0 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81aeb18640 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x558541713300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f81adb16640 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 0x558541712dc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f819c012940 tx=0x7f819c012af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfda32de9d1da372 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f818dffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.264+0000 7f818dffb640 1 -- 172.21.15.3:0/3492256055 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c010840 con 0x5585413332f0 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f818dffb640 1 -- 172.21.15.3:0/3492256055 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f819c0109e0 con 0x5585413332f0 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f81b75c3740 1 -- 172.21.15.3:0/3492256055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f819c00fe50 con 0x5585413332f0 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f818dffb640 1 -- 172.21.15.3:0/3492256055 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f819c010c00 con 0x5585413332f0 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f81b75c3740 10 client.16134 fetch_fsmap learned FSMap version 29 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f81b75c3740 1 -- 172.21.15.3:0/3492256055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55854178b840 con 0x5585413332f0 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f818dffb640 1 -- 172.21.15.3:0/3492256055 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f819c00fe50 con 0x5585413332f0 2024-09-21T06:16:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f818dffb640 1 -- 172.21.15.3:0/3492256055 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f819c02d070 con 0x5585413332f0 2024-09-21T06:16:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f81b75c3740 10 client.16134 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:16:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f81b75c3740 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f81b75c3740 20 client.16134 populate_metadata read hostname 'smithi003' 2024-09-21T06:16:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f81b75c3740 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55854135c1b0 0x55854178df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f81b75c3740 4 client.16134 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:16:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f81b75c3740 1 -- 172.21.15.3:0/3492256055 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585416740f0 con 0x55854135c1b0 2024-09-21T06:16:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.265+0000 7f81aeb18640 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55854135c1b0 0x55854178df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.266+0000 7f818dffb640 10 client.16134 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.266+0000 7f81aeb18640 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55854135c1b0 0x55854178df50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99be0f4bb2c6b937 server_cookie=be2fe5ca7c71dc4f in_seq=0 out_seq=0 2024-09-21T06:16:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f818dffb640 1 -- 172.21.15.3:0/3492256055 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585416740f0 con 0x55854135c1b0 2024-09-21T06:16:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f818dffb640 10 client.16134 handle_command_reply: tid=0 2024-09-21T06:16:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f81b75c3740 1 client.16134 shutdown 2024-09-21T06:16:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f81b75c3740 1 -- 172.21.15.3:0/3492256055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 msgr2=0x558541712dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f81b75c3740 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 0x558541712dc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f819c012940 tx=0x7f819c012af0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f81b75c3740 1 -- 172.21.15.3:0/3492256055 shutdown_connections 2024-09-21T06:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f81b75c3740 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55854135c1b0 0x55854178df50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f81b75c3740 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55854133b4d0 0x558541713300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f81b75c3740 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5585413332f0 0x558541712dc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f818dffb640 0 client.16134 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f81b75c3740 1 --2- 172.21.15.3:0/3492256055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55854131ad80 0x558541715900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f81b75c3740 1 -- 172.21.15.3:0/3492256055 >> 172.21.15.3:0/3492256055 conn(0x558541350300 msgr2=0x558541703150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f81b75c3740 1 -- 172.21.15.3:0/3492256055 shutdown_connections 2024-09-21T06:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f81b75c3740 1 -- 172.21.15.3:0/3492256055 wait complete. 2024-09-21T06:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.327+0000 7f81b75c3740 20 client.16134 trim_cache size 0 max 16384 2024-09-21T06:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:16:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:16:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:16:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:16:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.328+0000 7f81b057b640 1 -- 172.21.15.3:0/1736838870 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f81900b6680 msgr2=0x7f81900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.328+0000 7f81b057b640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f81900b6680 0x7f81900b8b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f81981060e0 tx=0x7f819810e550 comp rx=0 tx=0).stop 2024-09-21T06:16:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.328+0000 7f81b057b640 1 -- 172.21.15.3:0/1736838870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81a81029b0 msgr2=0x7f81a8199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.328+0000 7f81b057b640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81a81029b0 0x7f81a8199330 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8198033040 tx=0x7f81980316f0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.328+0000 7f81b057b640 1 -- 172.21.15.3:0/1736838870 shutdown_connections 2024-09-21T06:16:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.328+0000 7f81b057b640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f81900b6680 0x7f81900b8b90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.328+0000 7f81b057b640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81a8108c30 0x7f81a819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.328+0000 7f81b057b640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81a81082e0 0x7f81a81998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.328+0000 7f81b057b640 1 --2- 172.21.15.3:0/1736838870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81a81029b0 0x7f81a8199330 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.328+0000 7f81b057b640 1 -- 172.21.15.3:0/1736838870 >> 172.21.15.3:0/1736838870 conn(0x7f81a807a740 msgr2=0x7f81a81063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.328+0000 7f81b057b640 1 -- 172.21.15.3:0/1736838870 shutdown_connections 2024-09-21T06:16:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:16:51.328+0000 7f81b057b640 1 -- 172.21.15.3:0/1736838870 wait complete. 2024-09-21T06:16:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:51 smithi003 ceph-mon[31227]: pgmap v380: 97 pgs: 97 active+clean; 59 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 5.1 MiB/s wr, 406 op/s 2024-09-21T06:16:51.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:51 smithi137 ceph-mon[34464]: pgmap v380: 97 pgs: 97 active+clean; 59 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 5.1 MiB/s wr, 406 op/s 2024-09-21T06:16:51.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:51 smithi204 ceph-mon[34659]: pgmap v380: 97 pgs: 97 active+clean; 59 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 5.1 MiB/s wr, 406 op/s 2024-09-21T06:16:51.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 67745 16.00 MB/sec warmup 24 sec latency 27.584 ms 2024-09-21T06:16:52.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 70915 16.07 MB/sec warmup 25 sec latency 12.087 ms 2024-09-21T06:16:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:53 smithi003 ceph-mon[31227]: pgmap v381: 97 pgs: 97 active+clean; 65 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.8 MiB/s wr, 414 op/s 2024-09-21T06:16:53.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:53 smithi137 ceph-mon[34464]: pgmap v381: 97 pgs: 97 active+clean; 65 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.8 MiB/s wr, 414 op/s 2024-09-21T06:16:53.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:53 smithi204 ceph-mon[34659]: pgmap v381: 97 pgs: 97 active+clean; 65 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.8 MiB/s wr, 414 op/s 2024-09-21T06:16:53.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 72920 15.73 MB/sec warmup 26 sec latency 14.813 ms 2024-09-21T06:16:54.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 75205 15.66 MB/sec warmup 27 sec latency 11.612 ms 2024-09-21T06:16:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:55 smithi003 ceph-mon[31227]: pgmap v382: 97 pgs: 97 active+clean; 65 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.6 MiB/s wr, 317 op/s 2024-09-21T06:16:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:55 smithi137 ceph-mon[34464]: pgmap v382: 97 pgs: 97 active+clean; 65 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.6 MiB/s wr, 317 op/s 2024-09-21T06:16:55.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:55 smithi204 ceph-mon[34659]: pgmap v382: 97 pgs: 97 active+clean; 65 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.6 MiB/s wr, 317 op/s 2024-09-21T06:16:55.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 78374 15.76 MB/sec warmup 28 sec latency 12.984 ms 2024-09-21T06:16:56.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 81065 15.66 MB/sec warmup 29 sec latency 12.371 ms 2024-09-21T06:16:57.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:57 smithi003 ceph-mon[31227]: pgmap v383: 97 pgs: 97 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.1 MiB/s wr, 449 op/s 2024-09-21T06:16:57.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:57 smithi137 ceph-mon[34464]: pgmap v383: 97 pgs: 97 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.1 MiB/s wr, 449 op/s 2024-09-21T06:16:57.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:57 smithi204 ceph-mon[34659]: pgmap v383: 97 pgs: 97 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.1 MiB/s wr, 449 op/s 2024-09-21T06:16:57.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 82856 15.43 MB/sec warmup 30 sec latency 42.093 ms 2024-09-21T06:16:58.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 85636 15.48 MB/sec warmup 31 sec latency 10.336 ms 2024-09-21T06:16:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:16:59 smithi003 ceph-mon[31227]: pgmap v384: 97 pgs: 97 active+clean; 33 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.8 MiB/s wr, 431 op/s 2024-09-21T06:16:59.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:16:59 smithi137 ceph-mon[34464]: pgmap v384: 97 pgs: 97 active+clean; 33 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.8 MiB/s wr, 431 op/s 2024-09-21T06:16:59.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:16:59 smithi204 ceph-mon[34659]: pgmap v384: 97 pgs: 97 active+clean; 33 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.8 MiB/s wr, 431 op/s 2024-09-21T06:16:59.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 88855 15.55 MB/sec warmup 32 sec latency 15.333 ms 2024-09-21T06:17:00.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 89784 15.18 MB/sec warmup 33 sec latency 24.557 ms 2024-09-21T06:17:01.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:01 smithi003 ceph-mon[31227]: pgmap v385: 97 pgs: 97 active+clean; 35 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.7 MiB/s wr, 385 op/s 2024-09-21T06:17:01.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:01 smithi137 ceph-mon[34464]: pgmap v385: 97 pgs: 97 active+clean; 35 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.7 MiB/s wr, 385 op/s 2024-09-21T06:17:01.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:01 smithi204 ceph-mon[34659]: pgmap v385: 97 pgs: 97 active+clean; 35 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.7 MiB/s wr, 385 op/s 2024-09-21T06:17:01.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 92642 15.24 MB/sec warmup 34 sec latency 10.563 ms 2024-09-21T06:17:02.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 95760 15.30 MB/sec warmup 35 sec latency 15.603 ms 2024-09-21T06:17:03.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:03 smithi003 ceph-mon[31227]: pgmap v386: 97 pgs: 97 active+clean; 38 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 404 op/s 2024-09-21T06:17:03.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:03 smithi137 ceph-mon[34464]: pgmap v386: 97 pgs: 97 active+clean; 38 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 404 op/s 2024-09-21T06:17:03.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:03 smithi204 ceph-mon[34659]: pgmap v386: 97 pgs: 97 active+clean; 38 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 404 op/s 2024-09-21T06:17:03.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 97592 15.07 MB/sec warmup 36 sec latency 15.209 ms 2024-09-21T06:17:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.428+0000 7f3e36c5c640 1 -- 172.21.15.3:0/2454887039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e30109400 msgr2=0x7f3e30109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.428+0000 7f3e36c5c640 1 --2- 172.21.15.3:0/2454887039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e30109400 0x7f3e30109810 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009900 tx=0x7f3e1c02a410 comp rx=0 tx=0).stop 2024-09-21T06:17:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.428+0000 7f3e36c5c640 1 -- 172.21.15.3:0/2454887039 shutdown_connections 2024-09-21T06:17:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.428+0000 7f3e36c5c640 1 --2- 172.21.15.3:0/2454887039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e30109d50 0x7f3e30110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.428+0000 7f3e36c5c640 1 --2- 172.21.15.3:0/2454887039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e30109400 0x7f3e30109810 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.428+0000 7f3e36c5c640 1 --2- 172.21.15.3:0/2454887039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e30103ad0 0x7f3e30103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.428+0000 7f3e36c5c640 1 -- 172.21.15.3:0/2454887039 >> 172.21.15.3:0/2454887039 conn(0x7f3e300ff510 msgr2=0x7f3e30101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.428+0000 7f3e36c5c640 1 -- 172.21.15.3:0/2454887039 shutdown_connections 2024-09-21T06:17:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.428+0000 7f3e36c5c640 1 -- 172.21.15.3:0/2454887039 wait complete. 2024-09-21T06:17:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.429+0000 7f3e36c5c640 1 Processor -- start 2024-09-21T06:17:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.429+0000 7f3e36c5c640 1 -- start start 2024-09-21T06:17:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e36c5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e30103ad0 0x7f3e3019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e36c5c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e30109400 0x7f3e3019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e36c5c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e30109d50 0x7f3e3019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e36c5c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e30113a10 con 0x7f3e30109400 2024-09-21T06:17:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e36c5c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3e30113890 con 0x7f3e30103ad0 2024-09-21T06:17:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e36c5c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e30113b90 con 0x7f3e30109d50 2024-09-21T06:17:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e30109400 0x7f3e3019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e30109400 0x7f3e3019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53944/0 (socket says 172.21.15.3:53944) 2024-09-21T06:17:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e349f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e30103ad0 0x7f3e3019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e2ffff640 1 -- 172.21.15.3:0/1005513693 learned_addr learned my addr 172.21.15.3:0/1005513693 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e351f9640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e30109d50 0x7f3e3019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e349f8640 1 -- 172.21.15.3:0/1005513693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e30109d50 msgr2=0x7f3e3019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e349f8640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e30109d50 0x7f3e3019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e349f8640 1 -- 172.21.15.3:0/1005513693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e30109400 msgr2=0x7f3e3019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e349f8640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e30109400 0x7f3e3019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e349f8640 1 -- 172.21.15.3:0/1005513693 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e301a0380 con 0x7f3e30103ad0 2024-09-21T06:17:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e2ffff640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e30109400 0x7f3e3019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.430+0000 7f3e349f8640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e30103ad0 0x7f3e3019a450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400e9f0 tx=0x7f3e2400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.431+0000 7f3e2dffb640 1 -- 172.21.15.3:0/1005513693 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2400ccc0 con 0x7f3e30103ad0 2024-09-21T06:17:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.431+0000 7f3e2dffb640 1 -- 172.21.15.3:0/1005513693 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e240108e0 con 0x7f3e30103ad0 2024-09-21T06:17:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.431+0000 7f3e36c5c640 1 -- 172.21.15.3:0/1005513693 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e301a0670 con 0x7f3e30103ad0 2024-09-21T06:17:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.431+0000 7f3e2dffb640 1 -- 172.21.15.3:0/1005513693 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24018880 con 0x7f3e30103ad0 2024-09-21T06:17:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.431+0000 7f3e36c5c640 1 -- 172.21.15.3:0/1005513693 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e301a6e00 con 0x7f3e30103ad0 2024-09-21T06:17:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.433+0000 7f3e3dbc9740 1 Processor -- start 2024-09-21T06:17:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.433+0000 7f3e3dbc9740 1 -- start start 2024-09-21T06:17:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.433+0000 7f3e2dffb640 1 -- 172.21.15.3:0/1005513693 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f3e24018a20 con 0x7f3e30103ad0 2024-09-21T06:17:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.433+0000 7f3e3dbc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f296d1b0 0x5608f2d129b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.433+0000 7f3e3dbc9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f29442f0 0x5608f2d12ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e3dbc9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d13430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e3dbc9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5608f2c876f0 con 0x5608f294c4d0 2024-09-21T06:17:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e3dbc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5608f2c76b20 con 0x5608f296d1b0 2024-09-21T06:17:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e3dbc9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5608f2c7a930 con 0x5608f29442f0 2024-09-21T06:17:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e349f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f29442f0 0x5608f2d12ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e351f9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f296d1b0 0x5608f2d129b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d13430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e351f9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f296d1b0 0x5608f2d129b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:52216/0 (socket says 172.21.15.3:52216) 2024-09-21T06:17:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e349f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f29442f0 0x5608f2d12ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:56328/0 (socket says 172.21.15.3:56328) 2024-09-21T06:17:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d13430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53954/0 (socket says 172.21.15.3:53954) 2024-09-21T06:17:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e351f9640 1 -- 172.21.15.3:0/2296553163 learned_addr learned my addr 172.21.15.3:0/2296553163 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e351f9640 1 -- 172.21.15.3:0/2296553163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f29442f0 msgr2=0x5608f2d12ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e351f9640 1 --2- 172.21.15.3:0/2296553163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f29442f0 0x5608f2d12ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e351f9640 1 -- 172.21.15.3:0/2296553163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 msgr2=0x5608f2d13430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e351f9640 1 --2- 172.21.15.3:0/2296553163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d13430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e351f9640 1 -- 172.21.15.3:0/2296553163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f2c3c530 con 0x5608f296d1b0 2024-09-21T06:17:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e351f9640 1 --2- 172.21.15.3:0/2296553163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f296d1b0 0x5608f2d129b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e28011440 tx=0x7f3e28011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b37ba105a5252162 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.434+0000 7f3e2dffb640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3e200b6700 0x7f3e200b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.435+0000 7f3e349f8640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3e200b6700 0x7f3e200b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.435+0000 7f3e1affd640 1 -- 172.21.15.3:0/2296553163 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28004d20 con 0x5608f296d1b0 2024-09-21T06:17:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.435+0000 7f3e1affd640 1 -- 172.21.15.3:0/2296553163 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e28004050 con 0x5608f296d1b0 2024-09-21T06:17:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.435+0000 7f3e1affd640 1 -- 172.21.15.3:0/2296553163 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2800f7a0 con 0x5608f296d1b0 2024-09-21T06:17:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.435+0000 7f3e2dffb640 1 -- 172.21.15.3:0/1005513693 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f3e240e79b0 con 0x7f3e30103ad0 2024-09-21T06:17:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.435+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/2296553163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f296d1b0 msgr2=0x5608f2d129b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.435+0000 7f3e349f8640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3e200b6700 0x7f3e200b8c10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3e240e51f0 tx=0x7f3e240eb500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.435+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/2296553163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f296d1b0 0x5608f2d129b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e28011440 tx=0x7f3e28011900 comp rx=0 tx=0).stop 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.435+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/2296553163 shutdown_connections 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.435+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/2296553163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f29442f0 0x5608f2d12ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.435+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/2296553163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d13430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.435+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/2296553163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f296d1b0 0x5608f2d129b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.435+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/2296553163 >> 172.21.15.3:0/2296553163 conn(0x5608f2961300 msgr2=0x5608f2d108a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.436+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/2296553163 shutdown_connections 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.436+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/2296553163 wait complete. 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.436+0000 7f3e3dbc9740 1 -- start start 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.436+0000 7f3e3dbc9740 1 Processor -- start 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.436+0000 7f3e3dbc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 0x5608f2d1e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.436+0000 7f3e3dbc9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d1e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.436+0000 7f3e3dbc9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 0x5608f2d2a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.436+0000 7f3e3dbc9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5608f2baeab0 con 0x5608f294c4d0 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e3dbc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5608f2c1c730 con 0x5608f29442f0 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e3dbc9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5608f2d21130 con 0x5608f296d1b0 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e3dbc9740 10 client.0 _ll_register_callbacks cb 0x5608f2c706b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d1e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d1e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53962/0 (socket says 172.21.15.3:53962) 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e351f9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 0x5608f2d1e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e2ffff640 1 -- 172.21.15.3:0/810461204 learned_addr learned my addr 172.21.15.3:0/810461204 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e351f9640 1 --2- 172.21.15.3:0/810461204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 0x5608f2d2a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e2ffff640 1 -- 172.21.15.3:0/810461204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 msgr2=0x5608f2d2a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e2ffff640 1 --2- 172.21.15.3:0/810461204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 0x5608f2d2a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e2ffff640 1 -- 172.21.15.3:0/810461204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 msgr2=0x5608f2d1e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e2ffff640 1 --2- 172.21.15.3:0/810461204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 0x5608f2d1e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e2ffff640 1 -- 172.21.15.3:0/810461204 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f29d8790 con 0x5608f294c4d0 2024-09-21T06:17:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e2ffff640 1 --2- 172.21.15.3:0/810461204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d1e5f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c006010 tx=0x7f3e1c031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da76b8b7e6f41f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e0ffff640 1 -- 172.21.15.3:0/810461204 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c035040 con 0x5608f294c4d0 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e0ffff640 1 -- 172.21.15.3:0/810461204 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f3e1c004450 con 0x5608f294c4d0 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.437+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/810461204 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e28004050 con 0x5608f294c4d0 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.438+0000 7f3e0ffff640 1 -- 172.21.15.3:0/810461204 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e1c0318f0 con 0x5608f294c4d0 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.438+0000 7f3e3dbc9740 10 client.16146 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.438+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/810461204 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5608f2a51620 con 0x5608f294c4d0 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.438+0000 7f3e0ffff640 1 -- 172.21.15.3:0/810461204 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e28004050 con 0x5608f294c4d0 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.438+0000 7f3e0ffff640 1 -- 172.21.15.3:0/810461204 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3e1c03d830 con 0x5608f294c4d0 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.438+0000 7f3e3dbc9740 10 client.16146 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.438+0000 7f3e3dbc9740 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.438+0000 7f3e3dbc9740 20 client.16146 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.438+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/810461204 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5608f292bd80 0x5608f2d30280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.438+0000 7f3e3dbc9740 4 client.16146 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.438+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/810461204 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608f2c89640 con 0x5608f292bd80 2024-09-21T06:17:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.438+0000 7f3e351f9640 1 --2- 172.21.15.3:0/810461204 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5608f292bd80 0x5608f2d30280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.439+0000 7f3e0ffff640 10 client.16146 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.439+0000 7f3e351f9640 1 --2- 172.21.15.3:0/810461204 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5608f292bd80 0x5608f2d30280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e62b447aac903924 server_cookie=2b52dfb26e6db0d6 in_seq=0 out_seq=0 2024-09-21T06:17:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.440+0000 7f3e0ffff640 1 -- 172.21.15.3:0/810461204 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5608f2c89640 con 0x5608f292bd80 2024-09-21T06:17:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.440+0000 7f3e0ffff640 10 client.16146 handle_command_reply: tid=0 2024-09-21T06:17:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.440+0000 7f3e3dbc9740 1 client.16146 shutdown 2024-09-21T06:17:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.440+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/810461204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 msgr2=0x5608f2d1e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.440+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/810461204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d1e5f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c006010 tx=0x7f3e1c031740 comp rx=0 tx=0).stop 2024-09-21T06:17:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.440+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/810461204 shutdown_connections 2024-09-21T06:17:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.440+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/810461204 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5608f292bd80 0x5608f2d30280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.440+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/810461204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 0x5608f2d2a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.440+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/810461204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d1e5f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.440+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/810461204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 0x5608f2d1e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.440+0000 7f3e0ffff640 0 client.16146 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.440+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/810461204 >> 172.21.15.3:0/810461204 conn(0x5608f2961300 msgr2=0x5608f2d139e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.441+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/810461204 shutdown_connections 2024-09-21T06:17:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.441+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/810461204 wait complete. 2024-09-21T06:17:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.441+0000 7f3e3dbc9740 20 client.16146 trim_cache size 0 max 16384 2024-09-21T06:17:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.457+0000 7f3e3dbc9740 1 Processor -- start 2024-09-21T06:17:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.457+0000 7f3e3dbc9740 1 -- start start 2024-09-21T06:17:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.457+0000 7f3e3dbc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 0x5608f2d143a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.457+0000 7f3e3dbc9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d1b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.457+0000 7f3e3dbc9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 0x5608f2d1b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.457+0000 7f3e3dbc9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e301af6a0 con 0x5608f294c4d0 2024-09-21T06:17:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.457+0000 7f3e3dbc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3e301b1240 con 0x5608f29442f0 2024-09-21T06:17:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.457+0000 7f3e3dbc9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e301afd70 con 0x5608f296d1b0 2024-09-21T06:17:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.457+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d1b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.457+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d1b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53978/0 (socket says 172.21.15.3:53978) 2024-09-21T06:17:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.457+0000 7f3e2ffff640 1 -- 172.21.15.3:0/852482164 learned_addr learned my addr 172.21.15.3:0/852482164 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.457+0000 7f3e351f9640 1 --2- 172.21.15.3:0/852482164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 0x5608f2d143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.458+0000 7f3e351f9640 1 --2- 172.21.15.3:0/852482164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 0x5608f2d1b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.458+0000 7f3e2ffff640 1 -- 172.21.15.3:0/852482164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 msgr2=0x5608f2d1b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.458+0000 7f3e2ffff640 1 --2- 172.21.15.3:0/852482164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 0x5608f2d1b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.458+0000 7f3e2ffff640 1 -- 172.21.15.3:0/852482164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 msgr2=0x5608f2d143a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.458+0000 7f3e2ffff640 1 --2- 172.21.15.3:0/852482164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 0x5608f2d143a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.458+0000 7f3e2ffff640 1 -- 172.21.15.3:0/852482164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f2d151c0 con 0x5608f294c4d0 2024-09-21T06:17:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.458+0000 7f3e2ffff640 1 --2- 172.21.15.3:0/852482164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d1b280 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c03dfd0 tx=0x7f3e1c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653a4d12984c6eed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.458+0000 7f3e1a7fc640 1 -- 172.21.15.3:0/852482164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c054020 con 0x5608f294c4d0 2024-09-21T06:17:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.458+0000 7f3e1a7fc640 1 -- 172.21.15.3:0/852482164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e1c004250 con 0x5608f294c4d0 2024-09-21T06:17:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.458+0000 7f3e1a7fc640 1 -- 172.21.15.3:0/852482164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c035030 con 0x5608f294c4d0 2024-09-21T06:17:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.459+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/852482164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 msgr2=0x5608f2d1b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.459+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/852482164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d1b280 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c03dfd0 tx=0x7f3e1c005780 comp rx=0 tx=0).stop 2024-09-21T06:17:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.459+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/852482164 shutdown_connections 2024-09-21T06:17:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.459+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/852482164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 0x5608f2d1b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.459+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/852482164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d1b280 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.459+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/852482164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 0x5608f2d143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.459+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/852482164 >> 172.21.15.3:0/852482164 conn(0x5608f2961300 msgr2=0x5608f2d29310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.459+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/852482164 shutdown_connections 2024-09-21T06:17:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.459+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/852482164 wait complete. 2024-09-21T06:17:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.459+0000 7f3e3dbc9740 1 -- start start 2024-09-21T06:17:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e3dbc9740 1 Processor -- start 2024-09-21T06:17:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e3dbc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 0x5608f2d242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e3dbc9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e3dbc9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 0x5608f2d24d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e3dbc9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e301b0b10 con 0x5608f294c4d0 2024-09-21T06:17:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e3dbc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3e301b0440 con 0x5608f29442f0 2024-09-21T06:17:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e3dbc9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e301ad680 con 0x5608f296d1b0 2024-09-21T06:17:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e3dbc9740 10 client.0 _ll_register_callbacks cb 0x7f3e1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e351f9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 0x5608f2d242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e351f9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 0x5608f2d242b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:52240/0 (socket says 172.21.15.3:52240) 2024-09-21T06:17:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e351f9640 1 -- 172.21.15.3:0/3615349962 learned_addr learned my addr 172.21.15.3:0/3615349962 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e351f9640 1 --2- 172.21.15.3:0/3615349962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 0x5608f2d24d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e2ffff640 1 -- 172.21.15.3:0/3615349962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 msgr2=0x5608f2d24d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e2ffff640 1 --2- 172.21.15.3:0/3615349962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 0x5608f2d24d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e2ffff640 1 -- 172.21.15.3:0/3615349962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 msgr2=0x5608f2d242b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.460+0000 7f3e2ffff640 1 --2- 172.21.15.3:0/3615349962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 0x5608f2d242b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.461+0000 7f3e2ffff640 1 -- 172.21.15.3:0/3615349962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f2d99c50 con 0x5608f294c4d0 2024-09-21T06:17:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.461+0000 7f3e2ffff640 1 --2- 172.21.15.3:0/3615349962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d247f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c03ddc0 tx=0x7f3e1c031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd8470058dc851b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.461+0000 7f3e0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.461+0000 7f3e0ffff640 1 -- 172.21.15.3:0/3615349962 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c054030 con 0x5608f294c4d0 2024-09-21T06:17:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.461+0000 7f3e0ffff640 1 -- 172.21.15.3:0/3615349962 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f3e1c03d650 con 0x5608f294c4d0 2024-09-21T06:17:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.461+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/3615349962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e1c004250 con 0x5608f294c4d0 2024-09-21T06:17:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.461+0000 7f3e0ffff640 1 -- 172.21.15.3:0/3615349962 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e1c031e20 con 0x5608f294c4d0 2024-09-21T06:17:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.461+0000 7f3e3dbc9740 10 client.16152 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.461+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/3615349962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5608f2d9ae20 con 0x5608f294c4d0 2024-09-21T06:17:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.461+0000 7f3e0ffff640 1 -- 172.21.15.3:0/3615349962 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e1c004250 con 0x5608f294c4d0 2024-09-21T06:17:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.462+0000 7f3e0ffff640 1 -- 172.21.15.3:0/3615349962 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3e1c0320d0 con 0x5608f294c4d0 2024-09-21T06:17:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.462+0000 7f3e3dbc9740 10 client.16152 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.462+0000 7f3e3dbc9740 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.462+0000 7f3e3dbc9740 20 client.16152 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.462+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/3615349962 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5608f29011c0 0x5608f2d9d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.462+0000 7f3e3dbc9740 4 client.16152 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:17:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.462+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/3615349962 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608f2c850f0 con 0x5608f29011c0 2024-09-21T06:17:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.462+0000 7f3e351f9640 1 --2- 172.21.15.3:0/3615349962 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5608f29011c0 0x5608f2d9d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.462+0000 7f3e351f9640 1 --2- 172.21.15.3:0/3615349962 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5608f29011c0 0x5608f2d9d4c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c907d8092234b3 server_cookie=41990be8c756b497 in_seq=0 out_seq=0 2024-09-21T06:17:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.462+0000 7f3e0ffff640 10 client.16152 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.582+0000 7f3e0ffff640 1 -- 172.21.15.3:0/3615349962 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608f2c850f0 con 0x5608f29011c0 2024-09-21T06:17:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.582+0000 7f3e0ffff640 10 client.16152 handle_command_reply: tid=0 2024-09-21T06:17:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.582+0000 7f3e3dbc9740 1 client.16152 shutdown 2024-09-21T06:17:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.583+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/3615349962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 msgr2=0x5608f2d247f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.583+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/3615349962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d247f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c03ddc0 tx=0x7f3e1c031740 comp rx=0 tx=0).stop 2024-09-21T06:17:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.583+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/3615349962 shutdown_connections 2024-09-21T06:17:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.583+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/3615349962 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5608f29011c0 0x5608f2d9d4c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.583+0000 7f3e0ffff640 0 client.16152 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.583+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/3615349962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608f296d1b0 0x5608f2d24d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.583+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/3615349962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608f294c4d0 0x5608f2d247f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.583+0000 7f3e3dbc9740 1 --2- 172.21.15.3:0/3615349962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608f29442f0 0x5608f2d242b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.583+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/3615349962 >> 172.21.15.3:0/3615349962 conn(0x5608f2961300 msgr2=0x5608f2d14170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.583+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/3615349962 shutdown_connections 2024-09-21T06:17:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.583+0000 7f3e3dbc9740 1 -- 172.21.15.3:0/3615349962 wait complete. 2024-09-21T06:17:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.583+0000 7f3e3dbc9740 20 client.16152 trim_cache size 0 max 16384 2024-09-21T06:17:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:17:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:17:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:17:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:17:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.584+0000 7f3e36c5c640 1 -- 172.21.15.3:0/1005513693 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3e200b6700 msgr2=0x7f3e200b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.584+0000 7f3e36c5c640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3e200b6700 0x7f3e200b8c10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3e240e51f0 tx=0x7f3e240eb500 comp rx=0 tx=0).stop 2024-09-21T06:17:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.584+0000 7f3e36c5c640 1 -- 172.21.15.3:0/1005513693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e30103ad0 msgr2=0x7f3e3019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.584+0000 7f3e36c5c640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e30103ad0 0x7f3e3019a450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400e9f0 tx=0x7f3e2400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.584+0000 7f3e36c5c640 1 -- 172.21.15.3:0/1005513693 shutdown_connections 2024-09-21T06:17:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.584+0000 7f3e36c5c640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3e200b6700 0x7f3e200b8c10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.584+0000 7f3e36c5c640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e30109d50 0x7f3e3019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.584+0000 7f3e36c5c640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e30109400 0x7f3e3019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.584+0000 7f3e36c5c640 1 --2- 172.21.15.3:0/1005513693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e30103ad0 0x7f3e3019a450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.584+0000 7f3e36c5c640 1 -- 172.21.15.3:0/1005513693 >> 172.21.15.3:0/1005513693 conn(0x7f3e300ff510 msgr2=0x7f3e30101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.584+0000 7f3e36c5c640 1 -- 172.21.15.3:0/1005513693 shutdown_connections 2024-09-21T06:17:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:04.584+0000 7f3e36c5c640 1 -- 172.21.15.3:0/1005513693 wait complete. 2024-09-21T06:17:04.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 99810 15.04 MB/sec warmup 37 sec latency 61.343 ms 2024-09-21T06:17:05.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:05 smithi003 ceph-mon[31227]: pgmap v387: 97 pgs: 97 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 276 op/s 2024-09-21T06:17:05.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:05 smithi137 ceph-mon[34464]: pgmap v387: 97 pgs: 97 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 276 op/s 2024-09-21T06:17:05.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:05 smithi204 ceph-mon[34659]: pgmap v387: 97 pgs: 97 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 276 op/s 2024-09-21T06:17:05.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 102992 15.11 MB/sec warmup 38 sec latency 16.493 ms 2024-09-21T06:17:06.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 105111 14.91 MB/sec warmup 39 sec latency 12.248 ms 2024-09-21T06:17:07.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:17:07.586 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:07 smithi003 ceph-mon[31227]: pgmap v388: 97 pgs: 97 active+clean; 40 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 391 op/s 2024-09-21T06:17:07.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:07 smithi137 ceph-mon[34464]: pgmap v388: 97 pgs: 97 active+clean; 40 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 391 op/s 2024-09-21T06:17:07.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:07 smithi204 ceph-mon[34659]: pgmap v388: 97 pgs: 97 active+clean; 40 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 391 op/s 2024-09-21T06:17:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:07.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 107112 14.87 MB/sec warmup 40 sec latency 14.453 ms 2024-09-21T06:17:08.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.358+0000 7f52b37db640 1 -- 172.21.15.3:0/788925163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ac110200 msgr2=0x7f52ac112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.358+0000 7f52b37db640 1 --2- 172.21.15.3:0/788925163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ac110200 0x7f52ac112620 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009ca0 tx=0x7f52a402a950 comp rx=0 tx=0).stop 2024-09-21T06:17:08.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.358+0000 7f52b37db640 1 -- 172.21.15.3:0/788925163 shutdown_connections 2024-09-21T06:17:08.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.358+0000 7f52b37db640 1 --2- 172.21.15.3:0/788925163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ac110200 0x7f52ac112620 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.358+0000 7f52b37db640 1 --2- 172.21.15.3:0/788925163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52ac102fa0 0x7f52ac10fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.358+0000 7f52b37db640 1 --2- 172.21.15.3:0/788925163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ac102650 0x7f52ac102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.358+0000 7f52b37db640 1 -- 172.21.15.3:0/788925163 >> 172.21.15.3:0/788925163 conn(0x7f52ac079760 msgr2=0x7f52ac079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:08.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.358+0000 7f52b37db640 1 -- 172.21.15.3:0/788925163 shutdown_connections 2024-09-21T06:17:08.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.358+0000 7f52b37db640 1 -- 172.21.15.3:0/788925163 wait complete. 2024-09-21T06:17:08.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.358+0000 7f52b37db640 1 Processor -- start 2024-09-21T06:17:08.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.358+0000 7f52b37db640 1 -- start start 2024-09-21T06:17:08.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b37db640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52ac102650 0x7f52ac19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b37db640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ac102fa0 0x7f52ac19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b37db640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ac110200 0x7f52ac1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b37db640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac115ca0 con 0x7f52ac102fa0 2024-09-21T06:17:08.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b37db640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac115b20 con 0x7f52ac102650 2024-09-21T06:17:08.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b37db640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac115e20 con 0x7f52ac110200 2024-09-21T06:17:08.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b1577640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52ac102650 0x7f52ac19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b1d78640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ac110200 0x7f52ac1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b0d76640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ac102fa0 0x7f52ac19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b1577640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52ac102650 0x7f52ac19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49360/0 (socket says 172.21.15.3:49360) 2024-09-21T06:17:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b1577640 1 -- 172.21.15.3:0/854504359 learned_addr learned my addr 172.21.15.3:0/854504359 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b1577640 1 -- 172.21.15.3:0/854504359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ac110200 msgr2=0x7f52ac1a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b1577640 1 --2- 172.21.15.3:0/854504359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ac110200 0x7f52ac1a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b1577640 1 -- 172.21.15.3:0/854504359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ac102fa0 msgr2=0x7f52ac19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b1577640 1 --2- 172.21.15.3:0/854504359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ac102fa0 0x7f52ac19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b1577640 1 -- 172.21.15.3:0/854504359 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac1a25e0 con 0x7f52ac102650 2024-09-21T06:17:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.359+0000 7f52b0d76640 1 --2- 172.21.15.3:0/854504359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ac102fa0 0x7f52ac19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.360+0000 7f52b1577640 1 --2- 172.21.15.3:0/854504359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52ac102650 0x7f52ac19c670 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f52a800e9f0 tx=0x7f52a800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.360+0000 7f52a27fc640 1 -- 172.21.15.3:0/854504359 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a800cd80 con 0x7f52ac102650 2024-09-21T06:17:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.360+0000 7f52b37db640 1 -- 172.21.15.3:0/854504359 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ac1a28d0 con 0x7f52ac102650 2024-09-21T06:17:08.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.360+0000 7f52a27fc640 1 -- 172.21.15.3:0/854504359 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a80054d0 con 0x7f52ac102650 2024-09-21T06:17:08.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.360+0000 7f52a27fc640 1 -- 172.21.15.3:0/854504359 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8010690 con 0x7f52ac102650 2024-09-21T06:17:08.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.360+0000 7f52b37db640 1 -- 172.21.15.3:0/854504359 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ac1a9030 con 0x7f52ac102650 2024-09-21T06:17:08.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.363+0000 7f52b9d4f740 1 Processor -- start 2024-09-21T06:17:08.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.363+0000 7f52b9d4f740 1 -- start start 2024-09-21T06:17:08.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.363+0000 7f52b9d4f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28493070 0x55bd2883a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.363+0000 7f52b9d4f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 0x55bd2883ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.363+0000 7f52b9d4f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28472390 0x55bd2883b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.363+0000 7f52b9d4f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bd287b0e50 con 0x55bd28493070 2024-09-21T06:17:08.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52b9d4f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bd287a0280 con 0x55bd28472390 2024-09-21T06:17:08.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52b9d4f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bd287a4090 con 0x55bd2846a1b0 2024-09-21T06:17:08.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52b1d78640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28493070 0x55bd2883a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52b1d78640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28493070 0x55bd2883a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59038/0 (socket says 172.21.15.3:59038) 2024-09-21T06:17:08.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52b1d78640 1 -- 172.21.15.3:0/470843161 learned_addr learned my addr 172.21.15.3:0/470843161 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:08.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52b0d76640 1 --2- 172.21.15.3:0/470843161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28472390 0x55bd2883b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52a27fc640 1 -- 172.21.15.3:0/854504359 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f52a8004e90 con 0x7f52ac102650 2024-09-21T06:17:08.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52b1d78640 1 -- 172.21.15.3:0/470843161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 msgr2=0x55bd2883ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52b1d78640 1 --2- 172.21.15.3:0/470843161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 0x55bd2883ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52b1d78640 1 -- 172.21.15.3:0/470843161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28472390 msgr2=0x55bd2883b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52b1d78640 1 --2- 172.21.15.3:0/470843161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28472390 0x55bd2883b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52b1d78640 1 -- 172.21.15.3:0/470843161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd284fe650 con 0x55bd28493070 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52b0d76640 1 --2- 172.21.15.3:0/470843161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28472390 0x55bd2883b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.364+0000 7f52b1d78640 1 --2- 172.21.15.3:0/470843161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28493070 0x55bd2883a650 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f52a4035000 tx=0x7f52a40093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310b2e50e5ef8c05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.365+0000 7f52977fe640 1 -- 172.21.15.3:0/470843161 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4048070 con 0x55bd28493070 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.365+0000 7f52977fe640 1 -- 172.21.15.3:0/470843161 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a4003e20 con 0x55bd28493070 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.365+0000 7f52977fe640 1 -- 172.21.15.3:0/470843161 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4034040 con 0x55bd28493070 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.365+0000 7f52a27fc640 1 --2- 172.21.15.3:0/854504359 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f52980b6680 0x7f52980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.365+0000 7f52b1577640 1 --2- 172.21.15.3:0/854504359 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f52980b6680 0x7f52980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.365+0000 7f52a27fc640 1 -- 172.21.15.3:0/854504359 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f52a80e7ab0 con 0x7f52ac102650 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.366+0000 7f52b9d4f740 1 -- 172.21.15.3:0/470843161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28493070 msgr2=0x55bd2883a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.366+0000 7f52b1577640 1 --2- 172.21.15.3:0/854504359 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f52980b6680 0x7f52980b8b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55bd288392d0 tx=0x7f52a807a6b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.366+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/470843161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28493070 0x55bd2883a650 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f52a4035000 tx=0x7f52a40093a0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.366+0000 7f52b9d4f740 1 -- 172.21.15.3:0/470843161 shutdown_connections 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.366+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/470843161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 0x55bd2883ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.366+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/470843161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28472390 0x55bd2883b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.366+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/470843161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28493070 0x55bd2883a650 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.366+0000 7f52b9d4f740 1 -- 172.21.15.3:0/470843161 >> 172.21.15.3:0/470843161 conn(0x55bd28487460 msgr2=0x55bd28838540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.366+0000 7f52b9d4f740 1 -- 172.21.15.3:0/470843161 shutdown_connections 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.366+0000 7f52b9d4f740 1 -- 172.21.15.3:0/470843161 wait complete. 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.366+0000 7f52b9d4f740 1 -- start start 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.366+0000 7f52b9d4f740 1 Processor -- start 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b9d4f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 0x55bd28845cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b9d4f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 0x55bd28846210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b9d4f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28493070 0x55bd28846770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b9d4f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bd28601700 con 0x55bd28472390 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b9d4f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bd2873b110 con 0x55bd28493070 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b9d4f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bd28848dd0 con 0x55bd2846a1b0 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b0d76640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 0x55bd28846210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b0d76640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 0x55bd28846210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59044/0 (socket says 172.21.15.3:59044) 2024-09-21T06:17:08.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b0d76640 1 -- 172.21.15.3:0/2889392184 learned_addr learned my addr 172.21.15.3:0/2889392184 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:08.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b1d78640 1 --2- 172.21.15.3:0/2889392184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 0x55bd28845cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b9d4f740 10 client.0 _ll_register_callbacks cb 0x55bd28754990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:08.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b1d78640 1 --2- 172.21.15.3:0/2889392184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28493070 0x55bd28846770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b0d76640 1 -- 172.21.15.3:0/2889392184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 msgr2=0x55bd28845cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b0d76640 1 --2- 172.21.15.3:0/2889392184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 0x55bd28845cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b0d76640 1 -- 172.21.15.3:0/2889392184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28493070 msgr2=0x55bd28846770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b0d76640 1 --2- 172.21.15.3:0/2889392184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28493070 0x55bd28846770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.367+0000 7f52b0d76640 1 -- 172.21.15.3:0/2889392184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd28768c40 con 0x55bd28472390 2024-09-21T06:17:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.368+0000 7f52b0d76640 1 --2- 172.21.15.3:0/2889392184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 0x55bd28846210 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f529c012d20 tx=0x7f529c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f341a4a0683b838d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.368+0000 7f5294ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.368+0000 7f5294ff9640 1 -- 172.21.15.3:0/2889392184 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c0120d0 con 0x55bd28472390 2024-09-21T06:17:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.368+0000 7f5294ff9640 1 -- 172.21.15.3:0/2889392184 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f529c0047a0 con 0x55bd28472390 2024-09-21T06:17:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.368+0000 7f52b9d4f740 1 -- 172.21.15.3:0/2889392184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a4003e20 con 0x55bd28472390 2024-09-21T06:17:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.368+0000 7f5294ff9640 1 -- 172.21.15.3:0/2889392184 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f529c012270 con 0x55bd28472390 2024-09-21T06:17:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.368+0000 7f52b9d4f740 10 client.16164 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.368+0000 7f52b9d4f740 1 -- 172.21.15.3:0/2889392184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd28854b10 con 0x55bd28472390 2024-09-21T06:17:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.368+0000 7f5294ff9640 1 -- 172.21.15.3:0/2889392184 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a4003e20 con 0x55bd28472390 2024-09-21T06:17:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.368+0000 7f5294ff9640 1 -- 172.21.15.3:0/2889392184 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f529c0256b0 con 0x55bd28472390 2024-09-21T06:17:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.368+0000 7f52b9d4f740 10 client.16164 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.368+0000 7f52b9d4f740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.369+0000 7f52b9d4f740 20 client.16164 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.369+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/2889392184 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55bd284519f0 0x55bd28857fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.369+0000 7f52b9d4f740 4 client.16164 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.369+0000 7f52b9d4f740 1 -- 172.21.15.3:0/2889392184 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd287b2da0 con 0x55bd284519f0 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.369+0000 7f52b1d78640 1 --2- 172.21.15.3:0/2889392184 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55bd284519f0 0x55bd28857fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.369+0000 7f52b1d78640 1 --2- 172.21.15.3:0/2889392184 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55bd284519f0 0x55bd28857fc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c894dde4a8976fc9 server_cookie=54ef1217e35ee0a8 in_seq=0 out_seq=0 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.369+0000 7f5294ff9640 10 client.16164 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.370+0000 7f5294ff9640 1 -- 172.21.15.3:0/2889392184 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd287b2da0 con 0x55bd284519f0 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.370+0000 7f5294ff9640 10 client.16164 handle_command_reply: tid=0 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.370+0000 7f52b9d4f740 1 client.16164 shutdown 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.370+0000 7f52b9d4f740 1 -- 172.21.15.3:0/2889392184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 msgr2=0x55bd28846210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.370+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/2889392184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 0x55bd28846210 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f529c012d20 tx=0x7f529c00f790 comp rx=0 tx=0).stop 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.370+0000 7f52b9d4f740 1 -- 172.21.15.3:0/2889392184 shutdown_connections 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.370+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/2889392184 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55bd284519f0 0x55bd28857fc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.370+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/2889392184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28493070 0x55bd28846770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.371+0000 7f5294ff9640 0 client.16164 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.371+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/2889392184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 0x55bd28846210 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.371+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/2889392184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 0x55bd28845cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.371+0000 7f52b9d4f740 1 -- 172.21.15.3:0/2889392184 >> 172.21.15.3:0/2889392184 conn(0x55bd28487460 msgr2=0x55bd2883d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.371+0000 7f52b9d4f740 1 -- 172.21.15.3:0/2889392184 shutdown_connections 2024-09-21T06:17:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.371+0000 7f52b9d4f740 1 -- 172.21.15.3:0/2889392184 wait complete. 2024-09-21T06:17:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.371+0000 7f52b9d4f740 20 client.16164 trim_cache size 0 max 16384 2024-09-21T06:17:08.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.389+0000 7f52b9d4f740 1 Processor -- start 2024-09-21T06:17:08.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.389+0000 7f52b9d4f740 1 -- start start 2024-09-21T06:17:08.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b9d4f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 0x55bd2883c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b9d4f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 0x55bd28847900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b9d4f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28493070 0x55bd28837b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b9d4f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1b1700 con 0x55bd28472390 2024-09-21T06:17:08.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b9d4f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1b32a0 con 0x55bd28493070 2024-09-21T06:17:08.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b9d4f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1b1dd0 con 0x55bd2846a1b0 2024-09-21T06:17:08.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b0d76640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 0x55bd28847900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b0d76640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 0x55bd28847900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59050/0 (socket says 172.21.15.3:59050) 2024-09-21T06:17:08.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b0d76640 1 -- 172.21.15.3:0/2741879819 learned_addr learned my addr 172.21.15.3:0/2741879819 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:08.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b1d78640 1 --2- 172.21.15.3:0/2741879819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28493070 0x55bd28837b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b1d78640 1 --2- 172.21.15.3:0/2741879819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 0x55bd2883c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b0d76640 1 -- 172.21.15.3:0/2741879819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 msgr2=0x55bd2883c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b0d76640 1 --2- 172.21.15.3:0/2741879819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 0x55bd2883c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b0d76640 1 -- 172.21.15.3:0/2741879819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28493070 msgr2=0x55bd28837b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b0d76640 1 --2- 172.21.15.3:0/2741879819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28493070 0x55bd28837b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b0d76640 1 -- 172.21.15.3:0/2741879819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd28854940 con 0x55bd28472390 2024-09-21T06:17:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.390+0000 7f52b0d76640 1 --2- 172.21.15.3:0/2741879819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 0x55bd28847900 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f529c009730 tx=0x7f529c012990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa5608cf437c565 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.391+0000 7f5296ffd640 1 -- 172.21.15.3:0/2741879819 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c025440 con 0x55bd28472390 2024-09-21T06:17:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.391+0000 7f5296ffd640 1 -- 172.21.15.3:0/2741879819 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c0054f0 con 0x55bd28472390 2024-09-21T06:17:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.391+0000 7f5296ffd640 1 -- 172.21.15.3:0/2741879819 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c003650 con 0x55bd28472390 2024-09-21T06:17:08.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.391+0000 7f52b9d4f740 1 -- 172.21.15.3:0/2741879819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 msgr2=0x55bd28847900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.391+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/2741879819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 0x55bd28847900 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f529c009730 tx=0x7f529c012990 comp rx=0 tx=0).stop 2024-09-21T06:17:08.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.392+0000 7f52b9d4f740 1 -- 172.21.15.3:0/2741879819 shutdown_connections 2024-09-21T06:17:08.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.392+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/2741879819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28493070 0x55bd28837b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.392+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/2741879819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd28472390 0x55bd28847900 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.392+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/2741879819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd2846a1b0 0x55bd2883c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.392+0000 7f52b9d4f740 1 -- 172.21.15.3:0/2741879819 >> 172.21.15.3:0/2741879819 conn(0x55bd28487460 msgr2=0x55bd288471f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.392+0000 7f52b9d4f740 1 -- 172.21.15.3:0/2741879819 shutdown_connections 2024-09-21T06:17:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.392+0000 7f52b9d4f740 1 -- 172.21.15.3:0/2741879819 wait complete. 2024-09-21T06:17:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.392+0000 7f52b9d4f740 1 -- start start 2024-09-21T06:17:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.392+0000 7f52b9d4f740 1 Processor -- start 2024-09-21T06:17:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b9d4f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd2846a1b0 0x55bd2884ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b9d4f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28472390 0x55bd28849f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b9d4f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd28493070 0x55bd2884a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b9d4f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1b2b70 con 0x55bd2846a1b0 2024-09-21T06:17:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b9d4f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1b24a0 con 0x55bd28472390 2024-09-21T06:17:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b9d4f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1af6e0 con 0x55bd28493070 2024-09-21T06:17:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b1d78640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd2846a1b0 0x55bd2884ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b9d4f740 10 client.0 _ll_register_callbacks cb 0x7f529c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b0d76640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28472390 0x55bd28849f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b1d78640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd2846a1b0 0x55bd2884ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59064/0 (socket says 172.21.15.3:59064) 2024-09-21T06:17:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b1d78640 1 -- 172.21.15.3:0/3769774206 learned_addr learned my addr 172.21.15.3:0/3769774206 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b1d78640 1 --2- 172.21.15.3:0/3769774206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd28493070 0x55bd2884a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b1d78640 1 -- 172.21.15.3:0/3769774206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd28493070 msgr2=0x55bd2884a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b1d78640 1 --2- 172.21.15.3:0/3769774206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd28493070 0x55bd2884a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b1d78640 1 -- 172.21.15.3:0/3769774206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28472390 msgr2=0x55bd28849f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b1d78640 1 --2- 172.21.15.3:0/3769774206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28472390 0x55bd28849f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b1d78640 1 -- 172.21.15.3:0/3769774206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd28851ff0 con 0x55bd2846a1b0 2024-09-21T06:17:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b0d76640 1 --2- 172.21.15.3:0/3769774206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28472390 0x55bd28849f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f52b1d78640 1 --2- 172.21.15.3:0/3769774206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd2846a1b0 0x55bd2884ea70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f52a4005e30 tx=0x7f529c0128a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca3cc4a78e4823f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.393+0000 7f5294ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f5294ff9640 1 -- 172.21.15.3:0/3769774206 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4048040 con 0x55bd2846a1b0 2024-09-21T06:17:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f5294ff9640 1 -- 172.21.15.3:0/3769774206 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f52a4003a30 con 0x55bd2846a1b0 2024-09-21T06:17:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f52b9d4f740 1 -- 172.21.15.3:0/3769774206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f529c0054f0 con 0x55bd2846a1b0 2024-09-21T06:17:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f5294ff9640 1 -- 172.21.15.3:0/3769774206 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52a4003c30 con 0x55bd2846a1b0 2024-09-21T06:17:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f52b9d4f740 10 client.16170 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f52b9d4f740 1 -- 172.21.15.3:0/3769774206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd288c29d0 con 0x55bd2846a1b0 2024-09-21T06:17:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f5294ff9640 1 -- 172.21.15.3:0/3769774206 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c0054f0 con 0x55bd2846a1b0 2024-09-21T06:17:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f5294ff9640 1 -- 172.21.15.3:0/3769774206 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f52a4062440 con 0x55bd2846a1b0 2024-09-21T06:17:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f52b9d4f740 10 client.16170 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f52b9d4f740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f52b9d4f740 20 client.16170 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/3769774206 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55bd28426da0 0x55bd288c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f52b9d4f740 4 client.16170 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:17:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f52b9d4f740 1 -- 172.21.15.3:0/3769774206 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bd287ae850 con 0x55bd28426da0 2024-09-21T06:17:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.394+0000 7f52b0d76640 1 --2- 172.21.15.3:0/3769774206 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55bd28426da0 0x55bd288c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.395+0000 7f52b0d76640 1 --2- 172.21.15.3:0/3769774206 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55bd28426da0 0x55bd288c4cb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f224863d2f43cf25 server_cookie=cf92fe14b88554d5 in_seq=0 out_seq=0 2024-09-21T06:17:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.395+0000 7f5294ff9640 10 client.16170 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:08.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.396+0000 7f5294ff9640 1 -- 172.21.15.3:0/3769774206 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x55bd287ae850 con 0x55bd28426da0 2024-09-21T06:17:08.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.396+0000 7f5294ff9640 10 client.16170 handle_command_reply: tid=0 2024-09-21T06:17:08.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.396+0000 7f52b9d4f740 1 client.16170 shutdown 2024-09-21T06:17:08.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.396+0000 7f52b9d4f740 1 -- 172.21.15.3:0/3769774206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd2846a1b0 msgr2=0x55bd2884ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.396+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/3769774206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd2846a1b0 0x55bd2884ea70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f52a4005e30 tx=0x7f529c0128a0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.396+0000 7f52b9d4f740 1 -- 172.21.15.3:0/3769774206 shutdown_connections 2024-09-21T06:17:08.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.396+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/3769774206 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55bd28426da0 0x55bd288c4cb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/3769774206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd28493070 0x55bd2884a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f5294ff9640 0 client.16170 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:08.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/3769774206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd28472390 0x55bd28849f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b9d4f740 1 --2- 172.21.15.3:0/3769774206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd2846a1b0 0x55bd2884ea70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b9d4f740 1 -- 172.21.15.3:0/3769774206 >> 172.21.15.3:0/3769774206 conn(0x55bd28487460 msgr2=0x55bd2883be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:08.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b9d4f740 1 -- 172.21.15.3:0/3769774206 shutdown_connections 2024-09-21T06:17:08.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b9d4f740 1 -- 172.21.15.3:0/3769774206 wait complete. 2024-09-21T06:17:08.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b9d4f740 20 client.16170 trim_cache size 0 max 16384 2024-09-21T06:17:08.404 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:17:08.404 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:17:08.404 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:17:08.404 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:17:08.404 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:17:08.404 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:17:08.404 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:08.404 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:08.404 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:08.404 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:08.405 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:08.406 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:08.406 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:08.406 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:17:08.406 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:17:08.406 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:17:08.406 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:08.406 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:08.406 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:08.406 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:08.406 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:08.406 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:17:08.406 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:17:08.406 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:17:08.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b37db640 1 -- 172.21.15.3:0/854504359 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f52980b6680 msgr2=0x7f52980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b37db640 1 --2- 172.21.15.3:0/854504359 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f52980b6680 0x7f52980b8b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55bd288392d0 tx=0x7f52a807a6b0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b37db640 1 -- 172.21.15.3:0/854504359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52ac102650 msgr2=0x7f52ac19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b37db640 1 --2- 172.21.15.3:0/854504359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52ac102650 0x7f52ac19c670 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f52a800e9f0 tx=0x7f52a800edf0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b37db640 1 -- 172.21.15.3:0/854504359 shutdown_connections 2024-09-21T06:17:08.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b37db640 1 --2- 172.21.15.3:0/854504359 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f52980b6680 0x7f52980b8b90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b37db640 1 --2- 172.21.15.3:0/854504359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ac110200 0x7f52ac1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b37db640 1 --2- 172.21.15.3:0/854504359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52ac102fa0 0x7f52ac19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b37db640 1 --2- 172.21.15.3:0/854504359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52ac102650 0x7f52ac19c670 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.397+0000 7f52b37db640 1 -- 172.21.15.3:0/854504359 >> 172.21.15.3:0/854504359 conn(0x7f52ac079760 msgr2=0x7f52ac109da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:08.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.398+0000 7f52b37db640 1 -- 172.21.15.3:0/854504359 shutdown_connections 2024-09-21T06:17:08.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:08.398+0000 7f52b37db640 1 -- 172.21.15.3:0/854504359 wait complete. 2024-09-21T06:17:08.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 110269 14.96 MB/sec warmup 41 sec latency 12.929 ms 2024-09-21T06:17:09.148 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:17:09.148 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:17:09.469 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:09.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:09 smithi003 ceph-mon[31227]: pgmap v389: 97 pgs: 97 active+clean; 37 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 293 op/s 2024-09-21T06:17:09.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:09 smithi137 ceph-mon[34464]: pgmap v389: 97 pgs: 97 active+clean; 37 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 293 op/s 2024-09-21T06:17:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:09 smithi204 ceph-mon[34659]: pgmap v389: 97 pgs: 97 active+clean; 37 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 293 op/s 2024-09-21T06:17:09.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 112985 14.91 MB/sec warmup 42 sec latency 13.223 ms 2024-09-21T06:17:10.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.154+0000 7fe7d3f81640 1 -- 172.21.15.3:0/2943557211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc104690 msgr2=0x7fe7cc104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.154+0000 7fe7d3f81640 1 --2- 172.21.15.3:0/2943557211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc104690 0x7fe7cc104aa0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009ca0 tx=0x7fe7bc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.154+0000 7fe7d3f81640 1 -- 172.21.15.3:0/2943557211 shutdown_connections 2024-09-21T06:17:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.154+0000 7fe7d3f81640 1 --2- 172.21.15.3:0/2943557211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7cc101da0 0x7fe7cc1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.154+0000 7fe7d3f81640 1 --2- 172.21.15.3:0/2943557211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc102890 0x7fe7cc112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.154+0000 7fe7d3f81640 1 --2- 172.21.15.3:0/2943557211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc104690 0x7fe7cc104aa0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.154+0000 7fe7d3f81640 1 -- 172.21.15.3:0/2943557211 >> 172.21.15.3:0/2943557211 conn(0x7fe7cc077c60 msgr2=0x7fe7cc0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.154+0000 7fe7d3f81640 1 -- 172.21.15.3:0/2943557211 shutdown_connections 2024-09-21T06:17:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.154+0000 7fe7d3f81640 1 -- 172.21.15.3:0/2943557211 wait complete. 2024-09-21T06:17:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.155+0000 7fe7d3f81640 1 Processor -- start 2024-09-21T06:17:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.155+0000 7fe7d3f81640 1 -- start start 2024-09-21T06:17:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.155+0000 7fe7d3f81640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc101da0 0x7fe7cc19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.155+0000 7fe7d3f81640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc102890 0x7fe7cc19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.155+0000 7fe7d3f81640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7cc104690 0x7fe7cc1a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.155+0000 7fe7d3f81640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc115ca0 con 0x7fe7cc101da0 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.155+0000 7fe7d3f81640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc115b20 con 0x7fe7cc102890 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d3f81640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc115e20 con 0x7fe7cc104690 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d1d1d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc101da0 0x7fe7cc19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d1d1d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc101da0 0x7fe7cc19c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59076/0 (socket says 172.21.15.3:59076) 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d1d1d640 1 -- 172.21.15.3:0/1085574172 learned_addr learned my addr 172.21.15.3:0/1085574172 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d151c640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc102890 0x7fe7cc19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d251e640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7cc104690 0x7fe7cc1a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d1d1d640 1 -- 172.21.15.3:0/1085574172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7cc104690 msgr2=0x7fe7cc1a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d1d1d640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7cc104690 0x7fe7cc1a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d1d1d640 1 -- 172.21.15.3:0/1085574172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc102890 msgr2=0x7fe7cc19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d1d1d640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc102890 0x7fe7cc19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d1d1d640 1 -- 172.21.15.3:0/1085574172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc1a2620 con 0x7fe7cc101da0 2024-09-21T06:17:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d1d1d640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc101da0 0x7fe7cc19c6b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc033040 tx=0x7fe7bc0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7caffd640 1 -- 172.21.15.3:0/1085574172 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc003b70 con 0x7fe7cc101da0 2024-09-21T06:17:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7caffd640 1 -- 172.21.15.3:0/1085574172 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7bc003d10 con 0x7fe7cc101da0 2024-09-21T06:17:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d3f81640 1 -- 172.21.15.3:0/1085574172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7cc1a28b0 con 0x7fe7cc101da0 2024-09-21T06:17:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7caffd640 1 -- 172.21.15.3:0/1085574172 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc02f9e0 con 0x7fe7cc101da0 2024-09-21T06:17:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.156+0000 7fe7d3f81640 1 -- 172.21.15.3:0/1085574172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7cc1a9070 con 0x7fe7cc101da0 2024-09-21T06:17:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.157+0000 7fe7d3f81640 1 -- 172.21.15.3:0/1085574172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7cc1021b0 con 0x7fe7cc101da0 2024-09-21T06:17:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.160+0000 7fe7caffd640 1 -- 172.21.15.3:0/1085574172 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fe7bc02fb80 con 0x7fe7cc101da0 2024-09-21T06:17:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.161+0000 7fe7caffd640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe7b40b6700 0x7fe7b40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.161+0000 7fe7d151c640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe7b40b6700 0x7fe7b40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.161+0000 7fe7caffd640 1 -- 172.21.15.3:0/1085574172 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe7bc108b20 con 0x7fe7cc101da0 2024-09-21T06:17:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.162+0000 7fe7d151c640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe7b40b6700 0x7fe7b40b8c10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc104d70 tx=0x7fe7c0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:10.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.164+0000 7fe7caffd640 1 -- 172.21.15.3:0/1085574172 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe7bc0cf050 con 0x7fe7cc101da0 2024-09-21T06:17:10.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.405+0000 7fe7d3f81640 1 -- 172.21.15.3:0/1085574172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7cc10dc60 con 0x7fe7cc101da0 2024-09-21T06:17:10.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.406+0000 7fe7caffd640 1 -- 172.21.15.3:0/1085574172 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7fe7bc0d3430 con 0x7fe7cc101da0 2024-09-21T06:17:10.408 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:17:10.408 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-09-21T06:16:31:270114+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:16:30.910049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:17:10.409 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-09-21T06:17:10.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.409+0000 7fe7d3f81640 1 -- 172.21.15.3:0/1085574172 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe7b40b6700 msgr2=0x7fe7b40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.409+0000 7fe7d3f81640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe7b40b6700 0x7fe7b40b8c10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc104d70 tx=0x7fe7c0005f90 comp rx=0 tx=0).stop 2024-09-21T06:17:10.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.409+0000 7fe7d3f81640 1 -- 172.21.15.3:0/1085574172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc101da0 msgr2=0x7fe7cc19c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.409+0000 7fe7d3f81640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc101da0 0x7fe7cc19c6b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc033040 tx=0x7fe7bc0316f0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.409+0000 7fe7d3f81640 1 -- 172.21.15.3:0/1085574172 shutdown_connections 2024-09-21T06:17:10.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.409+0000 7fe7d3f81640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe7b40b6700 0x7fe7b40b8c10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.409+0000 7fe7d3f81640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7cc104690 0x7fe7cc1a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.409+0000 7fe7d3f81640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc102890 0x7fe7cc19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.409+0000 7fe7d3f81640 1 --2- 172.21.15.3:0/1085574172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7cc101da0 0x7fe7cc19c6b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.409+0000 7fe7d3f81640 1 -- 172.21.15.3:0/1085574172 >> 172.21.15.3:0/1085574172 conn(0x7fe7cc077c60 msgr2=0x7fe7cc0ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:10.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.409+0000 7fe7d3f81640 1 -- 172.21.15.3:0/1085574172 shutdown_connections 2024-09-21T06:17:10.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:10.409+0000 7fe7d3f81640 1 -- 172.21.15.3:0/1085574172 wait complete. 2024-09-21T06:17:10.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 114713 14.77 MB/sec warmup 43 sec latency 23.810 ms 2024-09-21T06:17:11.102 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:17:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:11 smithi003 ceph-mon[31227]: pgmap v390: 97 pgs: 97 active+clean; 39 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 393 op/s 2024-09-21T06:17:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:11 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1085574172' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:17:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:11 smithi137 ceph-mon[34464]: pgmap v390: 97 pgs: 97 active+clean; 39 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 393 op/s 2024-09-21T06:17:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:11 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1085574172' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:17:11.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:11 smithi204 ceph-mon[34659]: pgmap v390: 97 pgs: 97 active+clean; 39 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 393 op/s 2024-09-21T06:17:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:11 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1085574172' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:17:11.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.888+0000 7f5429c58640 1 -- 172.21.15.3:0/2848003446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54241083d0 msgr2=0x7f54241087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:11.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.888+0000 7f5429c58640 1 --2- 172.21.15.3:0/2848003446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54241083d0 0x7f54241087e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5418009900 tx=0x7f541802a410 comp rx=0 tx=0).stop 2024-09-21T06:17:11.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.888+0000 7f5429c58640 1 -- 172.21.15.3:0/2848003446 shutdown_connections 2024-09-21T06:17:11.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.888+0000 7f5429c58640 1 --2- 172.21.15.3:0/2848003446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5424108d20 0x7f542410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.888+0000 7f5429c58640 1 --2- 172.21.15.3:0/2848003446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54241083d0 0x7f54241087e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.888+0000 7f5429c58640 1 --2- 172.21.15.3:0/2848003446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5424102ae0 0x7f5424102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.888+0000 7f5429c58640 1 -- 172.21.15.3:0/2848003446 >> 172.21.15.3:0/2848003446 conn(0x7f5424079760 msgr2=0x7f5424079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:11.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.888+0000 7f5429c58640 1 -- 172.21.15.3:0/2848003446 shutdown_connections 2024-09-21T06:17:11.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.888+0000 7f5429c58640 1 -- 172.21.15.3:0/2848003446 wait complete. 2024-09-21T06:17:11.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.889+0000 7f5429c58640 1 Processor -- start 2024-09-21T06:17:11.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.889+0000 7f5429c58640 1 -- start start 2024-09-21T06:17:11.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f5429c58640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5424102ae0 0x7f54241993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f5429c58640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54241083d0 0x7f5424199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f5429c58640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5424108d20 0x7f542419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f5429c58640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5424112990 con 0x7f5424102ae0 2024-09-21T06:17:11.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f5429c58640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5424112810 con 0x7f5424108d20 2024-09-21T06:17:11.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f5429c58640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5424112b10 con 0x7f54241083d0 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f54237fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5424102ae0 0x7f54241993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f54237fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5424102ae0 0x7f54241993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59092/0 (socket says 172.21.15.3:59092) 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f54237fe640 1 -- 172.21.15.3:0/248541971 learned_addr learned my addr 172.21.15.3:0/248541971 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f5422ffd640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54241083d0 0x7f5424199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f5423fff640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5424108d20 0x7f542419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f54237fe640 1 -- 172.21.15.3:0/248541971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54241083d0 msgr2=0x7f5424199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f54237fe640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54241083d0 0x7f5424199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f54237fe640 1 -- 172.21.15.3:0/248541971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5424108d20 msgr2=0x7f542419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f54237fe640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5424108d20 0x7f542419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f54237fe640 1 -- 172.21.15.3:0/248541971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542419f330 con 0x7f5424102ae0 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.890+0000 7f54237fe640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5424102ae0 0x7f54241993c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f541400db80 tx=0x7f541400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.891+0000 7f5420ff9640 1 -- 172.21.15.3:0/248541971 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414017070 con 0x7f5424102ae0 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.891+0000 7f5420ff9640 1 -- 172.21.15.3:0/248541971 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5414003b70 con 0x7f5424102ae0 2024-09-21T06:17:11.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.891+0000 7f5429c58640 1 -- 172.21.15.3:0/248541971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542419f620 con 0x7f5424102ae0 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.891+0000 7f5420ff9640 1 -- 172.21.15.3:0/248541971 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54140106c0 con 0x7f5424102ae0 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.891+0000 7f5429c58640 1 -- 172.21.15.3:0/248541971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54241a5d80 con 0x7f5424102ae0 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f54301cc740 1 Processor -- start 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f5420ff9640 1 -- 172.21.15.3:0/248541971 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f5414004020 con 0x7f5424102ae0 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f54301cc740 1 -- start start 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f54301cc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d5116d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f54301cc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50d9b1b0 0x562d5116ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f54301cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d5116e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f54301cc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562d510e1ea0 con 0x562d50dc4070 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f54301cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d510d12d0 con 0x562d50da3390 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f54301cc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562d510d50e0 con 0x562d50d9b1b0 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f5423fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d5116d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f5423fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d5116d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59094/0 (socket says 172.21.15.3:59094) 2024-09-21T06:17:11.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f5423fff640 1 -- 172.21.15.3:0/3816438713 learned_addr learned my addr 172.21.15.3:0/3816438713 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:11.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f54237fe640 1 --2- 172.21.15.3:0/3816438713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50d9b1b0 0x562d5116ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.894+0000 7f5422ffd640 1 --2- 172.21.15.3:0/3816438713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d5116e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.895+0000 7f5423fff640 1 -- 172.21.15.3:0/3816438713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50d9b1b0 msgr2=0x562d5116ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:11.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.895+0000 7f5423fff640 1 --2- 172.21.15.3:0/3816438713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50d9b1b0 0x562d5116ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.895+0000 7f5423fff640 1 -- 172.21.15.3:0/3816438713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 msgr2=0x562d5116e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:11.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.895+0000 7f5423fff640 1 --2- 172.21.15.3:0/3816438713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d5116e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.895+0000 7f5423fff640 1 -- 172.21.15.3:0/3816438713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d50e2f650 con 0x562d50dc4070 2024-09-21T06:17:11.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.895+0000 7f54237fe640 1 --2- 172.21.15.3:0/3816438713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50d9b1b0 0x562d5116ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:11.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.895+0000 7f5420ff9640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f540c0b6680 0x7f540c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.895+0000 7f54237fe640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f540c0b6680 0x7f540c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.895+0000 7f5423fff640 1 --2- 172.21.15.3:0/3816438713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d5116d8b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f541c004620 tx=0x7f541c00bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7156bb41a4e661ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:11.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.895+0000 7f5409ffb640 1 -- 172.21.15.3:0/3816438713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c003ba0 con 0x562d50dc4070 2024-09-21T06:17:11.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.896+0000 7f5420ff9640 1 -- 172.21.15.3:0/248541971 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f54140e76e0 con 0x7f5424102ae0 2024-09-21T06:17:11.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.896+0000 7f5409ffb640 1 -- 172.21.15.3:0/3816438713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f541c015070 con 0x562d50dc4070 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.896+0000 7f5409ffb640 1 -- 172.21.15.3:0/3816438713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c0116a0 con 0x562d50dc4070 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.897+0000 7f54301cc740 1 -- 172.21.15.3:0/3816438713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 msgr2=0x562d5116d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.897+0000 7f54237fe640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f540c0b6680 0x7f540c0b8b90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f54140e4f20 tx=0x7f54140ed550 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.897+0000 7f54301cc740 1 --2- 172.21.15.3:0/3816438713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d5116d8b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f541c004620 tx=0x7f541c00bb10 comp rx=0 tx=0).stop 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.897+0000 7f54301cc740 1 -- 172.21.15.3:0/3816438713 shutdown_connections 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.897+0000 7f54301cc740 1 --2- 172.21.15.3:0/3816438713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50d9b1b0 0x562d5116ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.897+0000 7f54301cc740 1 --2- 172.21.15.3:0/3816438713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d5116e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.897+0000 7f54301cc740 1 --2- 172.21.15.3:0/3816438713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d5116d8b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.897+0000 7f54301cc740 1 -- 172.21.15.3:0/3816438713 >> 172.21.15.3:0/3816438713 conn(0x562d50db8460 msgr2=0x562d5116b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.897+0000 7f54301cc740 1 -- 172.21.15.3:0/3816438713 shutdown_connections 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.897+0000 7f54301cc740 1 -- 172.21.15.3:0/3816438713 wait complete. 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.897+0000 7f54301cc740 1 -- start start 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.897+0000 7f54301cc740 1 Processor -- start 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f54301cc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50d9b1b0 0x562d511774e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f54301cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d51177aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f54301cc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d511871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f54301cc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562d50f32700 con 0x562d50dc4070 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f54301cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d5106c160 con 0x562d50da3390 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f54301cc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562d511690c0 con 0x562d50d9b1b0 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f54301cc740 10 client.0 _ll_register_callbacks cb 0x562d510859e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f5423fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d511871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f5423fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d511871e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59098/0 (socket says 172.21.15.3:59098) 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f5423fff640 1 -- 172.21.15.3:0/2281281334 learned_addr learned my addr 172.21.15.3:0/2281281334 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f5422ffd640 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d51177aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f5423fff640 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50d9b1b0 0x562d511774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f5423fff640 1 -- 172.21.15.3:0/2281281334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50d9b1b0 msgr2=0x562d511774e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f5423fff640 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50d9b1b0 0x562d511774e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f5423fff640 1 -- 172.21.15.3:0/2281281334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 msgr2=0x562d51177aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f5423fff640 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d51177aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.898+0000 7f5423fff640 1 -- 172.21.15.3:0/2281281334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d51099c90 con 0x562d50dc4070 2024-09-21T06:17:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.899+0000 7f5422ffd640 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d51177aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.899+0000 7f5423fff640 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d511871e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x562d5116ff00 tx=0x7f541c01ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9e19176c8e1c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.899+0000 7f5402ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.899+0000 7f5402ffd640 1 -- 172.21.15.3:0/2281281334 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c005330 con 0x562d50dc4070 2024-09-21T06:17:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.899+0000 7f5402ffd640 1 -- 172.21.15.3:0/2281281334 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f541c0054d0 con 0x562d50dc4070 2024-09-21T06:17:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.899+0000 7f54301cc740 1 -- 172.21.15.3:0/2281281334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f541c015070 con 0x562d50dc4070 2024-09-21T06:17:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.899+0000 7f5402ffd640 1 -- 172.21.15.3:0/2281281334 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f541c01a470 con 0x562d50dc4070 2024-09-21T06:17:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.899+0000 7f54301cc740 10 client.16188 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.899+0000 7f54301cc740 1 -- 172.21.15.3:0/2281281334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562d51189f90 con 0x562d50dc4070 2024-09-21T06:17:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.899+0000 7f5402ffd640 1 -- 172.21.15.3:0/2281281334 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f541c015070 con 0x562d50dc4070 2024-09-21T06:17:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.900+0000 7f5402ffd640 1 -- 172.21.15.3:0/2281281334 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f541c011440 con 0x562d50dc4070 2024-09-21T06:17:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.900+0000 7f54301cc740 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.900+0000 7f54301cc740 10 client.16188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.900+0000 7f54301cc740 20 client.16188 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.900+0000 7f54301cc740 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562d50d829f0 0x562d5118d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.900+0000 7f54301cc740 4 client.16188 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.900+0000 7f54301cc740 1 -- 172.21.15.3:0/2281281334 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d510e3df0 con 0x562d50d829f0 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.900+0000 7f5422ffd640 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562d50d829f0 0x562d5118d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.901+0000 7f5402ffd640 10 client.16188 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.901+0000 7f5422ffd640 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562d50d829f0 0x562d5118d350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8d10df28e914d918 server_cookie=50e8f9ac9b1ccaf4 in_seq=0 out_seq=0 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.901+0000 7f5402ffd640 1 -- 172.21.15.3:0/2281281334 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d510e3df0 con 0x562d50d829f0 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.901+0000 7f5402ffd640 10 client.16188 handle_command_reply: tid=0 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.901+0000 7f54301cc740 1 client.16188 shutdown 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.902+0000 7f54301cc740 1 -- 172.21.15.3:0/2281281334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 msgr2=0x562d511871e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.902+0000 7f54301cc740 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d511871e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x562d5116ff00 tx=0x7f541c01ac10 comp rx=0 tx=0).stop 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.902+0000 7f54301cc740 1 -- 172.21.15.3:0/2281281334 shutdown_connections 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.902+0000 7f54301cc740 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562d50d829f0 0x562d5118d350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.902+0000 7f54301cc740 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d511871e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.902+0000 7f54301cc740 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d51177aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.902+0000 7f54301cc740 1 --2- 172.21.15.3:0/2281281334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50d9b1b0 0x562d511774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.902+0000 7f54301cc740 1 -- 172.21.15.3:0/2281281334 >> 172.21.15.3:0/2281281334 conn(0x562d50db8460 msgr2=0x562d51170a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:11.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.902+0000 7f5402ffd640 0 client.16188 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:17:11.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.902+0000 7f54301cc740 1 -- 172.21.15.3:0/2281281334 shutdown_connections 2024-09-21T06:17:11.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.902+0000 7f54301cc740 1 -- 172.21.15.3:0/2281281334 wait complete. 2024-09-21T06:17:11.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.902+0000 7f54301cc740 20 client.16188 trim_cache size 0 max 16384 2024-09-21T06:17:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f54301cc740 1 Processor -- start 2024-09-21T06:17:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f54301cc740 1 -- start start 2024-09-21T06:17:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f54301cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50d9b1b0 0x562d511719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f54301cc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50da3390 0x562d51178f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f54301cc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d511871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f54301cc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f54241ad510 con 0x562d50dc4070 2024-09-21T06:17:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f54301cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f54241af0b0 con 0x562d50d9b1b0 2024-09-21T06:17:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f54301cc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f54241adbe0 con 0x562d50da3390 2024-09-21T06:17:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f5422ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50da3390 0x562d51178f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f5423fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d511871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f5423fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50d9b1b0 0x562d511719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f5422ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50da3390 0x562d51178f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:38704/0 (socket says 172.21.15.3:38704) 2024-09-21T06:17:11.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f5422ffd640 1 -- 172.21.15.3:0/4034101052 learned_addr learned my addr 172.21.15.3:0/4034101052 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f5422ffd640 1 -- 172.21.15.3:0/4034101052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50d9b1b0 msgr2=0x562d511719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f5422ffd640 1 --2- 172.21.15.3:0/4034101052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50d9b1b0 0x562d511719c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f5422ffd640 1 -- 172.21.15.3:0/4034101052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 msgr2=0x562d511871e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f5422ffd640 1 --2- 172.21.15.3:0/4034101052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d511871e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.916+0000 7f5422ffd640 1 -- 172.21.15.3:0/4034101052 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d5116e6e0 con 0x562d50da3390 2024-09-21T06:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.917+0000 7f5422ffd640 1 --2- 172.21.15.3:0/4034101052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50da3390 0x562d51178f10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5418009900 tx=0x7f5418031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b287e7cf26ed43b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.917+0000 7f54097fa640 1 -- 172.21.15.3:0/4034101052 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418035040 con 0x562d50da3390 2024-09-21T06:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.917+0000 7f54097fa640 1 -- 172.21.15.3:0/4034101052 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5418004400 con 0x562d50da3390 2024-09-21T06:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.917+0000 7f54097fa640 1 -- 172.21.15.3:0/4034101052 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54180035f0 con 0x562d50da3390 2024-09-21T06:17:11.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.918+0000 7f54301cc740 1 -- 172.21.15.3:0/4034101052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50da3390 msgr2=0x562d51178f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:11.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.918+0000 7f54301cc740 1 --2- 172.21.15.3:0/4034101052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50da3390 0x562d51178f10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5418009900 tx=0x7f5418031400 comp rx=0 tx=0).stop 2024-09-21T06:17:11.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.918+0000 7f54301cc740 1 -- 172.21.15.3:0/4034101052 shutdown_connections 2024-09-21T06:17:11.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.918+0000 7f54301cc740 1 --2- 172.21.15.3:0/4034101052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50da3390 0x562d51178f10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.918+0000 7f54301cc740 1 --2- 172.21.15.3:0/4034101052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50dc4070 0x562d511871e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.918+0000 7f54301cc740 1 --2- 172.21.15.3:0/4034101052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50d9b1b0 0x562d511719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.918+0000 7f54301cc740 1 -- 172.21.15.3:0/4034101052 >> 172.21.15.3:0/4034101052 conn(0x562d50db8460 msgr2=0x562d51178800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:11.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.918+0000 7f54301cc740 1 -- 172.21.15.3:0/4034101052 shutdown_connections 2024-09-21T06:17:11.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.918+0000 7f54301cc740 1 -- 172.21.15.3:0/4034101052 wait complete. 2024-09-21T06:17:11.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.918+0000 7f54301cc740 1 -- start start 2024-09-21T06:17:11.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.918+0000 7f54301cc740 1 Processor -- start 2024-09-21T06:17:11.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f54301cc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50d9b1b0 0x562d511878c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f54301cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d51182d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f54301cc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50dc4070 0x562d511832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f54301cc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f54241ae980 con 0x562d50d9b1b0 2024-09-21T06:17:11.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f54301cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f54241ae2b0 con 0x562d50da3390 2024-09-21T06:17:11.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f54301cc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f54241ab4f0 con 0x562d50dc4070 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f54301cc740 10 client.0 _ll_register_callbacks cb 0x7f5418009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f5423fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50d9b1b0 0x562d511878c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f5423fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50d9b1b0 0x562d511878c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59116/0 (socket says 172.21.15.3:59116) 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f5423fff640 1 -- 172.21.15.3:0/3929256544 learned_addr learned my addr 172.21.15.3:0/3929256544 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f5422ffd640 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d51182d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f5423fff640 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50dc4070 0x562d511832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f5422ffd640 1 -- 172.21.15.3:0/3929256544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50dc4070 msgr2=0x562d511832b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f5422ffd640 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50dc4070 0x562d511832b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f5422ffd640 1 -- 172.21.15.3:0/3929256544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50d9b1b0 msgr2=0x562d511878c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f5422ffd640 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50d9b1b0 0x562d511878c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f5422ffd640 1 -- 172.21.15.3:0/3929256544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d51183a40 con 0x562d50da3390 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.919+0000 7f5423fff640 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50d9b1b0 0x562d511878c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.920+0000 7f5422ffd640 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d51182d70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5418009230 tx=0x7f5418032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6bc25a879abdb05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.920+0000 7f5402ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.920+0000 7f5402ffd640 1 -- 172.21.15.3:0/3929256544 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418035070 con 0x562d50da3390 2024-09-21T06:17:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.920+0000 7f54301cc740 1 -- 172.21.15.3:0/3929256544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5418004400 con 0x562d50da3390 2024-09-21T06:17:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.920+0000 7f5402ffd640 1 -- 172.21.15.3:0/3929256544 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5418005450 con 0x562d50da3390 2024-09-21T06:17:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.920+0000 7f5402ffd640 1 -- 172.21.15.3:0/3929256544 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5418005650 con 0x562d50da3390 2024-09-21T06:17:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.920+0000 7f54301cc740 10 client.25778 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.920+0000 7f54301cc740 1 -- 172.21.15.3:0/3929256544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562d511f7a10 con 0x562d50da3390 2024-09-21T06:17:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.920+0000 7f5402ffd640 1 -- 172.21.15.3:0/3929256544 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5418004400 con 0x562d50da3390 2024-09-21T06:17:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.921+0000 7f5402ffd640 1 -- 172.21.15.3:0/3929256544 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5418003a50 con 0x562d50da3390 2024-09-21T06:17:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.921+0000 7f54301cc740 10 client.25778 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.921+0000 7f54301cc740 10 client.25778 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:17:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.921+0000 7f54301cc740 20 client.25778 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.921+0000 7f54301cc740 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562d50d57da0 0x562d511fa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.921+0000 7f54301cc740 4 client.25778 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:17:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.921+0000 7f54301cc740 1 -- 172.21.15.3:0/3929256544 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d511649e0 con 0x562d50d57da0 2024-09-21T06:17:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.921+0000 7f5423fff640 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562d50d57da0 0x562d511fa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.921+0000 7f5402ffd640 10 client.25778 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:17:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.921+0000 7f5423fff640 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562d50d57da0 0x562d511fa1b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6233f1c677be23db server_cookie=861d3880b85e6bc1 in_seq=0 out_seq=0 2024-09-21T06:17:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f5402ffd640 1 -- 172.21.15.3:0/3929256544 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d511649e0 con 0x562d50d57da0 2024-09-21T06:17:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f5402ffd640 10 client.25778 handle_command_reply: tid=0 2024-09-21T06:17:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f54301cc740 1 client.25778 shutdown 2024-09-21T06:17:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f54301cc740 1 -- 172.21.15.3:0/3929256544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 msgr2=0x562d51182d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f54301cc740 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d51182d70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5418009230 tx=0x7f5418032ce0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f54301cc740 1 -- 172.21.15.3:0/3929256544 shutdown_connections 2024-09-21T06:17:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f54301cc740 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562d50d57da0 0x562d511fa1b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f54301cc740 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d50dc4070 0x562d511832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f54301cc740 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d50da3390 0x562d51182d70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f54301cc740 1 --2- 172.21.15.3:0/3929256544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d50d9b1b0 0x562d511878c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f5402ffd640 0 client.25778 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:17:11.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f54301cc740 1 -- 172.21.15.3:0/3929256544 >> 172.21.15.3:0/3929256544 conn(0x562d50db8460 msgr2=0x562d51168570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:11.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f54301cc740 1 -- 172.21.15.3:0/3929256544 shutdown_connections 2024-09-21T06:17:11.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f54301cc740 1 -- 172.21.15.3:0/3929256544 wait complete. 2024-09-21T06:17:11.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.922+0000 7f54301cc740 20 client.25778 trim_cache size 0 max 16384 2024-09-21T06:17:11.928 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:17:11.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.923+0000 7f5429c58640 1 -- 172.21.15.3:0/248541971 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f540c0b6680 msgr2=0x7f540c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.923+0000 7f5429c58640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f540c0b6680 0x7f540c0b8b90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f54140e4f20 tx=0x7f54140ed550 comp rx=0 tx=0).stop 2024-09-21T06:17:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.923+0000 7f5429c58640 1 -- 172.21.15.3:0/248541971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5424102ae0 msgr2=0x7f54241993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.923+0000 7f5429c58640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5424102ae0 0x7f54241993c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f541400db80 tx=0x7f541400b630 comp rx=0 tx=0).stop 2024-09-21T06:17:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.923+0000 7f5429c58640 1 -- 172.21.15.3:0/248541971 shutdown_connections 2024-09-21T06:17:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.923+0000 7f5429c58640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f540c0b6680 0x7f540c0b8b90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.923+0000 7f5429c58640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54241083d0 0x7f5424199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.923+0000 7f5429c58640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5424108d20 0x7f542419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.923+0000 7f5429c58640 1 --2- 172.21.15.3:0/248541971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5424102ae0 0x7f54241993c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.923+0000 7f5429c58640 1 -- 172.21.15.3:0/248541971 >> 172.21.15.3:0/248541971 conn(0x7f5424079760 msgr2=0x7f5424104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.923+0000 7f5429c58640 1 -- 172.21.15.3:0/248541971 shutdown_connections 2024-09-21T06:17:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:11.923+0000 7f5429c58640 1 -- 172.21.15.3:0/248541971 wait complete. 2024-09-21T06:17:11.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 117483 14.82 MB/sec warmup 44 sec latency 16.799 ms 2024-09-21T06:17:12.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:17:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:12.946 INFO:tasks.workunit.client.0.smithi003.stdout: 1 120609 14.88 MB/sec warmup 45 sec latency 12.323 ms 2024-09-21T06:17:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.485+0000 7f95a8425640 1 -- 172.21.15.3:0/3445605276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a0105380 msgr2=0x7f95a010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.485+0000 7f95a8425640 1 --2- 172.21.15.3:0/3445605276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a0105380 0x7f95a010d8d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9590009900 tx=0x7f959002a410 comp rx=0 tx=0).stop 2024-09-21T06:17:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.485+0000 7f95a8425640 1 -- 172.21.15.3:0/3445605276 shutdown_connections 2024-09-21T06:17:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.485+0000 7f95a8425640 1 --2- 172.21.15.3:0/3445605276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a010de60 0x7f95a0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.485+0000 7f95a8425640 1 --2- 172.21.15.3:0/3445605276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a0105380 0x7f95a010d8d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.485+0000 7f95a8425640 1 --2- 172.21.15.3:0/3445605276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95a0104a30 0x7f95a0104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.485+0000 7f95a8425640 1 -- 172.21.15.3:0/3445605276 >> 172.21.15.3:0/3445605276 conn(0x7f95a00797b0 msgr2=0x7f95a0079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.485+0000 7f95a8425640 1 -- 172.21.15.3:0/3445605276 shutdown_connections 2024-09-21T06:17:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.485+0000 7f95a8425640 1 -- 172.21.15.3:0/3445605276 wait complete. 2024-09-21T06:17:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.486+0000 7f95a8425640 1 Processor -- start 2024-09-21T06:17:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.486+0000 7f95a8425640 1 -- start start 2024-09-21T06:17:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.486+0000 7f95a8425640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95a0104a30 0x7f95a019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.486+0000 7f95a8425640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a0105380 0x7f95a019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.486+0000 7f95a8425640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a010de60 0x7f95a019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.486+0000 7f95a8425640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f95a0113aa0 con 0x7f95a0105380 2024-09-21T06:17:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.486+0000 7f95a8425640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95a0113920 con 0x7f95a0104a30 2024-09-21T06:17:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.486+0000 7f95a8425640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95a0113c20 con 0x7f95a010de60 2024-09-21T06:17:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.486+0000 7f95a59c0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a0105380 0x7f95a019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.486+0000 7f95a61c1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95a0104a30 0x7f95a019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.486+0000 7f95a59c0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a0105380 0x7f95a019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59136/0 (socket says 172.21.15.3:59136) 2024-09-21T06:17:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.486+0000 7f95a59c0640 1 -- 172.21.15.3:0/2396175781 learned_addr learned my addr 172.21.15.3:0/2396175781 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.487+0000 7f95a69c2640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a010de60 0x7f95a019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.487+0000 7f95a61c1640 1 -- 172.21.15.3:0/2396175781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a010de60 msgr2=0x7f95a019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.487+0000 7f95a61c1640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a010de60 0x7f95a019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.487+0000 7f95a61c1640 1 -- 172.21.15.3:0/2396175781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a0105380 msgr2=0x7f95a019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.487+0000 7f95a61c1640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a0105380 0x7f95a019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.487+0000 7f95a61c1640 1 -- 172.21.15.3:0/2396175781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a01a0440 con 0x7f95a0104a30 2024-09-21T06:17:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.487+0000 7f95a59c0640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a0105380 0x7f95a019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.487+0000 7f95a61c1640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95a0104a30 0x7f95a019a4d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f959c00e9f0 tx=0x7f959c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.487+0000 7f95977fe640 1 -- 172.21.15.3:0/2396175781 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c00ccc0 con 0x7f95a0104a30 2024-09-21T06:17:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.487+0000 7f95a8425640 1 -- 172.21.15.3:0/2396175781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a01a0730 con 0x7f95a0104a30 2024-09-21T06:17:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.488+0000 7f95a8425640 1 -- 172.21.15.3:0/2396175781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a01a6e90 con 0x7f95a0104a30 2024-09-21T06:17:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.488+0000 7f95977fe640 1 -- 172.21.15.3:0/2396175781 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959c0108e0 con 0x7f95a0104a30 2024-09-21T06:17:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.488+0000 7f95977fe640 1 -- 172.21.15.3:0/2396175781 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c018880 con 0x7f95a0104a30 2024-09-21T06:17:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.490+0000 7f95977fe640 1 -- 172.21.15.3:0/2396175781 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f959c018a20 con 0x7f95a0104a30 2024-09-21T06:17:13.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.490+0000 7f95ae999740 1 Processor -- start 2024-09-21T06:17:13.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.491+0000 7f95ae999740 1 -- start start 2024-09-21T06:17:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.491+0000 7f95ae999740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 0x558d296526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.491+0000 7f95ae999740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292821b0 0x558d29652be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.491+0000 7f95ae999740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 0x558d29653120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.491+0000 7f95ae999740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558d295c8ea0 con 0x558d292ab070 2024-09-21T06:17:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.491+0000 7f95ae999740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558d295b82d0 con 0x558d2928a390 2024-09-21T06:17:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.491+0000 7f95ae999740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558d295bc0e0 con 0x558d292821b0 2024-09-21T06:17:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.491+0000 7f95a69c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 0x558d296526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.491+0000 7f95a61c1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292821b0 0x558d29652be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.491+0000 7f95a69c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 0x558d296526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59142/0 (socket says 172.21.15.3:59142) 2024-09-21T06:17:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.491+0000 7f95a69c2640 1 -- 172.21.15.3:0/1304579968 learned_addr learned my addr 172.21.15.3:0/1304579968 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.491+0000 7f95a59c0640 1 --2- 172.21.15.3:0/1304579968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 0x558d29653120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.492+0000 7f95a69c2640 1 -- 172.21.15.3:0/1304579968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292821b0 msgr2=0x558d29652be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.492+0000 7f95a69c2640 1 --2- 172.21.15.3:0/1304579968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292821b0 0x558d29652be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.492+0000 7f95a69c2640 1 -- 172.21.15.3:0/1304579968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 msgr2=0x558d29653120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.492+0000 7f95a69c2640 1 --2- 172.21.15.3:0/1304579968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 0x558d29653120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.492+0000 7f95a69c2640 1 -- 172.21.15.3:0/1304579968 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d29316650 con 0x558d292ab070 2024-09-21T06:17:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.492+0000 7f95977fe640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f958c0b6700 0x7f958c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.492+0000 7f95a69c2640 1 --2- 172.21.15.3:0/1304579968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 0x558d296526a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9598009fd0 tx=0x7f95980118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395b9b4874f1ba97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.492+0000 7f95977fe640 1 -- 172.21.15.3:0/2396175781 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f959c0e7850 con 0x7f95a0104a30 2024-09-21T06:17:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.492+0000 7f95a61c1640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f958c0b6700 0x7f958c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.492+0000 7f9594d9a640 1 -- 172.21.15.3:0/1304579968 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598004d20 con 0x558d292ab070 2024-09-21T06:17:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.492+0000 7f9594d9a640 1 -- 172.21.15.3:0/1304579968 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9598004050 con 0x558d292ab070 2024-09-21T06:17:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.492+0000 7f9594d9a640 1 -- 172.21.15.3:0/1304579968 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598005280 con 0x558d292ab070 2024-09-21T06:17:13.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.493+0000 7f95ae999740 1 -- 172.21.15.3:0/1304579968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 msgr2=0x558d296526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:13.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.493+0000 7f95ae999740 1 --2- 172.21.15.3:0/1304579968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 0x558d296526a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9598009fd0 tx=0x7f95980118d0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.493+0000 7f95a61c1640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f958c0b6700 0x7f958c0b8c10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x558d29651320 tx=0x7f959c0ed530 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.493+0000 7f95ae999740 1 -- 172.21.15.3:0/1304579968 shutdown_connections 2024-09-21T06:17:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.493+0000 7f95ae999740 1 --2- 172.21.15.3:0/1304579968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292821b0 0x558d29652be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.493+0000 7f95ae999740 1 --2- 172.21.15.3:0/1304579968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 0x558d29653120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.493+0000 7f95ae999740 1 --2- 172.21.15.3:0/1304579968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 0x558d296526a0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.493+0000 7f95ae999740 1 -- 172.21.15.3:0/1304579968 >> 172.21.15.3:0/1304579968 conn(0x558d2929f460 msgr2=0x558d29650590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.493+0000 7f95ae999740 1 -- 172.21.15.3:0/1304579968 shutdown_connections 2024-09-21T06:17:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.493+0000 7f95ae999740 1 -- 172.21.15.3:0/1304579968 wait complete. 2024-09-21T06:17:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.494+0000 7f95ae999740 1 -- start start 2024-09-21T06:17:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.494+0000 7f95ae999740 1 Processor -- start 2024-09-21T06:17:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.494+0000 7f95ae999740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d292821b0 0x558d2965dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.494+0000 7f95ae999740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2928a390 0x558d2965e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.494+0000 7f95ae999740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 0x558d2965e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.494+0000 7f95ae999740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558d29419700 con 0x558d292ab070 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.494+0000 7f95ae999740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558d29553160 con 0x558d292821b0 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.494+0000 7f95ae999740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558d29660e80 con 0x558d2928a390 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.494+0000 7f95ae999740 10 client.0 _ll_register_callbacks cb 0x558d2956c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.494+0000 7f95a69c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 0x558d2965e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.494+0000 7f95a69c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 0x558d2965e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59158/0 (socket says 172.21.15.3:59158) 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.494+0000 7f95a69c2640 1 -- 172.21.15.3:0/2779318047 learned_addr learned my addr 172.21.15.3:0/2779318047 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.494+0000 7f95a69c2640 1 --2- 172.21.15.3:0/2779318047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d292821b0 0x558d2965dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f95a59c0640 1 --2- 172.21.15.3:0/2779318047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2928a390 0x558d2965e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f95a69c2640 1 -- 172.21.15.3:0/2779318047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2928a390 msgr2=0x558d2965e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f95a69c2640 1 --2- 172.21.15.3:0/2779318047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2928a390 0x558d2965e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f95a69c2640 1 -- 172.21.15.3:0/2779318047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d292821b0 msgr2=0x558d2965dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f95a69c2640 1 --2- 172.21.15.3:0/2779318047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d292821b0 0x558d2965dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f95a69c2640 1 -- 172.21.15.3:0/2779318047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d29580c90 con 0x558d292ab070 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f95a69c2640 1 --2- 172.21.15.3:0/2779318047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 0x558d2965e720 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9598003fd0 tx=0x7f9598005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176128a8f9e4957c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f9589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f9589ffb640 1 -- 172.21.15.3:0/2779318047 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959800f4e0 con 0x558d292ab070 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f9589ffb640 1 -- 172.21.15.3:0/2779318047 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f959800f680 con 0x558d292ab070 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f95ae999740 1 -- 172.21.15.3:0/2779318047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9598004050 con 0x558d292ab070 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f9589ffb640 1 -- 172.21.15.3:0/2779318047 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f959800f880 con 0x558d292ab070 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f95ae999740 10 client.16206 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.495+0000 7f95ae999740 1 -- 172.21.15.3:0/2779318047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d2966cb10 con 0x558d292ab070 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.496+0000 7f9589ffb640 1 -- 172.21.15.3:0/2779318047 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9598004050 con 0x558d292ab070 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.496+0000 7f9589ffb640 1 -- 172.21.15.3:0/2779318047 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9598028850 con 0x558d292ab070 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.496+0000 7f95ae999740 10 client.16206 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.496+0000 7f95ae999740 10 client.16206 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.496+0000 7f95ae999740 20 client.16206 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.496+0000 7f95ae999740 1 --2- 172.21.15.3:0/2779318047 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x558d292699f0 0x558d2966fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.496+0000 7f95ae999740 4 client.16206 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.496+0000 7f95ae999740 1 -- 172.21.15.3:0/2779318047 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d295cadf0 con 0x558d292699f0 2024-09-21T06:17:13.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.496+0000 7f95a59c0640 1 --2- 172.21.15.3:0/2779318047 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x558d292699f0 0x558d2966fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.496+0000 7f95a59c0640 1 --2- 172.21.15.3:0/2779318047 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x558d292699f0 0x558d2966fe70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=68f8202cd031c647 server_cookie=84e7ea73980533c2 in_seq=0 out_seq=0 2024-09-21T06:17:13.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.496+0000 7f9589ffb640 10 client.16206 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:17:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.497+0000 7f9589ffb640 1 -- 172.21.15.3:0/2779318047 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558d295cadf0 con 0x558d292699f0 2024-09-21T06:17:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.497+0000 7f9589ffb640 10 client.16206 handle_command_reply: tid=0 2024-09-21T06:17:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.497+0000 7f95ae999740 1 client.16206 shutdown 2024-09-21T06:17:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.497+0000 7f95ae999740 1 -- 172.21.15.3:0/2779318047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 msgr2=0x558d2965e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.497+0000 7f95ae999740 1 --2- 172.21.15.3:0/2779318047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 0x558d2965e720 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9598003fd0 tx=0x7f9598005a50 comp rx=0 tx=0).stop 2024-09-21T06:17:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.497+0000 7f95ae999740 1 -- 172.21.15.3:0/2779318047 shutdown_connections 2024-09-21T06:17:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.497+0000 7f95ae999740 1 --2- 172.21.15.3:0/2779318047 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x558d292699f0 0x558d2966fe70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.497+0000 7f95ae999740 1 --2- 172.21.15.3:0/2779318047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2928a390 0x558d2965e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.497+0000 7f95ae999740 1 --2- 172.21.15.3:0/2779318047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292ab070 0x558d2965e720 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.497+0000 7f95ae999740 1 --2- 172.21.15.3:0/2779318047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d292821b0 0x558d2965dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.498+0000 7f95ae999740 1 -- 172.21.15.3:0/2779318047 >> 172.21.15.3:0/2779318047 conn(0x558d2929f460 msgr2=0x558d29655830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.498+0000 7f9589ffb640 0 client.16206 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:17:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.498+0000 7f95ae999740 1 -- 172.21.15.3:0/2779318047 shutdown_connections 2024-09-21T06:17:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.498+0000 7f95ae999740 1 -- 172.21.15.3:0/2779318047 wait complete. 2024-09-21T06:17:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.498+0000 7f95ae999740 20 client.16206 trim_cache size 0 max 16384 2024-09-21T06:17:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.511+0000 7f95ae999740 1 Processor -- start 2024-09-21T06:17:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.511+0000 7f95ae999740 1 -- start start 2024-09-21T06:17:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.511+0000 7f95ae999740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 0x558d2965ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.511+0000 7f95ae999740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 0x558d2965b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.511+0000 7f95ae999740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292ab070 0x558d296536d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.511+0000 7f95ae999740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9584012b00 con 0x558d292821b0 2024-09-21T06:17:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.511+0000 7f95ae999740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95840146a0 con 0x558d2928a390 2024-09-21T06:17:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.511+0000 7f95ae999740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95840131d0 con 0x558d292ab070 2024-09-21T06:17:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f95a69c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 0x558d2965ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f95a69c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 0x558d2965ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59166/0 (socket says 172.21.15.3:59166) 2024-09-21T06:17:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f95a69c2640 1 -- 172.21.15.3:0/2306927782 learned_addr learned my addr 172.21.15.3:0/2306927782 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f95a59c0640 1 --2- 172.21.15.3:0/2306927782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 0x558d2965b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f95a69c2640 1 --2- 172.21.15.3:0/2306927782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292ab070 0x558d296536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f95a69c2640 1 -- 172.21.15.3:0/2306927782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292ab070 msgr2=0x558d296536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f95a69c2640 1 --2- 172.21.15.3:0/2306927782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292ab070 0x558d296536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f95a69c2640 1 -- 172.21.15.3:0/2306927782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 msgr2=0x558d2965b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:13.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f95a69c2640 1 --2- 172.21.15.3:0/2306927782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 0x558d2965b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f95a69c2640 1 -- 172.21.15.3:0/2306927782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d29654990 con 0x558d292821b0 2024-09-21T06:17:13.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f95a69c2640 1 --2- 172.21.15.3:0/2306927782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 0x558d2965ae90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9598005b40 tx=0x7f9598005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f364674ebbff724 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:13.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f958bfff640 1 -- 172.21.15.3:0/2306927782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959800f4e0 con 0x558d292821b0 2024-09-21T06:17:13.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f958bfff640 1 -- 172.21.15.3:0/2306927782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959800f680 con 0x558d292821b0 2024-09-21T06:17:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.512+0000 7f958bfff640 1 -- 172.21.15.3:0/2306927782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598028910 con 0x558d292821b0 2024-09-21T06:17:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.513+0000 7f95ae999740 1 -- 172.21.15.3:0/2306927782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 msgr2=0x558d2965ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.513+0000 7f95ae999740 1 --2- 172.21.15.3:0/2306927782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 0x558d2965ae90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9598005b40 tx=0x7f9598005bd0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.513+0000 7f95ae999740 1 -- 172.21.15.3:0/2306927782 shutdown_connections 2024-09-21T06:17:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.513+0000 7f95ae999740 1 --2- 172.21.15.3:0/2306927782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292ab070 0x558d296536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.513+0000 7f95ae999740 1 --2- 172.21.15.3:0/2306927782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 0x558d2965b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.513+0000 7f95ae999740 1 --2- 172.21.15.3:0/2306927782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 0x558d2965ae90 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.513+0000 7f95ae999740 1 -- 172.21.15.3:0/2306927782 >> 172.21.15.3:0/2306927782 conn(0x558d2929f460 msgr2=0x558d2965c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.513+0000 7f95ae999740 1 -- 172.21.15.3:0/2306927782 shutdown_connections 2024-09-21T06:17:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.513+0000 7f95ae999740 1 -- 172.21.15.3:0/2306927782 wait complete. 2024-09-21T06:17:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95ae999740 1 -- start start 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95ae999740 1 Processor -- start 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95ae999740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 0x558d296665c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95ae999740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 0x558d29666b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95ae999740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292ab070 0x558d29661c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95ae999740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9584013f70 con 0x558d292821b0 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95ae999740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95840138a0 con 0x558d2928a390 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95ae999740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9584010ae0 con 0x558d292ab070 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95ae999740 10 client.0 _ll_register_callbacks cb 0x7f95980093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95a69c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 0x558d296665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95a59c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 0x558d29666b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95a69c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 0x558d296665c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59180/0 (socket says 172.21.15.3:59180) 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95a69c2640 1 -- 172.21.15.3:0/1523738696 learned_addr learned my addr 172.21.15.3:0/1523738696 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95a69c2640 1 --2- 172.21.15.3:0/1523738696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292ab070 0x558d29661c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95a59c0640 1 -- 172.21.15.3:0/1523738696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292ab070 msgr2=0x558d29661c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95a59c0640 1 --2- 172.21.15.3:0/1523738696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292ab070 0x558d29661c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.514+0000 7f95a59c0640 1 -- 172.21.15.3:0/1523738696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 msgr2=0x558d296665c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.515+0000 7f95a59c0640 1 --2- 172.21.15.3:0/1523738696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 0x558d296665c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.515+0000 7f95a59c0640 1 -- 172.21.15.3:0/1523738696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d296623b0 con 0x558d2928a390 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.515+0000 7f95a69c2640 1 --2- 172.21.15.3:0/1523738696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 0x558d296665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.515+0000 7f95a59c0640 1 --2- 172.21.15.3:0/1523738696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 0x558d29666b80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9590031440 tx=0x7f95900044c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11ba88029384ec0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.515+0000 7f9589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.515+0000 7f9589ffb640 1 -- 172.21.15.3:0/1523738696 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959004c070 con 0x558d2928a390 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.515+0000 7f95ae999740 1 -- 172.21.15.3:0/1523738696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f959800f680 con 0x558d2928a390 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.515+0000 7f9589ffb640 1 -- 172.21.15.3:0/1523738696 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f9590003ac0 con 0x558d2928a390 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.515+0000 7f9589ffb640 1 -- 172.21.15.3:0/1523738696 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9590035040 con 0x558d2928a390 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.515+0000 7f95ae999740 10 client.25796 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.515+0000 7f95ae999740 1 -- 172.21.15.3:0/1523738696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d296da380 con 0x558d2928a390 2024-09-21T06:17:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.515+0000 7f9589ffb640 1 -- 172.21.15.3:0/1523738696 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959800f680 con 0x558d2928a390 2024-09-21T06:17:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.516+0000 7f9589ffb640 1 -- 172.21.15.3:0/1523738696 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9590004960 con 0x558d2928a390 2024-09-21T06:17:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.516+0000 7f95ae999740 10 client.25796 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.516+0000 7f95ae999740 10 client.25796 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:17:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.516+0000 7f95ae999740 20 client.25796 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.516+0000 7f95ae999740 1 --2- 172.21.15.3:0/1523738696 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x558d2923eda0 0x558d296dcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.516+0000 7f95ae999740 4 client.25796 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:17:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.516+0000 7f95ae999740 1 -- 172.21.15.3:0/1523738696 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d295c68a0 con 0x558d2923eda0 2024-09-21T06:17:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.516+0000 7f95a69c2640 1 --2- 172.21.15.3:0/1523738696 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x558d2923eda0 0x558d296dcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.516+0000 7f95a69c2640 1 --2- 172.21.15.3:0/1523738696 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x558d2923eda0 0x558d296dcb20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a7c33d1b66ddab6 server_cookie=d99b59bc65d6d5b0 in_seq=0 out_seq=0 2024-09-21T06:17:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.516+0000 7f9589ffb640 10 client.25796 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:17:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f9589ffb640 1 -- 172.21.15.3:0/1523738696 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d295c68a0 con 0x558d2923eda0 2024-09-21T06:17:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f9589ffb640 10 client.25796 handle_command_reply: tid=0 2024-09-21T06:17:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f95ae999740 1 client.25796 shutdown 2024-09-21T06:17:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f95ae999740 1 -- 172.21.15.3:0/1523738696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 msgr2=0x558d29666b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f95ae999740 1 --2- 172.21.15.3:0/1523738696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 0x558d29666b80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9590031440 tx=0x7f95900044c0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f95ae999740 1 -- 172.21.15.3:0/1523738696 shutdown_connections 2024-09-21T06:17:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f95ae999740 1 --2- 172.21.15.3:0/1523738696 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x558d2923eda0 0x558d296dcb20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f95ae999740 1 --2- 172.21.15.3:0/1523738696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d292ab070 0x558d29661c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f95ae999740 1 --2- 172.21.15.3:0/1523738696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d2928a390 0x558d29666b80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f9589ffb640 0 client.25796 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:17:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f95ae999740 1 --2- 172.21.15.3:0/1523738696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d292821b0 0x558d296665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f95ae999740 1 -- 172.21.15.3:0/1523738696 >> 172.21.15.3:0/1523738696 conn(0x558d2929f460 msgr2=0x558d296690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f95ae999740 1 -- 172.21.15.3:0/1523738696 shutdown_connections 2024-09-21T06:17:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f95ae999740 1 -- 172.21.15.3:0/1523738696 wait complete. 2024-09-21T06:17:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.517+0000 7f95ae999740 20 client.25796 trim_cache size 0 max 16384 2024-09-21T06:17:13.524 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:17:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.518+0000 7f95a8425640 1 -- 172.21.15.3:0/2396175781 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f958c0b6700 msgr2=0x7f958c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.518+0000 7f95a8425640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f958c0b6700 0x7f958c0b8c10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x558d29651320 tx=0x7f959c0ed530 comp rx=0 tx=0).stop 2024-09-21T06:17:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.518+0000 7f95a8425640 1 -- 172.21.15.3:0/2396175781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95a0104a30 msgr2=0x7f95a019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.518+0000 7f95a8425640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95a0104a30 0x7f95a019a4d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f959c00e9f0 tx=0x7f959c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.518+0000 7f95a8425640 1 -- 172.21.15.3:0/2396175781 shutdown_connections 2024-09-21T06:17:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.518+0000 7f95a8425640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f958c0b6700 0x7f958c0b8c10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.518+0000 7f95a8425640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a010de60 0x7f95a019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.518+0000 7f95a8425640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95a0105380 0x7f95a019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.518+0000 7f95a8425640 1 --2- 172.21.15.3:0/2396175781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95a0104a30 0x7f95a019a4d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.518+0000 7f95a8425640 1 -- 172.21.15.3:0/2396175781 >> 172.21.15.3:0/2396175781 conn(0x7f95a00797b0 msgr2=0x7f95a01008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.518+0000 7f95a8425640 1 -- 172.21.15.3:0/2396175781 shutdown_connections 2024-09-21T06:17:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:13.518+0000 7f95a8425640 1 -- 172.21.15.3:0/2396175781 wait complete. 2024-09-21T06:17:13.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:13 smithi137 ceph-mon[34464]: pgmap v391: 97 pgs: 97 active+clean; 40 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.9 MiB/s wr, 391 op/s 2024-09-21T06:17:13.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:13 smithi204 ceph-mon[34659]: pgmap v391: 97 pgs: 97 active+clean; 40 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.9 MiB/s wr, 391 op/s 2024-09-21T06:17:13.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:13 smithi003 ceph-mon[31227]: pgmap v391: 97 pgs: 97 active+clean; 40 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.9 MiB/s wr, 391 op/s 2024-09-21T06:17:13.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 122086 14.70 MB/sec warmup 46 sec latency 22.029 ms 2024-09-21T06:17:14.401 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:17:14.672 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:14.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 124655 14.69 MB/sec warmup 47 sec latency 14.728 ms 2024-09-21T06:17:15.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.207+0000 7f75652b7640 1 -- 172.21.15.3:0/2610239264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f756010de60 msgr2=0x7f7560110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:15.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.207+0000 7f75652b7640 1 --2- 172.21.15.3:0/2610239264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f756010de60 0x7f7560110330 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7558009ca0 tx=0x7f755802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.207+0000 7f75652b7640 1 -- 172.21.15.3:0/2610239264 shutdown_connections 2024-09-21T06:17:15.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.207+0000 7f75652b7640 1 --2- 172.21.15.3:0/2610239264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7560105380 0x7f756010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.207+0000 7f75652b7640 1 --2- 172.21.15.3:0/2610239264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f756010de60 0x7f7560110330 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.207+0000 7f75652b7640 1 --2- 172.21.15.3:0/2610239264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7560104a30 0x7f7560104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.207+0000 7f75652b7640 1 -- 172.21.15.3:0/2610239264 >> 172.21.15.3:0/2610239264 conn(0x7f7560079790 msgr2=0x7f7560079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:15.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.207+0000 7f75652b7640 1 -- 172.21.15.3:0/2610239264 shutdown_connections 2024-09-21T06:17:15.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.207+0000 7f75652b7640 1 -- 172.21.15.3:0/2610239264 wait complete. 2024-09-21T06:17:15.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.208+0000 7f75652b7640 1 Processor -- start 2024-09-21T06:17:15.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.208+0000 7f75652b7640 1 -- start start 2024-09-21T06:17:15.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.208+0000 7f75652b7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7560104a30 0x7f756019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.208+0000 7f75652b7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7560105380 0x7f756019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.208+0000 7f75652b7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f756010de60 0x7f756019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f75652b7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7560113aa0 con 0x7f756010de60 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f75652b7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7560113920 con 0x7f7560105380 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f75652b7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7560113c20 con 0x7f7560104a30 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f755effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7560104a30 0x7f756019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f755effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7560104a30 0x7f756019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:38778/0 (socket says 172.21.15.3:38778) 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f755effd640 1 -- 172.21.15.3:0/2224801272 learned_addr learned my addr 172.21.15.3:0/2224801272 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f755f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f756010de60 0x7f756019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f7557fff640 1 --2- 172.21.15.3:0/2224801272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7560105380 0x7f756019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f755effd640 1 -- 172.21.15.3:0/2224801272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7560105380 msgr2=0x7f756019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f755effd640 1 --2- 172.21.15.3:0/2224801272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7560105380 0x7f756019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f755effd640 1 -- 172.21.15.3:0/2224801272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f756010de60 msgr2=0x7f756019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:15.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f755effd640 1 --2- 172.21.15.3:0/2224801272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f756010de60 0x7f756019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f755effd640 1 -- 172.21.15.3:0/2224801272 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75601a0400 con 0x7f7560104a30 2024-09-21T06:17:15.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f755effd640 1 --2- 172.21.15.3:0/2224801272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7560104a30 0x7f756019a490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f755000ea60 tx=0x7f755000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:15.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.209+0000 7f755cff9640 1 -- 172.21.15.3:0/2224801272 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755000cde0 con 0x7f7560104a30 2024-09-21T06:17:15.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.210+0000 7f75652b7640 1 -- 172.21.15.3:0/2224801272 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75601a06f0 con 0x7f7560104a30 2024-09-21T06:17:15.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.210+0000 7f755cff9640 1 -- 172.21.15.3:0/2224801272 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75500035f0 con 0x7f7560104a30 2024-09-21T06:17:15.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.210+0000 7f755cff9640 1 -- 172.21.15.3:0/2224801272 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550010690 con 0x7f7560104a30 2024-09-21T06:17:15.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.210+0000 7f75652b7640 1 -- 172.21.15.3:0/2224801272 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7560102650 con 0x7f7560104a30 2024-09-21T06:17:15.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.212+0000 7f756b82b740 1 Processor -- start 2024-09-21T06:17:15.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f756b82b740 1 -- start start 2024-09-21T06:17:15.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f755cff9640 1 -- 172.21.15.3:0/2224801272 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f75500037a0 con 0x7f7560104a30 2024-09-21T06:17:15.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f756b82b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d342e070 0x55d3d37d56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f756b82b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d34051b0 0x55d3d37d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f756b82b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37d6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f756b82b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d3d374bea0 con 0x55d3d34051b0 2024-09-21T06:17:15.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f756b82b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d3d373b2d0 con 0x55d3d340d390 2024-09-21T06:17:15.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f756b82b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d3d373f0e0 con 0x55d3d342e070 2024-09-21T06:17:15.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f755effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d34051b0 0x55d3d37d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f755f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d342e070 0x55d3d37d56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f7557fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37d6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f755effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d34051b0 0x55d3d37d5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59210/0 (socket says 172.21.15.3:59210) 2024-09-21T06:17:15.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f755effd640 1 -- 172.21.15.3:0/2616428247 learned_addr learned my addr 172.21.15.3:0/2616428247 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:15.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.213+0000 7f7557fff640 1 --2- 172.21.15.3:0/2616428247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37d6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49546/0 (socket says 172.21.15.3:49546) 2024-09-21T06:17:15.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.214+0000 7f755f7fe640 1 -- 172.21.15.3:0/2616428247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 msgr2=0x55d3d37d6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:15.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.214+0000 7f755f7fe640 1 --2- 172.21.15.3:0/2616428247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37d6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.214+0000 7f755f7fe640 1 -- 172.21.15.3:0/2616428247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d34051b0 msgr2=0x55d3d37d5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:15.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.214+0000 7f755f7fe640 1 --2- 172.21.15.3:0/2616428247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d34051b0 0x55d3d37d5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.214+0000 7f755f7fe640 1 -- 172.21.15.3:0/2616428247 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3d3499650 con 0x55d3d342e070 2024-09-21T06:17:15.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.214+0000 7f755f7fe640 1 --2- 172.21.15.3:0/2616428247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d342e070 0x55d3d37d56a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7558005e30 tx=0x7f755802f410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74d0d793a2b08b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:15.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.214+0000 7f755cff9640 1 --2- 172.21.15.3:0/2224801272 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f754c0b6680 0x7f754c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.214+0000 7f75557fa640 1 -- 172.21.15.3:0/2616428247 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558031cf0 con 0x55d3d342e070 2024-09-21T06:17:15.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.214+0000 7f755effd640 1 --2- 172.21.15.3:0/2224801272 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f754c0b6680 0x7f754c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.215+0000 7f75557fa640 1 -- 172.21.15.3:0/2616428247 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7558036030 con 0x55d3d342e070 2024-09-21T06:17:15.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.215+0000 7f75557fa640 1 -- 172.21.15.3:0/2616428247 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558004040 con 0x55d3d342e070 2024-09-21T06:17:15.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.215+0000 7f755cff9640 1 -- 172.21.15.3:0/2224801272 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f75500e79d0 con 0x7f7560104a30 2024-09-21T06:17:15.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.215+0000 7f756b82b740 1 -- 172.21.15.3:0/2616428247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d342e070 msgr2=0x55d3d37d56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.215+0000 7f756b82b740 1 --2- 172.21.15.3:0/2616428247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d342e070 0x55d3d37d56a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7558005e30 tx=0x7f755802f410 comp rx=0 tx=0).stop 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.215+0000 7f755effd640 1 --2- 172.21.15.3:0/2224801272 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f754c0b6680 0x7f754c0b8b90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f75500aaad0 tx=0x7f75500a9040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.216+0000 7f756b82b740 1 -- 172.21.15.3:0/2616428247 shutdown_connections 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.216+0000 7f756b82b740 1 --2- 172.21.15.3:0/2616428247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37d6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.216+0000 7f756b82b740 1 --2- 172.21.15.3:0/2616428247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d34051b0 0x55d3d37d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.216+0000 7f756b82b740 1 --2- 172.21.15.3:0/2616428247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d342e070 0x55d3d37d56a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.216+0000 7f756b82b740 1 -- 172.21.15.3:0/2616428247 >> 172.21.15.3:0/2616428247 conn(0x55d3d3422460 msgr2=0x55d3d37d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.216+0000 7f756b82b740 1 -- 172.21.15.3:0/2616428247 shutdown_connections 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.216+0000 7f756b82b740 1 -- 172.21.15.3:0/2616428247 wait complete. 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.216+0000 7f756b82b740 1 -- start start 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.216+0000 7f756b82b740 1 Processor -- start 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f756b82b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d34051b0 0x55d3d37e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f756b82b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f756b82b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d342e070 0x55d3d37ecf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f756b82b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d3d359c700 con 0x55d3d342e070 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f756b82b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d3d36d6160 con 0x55d3d340d390 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f756b82b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d3d37e3ca0 con 0x55d3d34051b0 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f756b82b740 10 client.0 _ll_register_callbacks cb 0x55d3d36ef9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f755f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d342e070 0x55d3d37ecf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f7557fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f7557fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49560/0 (socket says 172.21.15.3:49560) 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f7557fff640 1 -- 172.21.15.3:0/564707529 learned_addr learned my addr 172.21.15.3:0/564707529 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f755f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d342e070 0x55d3d37ecf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59216/0 (socket says 172.21.15.3:59216) 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f755f7fe640 1 --2- 172.21.15.3:0/564707529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d34051b0 0x55d3d37e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f7557fff640 1 -- 172.21.15.3:0/564707529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d34051b0 msgr2=0x55d3d37e0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f7557fff640 1 --2- 172.21.15.3:0/564707529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d34051b0 0x55d3d37e0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f7557fff640 1 -- 172.21.15.3:0/564707529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d342e070 msgr2=0x55d3d37ecf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f7557fff640 1 --2- 172.21.15.3:0/564707529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d342e070 0x55d3d37ecf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f7557fff640 1 -- 172.21.15.3:0/564707529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3d3703c90 con 0x55d3d340d390 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.217+0000 7f755f7fe640 1 --2- 172.21.15.3:0/564707529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d34051b0 0x55d3d37e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.218+0000 7f7557fff640 1 --2- 172.21.15.3:0/564707529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e1240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7548009920 tx=0x7f7548009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da54a5d86d7b45d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.218+0000 7f75467fc640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.218+0000 7f75467fc640 1 -- 172.21.15.3:0/564707529 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754800bcf0 con 0x55d3d340d390 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.218+0000 7f75467fc640 1 -- 172.21.15.3:0/564707529 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f7548004030 con 0x55d3d340d390 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.218+0000 7f756b82b740 1 -- 172.21.15.3:0/564707529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7558036030 con 0x55d3d340d390 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.218+0000 7f75467fc640 1 -- 172.21.15.3:0/564707529 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f754802a050 con 0x55d3d340d390 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.218+0000 7f756b82b740 10 client.25808 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.218+0000 7f756b82b740 1 -- 172.21.15.3:0/564707529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3d37ef9b0 con 0x55d3d340d390 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.218+0000 7f75467fc640 1 -- 172.21.15.3:0/564707529 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7558036030 con 0x55d3d340d390 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.218+0000 7f75467fc640 1 -- 172.21.15.3:0/564707529 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f754801b440 con 0x55d3d340d390 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.219+0000 7f756b82b740 10 client.25808 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.219+0000 7f756b82b740 10 client.25808 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:17:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.219+0000 7f756b82b740 20 client.25808 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.219+0000 7f756b82b740 1 --2- 172.21.15.3:0/564707529 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d3d33ec9f0 0x55d3d37f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.219+0000 7f756b82b740 4 client.25808 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.219+0000 7f756b82b740 1 -- 172.21.15.3:0/564707529 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3d374ddf0 con 0x55d3d33ec9f0 2024-09-21T06:17:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.219+0000 7f755f7fe640 1 --2- 172.21.15.3:0/564707529 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d3d33ec9f0 0x55d3d37f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.219+0000 7f755f7fe640 1 --2- 172.21.15.3:0/564707529 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d3d33ec9f0 0x55d3d37f2f70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb458466d4dc5bc5 server_cookie=ad4b0e03ffe0d81a in_seq=0 out_seq=0 2024-09-21T06:17:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.219+0000 7f75467fc640 10 client.25808 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:17:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f75467fc640 1 -- 172.21.15.3:0/564707529 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d3d374ddf0 con 0x55d3d33ec9f0 2024-09-21T06:17:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f75467fc640 10 client.25808 handle_command_reply: tid=0 2024-09-21T06:17:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f756b82b740 1 client.25808 shutdown 2024-09-21T06:17:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f756b82b740 1 -- 172.21.15.3:0/564707529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 msgr2=0x55d3d37e1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f756b82b740 1 --2- 172.21.15.3:0/564707529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e1240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7548009920 tx=0x7f7548009de0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f756b82b740 1 -- 172.21.15.3:0/564707529 shutdown_connections 2024-09-21T06:17:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f756b82b740 1 --2- 172.21.15.3:0/564707529 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d3d33ec9f0 0x55d3d37f2f70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f756b82b740 1 --2- 172.21.15.3:0/564707529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d342e070 0x55d3d37ecf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f75467fc640 0 client.25808 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:17:15.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f756b82b740 1 --2- 172.21.15.3:0/564707529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e1240 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f756b82b740 1 --2- 172.21.15.3:0/564707529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d34051b0 0x55d3d37e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f756b82b740 1 -- 172.21.15.3:0/564707529 >> 172.21.15.3:0/564707529 conn(0x55d3d3422460 msgr2=0x55d3d37d8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:15.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f756b82b740 1 -- 172.21.15.3:0/564707529 shutdown_connections 2024-09-21T06:17:15.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f756b82b740 1 -- 172.21.15.3:0/564707529 wait complete. 2024-09-21T06:17:15.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.220+0000 7f756b82b740 20 client.25808 trim_cache size 0 max 16384 2024-09-21T06:17:15.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.235+0000 7f756b82b740 1 Processor -- start 2024-09-21T06:17:15.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.235+0000 7f756b82b740 1 -- start start 2024-09-21T06:17:15.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.235+0000 7f756b82b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d34051b0 0x55d3d37d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.235+0000 7f756b82b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.235+0000 7f756b82b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d342e070 0x55d3d37ece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.235+0000 7f756b82b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f75601ad490 con 0x55d3d342e070 2024-09-21T06:17:15.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.235+0000 7f756b82b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f75601af030 con 0x55d3d340d390 2024-09-21T06:17:15.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.235+0000 7f756b82b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75601adb60 con 0x55d3d34051b0 2024-09-21T06:17:15.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f755f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d342e070 0x55d3d37ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f7557fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f7557fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49574/0 (socket says 172.21.15.3:49574) 2024-09-21T06:17:15.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f7557fff640 1 -- 172.21.15.3:0/1375250140 learned_addr learned my addr 172.21.15.3:0/1375250140 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:15.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f755f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d342e070 0x55d3d37ece90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59226/0 (socket says 172.21.15.3:59226) 2024-09-21T06:17:15.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f755f7fe640 1 --2- 172.21.15.3:0/1375250140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d34051b0 0x55d3d37d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f7557fff640 1 -- 172.21.15.3:0/1375250140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d34051b0 msgr2=0x55d3d37d7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:15.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f7557fff640 1 --2- 172.21.15.3:0/1375250140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d34051b0 0x55d3d37d7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f7557fff640 1 -- 172.21.15.3:0/1375250140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d342e070 msgr2=0x55d3d37ece90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:15.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f7557fff640 1 --2- 172.21.15.3:0/1375250140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d342e070 0x55d3d37ece90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f7557fff640 1 -- 172.21.15.3:0/1375250140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3d37d7990 con 0x55d3d340d390 2024-09-21T06:17:15.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f7557fff640 1 --2- 172.21.15.3:0/1375250140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e2930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f754800a010 tx=0x7f7548009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c273e2d7851c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:15.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f7554ff9640 1 -- 172.21.15.3:0/1375250140 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548033070 con 0x55d3d340d390 2024-09-21T06:17:15.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f7554ff9640 1 -- 172.21.15.3:0/1375250140 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f754801baa0 con 0x55d3d340d390 2024-09-21T06:17:15.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.236+0000 7f7554ff9640 1 -- 172.21.15.3:0/1375250140 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75480035f0 con 0x55d3d340d390 2024-09-21T06:17:15.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.237+0000 7f756b82b740 1 -- 172.21.15.3:0/1375250140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 msgr2=0x55d3d37e2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:15.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.237+0000 7f756b82b740 1 --2- 172.21.15.3:0/1375250140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e2930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f754800a010 tx=0x7f7548009530 comp rx=0 tx=0).stop 2024-09-21T06:17:15.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.237+0000 7f756b82b740 1 -- 172.21.15.3:0/1375250140 shutdown_connections 2024-09-21T06:17:15.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.237+0000 7f756b82b740 1 --2- 172.21.15.3:0/1375250140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d342e070 0x55d3d37ece90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.237+0000 7f756b82b740 1 --2- 172.21.15.3:0/1375250140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e2930 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.237+0000 7f756b82b740 1 --2- 172.21.15.3:0/1375250140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d34051b0 0x55d3d37d7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.237+0000 7f756b82b740 1 -- 172.21.15.3:0/1375250140 >> 172.21.15.3:0/1375250140 conn(0x55d3d3422460 msgr2=0x55d3d37e2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:15.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.238+0000 7f756b82b740 1 -- 172.21.15.3:0/1375250140 shutdown_connections 2024-09-21T06:17:15.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.238+0000 7f756b82b740 1 -- 172.21.15.3:0/1375250140 wait complete. 2024-09-21T06:17:15.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.238+0000 7f756b82b740 1 -- start start 2024-09-21T06:17:15.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.238+0000 7f756b82b740 1 Processor -- start 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.238+0000 7f756b82b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d34051b0 0x55d3d37e5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.238+0000 7f756b82b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.238+0000 7f756b82b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d342e070 0x55d3d385c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f756b82b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f75601ae900 con 0x55d3d34051b0 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f755f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d34051b0 0x55d3d37e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f755f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d34051b0 0x55d3d37e5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45740/0 (socket says 172.21.15.3:45740) 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f7557fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f755f7fe640 1 -- 172.21.15.3:0/1812964322 learned_addr learned my addr 172.21.15.3:0/1812964322 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f756b82b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f75601ae230 con 0x55d3d340d390 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f756b82b740 1 -- 172.21.15.3:0/1812964322 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75601ab470 con 0x55d3d342e070 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f756b82b740 10 client.0 _ll_register_callbacks cb 0x7f7548010a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f755f7fe640 1 --2- 172.21.15.3:0/1812964322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d342e070 0x55d3d385c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f755f7fe640 1 -- 172.21.15.3:0/1812964322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d342e070 msgr2=0x55d3d385c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f755f7fe640 1 --2- 172.21.15.3:0/1812964322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d342e070 0x55d3d385c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f755f7fe640 1 -- 172.21.15.3:0/1812964322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 msgr2=0x55d3d37e5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:15.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f755f7fe640 1 --2- 172.21.15.3:0/1812964322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f755f7fe640 1 -- 172.21.15.3:0/1812964322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3d37e6020 con 0x55d3d34051b0 2024-09-21T06:17:15.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f7557fff640 1 --2- 172.21.15.3:0/1812964322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:15.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f755f7fe640 1 --2- 172.21.15.3:0/1812964322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d34051b0 0x55d3d37e5210 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f755802b0b0 tx=0x7f755802f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f036b48d6e51838 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:15.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.239+0000 7f75467fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:15.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f75467fc640 1 -- 172.21.15.3:0/1812964322 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558031d00 con 0x55d3d34051b0 2024-09-21T06:17:15.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f75467fc640 1 -- 172.21.15.3:0/1812964322 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f7558031450 con 0x55d3d34051b0 2024-09-21T06:17:15.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f756b82b740 1 -- 172.21.15.3:0/1812964322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f754801baa0 con 0x55d3d34051b0 2024-09-21T06:17:15.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f75467fc640 1 -- 172.21.15.3:0/1812964322 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7558031650 con 0x55d3d34051b0 2024-09-21T06:17:15.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f756b82b740 10 client.16230 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:15.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f756b82b740 1 -- 172.21.15.3:0/1812964322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3d385ded0 con 0x55d3d34051b0 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f75467fc640 1 -- 172.21.15.3:0/1812964322 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f754801baa0 con 0x55d3d34051b0 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f75467fc640 1 -- 172.21.15.3:0/1812964322 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7558043070 con 0x55d3d34051b0 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f756b82b740 10 client.16230 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f756b82b740 10 client.16230 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f756b82b740 20 client.16230 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f756b82b740 1 --2- 172.21.15.3:0/1812964322 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d3d33c1da0 0x55d3d38603d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f756b82b740 4 client.16230 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.240+0000 7f756b82b740 1 -- 172.21.15.3:0/1812964322 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3d37498a0 con 0x55d3d33c1da0 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f7557fff640 1 --2- 172.21.15.3:0/1812964322 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d3d33c1da0 0x55d3d38603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f7557fff640 1 --2- 172.21.15.3:0/1812964322 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d3d33c1da0 0x55d3d38603d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7df236c95a430c8 server_cookie=d76b51b3afd8277d in_seq=0 out_seq=0 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f75467fc640 10 client.16230 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f75467fc640 1 -- 172.21.15.3:0/1812964322 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3d37498a0 con 0x55d3d33c1da0 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f75467fc640 10 client.16230 handle_command_reply: tid=0 2024-09-21T06:17:15.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f756b82b740 1 client.16230 shutdown 2024-09-21T06:17:15.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f756b82b740 1 -- 172.21.15.3:0/1812964322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d34051b0 msgr2=0x55d3d37e5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:15.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f756b82b740 1 --2- 172.21.15.3:0/1812964322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d34051b0 0x55d3d37e5210 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f755802b0b0 tx=0x7f755802f410 comp rx=0 tx=0).stop 2024-09-21T06:17:15.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f756b82b740 1 -- 172.21.15.3:0/1812964322 shutdown_connections 2024-09-21T06:17:15.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f756b82b740 1 --2- 172.21.15.3:0/1812964322 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d3d33c1da0 0x55d3d38603d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f756b82b740 1 --2- 172.21.15.3:0/1812964322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3d342e070 0x55d3d385c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f756b82b740 1 --2- 172.21.15.3:0/1812964322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3d340d390 0x55d3d37e5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f756b82b740 1 --2- 172.21.15.3:0/1812964322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3d34051b0 0x55d3d37e5210 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f756b82b740 1 -- 172.21.15.3:0/1812964322 >> 172.21.15.3:0/1812964322 conn(0x55d3d3422460 msgr2=0x55d3d37e2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:15.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.241+0000 7f75467fc640 0 client.16230 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:17:15.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.242+0000 7f756b82b740 1 -- 172.21.15.3:0/1812964322 shutdown_connections 2024-09-21T06:17:15.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.242+0000 7f756b82b740 1 -- 172.21.15.3:0/1812964322 wait complete. 2024-09-21T06:17:15.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.242+0000 7f756b82b740 20 client.16230 trim_cache size 0 max 16384 2024-09-21T06:17:15.249 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:17:15.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.242+0000 7f75652b7640 1 -- 172.21.15.3:0/2224801272 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f754c0b6680 msgr2=0x7f754c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:15.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.242+0000 7f75652b7640 1 --2- 172.21.15.3:0/2224801272 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f754c0b6680 0x7f754c0b8b90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f75500aaad0 tx=0x7f75500a9040 comp rx=0 tx=0).stop 2024-09-21T06:17:15.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.242+0000 7f75652b7640 1 -- 172.21.15.3:0/2224801272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7560104a30 msgr2=0x7f756019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:15.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.242+0000 7f75652b7640 1 --2- 172.21.15.3:0/2224801272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7560104a30 0x7f756019a490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f755000ea60 tx=0x7f755000ef20 comp rx=0 tx=0).stop 2024-09-21T06:17:15.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.243+0000 7f75652b7640 1 -- 172.21.15.3:0/2224801272 shutdown_connections 2024-09-21T06:17:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.243+0000 7f75652b7640 1 --2- 172.21.15.3:0/2224801272 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f754c0b6680 0x7f754c0b8b90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.243+0000 7f75652b7640 1 --2- 172.21.15.3:0/2224801272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f756010de60 0x7f756019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.243+0000 7f75652b7640 1 --2- 172.21.15.3:0/2224801272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7560105380 0x7f756019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.243+0000 7f75652b7640 1 --2- 172.21.15.3:0/2224801272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7560104a30 0x7f756019a490 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.243+0000 7f75652b7640 1 -- 172.21.15.3:0/2224801272 >> 172.21.15.3:0/2224801272 conn(0x7f7560079790 msgr2=0x7f75601068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.243+0000 7f75652b7640 1 -- 172.21.15.3:0/2224801272 shutdown_connections 2024-09-21T06:17:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:15.243+0000 7f75652b7640 1 -- 172.21.15.3:0/2224801272 wait complete. 2024-09-21T06:17:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:15 smithi003 ceph-mon[31227]: pgmap v392: 97 pgs: 97 active+clean; 41 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 320 op/s 2024-09-21T06:17:15.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:15 smithi137 ceph-mon[34464]: pgmap v392: 97 pgs: 97 active+clean; 41 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 320 op/s 2024-09-21T06:17:15.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:15 smithi204 ceph-mon[34659]: pgmap v392: 97 pgs: 97 active+clean; 41 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 320 op/s 2024-09-21T06:17:15.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 127802 14.75 MB/sec warmup 48 sec latency 21.516 ms 2024-09-21T06:17:16.045 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:17:16.312 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.819+0000 7f637dfd9640 1 -- 172.21.15.3:0/3086743908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63781052b0 msgr2=0x7f637810d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.819+0000 7f637dfd9640 1 --2- 172.21.15.3:0/3086743908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63781052b0 0x7f637810d8f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6360009900 tx=0x7f636002a410 comp rx=0 tx=0).stop 2024-09-21T06:17:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.819+0000 7f637dfd9640 1 -- 172.21.15.3:0/3086743908 shutdown_connections 2024-09-21T06:17:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.819+0000 7f637dfd9640 1 --2- 172.21.15.3:0/3086743908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f637810de30 0x7f6378110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.819+0000 7f637dfd9640 1 --2- 172.21.15.3:0/3086743908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63781052b0 0x7f637810d8f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.819+0000 7f637dfd9640 1 --2- 172.21.15.3:0/3086743908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6378104880 0x7f6378104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.819+0000 7f637dfd9640 1 -- 172.21.15.3:0/3086743908 >> 172.21.15.3:0/3086743908 conn(0x7f6378079730 msgr2=0x7f6378079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.819+0000 7f637dfd9640 1 -- 172.21.15.3:0/3086743908 shutdown_connections 2024-09-21T06:17:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.819+0000 7f637dfd9640 1 -- 172.21.15.3:0/3086743908 wait complete. 2024-09-21T06:17:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.820+0000 7f637dfd9640 1 Processor -- start 2024-09-21T06:17:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.820+0000 7f637dfd9640 1 -- start start 2024-09-21T06:17:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.820+0000 7f637dfd9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6378104880 0x7f637819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.820+0000 7f637dfd9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63781052b0 0x7f637819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.820+0000 7f637dfd9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f637810de30 0x7f63781a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.820+0000 7f637dfd9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6378113a70 con 0x7f63781052b0 2024-09-21T06:17:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.820+0000 7f637dfd9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f63781138f0 con 0x7f6378104880 2024-09-21T06:17:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.820+0000 7f637dfd9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6378113bf0 con 0x7f637810de30 2024-09-21T06:17:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f636effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63781052b0 0x7f637819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f636effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63781052b0 0x7f637819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45758/0 (socket says 172.21.15.3:45758) 2024-09-21T06:17:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f636effd640 1 -- 172.21.15.3:0/1014674141 learned_addr learned my addr 172.21.15.3:0/1014674141 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f63777fe640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6378104880 0x7f637819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f6377fff640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f637810de30 0x7f63781a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f636effd640 1 -- 172.21.15.3:0/1014674141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f637810de30 msgr2=0x7f63781a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f636effd640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f637810de30 0x7f63781a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f636effd640 1 -- 172.21.15.3:0/1014674141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6378104880 msgr2=0x7f637819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f636effd640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6378104880 0x7f637819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f636effd640 1 -- 172.21.15.3:0/1014674141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63781a2610 con 0x7f63781052b0 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f63777fe640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6378104880 0x7f637819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f636effd640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63781052b0 0x7f637819cc40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6360032fd0 tx=0x7f6360031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f63757fa640 1 -- 172.21.15.3:0/1014674141 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636003f030 con 0x7f63781052b0 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f63757fa640 1 -- 172.21.15.3:0/1014674141 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6360004490 con 0x7f63781052b0 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f637dfd9640 1 -- 172.21.15.3:0/1014674141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63781a28a0 con 0x7f63781052b0 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.821+0000 7f63757fa640 1 -- 172.21.15.3:0/1014674141 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360035070 con 0x7f63781052b0 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.822+0000 7f637dfd9640 1 -- 172.21.15.3:0/1014674141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63781a9060 con 0x7f63781052b0 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.824+0000 7f638454d740 1 Processor -- start 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.824+0000 7f63757fa640 1 -- 172.21.15.3:0/1014674141 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f6360042030 con 0x7f63781052b0 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.824+0000 7f638454d740 1 -- start start 2024-09-21T06:17:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.824+0000 7f638454d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b4895070 0x55c0b4c3c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.824+0000 7f638454d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b486c1b0 0x55c0b4c3c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.824+0000 7f638454d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4874390 0x55c0b4c3cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.824+0000 7f638454d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c0b4bb2ea0 con 0x55c0b4874390 2024-09-21T06:17:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.824+0000 7f638454d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0b4ba22d0 con 0x55c0b4895070 2024-09-21T06:17:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.824+0000 7f638454d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c0b4ba60e0 con 0x55c0b486c1b0 2024-09-21T06:17:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.825+0000 7f6377fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b4895070 0x55c0b4c3c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.825+0000 7f63777fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b486c1b0 0x55c0b4c3c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.825+0000 7f636effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4874390 0x55c0b4c3cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.825+0000 7f6377fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b4895070 0x55c0b4c3c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:44776/0 (socket says 172.21.15.3:44776) 2024-09-21T06:17:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.825+0000 7f6377fff640 1 -- 172.21.15.3:0/2954831289 learned_addr learned my addr 172.21.15.3:0/2954831289 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.825+0000 7f63777fe640 1 -- 172.21.15.3:0/2954831289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b4895070 msgr2=0x55c0b4c3c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.825+0000 7f63777fe640 1 --2- 172.21.15.3:0/2954831289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b4895070 0x55c0b4c3c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.825+0000 7f63777fe640 1 -- 172.21.15.3:0/2954831289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4874390 msgr2=0x55c0b4c3cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:16.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.825+0000 7f63777fe640 1 --2- 172.21.15.3:0/2954831289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4874390 0x55c0b4c3cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.825+0000 7f63777fe640 1 -- 172.21.15.3:0/2954831289 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b4900650 con 0x55c0b486c1b0 2024-09-21T06:17:16.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.825+0000 7f63777fe640 1 --2- 172.21.15.3:0/2954831289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b486c1b0 0x55c0b4c3c800 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f637806bbc0 tx=0x7f6368011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed5103d28e6397e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:16.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.825+0000 7f63757fa640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f63640b6680 0x7f63640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.826+0000 7f636dffb640 1 -- 172.21.15.3:0/2954831289 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368003c30 con 0x55c0b486c1b0 2024-09-21T06:17:16.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.826+0000 7f6377fff640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f63640b6680 0x7f63640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.826+0000 7f636dffb640 1 -- 172.21.15.3:0/2954831289 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f636800fe20 con 0x55c0b486c1b0 2024-09-21T06:17:16.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.826+0000 7f63757fa640 1 -- 172.21.15.3:0/1014674141 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f63601079f0 con 0x7f63781052b0 2024-09-21T06:17:16.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.826+0000 7f636dffb640 1 -- 172.21.15.3:0/2954831289 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63680193e0 con 0x55c0b486c1b0 2024-09-21T06:17:16.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.827+0000 7f638454d740 1 -- 172.21.15.3:0/2954831289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b486c1b0 msgr2=0x55c0b4c3c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:16.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.827+0000 7f638454d740 1 --2- 172.21.15.3:0/2954831289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b486c1b0 0x55c0b4c3c800 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f637806bbc0 tx=0x7f6368011c70 comp rx=0 tx=0).stop 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.827+0000 7f638454d740 1 -- 172.21.15.3:0/2954831289 shutdown_connections 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.827+0000 7f638454d740 1 --2- 172.21.15.3:0/2954831289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b486c1b0 0x55c0b4c3c800 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.827+0000 7f638454d740 1 --2- 172.21.15.3:0/2954831289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4874390 0x55c0b4c3cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.827+0000 7f638454d740 1 --2- 172.21.15.3:0/2954831289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b4895070 0x55c0b4c3c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.827+0000 7f638454d740 1 -- 172.21.15.3:0/2954831289 >> 172.21.15.3:0/2954831289 conn(0x55c0b4889460 msgr2=0x55c0b4c3a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.827+0000 7f6377fff640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f63640b6680 0x7f63640b8b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6370009ee0 tx=0x7f6370005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.827+0000 7f638454d740 1 -- 172.21.15.3:0/2954831289 shutdown_connections 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.828+0000 7f638454d740 1 -- 172.21.15.3:0/2954831289 wait complete. 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.828+0000 7f638454d740 1 -- start start 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.828+0000 7f638454d740 1 Processor -- start 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f638454d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b486c1b0 0x55c0b4c478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f638454d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4874390 0x55c0b4c47e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f638454d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b4895070 0x55c0b4c48380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f638454d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c0b4a03700 con 0x55c0b4874390 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f638454d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0b4b3d160 con 0x55c0b486c1b0 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f638454d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c0b4c4aaf0 con 0x55c0b4895070 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f638454d740 10 client.0 _ll_register_callbacks cb 0x55c0b4b569e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f636effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4874390 0x55c0b4c47e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f636effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4874390 0x55c0b4c47e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45768/0 (socket says 172.21.15.3:45768) 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f636effd640 1 -- 172.21.15.3:0/3962433319 learned_addr learned my addr 172.21.15.3:0/3962433319 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f63777fe640 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b486c1b0 0x55c0b4c478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f6377fff640 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b4895070 0x55c0b4c48380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f63777fe640 1 -- 172.21.15.3:0/3962433319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b4895070 msgr2=0x55c0b4c48380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f63777fe640 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b4895070 0x55c0b4c48380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f63777fe640 1 -- 172.21.15.3:0/3962433319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4874390 msgr2=0x55c0b4c47e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f63777fe640 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4874390 0x55c0b4c47e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f63777fe640 1 -- 172.21.15.3:0/3962433319 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b4b6ac90 con 0x55c0b486c1b0 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f6377fff640 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b4895070 0x55c0b4c48380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:17:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f636effd640 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4874390 0x55c0b4c47e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f63777fe640 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b486c1b0 0x55c0b4c478e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6368000f80 tx=0x7f63680114b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f975f03f760d4cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.829+0000 7f6356ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f6356ffd640 1 -- 172.21.15.3:0/3962433319 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368028020 con 0x55c0b486c1b0 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f638454d740 1 -- 172.21.15.3:0/3962433319 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f636800fe20 con 0x55c0b486c1b0 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f6356ffd640 1 -- 172.21.15.3:0/3962433319 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f636800f970 con 0x55c0b486c1b0 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f6356ffd640 1 -- 172.21.15.3:0/3962433319 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6368033050 con 0x55c0b486c1b0 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f638454d740 10 client.25826 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f638454d740 1 -- 172.21.15.3:0/3962433319 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0b4c567a0 con 0x55c0b486c1b0 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f6356ffd640 1 -- 172.21.15.3:0/3962433319 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f636800fe20 con 0x55c0b486c1b0 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f6356ffd640 1 -- 172.21.15.3:0/3962433319 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6368018070 con 0x55c0b486c1b0 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f638454d740 10 client.25826 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f638454d740 10 client.25826 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f638454d740 20 client.25826 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f638454d740 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55c0b48539f0 0x55c0b4c5a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f638454d740 4 client.25826 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.830+0000 7f638454d740 1 -- 172.21.15.3:0/3962433319 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0b4bb4df0 con 0x55c0b48539f0 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.831+0000 7f636effd640 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55c0b48539f0 0x55c0b4c5a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.831+0000 7f6356ffd640 10 client.25826 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.831+0000 7f636effd640 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55c0b48539f0 0x55c0b4c5a060 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f9beee26e53643c server_cookie=60b2f3b38670f7d2 in_seq=0 out_seq=0 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f6356ffd640 1 -- 172.21.15.3:0/3962433319 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c0b4bb4df0 con 0x55c0b48539f0 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f6356ffd640 10 client.25826 handle_command_reply: tid=0 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f638454d740 1 client.25826 shutdown 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f638454d740 1 -- 172.21.15.3:0/3962433319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b486c1b0 msgr2=0x55c0b4c478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f638454d740 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b486c1b0 0x55c0b4c478e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6368000f80 tx=0x7f63680114b0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f638454d740 1 -- 172.21.15.3:0/3962433319 shutdown_connections 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f638454d740 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55c0b48539f0 0x55c0b4c5a060 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f638454d740 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b4895070 0x55c0b4c48380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f638454d740 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4874390 0x55c0b4c47e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f6356ffd640 0 client.25826 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:17:16.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f638454d740 1 --2- 172.21.15.3:0/3962433319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b486c1b0 0x55c0b4c478e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f638454d740 1 -- 172.21.15.3:0/3962433319 >> 172.21.15.3:0/3962433319 conn(0x55c0b4889460 msgr2=0x55c0b4c3f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:16.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f638454d740 1 -- 172.21.15.3:0/3962433319 shutdown_connections 2024-09-21T06:17:16.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f638454d740 1 -- 172.21.15.3:0/3962433319 wait complete. 2024-09-21T06:17:16.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.832+0000 7f638454d740 20 client.25826 trim_cache size 0 max 16384 2024-09-21T06:17:16.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f638454d740 1 Processor -- start 2024-09-21T06:17:16.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f638454d740 1 -- start start 2024-09-21T06:17:16.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f638454d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b486c1b0 0x55c0b4c47da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f638454d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b4874390 0x55c0b4c39760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f638454d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4895070 0x55c0b4c39ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f638454d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f63781b1850 con 0x55c0b4895070 2024-09-21T06:17:16.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f638454d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f63781b33f0 con 0x55c0b486c1b0 2024-09-21T06:17:16.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f638454d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63781b1f20 con 0x55c0b4874390 2024-09-21T06:17:16.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f6377fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4895070 0x55c0b4c39ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f63777fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b486c1b0 0x55c0b4c47da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f6377fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4895070 0x55c0b4c39ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45772/0 (socket says 172.21.15.3:45772) 2024-09-21T06:17:16.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f6377fff640 1 -- 172.21.15.3:0/789568173 learned_addr learned my addr 172.21.15.3:0/789568173 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:16.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f636effd640 1 --2- 172.21.15.3:0/789568173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b4874390 0x55c0b4c39760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f63777fe640 1 -- 172.21.15.3:0/789568173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b4874390 msgr2=0x55c0b4c39760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:16.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f63777fe640 1 --2- 172.21.15.3:0/789568173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b4874390 0x55c0b4c39760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f63777fe640 1 -- 172.21.15.3:0/789568173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4895070 msgr2=0x55c0b4c39ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:16.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f63777fe640 1 --2- 172.21.15.3:0/789568173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4895070 0x55c0b4c39ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f63777fe640 1 -- 172.21.15.3:0/789568173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b4c53d10 con 0x55c0b486c1b0 2024-09-21T06:17:16.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f636effd640 1 --2- 172.21.15.3:0/789568173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b4874390 0x55c0b4c39760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:17:16.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.847+0000 7f6377fff640 1 --2- 172.21.15.3:0/789568173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4895070 0x55c0b4c39ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:16.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.848+0000 7f63777fe640 1 --2- 172.21.15.3:0/789568173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b486c1b0 0x55c0b4c47da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6368006010 tx=0x7f6368009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b1030ae7b9d4f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:16.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.848+0000 7f636d7fa640 1 -- 172.21.15.3:0/789568173 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368028040 con 0x55c0b486c1b0 2024-09-21T06:17:16.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.848+0000 7f636d7fa640 1 -- 172.21.15.3:0/789568173 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6368004420 con 0x55c0b486c1b0 2024-09-21T06:17:16.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.848+0000 7f636d7fa640 1 -- 172.21.15.3:0/789568173 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368003c90 con 0x55c0b486c1b0 2024-09-21T06:17:16.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.849+0000 7f638454d740 1 -- 172.21.15.3:0/789568173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b486c1b0 msgr2=0x55c0b4c47da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:16.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.849+0000 7f638454d740 1 --2- 172.21.15.3:0/789568173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b486c1b0 0x55c0b4c47da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6368006010 tx=0x7f6368009f60 comp rx=0 tx=0).stop 2024-09-21T06:17:16.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.849+0000 7f638454d740 1 -- 172.21.15.3:0/789568173 shutdown_connections 2024-09-21T06:17:16.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.849+0000 7f638454d740 1 --2- 172.21.15.3:0/789568173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b4874390 0x55c0b4c39760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.849+0000 7f638454d740 1 --2- 172.21.15.3:0/789568173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4895070 0x55c0b4c39ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.849+0000 7f638454d740 1 --2- 172.21.15.3:0/789568173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b486c1b0 0x55c0b4c47da0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.849+0000 7f638454d740 1 -- 172.21.15.3:0/789568173 >> 172.21.15.3:0/789568173 conn(0x55c0b4889460 msgr2=0x55c0b4c47690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.849+0000 7f638454d740 1 -- 172.21.15.3:0/789568173 shutdown_connections 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.849+0000 7f638454d740 1 -- 172.21.15.3:0/789568173 wait complete. 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.849+0000 7f638454d740 1 -- start start 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.850+0000 7f638454d740 1 Processor -- start 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.850+0000 7f638454d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b486c1b0 0x55c0b4c4b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.850+0000 7f638454d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b4874390 0x55c0b4c4b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.850+0000 7f638454d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4895070 0x55c0b4cc2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.850+0000 7f638454d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f63781b2cc0 con 0x55c0b4895070 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.850+0000 7f638454d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f63781b25f0 con 0x55c0b4874390 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.850+0000 7f638454d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63781af830 con 0x55c0b486c1b0 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.850+0000 7f638454d740 10 client.0 _ll_register_callbacks cb 0x7f636010b390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:16.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.850+0000 7f6377fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4895070 0x55c0b4cc2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.850+0000 7f6377fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4895070 0x55c0b4cc2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45782/0 (socket says 172.21.15.3:45782) 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.850+0000 7f636effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b4874390 0x55c0b4c4b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.850+0000 7f6377fff640 1 -- 172.21.15.3:0/2145343502 learned_addr learned my addr 172.21.15.3:0/2145343502 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.850+0000 7f63777fe640 1 --2- 172.21.15.3:0/2145343502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b486c1b0 0x55c0b4c4b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.851+0000 7f636effd640 1 -- 172.21.15.3:0/2145343502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b486c1b0 msgr2=0x55c0b4c4b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.851+0000 7f636effd640 1 --2- 172.21.15.3:0/2145343502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b486c1b0 0x55c0b4c4b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.851+0000 7f636effd640 1 -- 172.21.15.3:0/2145343502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4895070 msgr2=0x55c0b4cc2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.851+0000 7f636effd640 1 --2- 172.21.15.3:0/2145343502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4895070 0x55c0b4cc2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.851+0000 7f636effd640 1 -- 172.21.15.3:0/2145343502 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b4c4bf50 con 0x55c0b4874390 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.851+0000 7f636effd640 1 --2- 172.21.15.3:0/2145343502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b4874390 0x55c0b4c4b680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f63601051e0 tx=0x7f636002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c357c24c5fd92c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.851+0000 7f6356ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.851+0000 7f6356ffd640 1 -- 172.21.15.3:0/2145343502 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360049040 con 0x55c0b4874390 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.851+0000 7f6356ffd640 1 -- 172.21.15.3:0/2145343502 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f63600d6e10 con 0x55c0b4874390 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.851+0000 7f638454d740 1 -- 172.21.15.3:0/2145343502 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6368004420 con 0x55c0b4874390 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.852+0000 7f6356ffd640 1 -- 172.21.15.3:0/2145343502 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63600cd280 con 0x55c0b4874390 2024-09-21T06:17:16.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.852+0000 7f638454d740 10 client.25832 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:16.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.852+0000 7f638454d740 1 -- 172.21.15.3:0/2145343502 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0b4cc4030 con 0x55c0b4874390 2024-09-21T06:17:16.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.852+0000 7f6356ffd640 1 -- 172.21.15.3:0/2145343502 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6368004420 con 0x55c0b4874390 2024-09-21T06:17:16.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.852+0000 7f6356ffd640 1 -- 172.21.15.3:0/2145343502 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f63600cebb0 con 0x55c0b4874390 2024-09-21T06:17:16.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.852+0000 7f638454d740 10 client.25832 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:16.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.852+0000 7f638454d740 10 client.25832 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:17:16.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.852+0000 7f638454d740 20 client.25832 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:16.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.852+0000 7f638454d740 1 --2- 172.21.15.3:0/2145343502 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55c0b4828da0 0x55c0b4cc6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:16.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.852+0000 7f638454d740 4 client.25832 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:17:16.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.852+0000 7f638454d740 1 -- 172.21.15.3:0/2145343502 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0b4bb08a0 con 0x55c0b4828da0 2024-09-21T06:17:16.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.852+0000 7f63777fe640 1 --2- 172.21.15.3:0/2145343502 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55c0b4828da0 0x55c0b4cc6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:16.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f63777fe640 1 --2- 172.21.15.3:0/2145343502 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55c0b4828da0 0x55c0b4cc6770 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=decec97338dfcaec server_cookie=a5472bdeb208408a in_seq=0 out_seq=0 2024-09-21T06:17:16.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f6356ffd640 10 client.25832 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:17:16.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f6356ffd640 1 -- 172.21.15.3:0/2145343502 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0b4bb08a0 con 0x55c0b4828da0 2024-09-21T06:17:16.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f6356ffd640 10 client.25832 handle_command_reply: tid=0 2024-09-21T06:17:16.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f638454d740 1 client.25832 shutdown 2024-09-21T06:17:16.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f638454d740 1 -- 172.21.15.3:0/2145343502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b4874390 msgr2=0x55c0b4c4b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:16.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f638454d740 1 --2- 172.21.15.3:0/2145343502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b4874390 0x55c0b4c4b680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f63601051e0 tx=0x7f636002a410 comp rx=0 tx=0).stop 2024-09-21T06:17:16.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f638454d740 1 -- 172.21.15.3:0/2145343502 shutdown_connections 2024-09-21T06:17:16.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f638454d740 1 --2- 172.21.15.3:0/2145343502 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55c0b4828da0 0x55c0b4cc6770 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f6356ffd640 0 client.25832 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:17:16.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f638454d740 1 --2- 172.21.15.3:0/2145343502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0b4895070 0x55c0b4cc2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f638454d740 1 --2- 172.21.15.3:0/2145343502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0b4874390 0x55c0b4c4b680 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f638454d740 1 --2- 172.21.15.3:0/2145343502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0b486c1b0 0x55c0b4c4b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f638454d740 1 -- 172.21.15.3:0/2145343502 >> 172.21.15.3:0/2145343502 conn(0x55c0b4889460 msgr2=0x55c0b4c3d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:16.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f638454d740 1 -- 172.21.15.3:0/2145343502 shutdown_connections 2024-09-21T06:17:16.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f638454d740 1 -- 172.21.15.3:0/2145343502 wait complete. 2024-09-21T06:17:16.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.853+0000 7f638454d740 20 client.25832 trim_cache size 0 max 16384 2024-09-21T06:17:16.860 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:17:16.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.854+0000 7f637dfd9640 1 -- 172.21.15.3:0/1014674141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f63640b6680 msgr2=0x7f63640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:16.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.854+0000 7f637dfd9640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f63640b6680 0x7f63640b8b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6370009ee0 tx=0x7f6370005f90 comp rx=0 tx=0).stop 2024-09-21T06:17:16.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.854+0000 7f637dfd9640 1 -- 172.21.15.3:0/1014674141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63781052b0 msgr2=0x7f637819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:16.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.854+0000 7f637dfd9640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63781052b0 0x7f637819cc40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6360032fd0 tx=0x7f6360031a20 comp rx=0 tx=0).stop 2024-09-21T06:17:16.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.854+0000 7f637dfd9640 1 -- 172.21.15.3:0/1014674141 shutdown_connections 2024-09-21T06:17:16.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.854+0000 7f637dfd9640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f63640b6680 0x7f63640b8b90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.854+0000 7f637dfd9640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f637810de30 0x7f63781a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.854+0000 7f637dfd9640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63781052b0 0x7f637819cc40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.854+0000 7f637dfd9640 1 --2- 172.21.15.3:0/1014674141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6378104880 0x7f637819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:16.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.854+0000 7f637dfd9640 1 -- 172.21.15.3:0/1014674141 >> 172.21.15.3:0/1014674141 conn(0x7f6378079730 msgr2=0x7f63780ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:16.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.854+0000 7f637dfd9640 1 -- 172.21.15.3:0/1014674141 shutdown_connections 2024-09-21T06:17:16.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:16.854+0000 7f637dfd9640 1 -- 172.21.15.3:0/1014674141 wait complete. 2024-09-21T06:17:16.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 129422 14.59 MB/sec warmup 49 sec latency 39.691 ms 2024-09-21T06:17:17.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:17 smithi003 ceph-mon[31227]: pgmap v393: 97 pgs: 97 active+clean; 49 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.3 MiB/s wr, 405 op/s 2024-09-21T06:17:17.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:17 smithi137 ceph-mon[34464]: pgmap v393: 97 pgs: 97 active+clean; 49 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.3 MiB/s wr, 405 op/s 2024-09-21T06:17:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:17 smithi204 ceph-mon[34659]: pgmap v393: 97 pgs: 97 active+clean; 49 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.3 MiB/s wr, 405 op/s 2024-09-21T06:17:17.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:17:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.689+0000 7fce76913640 1 -- 172.21.15.3:0/2456241903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce70108310 msgr2=0x7fce70108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.689+0000 7fce76913640 1 --2- 172.21.15.3:0/2456241903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce70108310 0x7fce70108720 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fce64009900 tx=0x7fce6402a410 comp rx=0 tx=0).stop 2024-09-21T06:17:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.689+0000 7fce76913640 1 -- 172.21.15.3:0/2456241903 shutdown_connections 2024-09-21T06:17:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.689+0000 7fce76913640 1 --2- 172.21.15.3:0/2456241903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce70108c60 0x7fce7010f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.689+0000 7fce76913640 1 --2- 172.21.15.3:0/2456241903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce70108310 0x7fce70108720 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.689+0000 7fce76913640 1 --2- 172.21.15.3:0/2456241903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce701029e0 0x7fce70102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.689+0000 7fce76913640 1 -- 172.21.15.3:0/2456241903 >> 172.21.15.3:0/2456241903 conn(0x7fce700fe3e0 msgr2=0x7fce70100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.689+0000 7fce76913640 1 -- 172.21.15.3:0/2456241903 shutdown_connections 2024-09-21T06:17:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.689+0000 7fce76913640 1 -- 172.21.15.3:0/2456241903 wait complete. 2024-09-21T06:17:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.690+0000 7fce76913640 1 Processor -- start 2024-09-21T06:17:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.690+0000 7fce76913640 1 -- start start 2024-09-21T06:17:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.690+0000 7fce76913640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce701029e0 0x7fce700fe910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.690+0000 7fce76913640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce70108310 0x7fce700fee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce76913640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce70108c60 0x7fce700ff390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce76913640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fce70112920 con 0x7fce70108310 2024-09-21T06:17:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce76913640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce701127a0 con 0x7fce70108c60 2024-09-21T06:17:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce76913640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce70112aa0 con 0x7fce701029e0 2024-09-21T06:17:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce6ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce701029e0 0x7fce700fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce6f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce70108310 0x7fce700fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce6ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce701029e0 0x7fce700fe910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:60822/0 (socket says 172.21.15.3:60822) 2024-09-21T06:17:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce6ffff640 1 -- 172.21.15.3:0/1990249278 learned_addr learned my addr 172.21.15.3:0/1990249278 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce74eb0640 1 --2- 172.21.15.3:0/1990249278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce70108c60 0x7fce700ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce6ffff640 1 -- 172.21.15.3:0/1990249278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce70108c60 msgr2=0x7fce700ff390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce6ffff640 1 --2- 172.21.15.3:0/1990249278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce70108c60 0x7fce700ff390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce6ffff640 1 -- 172.21.15.3:0/1990249278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce70108310 msgr2=0x7fce700fee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce6ffff640 1 --2- 172.21.15.3:0/1990249278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce70108310 0x7fce700fee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce6ffff640 1 -- 172.21.15.3:0/1990249278 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce701a8780 con 0x7fce701029e0 2024-09-21T06:17:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce6ffff640 1 --2- 172.21.15.3:0/1990249278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce701029e0 0x7fce700fe910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce6000e9f0 tx=0x7fce6000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.691+0000 7fce6d7fa640 1 -- 172.21.15.3:0/1990249278 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6000cd80 con 0x7fce701029e0 2024-09-21T06:17:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.692+0000 7fce76913640 1 -- 172.21.15.3:0/1990249278 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce701a8a70 con 0x7fce701029e0 2024-09-21T06:17:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.692+0000 7fce6d7fa640 1 -- 172.21.15.3:0/1990249278 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce600108e0 con 0x7fce701029e0 2024-09-21T06:17:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.692+0000 7fce6d7fa640 1 -- 172.21.15.3:0/1990249278 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60018980 con 0x7fce701029e0 2024-09-21T06:17:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.692+0000 7fce76913640 1 -- 172.21.15.3:0/1990249278 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce701a8fe0 con 0x7fce701029e0 2024-09-21T06:17:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.694+0000 7fce7d876740 1 Processor -- start 2024-09-21T06:17:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.694+0000 7fce7d876740 1 -- start start 2024-09-21T06:17:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.694+0000 7fce6d7fa640 1 -- 172.21.15.3:0/1990249278 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fce60018b20 con 0x7fce701029e0 2024-09-21T06:17:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce7d876740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1ae1b0 0x56288e5539b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce7d876740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e1852f0 0x56288e553ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce7d876740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e18d4d0 0x56288e554430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce7d876740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56288e4c86f0 con 0x56288e18d4d0 2024-09-21T06:17:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce7d876740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56288e4b7b20 con 0x56288e1ae1b0 2024-09-21T06:17:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce7d876740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56288e4bb930 con 0x56288e1852f0 2024-09-21T06:17:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce74eb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1ae1b0 0x56288e5539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce6ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e1852f0 0x56288e553ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce6f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e18d4d0 0x56288e554430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce74eb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1ae1b0 0x56288e5539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:44818/0 (socket says 172.21.15.3:44818) 2024-09-21T06:17:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce74eb0640 1 -- 172.21.15.3:0/78169419 learned_addr learned my addr 172.21.15.3:0/78169419 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce6f7fe640 1 --2- 172.21.15.3:0/78169419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e18d4d0 0x56288e554430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45794/0 (socket says 172.21.15.3:45794) 2024-09-21T06:17:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce74eb0640 1 -- 172.21.15.3:0/78169419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e1852f0 msgr2=0x56288e553ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce74eb0640 1 --2- 172.21.15.3:0/78169419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e1852f0 0x56288e553ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce74eb0640 1 -- 172.21.15.3:0/78169419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e18d4d0 msgr2=0x56288e554430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce74eb0640 1 --2- 172.21.15.3:0/78169419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e18d4d0 0x56288e554430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce74eb0640 1 -- 172.21.15.3:0/78169419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56288e47d530 con 0x56288e1ae1b0 2024-09-21T06:17:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce6d7fa640 1 --2- 172.21.15.3:0/1990249278 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fce580b66b0 0x7fce580b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.695+0000 7fce74eb0640 1 --2- 172.21.15.3:0/78169419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1ae1b0 0x56288e5539b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce68009f80 tx=0x7fce680118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c48a205efe81528d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.696+0000 7fce6ffff640 1 --2- 172.21.15.3:0/1990249278 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fce580b66b0 0x7fce580b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.696+0000 7fce567fc640 1 -- 172.21.15.3:0/78169419 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68004d20 con 0x56288e1ae1b0 2024-09-21T06:17:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.696+0000 7fce6d7fa640 1 -- 172.21.15.3:0/1990249278 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fce600e79c0 con 0x7fce701029e0 2024-09-21T06:17:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.696+0000 7fce567fc640 1 -- 172.21.15.3:0/78169419 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce68004050 con 0x56288e1ae1b0 2024-09-21T06:17:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.696+0000 7fce567fc640 1 -- 172.21.15.3:0/78169419 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68005280 con 0x56288e1ae1b0 2024-09-21T06:17:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.697+0000 7fce7d876740 1 -- 172.21.15.3:0/78169419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1ae1b0 msgr2=0x56288e5539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.697+0000 7fce7d876740 1 --2- 172.21.15.3:0/78169419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1ae1b0 0x56288e5539b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce68009f80 tx=0x7fce680118d0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.697+0000 7fce6ffff640 1 --2- 172.21.15.3:0/1990249278 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fce580b66b0 0x7fce580b8bc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fce600e5200 tx=0x7fce600eb510 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.697+0000 7fce7d876740 1 -- 172.21.15.3:0/78169419 shutdown_connections 2024-09-21T06:17:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.697+0000 7fce7d876740 1 --2- 172.21.15.3:0/78169419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e1852f0 0x56288e553ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.697+0000 7fce7d876740 1 --2- 172.21.15.3:0/78169419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e18d4d0 0x56288e554430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.697+0000 7fce7d876740 1 --2- 172.21.15.3:0/78169419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1ae1b0 0x56288e5539b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.697+0000 7fce7d876740 1 -- 172.21.15.3:0/78169419 >> 172.21.15.3:0/78169419 conn(0x56288e1a2300 msgr2=0x56288e5518a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.697+0000 7fce7d876740 1 -- 172.21.15.3:0/78169419 shutdown_connections 2024-09-21T06:17:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.697+0000 7fce7d876740 1 -- 172.21.15.3:0/78169419 wait complete. 2024-09-21T06:17:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.698+0000 7fce7d876740 1 -- start start 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.698+0000 7fce7d876740 1 Processor -- start 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.698+0000 7fce7d876740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1852f0 0x56288e55f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.698+0000 7fce7d876740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e18d4d0 0x56288e55f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.698+0000 7fce7d876740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 0x56288e56b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.698+0000 7fce7d876740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56288e3efab0 con 0x56288e1ae1b0 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.698+0000 7fce7d876740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56288e45d730 con 0x56288e1852f0 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.698+0000 7fce7d876740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56288e562130 con 0x56288e18d4d0 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.698+0000 7fce7d876740 10 client.0 _ll_register_callbacks cb 0x56288e4b16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.698+0000 7fce74eb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1852f0 0x56288e55f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.698+0000 7fce74eb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1852f0 0x56288e55f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:44826/0 (socket says 172.21.15.3:44826) 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.698+0000 7fce74eb0640 1 -- 172.21.15.3:0/2607347749 learned_addr learned my addr 172.21.15.3:0/2607347749 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.698+0000 7fce74eb0640 1 --2- 172.21.15.3:0/2607347749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 0x56288e56b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.699+0000 7fce6f7fe640 1 --2- 172.21.15.3:0/2607347749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e18d4d0 0x56288e55f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.699+0000 7fce74eb0640 1 -- 172.21.15.3:0/2607347749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e18d4d0 msgr2=0x56288e55f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.699+0000 7fce74eb0640 1 --2- 172.21.15.3:0/2607347749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e18d4d0 0x56288e55f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.699+0000 7fce74eb0640 1 -- 172.21.15.3:0/2607347749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 msgr2=0x56288e56b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.699+0000 7fce74eb0640 1 --2- 172.21.15.3:0/2607347749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 0x56288e56b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.699+0000 7fce74eb0640 1 -- 172.21.15.3:0/2607347749 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56288e219790 con 0x56288e1852f0 2024-09-21T06:17:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.699+0000 7fce74eb0640 1 --2- 172.21.15.3:0/2607347749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1852f0 0x56288e55f050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce68003fd0 tx=0x7fce68004950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5e55603cf526576 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.699+0000 7fce4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.699+0000 7fce4f7fe640 1 -- 172.21.15.3:0/2607347749 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68004a00 con 0x56288e1852f0 2024-09-21T06:17:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.699+0000 7fce7d876740 1 -- 172.21.15.3:0/2607347749 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce68004050 con 0x56288e1852f0 2024-09-21T06:17:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.699+0000 7fce4f7fe640 1 -- 172.21.15.3:0/2607347749 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fce68004ba0 con 0x56288e1852f0 2024-09-21T06:17:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.699+0000 7fce4f7fe640 1 -- 172.21.15.3:0/2607347749 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce68036050 con 0x56288e1852f0 2024-09-21T06:17:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.700+0000 7fce7d876740 10 client.25844 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.700+0000 7fce7d876740 1 -- 172.21.15.3:0/2607347749 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56288e292620 con 0x56288e1852f0 2024-09-21T06:17:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.700+0000 7fce4f7fe640 1 -- 172.21.15.3:0/2607347749 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce68004050 con 0x56288e1852f0 2024-09-21T06:17:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.700+0000 7fce4f7fe640 1 -- 172.21.15.3:0/2607347749 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fce68005050 con 0x56288e1852f0 2024-09-21T06:17:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.700+0000 7fce7d876740 10 client.25844 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.700+0000 7fce7d876740 10 client.25844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.700+0000 7fce7d876740 20 client.25844 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.700+0000 7fce7d876740 1 --2- 172.21.15.3:0/2607347749 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56288e16cd80 0x56288e5710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.700+0000 7fce7d876740 4 client.25844 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.700+0000 7fce7d876740 1 -- 172.21.15.3:0/2607347749 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56288e4ca640 con 0x56288e16cd80 2024-09-21T06:17:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.700+0000 7fce6f7fe640 1 --2- 172.21.15.3:0/2607347749 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56288e16cd80 0x56288e5710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.701+0000 7fce6f7fe640 1 --2- 172.21.15.3:0/2607347749 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56288e16cd80 0x56288e5710e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbc155ba74c4a950 server_cookie=476520a3e2f0ab3d in_seq=0 out_seq=0 2024-09-21T06:17:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.701+0000 7fce4f7fe640 10 client.25844 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.701+0000 7fce4f7fe640 1 -- 172.21.15.3:0/2607347749 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56288e4ca640 con 0x56288e16cd80 2024-09-21T06:17:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.701+0000 7fce4f7fe640 10 client.25844 handle_command_reply: tid=0 2024-09-21T06:17:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.702+0000 7fce7d876740 1 client.25844 shutdown 2024-09-21T06:17:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.702+0000 7fce7d876740 1 -- 172.21.15.3:0/2607347749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1852f0 msgr2=0x56288e55f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.702+0000 7fce7d876740 1 --2- 172.21.15.3:0/2607347749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1852f0 0x56288e55f050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce68003fd0 tx=0x7fce68004950 comp rx=0 tx=0).stop 2024-09-21T06:17:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.702+0000 7fce7d876740 1 -- 172.21.15.3:0/2607347749 shutdown_connections 2024-09-21T06:17:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.702+0000 7fce7d876740 1 --2- 172.21.15.3:0/2607347749 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56288e16cd80 0x56288e5710e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.702+0000 7fce7d876740 1 --2- 172.21.15.3:0/2607347749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e18d4d0 0x56288e55f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.702+0000 7fce4f7fe640 0 client.25844 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.702+0000 7fce7d876740 1 --2- 172.21.15.3:0/2607347749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 0x56288e56b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.702+0000 7fce7d876740 1 --2- 172.21.15.3:0/2607347749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1852f0 0x56288e55f050 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.702+0000 7fce7d876740 1 -- 172.21.15.3:0/2607347749 >> 172.21.15.3:0/2607347749 conn(0x56288e1a2300 msgr2=0x56288e5549e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.702+0000 7fce7d876740 1 -- 172.21.15.3:0/2607347749 shutdown_connections 2024-09-21T06:17:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.702+0000 7fce7d876740 1 -- 172.21.15.3:0/2607347749 wait complete. 2024-09-21T06:17:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.702+0000 7fce7d876740 20 client.25844 trim_cache size 0 max 16384 2024-09-21T06:17:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.717+0000 7fce7d876740 1 Processor -- start 2024-09-21T06:17:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce7d876740 1 -- start start 2024-09-21T06:17:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce7d876740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1852f0 0x56288e554fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce7d876740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e18d4d0 0x56288e56b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce7d876740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 0x56288e55c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce7d876740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fce701b1930 con 0x56288e1ae1b0 2024-09-21T06:17:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce7d876740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce701b34d0 con 0x56288e1852f0 2024-09-21T06:17:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce7d876740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce701b2000 con 0x56288e18d4d0 2024-09-21T06:17:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce6f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e18d4d0 0x56288e56b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce74eb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1852f0 0x56288e554fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce6f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e18d4d0 0x56288e56b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:60838/0 (socket says 172.21.15.3:60838) 2024-09-21T06:17:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce74eb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1852f0 0x56288e554fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:44842/0 (socket says 172.21.15.3:44842) 2024-09-21T06:17:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce6f7fe640 1 -- 172.21.15.3:0/910549588 learned_addr learned my addr 172.21.15.3:0/910549588 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce74eb0640 1 --2- 172.21.15.3:0/910549588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 0x56288e55c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce6f7fe640 1 -- 172.21.15.3:0/910549588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1852f0 msgr2=0x56288e554fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce6f7fe640 1 --2- 172.21.15.3:0/910549588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1852f0 0x56288e554fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce6f7fe640 1 -- 172.21.15.3:0/910549588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 msgr2=0x56288e55c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce6f7fe640 1 --2- 172.21.15.3:0/910549588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 0x56288e55c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.718+0000 7fce6f7fe640 1 -- 172.21.15.3:0/910549588 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56288e5521a0 con 0x56288e18d4d0 2024-09-21T06:17:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.719+0000 7fce6f7fe640 1 --2- 172.21.15.3:0/910549588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e18d4d0 0x56288e56b320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce640096c0 tx=0x7fce64031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36ab20095edcda9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.719+0000 7fce55ffb640 1 -- 172.21.15.3:0/910549588 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64035040 con 0x56288e18d4d0 2024-09-21T06:17:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.719+0000 7fce55ffb640 1 -- 172.21.15.3:0/910549588 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce64004380 con 0x56288e18d4d0 2024-09-21T06:17:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.719+0000 7fce55ffb640 1 -- 172.21.15.3:0/910549588 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64004d10 con 0x56288e18d4d0 2024-09-21T06:17:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.720+0000 7fce7d876740 1 -- 172.21.15.3:0/910549588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e18d4d0 msgr2=0x56288e56b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.720+0000 7fce7d876740 1 --2- 172.21.15.3:0/910549588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e18d4d0 0x56288e56b320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce640096c0 tx=0x7fce64031400 comp rx=0 tx=0).stop 2024-09-21T06:17:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.720+0000 7fce7d876740 1 -- 172.21.15.3:0/910549588 shutdown_connections 2024-09-21T06:17:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.721+0000 7fce7d876740 1 --2- 172.21.15.3:0/910549588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e18d4d0 0x56288e56b320 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.721+0000 7fce7d876740 1 --2- 172.21.15.3:0/910549588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 0x56288e55c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.721+0000 7fce7d876740 1 --2- 172.21.15.3:0/910549588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e1852f0 0x56288e554fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.721+0000 7fce7d876740 1 -- 172.21.15.3:0/910549588 >> 172.21.15.3:0/910549588 conn(0x56288e1a2300 msgr2=0x56288e554890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.721+0000 7fce7d876740 1 -- 172.21.15.3:0/910549588 shutdown_connections 2024-09-21T06:17:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.721+0000 7fce7d876740 1 -- 172.21.15.3:0/910549588 wait complete. 2024-09-21T06:17:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.721+0000 7fce7d876740 1 -- start start 2024-09-21T06:17:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.722+0000 7fce7d876740 1 Processor -- start 2024-09-21T06:17:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.722+0000 7fce7d876740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e1852f0 0x56288e564580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.722+0000 7fce7d876740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e18d4d0 0x56288e564ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.722+0000 7fce7d876740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 0x56288e565020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.722+0000 7fce7d876740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fce701b2da0 con 0x56288e1ae1b0 2024-09-21T06:17:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.722+0000 7fce7d876740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce701b26d0 con 0x56288e18d4d0 2024-09-21T06:17:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.722+0000 7fce7d876740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce701af910 con 0x56288e1852f0 2024-09-21T06:17:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.722+0000 7fce7d876740 10 client.0 _ll_register_callbacks cb 0x7fce60005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.722+0000 7fce74eb0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e1852f0 0x56288e564580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.722+0000 7fce74eb0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e1852f0 0x56288e564580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:60852/0 (socket says 172.21.15.3:60852) 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.722+0000 7fce74eb0640 1 -- 172.21.15.3:0/2752349978 learned_addr learned my addr 172.21.15.3:0/2752349978 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.723+0000 7fce6f7fe640 1 --2- 172.21.15.3:0/2752349978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e18d4d0 0x56288e564ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.723+0000 7fce74eb0640 1 --2- 172.21.15.3:0/2752349978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 0x56288e565020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.723+0000 7fce74eb0640 1 -- 172.21.15.3:0/2752349978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e1852f0 msgr2=0x56288e564580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.723+0000 7fce74eb0640 1 --2- 172.21.15.3:0/2752349978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e1852f0 0x56288e564580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.723+0000 7fce74eb0640 1 -- 172.21.15.3:0/2752349978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e18d4d0 msgr2=0x56288e564ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.723+0000 7fce74eb0640 1 --2- 172.21.15.3:0/2752349978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e18d4d0 0x56288e564ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.723+0000 7fce74eb0640 1 -- 172.21.15.3:0/2752349978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56288e5d9b70 con 0x56288e1ae1b0 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.723+0000 7fce74eb0640 1 --2- 172.21.15.3:0/2752349978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 0x56288e565020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fce68005420 tx=0x7fce68005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4843a9838387479 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.723+0000 7fce4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.724+0000 7fce4f7fe640 1 -- 172.21.15.3:0/2752349978 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68003da0 con 0x56288e1ae1b0 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.724+0000 7fce4f7fe640 1 -- 172.21.15.3:0/2752349978 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fce680049c0 con 0x56288e1ae1b0 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.724+0000 7fce7d876740 1 -- 172.21.15.3:0/2752349978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce64004380 con 0x56288e1ae1b0 2024-09-21T06:17:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.724+0000 7fce4f7fe640 1 -- 172.21.15.3:0/2752349978 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce6804c050 con 0x56288e1ae1b0 2024-09-21T06:17:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.724+0000 7fce7d876740 10 client.16266 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.724+0000 7fce7d876740 1 -- 172.21.15.3:0/2752349978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56288e5dae00 con 0x56288e1ae1b0 2024-09-21T06:17:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.724+0000 7fce4f7fe640 1 -- 172.21.15.3:0/2752349978 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce64004380 con 0x56288e1ae1b0 2024-09-21T06:17:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.725+0000 7fce4f7fe640 1 -- 172.21.15.3:0/2752349978 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fce6800f600 con 0x56288e1ae1b0 2024-09-21T06:17:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.725+0000 7fce7d876740 10 client.16266 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.725+0000 7fce7d876740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.725+0000 7fce7d876740 20 client.16266 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.725+0000 7fce7d876740 1 --2- 172.21.15.3:0/2752349978 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56288e1421c0 0x56288e5dd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.725+0000 7fce7d876740 4 client.16266 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:17:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.725+0000 7fce7d876740 1 -- 172.21.15.3:0/2752349978 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56288e4c60f0 con 0x56288e1421c0 2024-09-21T06:17:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.725+0000 7fce6f7fe640 1 --2- 172.21.15.3:0/2752349978 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56288e1421c0 0x56288e5dd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.727+0000 7fce4f7fe640 10 client.16266 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.727+0000 7fce6f7fe640 1 --2- 172.21.15.3:0/2752349978 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56288e1421c0 0x56288e5dd590 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76b74d741dd7a40 server_cookie=4344802dff65e2e5 in_seq=0 out_seq=0 2024-09-21T06:17:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.843+0000 7fce4f7fe640 1 -- 172.21.15.3:0/2752349978 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56288e4c60f0 con 0x56288e1421c0 2024-09-21T06:17:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.843+0000 7fce4f7fe640 10 client.16266 handle_command_reply: tid=0 2024-09-21T06:17:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.843+0000 7fce7d876740 1 client.16266 shutdown 2024-09-21T06:17:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.843+0000 7fce7d876740 1 -- 172.21.15.3:0/2752349978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 msgr2=0x56288e565020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.843+0000 7fce7d876740 1 --2- 172.21.15.3:0/2752349978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 0x56288e565020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fce68005420 tx=0x7fce68005fd0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.843+0000 7fce7d876740 1 -- 172.21.15.3:0/2752349978 shutdown_connections 2024-09-21T06:17:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.843+0000 7fce7d876740 1 --2- 172.21.15.3:0/2752349978 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56288e1421c0 0x56288e5dd590 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.843+0000 7fce7d876740 1 --2- 172.21.15.3:0/2752349978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56288e1ae1b0 0x56288e565020 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.843+0000 7fce7d876740 1 --2- 172.21.15.3:0/2752349978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288e18d4d0 0x56288e564ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.843+0000 7fce7d876740 1 --2- 172.21.15.3:0/2752349978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56288e1852f0 0x56288e564580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.843+0000 7fce4f7fe640 0 client.16266 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.843+0000 7fce7d876740 1 -- 172.21.15.3:0/2752349978 >> 172.21.15.3:0/2752349978 conn(0x56288e1a2300 msgr2=0x56288e551880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce7d876740 1 -- 172.21.15.3:0/2752349978 shutdown_connections 2024-09-21T06:17:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce7d876740 1 -- 172.21.15.3:0/2752349978 wait complete. 2024-09-21T06:17:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce7d876740 20 client.16266 trim_cache size 0 max 16384 2024-09-21T06:17:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:17:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:17:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:17:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce76913640 1 -- 172.21.15.3:0/1990249278 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fce580b66b0 msgr2=0x7fce580b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce76913640 1 --2- 172.21.15.3:0/1990249278 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fce580b66b0 0x7fce580b8bc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fce600e5200 tx=0x7fce600eb510 comp rx=0 tx=0).stop 2024-09-21T06:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce76913640 1 -- 172.21.15.3:0/1990249278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce701029e0 msgr2=0x7fce700fe910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce76913640 1 --2- 172.21.15.3:0/1990249278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce701029e0 0x7fce700fe910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce6000e9f0 tx=0x7fce6000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce76913640 1 -- 172.21.15.3:0/1990249278 shutdown_connections 2024-09-21T06:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce76913640 1 --2- 172.21.15.3:0/1990249278 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fce580b66b0 0x7fce580b8bc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce76913640 1 --2- 172.21.15.3:0/1990249278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce70108c60 0x7fce700ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce76913640 1 --2- 172.21.15.3:0/1990249278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce70108310 0x7fce700fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce76913640 1 --2- 172.21.15.3:0/1990249278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce701029e0 0x7fce700fe910 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce76913640 1 -- 172.21.15.3:0/1990249278 >> 172.21.15.3:0/1990249278 conn(0x7fce700fe3e0 msgr2=0x7fce7006bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce76913640 1 -- 172.21.15.3:0/1990249278 shutdown_connections 2024-09-21T06:17:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:17.844+0000 7fce76913640 1 -- 172.21.15.3:0/1990249278 wait complete. 2024-09-21T06:17:17.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 131708 14.57 MB/sec warmup 50 sec latency 49.352 ms 2024-09-21T06:17:17.989 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.614+0000 7faab0c35640 1 -- 172.21.15.3:0/2937555456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaac102ae0 msgr2=0x7faaac102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.614+0000 7faab0c35640 1 --2- 172.21.15.3:0/2937555456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaac102ae0 0x7faaac102ef0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009900 tx=0x7faa9c02a410 comp rx=0 tx=0).stop 2024-09-21T06:17:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.614+0000 7faab0c35640 1 -- 172.21.15.3:0/2937555456 shutdown_connections 2024-09-21T06:17:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.614+0000 7faab0c35640 1 --2- 172.21.15.3:0/2937555456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaac1083d0 0x7faaac1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.614+0000 7faab0c35640 1 --2- 172.21.15.3:0/2937555456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaac108d20 0x7faaac10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.614+0000 7faab0c35640 1 --2- 172.21.15.3:0/2937555456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaac102ae0 0x7faaac102ef0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.614+0000 7faab0c35640 1 -- 172.21.15.3:0/2937555456 >> 172.21.15.3:0/2937555456 conn(0x7faaac079790 msgr2=0x7faaac079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.614+0000 7faab0c35640 1 -- 172.21.15.3:0/2937555456 shutdown_connections 2024-09-21T06:17:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.614+0000 7faab0c35640 1 -- 172.21.15.3:0/2937555456 wait complete. 2024-09-21T06:17:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.615+0000 7faab0c35640 1 Processor -- start 2024-09-21T06:17:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.615+0000 7faab0c35640 1 -- start start 2024-09-21T06:17:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faab0c35640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaac102ae0 0x7faaac110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faab0c35640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaac1083d0 0x7faaac110570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faab0c35640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaac108d20 0x7faaac110ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faab0c35640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faaac113b00 con 0x7faaac1083d0 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faab0c35640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faaac113980 con 0x7faaac102ae0 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faab0c35640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faaac113c80 con 0x7faaac108d20 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faaa3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaac1083d0 0x7faaac110570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faaaad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaac108d20 0x7faaac110ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faaaa59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaac102ae0 0x7faaac110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faaa3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaac1083d0 0x7faaac110570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45830/0 (socket says 172.21.15.3:45830) 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faaaad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaac108d20 0x7faaac110ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:60868/0 (socket says 172.21.15.3:60868) 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faaa3fff640 1 -- 172.21.15.3:0/1089677234 learned_addr learned my addr 172.21.15.3:0/1089677234 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faaaa59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaac102ae0 0x7faaac110030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:44880/0 (socket says 172.21.15.3:44880) 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faaaa59c640 1 -- 172.21.15.3:0/1089677234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaac108d20 msgr2=0x7faaac110ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faaaa59c640 1 --2- 172.21.15.3:0/1089677234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaac108d20 0x7faaac110ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faaaa59c640 1 -- 172.21.15.3:0/1089677234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaac1083d0 msgr2=0x7faaac110570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faaaa59c640 1 --2- 172.21.15.3:0/1089677234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaac1083d0 0x7faaac110570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.616+0000 7faaaa59c640 1 -- 172.21.15.3:0/1089677234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaac1a86a0 con 0x7faaac102ae0 2024-09-21T06:17:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.617+0000 7faaaa59c640 1 --2- 172.21.15.3:0/1089677234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaac102ae0 0x7faaac110030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faa9c006010 tx=0x7faa9c004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:18.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.617+0000 7faaa37fe640 1 -- 172.21.15.3:0/1089677234 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c03f070 con 0x7faaac102ae0 2024-09-21T06:17:18.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.617+0000 7faab0c35640 1 -- 172.21.15.3:0/1089677234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaac1a8930 con 0x7faaac102ae0 2024-09-21T06:17:18.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.617+0000 7faab0c35640 1 -- 172.21.15.3:0/1089677234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaac1a8e60 con 0x7faaac102ae0 2024-09-21T06:17:18.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.617+0000 7faaa37fe640 1 -- 172.21.15.3:0/1089677234 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa9c003ca0 con 0x7faaac102ae0 2024-09-21T06:17:18.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.617+0000 7faaa37fe640 1 -- 172.21.15.3:0/1089677234 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c035070 con 0x7faaac102ae0 2024-09-21T06:17:18.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faab71a9740 1 Processor -- start 2024-09-21T06:17:18.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faab71a9740 1 -- start start 2024-09-21T06:17:18.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faaa37fe640 1 -- 172.21.15.3:0/1089677234 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7faa9c043400 con 0x7faaac102ae0 2024-09-21T06:17:18.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faab71a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827acb070 0x55a827e722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faab71a9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827aa21b0 0x55a827e72800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faab71a9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827aaa390 0x55a827e72d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faab71a9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a827de8ea0 con 0x55a827aaa390 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faab71a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a827dd82d0 con 0x55a827acb070 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faab71a9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a827ddc0e0 con 0x55a827aa21b0 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faaaa59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827aa21b0 0x55a827e72800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faaa3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827aaa390 0x55a827e72d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faaa3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827aaa390 0x55a827e72d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45832/0 (socket says 172.21.15.3:45832) 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faaa3fff640 1 -- 172.21.15.3:0/2481801251 learned_addr learned my addr 172.21.15.3:0/2481801251 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.620+0000 7faaaad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827acb070 0x55a827e722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.621+0000 7faaaad9d640 1 -- 172.21.15.3:0/2481801251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827aa21b0 msgr2=0x55a827e72800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.621+0000 7faaaad9d640 1 --2- 172.21.15.3:0/2481801251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827aa21b0 0x55a827e72800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.621+0000 7faaaad9d640 1 -- 172.21.15.3:0/2481801251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827aaa390 msgr2=0x55a827e72d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.621+0000 7faaaad9d640 1 --2- 172.21.15.3:0/2481801251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827aaa390 0x55a827e72d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.621+0000 7faaaad9d640 1 -- 172.21.15.3:0/2481801251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a827b36650 con 0x55a827acb070 2024-09-21T06:17:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.621+0000 7faaaad9d640 1 --2- 172.21.15.3:0/2481801251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827acb070 0x55a827e722c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009f00 tx=0x7faaa4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33bf1339bc12f525 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.621+0000 7faaa0ff9640 1 -- 172.21.15.3:0/2481801251 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4004d20 con 0x55a827acb070 2024-09-21T06:17:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.621+0000 7faaa37fe640 1 --2- 172.21.15.3:0/1089677234 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faa980b6700 0x7faa980b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.621+0000 7faaa0ff9640 1 -- 172.21.15.3:0/2481801251 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa4004030 con 0x55a827acb070 2024-09-21T06:17:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.621+0000 7faaa0ff9640 1 -- 172.21.15.3:0/2481801251 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4005260 con 0x55a827acb070 2024-09-21T06:17:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.622+0000 7faaaa59c640 1 --2- 172.21.15.3:0/1089677234 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faa980b6700 0x7faa980b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.622+0000 7faaa37fe640 1 -- 172.21.15.3:0/1089677234 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7faa9c107af0 con 0x7faaac102ae0 2024-09-21T06:17:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.622+0000 7faab71a9740 1 -- 172.21.15.3:0/2481801251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827acb070 msgr2=0x55a827e722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.622+0000 7faab71a9740 1 --2- 172.21.15.3:0/2481801251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827acb070 0x55a827e722c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009f00 tx=0x7faaa4011810 comp rx=0 tx=0).stop 2024-09-21T06:17:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.622+0000 7faaaa59c640 1 --2- 172.21.15.3:0/1089677234 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faa980b6700 0x7faa980b8c10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faa9c105330 tx=0x7faa9c10b640 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.623+0000 7faab71a9740 1 -- 172.21.15.3:0/2481801251 shutdown_connections 2024-09-21T06:17:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.623+0000 7faab71a9740 1 --2- 172.21.15.3:0/2481801251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827aa21b0 0x55a827e72800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.623+0000 7faab71a9740 1 --2- 172.21.15.3:0/2481801251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827aaa390 0x55a827e72d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.623+0000 7faab71a9740 1 --2- 172.21.15.3:0/2481801251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827acb070 0x55a827e722c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.623+0000 7faab71a9740 1 -- 172.21.15.3:0/2481801251 >> 172.21.15.3:0/2481801251 conn(0x55a827abf460 msgr2=0x55a827e701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.623+0000 7faab71a9740 1 -- 172.21.15.3:0/2481801251 shutdown_connections 2024-09-21T06:17:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.623+0000 7faab71a9740 1 -- 172.21.15.3:0/2481801251 wait complete. 2024-09-21T06:17:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.623+0000 7faab71a9740 1 -- start start 2024-09-21T06:17:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.623+0000 7faab71a9740 1 Processor -- start 2024-09-21T06:17:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faab71a9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827aa21b0 0x55a827e7d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faab71a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827aaa390 0x55a827e7de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faab71a9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827acb070 0x55a827e7e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faab71a9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a827c39700 con 0x55a827aa21b0 2024-09-21T06:17:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faab71a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a827d73160 con 0x55a827aaa390 2024-09-21T06:17:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faab71a9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a827e80aa0 con 0x55a827acb070 2024-09-21T06:17:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faab71a9740 10 client.0 _ll_register_callbacks cb 0x55a827d8c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faaaad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827aa21b0 0x55a827e7d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faaa3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827aaa390 0x55a827e7de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faaa3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827aaa390 0x55a827e7de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:44900/0 (socket says 172.21.15.3:44900) 2024-09-21T06:17:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faaa3fff640 1 -- 172.21.15.3:0/3332490244 learned_addr learned my addr 172.21.15.3:0/3332490244 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faaaad9d640 1 --2- 172.21.15.3:0/3332490244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827acb070 0x55a827e7e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faaa3fff640 1 -- 172.21.15.3:0/3332490244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827acb070 msgr2=0x55a827e7e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faaa3fff640 1 --2- 172.21.15.3:0/3332490244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827acb070 0x55a827e7e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faaa3fff640 1 -- 172.21.15.3:0/3332490244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827aa21b0 msgr2=0x55a827e7d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faaa3fff640 1 --2- 172.21.15.3:0/3332490244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827aa21b0 0x55a827e7d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.624+0000 7faaa3fff640 1 -- 172.21.15.3:0/3332490244 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a827da0c90 con 0x55a827aaa390 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.625+0000 7faaa3fff640 1 --2- 172.21.15.3:0/3332490244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827aaa390 0x55a827e7de10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa94010a50 tx=0x7faa94010f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=333b6c6537a5fdb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.625+0000 7faa91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.625+0000 7faa91ffb640 1 -- 172.21.15.3:0/3332490244 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9401e060 con 0x55a827aaa390 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.625+0000 7faa91ffb640 1 -- 172.21.15.3:0/3332490244 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7faa9400fbd0 con 0x55a827aaa390 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.625+0000 7faab71a9740 1 -- 172.21.15.3:0/3332490244 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa4004030 con 0x55a827aaa390 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.625+0000 7faa91ffb640 1 -- 172.21.15.3:0/3332490244 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa9400fdd0 con 0x55a827aaa390 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.625+0000 7faab71a9740 10 client.25862 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.625+0000 7faab71a9740 1 -- 172.21.15.3:0/3332490244 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a827e8c7d0 con 0x55a827aaa390 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.625+0000 7faa91ffb640 1 -- 172.21.15.3:0/3332490244 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa4004030 con 0x55a827aaa390 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.626+0000 7faa91ffb640 1 -- 172.21.15.3:0/3332490244 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7faa94018490 con 0x55a827aaa390 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.626+0000 7faab71a9740 10 client.25862 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.626+0000 7faab71a9740 10 client.25862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.626+0000 7faab71a9740 20 client.25862 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.626+0000 7faab71a9740 1 --2- 172.21.15.3:0/3332490244 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a827a899f0 0x55a827e8fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.626+0000 7faab71a9740 4 client.25862 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:18.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.626+0000 7faab71a9740 1 -- 172.21.15.3:0/3332490244 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a827deadf0 con 0x55a827a899f0 2024-09-21T06:17:18.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.626+0000 7faaaad9d640 1 --2- 172.21.15.3:0/3332490244 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a827a899f0 0x55a827e8fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.626+0000 7faa91ffb640 10 client.25862 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:18.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.626+0000 7faaaad9d640 1 --2- 172.21.15.3:0/3332490244 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a827a899f0 0x55a827e8fd60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=514786821882c18c server_cookie=2054cf98bebfb00b in_seq=0 out_seq=0 2024-09-21T06:17:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.627+0000 7faa91ffb640 1 -- 172.21.15.3:0/3332490244 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a827deadf0 con 0x55a827a899f0 2024-09-21T06:17:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.627+0000 7faa91ffb640 10 client.25862 handle_command_reply: tid=0 2024-09-21T06:17:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.627+0000 7faab71a9740 1 client.25862 shutdown 2024-09-21T06:17:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.627+0000 7faab71a9740 1 -- 172.21.15.3:0/3332490244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827aaa390 msgr2=0x55a827e7de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.628+0000 7faab71a9740 1 --2- 172.21.15.3:0/3332490244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827aaa390 0x55a827e7de10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa94010a50 tx=0x7faa94010f10 comp rx=0 tx=0).stop 2024-09-21T06:17:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.628+0000 7faab71a9740 1 -- 172.21.15.3:0/3332490244 shutdown_connections 2024-09-21T06:17:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.628+0000 7faab71a9740 1 --2- 172.21.15.3:0/3332490244 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a827a899f0 0x55a827e8fd60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.628+0000 7faab71a9740 1 --2- 172.21.15.3:0/3332490244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827acb070 0x55a827e7e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.628+0000 7faab71a9740 1 --2- 172.21.15.3:0/3332490244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827aaa390 0x55a827e7de10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.628+0000 7faab71a9740 1 --2- 172.21.15.3:0/3332490244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827aa21b0 0x55a827e7d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.628+0000 7faab71a9740 1 -- 172.21.15.3:0/3332490244 >> 172.21.15.3:0/3332490244 conn(0x55a827abf460 msgr2=0x55a827e754a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.628+0000 7faa91ffb640 0 client.25862 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:18.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.628+0000 7faab71a9740 1 -- 172.21.15.3:0/3332490244 shutdown_connections 2024-09-21T06:17:18.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.628+0000 7faab71a9740 1 -- 172.21.15.3:0/3332490244 wait complete. 2024-09-21T06:17:18.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.628+0000 7faab71a9740 20 client.25862 trim_cache size 0 max 16384 2024-09-21T06:17:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.641+0000 7faab71a9740 1 Processor -- start 2024-09-21T06:17:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.641+0000 7faab71a9740 1 -- start start 2024-09-21T06:17:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.641+0000 7faab71a9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827aa21b0 0x55a827e7e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.641+0000 7faab71a9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827acb070 0x55a827e89c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faab71a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827a5eda0 0x55a827e6f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faab71a9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1b1750 con 0x55a827acb070 2024-09-21T06:17:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faab71a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1b32f0 con 0x55a827a5eda0 2024-09-21T06:17:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faab71a9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1b1e20 con 0x55a827aa21b0 2024-09-21T06:17:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faaa3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827acb070 0x55a827e89c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faaa3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827acb070 0x55a827e89c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45848/0 (socket says 172.21.15.3:45848) 2024-09-21T06:17:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faaa3fff640 1 -- 172.21.15.3:0/2711432041 learned_addr learned my addr 172.21.15.3:0/2711432041 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faaaad9d640 1 --2- 172.21.15.3:0/2711432041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827aa21b0 0x55a827e7e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faaaad9d640 1 --2- 172.21.15.3:0/2711432041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827a5eda0 0x55a827e6f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faaaad9d640 1 -- 172.21.15.3:0/2711432041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827a5eda0 msgr2=0x55a827e6f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faaaad9d640 1 --2- 172.21.15.3:0/2711432041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827a5eda0 0x55a827e6f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faaaad9d640 1 -- 172.21.15.3:0/2711432041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827acb070 msgr2=0x55a827e89c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faaaad9d640 1 --2- 172.21.15.3:0/2711432041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827acb070 0x55a827e89c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faaaad9d640 1 -- 172.21.15.3:0/2711432041 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a827e8c5e0 con 0x55a827aa21b0 2024-09-21T06:17:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faaaad9d640 1 --2- 172.21.15.3:0/2711432041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827aa21b0 0x55a827e7e290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009270 tx=0x7faaa4012f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84112d4ff7295c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.642+0000 7faa93fff640 1 -- 172.21.15.3:0/2711432041 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa401e040 con 0x55a827aa21b0 2024-09-21T06:17:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.643+0000 7faa93fff640 1 -- 172.21.15.3:0/2711432041 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa401f040 con 0x55a827aa21b0 2024-09-21T06:17:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.643+0000 7faa93fff640 1 -- 172.21.15.3:0/2711432041 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa40052b0 con 0x55a827aa21b0 2024-09-21T06:17:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.643+0000 7faab71a9740 1 -- 172.21.15.3:0/2711432041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827aa21b0 msgr2=0x55a827e7e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.643+0000 7faab71a9740 1 --2- 172.21.15.3:0/2711432041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827aa21b0 0x55a827e7e290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009270 tx=0x7faaa4012f90 comp rx=0 tx=0).stop 2024-09-21T06:17:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.643+0000 7faab71a9740 1 -- 172.21.15.3:0/2711432041 shutdown_connections 2024-09-21T06:17:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.643+0000 7faab71a9740 1 --2- 172.21.15.3:0/2711432041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827a5eda0 0x55a827e6f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.643+0000 7faab71a9740 1 --2- 172.21.15.3:0/2711432041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827acb070 0x55a827e89c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.643+0000 7faab71a9740 1 --2- 172.21.15.3:0/2711432041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827aa21b0 0x55a827e7e290 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.644+0000 7faab71a9740 1 -- 172.21.15.3:0/2711432041 >> 172.21.15.3:0/2711432041 conn(0x55a827abf460 msgr2=0x55a827e7c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.644+0000 7faab71a9740 1 -- 172.21.15.3:0/2711432041 shutdown_connections 2024-09-21T06:17:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.644+0000 7faab71a9740 1 -- 172.21.15.3:0/2711432041 wait complete. 2024-09-21T06:17:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.644+0000 7faab71a9740 1 -- start start 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.644+0000 7faab71a9740 1 Processor -- start 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.644+0000 7faab71a9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827a5eda0 0x55a827e86680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.644+0000 7faab71a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827aa21b0 0x55a827e81b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.644+0000 7faab71a9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827acb070 0x55a827e82220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.644+0000 7faab71a9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1b2bc0 con 0x55a827acb070 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.644+0000 7faab71a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1b24f0 con 0x55a827aa21b0 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.644+0000 7faab71a9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1af730 con 0x55a827a5eda0 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faab71a9740 10 client.0 _ll_register_callbacks cb 0x7faa9400a870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faaaad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827a5eda0 0x55a827e86680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faaaad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827acb070 0x55a827e82220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faaa3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827aa21b0 0x55a827e81b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faaaad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827acb070 0x55a827e82220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45858/0 (socket says 172.21.15.3:45858) 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faaaad9d640 1 -- 172.21.15.3:0/1322123465 learned_addr learned my addr 172.21.15.3:0/1322123465 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faaa3fff640 1 -- 172.21.15.3:0/1322123465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827a5eda0 msgr2=0x55a827e86680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faaa3fff640 1 --2- 172.21.15.3:0/1322123465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827a5eda0 0x55a827e86680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faaa3fff640 1 -- 172.21.15.3:0/1322123465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827acb070 msgr2=0x55a827e82220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faaa3fff640 1 --2- 172.21.15.3:0/1322123465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827acb070 0x55a827e82220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faaa3fff640 1 -- 172.21.15.3:0/1322123465 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a827ef9720 con 0x55a827aa21b0 2024-09-21T06:17:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faaaad9d640 1 --2- 172.21.15.3:0/1322123465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827acb070 0x55a827e82220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faaa3fff640 1 --2- 172.21.15.3:0/1322123465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827aa21b0 0x55a827e81b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faa94018a20 tx=0x7faa94018cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2459aafc684133 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faa91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faa91ffb640 1 -- 172.21.15.3:0/1322123465 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94030060 con 0x55a827aa21b0 2024-09-21T06:17:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faab71a9740 1 -- 172.21.15.3:0/1322123465 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa401f040 con 0x55a827aa21b0 2024-09-21T06:17:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.645+0000 7faa91ffb640 1 -- 172.21.15.3:0/1322123465 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7faa9400fbd0 con 0x55a827aa21b0 2024-09-21T06:17:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.646+0000 7faa91ffb640 1 -- 172.21.15.3:0/1322123465 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa94018ea0 con 0x55a827aa21b0 2024-09-21T06:17:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.646+0000 7faab71a9740 10 client.25868 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.646+0000 7faab71a9740 1 -- 172.21.15.3:0/1322123465 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a827efa810 con 0x55a827aa21b0 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.646+0000 7faa91ffb640 1 -- 172.21.15.3:0/1322123465 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa401f040 con 0x55a827aa21b0 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.646+0000 7faa91ffb640 1 -- 172.21.15.3:0/1322123465 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7faa94004900 con 0x55a827aa21b0 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.646+0000 7faab71a9740 10 client.25868 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.646+0000 7faab71a9740 10 client.25868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.646+0000 7faab71a9740 20 client.25868 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.646+0000 7faab71a9740 1 --2- 172.21.15.3:0/1322123465 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a827efb020 0x55a827efd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.646+0000 7faab71a9740 4 client.25868 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.646+0000 7faab71a9740 1 -- 172.21.15.3:0/1322123465 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a827de68a0 con 0x55a827efb020 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.646+0000 7faaaad9d640 1 --2- 172.21.15.3:0/1322123465 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a827efb020 0x55a827efd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.647+0000 7faaaad9d640 1 --2- 172.21.15.3:0/1322123465 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a827efb020 0x55a827efd440 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=706f614ffeffccb3 server_cookie=72879bb6ffe3659f in_seq=0 out_seq=0 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.647+0000 7faa91ffb640 10 client.25868 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.647+0000 7faa91ffb640 1 -- 172.21.15.3:0/1322123465 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a827de68a0 con 0x55a827efb020 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.647+0000 7faa91ffb640 10 client.25868 handle_command_reply: tid=0 2024-09-21T06:17:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.647+0000 7faab71a9740 1 client.25868 shutdown 2024-09-21T06:17:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.647+0000 7faab71a9740 1 -- 172.21.15.3:0/1322123465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827aa21b0 msgr2=0x55a827e81b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.647+0000 7faab71a9740 1 --2- 172.21.15.3:0/1322123465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827aa21b0 0x55a827e81b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faa94018a20 tx=0x7faa94018cb0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.647+0000 7faab71a9740 1 -- 172.21.15.3:0/1322123465 shutdown_connections 2024-09-21T06:17:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.647+0000 7faab71a9740 1 --2- 172.21.15.3:0/1322123465 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a827efb020 0x55a827efd440 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faab71a9740 1 --2- 172.21.15.3:0/1322123465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a827acb070 0x55a827e82220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faab71a9740 1 --2- 172.21.15.3:0/1322123465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a827aa21b0 0x55a827e81b30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faab71a9740 1 --2- 172.21.15.3:0/1322123465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a827a5eda0 0x55a827e86680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faa91ffb640 0 client.25868 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faab71a9740 1 -- 172.21.15.3:0/1322123465 >> 172.21.15.3:0/1322123465 conn(0x55a827abf460 msgr2=0x55a827e732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faab71a9740 1 -- 172.21.15.3:0/1322123465 shutdown_connections 2024-09-21T06:17:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faab71a9740 1 -- 172.21.15.3:0/1322123465 wait complete. 2024-09-21T06:17:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faab71a9740 20 client.25868 trim_cache size 0 max 16384 2024-09-21T06:17:18.654 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:17:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faab0c35640 1 -- 172.21.15.3:0/1089677234 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faa980b6700 msgr2=0x7faa980b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faab0c35640 1 --2- 172.21.15.3:0/1089677234 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faa980b6700 0x7faa980b8c10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faa9c105330 tx=0x7faa9c10b640 comp rx=0 tx=0).stop 2024-09-21T06:17:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faab0c35640 1 -- 172.21.15.3:0/1089677234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaac102ae0 msgr2=0x7faaac110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faab0c35640 1 --2- 172.21.15.3:0/1089677234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaac102ae0 0x7faaac110030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faa9c006010 tx=0x7faa9c004060 comp rx=0 tx=0).stop 2024-09-21T06:17:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faab0c35640 1 -- 172.21.15.3:0/1089677234 shutdown_connections 2024-09-21T06:17:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.648+0000 7faab0c35640 1 --2- 172.21.15.3:0/1089677234 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faa980b6700 0x7faa980b8c10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.649+0000 7faab0c35640 1 --2- 172.21.15.3:0/1089677234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaac108d20 0x7faaac110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.649+0000 7faab0c35640 1 --2- 172.21.15.3:0/1089677234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaac1083d0 0x7faaac110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.649+0000 7faab0c35640 1 --2- 172.21.15.3:0/1089677234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaac102ae0 0x7faaac110030 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.649+0000 7faab0c35640 1 -- 172.21.15.3:0/1089677234 >> 172.21.15.3:0/1089677234 conn(0x7faaac079790 msgr2=0x7faaac106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.649+0000 7faab0c35640 1 -- 172.21.15.3:0/1089677234 shutdown_connections 2024-09-21T06:17:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:18.649+0000 7faab0c35640 1 -- 172.21.15.3:0/1089677234 wait complete. 2024-09-21T06:17:18.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 134777 14.63 MB/sec warmup 51 sec latency 25.227 ms 2024-09-21T06:17:19.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:19 smithi003 ceph-mon[31227]: pgmap v394: 97 pgs: 97 active+clean; 50 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-09-21T06:17:19.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:19 smithi137 ceph-mon[34464]: pgmap v394: 97 pgs: 97 active+clean; 50 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-09-21T06:17:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:19 smithi204 ceph-mon[34659]: pgmap v394: 97 pgs: 97 active+clean; 50 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-09-21T06:17:19.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 136525 14.48 MB/sec warmup 52 sec latency 12.703 ms 2024-09-21T06:17:20.349 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:17:20.349 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:17:20.619 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:20.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 138929 14.47 MB/sec warmup 53 sec latency 19.915 ms 2024-09-21T06:17:21.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.135+0000 7ff791526640 1 -- 172.21.15.3:0/3672808038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78c1083a0 msgr2=0x7ff78c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:21.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.135+0000 7ff791526640 1 --2- 172.21.15.3:0/3672808038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78c1083a0 0x7ff78c1087b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff774009900 tx=0x7ff77402a410 comp rx=0 tx=0).stop 2024-09-21T06:17:21.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.135+0000 7ff791526640 1 -- 172.21.15.3:0/3672808038 shutdown_connections 2024-09-21T06:17:21.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.135+0000 7ff791526640 1 --2- 172.21.15.3:0/3672808038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff78c108cf0 0x7ff78c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.135+0000 7ff791526640 1 --2- 172.21.15.3:0/3672808038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78c1083a0 0x7ff78c1087b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.135+0000 7ff791526640 1 --2- 172.21.15.3:0/3672808038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff78c102a60 0x7ff78c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.135+0000 7ff791526640 1 -- 172.21.15.3:0/3672808038 >> 172.21.15.3:0/3672808038 conn(0x7ff78c079730 msgr2=0x7ff78c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:21.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.135+0000 7ff791526640 1 -- 172.21.15.3:0/3672808038 shutdown_connections 2024-09-21T06:17:21.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.135+0000 7ff791526640 1 -- 172.21.15.3:0/3672808038 wait complete. 2024-09-21T06:17:21.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.136+0000 7ff791526640 1 Processor -- start 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.136+0000 7ff791526640 1 -- start start 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.136+0000 7ff791526640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff78c102a60 0x7ff78c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.136+0000 7ff791526640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff78c1083a0 0x7ff78c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.136+0000 7ff791526640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78c108cf0 0x7ff78c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.136+0000 7ff791526640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c1129b0 con 0x7ff78c108cf0 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.136+0000 7ff791526640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c112830 con 0x7ff78c102a60 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.136+0000 7ff791526640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c112b30 con 0x7ff78c1083a0 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff783fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff78c1083a0 0x7ff78c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff78affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff78c102a60 0x7ff78c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff783fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff78c1083a0 0x7ff78c19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:60928/0 (socket says 172.21.15.3:60928) 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff78b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78c108cf0 0x7ff78c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff783fff640 1 -- 172.21.15.3:0/3916607933 learned_addr learned my addr 172.21.15.3:0/3916607933 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff78b7fe640 1 -- 172.21.15.3:0/3916607933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff78c1083a0 msgr2=0x7ff78c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff78b7fe640 1 --2- 172.21.15.3:0/3916607933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff78c1083a0 0x7ff78c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff78b7fe640 1 -- 172.21.15.3:0/3916607933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff78c102a60 msgr2=0x7ff78c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:21.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff78b7fe640 1 --2- 172.21.15.3:0/3916607933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff78c102a60 0x7ff78c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff78b7fe640 1 -- 172.21.15.3:0/3916607933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c1a0400 con 0x7ff78c108cf0 2024-09-21T06:17:21.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff783fff640 1 --2- 172.21.15.3:0/3916607933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff78c1083a0 0x7ff78c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:21.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff78b7fe640 1 --2- 172.21.15.3:0/3916607933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78c108cf0 0x7ff78c19fcd0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff78400e940 tx=0x7ff78400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:21.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff788ff9640 1 -- 172.21.15.3:0/3916607933 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784004d20 con 0x7ff78c108cf0 2024-09-21T06:17:21.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.137+0000 7ff791526640 1 -- 172.21.15.3:0/3916607933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78c1a06f0 con 0x7ff78c108cf0 2024-09-21T06:17:21.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.138+0000 7ff788ff9640 1 -- 172.21.15.3:0/3916607933 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff784005020 con 0x7ff78c108cf0 2024-09-21T06:17:21.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.138+0000 7ff788ff9640 1 -- 172.21.15.3:0/3916607933 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7840116d0 con 0x7ff78c108cf0 2024-09-21T06:17:21.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.138+0000 7ff791526640 1 -- 172.21.15.3:0/3916607933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78c1a6e50 con 0x7ff78c108cf0 2024-09-21T06:17:21.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.140+0000 7ff788ff9640 1 -- 172.21.15.3:0/3916607933 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7ff784011870 con 0x7ff78c108cf0 2024-09-21T06:17:21.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.140+0000 7ff797a9a740 1 Processor -- start 2024-09-21T06:17:21.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.141+0000 7ff797a9a740 1 -- start start 2024-09-21T06:17:21.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.141+0000 7ff797a9a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79fc3350 0x562c7a36a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.141+0000 7ff797a9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79f9a490 0x562c7a36a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.141+0000 7ff797a9a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fa2670 0x562c7a36aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.141+0000 7ff797a9a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562c7a2e0e60 con 0x562c79fc3350 2024-09-21T06:17:21.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.141+0000 7ff797a9a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562c7a2d0290 con 0x562c79fa2670 2024-09-21T06:17:21.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.141+0000 7ff797a9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c7a2d40a0 con 0x562c79f9a490 2024-09-21T06:17:21.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.141+0000 7ff783fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79fc3350 0x562c7a36a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.141+0000 7ff78affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79f9a490 0x562c7a36a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.141+0000 7ff783fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79fc3350 0x562c7a36a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45880/0 (socket says 172.21.15.3:45880) 2024-09-21T06:17:21.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.141+0000 7ff78affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79f9a490 0x562c7a36a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:60942/0 (socket says 172.21.15.3:60942) 2024-09-21T06:17:21.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.141+0000 7ff783fff640 1 -- 172.21.15.3:0/1381600922 learned_addr learned my addr 172.21.15.3:0/1381600922 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:21.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.141+0000 7ff783fff640 1 --2- 172.21.15.3:0/1381600922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fa2670 0x562c7a36aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.142+0000 7ff78affd640 1 -- 172.21.15.3:0/1381600922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fa2670 msgr2=0x562c7a36aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:21.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.142+0000 7ff78affd640 1 --2- 172.21.15.3:0/1381600922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fa2670 0x562c7a36aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.142+0000 7ff78affd640 1 -- 172.21.15.3:0/1381600922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79fc3350 msgr2=0x562c7a36a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:21.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.142+0000 7ff78affd640 1 --2- 172.21.15.3:0/1381600922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79fc3350 0x562c7a36a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.142+0000 7ff78affd640 1 -- 172.21.15.3:0/1381600922 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c7a02e930 con 0x562c79f9a490 2024-09-21T06:17:21.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.142+0000 7ff788ff9640 1 --2- 172.21.15.3:0/3916607933 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7780b6700 0x7ff7780b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.142+0000 7ff78affd640 1 --2- 172.21.15.3:0/1381600922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79f9a490 0x562c7a36a970 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009c20 tx=0x7ff77c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3c645864ce76682 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:21.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.142+0000 7ff783fff640 1 --2- 172.21.15.3:0/3916607933 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7780b6700 0x7ff7780b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.142+0000 7ff788ff9640 1 -- 172.21.15.3:0/3916607933 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff7840e7730 con 0x7ff78c108cf0 2024-09-21T06:17:21.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.142+0000 7ff7817fa640 1 -- 172.21.15.3:0/1381600922 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c009dd0 con 0x562c79f9a490 2024-09-21T06:17:21.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.142+0000 7ff7817fa640 1 -- 172.21.15.3:0/1381600922 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff77c005020 con 0x562c79f9a490 2024-09-21T06:17:21.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.142+0000 7ff7817fa640 1 -- 172.21.15.3:0/1381600922 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c004a00 con 0x562c79f9a490 2024-09-21T06:17:21.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.143+0000 7ff797a9a740 1 -- 172.21.15.3:0/1381600922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79f9a490 msgr2=0x562c7a36a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.143+0000 7ff797a9a740 1 --2- 172.21.15.3:0/1381600922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79f9a490 0x562c7a36a970 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009c20 tx=0x7ff77c011810 comp rx=0 tx=0).stop 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.143+0000 7ff783fff640 1 --2- 172.21.15.3:0/3916607933 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7780b6700 0x7ff7780b8c10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff774005f70 tx=0x7ff774041040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 -- 172.21.15.3:0/1381600922 shutdown_connections 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 --2- 172.21.15.3:0/1381600922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79f9a490 0x562c7a36a970 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 --2- 172.21.15.3:0/1381600922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fa2670 0x562c7a36aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 --2- 172.21.15.3:0/1381600922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79fc3350 0x562c7a36a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 -- 172.21.15.3:0/1381600922 >> 172.21.15.3:0/1381600922 conn(0x562c79fb7740 msgr2=0x562c7a368320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 -- 172.21.15.3:0/1381600922 shutdown_connections 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 -- 172.21.15.3:0/1381600922 wait complete. 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 -- start start 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 Processor -- start 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79f9a490 0x562c7a37a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fa2670 0x562c7a37a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79fc3350 0x562c7a3862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562c7a131890 con 0x562c79fc3350 2024-09-21T06:17:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562c7a26ad40 con 0x562c79fa2670 2024-09-21T06:17:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.144+0000 7ff797a9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c7a37cf70 con 0x562c79f9a490 2024-09-21T06:17:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff78affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79f9a490 0x562c7a37a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff78affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79f9a490 0x562c7a37a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:60944/0 (socket says 172.21.15.3:60944) 2024-09-21T06:17:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff783fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fa2670 0x562c7a37a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff78affd640 1 -- 172.21.15.3:0/551284295 learned_addr learned my addr 172.21.15.3:0/551284295 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff78b7fe640 1 --2- 172.21.15.3:0/551284295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79fc3350 0x562c7a3862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff797a9a740 10 client.0 _ll_register_callbacks cb 0x562c7a2cec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff783fff640 1 -- 172.21.15.3:0/551284295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79f9a490 msgr2=0x562c7a37a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff783fff640 1 --2- 172.21.15.3:0/551284295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79f9a490 0x562c7a37a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff783fff640 1 -- 172.21.15.3:0/551284295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79fc3350 msgr2=0x562c7a3862d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff783fff640 1 --2- 172.21.15.3:0/551284295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79fc3350 0x562c7a3862d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff783fff640 1 -- 172.21.15.3:0/551284295 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c7a386810 con 0x562c79fa2670 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff78b7fe640 1 --2- 172.21.15.3:0/551284295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79fc3350 0x562c7a3862d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff783fff640 1 --2- 172.21.15.3:0/551284295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fa2670 0x562c7a37a8c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff774005ab0 tx=0x7ff77402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f13e3d109540e1dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff7727fc640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.145+0000 7ff7727fc640 1 -- 172.21.15.3:0/551284295 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774050070 con 0x562c79fa2670 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff797a9a740 1 -- 172.21.15.3:0/551284295 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff77c005020 con 0x562c79fa2670 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff7727fc640 1 -- 172.21.15.3:0/551284295 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff774031970 con 0x562c79fa2670 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff7727fc640 1 -- 172.21.15.3:0/551284295 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff77405c050 con 0x562c79fa2670 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff797a9a740 10 client.25880 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff797a9a740 1 -- 172.21.15.3:0/551284295 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562c7a389110 con 0x562c79fa2670 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff7727fc640 1 -- 172.21.15.3:0/551284295 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff77c005020 con 0x562c79fa2670 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff7727fc640 1 -- 172.21.15.3:0/551284295 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff77403dce0 con 0x562c79fa2670 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff797a9a740 10 client.25880 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff797a9a740 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff797a9a740 20 client.25880 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:21.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff797a9a740 1 --2- 172.21.15.3:0/551284295 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562c79f81cd0 0x562c7a38c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff797a9a740 4 client.25880 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:21.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff797a9a740 1 -- 172.21.15.3:0/551284295 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c7a2e2db0 con 0x562c79f81cd0 2024-09-21T06:17:21.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.146+0000 7ff78affd640 1 --2- 172.21.15.3:0/551284295 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562c79f81cd0 0x562c7a38c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.147+0000 7ff7727fc640 10 client.25880 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:21.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.147+0000 7ff78affd640 1 --2- 172.21.15.3:0/551284295 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562c79f81cd0 0x562c7a38c660 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=649531216b45c5f4 server_cookie=a9b82f8d4f95a109 in_seq=0 out_seq=0 2024-09-21T06:17:21.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff7727fc640 1 -- 172.21.15.3:0/551284295 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562c7a2e2db0 con 0x562c79f81cd0 2024-09-21T06:17:21.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff7727fc640 10 client.25880 handle_command_reply: tid=0 2024-09-21T06:17:21.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff797a9a740 1 client.25880 shutdown 2024-09-21T06:17:21.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff797a9a740 1 -- 172.21.15.3:0/551284295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fa2670 msgr2=0x562c7a37a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:21.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff797a9a740 1 --2- 172.21.15.3:0/551284295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fa2670 0x562c7a37a8c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff774005ab0 tx=0x7ff77402a410 comp rx=0 tx=0).stop 2024-09-21T06:17:21.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff797a9a740 1 -- 172.21.15.3:0/551284295 shutdown_connections 2024-09-21T06:17:21.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff797a9a740 1 --2- 172.21.15.3:0/551284295 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562c79f81cd0 0x562c7a38c660 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff797a9a740 1 --2- 172.21.15.3:0/551284295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79fc3350 0x562c7a3862d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff7727fc640 0 client.25880 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:21.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff797a9a740 1 --2- 172.21.15.3:0/551284295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fa2670 0x562c7a37a8c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff797a9a740 1 --2- 172.21.15.3:0/551284295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79f9a490 0x562c7a37a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff797a9a740 1 -- 172.21.15.3:0/551284295 >> 172.21.15.3:0/551284295 conn(0x562c79fb7740 msgr2=0x562c7a366850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:21.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff797a9a740 1 -- 172.21.15.3:0/551284295 shutdown_connections 2024-09-21T06:17:21.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff797a9a740 1 -- 172.21.15.3:0/551284295 wait complete. 2024-09-21T06:17:21.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.148+0000 7ff797a9a740 20 client.25880 trim_cache size 0 max 16384 2024-09-21T06:17:21.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.167+0000 7ff797a9a740 1 Processor -- start 2024-09-21T06:17:21.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.167+0000 7ff797a9a740 1 -- start start 2024-09-21T06:17:21.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.167+0000 7ff797a9a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 0x562c7a366850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.167+0000 7ff797a9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79fa2670 0x562c7a366d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.167+0000 7ff797a9a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fc3350 0x562c7a377370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.167+0000 7ff797a9a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c1ad3f0 con 0x562c79f9a490 2024-09-21T06:17:21.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.167+0000 7ff797a9a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c1aef90 con 0x562c79fc3350 2024-09-21T06:17:21.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.167+0000 7ff797a9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c1adac0 con 0x562c79fa2670 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.168+0000 7ff78affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 0x562c7a366850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.168+0000 7ff78affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 0x562c7a366850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45896/0 (socket says 172.21.15.3:45896) 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.168+0000 7ff78affd640 1 -- 172.21.15.3:0/3267756201 learned_addr learned my addr 172.21.15.3:0/3267756201 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.168+0000 7ff78b7fe640 1 --2- 172.21.15.3:0/3267756201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fc3350 0x562c7a377370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.168+0000 7ff783fff640 1 --2- 172.21.15.3:0/3267756201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79fa2670 0x562c7a366d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.168+0000 7ff78affd640 1 -- 172.21.15.3:0/3267756201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79fa2670 msgr2=0x562c7a366d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.168+0000 7ff78affd640 1 --2- 172.21.15.3:0/3267756201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79fa2670 0x562c7a366d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.168+0000 7ff78affd640 1 -- 172.21.15.3:0/3267756201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fc3350 msgr2=0x562c7a377370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.168+0000 7ff78affd640 1 --2- 172.21.15.3:0/3267756201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fc3350 0x562c7a377370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.168+0000 7ff78affd640 1 -- 172.21.15.3:0/3267756201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c7a36c990 con 0x562c79f9a490 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.168+0000 7ff78affd640 1 --2- 172.21.15.3:0/3267756201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 0x562c7a366850 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0099b0 tx=0x7ff77c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f41a463e0197517 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.168+0000 7ff780ff9640 1 -- 172.21.15.3:0/3267756201 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c031070 con 0x562c79f9a490 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.169+0000 7ff780ff9640 1 -- 172.21.15.3:0/3267756201 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff77c0054b0 con 0x562c79f9a490 2024-09-21T06:17:21.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.169+0000 7ff780ff9640 1 -- 172.21.15.3:0/3267756201 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c019410 con 0x562c79f9a490 2024-09-21T06:17:21.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.169+0000 7ff797a9a740 1 -- 172.21.15.3:0/3267756201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 msgr2=0x562c7a366850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:21.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.169+0000 7ff797a9a740 1 --2- 172.21.15.3:0/3267756201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 0x562c7a366850 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0099b0 tx=0x7ff77c005d30 comp rx=0 tx=0).stop 2024-09-21T06:17:21.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.169+0000 7ff797a9a740 1 -- 172.21.15.3:0/3267756201 shutdown_connections 2024-09-21T06:17:21.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.169+0000 7ff797a9a740 1 --2- 172.21.15.3:0/3267756201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79fa2670 0x562c7a366d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.169+0000 7ff797a9a740 1 --2- 172.21.15.3:0/3267756201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fc3350 0x562c7a377370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.169+0000 7ff797a9a740 1 --2- 172.21.15.3:0/3267756201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 0x562c7a366850 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.169+0000 7ff797a9a740 1 -- 172.21.15.3:0/3267756201 >> 172.21.15.3:0/3267756201 conn(0x562c79fb7740 msgr2=0x562c7a378a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.170+0000 7ff797a9a740 1 -- 172.21.15.3:0/3267756201 shutdown_connections 2024-09-21T06:17:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.170+0000 7ff797a9a740 1 -- 172.21.15.3:0/3267756201 wait complete. 2024-09-21T06:17:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.170+0000 7ff797a9a740 1 -- start start 2024-09-21T06:17:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.170+0000 7ff797a9a740 1 Processor -- start 2024-09-21T06:17:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff797a9a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 0x562c7a382440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff797a9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79fa2670 0x562c7a382980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff797a9a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fc3350 0x562c7a37d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff797a9a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c1ae860 con 0x562c79f9a490 2024-09-21T06:17:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff797a9a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c1ae190 con 0x562c79fc3350 2024-09-21T06:17:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff797a9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c1ab3d0 con 0x562c79fa2670 2024-09-21T06:17:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff78affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 0x562c7a382440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff78affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 0x562c7a382440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45904/0 (socket says 172.21.15.3:45904) 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff78affd640 1 -- 172.21.15.3:0/3913113583 learned_addr learned my addr 172.21.15.3:0/3913113583 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff797a9a740 10 client.0 _ll_register_callbacks cb 0x7ff774003770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff78b7fe640 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fc3350 0x562c7a37d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff783fff640 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79fa2670 0x562c7a382980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff78affd640 1 -- 172.21.15.3:0/3913113583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79fa2670 msgr2=0x562c7a382980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff78affd640 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79fa2670 0x562c7a382980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff78affd640 1 -- 172.21.15.3:0/3913113583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fc3350 msgr2=0x562c7a37d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff78affd640 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fc3350 0x562c7a37d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff78affd640 1 -- 172.21.15.3:0/3913113583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c7a37d780 con 0x562c79f9a490 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff783fff640 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79fa2670 0x562c7a382980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff78b7fe640 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fc3350 0x562c7a37d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff78affd640 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 0x562c7a382440 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff77c024cd0 tx=0x7ff77c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b932f4049b9be39b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.171+0000 7ff7727fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:21.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff7727fc640 1 -- 172.21.15.3:0/3913113583 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c031040 con 0x562c79f9a490 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff7727fc640 1 -- 172.21.15.3:0/3913113583 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff77c016040 con 0x562c79f9a490 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff797a9a740 1 -- 172.21.15.3:0/3913113583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff77c0054b0 con 0x562c79f9a490 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff7727fc640 1 -- 172.21.15.3:0/3913113583 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff77c005b90 con 0x562c79f9a490 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff797a9a740 10 client.16302 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff797a9a740 1 -- 172.21.15.3:0/3913113583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562c7a3f78a0 con 0x562c79f9a490 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff7727fc640 1 -- 172.21.15.3:0/3913113583 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff77c0054b0 con 0x562c79f9a490 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff7727fc640 1 -- 172.21.15.3:0/3913113583 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff77c036890 con 0x562c79f9a490 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff797a9a740 10 client.16302 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff797a9a740 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff797a9a740 20 client.16302 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff797a9a740 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562c79f57080 0x562c7a3f9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff797a9a740 4 client.16302 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.172+0000 7ff797a9a740 1 -- 172.21.15.3:0/3913113583 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562c7a2de860 con 0x562c79f57080 2024-09-21T06:17:21.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.173+0000 7ff783fff640 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562c79f57080 0x562c7a3f9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.173+0000 7ff783fff640 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562c79f57080 0x562c7a3f9df0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fafdce966e4eed server_cookie=fdfa6c120fb48b81 in_seq=0 out_seq=0 2024-09-21T06:17:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.173+0000 7ff7727fc640 10 client.16302 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff7727fc640 1 -- 172.21.15.3:0/3913113583 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c7a2de860 con 0x562c79f57080 2024-09-21T06:17:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff7727fc640 10 client.16302 handle_command_reply: tid=0 2024-09-21T06:17:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff797a9a740 1 client.16302 shutdown 2024-09-21T06:17:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff797a9a740 1 -- 172.21.15.3:0/3913113583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 msgr2=0x562c7a382440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff797a9a740 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 0x562c7a382440 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff77c024cd0 tx=0x7ff77c009ba0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff797a9a740 1 -- 172.21.15.3:0/3913113583 shutdown_connections 2024-09-21T06:17:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff797a9a740 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562c79f57080 0x562c7a3f9df0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff797a9a740 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c79fa2670 0x562c7a382980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff797a9a740 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c79fc3350 0x562c7a37d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff797a9a740 1 --2- 172.21.15.3:0/3913113583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c79f9a490 0x562c7a382440 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff797a9a740 1 -- 172.21.15.3:0/3913113583 >> 172.21.15.3:0/3913113583 conn(0x562c79fb7740 msgr2=0x562c7a366850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:21.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff7727fc640 0 client.16302 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:21.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff797a9a740 1 -- 172.21.15.3:0/3913113583 shutdown_connections 2024-09-21T06:17:21.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff797a9a740 1 -- 172.21.15.3:0/3913113583 wait complete. 2024-09-21T06:17:21.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.175+0000 7ff797a9a740 20 client.16302 trim_cache size 0 max 16384 2024-09-21T06:17:21.180 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:17:21.180 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:17:21.180 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9", 2024-09-21T06:17:21.180 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:17:21.180 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:17:21.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.176+0000 7ff791526640 1 -- 172.21.15.3:0/3916607933 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7780b6700 msgr2=0x7ff7780b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:21.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.176+0000 7ff791526640 1 --2- 172.21.15.3:0/3916607933 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7780b6700 0x7ff7780b8c10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff774005f70 tx=0x7ff774041040 comp rx=0 tx=0).stop 2024-09-21T06:17:21.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.176+0000 7ff791526640 1 -- 172.21.15.3:0/3916607933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78c108cf0 msgr2=0x7ff78c19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:21.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.176+0000 7ff791526640 1 --2- 172.21.15.3:0/3916607933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78c108cf0 0x7ff78c19fcd0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff78400e940 tx=0x7ff78400ee00 comp rx=0 tx=0).stop 2024-09-21T06:17:21.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.176+0000 7ff791526640 1 -- 172.21.15.3:0/3916607933 shutdown_connections 2024-09-21T06:17:21.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.176+0000 7ff791526640 1 --2- 172.21.15.3:0/3916607933 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7780b6700 0x7ff7780b8c10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.176+0000 7ff791526640 1 --2- 172.21.15.3:0/3916607933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff78c1083a0 0x7ff78c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.176+0000 7ff791526640 1 --2- 172.21.15.3:0/3916607933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78c108cf0 0x7ff78c19fcd0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.176+0000 7ff791526640 1 --2- 172.21.15.3:0/3916607933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff78c102a60 0x7ff78c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.176+0000 7ff791526640 1 -- 172.21.15.3:0/3916607933 >> 172.21.15.3:0/3916607933 conn(0x7ff78c079730 msgr2=0x7ff78c06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:21.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.176+0000 7ff791526640 1 -- 172.21.15.3:0/3916607933 shutdown_connections 2024-09-21T06:17:21.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:21.176+0000 7ff791526640 1 -- 172.21.15.3:0/3916607933 wait complete. 2024-09-21T06:17:21.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:21 smithi137 ceph-mon[34464]: pgmap v395: 97 pgs: 97 active+clean; 44 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.9 MiB/s wr, 332 op/s 2024-09-21T06:17:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:21 smithi204 ceph-mon[34659]: pgmap v395: 97 pgs: 97 active+clean; 44 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.9 MiB/s wr, 332 op/s 2024-09-21T06:17:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:21 smithi003 ceph-mon[31227]: pgmap v395: 97 pgs: 97 active+clean; 44 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.9 MiB/s wr, 332 op/s 2024-09-21T06:17:21.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 141895 14.52 MB/sec warmup 54 sec latency 48.999 ms 2024-09-21T06:17:22.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:17:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:22.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:22 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:17:22.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:22 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:22.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:22 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:17:22.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:22 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:17:22.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:22 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:17:22.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:22 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:22.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:22 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:22.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:22 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:22.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:22 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:17:22.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:22 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:22.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:22 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:17:22.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:22 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:17:22.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:22 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:17:22.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:22 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:22.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:22 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:22.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:22 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:22.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:22 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:17:22.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:22 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:22.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:22 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:17:22.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:22 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:17:22.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:22 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:17:22.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:22 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:22.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:22 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:22.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:22 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:22.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 143560 14.38 MB/sec warmup 55 sec latency 24.660 ms 2024-09-21T06:17:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.049+0000 7f2e09ce3640 1 -- 172.21.15.3:0/285797728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e0410de00 msgr2=0x7f2e04110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.049+0000 7f2e09ce3640 1 --2- 172.21.15.3:0/285797728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e0410de00 0x7f2e04110310 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009ca0 tx=0x7f2dfc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.049+0000 7f2e09ce3640 1 -- 172.21.15.3:0/285797728 shutdown_connections 2024-09-21T06:17:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.049+0000 7f2e09ce3640 1 --2- 172.21.15.3:0/285797728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e0410de00 0x7f2e04110310 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.049+0000 7f2e09ce3640 1 --2- 172.21.15.3:0/285797728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e04105280 0x7f2e0410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.049+0000 7f2e09ce3640 1 --2- 172.21.15.3:0/285797728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e04104850 0x7f2e04104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.049+0000 7f2e09ce3640 1 -- 172.21.15.3:0/285797728 >> 172.21.15.3:0/285797728 conn(0x7f2e04079730 msgr2=0x7f2e04079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.049+0000 7f2e09ce3640 1 -- 172.21.15.3:0/285797728 shutdown_connections 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.049+0000 7f2e09ce3640 1 -- 172.21.15.3:0/285797728 wait complete. 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.050+0000 7f2e09ce3640 1 Processor -- start 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.050+0000 7f2e09ce3640 1 -- start start 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e09ce3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e04104850 0x7f2e0419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e09ce3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e04105280 0x7f2e0419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e09ce3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e0410de00 0x7f2e0419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e09ce3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e04113a80 con 0x7f2e0410de00 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e09ce3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e04113900 con 0x7f2e04105280 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e09ce3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e04113c00 con 0x7f2e04104850 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e03fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e0410de00 0x7f2e0419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e03fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e0410de00 0x7f2e0419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45920/0 (socket says 172.21.15.3:45920) 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e02ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e04105280 0x7f2e0419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e03fff640 1 -- 172.21.15.3:0/3800131141 learned_addr learned my addr 172.21.15.3:0/3800131141 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e037fe640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e04104850 0x7f2e0419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e02ffd640 1 -- 172.21.15.3:0/3800131141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e04104850 msgr2=0x7f2e0419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e02ffd640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e04104850 0x7f2e0419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e02ffd640 1 -- 172.21.15.3:0/3800131141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e0410de00 msgr2=0x7f2e0419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e02ffd640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e0410de00 0x7f2e0419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e02ffd640 1 -- 172.21.15.3:0/3800131141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e041a02e0 con 0x7f2e04105280 2024-09-21T06:17:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.051+0000 7f2e03fff640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e0410de00 0x7f2e0419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.052+0000 7f2e02ffd640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e04105280 0x7f2e0419a9d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2e0406b950 tx=0x7f2df800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.052+0000 7f2e00ff9640 1 -- 172.21.15.3:0/3800131141 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df800cc40 con 0x7f2e04105280 2024-09-21T06:17:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.052+0000 7f2e00ff9640 1 -- 172.21.15.3:0/3800131141 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df800cde0 con 0x7f2e04105280 2024-09-21T06:17:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.052+0000 7f2e09ce3640 1 -- 172.21.15.3:0/3800131141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e041a05d0 con 0x7f2e04105280 2024-09-21T06:17:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.052+0000 7f2e00ff9640 1 -- 172.21.15.3:0/3800131141 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8018780 con 0x7f2e04105280 2024-09-21T06:17:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.052+0000 7f2e09ce3640 1 -- 172.21.15.3:0/3800131141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e041a6df0 con 0x7f2e04105280 2024-09-21T06:17:23.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.054+0000 7f2e10257740 1 Processor -- start 2024-09-21T06:17:23.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e10257740 1 -- start start 2024-09-21T06:17:23.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e10257740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef347070 0x5649ef6ee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e10257740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e10257740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e10257740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5649ef664e50 con 0x5649ef326390 2024-09-21T06:17:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e10257740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5649ef654280 con 0x5649ef31e1b0 2024-09-21T06:17:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e10257740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5649ef658090 con 0x5649ef347070 2024-09-21T06:17:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e00ff9640 1 -- 172.21.15.3:0/3800131141 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f2df8010a90 con 0x7f2e04105280 2024-09-21T06:17:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6ee7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:44998/0 (socket says 172.21.15.3:44998) 2024-09-21T06:17:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e03fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef347070 0x5649ef6ee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e037fe640 1 -- 172.21.15.3:0/620512447 learned_addr learned my addr 172.21.15.3:0/620512447 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e02ffd640 1 --2- 172.21.15.3:0/620512447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e037fe640 1 -- 172.21.15.3:0/620512447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef347070 msgr2=0x5649ef6ee270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e037fe640 1 --2- 172.21.15.3:0/620512447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef347070 0x5649ef6ee270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e037fe640 1 -- 172.21.15.3:0/620512447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 msgr2=0x5649ef6eecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e037fe640 1 --2- 172.21.15.3:0/620512447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6eecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.055+0000 7f2e037fe640 1 -- 172.21.15.3:0/620512447 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ef3b2650 con 0x5649ef31e1b0 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.056+0000 7f2e037fe640 1 --2- 172.21.15.3:0/620512447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6ee7b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2df400ac70 tx=0x7f2df400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac06d065b69eeb7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.056+0000 7f2de9ffb640 1 -- 172.21.15.3:0/620512447 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4005220 con 0x5649ef31e1b0 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.056+0000 7f2de9ffb640 1 -- 172.21.15.3:0/620512447 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df40047a0 con 0x5649ef31e1b0 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.056+0000 7f2de9ffb640 1 -- 172.21.15.3:0/620512447 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4018780 con 0x5649ef31e1b0 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.056+0000 7f2e00ff9640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2dec0b66b0 0x7f2dec0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.056+0000 7f2e03fff640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2dec0b66b0 0x7f2dec0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.056+0000 7f2e00ff9640 1 -- 172.21.15.3:0/3800131141 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2df80e7960 con 0x7f2e04105280 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.057+0000 7f2e10257740 1 -- 172.21.15.3:0/620512447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 msgr2=0x5649ef6ee7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.057+0000 7f2e10257740 1 --2- 172.21.15.3:0/620512447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6ee7b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2df400ac70 tx=0x7f2df400f790 comp rx=0 tx=0).stop 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.057+0000 7f2e03fff640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2dec0b66b0 0x7f2dec0b8bc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0094e0 tx=0x7f2dfc005c50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.057+0000 7f2e10257740 1 -- 172.21.15.3:0/620512447 shutdown_connections 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.057+0000 7f2e10257740 1 --2- 172.21.15.3:0/620512447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6eecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.057+0000 7f2e10257740 1 --2- 172.21.15.3:0/620512447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6ee7b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.057+0000 7f2e10257740 1 --2- 172.21.15.3:0/620512447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef347070 0x5649ef6ee270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.057+0000 7f2e10257740 1 -- 172.21.15.3:0/620512447 >> 172.21.15.3:0/620512447 conn(0x5649ef33b460 msgr2=0x5649ef6ec160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.057+0000 7f2e10257740 1 -- 172.21.15.3:0/620512447 shutdown_connections 2024-09-21T06:17:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.057+0000 7f2e10257740 1 -- 172.21.15.3:0/620512447 wait complete. 2024-09-21T06:17:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.057+0000 7f2e10257740 1 -- start start 2024-09-21T06:17:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e10257740 1 Processor -- start 2024-09-21T06:17:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e10257740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef31e1b0 0x5649ef6f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e10257740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e10257740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef347070 0x5649ef705d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e10257740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5649ef4b5700 con 0x5649ef326390 2024-09-21T06:17:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e10257740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5649ef5ef110 con 0x5649ef347070 2024-09-21T06:17:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e10257740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5649ef6fca50 con 0x5649ef31e1b0 2024-09-21T06:17:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e03fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef347070 0x5649ef705d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e10257740 10 client.0 _ll_register_callbacks cb 0x5649ef608990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e02ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e037fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef31e1b0 0x5649ef6f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e03fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef347070 0x5649ef705d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:45000/0 (socket says 172.21.15.3:45000) 2024-09-21T06:17:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e03fff640 1 -- 172.21.15.3:0/4145186412 learned_addr learned my addr 172.21.15.3:0/4145186412 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.058+0000 7f2e02ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6f9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45936/0 (socket says 172.21.15.3:45936) 2024-09-21T06:17:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.059+0000 7f2e03fff640 1 -- 172.21.15.3:0/4145186412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef31e1b0 msgr2=0x5649ef6f98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.059+0000 7f2e03fff640 1 --2- 172.21.15.3:0/4145186412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef31e1b0 0x5649ef6f98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.059+0000 7f2e03fff640 1 -- 172.21.15.3:0/4145186412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 msgr2=0x5649ef6f9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.059+0000 7f2e03fff640 1 --2- 172.21.15.3:0/4145186412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6f9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.059+0000 7f2e03fff640 1 -- 172.21.15.3:0/4145186412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ef61cc40 con 0x5649ef347070 2024-09-21T06:17:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.059+0000 7f2e037fe640 1 --2- 172.21.15.3:0/4145186412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef31e1b0 0x5649ef6f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.059+0000 7f2e03fff640 1 --2- 172.21.15.3:0/4145186412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef347070 0x5649ef705d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc031a30 tx=0x7f2dfc02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57e4d83f02d01ff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.059+0000 7f2de2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.059+0000 7f2de2ffd640 1 -- 172.21.15.3:0/4145186412 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc004310 con 0x5649ef347070 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.059+0000 7f2de2ffd640 1 -- 172.21.15.3:0/4145186412 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2dfc0044b0 con 0x5649ef347070 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.059+0000 7f2e10257740 1 -- 172.21.15.3:0/4145186412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df40047a0 con 0x5649ef347070 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.060+0000 7f2de2ffd640 1 -- 172.21.15.3:0/4145186412 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2dfc0046b0 con 0x5649ef347070 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.060+0000 7f2e10257740 10 client.25898 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.060+0000 7f2e10257740 1 -- 172.21.15.3:0/4145186412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5649ef708700 con 0x5649ef347070 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.060+0000 7f2de2ffd640 1 -- 172.21.15.3:0/4145186412 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df40047a0 con 0x5649ef347070 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.060+0000 7f2de2ffd640 1 -- 172.21.15.3:0/4145186412 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2dfc052070 con 0x5649ef347070 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.060+0000 7f2e10257740 10 client.25898 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.060+0000 7f2e10257740 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.060+0000 7f2e10257740 20 client.25898 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.060+0000 7f2e10257740 1 --2- 172.21.15.3:0/4145186412 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5649ef3059f0 0x5649ef70bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.060+0000 7f2e10257740 4 client.25898 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.060+0000 7f2e10257740 1 -- 172.21.15.3:0/4145186412 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649ef666da0 con 0x5649ef3059f0 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.060+0000 7f2e037fe640 1 --2- 172.21.15.3:0/4145186412 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5649ef3059f0 0x5649ef70bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.061+0000 7f2de2ffd640 10 client.25898 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.061+0000 7f2e037fe640 1 --2- 172.21.15.3:0/4145186412 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5649ef3059f0 0x5649ef70bc50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f05d5ced41da9a server_cookie=4ef60466b75f10b8 in_seq=0 out_seq=0 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.062+0000 7f2de2ffd640 1 -- 172.21.15.3:0/4145186412 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649ef666da0 con 0x5649ef3059f0 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.062+0000 7f2de2ffd640 10 client.25898 handle_command_reply: tid=0 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.062+0000 7f2e10257740 1 client.25898 shutdown 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.062+0000 7f2e10257740 1 -- 172.21.15.3:0/4145186412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef347070 msgr2=0x5649ef705d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.062+0000 7f2e10257740 1 --2- 172.21.15.3:0/4145186412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef347070 0x5649ef705d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc031a30 tx=0x7f2dfc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.062+0000 7f2e10257740 1 -- 172.21.15.3:0/4145186412 shutdown_connections 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.062+0000 7f2e10257740 1 --2- 172.21.15.3:0/4145186412 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5649ef3059f0 0x5649ef70bc50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.062+0000 7f2e10257740 1 --2- 172.21.15.3:0/4145186412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef347070 0x5649ef705d40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.062+0000 7f2e10257740 1 --2- 172.21.15.3:0/4145186412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6f9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.062+0000 7f2e10257740 1 --2- 172.21.15.3:0/4145186412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef31e1b0 0x5649ef6f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.062+0000 7f2de2ffd640 0 client.25898 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:23.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.062+0000 7f2e10257740 1 -- 172.21.15.3:0/4145186412 >> 172.21.15.3:0/4145186412 conn(0x5649ef33b460 msgr2=0x5649ef6f1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:23.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.063+0000 7f2e10257740 1 -- 172.21.15.3:0/4145186412 shutdown_connections 2024-09-21T06:17:23.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.063+0000 7f2e10257740 1 -- 172.21.15.3:0/4145186412 wait complete. 2024-09-21T06:17:23.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.063+0000 7f2e10257740 20 client.25898 trim_cache size 0 max 16384 2024-09-21T06:17:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.082+0000 7f2e10257740 1 Processor -- start 2024-09-21T06:17:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.082+0000 7f2e10257740 1 -- start start 2024-09-21T06:17:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.082+0000 7f2e10257740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.082+0000 7f2e10257740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef326390 0x5649ef6eb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.082+0000 7f2e10257740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef347070 0x5649ef6ebc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.082+0000 7f2e10257740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e041af650 con 0x5649ef347070 2024-09-21T06:17:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.082+0000 7f2e10257740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e041b11f0 con 0x5649ef31e1b0 2024-09-21T06:17:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.082+0000 7f2e10257740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e041afd20 con 0x5649ef326390 2024-09-21T06:17:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.082+0000 7f2e03fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef347070 0x5649ef6ebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.082+0000 7f2e03fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef347070 0x5649ef6ebc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45946/0 (socket says 172.21.15.3:45946) 2024-09-21T06:17:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.082+0000 7f2e03fff640 1 -- 172.21.15.3:0/395939142 learned_addr learned my addr 172.21.15.3:0/395939142 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.082+0000 7f2e037fe640 1 --2- 172.21.15.3:0/395939142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.082+0000 7f2e02ffd640 1 --2- 172.21.15.3:0/395939142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef326390 0x5649ef6eb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.083+0000 7f2e03fff640 1 -- 172.21.15.3:0/395939142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef326390 msgr2=0x5649ef6eb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:23.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.083+0000 7f2e03fff640 1 --2- 172.21.15.3:0/395939142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef326390 0x5649ef6eb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.083+0000 7f2e03fff640 1 -- 172.21.15.3:0/395939142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 msgr2=0x5649ef6ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.083+0000 7f2e03fff640 1 --2- 172.21.15.3:0/395939142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6ef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.083+0000 7f2e03fff640 1 -- 172.21.15.3:0/395939142 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ef6fab90 con 0x5649ef347070 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.083+0000 7f2e02ffd640 1 --2- 172.21.15.3:0/395939142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef326390 0x5649ef6eb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.083+0000 7f2e03fff640 1 --2- 172.21.15.3:0/395939142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef347070 0x5649ef6ebc50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc006010 tx=0x7f2dfc02f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8478e10d261d082f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.083+0000 7f2de97fa640 1 -- 172.21.15.3:0/395939142 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc061070 con 0x5649ef347070 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.083+0000 7f2de97fa640 1 -- 172.21.15.3:0/395939142 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dfc036040 con 0x5649ef347070 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.083+0000 7f2de97fa640 1 -- 172.21.15.3:0/395939142 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc02f8c0 con 0x5649ef347070 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.084+0000 7f2e10257740 1 -- 172.21.15.3:0/395939142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef347070 msgr2=0x5649ef6ebc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.084+0000 7f2e10257740 1 --2- 172.21.15.3:0/395939142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef347070 0x5649ef6ebc50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc006010 tx=0x7f2dfc02f410 comp rx=0 tx=0).stop 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.084+0000 7f2e10257740 1 -- 172.21.15.3:0/395939142 shutdown_connections 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.084+0000 7f2e10257740 1 --2- 172.21.15.3:0/395939142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef326390 0x5649ef6eb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.084+0000 7f2e10257740 1 --2- 172.21.15.3:0/395939142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef347070 0x5649ef6ebc50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.084+0000 7f2e10257740 1 --2- 172.21.15.3:0/395939142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6ef9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.084+0000 7f2e10257740 1 -- 172.21.15.3:0/395939142 >> 172.21.15.3:0/395939142 conn(0x5649ef33b460 msgr2=0x5649ef6ef2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.084+0000 7f2e10257740 1 -- 172.21.15.3:0/395939142 shutdown_connections 2024-09-21T06:17:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.084+0000 7f2e10257740 1 -- 172.21.15.3:0/395939142 wait complete. 2024-09-21T06:17:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e10257740 1 -- start start 2024-09-21T06:17:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e10257740 1 Processor -- start 2024-09-21T06:17:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e10257740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6fdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e10257740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6fe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e10257740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef347070 0x5649ef6fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e10257740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e041b0ac0 con 0x5649ef326390 2024-09-21T06:17:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e10257740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e041b03f0 con 0x5649ef31e1b0 2024-09-21T06:17:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e10257740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e041ad630 con 0x5649ef347070 2024-09-21T06:17:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e10257740 10 client.0 _ll_register_callbacks cb 0x7f2dfc009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e02ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6fdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e02ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6fe170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45948/0 (socket says 172.21.15.3:45948) 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e02ffd640 1 -- 172.21.15.3:0/3492785903 learned_addr learned my addr 172.21.15.3:0/3492785903 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.085+0000 7f2e03fff640 1 --2- 172.21.15.3:0/3492785903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef347070 0x5649ef6fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2e037fe640 1 -- 172.21.15.3:0/3492785903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef347070 msgr2=0x5649ef6fe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2e037fe640 1 --2- 172.21.15.3:0/3492785903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef347070 0x5649ef6fe6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2e037fe640 1 -- 172.21.15.3:0/3492785903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 msgr2=0x5649ef6fe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2e037fe640 1 --2- 172.21.15.3:0/3492785903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6fe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2e037fe640 1 -- 172.21.15.3:0/3492785903 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ef775650 con 0x5649ef31e1b0 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2e02ffd640 1 --2- 172.21.15.3:0/3492785903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2e037fe640 1 --2- 172.21.15.3:0/3492785903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6fdc30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2df4000c00 tx=0x7f2df400af70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3827fad5b0eac0c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2de2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2de2ffd640 1 -- 172.21.15.3:0/3492785903 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4004580 con 0x5649ef31e1b0 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2e10257740 1 -- 172.21.15.3:0/3492785903 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dfc036040 con 0x5649ef31e1b0 2024-09-21T06:17:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2de2ffd640 1 -- 172.21.15.3:0/3492785903 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2df4004720 con 0x5649ef31e1b0 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2de2ffd640 1 -- 172.21.15.3:0/3492785903 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2df403a050 con 0x5649ef31e1b0 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2e10257740 10 client.25904 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.086+0000 7f2e10257740 1 -- 172.21.15.3:0/3492785903 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5649ef776890 con 0x5649ef31e1b0 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.087+0000 7f2de2ffd640 1 -- 172.21.15.3:0/3492785903 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dfc036040 con 0x5649ef31e1b0 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.087+0000 7f2de2ffd640 1 -- 172.21.15.3:0/3492785903 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2df4033740 con 0x5649ef31e1b0 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.087+0000 7f2e10257740 10 client.25904 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.087+0000 7f2e10257740 10 client.25904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.087+0000 7f2e10257740 20 client.25904 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.087+0000 7f2e10257740 1 --2- 172.21.15.3:0/3492785903 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5649ef2dada0 0x5649ef778ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.087+0000 7f2e10257740 4 client.25904 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.087+0000 7f2e10257740 1 -- 172.21.15.3:0/3492785903 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5649ef662850 con 0x5649ef2dada0 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.087+0000 7f2e02ffd640 1 --2- 172.21.15.3:0/3492785903 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5649ef2dada0 0x5649ef778ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2de2ffd640 10 client.25904 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2e02ffd640 1 --2- 172.21.15.3:0/3492785903 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5649ef2dada0 0x5649ef778ce0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=797c9e2462f428e9 server_cookie=4bdd443ba7e1801d in_seq=0 out_seq=0 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2de2ffd640 1 -- 172.21.15.3:0/3492785903 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x5649ef662850 con 0x5649ef2dada0 2024-09-21T06:17:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2de2ffd640 10 client.25904 handle_command_reply: tid=0 2024-09-21T06:17:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2e10257740 1 client.25904 shutdown 2024-09-21T06:17:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2e10257740 1 -- 172.21.15.3:0/3492785903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 msgr2=0x5649ef6fdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2e10257740 1 --2- 172.21.15.3:0/3492785903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6fdc30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2df4000c00 tx=0x7f2df400af70 comp rx=0 tx=0).stop 2024-09-21T06:17:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2e10257740 1 -- 172.21.15.3:0/3492785903 shutdown_connections 2024-09-21T06:17:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2e10257740 1 --2- 172.21.15.3:0/3492785903 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5649ef2dada0 0x5649ef778ce0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2e10257740 1 --2- 172.21.15.3:0/3492785903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649ef347070 0x5649ef6fe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2e10257740 1 --2- 172.21.15.3:0/3492785903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649ef326390 0x5649ef6fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2e10257740 1 --2- 172.21.15.3:0/3492785903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649ef31e1b0 0x5649ef6fdc30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2e10257740 1 -- 172.21.15.3:0/3492785903 >> 172.21.15.3:0/3492785903 conn(0x5649ef33b460 msgr2=0x5649ef6fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2de2ffd640 0 client.25904 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2e10257740 1 -- 172.21.15.3:0/3492785903 shutdown_connections 2024-09-21T06:17:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.088+0000 7f2e10257740 1 -- 172.21.15.3:0/3492785903 wait complete. 2024-09-21T06:17:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.089+0000 7f2e10257740 20 client.25904 trim_cache size 0 max 16384 2024-09-21T06:17:23.095 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:17:23.095 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:17:23.095 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-21T06:17:23.095 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:17:23.095 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:23.095 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9", 2024-09-21T06:17:23.095 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-21T06:17:23.095 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:17:23.095 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:17:23.095 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:17:23.095 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:17:23.095 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:17:23.095 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:17:23.096 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:23.097 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:23.097 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:23.097 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:23.097 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:23.097 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:17:23.097 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:17:23.097 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:17:23.097 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:23.097 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:23.097 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:23.097 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:23.097 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:23.097 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:17:23.098 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:17:23.098 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:17:23.098 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:23.098 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:23.098 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:23.098 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:23.098 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:23.098 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:17:23.098 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:17:23.098 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:17:23.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.089+0000 7f2e09ce3640 1 -- 172.21.15.3:0/3800131141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2dec0b66b0 msgr2=0x7f2dec0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:23.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.089+0000 7f2e09ce3640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2dec0b66b0 0x7f2dec0b8bc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0094e0 tx=0x7f2dfc005c50 comp rx=0 tx=0).stop 2024-09-21T06:17:23.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.089+0000 7f2e09ce3640 1 -- 172.21.15.3:0/3800131141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e04105280 msgr2=0x7f2e0419a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:23.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.089+0000 7f2e09ce3640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e04105280 0x7f2e0419a9d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2e0406b950 tx=0x7f2df800ee50 comp rx=0 tx=0).stop 2024-09-21T06:17:23.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.089+0000 7f2e09ce3640 1 -- 172.21.15.3:0/3800131141 shutdown_connections 2024-09-21T06:17:23.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.089+0000 7f2e09ce3640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2dec0b66b0 0x7f2dec0b8bc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.089+0000 7f2e09ce3640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e0410de00 0x7f2e0419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.089+0000 7f2e09ce3640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e04105280 0x7f2e0419a9d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.089+0000 7f2e09ce3640 1 --2- 172.21.15.3:0/3800131141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e04104850 0x7f2e0419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:23.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.089+0000 7f2e09ce3640 1 -- 172.21.15.3:0/3800131141 >> 172.21.15.3:0/3800131141 conn(0x7f2e04079730 msgr2=0x7f2e04100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:23.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.089+0000 7f2e09ce3640 1 -- 172.21.15.3:0/3800131141 shutdown_connections 2024-09-21T06:17:23.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:23.089+0000 7f2e09ce3640 1 -- 172.21.15.3:0/3800131141 wait complete. 2024-09-21T06:17:23.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:23 smithi137 ceph-mon[34464]: pgmap v396: 97 pgs: 97 active+clean; 48 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 322 op/s 2024-09-21T06:17:23.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:23 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:17:23.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:23 smithi204 ceph-mon[34659]: pgmap v396: 97 pgs: 97 active+clean; 48 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 322 op/s 2024-09-21T06:17:23.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:23 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:17:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:23 smithi003 ceph-mon[31227]: pgmap v396: 97 pgs: 97 active+clean; 48 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 322 op/s 2024-09-21T06:17:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:23 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:17:23.905 INFO:tasks.cephfs.filesystem:scrub status for tag:a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9 - {'path': '/', 'tag': 'a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9', 'options': 'recursive,force'} 2024-09-21T06:17:23.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 145921 14.38 MB/sec warmup 56 sec latency 9.667 ms 2024-09-21T06:17:24.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:17:24.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:17:24.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:17:24.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:17:24.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:17:24.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:17:24.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:17:24.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:17:24.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:17:24.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 148469 14.34 MB/sec warmup 57 sec latency 85.657 ms 2024-09-21T06:17:25.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:25 smithi137 ceph-mon[34464]: pgmap v397: 97 pgs: 97 active+clean; 48 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 307 op/s 2024-09-21T06:17:25.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:25 smithi204 ceph-mon[34659]: pgmap v397: 97 pgs: 97 active+clean; 48 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 307 op/s 2024-09-21T06:17:25.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:25 smithi003 ceph-mon[31227]: pgmap v397: 97 pgs: 97 active+clean; 48 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 307 op/s 2024-09-21T06:17:25.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 150450 14.30 MB/sec warmup 58 sec latency 13.567 ms 2024-09-21T06:17:26.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 152978 14.29 MB/sec warmup 59 sec latency 13.998 ms 2024-09-21T06:17:27.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:27 smithi137 ceph-mon[34464]: pgmap v398: 97 pgs: 97 active+clean; 56 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.4 MiB/s wr, 393 op/s 2024-09-21T06:17:27.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:27 smithi204 ceph-mon[34659]: pgmap v398: 97 pgs: 97 active+clean; 56 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.4 MiB/s wr, 393 op/s 2024-09-21T06:17:27.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:27 smithi003 ceph-mon[31227]: pgmap v398: 97 pgs: 97 active+clean; 56 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.4 MiB/s wr, 393 op/s 2024-09-21T06:17:27.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 156163 14.35 MB/sec warmup 60 sec latency 21.251 ms 2024-09-21T06:17:28.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 157869 14.22 MB/sec warmup 61 sec latency 13.223 ms 2024-09-21T06:17:29.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:29 smithi137 ceph-mon[34464]: pgmap v399: 97 pgs: 97 active+clean; 57 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-09-21T06:17:29.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:29 smithi204 ceph-mon[34659]: pgmap v399: 97 pgs: 97 active+clean; 57 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-09-21T06:17:29.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:29 smithi003 ceph-mon[31227]: pgmap v399: 97 pgs: 97 active+clean; 57 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-09-21T06:17:29.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 160048 14.22 MB/sec warmup 62 sec latency 65.512 ms 2024-09-21T06:17:30.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 162828 14.20 MB/sec warmup 63 sec latency 12.581 ms 2024-09-21T06:17:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.947+0000 7f10be5a1640 1 -- 172.21.15.3:0/554820361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b8108c30 msgr2=0x7f10b810f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.947+0000 7f10be5a1640 1 --2- 172.21.15.3:0/554820361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b8108c30 0x7f10b810f460 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009ca0 tx=0x7f10b002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.948+0000 7f10be5a1640 1 -- 172.21.15.3:0/554820361 shutdown_connections 2024-09-21T06:17:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.948+0000 7f10be5a1640 1 --2- 172.21.15.3:0/554820361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10b81082e0 0x7f10b81086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.948+0000 7f10be5a1640 1 --2- 172.21.15.3:0/554820361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b8108c30 0x7f10b810f460 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.948+0000 7f10be5a1640 1 --2- 172.21.15.3:0/554820361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10b81029b0 0x7f10b8102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.948+0000 7f10be5a1640 1 -- 172.21.15.3:0/554820361 >> 172.21.15.3:0/554820361 conn(0x7f10b807a740 msgr2=0x7f10b807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.948+0000 7f10be5a1640 1 -- 172.21.15.3:0/554820361 shutdown_connections 2024-09-21T06:17:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.948+0000 7f10be5a1640 1 -- 172.21.15.3:0/554820361 wait complete. 2024-09-21T06:17:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.948+0000 7f10be5a1640 1 Processor -- start 2024-09-21T06:17:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.948+0000 7f10be5a1640 1 -- start start 2024-09-21T06:17:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10be5a1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b81029b0 0x7f10b819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10be5a1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10b81082e0 0x7f10b819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10be5a1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10b8108c30 0x7f10b819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10be5a1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f10b8112940 con 0x7f10b81029b0 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10be5a1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81127c0 con 0x7f10b8108c30 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10be5a1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f10b8112ac0 con 0x7f10b81082e0 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b81029b0 0x7f10b819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10b81082e0 0x7f10b819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b81029b0 0x7f10b819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44278/0 (socket says 172.21.15.3:44278) 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10b7fff640 1 -- 172.21.15.3:0/643578983 learned_addr learned my addr 172.21.15.3:0/643578983 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10bcb3e640 1 --2- 172.21.15.3:0/643578983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10b8108c30 0x7f10b819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10b77fe640 1 -- 172.21.15.3:0/643578983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10b8108c30 msgr2=0x7f10b819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10b77fe640 1 --2- 172.21.15.3:0/643578983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10b8108c30 0x7f10b819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10b77fe640 1 -- 172.21.15.3:0/643578983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b81029b0 msgr2=0x7f10b819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10b77fe640 1 --2- 172.21.15.3:0/643578983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b81029b0 0x7f10b819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.949+0000 7f10b77fe640 1 -- 172.21.15.3:0/643578983 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b81a0380 con 0x7f10b81082e0 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.950+0000 7f10b77fe640 1 --2- 172.21.15.3:0/643578983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10b81082e0 0x7f10b819a9f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f10b806c9c0 tx=0x7f10ac00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.950+0000 7f10b57fa640 1 -- 172.21.15.3:0/643578983 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00cc40 con 0x7f10b81082e0 2024-09-21T06:17:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.950+0000 7f10b57fa640 1 -- 172.21.15.3:0/643578983 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ac00cde0 con 0x7f10b81082e0 2024-09-21T06:17:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.950+0000 7f10be5a1640 1 -- 172.21.15.3:0/643578983 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b81a0670 con 0x7f10b81082e0 2024-09-21T06:17:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.950+0000 7f10b57fa640 1 -- 172.21.15.3:0/643578983 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac004a00 con 0x7f10b81082e0 2024-09-21T06:17:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.950+0000 7f10be5a1640 1 -- 172.21.15.3:0/643578983 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b81a6e00 con 0x7f10b81082e0 2024-09-21T06:17:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10c55e6740 1 Processor -- start 2024-09-21T06:17:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10c55e6740 1 -- start start 2024-09-21T06:17:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10c55e6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83541b0 0x55f3d86f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10c55e6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d832b2f0 0x55f3d86f9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10c55e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d83334d0 0x55f3d86fa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10c55e6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f3d866e6f0 con 0x55f3d832b2f0 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10c55e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f3d865db20 con 0x55f3d83334d0 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10c55e6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f3d8661930 con 0x55f3d83541b0 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d832b2f0 0x55f3d86f9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d832b2f0 0x55f3d86f9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44290/0 (socket says 172.21.15.3:44290) 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10b7fff640 1 -- 172.21.15.3:0/1017262297 learned_addr learned my addr 172.21.15.3:0/1017262297 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10b57fa640 1 -- 172.21.15.3:0/643578983 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f10ac004ba0 con 0x7f10b81082e0 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10bcb3e640 1 --2- 172.21.15.3:0/1017262297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83541b0 0x55f3d86f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10b77fe640 1 --2- 172.21.15.3:0/1017262297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d83334d0 0x55f3d86fa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10b7fff640 1 -- 172.21.15.3:0/1017262297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83541b0 msgr2=0x55f3d86f99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10b7fff640 1 --2- 172.21.15.3:0/1017262297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83541b0 0x55f3d86f99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10b7fff640 1 -- 172.21.15.3:0/1017262297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d83334d0 msgr2=0x55f3d86fa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10b7fff640 1 --2- 172.21.15.3:0/1017262297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d83334d0 0x55f3d86fa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10b7fff640 1 -- 172.21.15.3:0/1017262297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3d8623530 con 0x55f3d832b2f0 2024-09-21T06:17:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.953+0000 7f10b77fe640 1 --2- 172.21.15.3:0/1017262297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d83334d0 0x55f3d86fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:17:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.954+0000 7f10b7fff640 1 --2- 172.21.15.3:0/1017262297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d832b2f0 0x55f3d86f9ef0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009dc0 tx=0x7f10a8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa14f8a5e58a058 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.954+0000 7f109e7fc640 1 -- 172.21.15.3:0/1017262297 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8005220 con 0x55f3d832b2f0 2024-09-21T06:17:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.954+0000 7f109e7fc640 1 -- 172.21.15.3:0/1017262297 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a8004c50 con 0x55f3d832b2f0 2024-09-21T06:17:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.954+0000 7f109e7fc640 1 -- 172.21.15.3:0/1017262297 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8019830 con 0x55f3d832b2f0 2024-09-21T06:17:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.954+0000 7f10b57fa640 1 --2- 172.21.15.3:0/643578983 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f10a00b6680 0x7f10a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.954+0000 7f10bcb3e640 1 --2- 172.21.15.3:0/643578983 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f10a00b6680 0x7f10a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.954+0000 7f10b57fa640 1 -- 172.21.15.3:0/643578983 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f10ac0e75e0 con 0x7f10b81082e0 2024-09-21T06:17:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.955+0000 7f10c55e6740 1 -- 172.21.15.3:0/1017262297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d832b2f0 msgr2=0x55f3d86f9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.955+0000 7f10bcb3e640 1 --2- 172.21.15.3:0/643578983 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f10a00b6680 0x7f10a00b8b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f10b0006010 tx=0x7f10b0005d80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.955+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1017262297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d832b2f0 0x55f3d86f9ef0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009dc0 tx=0x7f10a8011810 comp rx=0 tx=0).stop 2024-09-21T06:17:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.955+0000 7f10c55e6740 1 -- 172.21.15.3:0/1017262297 shutdown_connections 2024-09-21T06:17:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.955+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1017262297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d83334d0 0x55f3d86fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.955+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1017262297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d832b2f0 0x55f3d86f9ef0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.955+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1017262297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83541b0 0x55f3d86f99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.955+0000 7f10c55e6740 1 -- 172.21.15.3:0/1017262297 >> 172.21.15.3:0/1017262297 conn(0x55f3d8348300 msgr2=0x55f3d86f78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.955+0000 7f10c55e6740 1 -- 172.21.15.3:0/1017262297 shutdown_connections 2024-09-21T06:17:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.955+0000 7f10c55e6740 1 -- 172.21.15.3:0/1017262297 wait complete. 2024-09-21T06:17:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.955+0000 7f10c55e6740 1 -- start start 2024-09-21T06:17:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.956+0000 7f10c55e6740 1 Processor -- start 2024-09-21T06:17:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.956+0000 7f10c55e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d832b2f0 0x55f3d8705020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.956+0000 7f10c55e6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 0x55f3d8705560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.956+0000 7f10c55e6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83541b0 0x55f3d8711420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.956+0000 7f10c55e6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f3d8595ab0 con 0x55f3d83334d0 2024-09-21T06:17:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.956+0000 7f10c55e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f3d8603730 con 0x55f3d832b2f0 2024-09-21T06:17:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.956+0000 7f10c55e6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f3d8708130 con 0x55f3d83541b0 2024-09-21T06:17:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.956+0000 7f10c55e6740 10 client.0 _ll_register_callbacks cb 0x55f3d86576b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.956+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 0x55f3d8705560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.956+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d832b2f0 0x55f3d8705020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.956+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 0x55f3d8705560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44304/0 (socket says 172.21.15.3:44304) 2024-09-21T06:17:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.956+0000 7f10b77fe640 1 -- 172.21.15.3:0/1940290072 learned_addr learned my addr 172.21.15.3:0/1940290072 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.956+0000 7f10bcb3e640 1 --2- 172.21.15.3:0/1940290072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83541b0 0x55f3d8711420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10b7fff640 1 -- 172.21.15.3:0/1940290072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83541b0 msgr2=0x55f3d8711420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10b7fff640 1 --2- 172.21.15.3:0/1940290072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83541b0 0x55f3d8711420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10b7fff640 1 -- 172.21.15.3:0/1940290072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 msgr2=0x55f3d8705560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10b7fff640 1 --2- 172.21.15.3:0/1940290072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 0x55f3d8705560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10b7fff640 1 -- 172.21.15.3:0/1940290072 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3d83bf790 con 0x55f3d832b2f0 2024-09-21T06:17:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10b77fe640 1 --2- 172.21.15.3:0/1940290072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 0x55f3d8705560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10b7fff640 1 --2- 172.21.15.3:0/1940290072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d832b2f0 0x55f3d8705020 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f10a8019510 tx=0x7f10a80050b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc86a2796412f8c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10977fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10977fe640 1 -- 172.21.15.3:0/1940290072 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8004450 con 0x55f3d832b2f0 2024-09-21T06:17:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10977fe640 1 -- 172.21.15.3:0/1940290072 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f10a80035f0 con 0x55f3d832b2f0 2024-09-21T06:17:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10c55e6740 1 -- 172.21.15.3:0/1940290072 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a8004c50 con 0x55f3d832b2f0 2024-09-21T06:17:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10977fe640 1 -- 172.21.15.3:0/1940290072 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10a8032050 con 0x55f3d832b2f0 2024-09-21T06:17:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10c55e6740 10 client.25913 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.957+0000 7f10c55e6740 1 -- 172.21.15.3:0/1940290072 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3d8438620 con 0x55f3d832b2f0 2024-09-21T06:17:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.958+0000 7f10977fe640 1 -- 172.21.15.3:0/1940290072 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a8004c50 con 0x55f3d832b2f0 2024-09-21T06:17:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.958+0000 7f10977fe640 1 -- 172.21.15.3:0/1940290072 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f10a8003ce0 con 0x55f3d832b2f0 2024-09-21T06:17:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.958+0000 7f10c55e6740 10 client.25913 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.958+0000 7f10c55e6740 10 client.25913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.958+0000 7f10c55e6740 20 client.25913 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.958+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1940290072 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f3d8312d80 0x55f3d8717010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.958+0000 7f10c55e6740 4 client.25913 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.958+0000 7f10c55e6740 1 -- 172.21.15.3:0/1940290072 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3d8670640 con 0x55f3d8312d80 2024-09-21T06:17:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.958+0000 7f10b77fe640 1 --2- 172.21.15.3:0/1940290072 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f3d8312d80 0x55f3d8717010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.959+0000 7f10b77fe640 1 --2- 172.21.15.3:0/1940290072 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f3d8312d80 0x55f3d8717010 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=691632fd3c83e82b server_cookie=8055216f28b9c644 in_seq=0 out_seq=0 2024-09-21T06:17:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.959+0000 7f10977fe640 10 client.25913 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10977fe640 1 -- 172.21.15.3:0/1940290072 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f3d8670640 con 0x55f3d8312d80 2024-09-21T06:17:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10977fe640 10 client.25913 handle_command_reply: tid=0 2024-09-21T06:17:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10c55e6740 1 client.25913 shutdown 2024-09-21T06:17:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10c55e6740 1 -- 172.21.15.3:0/1940290072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d832b2f0 msgr2=0x55f3d8705020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1940290072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d832b2f0 0x55f3d8705020 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f10a8019510 tx=0x7f10a80050b0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10c55e6740 1 -- 172.21.15.3:0/1940290072 shutdown_connections 2024-09-21T06:17:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1940290072 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f3d8312d80 0x55f3d8717010 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1940290072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83541b0 0x55f3d8711420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1940290072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 0x55f3d8705560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10977fe640 0 client.25913 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1940290072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d832b2f0 0x55f3d8705020 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10c55e6740 1 -- 172.21.15.3:0/1940290072 >> 172.21.15.3:0/1940290072 conn(0x55f3d8348300 msgr2=0x55f3d86fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10c55e6740 1 -- 172.21.15.3:0/1940290072 shutdown_connections 2024-09-21T06:17:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10c55e6740 1 -- 172.21.15.3:0/1940290072 wait complete. 2024-09-21T06:17:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.960+0000 7f10c55e6740 20 client.25913 trim_cache size 0 max 16384 2024-09-21T06:17:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.977+0000 7f10c55e6740 1 Processor -- start 2024-09-21T06:17:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.977+0000 7f10c55e6740 1 -- start start 2024-09-21T06:17:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10c55e6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d832b2f0 0x55f3d86fb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10c55e6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 0x55f3d8706c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10c55e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d83541b0 0x55f3d8711320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10c55e6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81af6b0 con 0x55f3d83334d0 2024-09-21T06:17:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10c55e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81b1250 con 0x55f3d83541b0 2024-09-21T06:17:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10c55e6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81afd80 con 0x55f3d832b2f0 2024-09-21T06:17:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 0x55f3d8706c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 0x55f3d8706c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44310/0 (socket says 172.21.15.3:44310) 2024-09-21T06:17:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10b77fe640 1 -- 172.21.15.3:0/3733546893 learned_addr learned my addr 172.21.15.3:0/3733546893 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10bcb3e640 1 --2- 172.21.15.3:0/3733546893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d83541b0 0x55f3d8711320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10b7fff640 1 --2- 172.21.15.3:0/3733546893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d832b2f0 0x55f3d86fb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10b77fe640 1 -- 172.21.15.3:0/3733546893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d832b2f0 msgr2=0x55f3d86fb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10b77fe640 1 --2- 172.21.15.3:0/3733546893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d832b2f0 0x55f3d86fb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10b77fe640 1 -- 172.21.15.3:0/3733546893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d83541b0 msgr2=0x55f3d8711320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10b77fe640 1 --2- 172.21.15.3:0/3733546893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d83541b0 0x55f3d8711320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10b77fe640 1 -- 172.21.15.3:0/3733546893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3d8707990 con 0x55f3d83334d0 2024-09-21T06:17:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.978+0000 7f10b77fe640 1 --2- 172.21.15.3:0/3733546893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 0x55f3d8706c50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0ad400 tx=0x7f10ac0053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc4e66908d41c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.979+0000 7f109dffb640 1 -- 172.21.15.3:0/3733546893 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac01c070 con 0x55f3d83334d0 2024-09-21T06:17:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.979+0000 7f109dffb640 1 -- 172.21.15.3:0/3733546893 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ac0adb70 con 0x55f3d83334d0 2024-09-21T06:17:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.979+0000 7f109dffb640 1 -- 172.21.15.3:0/3733546893 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0e9b60 con 0x55f3d83334d0 2024-09-21T06:17:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.979+0000 7f10c55e6740 1 -- 172.21.15.3:0/3733546893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 msgr2=0x55f3d8706c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.979+0000 7f10c55e6740 1 --2- 172.21.15.3:0/3733546893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 0x55f3d8706c50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0ad400 tx=0x7f10ac0053f0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.980+0000 7f10c55e6740 1 -- 172.21.15.3:0/3733546893 shutdown_connections 2024-09-21T06:17:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.980+0000 7f10c55e6740 1 --2- 172.21.15.3:0/3733546893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d83541b0 0x55f3d8711320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.980+0000 7f10c55e6740 1 --2- 172.21.15.3:0/3733546893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83334d0 0x55f3d8706c50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.980+0000 7f10c55e6740 1 --2- 172.21.15.3:0/3733546893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d832b2f0 0x55f3d86fb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.980+0000 7f10c55e6740 1 -- 172.21.15.3:0/3733546893 >> 172.21.15.3:0/3733546893 conn(0x55f3d8348300 msgr2=0x55f3d8706540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.980+0000 7f10c55e6740 1 -- 172.21.15.3:0/3733546893 shutdown_connections 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.980+0000 7f10c55e6740 1 -- 172.21.15.3:0/3733546893 wait complete. 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.980+0000 7f10c55e6740 1 -- start start 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.980+0000 7f10c55e6740 1 Processor -- start 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.980+0000 7f10c55e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d832b2f0 0x55f3d870a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.980+0000 7f10c55e6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83334d0 0x55f3d870abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10c55e6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83541b0 0x55f3d877f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10c55e6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81b0b20 con 0x55f3d83541b0 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10c55e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81b0450 con 0x55f3d832b2f0 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10c55e6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81ad690 con 0x55f3d83334d0 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10c55e6740 10 client.0 _ll_register_callbacks cb 0x7f10ac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10bcb3e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83541b0 0x55f3d877f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10bcb3e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83541b0 0x55f3d877f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44314/0 (socket says 172.21.15.3:44314) 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d832b2f0 0x55f3d870a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83334d0 0x55f3d870abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10bcb3e640 1 -- 172.21.15.3:0/1607851676 learned_addr learned my addr 172.21.15.3:0/1607851676 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10b7fff640 1 -- 172.21.15.3:0/1607851676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83334d0 msgr2=0x55f3d870abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10b7fff640 1 --2- 172.21.15.3:0/1607851676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83334d0 0x55f3d870abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10b7fff640 1 -- 172.21.15.3:0/1607851676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83541b0 msgr2=0x55f3d877f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10b7fff640 1 --2- 172.21.15.3:0/1607851676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83541b0 0x55f3d877f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10b7fff640 1 -- 172.21.15.3:0/1607851676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3d870b450 con 0x55f3d832b2f0 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10b77fe640 1 --2- 172.21.15.3:0/1607851676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83334d0 0x55f3d870abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10b7fff640 1 --2- 172.21.15.3:0/1607851676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d832b2f0 0x55f3d870a670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f10a8019870 tx=0x7f10a8009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=872e01c89e3873a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10977fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.981+0000 7f10977fe640 1 -- 172.21.15.3:0/1607851676 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8004bd0 con 0x55f3d832b2f0 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10977fe640 1 -- 172.21.15.3:0/1607851676 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f10a8004250 con 0x55f3d832b2f0 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10c55e6740 1 -- 172.21.15.3:0/1607851676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10ac0adb70 con 0x55f3d832b2f0 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10977fe640 1 -- 172.21.15.3:0/1607851676 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10a80099d0 con 0x55f3d832b2f0 2024-09-21T06:17:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10c55e6740 10 client.25919 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10c55e6740 1 -- 172.21.15.3:0/1607851676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3d8780eb0 con 0x55f3d832b2f0 2024-09-21T06:17:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10977fe640 1 -- 172.21.15.3:0/1607851676 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ac0adb70 con 0x55f3d832b2f0 2024-09-21T06:17:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10977fe640 1 -- 172.21.15.3:0/1607851676 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f10a80035f0 con 0x55f3d832b2f0 2024-09-21T06:17:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10c55e6740 10 client.25919 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10c55e6740 10 client.25919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10c55e6740 20 client.25919 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1607851676 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f3d82e81c0 0x55f3d8783670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10c55e6740 4 client.25919 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:17:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10c55e6740 1 -- 172.21.15.3:0/1607851676 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3d866c0f0 con 0x55f3d82e81c0 2024-09-21T06:17:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.982+0000 7f10b77fe640 1 --2- 172.21.15.3:0/1607851676 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f3d82e81c0 0x55f3d8783670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.983+0000 7f10977fe640 10 client.25919 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:30.983+0000 7f10b77fe640 1 --2- 172.21.15.3:0/1607851676 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f3d82e81c0 0x55f3d8783670 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21aa422a5b1343f8 server_cookie=3473c82568e3c471 in_seq=0 out_seq=0 2024-09-21T06:17:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10977fe640 1 -- 172.21.15.3:0/1607851676 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3d866c0f0 con 0x55f3d82e81c0 2024-09-21T06:17:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10977fe640 10 client.25919 handle_command_reply: tid=0 2024-09-21T06:17:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10c55e6740 1 client.25919 shutdown 2024-09-21T06:17:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10c55e6740 1 -- 172.21.15.3:0/1607851676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d832b2f0 msgr2=0x55f3d870a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1607851676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d832b2f0 0x55f3d870a670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f10a8019870 tx=0x7f10a8009530 comp rx=0 tx=0).stop 2024-09-21T06:17:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10c55e6740 1 -- 172.21.15.3:0/1607851676 shutdown_connections 2024-09-21T06:17:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1607851676 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f3d82e81c0 0x55f3d8783670 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1607851676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d83334d0 0x55f3d870abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1607851676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3d83541b0 0x55f3d877f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10c55e6740 1 --2- 172.21.15.3:0/1607851676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3d832b2f0 0x55f3d870a670 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10c55e6740 1 -- 172.21.15.3:0/1607851676 >> 172.21.15.3:0/1607851676 conn(0x55f3d8348300 msgr2=0x55f3d86fb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10977fe640 0 client.25919 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10c55e6740 1 -- 172.21.15.3:0/1607851676 shutdown_connections 2024-09-21T06:17:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10c55e6740 1 -- 172.21.15.3:0/1607851676 wait complete. 2024-09-21T06:17:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.102+0000 7f10c55e6740 20 client.25919 trim_cache size 0 max 16384 2024-09-21T06:17:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:17:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:17:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:17:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:17:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.103+0000 7f10be5a1640 1 -- 172.21.15.3:0/643578983 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f10a00b6680 msgr2=0x7f10a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.103+0000 7f10be5a1640 1 --2- 172.21.15.3:0/643578983 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f10a00b6680 0x7f10a00b8b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f10b0006010 tx=0x7f10b0005d80 comp rx=0 tx=0).stop 2024-09-21T06:17:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.103+0000 7f10be5a1640 1 -- 172.21.15.3:0/643578983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10b81082e0 msgr2=0x7f10b819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.103+0000 7f10be5a1640 1 --2- 172.21.15.3:0/643578983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10b81082e0 0x7f10b819a9f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f10b806c9c0 tx=0x7f10ac00ee50 comp rx=0 tx=0).stop 2024-09-21T06:17:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.103+0000 7f10be5a1640 1 -- 172.21.15.3:0/643578983 shutdown_connections 2024-09-21T06:17:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.103+0000 7f10be5a1640 1 --2- 172.21.15.3:0/643578983 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f10a00b6680 0x7f10a00b8b90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.103+0000 7f10be5a1640 1 --2- 172.21.15.3:0/643578983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10b81082e0 0x7f10b819a9f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.103+0000 7f10be5a1640 1 --2- 172.21.15.3:0/643578983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10b8108c30 0x7f10b819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.103+0000 7f10be5a1640 1 --2- 172.21.15.3:0/643578983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b81029b0 0x7f10b819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.103+0000 7f10be5a1640 1 -- 172.21.15.3:0/643578983 >> 172.21.15.3:0/643578983 conn(0x7f10b807a740 msgr2=0x7f10b8101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.104+0000 7f10be5a1640 1 -- 172.21.15.3:0/643578983 shutdown_connections 2024-09-21T06:17:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:31.104+0000 7f10be5a1640 1 -- 172.21.15.3:0/643578983 wait complete. 2024-09-21T06:17:31.641 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:31 smithi003 ceph-mon[31227]: pgmap v400: 97 pgs: 97 active+clean; 61 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 400 op/s 2024-09-21T06:17:31.642 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:31 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:17:31.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:31 smithi137 ceph-mon[34464]: pgmap v400: 97 pgs: 97 active+clean; 61 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 400 op/s 2024-09-21T06:17:31.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:31 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:17:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:31 smithi204 ceph-mon[34659]: pgmap v400: 97 pgs: 97 active+clean; 61 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 400 op/s 2024-09-21T06:17:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:31 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:17:31.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 164233 14.10 MB/sec warmup 64 sec latency 81.507 ms 2024-09-21T06:17:32.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:32 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:17:32.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:32 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:17:32.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:32 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:17:32.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:32 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:17:32.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:32 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:17:32.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:32 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:17:32.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 167086 14.15 MB/sec warmup 65 sec latency 39.118 ms 2024-09-21T06:17:33.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:33 smithi137 ceph-mon[34464]: pgmap v401: 97 pgs: 97 active+clean; 63 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 387 op/s 2024-09-21T06:17:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:33 smithi204 ceph-mon[34659]: pgmap v401: 97 pgs: 97 active+clean; 63 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 387 op/s 2024-09-21T06:17:33.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:33 smithi003 ceph-mon[31227]: pgmap v401: 97 pgs: 97 active+clean; 63 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 387 op/s 2024-09-21T06:17:33.947 INFO:tasks.workunit.client.0.smithi003.stdout: 1 169916 14.14 MB/sec warmup 66 sec latency 16.914 ms 2024-09-21T06:17:34.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 171318 14.04 MB/sec warmup 67 sec latency 65.173 ms 2024-09-21T06:17:35.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:35 smithi137 ceph-mon[34464]: pgmap v402: 97 pgs: 97 active+clean; 63 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 300 op/s 2024-09-21T06:17:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:35 smithi204 ceph-mon[34659]: pgmap v402: 97 pgs: 97 active+clean; 63 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 300 op/s 2024-09-21T06:17:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:35 smithi003 ceph-mon[31227]: pgmap v402: 97 pgs: 97 active+clean; 63 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 300 op/s 2024-09-21T06:17:35.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 174207 14.09 MB/sec warmup 68 sec latency 21.412 ms 2024-09-21T06:17:36.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 176838 14.07 MB/sec warmup 69 sec latency 12.950 ms 2024-09-21T06:17:37.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:37 smithi137 ceph-mon[34464]: pgmap v403: 97 pgs: 97 active+clean; 54 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 433 op/s 2024-09-21T06:17:37.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:37 smithi204 ceph-mon[34659]: pgmap v403: 97 pgs: 97 active+clean; 54 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 433 op/s 2024-09-21T06:17:37.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:37 smithi003 ceph-mon[31227]: pgmap v403: 97 pgs: 97 active+clean; 54 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 433 op/s 2024-09-21T06:17:37.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 178624 13.99 MB/sec warmup 70 sec latency 18.911 ms 2024-09-21T06:17:38.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 181333 14.03 MB/sec warmup 71 sec latency 12.310 ms 2024-09-21T06:17:39.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:39 smithi137 ceph-mon[34464]: pgmap v404: 97 pgs: 97 active+clean; 54 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 370 op/s 2024-09-21T06:17:39.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:39 smithi204 ceph-mon[34659]: pgmap v404: 97 pgs: 97 active+clean; 54 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 370 op/s 2024-09-21T06:17:39.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:39 smithi003 ceph-mon[31227]: pgmap v404: 97 pgs: 97 active+clean; 54 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 370 op/s 2024-09-21T06:17:39.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 184147 14.02 MB/sec warmup 72 sec latency 63.729 ms 2024-09-21T06:17:40.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 185909 13.98 MB/sec warmup 73 sec latency 18.986 ms 2024-09-21T06:17:41.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:41 smithi137 ceph-mon[34464]: pgmap v405: 97 pgs: 97 active+clean; 58 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 451 op/s 2024-09-21T06:17:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:41 smithi204 ceph-mon[34659]: pgmap v405: 97 pgs: 97 active+clean; 58 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 451 op/s 2024-09-21T06:17:41.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:41 smithi003 ceph-mon[31227]: pgmap v405: 97 pgs: 97 active+clean; 58 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 451 op/s 2024-09-21T06:17:41.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 188463 13.98 MB/sec warmup 74 sec latency 13.029 ms 2024-09-21T06:17:42.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 191572 14.03 MB/sec warmup 75 sec latency 14.008 ms 2024-09-21T06:17:43.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:43 smithi137 ceph-mon[34464]: pgmap v406: 97 pgs: 97 active+clean; 58 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 391 op/s 2024-09-21T06:17:43.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:43 smithi204 ceph-mon[34659]: pgmap v406: 97 pgs: 97 active+clean; 58 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 391 op/s 2024-09-21T06:17:43.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:43 smithi003 ceph-mon[31227]: pgmap v406: 97 pgs: 97 active+clean; 58 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 391 op/s 2024-09-21T06:17:43.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 193883 13.95 MB/sec warmup 76 sec latency 16.191 ms 2024-09-21T06:17:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.209+0000 7f2e48157640 1 -- 172.21.15.3:0/2642401318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40108c30 msgr2=0x7f2e4010f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.209+0000 7f2e48157640 1 --2- 172.21.15.3:0/2642401318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40108c30 0x7f2e4010f460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009ca0 tx=0x7f2e3802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.209+0000 7f2e48157640 1 -- 172.21.15.3:0/2642401318 shutdown_connections 2024-09-21T06:17:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.209+0000 7f2e48157640 1 --2- 172.21.15.3:0/2642401318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40108c30 0x7f2e4010f460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.209+0000 7f2e48157640 1 --2- 172.21.15.3:0/2642401318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e401082e0 0x7f2e401086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.209+0000 7f2e48157640 1 --2- 172.21.15.3:0/2642401318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e401029b0 0x7f2e40102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.209+0000 7f2e48157640 1 -- 172.21.15.3:0/2642401318 >> 172.21.15.3:0/2642401318 conn(0x7f2e4007a740 msgr2=0x7f2e4007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.209+0000 7f2e48157640 1 -- 172.21.15.3:0/2642401318 shutdown_connections 2024-09-21T06:17:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.209+0000 7f2e48157640 1 -- 172.21.15.3:0/2642401318 wait complete. 2024-09-21T06:17:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.210+0000 7f2e48157640 1 Processor -- start 2024-09-21T06:17:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.210+0000 7f2e48157640 1 -- start start 2024-09-21T06:17:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.210+0000 7f2e48157640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e401029b0 0x7f2e401992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.210+0000 7f2e48157640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e401082e0 0x7f2e40199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.210+0000 7f2e48157640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40108c30 0x7f2e4019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.210+0000 7f2e48157640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401128f0 con 0x7f2e401082e0 2024-09-21T06:17:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.210+0000 7f2e48157640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40112770 con 0x7f2e401029b0 2024-09-21T06:17:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.210+0000 7f2e48157640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40112a70 con 0x7f2e40108c30 2024-09-21T06:17:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.210+0000 7f2e456f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e401082e0 0x7f2e40199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.210+0000 7f2e456f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e401082e0 0x7f2e40199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53918/0 (socket says 172.21.15.3:53918) 2024-09-21T06:17:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.210+0000 7f2e456f2640 1 -- 172.21.15.3:0/2076043739 learned_addr learned my addr 172.21.15.3:0/2076043739 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.210+0000 7f2e45ef3640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e401029b0 0x7f2e401992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.210+0000 7f2e466f4640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40108c30 0x7f2e4019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.211+0000 7f2e456f2640 1 -- 172.21.15.3:0/2076043739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40108c30 msgr2=0x7f2e4019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.211+0000 7f2e456f2640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40108c30 0x7f2e4019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.211+0000 7f2e456f2640 1 -- 172.21.15.3:0/2076043739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e401029b0 msgr2=0x7f2e401992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.211+0000 7f2e456f2640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e401029b0 0x7f2e401992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.211+0000 7f2e456f2640 1 -- 172.21.15.3:0/2076043739 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4019f200 con 0x7f2e401082e0 2024-09-21T06:17:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.211+0000 7f2e45ef3640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e401029b0 0x7f2e401992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.211+0000 7f2e466f4640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40108c30 0x7f2e4019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:17:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.211+0000 7f2e456f2640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e401082e0 0x7f2e40199870 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000e970 tx=0x7f2e3000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.211+0000 7f2e36ffd640 1 -- 172.21.15.3:0/2076043739 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000cd80 con 0x7f2e401082e0 2024-09-21T06:17:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.211+0000 7f2e48157640 1 -- 172.21.15.3:0/2076043739 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4019f4f0 con 0x7f2e401082e0 2024-09-21T06:17:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.211+0000 7f2e36ffd640 1 -- 172.21.15.3:0/2076043739 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e300035f0 con 0x7f2e401082e0 2024-09-21T06:17:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.211+0000 7f2e36ffd640 1 -- 172.21.15.3:0/2076043739 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30010690 con 0x7f2e401082e0 2024-09-21T06:17:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.211+0000 7f2e48157640 1 -- 172.21.15.3:0/2076043739 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e401a5c80 con 0x7f2e401082e0 2024-09-21T06:17:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.214+0000 7f2e36ffd640 1 -- 172.21.15.3:0/2076043739 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f2e30003aa0 con 0x7f2e401082e0 2024-09-21T06:17:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.214+0000 7f2e4f0c4740 1 Processor -- start 2024-09-21T06:17:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.214+0000 7f2e4f0c4740 1 -- start start 2024-09-21T06:17:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.214+0000 7f2e4f0c4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09eff31b0 0x55b09f3989b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.214+0000 7f2e4f0c4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09efca2f0 0x55b09f398ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.214+0000 7f2e4f0c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efd24d0 0x55b09f399430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.214+0000 7f2e4f0c4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b09f30d6f0 con 0x55b09efca2f0 2024-09-21T06:17:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.214+0000 7f2e4f0c4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b09f2fcb20 con 0x55b09eff31b0 2024-09-21T06:17:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.214+0000 7f2e4f0c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b09f300930 con 0x55b09efd24d0 2024-09-21T06:17:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e45ef3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09efca2f0 0x55b09f398ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e45ef3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09efca2f0 0x55b09f398ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53924/0 (socket says 172.21.15.3:53924) 2024-09-21T06:17:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e45ef3640 1 -- 172.21.15.3:0/2516229407 learned_addr learned my addr 172.21.15.3:0/2516229407 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e466f4640 1 --2- 172.21.15.3:0/2516229407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09eff31b0 0x55b09f3989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e456f2640 1 --2- 172.21.15.3:0/2516229407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efd24d0 0x55b09f399430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e466f4640 1 -- 172.21.15.3:0/2516229407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efd24d0 msgr2=0x55b09f399430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e36ffd640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e2c0b6680 0x7f2e2c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e466f4640 1 --2- 172.21.15.3:0/2516229407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efd24d0 0x55b09f399430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e466f4640 1 -- 172.21.15.3:0/2516229407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09efca2f0 msgr2=0x55b09f398ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e466f4640 1 --2- 172.21.15.3:0/2516229407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09efca2f0 0x55b09f398ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e466f4640 1 -- 172.21.15.3:0/2516229407 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b09f2c2530 con 0x55b09eff31b0 2024-09-21T06:17:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e36ffd640 1 -- 172.21.15.3:0/2076043739 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2e300e7a10 con 0x7f2e401082e0 2024-09-21T06:17:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e466f4640 1 --2- 172.21.15.3:0/2516229407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09eff31b0 0x55b09f3989b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009ca0 tx=0x7f2e38009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfb2b54b5c70dc26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e466f4640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e2c0b6680 0x7f2e2c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.215+0000 7f2e2bfff640 1 -- 172.21.15.3:0/2516229407 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38031d50 con 0x55b09eff31b0 2024-09-21T06:17:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.216+0000 7f2e2bfff640 1 -- 172.21.15.3:0/2516229407 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e38040070 con 0x55b09eff31b0 2024-09-21T06:17:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.216+0000 7f2e2bfff640 1 -- 172.21.15.3:0/2516229407 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e380046f0 con 0x55b09eff31b0 2024-09-21T06:17:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.217+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/2516229407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09eff31b0 msgr2=0x55b09f3989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.217+0000 7f2e466f4640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e2c0b6680 0x7f2e2c0b8b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2e38045000 tx=0x7f2e3802f660 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.217+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/2516229407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09eff31b0 0x55b09f3989b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009ca0 tx=0x7f2e38009440 comp rx=0 tx=0).stop 2024-09-21T06:17:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.217+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/2516229407 shutdown_connections 2024-09-21T06:17:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.217+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/2516229407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efd24d0 0x55b09f399430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.217+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/2516229407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09efca2f0 0x55b09f398ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.217+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/2516229407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09eff31b0 0x55b09f3989b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.217+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/2516229407 >> 172.21.15.3:0/2516229407 conn(0x55b09efe7300 msgr2=0x55b09f3968a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.217+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/2516229407 shutdown_connections 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.217+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/2516229407 wait complete. 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.217+0000 7f2e4f0c4740 1 -- start start 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.217+0000 7f2e4f0c4740 1 Processor -- start 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e4f0c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efca2f0 0x55b09f3a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e4f0c4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efd24d0 0x55b09f3a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e4f0c4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09eff31b0 0x55b09f3b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e4f0c4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b09f234ab0 con 0x55b09eff31b0 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e4f0c4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b09f2a2730 con 0x55b09efd24d0 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e4f0c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b09f3a70d0 con 0x55b09efca2f0 2024-09-21T06:17:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e4f0c4740 10 client.0 _ll_register_callbacks cb 0x55b09f2f66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e466f4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09eff31b0 0x55b09f3b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e456f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efd24d0 0x55b09f3a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e45ef3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efca2f0 0x55b09f3a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e466f4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09eff31b0 0x55b09f3b03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53930/0 (socket says 172.21.15.3:53930) 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e466f4640 1 -- 172.21.15.3:0/2607856651 learned_addr learned my addr 172.21.15.3:0/2607856651 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e456f2640 1 -- 172.21.15.3:0/2607856651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efca2f0 msgr2=0x55b09f3a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e456f2640 1 --2- 172.21.15.3:0/2607856651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efca2f0 0x55b09f3a40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e456f2640 1 -- 172.21.15.3:0/2607856651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09eff31b0 msgr2=0x55b09f3b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e456f2640 1 --2- 172.21.15.3:0/2607856651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09eff31b0 0x55b09f3b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e456f2640 1 -- 172.21.15.3:0/2607856651 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b09f05e790 con 0x55b09efd24d0 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e45ef3640 1 --2- 172.21.15.3:0/2607856651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efca2f0 0x55b09f3a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.218+0000 7f2e466f4640 1 --2- 172.21.15.3:0/2607856651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09eff31b0 0x55b09f3b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.219+0000 7f2e456f2640 1 --2- 172.21.15.3:0/2607856651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efd24d0 0x55b09f3a4600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e300e51a0 tx=0x7f2e300eb150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb8289a2802dc498 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.219+0000 7f2e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.219+0000 7f2e297fa640 1 -- 172.21.15.3:0/2607856651 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300f0070 con 0x55b09efd24d0 2024-09-21T06:17:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.219+0000 7f2e297fa640 1 -- 172.21.15.3:0/2607856651 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2e300a91f0 con 0x55b09efd24d0 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.219+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/2607856651 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e38040070 con 0x55b09efd24d0 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.219+0000 7f2e297fa640 1 -- 172.21.15.3:0/2607856651 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e300a9740 con 0x55b09efd24d0 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.219+0000 7f2e4f0c4740 10 client.25931 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.219+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/2607856651 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b09f0d7620 con 0x55b09efd24d0 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.219+0000 7f2e297fa640 1 -- 172.21.15.3:0/2607856651 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e38040070 con 0x55b09efd24d0 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.219+0000 7f2e297fa640 1 -- 172.21.15.3:0/2607856651 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2e300a93f0 con 0x55b09efd24d0 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.220+0000 7f2e4f0c4740 10 client.25931 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.220+0000 7f2e4f0c4740 10 client.25931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.220+0000 7f2e4f0c4740 20 client.25931 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.220+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/2607856651 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b09efb1d80 0x55b09f3b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.220+0000 7f2e4f0c4740 4 client.25931 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.220+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/2607856651 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b09f30f640 con 0x55b09efb1d80 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.220+0000 7f2e45ef3640 1 --2- 172.21.15.3:0/2607856651 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b09efb1d80 0x55b09f3b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.220+0000 7f2e45ef3640 1 --2- 172.21.15.3:0/2607856651 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b09efb1d80 0x55b09f3b61b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a43b2f9ec27c94f7 server_cookie=abc621170a371309 in_seq=0 out_seq=0 2024-09-21T06:17:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.220+0000 7f2e297fa640 10 client.25931 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e297fa640 1 -- 172.21.15.3:0/2607856651 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b09f30f640 con 0x55b09efb1d80 2024-09-21T06:17:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e297fa640 10 client.25931 handle_command_reply: tid=0 2024-09-21T06:17:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e4f0c4740 1 client.25931 shutdown 2024-09-21T06:17:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/2607856651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efd24d0 msgr2=0x55b09f3a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/2607856651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efd24d0 0x55b09f3a4600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e300e51a0 tx=0x7f2e300eb150 comp rx=0 tx=0).stop 2024-09-21T06:17:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/2607856651 shutdown_connections 2024-09-21T06:17:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/2607856651 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b09efb1d80 0x55b09f3b61b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/2607856651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09eff31b0 0x55b09f3b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/2607856651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efd24d0 0x55b09f3a4600 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/2607856651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efca2f0 0x55b09f3a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e297fa640 0 client.25931 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/2607856651 >> 172.21.15.3:0/2607856651 conn(0x55b09efe7300 msgr2=0x55b09f3999e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/2607856651 shutdown_connections 2024-09-21T06:17:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/2607856651 wait complete. 2024-09-21T06:17:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.221+0000 7f2e4f0c4740 20 client.25931 trim_cache size 0 max 16384 2024-09-21T06:17:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e4f0c4740 1 Processor -- start 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e4f0c4740 1 -- start start 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e4f0c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efca2f0 0x55b09f3a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e4f0c4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efd24d0 0x55b09f3968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e4f0c4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09eff31b0 0x55b09f3b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e4f0c4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401ae600 con 0x55b09eff31b0 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e4f0c4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401b01a0 con 0x55b09efd24d0 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e4f0c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401aecd0 con 0x55b09efca2f0 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e466f4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09eff31b0 0x55b09f3b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e45ef3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efca2f0 0x55b09f3a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e466f4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09eff31b0 0x55b09f3b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53942/0 (socket says 172.21.15.3:53942) 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e456f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efd24d0 0x55b09f3968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e466f4640 1 -- 172.21.15.3:0/1086851518 learned_addr learned my addr 172.21.15.3:0/1086851518 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e45ef3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efca2f0 0x55b09f3a45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:52030/0 (socket says 172.21.15.3:52030) 2024-09-21T06:17:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e45ef3640 1 -- 172.21.15.3:0/1086851518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efd24d0 msgr2=0x55b09f3968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e45ef3640 1 --2- 172.21.15.3:0/1086851518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efd24d0 0x55b09f3968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e45ef3640 1 -- 172.21.15.3:0/1086851518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09eff31b0 msgr2=0x55b09f3b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e45ef3640 1 --2- 172.21.15.3:0/1086851518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09eff31b0 0x55b09f3b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.238+0000 7f2e45ef3640 1 -- 172.21.15.3:0/1086851518 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b09f39b250 con 0x55b09efca2f0 2024-09-21T06:17:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.239+0000 7f2e45ef3640 1 --2- 172.21.15.3:0/1086851518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efca2f0 0x55b09f3a45a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55b09f3b6950 tx=0x7f2e3c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c9255d7e43c2076 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.239+0000 7f2e2b7fe640 1 -- 172.21.15.3:0/1086851518 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c012110 con 0x55b09efca2f0 2024-09-21T06:17:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.239+0000 7f2e2b7fe640 1 -- 172.21.15.3:0/1086851518 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e3c0054d0 con 0x55b09efca2f0 2024-09-21T06:17:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.239+0000 7f2e2b7fe640 1 -- 172.21.15.3:0/1086851518 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c004d80 con 0x55b09efca2f0 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.240+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/1086851518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efca2f0 msgr2=0x55b09f3a45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.240+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/1086851518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efca2f0 0x55b09f3a45a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55b09f3b6950 tx=0x7f2e3c00f790 comp rx=0 tx=0).stop 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.240+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/1086851518 shutdown_connections 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.240+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/1086851518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09eff31b0 0x55b09f3b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.240+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/1086851518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efd24d0 0x55b09f3968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.240+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/1086851518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09efca2f0 0x55b09f3a45a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.240+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/1086851518 >> 172.21.15.3:0/1086851518 conn(0x55b09efe7300 msgr2=0x55b09f3961c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.240+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/1086851518 shutdown_connections 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.240+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/1086851518 wait complete. 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.240+0000 7f2e4f0c4740 1 -- start start 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.240+0000 7f2e4f0c4740 1 Processor -- start 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e4f0c4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efca2f0 0x55b09f3bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e4f0c4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09efd24d0 0x55b09f3bb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e4f0c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09eff31b0 0x55b09f41ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e4f0c4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401afa70 con 0x55b09efd24d0 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e4f0c4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401af3a0 con 0x55b09efca2f0 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e4f0c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401ac5e0 con 0x55b09eff31b0 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e4f0c4740 10 client.0 _ll_register_callbacks cb 0x7f2e300056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e456f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09efd24d0 0x55b09f3bb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e466f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09eff31b0 0x55b09f41ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e45ef3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efca2f0 0x55b09f3bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e456f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09efd24d0 0x55b09f3bb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53948/0 (socket says 172.21.15.3:53948) 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e456f2640 1 -- 172.21.15.3:0/4134636840 learned_addr learned my addr 172.21.15.3:0/4134636840 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e456f2640 1 -- 172.21.15.3:0/4134636840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09eff31b0 msgr2=0x55b09f41ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e456f2640 1 --2- 172.21.15.3:0/4134636840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09eff31b0 0x55b09f41ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e456f2640 1 -- 172.21.15.3:0/4134636840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efca2f0 msgr2=0x55b09f3bb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e456f2640 1 --2- 172.21.15.3:0/4134636840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efca2f0 0x55b09f3bb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e456f2640 1 -- 172.21.15.3:0/4134636840 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b09f41b230 con 0x55b09efd24d0 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e45ef3640 1 --2- 172.21.15.3:0/4134636840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efca2f0 0x55b09f3bb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e456f2640 1 --2- 172.21.15.3:0/4134636840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09efd24d0 0x55b09f3bb920 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2e300a9e70 tx=0x7f2e300acf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc16a208739ade5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.241+0000 7f2e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e297fa640 1 -- 172.21.15.3:0/4134636840 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300f0040 con 0x55b09efd24d0 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e297fa640 1 -- 172.21.15.3:0/4134636840 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2e300aedd0 con 0x55b09efd24d0 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/4134636840 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e3c0054d0 con 0x55b09efd24d0 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e297fa640 1 -- 172.21.15.3:0/4134636840 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e300ad010 con 0x55b09efd24d0 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e4f0c4740 10 client.16356 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/4134636840 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b09f4225e0 con 0x55b09efd24d0 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e297fa640 1 -- 172.21.15.3:0/4134636840 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e3c0054d0 con 0x55b09efd24d0 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e297fa640 1 -- 172.21.15.3:0/4134636840 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2e300a91f0 con 0x55b09efd24d0 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e4f0c4740 10 client.16356 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e4f0c4740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e4f0c4740 20 client.16356 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/4134636840 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b09ef871c0 0x55b09f424e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e4f0c4740 4 client.16356 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:17:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/4134636840 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b09f30b0f0 con 0x55b09ef871c0 2024-09-21T06:17:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.242+0000 7f2e45ef3640 1 --2- 172.21.15.3:0/4134636840 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b09ef871c0 0x55b09f424e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.243+0000 7f2e45ef3640 1 --2- 172.21.15.3:0/4134636840 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b09ef871c0 0x55b09f424e00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73d4d240631d8734 server_cookie=54849f084bbb817 in_seq=0 out_seq=0 2024-09-21T06:17:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.243+0000 7f2e297fa640 10 client.16356 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e297fa640 1 -- 172.21.15.3:0/4134636840 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b09f30b0f0 con 0x55b09ef871c0 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e297fa640 10 client.16356 handle_command_reply: tid=0 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e4f0c4740 1 client.16356 shutdown 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/4134636840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09efd24d0 msgr2=0x55b09f3bb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/4134636840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09efd24d0 0x55b09f3bb920 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2e300a9e70 tx=0x7f2e300acf40 comp rx=0 tx=0).stop 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/4134636840 shutdown_connections 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/4134636840 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b09ef871c0 0x55b09f424e00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/4134636840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09eff31b0 0x55b09f41ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/4134636840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b09efd24d0 0x55b09f3bb920 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e4f0c4740 1 --2- 172.21.15.3:0/4134636840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b09efca2f0 0x55b09f3bb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/4134636840 >> 172.21.15.3:0/4134636840 conn(0x55b09efe7300 msgr2=0x55b09f394ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e297fa640 0 client.16356 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/4134636840 shutdown_connections 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.377+0000 7f2e4f0c4740 1 -- 172.21.15.3:0/4134636840 wait complete. 2024-09-21T06:17:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.378+0000 7f2e4f0c4740 20 client.16356 trim_cache size 0 max 16384 2024-09-21T06:17:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:17:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:17:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:17:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:17:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.378+0000 7f2e48157640 1 -- 172.21.15.3:0/2076043739 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e2c0b6680 msgr2=0x7f2e2c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.378+0000 7f2e48157640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e2c0b6680 0x7f2e2c0b8b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2e38045000 tx=0x7f2e3802f660 comp rx=0 tx=0).stop 2024-09-21T06:17:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.378+0000 7f2e48157640 1 -- 172.21.15.3:0/2076043739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e401082e0 msgr2=0x7f2e40199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.378+0000 7f2e48157640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e401082e0 0x7f2e40199870 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000e970 tx=0x7f2e3000ee30 comp rx=0 tx=0).stop 2024-09-21T06:17:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.378+0000 7f2e48157640 1 -- 172.21.15.3:0/2076043739 shutdown_connections 2024-09-21T06:17:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.378+0000 7f2e48157640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e2c0b6680 0x7f2e2c0b8b90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.378+0000 7f2e48157640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40108c30 0x7f2e4019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.378+0000 7f2e48157640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e401082e0 0x7f2e40199870 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.378+0000 7f2e48157640 1 --2- 172.21.15.3:0/2076043739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e401029b0 0x7f2e401992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.378+0000 7f2e48157640 1 -- 172.21.15.3:0/2076043739 >> 172.21.15.3:0/2076043739 conn(0x7f2e4007a740 msgr2=0x7f2e40106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.379+0000 7f2e48157640 1 -- 172.21.15.3:0/2076043739 shutdown_connections 2024-09-21T06:17:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:44.379+0000 7f2e48157640 1 -- 172.21.15.3:0/2076043739 wait complete. 2024-09-21T06:17:44.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 195512 13.93 MB/sec warmup 77 sec latency 81.421 ms 2024-09-21T06:17:45.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:45 smithi137 ceph-mon[34464]: pgmap v407: 97 pgs: 97 active+clean; 59 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 336 op/s 2024-09-21T06:17:45.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:45 smithi204 ceph-mon[34659]: pgmap v407: 97 pgs: 97 active+clean; 59 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 336 op/s 2024-09-21T06:17:45.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:45 smithi003 ceph-mon[31227]: pgmap v407: 97 pgs: 97 active+clean; 59 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 336 op/s 2024-09-21T06:17:45.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 198646 13.98 MB/sec warmup 78 sec latency 26.472 ms 2024-09-21T06:17:46.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 200680 13.89 MB/sec warmup 79 sec latency 16.441 ms 2024-09-21T06:17:47.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:47 smithi204 ceph-mon[34659]: pgmap v408: 97 pgs: 97 active+clean; 53 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.6 MiB/s wr, 437 op/s 2024-09-21T06:17:47.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:47 smithi003 ceph-mon[31227]: pgmap v408: 97 pgs: 97 active+clean; 53 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.6 MiB/s wr, 437 op/s 2024-09-21T06:17:47.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:47 smithi137 ceph-mon[34464]: pgmap v408: 97 pgs: 97 active+clean; 53 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.6 MiB/s wr, 437 op/s 2024-09-21T06:17:47.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 202766 13.89 MB/sec warmup 80 sec latency 13.984 ms 2024-09-21T06:17:48.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 205949 13.95 MB/sec warmup 81 sec latency 15.417 ms 2024-09-21T06:17:49.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:49 smithi204 ceph-mon[34659]: pgmap v409: 97 pgs: 97 active+clean; 54 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-09-21T06:17:49.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:49 smithi003 ceph-mon[31227]: pgmap v409: 97 pgs: 97 active+clean; 54 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-09-21T06:17:49.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:49 smithi137 ceph-mon[34464]: pgmap v409: 97 pgs: 97 active+clean; 54 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-09-21T06:17:49.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 208177 13.87 MB/sec warmup 82 sec latency 12.652 ms 2024-09-21T06:17:50.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 209783 13.85 MB/sec warmup 83 sec latency 64.003 ms 2024-09-21T06:17:51.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:51 smithi204 ceph-mon[34659]: pgmap v410: 97 pgs: 97 active+clean; 55 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.0 MiB/s wr, 455 op/s 2024-09-21T06:17:51.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:51 smithi003 ceph-mon[31227]: pgmap v410: 97 pgs: 97 active+clean; 55 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.0 MiB/s wr, 455 op/s 2024-09-21T06:17:51.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:51 smithi137 ceph-mon[34464]: pgmap v410: 97 pgs: 97 active+clean; 55 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.0 MiB/s wr, 455 op/s 2024-09-21T06:17:51.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 212893 13.89 MB/sec warmup 84 sec latency 30.130 ms 2024-09-21T06:17:52.948 INFO:tasks.workunit.client.0.smithi003.stdout: 1 214853 13.81 MB/sec warmup 85 sec latency 19.956 ms 2024-09-21T06:17:53.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:53 smithi204 ceph-mon[34659]: pgmap v411: 97 pgs: 97 active+clean; 55 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 381 op/s 2024-09-21T06:17:53.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:53 smithi003 ceph-mon[31227]: pgmap v411: 97 pgs: 97 active+clean; 55 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 381 op/s 2024-09-21T06:17:53.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:53 smithi137 ceph-mon[34464]: pgmap v411: 97 pgs: 97 active+clean; 55 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 381 op/s 2024-09-21T06:17:53.906 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:17:53.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 217042 13.81 MB/sec warmup 86 sec latency 15.651 ms 2024-09-21T06:17:54.176 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:54.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.719+0000 7f117a6ee640 1 -- 172.21.15.3:0/1760082651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1174102a60 msgr2=0x7f1174102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:54.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.719+0000 7f117a6ee640 1 --2- 172.21.15.3:0/1760082651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1174102a60 0x7f1174102e70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1164009900 tx=0x7f116402a410 comp rx=0 tx=0).stop 2024-09-21T06:17:54.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.719+0000 7f117a6ee640 1 -- 172.21.15.3:0/1760082651 shutdown_connections 2024-09-21T06:17:54.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.719+0000 7f117a6ee640 1 --2- 172.21.15.3:0/1760082651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1174108cf0 0x7f117410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.719+0000 7f117a6ee640 1 --2- 172.21.15.3:0/1760082651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11741083a0 0x7f11741087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.719+0000 7f117a6ee640 1 --2- 172.21.15.3:0/1760082651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1174102a60 0x7f1174102e70 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.719+0000 7f117a6ee640 1 -- 172.21.15.3:0/1760082651 >> 172.21.15.3:0/1760082651 conn(0x7f1174079730 msgr2=0x7f1174079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.719+0000 7f117a6ee640 1 -- 172.21.15.3:0/1760082651 shutdown_connections 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.719+0000 7f117a6ee640 1 -- 172.21.15.3:0/1760082651 wait complete. 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.720+0000 7f117a6ee640 1 Processor -- start 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.720+0000 7f117a6ee640 1 -- start start 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f117a6ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1174102a60 0x7f117419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f117a6ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11741083a0 0x7f117419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f117a6ee640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1174108cf0 0x7f117419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f117a6ee640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1174112960 con 0x7f11741083a0 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f117a6ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f11741127e0 con 0x7f1174102a60 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f117a6ee640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1174112ae0 con 0x7f1174108cf0 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f11737fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11741083a0 0x7f117419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f11737fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11741083a0 0x7f117419b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54184/0 (socket says 172.21.15.3:54184) 2024-09-21T06:17:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f11737fe640 1 -- 172.21.15.3:0/2711594985 learned_addr learned my addr 172.21.15.3:0/2711594985 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:54.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f1173fff640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1174102a60 0x7f117419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f1178c8b640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1174108cf0 0x7f117419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f11737fe640 1 -- 172.21.15.3:0/2711594985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1174108cf0 msgr2=0x7f117419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:54.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f11737fe640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1174108cf0 0x7f117419bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f11737fe640 1 -- 172.21.15.3:0/2711594985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1174102a60 msgr2=0x7f117419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:54.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f11737fe640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1174102a60 0x7f117419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f11737fe640 1 -- 172.21.15.3:0/2711594985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11741a6520 con 0x7f11741083a0 2024-09-21T06:17:54.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f11737fe640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11741083a0 0x7f117419b7d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f116800e9c0 tx=0x7f116800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:54.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.721+0000 7f11717fa640 1 -- 172.21.15.3:0/2711594985 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116800cd80 con 0x7f11741083a0 2024-09-21T06:17:54.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.722+0000 7f11717fa640 1 -- 172.21.15.3:0/2711594985 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11680054d0 con 0x7f11741083a0 2024-09-21T06:17:54.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.722+0000 7f117a6ee640 1 -- 172.21.15.3:0/2711594985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11741a6810 con 0x7f11741083a0 2024-09-21T06:17:54.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.722+0000 7f11717fa640 1 -- 172.21.15.3:0/2711594985 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168010430 con 0x7f11741083a0 2024-09-21T06:17:54.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.722+0000 7f117a6ee640 1 -- 172.21.15.3:0/2711594985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11741a6d50 con 0x7f11741083a0 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.724+0000 7f1180c62740 1 Processor -- start 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1180c62740 1 -- start start 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1180c62740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43937a070 0x55b439721650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1180c62740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4393511b0 0x55b439721b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1180c62740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b439359390 0x55b4397220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1180c62740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b439697e50 con 0x55b439359390 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1180c62740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b439687280 con 0x55b4393511b0 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1180c62740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b43968b090 con 0x55b43937a070 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f11717fa640 1 -- 172.21.15.3:0/2711594985 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f1168004f90 con 0x7f11741083a0 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1173fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4393511b0 0x55b439721b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1178c8b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43937a070 0x55b439721650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f11737fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b439359390 0x55b4397220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1173fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4393511b0 0x55b439721b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:39420/0 (socket says 172.21.15.3:39420) 2024-09-21T06:17:54.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1178c8b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43937a070 0x55b439721650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:57852/0 (socket says 172.21.15.3:57852) 2024-09-21T06:17:54.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1173fff640 1 -- 172.21.15.3:0/2987402856 learned_addr learned my addr 172.21.15.3:0/2987402856 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:54.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1173fff640 1 -- 172.21.15.3:0/2987402856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43937a070 msgr2=0x55b439721650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:54.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1173fff640 1 --2- 172.21.15.3:0/2987402856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43937a070 0x55b439721650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1173fff640 1 -- 172.21.15.3:0/2987402856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b439359390 msgr2=0x55b4397220d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:54.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1173fff640 1 --2- 172.21.15.3:0/2987402856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b439359390 0x55b4397220d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1173fff640 1 -- 172.21.15.3:0/2987402856 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4393e5650 con 0x55b4393511b0 2024-09-21T06:17:54.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.725+0000 7f1178c8b640 1 --2- 172.21.15.3:0/2987402856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43937a070 0x55b439721650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:54.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.726+0000 7f1173fff640 1 --2- 172.21.15.3:0/2987402856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4393511b0 0x55b439721b90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1164009900 tx=0x7f1164032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5513b1a9ab9c755c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:54.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.726+0000 7f11717fa640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f115c0b66b0 0x7f115c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.726+0000 7f115a7fc640 1 -- 172.21.15.3:0/2987402856 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164040070 con 0x55b4393511b0 2024-09-21T06:17:54.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.726+0000 7f1178c8b640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f115c0b66b0 0x7f115c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.726+0000 7f11717fa640 1 -- 172.21.15.3:0/2711594985 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f11680e7ae0 con 0x7f11741083a0 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.726+0000 7f115a7fc640 1 -- 172.21.15.3:0/2987402856 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1164004030 con 0x55b4393511b0 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.726+0000 7f115a7fc640 1 -- 172.21.15.3:0/2987402856 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164035040 con 0x55b4393511b0 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.727+0000 7f1178c8b640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f115c0b66b0 0x7f115c0b8bc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f116c009fd0 tx=0x7f116c009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.727+0000 7f1180c62740 1 -- 172.21.15.3:0/2987402856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4393511b0 msgr2=0x55b439721b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.727+0000 7f1180c62740 1 --2- 172.21.15.3:0/2987402856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4393511b0 0x55b439721b90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1164009900 tx=0x7f1164032b20 comp rx=0 tx=0).stop 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.727+0000 7f1180c62740 1 -- 172.21.15.3:0/2987402856 shutdown_connections 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.727+0000 7f1180c62740 1 --2- 172.21.15.3:0/2987402856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b439359390 0x55b4397220d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.727+0000 7f1180c62740 1 --2- 172.21.15.3:0/2987402856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4393511b0 0x55b439721b90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.727+0000 7f1180c62740 1 --2- 172.21.15.3:0/2987402856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43937a070 0x55b439721650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.727+0000 7f1180c62740 1 -- 172.21.15.3:0/2987402856 >> 172.21.15.3:0/2987402856 conn(0x55b43936e460 msgr2=0x55b43971f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.728+0000 7f1180c62740 1 -- 172.21.15.3:0/2987402856 shutdown_connections 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.728+0000 7f1180c62740 1 -- 172.21.15.3:0/2987402856 wait complete. 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.728+0000 7f1180c62740 1 -- start start 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.728+0000 7f1180c62740 1 Processor -- start 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.728+0000 7f1180c62740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4393511b0 0x55b43972cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1180c62740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b439359390 0x55b43972d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1180c62740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 0x55b4397390c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1180c62740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b4394e8700 con 0x55b43937a070 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1180c62740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b439622110 con 0x55b4393511b0 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1180c62740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b43972fdd0 con 0x55b439359390 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1180c62740 10 client.0 _ll_register_callbacks cb 0x55b43963b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1178c8b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 0x55b4397390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f11737fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b439359390 0x55b43972d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1178c8b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 0x55b4397390c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54204/0 (socket says 172.21.15.3:54204) 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1178c8b640 1 -- 172.21.15.3:0/4199979682 learned_addr learned my addr 172.21.15.3:0/4199979682 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1173fff640 1 --2- 172.21.15.3:0/4199979682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4393511b0 0x55b43972cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1178c8b640 1 -- 172.21.15.3:0/4199979682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b439359390 msgr2=0x55b43972d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1178c8b640 1 --2- 172.21.15.3:0/4199979682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b439359390 0x55b43972d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1178c8b640 1 -- 172.21.15.3:0/4199979682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4393511b0 msgr2=0x55b43972cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1178c8b640 1 --2- 172.21.15.3:0/4199979682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4393511b0 0x55b43972cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1178c8b640 1 -- 172.21.15.3:0/4199979682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b43964fc40 con 0x55b43937a070 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f1178c8b640 1 --2- 172.21.15.3:0/4199979682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 0x55b4397390c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f116c011e30 tx=0x7f116c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6d35bc9811b1ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.729+0000 7f11537fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.730+0000 7f11537fe640 1 -- 172.21.15.3:0/4199979682 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c01f9b0 con 0x55b43937a070 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.730+0000 7f1180c62740 1 -- 172.21.15.3:0/4199979682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1164004030 con 0x55b43937a070 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.730+0000 7f11537fe640 1 -- 172.21.15.3:0/4199979682 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f116c01fb50 con 0x55b43937a070 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.730+0000 7f11537fe640 1 -- 172.21.15.3:0/4199979682 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f116c01fd50 con 0x55b43937a070 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.730+0000 7f1180c62740 10 client.16368 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.730+0000 7f1180c62740 1 -- 172.21.15.3:0/4199979682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b43973bb40 con 0x55b43937a070 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.730+0000 7f11537fe640 1 -- 172.21.15.3:0/4199979682 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1164004030 con 0x55b43937a070 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.730+0000 7f11537fe640 1 -- 172.21.15.3:0/4199979682 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f116c004050 con 0x55b43937a070 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.730+0000 7f1180c62740 10 client.16368 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.730+0000 7f1180c62740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.731+0000 7f1180c62740 20 client.16368 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:54.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.731+0000 7f1180c62740 1 --2- 172.21.15.3:0/4199979682 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b4393389f0 0x55b43973f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.731+0000 7f1180c62740 4 client.16368 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.731+0000 7f1180c62740 1 -- 172.21.15.3:0/4199979682 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b439699da0 con 0x55b4393389f0 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.731+0000 7f1173fff640 1 --2- 172.21.15.3:0/4199979682 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b4393389f0 0x55b43973f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.731+0000 7f1173fff640 1 --2- 172.21.15.3:0/4199979682 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b4393389f0 0x55b43973f0e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b54b62effb776be server_cookie=898247e46a750e25 in_seq=0 out_seq=0 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.731+0000 7f11537fe640 10 client.16368 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.732+0000 7f11537fe640 1 -- 172.21.15.3:0/4199979682 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b439699da0 con 0x55b4393389f0 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.732+0000 7f11537fe640 10 client.16368 handle_command_reply: tid=0 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.732+0000 7f1180c62740 1 client.16368 shutdown 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.732+0000 7f1180c62740 1 -- 172.21.15.3:0/4199979682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 msgr2=0x55b4397390c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.732+0000 7f1180c62740 1 --2- 172.21.15.3:0/4199979682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 0x55b4397390c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f116c011e30 tx=0x7f116c004ab0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.732+0000 7f1180c62740 1 -- 172.21.15.3:0/4199979682 shutdown_connections 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.732+0000 7f1180c62740 1 --2- 172.21.15.3:0/4199979682 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b4393389f0 0x55b43973f0e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.732+0000 7f1180c62740 1 --2- 172.21.15.3:0/4199979682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b439359390 0x55b43972d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.732+0000 7f1180c62740 1 --2- 172.21.15.3:0/4199979682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 0x55b4397390c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.732+0000 7f11537fe640 0 client.16368 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:54.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.732+0000 7f1180c62740 1 --2- 172.21.15.3:0/4199979682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4393511b0 0x55b43972cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.732+0000 7f1180c62740 1 -- 172.21.15.3:0/4199979682 >> 172.21.15.3:0/4199979682 conn(0x55b43936e460 msgr2=0x55b4397247e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:54.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.733+0000 7f1180c62740 1 -- 172.21.15.3:0/4199979682 shutdown_connections 2024-09-21T06:17:54.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.733+0000 7f1180c62740 1 -- 172.21.15.3:0/4199979682 wait complete. 2024-09-21T06:17:54.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.733+0000 7f1180c62740 20 client.16368 trim_cache size 0 max 16384 2024-09-21T06:17:54.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.751+0000 7f1180c62740 1 Processor -- start 2024-09-21T06:17:54.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.751+0000 7f1180c62740 1 -- start start 2024-09-21T06:17:54.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1180c62740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4393511b0 0x55b43972d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1180c62740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b439359390 0x55b439738fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1180c62740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 0x55b439729f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1180c62740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f11741ad4f0 con 0x55b43937a070 2024-09-21T06:17:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1180c62740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f11741af090 con 0x55b439359390 2024-09-21T06:17:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1180c62740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f11741adbc0 con 0x55b4393511b0 2024-09-21T06:17:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1178c8b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 0x55b439729f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1178c8b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 0x55b439729f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54216/0 (socket says 172.21.15.3:54216) 2024-09-21T06:17:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1178c8b640 1 -- 172.21.15.3:0/3261049809 learned_addr learned my addr 172.21.15.3:0/3261049809 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f11737fe640 1 --2- 172.21.15.3:0/3261049809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b439359390 0x55b439738fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1173fff640 1 --2- 172.21.15.3:0/3261049809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4393511b0 0x55b43972d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1178c8b640 1 -- 172.21.15.3:0/3261049809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4393511b0 msgr2=0x55b43972d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1178c8b640 1 --2- 172.21.15.3:0/3261049809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4393511b0 0x55b43972d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1178c8b640 1 -- 172.21.15.3:0/3261049809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b439359390 msgr2=0x55b439738fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1178c8b640 1 --2- 172.21.15.3:0/3261049809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b439359390 0x55b439738fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1178c8b640 1 -- 172.21.15.3:0/3261049809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b43972bc70 con 0x55b43937a070 2024-09-21T06:17:54.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.752+0000 7f1178c8b640 1 --2- 172.21.15.3:0/3261049809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 0x55b439729f00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f116c009690 tx=0x7f116c011fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e8c8f76023214e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:54.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.753+0000 7f1159ffb640 1 -- 172.21.15.3:0/3261049809 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c004660 con 0x55b43937a070 2024-09-21T06:17:54.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.753+0000 7f1159ffb640 1 -- 172.21.15.3:0/3261049809 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f116c02c070 con 0x55b43937a070 2024-09-21T06:17:54.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.753+0000 7f1159ffb640 1 -- 172.21.15.3:0/3261049809 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c003970 con 0x55b43937a070 2024-09-21T06:17:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.753+0000 7f1180c62740 1 -- 172.21.15.3:0/3261049809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 msgr2=0x55b439729f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.753+0000 7f1180c62740 1 --2- 172.21.15.3:0/3261049809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 0x55b439729f00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f116c009690 tx=0x7f116c011fa0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.754+0000 7f1180c62740 1 -- 172.21.15.3:0/3261049809 shutdown_connections 2024-09-21T06:17:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.754+0000 7f1180c62740 1 --2- 172.21.15.3:0/3261049809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b43937a070 0x55b439729f00 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.754+0000 7f1180c62740 1 --2- 172.21.15.3:0/3261049809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b439359390 0x55b439738fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.754+0000 7f1180c62740 1 --2- 172.21.15.3:0/3261049809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4393511b0 0x55b43972d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.754+0000 7f1180c62740 1 -- 172.21.15.3:0/3261049809 >> 172.21.15.3:0/3261049809 conn(0x55b43936e460 msgr2=0x55b43972cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.754+0000 7f1180c62740 1 -- 172.21.15.3:0/3261049809 shutdown_connections 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.754+0000 7f1180c62740 1 -- 172.21.15.3:0/3261049809 wait complete. 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.754+0000 7f1180c62740 1 -- start start 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.754+0000 7f1180c62740 1 Processor -- start 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1180c62740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4393511b0 0x55b439731290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1180c62740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b439359390 0x55b4397317d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1180c62740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b43937a070 0x55b4397a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1180c62740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f11741ae960 con 0x55b4393511b0 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1180c62740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f11741ae290 con 0x55b43937a070 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1180c62740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f11741ab4d0 con 0x55b439359390 2024-09-21T06:17:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1180c62740 10 client.0 _ll_register_callbacks cb 0x7f116c009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1173fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4393511b0 0x55b439731290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1173fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4393511b0 0x55b439731290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54230/0 (socket says 172.21.15.3:54230) 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f11737fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b439359390 0x55b4397317d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1173fff640 1 -- 172.21.15.3:0/2913449406 learned_addr learned my addr 172.21.15.3:0/2913449406 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1178c8b640 1 --2- 172.21.15.3:0/2913449406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b43937a070 0x55b4397a8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1173fff640 1 -- 172.21.15.3:0/2913449406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b439359390 msgr2=0x55b4397317d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1173fff640 1 --2- 172.21.15.3:0/2913449406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b439359390 0x55b4397317d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1173fff640 1 -- 172.21.15.3:0/2913449406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b43937a070 msgr2=0x55b4397a8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1173fff640 1 --2- 172.21.15.3:0/2913449406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b43937a070 0x55b4397a8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1173fff640 1 -- 172.21.15.3:0/2913449406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b439731ff0 con 0x55b4393511b0 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1178c8b640 1 --2- 172.21.15.3:0/2913449406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b43937a070 0x55b4397a8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f1173fff640 1 --2- 172.21.15.3:0/2913449406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4393511b0 0x55b439731290 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1164009900 tx=0x7f1164005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7c21ae212021f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f11537fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.755+0000 7f11537fe640 1 -- 172.21.15.3:0/2913449406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164040070 con 0x55b4393511b0 2024-09-21T06:17:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f11537fe640 1 -- 172.21.15.3:0/2913449406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f11640314a0 con 0x55b4393511b0 2024-09-21T06:17:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f1180c62740 1 -- 172.21.15.3:0/2913449406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f116c02c070 con 0x55b4393511b0 2024-09-21T06:17:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f11537fe640 1 -- 172.21.15.3:0/2913449406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1164035040 con 0x55b4393511b0 2024-09-21T06:17:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f1180c62740 10 client.16374 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f1180c62740 1 -- 172.21.15.3:0/2913449406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4397aa030 con 0x55b4393511b0 2024-09-21T06:17:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f11537fe640 1 -- 172.21.15.3:0/2913449406 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f116c02c070 con 0x55b4393511b0 2024-09-21T06:17:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f11537fe640 1 -- 172.21.15.3:0/2913449406 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f11640047c0 con 0x55b4393511b0 2024-09-21T06:17:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f1180c62740 10 client.16374 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f1180c62740 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f1180c62740 20 client.16374 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f1180c62740 1 --2- 172.21.15.3:0/2913449406 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b43930dda0 0x55b4397ac510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f1180c62740 4 client.16374 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:17:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f1180c62740 1 -- 172.21.15.3:0/2913449406 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b439695850 con 0x55b43930dda0 2024-09-21T06:17:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.756+0000 7f11737fe640 1 --2- 172.21.15.3:0/2913449406 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b43930dda0 0x55b4397ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.757+0000 7f11537fe640 10 client.16374 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.757+0000 7f11737fe640 1 --2- 172.21.15.3:0/2913449406 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b43930dda0 0x55b4397ac510 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd7864cea069985 server_cookie=a67ddb8487b8f9c5 in_seq=0 out_seq=0 2024-09-21T06:17:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.757+0000 7f11537fe640 1 -- 172.21.15.3:0/2913449406 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x55b439695850 con 0x55b43930dda0 2024-09-21T06:17:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.757+0000 7f11537fe640 10 client.16374 handle_command_reply: tid=0 2024-09-21T06:17:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.758+0000 7f1180c62740 1 client.16374 shutdown 2024-09-21T06:17:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.758+0000 7f1180c62740 1 -- 172.21.15.3:0/2913449406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4393511b0 msgr2=0x55b439731290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.758+0000 7f1180c62740 1 --2- 172.21.15.3:0/2913449406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4393511b0 0x55b439731290 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1164009900 tx=0x7f1164005890 comp rx=0 tx=0).stop 2024-09-21T06:17:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.758+0000 7f1180c62740 1 -- 172.21.15.3:0/2913449406 shutdown_connections 2024-09-21T06:17:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.758+0000 7f1180c62740 1 --2- 172.21.15.3:0/2913449406 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b43930dda0 0x55b4397ac510 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.758+0000 7f1180c62740 1 --2- 172.21.15.3:0/2913449406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b439359390 0x55b4397317d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.758+0000 7f1180c62740 1 --2- 172.21.15.3:0/2913449406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b43937a070 0x55b4397a8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.758+0000 7f1180c62740 1 --2- 172.21.15.3:0/2913449406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4393511b0 0x55b439731290 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.758+0000 7f1180c62740 1 -- 172.21.15.3:0/2913449406 >> 172.21.15.3:0/2913449406 conn(0x55b43936e460 msgr2=0x55b439722630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.758+0000 7f11537fe640 0 client.16374 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.758+0000 7f1180c62740 1 -- 172.21.15.3:0/2913449406 shutdown_connections 2024-09-21T06:17:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.758+0000 7f1180c62740 1 -- 172.21.15.3:0/2913449406 wait complete. 2024-09-21T06:17:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.758+0000 7f1180c62740 20 client.16374 trim_cache size 0 max 16384 2024-09-21T06:17:54.764 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:17:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:17:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:17:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:17:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:17:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:17:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:17:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:17:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:17:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:17:54.767 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:17:54.767 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:17:54.767 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:17:54.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.759+0000 7f117a6ee640 1 -- 172.21.15.3:0/2711594985 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f115c0b66b0 msgr2=0x7f115c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:54.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.759+0000 7f117a6ee640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f115c0b66b0 0x7f115c0b8bc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f116c009fd0 tx=0x7f116c009500 comp rx=0 tx=0).stop 2024-09-21T06:17:54.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.759+0000 7f117a6ee640 1 -- 172.21.15.3:0/2711594985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11741083a0 msgr2=0x7f117419b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:54.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.759+0000 7f117a6ee640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11741083a0 0x7f117419b7d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f116800e9c0 tx=0x7f116800ee80 comp rx=0 tx=0).stop 2024-09-21T06:17:54.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.759+0000 7f117a6ee640 1 -- 172.21.15.3:0/2711594985 shutdown_connections 2024-09-21T06:17:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.759+0000 7f117a6ee640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f115c0b66b0 0x7f115c0b8bc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.759+0000 7f117a6ee640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1174108cf0 0x7f117419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.759+0000 7f117a6ee640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11741083a0 0x7f117419b7d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.759+0000 7f117a6ee640 1 --2- 172.21.15.3:0/2711594985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1174102a60 0x7f117419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.759+0000 7f117a6ee640 1 -- 172.21.15.3:0/2711594985 >> 172.21.15.3:0/2711594985 conn(0x7f1174079730 msgr2=0x7f1174101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.759+0000 7f117a6ee640 1 -- 172.21.15.3:0/2711594985 shutdown_connections 2024-09-21T06:17:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:54.759+0000 7f117a6ee640 1 -- 172.21.15.3:0/2711594985 wait complete. 2024-09-21T06:17:54.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 220180 13.87 MB/sec warmup 87 sec latency 21.902 ms 2024-09-21T06:17:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:55 smithi003 ceph-mon[31227]: pgmap v412: 97 pgs: 97 active+clean; 56 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 371 op/s 2024-09-21T06:17:55.693 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:17:55.693 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:17:55.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:55 smithi137 ceph-mon[34464]: pgmap v412: 97 pgs: 97 active+clean; 56 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 371 op/s 2024-09-21T06:17:55.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:55 smithi204 ceph-mon[34659]: pgmap v412: 97 pgs: 97 active+clean; 56 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 371 op/s 2024-09-21T06:17:55.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 221555 13.77 MB/sec warmup 88 sec latency 47.555 ms 2024-09-21T06:17:56.046 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.551+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/2214018030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab410de10 msgr2=0x7f5ab4110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.551+0000 7f5ab8cf2640 1 --2- 172.21.15.3:0/2214018030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab410de10 0x7f5ab4110320 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009900 tx=0x7f5aac02a410 comp rx=0 tx=0).stop 2024-09-21T06:17:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.551+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/2214018030 shutdown_connections 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.551+0000 7f5ab8cf2640 1 --2- 172.21.15.3:0/2214018030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab410de10 0x7f5ab4110320 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.551+0000 7f5ab8cf2640 1 --2- 172.21.15.3:0/2214018030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ab4105290 0x7f5ab410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.551+0000 7f5ab8cf2640 1 --2- 172.21.15.3:0/2214018030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ab4104860 0x7f5ab4104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.551+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/2214018030 >> 172.21.15.3:0/2214018030 conn(0x7f5ab4079740 msgr2=0x7f5ab4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.551+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/2214018030 shutdown_connections 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.551+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/2214018030 wait complete. 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.552+0000 7f5ab8cf2640 1 Processor -- start 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.552+0000 7f5ab8cf2640 1 -- start start 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.552+0000 7f5ab8cf2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ab4104860 0x7f5ab419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.552+0000 7f5ab8cf2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab4105290 0x7f5ab419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.552+0000 7f5ab8cf2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ab410de10 0x7f5ab419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.552+0000 7f5ab8cf2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4113a90 con 0x7f5ab4105290 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.552+0000 7f5ab8cf2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4113910 con 0x7f5ab4104860 2024-09-21T06:17:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.552+0000 7f5ab8cf2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4113c10 con 0x7f5ab410de10 2024-09-21T06:17:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.552+0000 7f5ab1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab4105290 0x7f5ab419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.552+0000 7f5ab259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ab4104860 0x7f5ab419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.552+0000 7f5ab1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab4105290 0x7f5ab419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34546/0 (socket says 172.21.15.3:34546) 2024-09-21T06:17:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.552+0000 7f5ab1d9b640 1 -- 172.21.15.3:0/1319683437 learned_addr learned my addr 172.21.15.3:0/1319683437 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.552+0000 7f5ab2d9d640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ab410de10 0x7f5ab419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.553+0000 7f5ab259c640 1 -- 172.21.15.3:0/1319683437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ab410de10 msgr2=0x7f5ab419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.553+0000 7f5ab259c640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ab410de10 0x7f5ab419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.553+0000 7f5ab259c640 1 -- 172.21.15.3:0/1319683437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab4105290 msgr2=0x7f5ab419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.553+0000 7f5ab259c640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab4105290 0x7f5ab419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.553+0000 7f5ab259c640 1 -- 172.21.15.3:0/1319683437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab41a0430 con 0x7f5ab4104860 2024-09-21T06:17:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.553+0000 7f5ab1d9b640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab4105290 0x7f5ab419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.553+0000 7f5ab259c640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ab4104860 0x7f5ab419a4c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005fd0 tx=0x7f5aa400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.553+0000 7f5aa37fe640 1 -- 172.21.15.3:0/1319683437 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4003970 con 0x7f5ab4104860 2024-09-21T06:17:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.553+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/1319683437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab41a0720 con 0x7f5ab4104860 2024-09-21T06:17:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.553+0000 7f5aa37fe640 1 -- 172.21.15.3:0/1319683437 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa4004030 con 0x7f5ab4104860 2024-09-21T06:17:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.553+0000 7f5aa37fe640 1 -- 172.21.15.3:0/1319683437 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4010720 con 0x7f5ab4104860 2024-09-21T06:17:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.553+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/1319683437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab41a6e80 con 0x7f5ab4104860 2024-09-21T06:17:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.554+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/1319683437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab406c1e0 con 0x7f5ab4104860 2024-09-21T06:17:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.556+0000 7f5aa37fe640 1 -- 172.21.15.3:0/1319683437 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f5aa4004b60 con 0x7f5ab4104860 2024-09-21T06:17:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.557+0000 7f5aa37fe640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5a9c0b6680 0x7f5a9c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.557+0000 7f5ab1d9b640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5a9c0b6680 0x7f5a9c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.557+0000 7f5aa37fe640 1 -- 172.21.15.3:0/1319683437 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5aa40e7fb0 con 0x7f5ab4104860 2024-09-21T06:17:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.557+0000 7f5ab1d9b640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5a9c0b6680 0x7f5a9c0b8b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5ab410fa40 tx=0x7f5aa8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.561+0000 7f5aa37fe640 1 -- 172.21.15.3:0/1319683437 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5aa40ae4e0 con 0x7f5ab4104860 2024-09-21T06:17:56.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.814+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/1319683437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ab4104c70 con 0x7f5ab4104860 2024-09-21T06:17:56.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.815+0000 7f5aa37fe640 1 -- 172.21.15.3:0/1319683437 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f5aa40b28c0 con 0x7f5ab4104860 2024-09-21T06:17:56.820 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:17:56.820 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-09-21T06:16:31:270114+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:16:30.910049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:17:56.820 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-09-21T06:17:56.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.818+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/1319683437 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5a9c0b6680 msgr2=0x7f5a9c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:56.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.818+0000 7f5ab8cf2640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5a9c0b6680 0x7f5a9c0b8b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5ab410fa40 tx=0x7f5aa8005f90 comp rx=0 tx=0).stop 2024-09-21T06:17:56.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.818+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/1319683437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ab4104860 msgr2=0x7f5ab419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:56.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.818+0000 7f5ab8cf2640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ab4104860 0x7f5ab419a4c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005fd0 tx=0x7f5aa400ef90 comp rx=0 tx=0).stop 2024-09-21T06:17:56.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.818+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/1319683437 shutdown_connections 2024-09-21T06:17:56.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.818+0000 7f5ab8cf2640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5a9c0b6680 0x7f5a9c0b8b90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:56.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.818+0000 7f5ab8cf2640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ab410de10 0x7f5ab419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:56.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.818+0000 7f5ab8cf2640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab4105290 0x7f5ab419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:56.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.818+0000 7f5ab8cf2640 1 --2- 172.21.15.3:0/1319683437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ab4104860 0x7f5ab419a4c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:56.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.818+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/1319683437 >> 172.21.15.3:0/1319683437 conn(0x7f5ab4079740 msgr2=0x7f5ab40ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:56.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.818+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/1319683437 shutdown_connections 2024-09-21T06:17:56.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:56.818+0000 7f5ab8cf2640 1 -- 172.21.15.3:0/1319683437 wait complete. 2024-09-21T06:17:56.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 223953 13.77 MB/sec warmup 89 sec latency 32.106 ms 2024-09-21T06:17:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.489+0000 7fb311f2b640 1 -- 172.21.15.3:0/2803579719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30c109400 msgr2=0x7fb30c109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.489+0000 7fb311f2b640 1 --2- 172.21.15.3:0/2803579719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30c109400 0x7fb30c109810 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009900 tx=0x7fb2f402a410 comp rx=0 tx=0).stop 2024-09-21T06:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.489+0000 7fb311f2b640 1 -- 172.21.15.3:0/2803579719 shutdown_connections 2024-09-21T06:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.489+0000 7fb311f2b640 1 --2- 172.21.15.3:0/2803579719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c109d50 0x7fb30c110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.489+0000 7fb311f2b640 1 --2- 172.21.15.3:0/2803579719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30c109400 0x7fb30c109810 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.489+0000 7fb311f2b640 1 --2- 172.21.15.3:0/2803579719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb30c103ad0 0x7fb30c103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.489+0000 7fb311f2b640 1 -- 172.21.15.3:0/2803579719 >> 172.21.15.3:0/2803579719 conn(0x7fb30c0ff510 msgr2=0x7fb30c101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.489+0000 7fb311f2b640 1 -- 172.21.15.3:0/2803579719 shutdown_connections 2024-09-21T06:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.490+0000 7fb311f2b640 1 -- 172.21.15.3:0/2803579719 wait complete. 2024-09-21T06:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.490+0000 7fb311f2b640 1 Processor -- start 2024-09-21T06:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.490+0000 7fb311f2b640 1 -- start start 2024-09-21T06:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.491+0000 7fb311f2b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c103ad0 0x7fb30c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.491+0000 7fb311f2b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb30c109400 0x7fb30c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.491+0000 7fb311f2b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30c109d50 0x7fb30c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.491+0000 7fb311f2b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c113a60 con 0x7fb30c109d50 2024-09-21T06:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.491+0000 7fb311f2b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1138e0 con 0x7fb30c109400 2024-09-21T06:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.491+0000 7fb311f2b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c113be0 con 0x7fb30c103ad0 2024-09-21T06:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.491+0000 7fb302ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb30c109400 0x7fb30c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.491+0000 7fb30bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30c109d50 0x7fb30c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.491+0000 7fb302ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb30c109400 0x7fb30c19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40084/0 (socket says 172.21.15.3:40084) 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.491+0000 7fb302ffd640 1 -- 172.21.15.3:0/1160448388 learned_addr learned my addr 172.21.15.3:0/1160448388 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.491+0000 7fb30b7fe640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c103ad0 0x7fb30c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.491+0000 7fb302ffd640 1 -- 172.21.15.3:0/1160448388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c103ad0 msgr2=0x7fb30c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.491+0000 7fb302ffd640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c103ad0 0x7fb30c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.492+0000 7fb302ffd640 1 -- 172.21.15.3:0/1160448388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30c109d50 msgr2=0x7fb30c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.492+0000 7fb302ffd640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30c109d50 0x7fb30c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.492+0000 7fb302ffd640 1 -- 172.21.15.3:0/1160448388 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c1a0370 con 0x7fb30c109400 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.492+0000 7fb30bfff640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30c109d50 0x7fb30c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.492+0000 7fb30b7fe640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c103ad0 0x7fb30c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.492+0000 7fb302ffd640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb30c109400 0x7fb30c19a9e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4005ab0 tx=0x7fb2f4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.492+0000 7fb3097fa640 1 -- 172.21.15.3:0/1160448388 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f403f070 con 0x7fb30c109400 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.492+0000 7fb311f2b640 1 -- 172.21.15.3:0/1160448388 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c1a0600 con 0x7fb30c109400 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.492+0000 7fb3097fa640 1 -- 172.21.15.3:0/1160448388 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2f4031b50 con 0x7fb30c109400 2024-09-21T06:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.492+0000 7fb3097fa640 1 -- 172.21.15.3:0/1160448388 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4035030 con 0x7fb30c109400 2024-09-21T06:17:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.492+0000 7fb311f2b640 1 -- 172.21.15.3:0/1160448388 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c1a6df0 con 0x7fb30c109400 2024-09-21T06:17:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.495+0000 7fb3097fa640 1 -- 172.21.15.3:0/1160448388 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fb2f4042070 con 0x7fb30c109400 2024-09-21T06:17:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.495+0000 7fb318e9b740 1 Processor -- start 2024-09-21T06:17:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.495+0000 7fb318e9b740 1 -- start start 2024-09-21T06:17:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.495+0000 7fb318e9b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.495+0000 7fb318e9b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63a8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.495+0000 7fb318e9b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fe24d0 0x55dfd63a9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.495+0000 7fb318e9b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dfd631d6f0 con 0x55dfd60031b0 2024-09-21T06:17:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.495+0000 7fb318e9b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dfd630cb20 con 0x55dfd5fe24d0 2024-09-21T06:17:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.495+0000 7fb318e9b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dfd6310930 con 0x55dfd5fda2f0 2024-09-21T06:17:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb30bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb30b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63a8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb30bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63a89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34570/0 (socket says 172.21.15.3:34570) 2024-09-21T06:17:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb30bfff640 1 -- 172.21.15.3:0/1643588722 learned_addr learned my addr 172.21.15.3:0/1643588722 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb302ffd640 1 --2- 172.21.15.3:0/1643588722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fe24d0 0x55dfd63a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb30bfff640 1 -- 172.21.15.3:0/1643588722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 msgr2=0x55dfd63a8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb30bfff640 1 --2- 172.21.15.3:0/1643588722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63a8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb30bfff640 1 -- 172.21.15.3:0/1643588722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fe24d0 msgr2=0x55dfd63a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb30bfff640 1 --2- 172.21.15.3:0/1643588722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fe24d0 0x55dfd63a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb30bfff640 1 -- 172.21.15.3:0/1643588722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd62d2530 con 0x55dfd60031b0 2024-09-21T06:17:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb30b7fe640 1 --2- 172.21.15.3:0/1643588722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63a8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb3097fa640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb2f80b6680 0x7fb2f80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb30bfff640 1 --2- 172.21.15.3:0/1643588722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63a89b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb3040117e0 tx=0x7fb304011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa342d0d06267f8a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb30b7fe640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb2f80b6680 0x7fb2f80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb3097fa640 1 -- 172.21.15.3:0/1160448388 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb2f4107c90 con 0x7fb30c109400 2024-09-21T06:17:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.496+0000 7fb301ffb640 1 -- 172.21.15.3:0/1643588722 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb304011310 con 0x55dfd60031b0 2024-09-21T06:17:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.497+0000 7fb301ffb640 1 -- 172.21.15.3:0/1643588722 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb304003e20 con 0x55dfd60031b0 2024-09-21T06:17:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.497+0000 7fb301ffb640 1 -- 172.21.15.3:0/1643588722 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3040044f0 con 0x55dfd60031b0 2024-09-21T06:17:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.498+0000 7fb318e9b740 1 -- 172.21.15.3:0/1643588722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 msgr2=0x55dfd63a89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.498+0000 7fb318e9b740 1 --2- 172.21.15.3:0/1643588722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63a89b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb3040117e0 tx=0x7fb304011ca0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.498+0000 7fb30b7fe640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb2f80b6680 0x7fb2f80b8b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009fd0 tx=0x7fb2fc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.498+0000 7fb318e9b740 1 -- 172.21.15.3:0/1643588722 shutdown_connections 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.498+0000 7fb318e9b740 1 --2- 172.21.15.3:0/1643588722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63a8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.498+0000 7fb318e9b740 1 --2- 172.21.15.3:0/1643588722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fe24d0 0x55dfd63a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.498+0000 7fb318e9b740 1 --2- 172.21.15.3:0/1643588722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63a89b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.498+0000 7fb318e9b740 1 -- 172.21.15.3:0/1643588722 >> 172.21.15.3:0/1643588722 conn(0x55dfd5ff7300 msgr2=0x55dfd63a68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.498+0000 7fb318e9b740 1 -- 172.21.15.3:0/1643588722 shutdown_connections 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.498+0000 7fb318e9b740 1 -- 172.21.15.3:0/1643588722 wait complete. 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.498+0000 7fb318e9b740 1 -- start start 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.498+0000 7fb318e9b740 1 Processor -- start 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb318e9b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb318e9b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fe24d0 0x55dfd63b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb318e9b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63c0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb318e9b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dfd6244ab0 con 0x55dfd60031b0 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb318e9b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dfd62b2730 con 0x55dfd5fe24d0 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb318e9b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dfd63b7130 con 0x55dfd5fda2f0 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb318e9b740 10 client.0 _ll_register_callbacks cb 0x55dfd63066b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb30bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63c0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb302ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fe24d0 0x55dfd63b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb30bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63c0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34572/0 (socket says 172.21.15.3:34572) 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb30b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb30bfff640 1 -- 172.21.15.3:0/2917743166 learned_addr learned my addr 172.21.15.3:0/2917743166 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb30bfff640 1 -- 172.21.15.3:0/2917743166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 msgr2=0x55dfd63b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb30bfff640 1 --2- 172.21.15.3:0/2917743166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb30bfff640 1 -- 172.21.15.3:0/2917743166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fe24d0 msgr2=0x55dfd63b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb30bfff640 1 --2- 172.21.15.3:0/2917743166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fe24d0 0x55dfd63b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb30bfff640 1 -- 172.21.15.3:0/2917743166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd606e790 con 0x55dfd60031b0 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb302ffd640 1 --2- 172.21.15.3:0/2917743166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fe24d0 0x55dfd63b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.499+0000 7fb30b7fe640 1 --2- 172.21.15.3:0/2917743166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb30bfff640 1 --2- 172.21.15.3:0/2917743166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63c0420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb304005c10 tx=0x7fb304005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25cc76308dd165b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb2eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb2eaffd640 1 -- 172.21.15.3:0/2917743166 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30400fbb0 con 0x55dfd60031b0 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb2eaffd640 1 -- 172.21.15.3:0/2917743166 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb30400fd50 con 0x55dfd60031b0 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb318e9b740 1 -- 172.21.15.3:0/2917743166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb304003e20 con 0x55dfd60031b0 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb2eaffd640 1 -- 172.21.15.3:0/2917743166 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb304004a40 con 0x55dfd60031b0 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb318e9b740 10 client.16392 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb318e9b740 1 -- 172.21.15.3:0/2917743166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfd60e7620 con 0x55dfd60031b0 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb2eaffd640 1 -- 172.21.15.3:0/2917743166 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb304003e20 con 0x55dfd60031b0 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb2eaffd640 1 -- 172.21.15.3:0/2917743166 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb304018020 con 0x55dfd60031b0 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb318e9b740 10 client.16392 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb318e9b740 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb318e9b740 20 client.16392 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2917743166 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dfd5fc1d80 0x55dfd63c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb318e9b740 4 client.16392 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.500+0000 7fb318e9b740 1 -- 172.21.15.3:0/2917743166 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfd631f640 con 0x55dfd5fc1d80 2024-09-21T06:17:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.501+0000 7fb30b7fe640 1 --2- 172.21.15.3:0/2917743166 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dfd5fc1d80 0x55dfd63c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.501+0000 7fb2eaffd640 10 client.16392 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.501+0000 7fb30b7fe640 1 --2- 172.21.15.3:0/2917743166 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dfd5fc1d80 0x55dfd63c60a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146e3ec7d1fed46c server_cookie=8c8f9a1a9d6076e2 in_seq=0 out_seq=0 2024-09-21T06:17:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.502+0000 7fb2eaffd640 1 -- 172.21.15.3:0/2917743166 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dfd631f640 con 0x55dfd5fc1d80 2024-09-21T06:17:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.502+0000 7fb2eaffd640 10 client.16392 handle_command_reply: tid=0 2024-09-21T06:17:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.502+0000 7fb318e9b740 1 client.16392 shutdown 2024-09-21T06:17:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.503+0000 7fb318e9b740 1 -- 172.21.15.3:0/2917743166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 msgr2=0x55dfd63c0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.503+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2917743166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63c0420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb304005c10 tx=0x7fb304005b00 comp rx=0 tx=0).stop 2024-09-21T06:17:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.503+0000 7fb318e9b740 1 -- 172.21.15.3:0/2917743166 shutdown_connections 2024-09-21T06:17:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.503+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2917743166 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dfd5fc1d80 0x55dfd63c60a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.503+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2917743166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63c0420 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.503+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2917743166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fe24d0 0x55dfd63b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.503+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2917743166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.503+0000 7fb318e9b740 1 -- 172.21.15.3:0/2917743166 >> 172.21.15.3:0/2917743166 conn(0x55dfd5ff7300 msgr2=0x55dfd63a99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.503+0000 7fb2eaffd640 0 client.16392 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.503+0000 7fb318e9b740 1 -- 172.21.15.3:0/2917743166 shutdown_connections 2024-09-21T06:17:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.503+0000 7fb318e9b740 1 -- 172.21.15.3:0/2917743166 wait complete. 2024-09-21T06:17:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.503+0000 7fb318e9b740 20 client.16392 trim_cache size 0 max 16384 2024-09-21T06:17:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.518+0000 7fb318e9b740 1 Processor -- start 2024-09-21T06:17:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.518+0000 7fb318e9b740 1 -- start start 2024-09-21T06:17:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb318e9b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fda2f0 0x55dfd63aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb318e9b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fe24d0 0x55dfd63c0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb318e9b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63c0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb318e9b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1af670 con 0x55dfd60031b0 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb318e9b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1b1210 con 0x55dfd5fda2f0 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb318e9b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1afd40 con 0x55dfd5fe24d0 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb30b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fda2f0 0x55dfd63aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb302ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fe24d0 0x55dfd63c0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb30b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fda2f0 0x55dfd63aa0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40108/0 (socket says 172.21.15.3:40108) 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb30b7fe640 1 -- 172.21.15.3:0/2225969261 learned_addr learned my addr 172.21.15.3:0/2225969261 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb30bfff640 1 --2- 172.21.15.3:0/2225969261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb302ffd640 1 -- 172.21.15.3:0/2225969261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fda2f0 msgr2=0x55dfd63aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb302ffd640 1 --2- 172.21.15.3:0/2225969261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fda2f0 0x55dfd63aa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb302ffd640 1 -- 172.21.15.3:0/2225969261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 msgr2=0x55dfd63c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb302ffd640 1 --2- 172.21.15.3:0/2225969261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63c0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.519+0000 7fb302ffd640 1 -- 172.21.15.3:0/2225969261 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd5fc1fb0 con 0x55dfd5fe24d0 2024-09-21T06:17:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.520+0000 7fb302ffd640 1 --2- 172.21.15.3:0/2225969261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fe24d0 0x55dfd63c0320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb2f41053c0 tx=0x7fb2f410b740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e58352628b21f588 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.520+0000 7fb3017fa640 1 -- 172.21.15.3:0/2225969261 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4049040 con 0x55dfd5fe24d0 2024-09-21T06:17:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.520+0000 7fb3017fa640 1 -- 172.21.15.3:0/2225969261 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2f40cd680 con 0x55dfd5fe24d0 2024-09-21T06:17:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.520+0000 7fb3017fa640 1 -- 172.21.15.3:0/2225969261 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f403b460 con 0x55dfd5fe24d0 2024-09-21T06:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.521+0000 7fb318e9b740 1 -- 172.21.15.3:0/2225969261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fe24d0 msgr2=0x55dfd63c0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.521+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2225969261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fe24d0 0x55dfd63c0320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb2f41053c0 tx=0x7fb2f410b740 comp rx=0 tx=0).stop 2024-09-21T06:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.521+0000 7fb318e9b740 1 -- 172.21.15.3:0/2225969261 shutdown_connections 2024-09-21T06:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.521+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2225969261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fe24d0 0x55dfd63c0320 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.521+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2225969261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd60031b0 0x55dfd63c0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.521+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2225969261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd5fda2f0 0x55dfd63aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.521+0000 7fb318e9b740 1 -- 172.21.15.3:0/2225969261 >> 172.21.15.3:0/2225969261 conn(0x55dfd5ff7300 msgr2=0x55dfd63a99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.521+0000 7fb318e9b740 1 -- 172.21.15.3:0/2225969261 shutdown_connections 2024-09-21T06:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.521+0000 7fb318e9b740 1 -- 172.21.15.3:0/2225969261 wait complete. 2024-09-21T06:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.521+0000 7fb318e9b740 1 -- start start 2024-09-21T06:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.521+0000 7fb318e9b740 1 Processor -- start 2024-09-21T06:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb318e9b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63b7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb318e9b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd5fe24d0 0x55dfd63b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb318e9b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd60031b0 0x55dfd63b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb318e9b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1b0ae0 con 0x55dfd5fe24d0 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb318e9b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1b0410 con 0x55dfd60031b0 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb318e9b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1ad650 con 0x55dfd5fda2f0 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb318e9b740 10 client.0 _ll_register_callbacks cb 0x7fb2f410b610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb30b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb302ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd5fe24d0 0x55dfd63b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb302ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd5fe24d0 0x55dfd63b7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34596/0 (socket says 172.21.15.3:34596) 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb302ffd640 1 -- 172.21.15.3:0/2358134170 learned_addr learned my addr 172.21.15.3:0/2358134170 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb30b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63b7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:40644/0 (socket says 172.21.15.3:40644) 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb30bfff640 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd60031b0 0x55dfd63b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb30bfff640 1 -- 172.21.15.3:0/2358134170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 msgr2=0x55dfd63b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb30bfff640 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63b7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb30bfff640 1 -- 172.21.15.3:0/2358134170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd5fe24d0 msgr2=0x55dfd63b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb30bfff640 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd5fe24d0 0x55dfd63b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb30bfff640 1 -- 172.21.15.3:0/2358134170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd642e970 con 0x55dfd60031b0 2024-09-21T06:17:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb30b7fe640 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb302ffd640 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd5fe24d0 0x55dfd63b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb30bfff640 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd60031b0 0x55dfd63b7dc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb304030040 tx=0x7fb304005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beb8ab6ea11e2be9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.522+0000 7fb2eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:17:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb2eaffd640 1 -- 172.21.15.3:0/2358134170 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30403d060 con 0x55dfd60031b0 2024-09-21T06:17:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb318e9b740 1 -- 172.21.15.3:0/2358134170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2f40cd680 con 0x55dfd60031b0 2024-09-21T06:17:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb2eaffd640 1 -- 172.21.15.3:0/2358134170 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb304025db0 con 0x55dfd60031b0 2024-09-21T06:17:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb2eaffd640 1 -- 172.21.15.3:0/2358134170 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb304025470 con 0x55dfd60031b0 2024-09-21T06:17:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb318e9b740 10 client.25979 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb318e9b740 1 -- 172.21.15.3:0/2358134170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfd642fb60 con 0x55dfd60031b0 2024-09-21T06:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb2eaffd640 1 -- 172.21.15.3:0/2358134170 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2f40cd680 con 0x55dfd60031b0 2024-09-21T06:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb2eaffd640 1 -- 172.21.15.3:0/2358134170 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb304018040 con 0x55dfd60031b0 2024-09-21T06:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb318e9b740 10 client.25979 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb318e9b740 10 client.25979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb318e9b740 20 client.25979 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dfd5f971c0 0x55dfd6432290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb318e9b740 4 client.25979 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.523+0000 7fb318e9b740 1 -- 172.21.15.3:0/2358134170 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfd631b0f0 con 0x55dfd5f971c0 2024-09-21T06:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.524+0000 7fb30b7fe640 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dfd5f971c0 0x55dfd6432290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.524+0000 7fb30b7fe640 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dfd5f971c0 0x55dfd6432290 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a44d225d76e47d40 server_cookie=b6ed321d41385ddb in_seq=0 out_seq=0 2024-09-21T06:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.524+0000 7fb2eaffd640 10 client.25979 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.663+0000 7fb2eaffd640 1 -- 172.21.15.3:0/2358134170 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dfd631b0f0 con 0x55dfd5f971c0 2024-09-21T06:17:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.663+0000 7fb2eaffd640 10 client.25979 handle_command_reply: tid=0 2024-09-21T06:17:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.663+0000 7fb318e9b740 1 client.25979 shutdown 2024-09-21T06:17:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.663+0000 7fb318e9b740 1 -- 172.21.15.3:0/2358134170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd60031b0 msgr2=0x55dfd63b7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.663+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd60031b0 0x55dfd63b7dc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb304030040 tx=0x7fb304005d30 comp rx=0 tx=0).stop 2024-09-21T06:17:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.663+0000 7fb318e9b740 1 -- 172.21.15.3:0/2358134170 shutdown_connections 2024-09-21T06:17:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.663+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dfd5f971c0 0x55dfd6432290 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.663+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd60031b0 0x55dfd63b7dc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.663+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfd5fe24d0 0x55dfd63b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.663+0000 7fb318e9b740 1 --2- 172.21.15.3:0/2358134170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd5fda2f0 0x55dfd63b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.663+0000 7fb2eaffd640 0 client.25979 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:17:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.663+0000 7fb318e9b740 1 -- 172.21.15.3:0/2358134170 >> 172.21.15.3:0/2358134170 conn(0x55dfd5ff7300 msgr2=0x55dfd63a9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.664+0000 7fb318e9b740 1 -- 172.21.15.3:0/2358134170 shutdown_connections 2024-09-21T06:17:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.664+0000 7fb318e9b740 1 -- 172.21.15.3:0/2358134170 wait complete. 2024-09-21T06:17:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.664+0000 7fb318e9b740 20 client.25979 trim_cache size 0 max 16384 2024-09-21T06:17:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:17:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:17:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:17:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:17:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.664+0000 7fb311f2b640 1 -- 172.21.15.3:0/1160448388 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb2f80b6680 msgr2=0x7fb2f80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.664+0000 7fb311f2b640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb2f80b6680 0x7fb2f80b8b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009fd0 tx=0x7fb2fc005f90 comp rx=0 tx=0).stop 2024-09-21T06:17:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.664+0000 7fb311f2b640 1 -- 172.21.15.3:0/1160448388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb30c109400 msgr2=0x7fb30c19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.664+0000 7fb311f2b640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb30c109400 0x7fb30c19a9e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4005ab0 tx=0x7fb2f4003b90 comp rx=0 tx=0).stop 2024-09-21T06:17:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.665+0000 7fb311f2b640 1 -- 172.21.15.3:0/1160448388 shutdown_connections 2024-09-21T06:17:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.665+0000 7fb311f2b640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb2f80b6680 0x7fb2f80b8b90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.665+0000 7fb311f2b640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30c109d50 0x7fb30c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.665+0000 7fb311f2b640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb30c109400 0x7fb30c19a9e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.665+0000 7fb311f2b640 1 --2- 172.21.15.3:0/1160448388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c103ad0 0x7fb30c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.665+0000 7fb311f2b640 1 -- 172.21.15.3:0/1160448388 >> 172.21.15.3:0/1160448388 conn(0x7fb30c0ff510 msgr2=0x7fb30c101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.665+0000 7fb311f2b640 1 -- 172.21.15.3:0/1160448388 shutdown_connections 2024-09-21T06:17:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:17:57.665+0000 7fb311f2b640 1 -- 172.21.15.3:0/1160448388 wait complete. 2024-09-21T06:17:57.831 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:57 smithi003 ceph-mon[31227]: pgmap v413: 97 pgs: 97 active+clean; 58 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 434 op/s 2024-09-21T06:17:57.831 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:57 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1319683437' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:17:57.861 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:17:57.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:57 smithi137 ceph-mon[34464]: pgmap v413: 97 pgs: 97 active+clean; 58 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 434 op/s 2024-09-21T06:17:57.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:57 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1319683437' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:17:57.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:57 smithi204 ceph-mon[34659]: pgmap v413: 97 pgs: 97 active+clean; 58 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 434 op/s 2024-09-21T06:17:57.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:57 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1319683437' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:17:57.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 226442 13.76 MB/sec warmup 90 sec latency 79.703 ms 2024-09-21T06:17:58.144 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:58.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.645+0000 7fe2549c1640 1 -- 172.21.15.3:0/3498898937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2501083a0 msgr2=0x7fe2501087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:58.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.645+0000 7fe2549c1640 1 --2- 172.21.15.3:0/3498898937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2501083a0 0x7fe2501087b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe238009900 tx=0x7fe23802a410 comp rx=0 tx=0).stop 2024-09-21T06:17:58.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.645+0000 7fe2549c1640 1 -- 172.21.15.3:0/3498898937 shutdown_connections 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.645+0000 7fe2549c1640 1 --2- 172.21.15.3:0/3498898937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe250108cf0 0x7fe25010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.645+0000 7fe2549c1640 1 --2- 172.21.15.3:0/3498898937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2501083a0 0x7fe2501087b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.645+0000 7fe2549c1640 1 --2- 172.21.15.3:0/3498898937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe250102a60 0x7fe250102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.645+0000 7fe2549c1640 1 -- 172.21.15.3:0/3498898937 >> 172.21.15.3:0/3498898937 conn(0x7fe250079730 msgr2=0x7fe250079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.645+0000 7fe2549c1640 1 -- 172.21.15.3:0/3498898937 shutdown_connections 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.645+0000 7fe2549c1640 1 -- 172.21.15.3:0/3498898937 wait complete. 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.646+0000 7fe2549c1640 1 Processor -- start 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.646+0000 7fe2549c1640 1 -- start start 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.646+0000 7fe2549c1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe250102a60 0x7fe25019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.646+0000 7fe2549c1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2501083a0 0x7fe25019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.646+0000 7fe2549c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe250108cf0 0x7fe25019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.646+0000 7fe2549c1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe250112960 con 0x7fe250108cf0 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.646+0000 7fe2549c1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501127e0 con 0x7fe250102a60 2024-09-21T06:17:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.646+0000 7fe2549c1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe250112ae0 con 0x7fe2501083a0 2024-09-21T06:17:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.646+0000 7fe24ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe250108cf0 0x7fe25019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.646+0000 7fe24e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe250102a60 0x7fe25019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe24ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe250108cf0 0x7fe25019bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34620/0 (socket says 172.21.15.3:34620) 2024-09-21T06:17:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe24ed9d640 1 -- 172.21.15.3:0/2857242061 learned_addr learned my addr 172.21.15.3:0/2857242061 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe247fff640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2501083a0 0x7fe25019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe24ed9d640 1 -- 172.21.15.3:0/2857242061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2501083a0 msgr2=0x7fe25019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe24ed9d640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2501083a0 0x7fe25019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe24ed9d640 1 -- 172.21.15.3:0/2857242061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe250102a60 msgr2=0x7fe25019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe24ed9d640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe250102a60 0x7fe25019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe24ed9d640 1 -- 172.21.15.3:0/2857242061 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2501a6590 con 0x7fe250108cf0 2024-09-21T06:17:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe247fff640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2501083a0 0x7fe25019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:17:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe24ed9d640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe250108cf0 0x7fe25019bd50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe24800eb10 tx=0x7fe24800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe2477fe640 1 -- 172.21.15.3:0/2857242061 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248004b20 con 0x7fe250108cf0 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe2477fe640 1 -- 172.21.15.3:0/2857242061 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe248004cc0 con 0x7fe250108cf0 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe2549c1640 1 -- 172.21.15.3:0/2857242061 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2501a6880 con 0x7fe250108cf0 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe2477fe640 1 -- 172.21.15.3:0/2857242061 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248010670 con 0x7fe250108cf0 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.647+0000 7fe2549c1640 1 -- 172.21.15.3:0/2857242061 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2501a6d90 con 0x7fe250108cf0 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.650+0000 7fe25af35740 1 Processor -- start 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe2477fe640 1 -- 172.21.15.3:0/2857242061 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fe248005020 con 0x7fe250108cf0 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe25af35740 1 -- start start 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe25af35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391a3070 0x55bc3954a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe25af35740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc3954abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe25af35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc3954b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe25af35740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bc394c0ea0 con 0x55bc3917a1b0 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe25af35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bc394b02d0 con 0x55bc391a3070 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe25af35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bc394b40e0 con 0x55bc39182390 2024-09-21T06:17:58.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe24e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc3954abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe24e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc3954abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34632/0 (socket says 172.21.15.3:34632) 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe24e59c640 1 -- 172.21.15.3:0/575493480 learned_addr learned my addr 172.21.15.3:0/575493480 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe247fff640 1 --2- 172.21.15.3:0/575493480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391a3070 0x55bc3954a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe247fff640 1 --2- 172.21.15.3:0/575493480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc3954b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.651+0000 7fe2477fe640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe23c0b6680 0x7fe23c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.652+0000 7fe24e59c640 1 -- 172.21.15.3:0/575493480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 msgr2=0x55bc3954b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.652+0000 7fe24e59c640 1 --2- 172.21.15.3:0/575493480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc3954b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.652+0000 7fe24e59c640 1 -- 172.21.15.3:0/575493480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391a3070 msgr2=0x55bc3954a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.652+0000 7fe24e59c640 1 --2- 172.21.15.3:0/575493480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391a3070 0x55bc3954a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.652+0000 7fe24e59c640 1 -- 172.21.15.3:0/575493480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc3920e650 con 0x55bc3917a1b0 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.652+0000 7fe247fff640 1 --2- 172.21.15.3:0/575493480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc3954b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.652+0000 7fe24ed9d640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe23c0b6680 0x7fe23c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.652+0000 7fe2477fe640 1 -- 172.21.15.3:0/2857242061 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe2480e7cd0 con 0x7fe250108cf0 2024-09-21T06:17:58.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.652+0000 7fe24e59c640 1 --2- 172.21.15.3:0/575493480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc3954abe0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe240009c90 tx=0x7fe240011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357d50bc968a8627 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.652+0000 7fe244ff9640 1 -- 172.21.15.3:0/575493480 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240005220 con 0x55bc3917a1b0 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.652+0000 7fe24ed9d640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe23c0b6680 0x7fe23c0b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe2480e5510 tx=0x7fe248000f40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.652+0000 7fe244ff9640 1 -- 172.21.15.3:0/575493480 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe240004c50 con 0x55bc3917a1b0 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.652+0000 7fe244ff9640 1 -- 172.21.15.3:0/575493480 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240019690 con 0x55bc3917a1b0 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.653+0000 7fe25af35740 1 -- 172.21.15.3:0/575493480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 msgr2=0x55bc3954abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.653+0000 7fe25af35740 1 --2- 172.21.15.3:0/575493480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc3954abe0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe240009c90 tx=0x7fe240011810 comp rx=0 tx=0).stop 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.653+0000 7fe25af35740 1 -- 172.21.15.3:0/575493480 shutdown_connections 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.653+0000 7fe25af35740 1 --2- 172.21.15.3:0/575493480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc3954b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.653+0000 7fe25af35740 1 --2- 172.21.15.3:0/575493480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc3954abe0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.653+0000 7fe25af35740 1 --2- 172.21.15.3:0/575493480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391a3070 0x55bc3954a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.653+0000 7fe25af35740 1 -- 172.21.15.3:0/575493480 >> 172.21.15.3:0/575493480 conn(0x55bc39197460 msgr2=0x55bc39548590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.653+0000 7fe25af35740 1 -- 172.21.15.3:0/575493480 shutdown_connections 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.653+0000 7fe25af35740 1 -- 172.21.15.3:0/575493480 wait complete. 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.654+0000 7fe25af35740 1 -- start start 2024-09-21T06:17:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.654+0000 7fe25af35740 1 Processor -- start 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.654+0000 7fe25af35740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39555cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.654+0000 7fe25af35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc39556230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.654+0000 7fe25af35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391a3070 0x55bc39556770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.654+0000 7fe25af35740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bc39311700 con 0x55bc3917a1b0 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.654+0000 7fe25af35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bc3944b160 con 0x55bc391a3070 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.654+0000 7fe25af35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bc39558e80 con 0x55bc39182390 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.654+0000 7fe25af35740 10 client.0 _ll_register_callbacks cb 0x55bc394649e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.654+0000 7fe24e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc39556230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.654+0000 7fe24e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc39556230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:40686/0 (socket says 172.21.15.3:40686) 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.654+0000 7fe24e59c640 1 -- 172.21.15.3:0/1858282603 learned_addr learned my addr 172.21.15.3:0/1858282603 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.655+0000 7fe247fff640 1 --2- 172.21.15.3:0/1858282603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39555cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.655+0000 7fe24e59c640 1 -- 172.21.15.3:0/1858282603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391a3070 msgr2=0x55bc39556770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.655+0000 7fe24e59c640 1 --2- 172.21.15.3:0/1858282603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391a3070 0x55bc39556770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.655+0000 7fe24e59c640 1 -- 172.21.15.3:0/1858282603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 msgr2=0x55bc39555cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.655+0000 7fe24e59c640 1 --2- 172.21.15.3:0/1858282603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39555cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.655+0000 7fe24e59c640 1 -- 172.21.15.3:0/1858282603 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc39478c90 con 0x55bc39182390 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.655+0000 7fe247fff640 1 --2- 172.21.15.3:0/1858282603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39555cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.655+0000 7fe24e59c640 1 --2- 172.21.15.3:0/1858282603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc39556230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe240005660 tx=0x7fe240005030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aacfb9fd2c3af688 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.655+0000 7fe235ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.655+0000 7fe235ffb640 1 -- 172.21.15.3:0/1858282603 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240029070 con 0x55bc39182390 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.655+0000 7fe235ffb640 1 -- 172.21.15.3:0/1858282603 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe240023630 con 0x55bc39182390 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.655+0000 7fe25af35740 1 -- 172.21.15.3:0/1858282603 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe240004c50 con 0x55bc39182390 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.656+0000 7fe235ffb640 1 -- 172.21.15.3:0/1858282603 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe240035050 con 0x55bc39182390 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.656+0000 7fe25af35740 10 client.25954 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.656+0000 7fe25af35740 1 -- 172.21.15.3:0/1858282603 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bc39564ad0 con 0x55bc39182390 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.656+0000 7fe235ffb640 1 -- 172.21.15.3:0/1858282603 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe240004c50 con 0x55bc39182390 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.656+0000 7fe235ffb640 1 -- 172.21.15.3:0/1858282603 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe240019550 con 0x55bc39182390 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.656+0000 7fe25af35740 10 client.25954 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.656+0000 7fe25af35740 10 client.25954 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.656+0000 7fe25af35740 20 client.25954 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:58.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.656+0000 7fe25af35740 1 --2- 172.21.15.3:0/1858282603 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55bc391619f0 0x55bc39567e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.656+0000 7fe25af35740 4 client.25954 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.656+0000 7fe25af35740 1 -- 172.21.15.3:0/1858282603 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc394c2df0 con 0x55bc391619f0 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.656+0000 7fe247fff640 1 --2- 172.21.15.3:0/1858282603 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55bc391619f0 0x55bc39567e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.657+0000 7fe247fff640 1 --2- 172.21.15.3:0/1858282603 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55bc391619f0 0x55bc39567e30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=15f1b4798738afc8 server_cookie=dc6ff52282d0f613 in_seq=0 out_seq=0 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.657+0000 7fe235ffb640 10 client.25954 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.657+0000 7fe235ffb640 1 -- 172.21.15.3:0/1858282603 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc394c2df0 con 0x55bc391619f0 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe235ffb640 10 client.25954 handle_command_reply: tid=0 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe25af35740 1 client.25954 shutdown 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe25af35740 1 -- 172.21.15.3:0/1858282603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 msgr2=0x55bc39556230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe25af35740 1 --2- 172.21.15.3:0/1858282603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc39556230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe240005660 tx=0x7fe240005030 comp rx=0 tx=0).stop 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe25af35740 1 -- 172.21.15.3:0/1858282603 shutdown_connections 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe25af35740 1 --2- 172.21.15.3:0/1858282603 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55bc391619f0 0x55bc39567e30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe235ffb640 0 client.25954 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe25af35740 1 --2- 172.21.15.3:0/1858282603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc39556230 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe25af35740 1 --2- 172.21.15.3:0/1858282603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391a3070 0x55bc39556770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe25af35740 1 --2- 172.21.15.3:0/1858282603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39555cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe25af35740 1 -- 172.21.15.3:0/1858282603 >> 172.21.15.3:0/1858282603 conn(0x55bc39197460 msgr2=0x55bc3954d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:58.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe25af35740 1 -- 172.21.15.3:0/1858282603 shutdown_connections 2024-09-21T06:17:58.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe25af35740 1 -- 172.21.15.3:0/1858282603 wait complete. 2024-09-21T06:17:58.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.658+0000 7fe25af35740 20 client.25954 trim_cache size 0 max 16384 2024-09-21T06:17:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.673+0000 7fe25af35740 1 Processor -- start 2024-09-21T06:17:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.673+0000 7fe25af35740 1 -- start start 2024-09-21T06:17:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.673+0000 7fe25af35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc391619f0 0x55bc3954c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.673+0000 7fe25af35740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39557920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.673+0000 7fe25af35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc39182390 0x55bc39552ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.673+0000 7fe25af35740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501af760 con 0x55bc3917a1b0 2024-09-21T06:17:58.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.673+0000 7fe25af35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501b1300 con 0x55bc39182390 2024-09-21T06:17:58.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.673+0000 7fe25af35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501afe30 con 0x55bc391619f0 2024-09-21T06:17:58.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.673+0000 7fe24e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39557920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.673+0000 7fe24e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39557920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34650/0 (socket says 172.21.15.3:34650) 2024-09-21T06:17:58.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.673+0000 7fe24e59c640 1 -- 172.21.15.3:0/3607778470 learned_addr learned my addr 172.21.15.3:0/3607778470 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:58.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.673+0000 7fe247fff640 1 --2- 172.21.15.3:0/3607778470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc391619f0 0x55bc3954c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.673+0000 7fe247fff640 1 --2- 172.21.15.3:0/3607778470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc39182390 0x55bc39552ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.674+0000 7fe24e59c640 1 -- 172.21.15.3:0/3607778470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc391619f0 msgr2=0x55bc3954c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:58.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.674+0000 7fe24e59c640 1 --2- 172.21.15.3:0/3607778470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc391619f0 0x55bc3954c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.674+0000 7fe24e59c640 1 -- 172.21.15.3:0/3607778470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc39182390 msgr2=0x55bc39552ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:58.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.674+0000 7fe24e59c640 1 --2- 172.21.15.3:0/3607778470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc39182390 0x55bc39552ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.674+0000 7fe24e59c640 1 -- 172.21.15.3:0/3607778470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc39556fb0 con 0x55bc3917a1b0 2024-09-21T06:17:58.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.674+0000 7fe24e59c640 1 --2- 172.21.15.3:0/3607778470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39557920 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe240009f80 tx=0x7fe240009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347ecbee8f02b796 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:58.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.674+0000 7fe237fff640 1 -- 172.21.15.3:0/3607778470 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240029040 con 0x55bc3917a1b0 2024-09-21T06:17:58.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.674+0000 7fe237fff640 1 -- 172.21.15.3:0/3607778470 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe240005690 con 0x55bc3917a1b0 2024-09-21T06:17:58.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.674+0000 7fe237fff640 1 -- 172.21.15.3:0/3607778470 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240019430 con 0x55bc3917a1b0 2024-09-21T06:17:58.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.675+0000 7fe25af35740 1 -- 172.21.15.3:0/3607778470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 msgr2=0x55bc39557920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:58.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.675+0000 7fe25af35740 1 --2- 172.21.15.3:0/3607778470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39557920 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe240009f80 tx=0x7fe240009640 comp rx=0 tx=0).stop 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.675+0000 7fe25af35740 1 -- 172.21.15.3:0/3607778470 shutdown_connections 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.675+0000 7fe25af35740 1 --2- 172.21.15.3:0/3607778470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc39182390 0x55bc39552ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.675+0000 7fe25af35740 1 --2- 172.21.15.3:0/3607778470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39557920 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.675+0000 7fe25af35740 1 --2- 172.21.15.3:0/3607778470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc391619f0 0x55bc3954c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.675+0000 7fe25af35740 1 -- 172.21.15.3:0/3607778470 >> 172.21.15.3:0/3607778470 conn(0x55bc39197460 msgr2=0x55bc39557210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.675+0000 7fe25af35740 1 -- 172.21.15.3:0/3607778470 shutdown_connections 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.675+0000 7fe25af35740 1 -- 172.21.15.3:0/3607778470 wait complete. 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.675+0000 7fe25af35740 1 -- start start 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe25af35740 1 Processor -- start 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe25af35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391619f0 0x55bc3955e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe25af35740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39559a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe25af35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc39559f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe25af35740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501b0bd0 con 0x55bc3917a1b0 2024-09-21T06:17:58.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe25af35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501b0500 con 0x55bc391619f0 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe25af35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501ad740 con 0x55bc39182390 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe25af35740 10 client.0 _ll_register_callbacks cb 0x7fe240005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe24e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39559a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe24e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39559a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34656/0 (socket says 172.21.15.3:34656) 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe247fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391619f0 0x55bc3955e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe24e59c640 1 -- 172.21.15.3:0/2550805658 learned_addr learned my addr 172.21.15.3:0/2550805658 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe247fff640 1 --2- 172.21.15.3:0/2550805658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc39559f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe24e59c640 1 -- 172.21.15.3:0/2550805658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 msgr2=0x55bc39559f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe24e59c640 1 --2- 172.21.15.3:0/2550805658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc39559f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe24e59c640 1 -- 172.21.15.3:0/2550805658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391619f0 msgr2=0x55bc3955e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe24e59c640 1 --2- 172.21.15.3:0/2550805658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391619f0 0x55bc3955e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.676+0000 7fe24e59c640 1 -- 172.21.15.3:0/2550805658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc3955a490 con 0x55bc3917a1b0 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.677+0000 7fe24e59c640 1 --2- 172.21.15.3:0/2550805658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39559a10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe240009730 tx=0x7fe240005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d4d261ef2dfece server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:58.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.677+0000 7fe235ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:17:58.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.677+0000 7fe235ffb640 1 -- 172.21.15.3:0/2550805658 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240029040 con 0x55bc3917a1b0 2024-09-21T06:17:58.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.677+0000 7fe235ffb640 1 -- 172.21.15.3:0/2550805658 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe240019430 con 0x55bc3917a1b0 2024-09-21T06:17:58.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.677+0000 7fe25af35740 1 -- 172.21.15.3:0/2550805658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc394be8a0 con 0x55bc3917a1b0 2024-09-21T06:17:58.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.677+0000 7fe235ffb640 1 -- 172.21.15.3:0/2550805658 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe240019650 con 0x55bc3917a1b0 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.677+0000 7fe25af35740 10 client.16416 fetch_fsmap learned FSMap version 29 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.677+0000 7fe25af35740 1 -- 172.21.15.3:0/2550805658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bc395d22f0 con 0x55bc3917a1b0 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.677+0000 7fe235ffb640 1 -- 172.21.15.3:0/2550805658 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bc394be8a0 con 0x55bc3917a1b0 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.677+0000 7fe235ffb640 1 -- 172.21.15.3:0/2550805658 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe240023430 con 0x55bc3917a1b0 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.678+0000 7fe25af35740 10 client.16416 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.678+0000 7fe25af35740 10 client.16416 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.678+0000 7fe25af35740 20 client.16416 populate_metadata read hostname 'smithi003' 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.678+0000 7fe25af35740 1 --2- 172.21.15.3:0/2550805658 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55bc391a3070 0x55bc395d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.678+0000 7fe25af35740 4 client.16416 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.678+0000 7fe25af35740 1 -- 172.21.15.3:0/2550805658 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc394bbcd0 con 0x55bc391a3070 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.678+0000 7fe247fff640 1 --2- 172.21.15.3:0/2550805658 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55bc391a3070 0x55bc395d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.678+0000 7fe235ffb640 10 client.16416 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.678+0000 7fe247fff640 1 --2- 172.21.15.3:0/2550805658 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55bc391a3070 0x55bc395d49d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f1188e3041dfb4f6 server_cookie=c96a2210881fa95c in_seq=0 out_seq=0 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe235ffb640 1 -- 172.21.15.3:0/2550805658 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc394bbcd0 con 0x55bc391a3070 2024-09-21T06:17:58.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe235ffb640 10 client.16416 handle_command_reply: tid=0 2024-09-21T06:17:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe25af35740 1 client.16416 shutdown 2024-09-21T06:17:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe25af35740 1 -- 172.21.15.3:0/2550805658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 msgr2=0x55bc39559a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe25af35740 1 --2- 172.21.15.3:0/2550805658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39559a10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe240009730 tx=0x7fe240005030 comp rx=0 tx=0).stop 2024-09-21T06:17:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe25af35740 1 -- 172.21.15.3:0/2550805658 shutdown_connections 2024-09-21T06:17:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe25af35740 1 --2- 172.21.15.3:0/2550805658 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55bc391a3070 0x55bc395d49d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe25af35740 1 --2- 172.21.15.3:0/2550805658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc39182390 0x55bc39559f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe25af35740 1 --2- 172.21.15.3:0/2550805658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc3917a1b0 0x55bc39559a10 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe25af35740 1 --2- 172.21.15.3:0/2550805658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc391619f0 0x55bc3955e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe25af35740 1 -- 172.21.15.3:0/2550805658 >> 172.21.15.3:0/2550805658 conn(0x55bc39197460 msgr2=0x55bc39557b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe235ffb640 0 client.16416 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:17:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe25af35740 1 -- 172.21.15.3:0/2550805658 shutdown_connections 2024-09-21T06:17:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe25af35740 1 -- 172.21.15.3:0/2550805658 wait complete. 2024-09-21T06:17:58.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.679+0000 7fe25af35740 20 client.16416 trim_cache size 0 max 16384 2024-09-21T06:17:58.685 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:17:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.680+0000 7fe2549c1640 1 -- 172.21.15.3:0/2857242061 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe23c0b6680 msgr2=0x7fe23c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.680+0000 7fe2549c1640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe23c0b6680 0x7fe23c0b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe2480e5510 tx=0x7fe248000f40 comp rx=0 tx=0).stop 2024-09-21T06:17:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.680+0000 7fe2549c1640 1 -- 172.21.15.3:0/2857242061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe250108cf0 msgr2=0x7fe25019bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.680+0000 7fe2549c1640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe250108cf0 0x7fe25019bd50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe24800eb10 tx=0x7fe24800c2d0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.680+0000 7fe2549c1640 1 -- 172.21.15.3:0/2857242061 shutdown_connections 2024-09-21T06:17:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.680+0000 7fe2549c1640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe23c0b6680 0x7fe23c0b8b90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.680+0000 7fe2549c1640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2501083a0 0x7fe25019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.680+0000 7fe2549c1640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe250108cf0 0x7fe25019bd50 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.680+0000 7fe2549c1640 1 --2- 172.21.15.3:0/2857242061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe250102a60 0x7fe25019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.680+0000 7fe2549c1640 1 -- 172.21.15.3:0/2857242061 >> 172.21.15.3:0/2857242061 conn(0x7fe250079730 msgr2=0x7fe2501044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.680+0000 7fe2549c1640 1 -- 172.21.15.3:0/2857242061 shutdown_connections 2024-09-21T06:17:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:17:58.680+0000 7fe2549c1640 1 -- 172.21.15.3:0/2857242061 wait complete. 2024-09-21T06:17:58.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 228128 13.70 MB/sec warmup 91 sec latency 29.645 ms 2024-09-21T06:17:59.591 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:17:59.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:17:59 smithi003 ceph-mon[31227]: pgmap v414: 97 pgs: 97 active+clean; 59 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 339 op/s 2024-09-21T06:17:59.871 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:17:59 smithi137 ceph-mon[34464]: pgmap v414: 97 pgs: 97 active+clean; 59 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 339 op/s 2024-09-21T06:17:59.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:17:59 smithi204 ceph-mon[34659]: pgmap v414: 97 pgs: 97 active+clean; 59 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 339 op/s 2024-09-21T06:17:59.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 230976 13.74 MB/sec warmup 92 sec latency 8.642 ms 2024-09-21T06:18:00.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.377+0000 7f29c6b67640 1 -- 172.21.15.3:0/2978800798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c01031c0 msgr2=0x7f29c01035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.377+0000 7f29c6b67640 1 --2- 172.21.15.3:0/2978800798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c01031c0 0x7f29c01035d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009900 tx=0x7f29b402a410 comp rx=0 tx=0).stop 2024-09-21T06:18:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.377+0000 7f29c6b67640 1 -- 172.21.15.3:0/2978800798 shutdown_connections 2024-09-21T06:18:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.377+0000 7f29c6b67640 1 --2- 172.21.15.3:0/2978800798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29c0105190 0x7f29c01055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.377+0000 7f29c6b67640 1 --2- 172.21.15.3:0/2978800798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c0105ae0 0x7f29c010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.377+0000 7f29c6b67640 1 --2- 172.21.15.3:0/2978800798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c01031c0 0x7f29c01035d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.377+0000 7f29c6b67640 1 -- 172.21.15.3:0/2978800798 >> 172.21.15.3:0/2978800798 conn(0x7f29c0077c60 msgr2=0x7f29c00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.377+0000 7f29c6b67640 1 -- 172.21.15.3:0/2978800798 shutdown_connections 2024-09-21T06:18:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.377+0000 7f29c6b67640 1 -- 172.21.15.3:0/2978800798 wait complete. 2024-09-21T06:18:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.378+0000 7f29c6b67640 1 Processor -- start 2024-09-21T06:18:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.378+0000 7f29c6b67640 1 -- start start 2024-09-21T06:18:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.378+0000 7f29c6b67640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c01031c0 0x7f29c019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.378+0000 7f29c6b67640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29c0105190 0x7f29c019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.378+0000 7f29c6b67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c0105ae0 0x7f29c019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.378+0000 7f29c6b67640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f29c0112980 con 0x7f29c0105ae0 2024-09-21T06:18:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.378+0000 7f29c6b67640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f29c0112800 con 0x7f29c01031c0 2024-09-21T06:18:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.378+0000 7f29c6b67640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29c0112b00 con 0x7f29c0105190 2024-09-21T06:18:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.378+0000 7f29c5104640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c0105ae0 0x7f29c019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.378+0000 7f29c4903640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c01031c0 0x7f29c019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.378+0000 7f29c5104640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c0105ae0 0x7f29c019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34666/0 (socket says 172.21.15.3:34666) 2024-09-21T06:18:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.378+0000 7f29c5104640 1 -- 172.21.15.3:0/1466236367 learned_addr learned my addr 172.21.15.3:0/1466236367 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.378+0000 7f29bffff640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29c0105190 0x7f29c019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.379+0000 7f29c4903640 1 -- 172.21.15.3:0/1466236367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29c0105190 msgr2=0x7f29c019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.379+0000 7f29c4903640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29c0105190 0x7f29c019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.379+0000 7f29c4903640 1 -- 172.21.15.3:0/1466236367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c0105ae0 msgr2=0x7f29c019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.379+0000 7f29c4903640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c0105ae0 0x7f29c019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.379+0000 7f29c4903640 1 -- 172.21.15.3:0/1466236367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c01a03d0 con 0x7f29c01031c0 2024-09-21T06:18:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.379+0000 7f29c5104640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c0105ae0 0x7f29c019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.379+0000 7f29c4903640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c01031c0 0x7f29c019a460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f29b402a930 tx=0x7f29b4031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.379+0000 7f29bdffb640 1 -- 172.21.15.3:0/1466236367 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b403f030 con 0x7f29c01031c0 2024-09-21T06:18:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.379+0000 7f29bdffb640 1 -- 172.21.15.3:0/1466236367 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b4004490 con 0x7f29c01031c0 2024-09-21T06:18:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.379+0000 7f29c6b67640 1 -- 172.21.15.3:0/1466236367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c01a0660 con 0x7f29c01031c0 2024-09-21T06:18:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.379+0000 7f29bdffb640 1 -- 172.21.15.3:0/1466236367 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4035070 con 0x7f29c01031c0 2024-09-21T06:18:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.380+0000 7f29c6b67640 1 -- 172.21.15.3:0/1466236367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c01a6e20 con 0x7f29c01031c0 2024-09-21T06:18:00.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.383+0000 7f29bdffb640 1 -- 172.21.15.3:0/1466236367 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f29b4042030 con 0x7f29c01031c0 2024-09-21T06:18:00.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.383+0000 7f29cd0db740 1 Processor -- start 2024-09-21T06:18:00.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.383+0000 7f29cd0db740 1 -- start start 2024-09-21T06:18:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.383+0000 7f29cd0db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf94070 0x56057c33b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.383+0000 7f29cd0db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf6b1b0 0x56057c33b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.383+0000 7f29cd0db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c33bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.383+0000 7f29cd0db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56057c2b1ea0 con 0x56057bf73390 2024-09-21T06:18:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.383+0000 7f29cd0db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56057c2a12d0 con 0x56057bf94070 2024-09-21T06:18:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.383+0000 7f29cd0db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56057c2a50e0 con 0x56057bf6b1b0 2024-09-21T06:18:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.383+0000 7f29c5104640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf94070 0x56057c33b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.383+0000 7f29c5104640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf94070 0x56057c33b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40176/0 (socket says 172.21.15.3:40176) 2024-09-21T06:18:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.383+0000 7f29c5104640 1 -- 172.21.15.3:0/4270713832 learned_addr learned my addr 172.21.15.3:0/4270713832 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.383+0000 7f29bffff640 1 --2- 172.21.15.3:0/4270713832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c33bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.384+0000 7f29c4903640 1 --2- 172.21.15.3:0/4270713832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf6b1b0 0x56057c33b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.384+0000 7f29c5104640 1 -- 172.21.15.3:0/4270713832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf6b1b0 msgr2=0x56057c33b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.384+0000 7f29c5104640 1 --2- 172.21.15.3:0/4270713832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf6b1b0 0x56057c33b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.384+0000 7f29c5104640 1 -- 172.21.15.3:0/4270713832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 msgr2=0x56057c33bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.384+0000 7f29c5104640 1 --2- 172.21.15.3:0/4270713832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c33bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.384+0000 7f29c5104640 1 -- 172.21.15.3:0/4270713832 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56057bfff650 con 0x56057bf94070 2024-09-21T06:18:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.385+0000 7f29bdffb640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f29b00b6680 0x7f29b00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.385+0000 7f29c5104640 1 --2- 172.21.15.3:0/4270713832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf94070 0x56057c33b2c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f29b80118b0 tx=0x7f29b8011d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d70f7d4917a64e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.385+0000 7f29c4903640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f29b00b6680 0x7f29b00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.385+0000 7f29aaffd640 1 -- 172.21.15.3:0/4270713832 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b801e070 con 0x56057bf94070 2024-09-21T06:18:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.385+0000 7f29bdffb640 1 -- 172.21.15.3:0/1466236367 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f29b4107920 con 0x7f29c01031c0 2024-09-21T06:18:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.385+0000 7f29aaffd640 1 -- 172.21.15.3:0/4270713832 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b800fe70 con 0x56057bf94070 2024-09-21T06:18:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.385+0000 7f29aaffd640 1 -- 172.21.15.3:0/4270713832 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8019da0 con 0x56057bf94070 2024-09-21T06:18:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.387+0000 7f29c4903640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f29b00b6680 0x7f29b00b8b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f29b4105160 tx=0x7f29b410d4d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.387+0000 7f29cd0db740 1 -- 172.21.15.3:0/4270713832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf94070 msgr2=0x56057c33b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.387+0000 7f29cd0db740 1 --2- 172.21.15.3:0/4270713832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf94070 0x56057c33b2c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f29b80118b0 tx=0x7f29b8011d70 comp rx=0 tx=0).stop 2024-09-21T06:18:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.387+0000 7f29cd0db740 1 -- 172.21.15.3:0/4270713832 shutdown_connections 2024-09-21T06:18:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.387+0000 7f29cd0db740 1 --2- 172.21.15.3:0/4270713832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf6b1b0 0x56057c33b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.387+0000 7f29cd0db740 1 --2- 172.21.15.3:0/4270713832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c33bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.387+0000 7f29cd0db740 1 --2- 172.21.15.3:0/4270713832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf94070 0x56057c33b2c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.387+0000 7f29cd0db740 1 -- 172.21.15.3:0/4270713832 >> 172.21.15.3:0/4270713832 conn(0x56057bf88460 msgr2=0x56057c3391b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.387+0000 7f29cd0db740 1 -- 172.21.15.3:0/4270713832 shutdown_connections 2024-09-21T06:18:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.387+0000 7f29cd0db740 1 -- 172.21.15.3:0/4270713832 wait complete. 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.387+0000 7f29cd0db740 1 -- start start 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.387+0000 7f29cd0db740 1 Processor -- start 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.388+0000 7f29cd0db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf6b1b0 0x56057c346930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.388+0000 7f29cd0db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c346e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.388+0000 7f29cd0db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf94070 0x56057c352d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.388+0000 7f29cd0db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56057c102700 con 0x56057bf73390 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.388+0000 7f29cd0db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56057c23c160 con 0x56057bf6b1b0 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.388+0000 7f29cd0db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56057c349aa0 con 0x56057bf94070 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.388+0000 7f29bffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c346e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.388+0000 7f29c5104640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf6b1b0 0x56057c346930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.388+0000 7f29c5104640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf6b1b0 0x56057c346930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40190/0 (socket says 172.21.15.3:40190) 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.388+0000 7f29bffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c346e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34690/0 (socket says 172.21.15.3:34690) 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.388+0000 7f29c5104640 1 -- 172.21.15.3:0/18734040 learned_addr learned my addr 172.21.15.3:0/18734040 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.388+0000 7f29cd0db740 10 client.0 _ll_register_callbacks cb 0x56057c2559e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.388+0000 7f29c5104640 1 --2- 172.21.15.3:0/18734040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf94070 0x56057c352d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.389+0000 7f29bffff640 1 -- 172.21.15.3:0/18734040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf94070 msgr2=0x56057c352d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.389+0000 7f29bffff640 1 --2- 172.21.15.3:0/18734040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf94070 0x56057c352d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.389+0000 7f29bffff640 1 -- 172.21.15.3:0/18734040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf6b1b0 msgr2=0x56057c346930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.389+0000 7f29bffff640 1 --2- 172.21.15.3:0/18734040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf6b1b0 0x56057c346930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.389+0000 7f29bffff640 1 -- 172.21.15.3:0/18734040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56057c269c90 con 0x56057bf73390 2024-09-21T06:18:00.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.389+0000 7f29c5104640 1 --2- 172.21.15.3:0/18734040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf6b1b0 0x56057c346930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:00.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.389+0000 7f29bffff640 1 --2- 172.21.15.3:0/18734040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c346e70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f29c0075820 tx=0x7f29ac012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50baa225d6edaa64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:00.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.389+0000 7f299ffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:00.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.389+0000 7f299ffff640 1 -- 172.21.15.3:0/18734040 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00f920 con 0x56057bf73390 2024-09-21T06:18:00.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.389+0000 7f29cd0db740 1 -- 172.21.15.3:0/18734040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b800fe70 con 0x56057bf73390 2024-09-21T06:18:00.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.389+0000 7f299ffff640 1 -- 172.21.15.3:0/18734040 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f29ac00fac0 con 0x56057bf73390 2024-09-21T06:18:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.390+0000 7f299ffff640 1 -- 172.21.15.3:0/18734040 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29ac00fce0 con 0x56057bf73390 2024-09-21T06:18:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.390+0000 7f29cd0db740 10 client.16428 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.390+0000 7f29cd0db740 1 -- 172.21.15.3:0/18734040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56057c3557e0 con 0x56057bf73390 2024-09-21T06:18:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.390+0000 7f299ffff640 1 -- 172.21.15.3:0/18734040 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b800fe70 con 0x56057bf73390 2024-09-21T06:18:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.390+0000 7f299ffff640 1 -- 172.21.15.3:0/18734040 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f29ac025420 con 0x56057bf73390 2024-09-21T06:18:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.390+0000 7f29cd0db740 10 client.16428 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.390+0000 7f29cd0db740 10 client.16428 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:18:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.390+0000 7f29cd0db740 20 client.16428 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.390+0000 7f29cd0db740 1 --2- 172.21.15.3:0/18734040 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56057bf529f0 0x56057c358d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.390+0000 7f29cd0db740 4 client.16428 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.390+0000 7f29cd0db740 1 -- 172.21.15.3:0/18734040 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56057c2b3df0 con 0x56057bf529f0 2024-09-21T06:18:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.391+0000 7f29c5104640 1 --2- 172.21.15.3:0/18734040 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56057bf529f0 0x56057c358d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.391+0000 7f299ffff640 10 client.16428 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:18:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.391+0000 7f29c5104640 1 --2- 172.21.15.3:0/18734040 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56057bf529f0 0x56057c358d90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dfefc8a56f002688 server_cookie=32bc5c4a8dad8460 in_seq=0 out_seq=0 2024-09-21T06:18:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.392+0000 7f299ffff640 1 -- 172.21.15.3:0/18734040 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56057c2b3df0 con 0x56057bf529f0 2024-09-21T06:18:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.392+0000 7f299ffff640 10 client.16428 handle_command_reply: tid=0 2024-09-21T06:18:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.392+0000 7f29cd0db740 1 client.16428 shutdown 2024-09-21T06:18:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.392+0000 7f29cd0db740 1 -- 172.21.15.3:0/18734040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 msgr2=0x56057c346e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.392+0000 7f29cd0db740 1 --2- 172.21.15.3:0/18734040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c346e70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f29c0075820 tx=0x7f29ac012b60 comp rx=0 tx=0).stop 2024-09-21T06:18:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.393+0000 7f29cd0db740 1 -- 172.21.15.3:0/18734040 shutdown_connections 2024-09-21T06:18:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.393+0000 7f29cd0db740 1 --2- 172.21.15.3:0/18734040 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56057bf529f0 0x56057c358d90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.393+0000 7f29cd0db740 1 --2- 172.21.15.3:0/18734040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf94070 0x56057c352d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.393+0000 7f29cd0db740 1 --2- 172.21.15.3:0/18734040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c346e70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.393+0000 7f29cd0db740 1 --2- 172.21.15.3:0/18734040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf6b1b0 0x56057c346930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.393+0000 7f299ffff640 0 client.16428 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:18:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.393+0000 7f29cd0db740 1 -- 172.21.15.3:0/18734040 >> 172.21.15.3:0/18734040 conn(0x56057bf88460 msgr2=0x56057c33e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.393+0000 7f29cd0db740 1 -- 172.21.15.3:0/18734040 shutdown_connections 2024-09-21T06:18:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.393+0000 7f29cd0db740 1 -- 172.21.15.3:0/18734040 wait complete. 2024-09-21T06:18:00.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.393+0000 7f29cd0db740 20 client.16428 trim_cache size 0 max 16384 2024-09-21T06:18:00.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.410+0000 7f29cd0db740 1 Processor -- start 2024-09-21T06:18:00.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.411+0000 7f29cd0db740 1 -- start start 2024-09-21T06:18:00.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.411+0000 7f29cd0db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf6b1b0 0x56057c346df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.411+0000 7f29cd0db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c352c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.411+0000 7f29cd0db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf94070 0x56057c343b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.411+0000 7f29cd0db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01af610 con 0x56057bf73390 2024-09-21T06:18:00.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.411+0000 7f29cd0db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01b11b0 con 0x56057bf6b1b0 2024-09-21T06:18:00.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.411+0000 7f29cd0db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01afce0 con 0x56057bf94070 2024-09-21T06:18:00.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.411+0000 7f29bffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c352c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.411+0000 7f29bffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c352c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34694/0 (socket says 172.21.15.3:34694) 2024-09-21T06:18:00.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.411+0000 7f29bffff640 1 -- 172.21.15.3:0/1758797690 learned_addr learned my addr 172.21.15.3:0/1758797690 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:00.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.412+0000 7f29bffff640 1 -- 172.21.15.3:0/1758797690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf94070 msgr2=0x56057c343b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:18:00.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.412+0000 7f29bffff640 1 --2- 172.21.15.3:0/1758797690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf94070 0x56057c343b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.412+0000 7f29bffff640 1 -- 172.21.15.3:0/1758797690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf6b1b0 msgr2=0x56057c346df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.412+0000 7f29bffff640 1 --2- 172.21.15.3:0/1758797690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf6b1b0 0x56057c346df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.412+0000 7f29bffff640 1 -- 172.21.15.3:0/1758797690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56057c347bf0 con 0x56057bf73390 2024-09-21T06:18:00.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.412+0000 7f29bffff640 1 --2- 172.21.15.3:0/1758797690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c352c90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f29ac025fd0 tx=0x7f29ac025cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7624d5f90087bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:00.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.412+0000 7f29aa7fc640 1 -- 172.21.15.3:0/1758797690 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac02a070 con 0x56057bf73390 2024-09-21T06:18:00.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.413+0000 7f29aa7fc640 1 -- 172.21.15.3:0/1758797690 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ac00fb30 con 0x56057bf73390 2024-09-21T06:18:00.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.413+0000 7f29aa7fc640 1 -- 172.21.15.3:0/1758797690 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0035f0 con 0x56057bf73390 2024-09-21T06:18:00.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.414+0000 7f29cd0db740 1 -- 172.21.15.3:0/1758797690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 msgr2=0x56057c352c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.414+0000 7f29cd0db740 1 --2- 172.21.15.3:0/1758797690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c352c90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f29ac025fd0 tx=0x7f29ac025cc0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.414+0000 7f29cd0db740 1 -- 172.21.15.3:0/1758797690 shutdown_connections 2024-09-21T06:18:00.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.414+0000 7f29cd0db740 1 --2- 172.21.15.3:0/1758797690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf94070 0x56057c343b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.414+0000 7f29cd0db740 1 --2- 172.21.15.3:0/1758797690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf73390 0x56057c352c90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.414+0000 7f29cd0db740 1 --2- 172.21.15.3:0/1758797690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf6b1b0 0x56057c346df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.414+0000 7f29cd0db740 1 -- 172.21.15.3:0/1758797690 >> 172.21.15.3:0/1758797690 conn(0x56057bf88460 msgr2=0x56057c3466e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:00.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.414+0000 7f29cd0db740 1 -- 172.21.15.3:0/1758797690 shutdown_connections 2024-09-21T06:18:00.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.415+0000 7f29cd0db740 1 -- 172.21.15.3:0/1758797690 wait complete. 2024-09-21T06:18:00.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.415+0000 7f29cd0db740 1 -- start start 2024-09-21T06:18:00.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.415+0000 7f29cd0db740 1 Processor -- start 2024-09-21T06:18:00.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.416+0000 7f29cd0db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf6b1b0 0x56057c34ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.416+0000 7f29cd0db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf73390 0x56057c34b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.416+0000 7f29cd0db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf94070 0x56057c34b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.416+0000 7f29cd0db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01b0a80 con 0x56057bf94070 2024-09-21T06:18:00.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.416+0000 7f29cd0db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01b03b0 con 0x56057bf73390 2024-09-21T06:18:00.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.416+0000 7f29cd0db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01ad5f0 con 0x56057bf6b1b0 2024-09-21T06:18:00.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.416+0000 7f29cd0db740 10 client.0 _ll_register_callbacks cb 0x7f29ac009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:00.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.416+0000 7f29c5104640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf6b1b0 0x56057c34ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.416+0000 7f29bffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf73390 0x56057c34b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.416+0000 7f29bffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf73390 0x56057c34b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40212/0 (socket says 172.21.15.3:40212) 2024-09-21T06:18:00.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.416+0000 7f29bffff640 1 -- 172.21.15.3:0/1540002459 learned_addr learned my addr 172.21.15.3:0/1540002459 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:00.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.416+0000 7f29c5104640 1 --2- 172.21.15.3:0/1540002459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf94070 0x56057c34b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.417+0000 7f29c5104640 1 -- 172.21.15.3:0/1540002459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf73390 msgr2=0x56057c34b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.417+0000 7f29c5104640 1 --2- 172.21.15.3:0/1540002459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf73390 0x56057c34b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.417+0000 7f29c5104640 1 -- 172.21.15.3:0/1540002459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf94070 msgr2=0x56057c34b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.417+0000 7f29c5104640 1 --2- 172.21.15.3:0/1540002459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf94070 0x56057c34b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.417+0000 7f29c5104640 1 -- 172.21.15.3:0/1540002459 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56057c3c6b60 con 0x56057bf6b1b0 2024-09-21T06:18:00.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.417+0000 7f29c5104640 1 --2- 172.21.15.3:0/1540002459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf6b1b0 0x56057c34ac50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x56057c33d820 tx=0x7f29b8000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc43a110a6fb6248 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:00.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.417+0000 7f299ffff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:18:00.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.417+0000 7f299ffff640 1 -- 172.21.15.3:0/1540002459 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b801e040 con 0x56057bf6b1b0 2024-09-21T06:18:00.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.417+0000 7f299ffff640 1 -- 172.21.15.3:0/1540002459 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f29b8019bc0 con 0x56057bf6b1b0 2024-09-21T06:18:00.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.417+0000 7f29cd0db740 1 -- 172.21.15.3:0/1540002459 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29ac00fb30 con 0x56057bf6b1b0 2024-09-21T06:18:00.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.418+0000 7f299ffff640 1 -- 172.21.15.3:0/1540002459 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29b803e050 con 0x56057bf6b1b0 2024-09-21T06:18:00.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.418+0000 7f29cd0db740 10 client.25975 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:00.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.418+0000 7f29cd0db740 1 -- 172.21.15.3:0/1540002459 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56057c3c7d70 con 0x56057bf6b1b0 2024-09-21T06:18:00.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.418+0000 7f299ffff640 1 -- 172.21.15.3:0/1540002459 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ac00fb30 con 0x56057bf6b1b0 2024-09-21T06:18:00.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.418+0000 7f299ffff640 1 -- 172.21.15.3:0/1540002459 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f29b8024040 con 0x56057bf6b1b0 2024-09-21T06:18:00.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.418+0000 7f29cd0db740 10 client.25975 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:00.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.418+0000 7f29cd0db740 10 client.25975 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:18:00.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.418+0000 7f29cd0db740 20 client.25975 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:00.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.418+0000 7f29cd0db740 1 --2- 172.21.15.3:0/1540002459 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56057bf27da0 0x56057c3ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.418+0000 7f29cd0db740 4 client.25975 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:18:00.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.418+0000 7f29cd0db740 1 -- 172.21.15.3:0/1540002459 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56057c2af8a0 con 0x56057bf27da0 2024-09-21T06:18:00.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.419+0000 7f29bffff640 1 --2- 172.21.15.3:0/1540002459 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56057bf27da0 0x56057c3ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.419+0000 7f299ffff640 10 client.25975 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:18:00.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.420+0000 7f29bffff640 1 --2- 172.21.15.3:0/1540002459 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56057bf27da0 0x56057c3ca1c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b964afb81d090031 server_cookie=7ffb8903b5d472ae in_seq=0 out_seq=0 2024-09-21T06:18:00.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.420+0000 7f299ffff640 1 -- 172.21.15.3:0/1540002459 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56057c2af8a0 con 0x56057bf27da0 2024-09-21T06:18:00.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.420+0000 7f299ffff640 10 client.25975 handle_command_reply: tid=0 2024-09-21T06:18:00.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.420+0000 7f29cd0db740 1 client.25975 shutdown 2024-09-21T06:18:00.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.420+0000 7f29cd0db740 1 -- 172.21.15.3:0/1540002459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf6b1b0 msgr2=0x56057c34ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.420+0000 7f29cd0db740 1 --2- 172.21.15.3:0/1540002459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf6b1b0 0x56057c34ac50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x56057c33d820 tx=0x7f29b8000f40 comp rx=0 tx=0).stop 2024-09-21T06:18:00.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.420+0000 7f29cd0db740 1 -- 172.21.15.3:0/1540002459 shutdown_connections 2024-09-21T06:18:00.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.420+0000 7f29cd0db740 1 --2- 172.21.15.3:0/1540002459 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56057bf27da0 0x56057c3ca1c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.420+0000 7f29cd0db740 1 --2- 172.21.15.3:0/1540002459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56057bf94070 0x56057c34b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.420+0000 7f299ffff640 0 client.25975 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:18:00.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.420+0000 7f29cd0db740 1 --2- 172.21.15.3:0/1540002459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56057bf73390 0x56057c34b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.420+0000 7f29cd0db740 1 --2- 172.21.15.3:0/1540002459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56057bf6b1b0 0x56057c34ac50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.420+0000 7f29cd0db740 1 -- 172.21.15.3:0/1540002459 >> 172.21.15.3:0/1540002459 conn(0x56057bf88460 msgr2=0x56057c33c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:00.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.421+0000 7f29cd0db740 1 -- 172.21.15.3:0/1540002459 shutdown_connections 2024-09-21T06:18:00.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.421+0000 7f29cd0db740 1 -- 172.21.15.3:0/1540002459 wait complete. 2024-09-21T06:18:00.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.421+0000 7f29cd0db740 20 client.25975 trim_cache size 0 max 16384 2024-09-21T06:18:00.425 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:18:00.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.422+0000 7f29c6b67640 1 -- 172.21.15.3:0/1466236367 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f29b00b6680 msgr2=0x7f29b00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.422+0000 7f29c6b67640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f29b00b6680 0x7f29b00b8b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f29b4105160 tx=0x7f29b410d4d0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.422+0000 7f29c6b67640 1 -- 172.21.15.3:0/1466236367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c01031c0 msgr2=0x7f29c019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.422+0000 7f29c6b67640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c01031c0 0x7f29c019a460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f29b402a930 tx=0x7f29b4031a20 comp rx=0 tx=0).stop 2024-09-21T06:18:00.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.423+0000 7f29c6b67640 1 -- 172.21.15.3:0/1466236367 shutdown_connections 2024-09-21T06:18:00.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.423+0000 7f29c6b67640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f29b00b6680 0x7f29b00b8b90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.423+0000 7f29c6b67640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29c0105190 0x7f29c019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.423+0000 7f29c6b67640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29c0105ae0 0x7f29c019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.423+0000 7f29c6b67640 1 --2- 172.21.15.3:0/1466236367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c01031c0 0x7f29c019a460 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.423+0000 7f29c6b67640 1 -- 172.21.15.3:0/1466236367 >> 172.21.15.3:0/1466236367 conn(0x7f29c0077c60 msgr2=0x7f29c0101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:00.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.423+0000 7f29c6b67640 1 -- 172.21.15.3:0/1466236367 shutdown_connections 2024-09-21T06:18:00.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:00.423+0000 7f29c6b67640 1 -- 172.21.15.3:0/1466236367 wait complete. 2024-09-21T06:18:00.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 231836 13.61 MB/sec warmup 93 sec latency 50.347 ms 2024-09-21T06:18:01.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:18:01.437 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:01.704 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:01 smithi003 ceph-mon[31227]: pgmap v415: 97 pgs: 97 active+clean; 52 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.3 MiB/s wr, 456 op/s 2024-09-21T06:18:01.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:01 smithi137 ceph-mon[34464]: pgmap v415: 97 pgs: 97 active+clean; 52 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.3 MiB/s wr, 456 op/s 2024-09-21T06:18:01.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:01 smithi204 ceph-mon[34659]: pgmap v415: 97 pgs: 97 active+clean; 52 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.3 MiB/s wr, 456 op/s 2024-09-21T06:18:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.944+0000 7fb6a0794640 1 -- 172.21.15.3:0/3040065461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb698108d10 msgr2=0x7fb69810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.944+0000 7fb6a0794640 1 --2- 172.21.15.3:0/3040065461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb698108d10 0x7fb69810f540 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb690009900 tx=0x7fb69002a410 comp rx=0 tx=0).stop 2024-09-21T06:18:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.944+0000 7fb6a0794640 1 -- 172.21.15.3:0/3040065461 shutdown_connections 2024-09-21T06:18:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.944+0000 7fb6a0794640 1 --2- 172.21.15.3:0/3040065461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6981083c0 0x7fb6981087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.944+0000 7fb6a0794640 1 --2- 172.21.15.3:0/3040065461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb698108d10 0x7fb69810f540 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.944+0000 7fb6a0794640 1 --2- 172.21.15.3:0/3040065461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb698102a80 0x7fb698102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.944+0000 7fb6a0794640 1 -- 172.21.15.3:0/3040065461 >> 172.21.15.3:0/3040065461 conn(0x7fb698079740 msgr2=0x7fb698079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.944+0000 7fb6a0794640 1 -- 172.21.15.3:0/3040065461 shutdown_connections 2024-09-21T06:18:01.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.944+0000 7fb6a0794640 1 -- 172.21.15.3:0/3040065461 wait complete. 2024-09-21T06:18:01.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.945+0000 7fb6a0794640 1 Processor -- start 2024-09-21T06:18:01.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.945+0000 7fb6a0794640 1 -- start start 2024-09-21T06:18:01.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.945+0000 7fb6a0794640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb698102a80 0x7fb69819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.945+0000 7fb6a0794640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6981083c0 0x7fb69819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.945+0000 7fb6a0794640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb698108d10 0x7fb69819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.945+0000 7fb6a0794640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb6981129d0 con 0x7fb6981083c0 2024-09-21T06:18:01.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.945+0000 7fb6a0794640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb698112850 con 0x7fb698108d10 2024-09-21T06:18:01.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.945+0000 7fb6a0794640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb698112b50 con 0x7fb698102a80 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.945+0000 7fb69dd2f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6981083c0 0x7fb69819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.945+0000 7fb69e530640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb698102a80 0x7fb69819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.945+0000 7fb69dd2f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6981083c0 0x7fb69819b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34708/0 (socket says 172.21.15.3:34708) 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.945+0000 7fb69ed31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb698108d10 0x7fb69819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.945+0000 7fb69dd2f640 1 -- 172.21.15.3:0/2331379875 learned_addr learned my addr 172.21.15.3:0/2331379875 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.946+0000 7fb69dd2f640 1 -- 172.21.15.3:0/2331379875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb698102a80 msgr2=0x7fb69819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.946+0000 7fb69dd2f640 1 --2- 172.21.15.3:0/2331379875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb698102a80 0x7fb69819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.946+0000 7fb69dd2f640 1 -- 172.21.15.3:0/2331379875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb698108d10 msgr2=0x7fb69819bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.946+0000 7fb69dd2f640 1 --2- 172.21.15.3:0/2331379875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb698108d10 0x7fb69819bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.946+0000 7fb69dd2f640 1 -- 172.21.15.3:0/2331379875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6981a6550 con 0x7fb6981083c0 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.946+0000 7fb69e530640 1 --2- 172.21.15.3:0/2331379875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb698102a80 0x7fb69819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.946+0000 7fb69dd2f640 1 --2- 172.21.15.3:0/2331379875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6981083c0 0x7fb69819b800 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00e9f0 tx=0x7fb68c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.946+0000 7fb6977fe640 1 -- 172.21.15.3:0/2331379875 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c00ccc0 con 0x7fb6981083c0 2024-09-21T06:18:01.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.946+0000 7fb6977fe640 1 -- 172.21.15.3:0/2331379875 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb68c0054d0 con 0x7fb6981083c0 2024-09-21T06:18:01.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.946+0000 7fb6a0794640 1 -- 172.21.15.3:0/2331379875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6981a6840 con 0x7fb6981083c0 2024-09-21T06:18:01.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.946+0000 7fb6977fe640 1 -- 172.21.15.3:0/2331379875 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c010430 con 0x7fb6981083c0 2024-09-21T06:18:01.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.946+0000 7fb6a0794640 1 -- 172.21.15.3:0/2331379875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6981a6d30 con 0x7fb6981083c0 2024-09-21T06:18:01.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 234469 13.65 MB/sec warmup 94 sec latency 39.827 ms 2024-09-21T06:18:01.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.949+0000 7fb6a6d08740 1 Processor -- start 2024-09-21T06:18:01.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.949+0000 7fb6a6d08740 1 -- start start 2024-09-21T06:18:01.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.949+0000 7fb6977fe640 1 -- 172.21.15.3:0/2331379875 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fb68c0046c0 con 0x7fb6981083c0 2024-09-21T06:18:01.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.950+0000 7fb6a6d08740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34d00070 0x562c350abab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.950+0000 7fb6a6d08740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cd71b0 0x562c350abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.950+0000 7fb6a6d08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34cdf390 0x562c350ac530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.950+0000 7fb6a6d08740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562c3501dea0 con 0x562c34cd71b0 2024-09-21T06:18:01.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.950+0000 7fb6a6d08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562c3500d2d0 con 0x562c34cdf390 2024-09-21T06:18:01.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.950+0000 7fb6a6d08740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c350110e0 con 0x562c34d00070 2024-09-21T06:18:01.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.950+0000 7fb69ed31640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34d00070 0x562c350abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.950+0000 7fb69ed31640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34d00070 0x562c350abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:40800/0 (socket says 172.21.15.3:40800) 2024-09-21T06:18:01.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.950+0000 7fb69ed31640 1 -- 172.21.15.3:0/1656174568 learned_addr learned my addr 172.21.15.3:0/1656174568 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:01.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.950+0000 7fb69e530640 1 --2- 172.21.15.3:0/1656174568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cd71b0 0x562c350abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.950+0000 7fb69dd2f640 1 --2- 172.21.15.3:0/1656174568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34cdf390 0x562c350ac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.951+0000 7fb69ed31640 1 -- 172.21.15.3:0/1656174568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34cdf390 msgr2=0x562c350ac530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:01.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.951+0000 7fb69ed31640 1 --2- 172.21.15.3:0/1656174568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34cdf390 0x562c350ac530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.951+0000 7fb69ed31640 1 -- 172.21.15.3:0/1656174568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cd71b0 msgr2=0x562c350abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:01.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.951+0000 7fb69ed31640 1 --2- 172.21.15.3:0/1656174568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cd71b0 0x562c350abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.951+0000 7fb69ed31640 1 -- 172.21.15.3:0/1656174568 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c34d6b650 con 0x562c34d00070 2024-09-21T06:18:01.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.951+0000 7fb6977fe640 1 --2- 172.21.15.3:0/2331379875 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb6800b66b0 0x7fb6800b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.951+0000 7fb69e530640 1 --2- 172.21.15.3:0/2331379875 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb6800b66b0 0x7fb6800b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.951+0000 7fb69ed31640 1 --2- 172.21.15.3:0/1656174568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34d00070 0x562c350abab0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb690009900 tx=0x7fb690005f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9576060c993889f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:01.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.951+0000 7fb6977fe640 1 -- 172.21.15.3:0/2331379875 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb68c0e79d0 con 0x7fb6981083c0 2024-09-21T06:18:01.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.951+0000 7fb69dd2f640 1 --2- 172.21.15.3:0/1656174568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34cdf390 0x562c350ac530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:01.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.951+0000 7fb694ff9640 1 -- 172.21.15.3:0/1656174568 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690040070 con 0x562c34d00070 2024-09-21T06:18:01.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.951+0000 7fb694ff9640 1 -- 172.21.15.3:0/1656174568 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6900044c0 con 0x562c34d00070 2024-09-21T06:18:01.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.951+0000 7fb694ff9640 1 -- 172.21.15.3:0/1656174568 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690035040 con 0x562c34d00070 2024-09-21T06:18:01.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.952+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1656174568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34d00070 msgr2=0x562c350abab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:01.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.952+0000 7fb69e530640 1 --2- 172.21.15.3:0/2331379875 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb6800b66b0 0x7fb6800b8bc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x562c350aa730 tx=0x7fb688009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:01.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.952+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1656174568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34d00070 0x562c350abab0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb690009900 tx=0x7fb690005f30 comp rx=0 tx=0).stop 2024-09-21T06:18:01.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.952+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1656174568 shutdown_connections 2024-09-21T06:18:01.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.952+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1656174568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34cdf390 0x562c350ac530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.952+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1656174568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cd71b0 0x562c350abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.952+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1656174568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34d00070 0x562c350abab0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.952+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1656174568 >> 172.21.15.3:0/1656174568 conn(0x562c34cf4460 msgr2=0x562c350a99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:01.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.953+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1656174568 shutdown_connections 2024-09-21T06:18:01.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.953+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1656174568 wait complete. 2024-09-21T06:18:01.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.954+0000 7fb6a6d08740 1 -- start start 2024-09-21T06:18:01.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.954+0000 7fb6a6d08740 1 Processor -- start 2024-09-21T06:18:01.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.954+0000 7fb6a6d08740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cd71b0 0x562c350b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.954+0000 7fb6a6d08740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cdf390 0x562c350b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.954+0000 7fb6a6d08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34d00070 0x562c350c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.954+0000 7fb6a6d08740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562c34e6e700 con 0x562c34cdf390 2024-09-21T06:18:01.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.954+0000 7fb6a6d08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562c34fa8160 con 0x562c34d00070 2024-09-21T06:18:01.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.954+0000 7fb6a6d08740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c350ba230 con 0x562c34cd71b0 2024-09-21T06:18:01.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.954+0000 7fb69e530640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cd71b0 0x562c350b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.954+0000 7fb69e530640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cd71b0 0x562c350b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:40812/0 (socket says 172.21.15.3:40812) 2024-09-21T06:18:01.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.954+0000 7fb69e530640 1 -- 172.21.15.3:0/1569338392 learned_addr learned my addr 172.21.15.3:0/1569338392 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:01.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.954+0000 7fb69dd2f640 1 --2- 172.21.15.3:0/1569338392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cdf390 0x562c350b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.954+0000 7fb69ed31640 1 --2- 172.21.15.3:0/1569338392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34d00070 0x562c350c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.955+0000 7fb6a6d08740 10 client.0 _ll_register_callbacks cb 0x562c34fc19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:01.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.955+0000 7fb69e530640 1 -- 172.21.15.3:0/1569338392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34d00070 msgr2=0x562c350c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:01.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.955+0000 7fb69e530640 1 --2- 172.21.15.3:0/1569338392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34d00070 0x562c350c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.955+0000 7fb69e530640 1 -- 172.21.15.3:0/1569338392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cdf390 msgr2=0x562c350b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.955+0000 7fb69e530640 1 --2- 172.21.15.3:0/1569338392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cdf390 0x562c350b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.955+0000 7fb69e530640 1 -- 172.21.15.3:0/1569338392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c34fd5c90 con 0x562c34cd71b0 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.955+0000 7fb69e530640 1 --2- 172.21.15.3:0/1569338392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cd71b0 0x562c350b71a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb68800fbd0 tx=0x7fb688004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1259d32d454012d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.955+0000 7fb67dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.955+0000 7fb67dffb640 1 -- 172.21.15.3:0/1569338392 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688011d60 con 0x562c34cd71b0 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.955+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1569338392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562c3501fdf0 con 0x562c34cd71b0 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.955+0000 7fb67dffb640 1 -- 172.21.15.3:0/1569338392 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb688003970 con 0x562c34cd71b0 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.956+0000 7fb67dffb640 1 -- 172.21.15.3:0/1569338392 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb688003b70 con 0x562c34cd71b0 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.956+0000 7fb6a6d08740 10 client.25987 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.956+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1569338392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562c350c5fd0 con 0x562c34cd71b0 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.956+0000 7fb67dffb640 1 -- 172.21.15.3:0/1569338392 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562c3501fdf0 con 0x562c34cd71b0 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.956+0000 7fb67dffb640 1 -- 172.21.15.3:0/1569338392 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb688027020 con 0x562c34cd71b0 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.956+0000 7fb6a6d08740 10 client.25987 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:01.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.956+0000 7fb6a6d08740 10 client.25987 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:18:01.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.956+0000 7fb6a6d08740 20 client.25987 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:01.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.956+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1569338392 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562c34cbe9f0 0x562c350c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.956+0000 7fb6a6d08740 4 client.25987 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:01.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.956+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1569338392 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c3501b8a0 con 0x562c34cbe9f0 2024-09-21T06:18:01.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.957+0000 7fb69dd2f640 1 --2- 172.21.15.3:0/1569338392 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562c34cbe9f0 0x562c350c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.957+0000 7fb67dffb640 10 client.25987 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:18:01.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.957+0000 7fb69dd2f640 1 --2- 172.21.15.3:0/1569338392 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562c34cbe9f0 0x562c350c93f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26bee893e433602d server_cookie=62e52d6e5c80197b in_seq=0 out_seq=0 2024-09-21T06:18:01.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb67dffb640 1 -- 172.21.15.3:0/1569338392 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562c3501b8a0 con 0x562c34cbe9f0 2024-09-21T06:18:01.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb67dffb640 10 client.25987 handle_command_reply: tid=0 2024-09-21T06:18:01.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb6a6d08740 1 client.25987 shutdown 2024-09-21T06:18:01.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1569338392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cd71b0 msgr2=0x562c350b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:01.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1569338392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cd71b0 0x562c350b71a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb68800fbd0 tx=0x7fb688004ab0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1569338392 shutdown_connections 2024-09-21T06:18:01.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1569338392 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562c34cbe9f0 0x562c350c93f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1569338392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34d00070 0x562c350c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1569338392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cdf390 0x562c350b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1569338392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cd71b0 0x562c350b71a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1569338392 >> 172.21.15.3:0/1569338392 conn(0x562c34cf4460 msgr2=0x562c350aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:01.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb67dffb640 0 client.25987 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:18:01.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1569338392 shutdown_connections 2024-09-21T06:18:01.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1569338392 wait complete. 2024-09-21T06:18:01.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.958+0000 7fb6a6d08740 20 client.25987 trim_cache size 0 max 16384 2024-09-21T06:18:01.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.973+0000 7fb6a6d08740 1 Processor -- start 2024-09-21T06:18:01.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.973+0000 7fb6a6d08740 1 -- start start 2024-09-21T06:18:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb6a6d08740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cd71b0 0x562c350b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb6a6d08740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cdf390 0x562c350c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb6a6d08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34d00070 0x562c350b4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb6a6d08740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb6981ad4e0 con 0x562c34cdf390 2024-09-21T06:18:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb6a6d08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6981af080 con 0x562c34d00070 2024-09-21T06:18:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb6a6d08740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb6981adbb0 con 0x562c34cd71b0 2024-09-21T06:18:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb69dd2f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cdf390 0x562c350c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb69dd2f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cdf390 0x562c350c3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34752/0 (socket says 172.21.15.3:34752) 2024-09-21T06:18:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb69dd2f640 1 -- 172.21.15.3:0/2162214753 learned_addr learned my addr 172.21.15.3:0/2162214753 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb69ed31640 1 --2- 172.21.15.3:0/2162214753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34d00070 0x562c350b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb69e530640 1 --2- 172.21.15.3:0/2162214753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cd71b0 0x562c350b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb69dd2f640 1 -- 172.21.15.3:0/2162214753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cd71b0 msgr2=0x562c350b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb69dd2f640 1 --2- 172.21.15.3:0/2162214753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cd71b0 0x562c350b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb69dd2f640 1 -- 172.21.15.3:0/2162214753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34d00070 msgr2=0x562c350b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:01.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb69dd2f640 1 --2- 172.21.15.3:0/2162214753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34d00070 0x562c350b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb69dd2f640 1 -- 172.21.15.3:0/2162214753 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c350adda0 con 0x562c34cdf390 2024-09-21T06:18:01.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.974+0000 7fb69ed31640 1 --2- 172.21.15.3:0/2162214753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34d00070 0x562c350b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:01.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.975+0000 7fb69dd2f640 1 --2- 172.21.15.3:0/2162214753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cdf390 0x562c350c3450 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00c6a0 tx=0x7fb68c0a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde768c56f84bd49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:01.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.975+0000 7fb67ffff640 1 -- 172.21.15.3:0/2162214753 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c0acfd0 con 0x562c34cdf390 2024-09-21T06:18:01.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.975+0000 7fb67ffff640 1 -- 172.21.15.3:0/2162214753 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb68c005680 con 0x562c34cdf390 2024-09-21T06:18:01.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.975+0000 7fb67ffff640 1 -- 172.21.15.3:0/2162214753 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c0acac0 con 0x562c34cdf390 2024-09-21T06:18:01.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.976+0000 7fb6a6d08740 1 -- 172.21.15.3:0/2162214753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cdf390 msgr2=0x562c350c3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.976+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/2162214753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cdf390 0x562c350c3450 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00c6a0 tx=0x7fb68c0a9040 comp rx=0 tx=0).stop 2024-09-21T06:18:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.976+0000 7fb6a6d08740 1 -- 172.21.15.3:0/2162214753 shutdown_connections 2024-09-21T06:18:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.976+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/2162214753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34d00070 0x562c350b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.976+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/2162214753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34cdf390 0x562c350c3450 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.976+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/2162214753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cd71b0 0x562c350b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.976+0000 7fb6a6d08740 1 -- 172.21.15.3:0/2162214753 >> 172.21.15.3:0/2162214753 conn(0x562c34cf4460 msgr2=0x562c350b6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.976+0000 7fb6a6d08740 1 -- 172.21.15.3:0/2162214753 shutdown_connections 2024-09-21T06:18:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.976+0000 7fb6a6d08740 1 -- 172.21.15.3:0/2162214753 wait complete. 2024-09-21T06:18:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.976+0000 7fb6a6d08740 1 -- start start 2024-09-21T06:18:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb6a6d08740 1 Processor -- start 2024-09-21T06:18:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb6a6d08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34cd71b0 0x562c350bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb6a6d08740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cdf390 0x562c350bb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb6a6d08740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34d00070 0x562c350bb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb69dd2f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cdf390 0x562c350bb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb69ed31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34d00070 0x562c350bb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb69e530640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34cd71b0 0x562c350bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb69dd2f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cdf390 0x562c350bb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:40826/0 (socket says 172.21.15.3:40826) 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb69ed31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34d00070 0x562c350bb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34764/0 (socket says 172.21.15.3:34764) 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb69dd2f640 1 -- 172.21.15.3:0/1634963815 learned_addr learned my addr 172.21.15.3:0/1634963815 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb69e530640 1 --2- 172.21.15.3:0/1634963815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34cd71b0 0x562c350bfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40262/0 (socket says 172.21.15.3:40262) 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1634963815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb6981ae950 con 0x562c34d00070 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1634963815 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6981ae280 con 0x562c34cd71b0 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1634963815 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb6981ab4c0 con 0x562c34cdf390 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.977+0000 7fb6a6d08740 10 client.0 _ll_register_callbacks cb 0x7fb688012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.978+0000 7fb69e530640 1 -- 172.21.15.3:0/1634963815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cdf390 msgr2=0x562c350bb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.978+0000 7fb69e530640 1 --2- 172.21.15.3:0/1634963815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cdf390 0x562c350bb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.978+0000 7fb69e530640 1 -- 172.21.15.3:0/1634963815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34d00070 msgr2=0x562c350bb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.978+0000 7fb69e530640 1 --2- 172.21.15.3:0/1634963815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34d00070 0x562c350bb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.978+0000 7fb69e530640 1 -- 172.21.15.3:0/1634963815 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c350bbe20 con 0x562c34cd71b0 2024-09-21T06:18:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.978+0000 7fb69ed31640 1 --2- 172.21.15.3:0/1634963815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34d00070 0x562c350bb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.978+0000 7fb69e530640 1 --2- 172.21.15.3:0/1634963815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34cd71b0 0x562c350bfca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb688032630 tx=0x7fb688011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39ce73d35af3743b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.978+0000 7fb67dffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:18:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.978+0000 7fb67dffb640 1 -- 172.21.15.3:0/1634963815 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688003970 con 0x562c34cd71b0 2024-09-21T06:18:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.978+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1634963815 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb68c005680 con 0x562c34cd71b0 2024-09-21T06:18:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.978+0000 7fb67dffb640 1 -- 172.21.15.3:0/1634963815 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb688032d40 con 0x562c34cd71b0 2024-09-21T06:18:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.979+0000 7fb67dffb640 1 -- 172.21.15.3:0/1634963815 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb688029440 con 0x562c34cd71b0 2024-09-21T06:18:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.979+0000 7fb6a6d08740 10 client.26027 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.979+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1634963815 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562c35133cb0 con 0x562c34cd71b0 2024-09-21T06:18:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.979+0000 7fb67dffb640 1 -- 172.21.15.3:0/1634963815 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb68c005680 con 0x562c34cd71b0 2024-09-21T06:18:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.979+0000 7fb67dffb640 1 -- 172.21.15.3:0/1634963815 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb688027040 con 0x562c34cd71b0 2024-09-21T06:18:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.979+0000 7fb6a6d08740 10 client.26027 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.979+0000 7fb6a6d08740 10 client.26027 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.979+0000 7fb6a6d08740 20 client.26027 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.979+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1634963815 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562c34c93da0 0x562c351364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.979+0000 7fb6a6d08740 4 client.26027 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.979+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1634963815 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c35018cd0 con 0x562c34c93da0 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.979+0000 7fb69dd2f640 1 --2- 172.21.15.3:0/1634963815 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562c34c93da0 0x562c351364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb67dffb640 10 client.26027 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb69dd2f640 1 --2- 172.21.15.3:0/1634963815 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562c34c93da0 0x562c351364f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33daf4e8089e3b8d server_cookie=44f065a17180527f in_seq=0 out_seq=0 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb67dffb640 1 -- 172.21.15.3:0/1634963815 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c35018cd0 con 0x562c34c93da0 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb67dffb640 10 client.26027 handle_command_reply: tid=0 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb6a6d08740 1 client.26027 shutdown 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1634963815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34cd71b0 msgr2=0x562c350bfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1634963815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34cd71b0 0x562c350bfca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb688032630 tx=0x7fb688011f90 comp rx=0 tx=0).stop 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1634963815 shutdown_connections 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1634963815 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562c34c93da0 0x562c351364f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1634963815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c34cdf390 0x562c350bb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1634963815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c34d00070 0x562c350bb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb6a6d08740 1 --2- 172.21.15.3:0/1634963815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562c34cd71b0 0x562c350bfca0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb67dffb640 0 client.26027 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:18:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.980+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1634963815 >> 172.21.15.3:0/1634963815 conn(0x562c34cf4460 msgr2=0x562c350acae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.981+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1634963815 shutdown_connections 2024-09-21T06:18:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.981+0000 7fb6a6d08740 1 -- 172.21.15.3:0/1634963815 wait complete. 2024-09-21T06:18:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:18:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.981+0000 7fb6a6d08740 20 client.26027 trim_cache size 0 max 16384 2024-09-21T06:18:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.981+0000 7fb6a0794640 1 -- 172.21.15.3:0/2331379875 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb6800b66b0 msgr2=0x7fb6800b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.982+0000 7fb6a0794640 1 --2- 172.21.15.3:0/2331379875 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb6800b66b0 0x7fb6800b8bc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x562c350aa730 tx=0x7fb688009500 comp rx=0 tx=0).stop 2024-09-21T06:18:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.982+0000 7fb6a0794640 1 -- 172.21.15.3:0/2331379875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6981083c0 msgr2=0x7fb69819b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.982+0000 7fb6a0794640 1 --2- 172.21.15.3:0/2331379875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6981083c0 0x7fb69819b800 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00e9f0 tx=0x7fb68c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.982+0000 7fb6a0794640 1 -- 172.21.15.3:0/2331379875 shutdown_connections 2024-09-21T06:18:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.982+0000 7fb6a0794640 1 --2- 172.21.15.3:0/2331379875 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb6800b66b0 0x7fb6800b8bc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.982+0000 7fb6a0794640 1 --2- 172.21.15.3:0/2331379875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb698108d10 0x7fb69819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.982+0000 7fb6a0794640 1 --2- 172.21.15.3:0/2331379875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6981083c0 0x7fb69819b800 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.982+0000 7fb6a0794640 1 --2- 172.21.15.3:0/2331379875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb698102a80 0x7fb69819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.982+0000 7fb6a0794640 1 -- 172.21.15.3:0/2331379875 >> 172.21.15.3:0/2331379875 conn(0x7fb698079740 msgr2=0x7fb698104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.982+0000 7fb6a0794640 1 -- 172.21.15.3:0/2331379875 shutdown_connections 2024-09-21T06:18:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:01.982+0000 7fb6a0794640 1 -- 172.21.15.3:0/2331379875 wait complete. 2024-09-21T06:18:02.668 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:18:02.939 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:02.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 237344 13.65 MB/sec warmup 95 sec latency 12.636 ms 2024-09-21T06:18:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.660+0000 7f1da9e3e640 1 -- 172.21.15.3:0/388645992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da4104840 msgr2=0x7f1da4104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.660+0000 7f1da9e3e640 1 --2- 172.21.15.3:0/388645992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da4104840 0x7f1da4104c50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009900 tx=0x7f1d9402a410 comp rx=0 tx=0).stop 2024-09-21T06:18:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.660+0000 7f1da9e3e640 1 -- 172.21.15.3:0/388645992 shutdown_connections 2024-09-21T06:18:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.660+0000 7f1da9e3e640 1 --2- 172.21.15.3:0/388645992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da410ddf0 0x7f1da4110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.660+0000 7f1da9e3e640 1 --2- 172.21.15.3:0/388645992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1da4105270 0x7f1da410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.660+0000 7f1da9e3e640 1 --2- 172.21.15.3:0/388645992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da4104840 0x7f1da4104c50 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.660+0000 7f1da9e3e640 1 -- 172.21.15.3:0/388645992 >> 172.21.15.3:0/388645992 conn(0x7f1da4079730 msgr2=0x7f1da4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.661+0000 7f1da9e3e640 1 -- 172.21.15.3:0/388645992 shutdown_connections 2024-09-21T06:18:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.661+0000 7f1da9e3e640 1 -- 172.21.15.3:0/388645992 wait complete. 2024-09-21T06:18:03.664 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:03 smithi003 ceph-mon[31227]: pgmap v416: 97 pgs: 97 active+clean; 53 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.4 MiB/s wr, 407 op/s 2024-09-21T06:18:03.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.661+0000 7f1da9e3e640 1 Processor -- start 2024-09-21T06:18:03.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.662+0000 7f1da9e3e640 1 -- start start 2024-09-21T06:18:03.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.662+0000 7f1da9e3e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1da4104840 0x7f1da41a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.662+0000 7f1da9e3e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da4105270 0x7f1da41a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.662+0000 7f1da9e3e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da410ddf0 0x7f1da41ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.662+0000 7f1da9e3e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4113a70 con 0x7f1da410ddf0 2024-09-21T06:18:03.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.662+0000 7f1da9e3e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41138f0 con 0x7f1da4104840 2024-09-21T06:18:03.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.662+0000 7f1da9e3e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4113bf0 con 0x7f1da4105270 2024-09-21T06:18:03.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.662+0000 7f1da37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1da4104840 0x7f1da41a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.662+0000 7f1da3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da410ddf0 0x7f1da41ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.662+0000 7f1da37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1da4104840 0x7f1da41a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40274/0 (socket says 172.21.15.3:40274) 2024-09-21T06:18:03.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.662+0000 7f1da3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da410ddf0 0x7f1da41ac7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34790/0 (socket says 172.21.15.3:34790) 2024-09-21T06:18:03.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.662+0000 7f1da37fe640 1 -- 172.21.15.3:0/3159670816 learned_addr learned my addr 172.21.15.3:0/3159670816 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:03.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.662+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da4105270 0x7f1da41a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.663+0000 7f1da3fff640 1 -- 172.21.15.3:0/3159670816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da4105270 msgr2=0x7f1da41a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:03.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.663+0000 7f1da3fff640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da4105270 0x7f1da41a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.663+0000 7f1da3fff640 1 -- 172.21.15.3:0/3159670816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1da4104840 msgr2=0x7f1da41a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:03.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.663+0000 7f1da3fff640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1da4104840 0x7f1da41a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.663+0000 7f1da3fff640 1 -- 172.21.15.3:0/3159670816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da41acd10 con 0x7f1da410ddf0 2024-09-21T06:18:03.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.663+0000 7f1da37fe640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1da4104840 0x7f1da41a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:03.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.663+0000 7f1da3fff640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da410ddf0 0x7f1da41ac7d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009fd0 tx=0x7f1d9c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:03.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.663+0000 7f1da0ff9640 1 -- 172.21.15.3:0/3159670816 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c00ce20 con 0x7f1da410ddf0 2024-09-21T06:18:03.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.663+0000 7f1da0ff9640 1 -- 172.21.15.3:0/3159670816 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9c005020 con 0x7f1da410ddf0 2024-09-21T06:18:03.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.663+0000 7f1da9e3e640 1 -- 172.21.15.3:0/3159670816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da41acf40 con 0x7f1da410ddf0 2024-09-21T06:18:03.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.663+0000 7f1da0ff9640 1 -- 172.21.15.3:0/3159670816 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c010690 con 0x7f1da410ddf0 2024-09-21T06:18:03.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.663+0000 7f1da9e3e640 1 -- 172.21.15.3:0/3159670816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da41ad3a0 con 0x7f1da410ddf0 2024-09-21T06:18:03.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.666+0000 7f1db03b2740 1 Processor -- start 2024-09-21T06:18:03.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.666+0000 7f1da0ff9640 1 -- 172.21.15.3:0/3159670816 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f1d9c0054d0 con 0x7f1da410ddf0 2024-09-21T06:18:03.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.666+0000 7f1db03b2740 1 -- start start 2024-09-21T06:18:03.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1db03b2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d14d070 0x56347d4f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1db03b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d1241b0 0x56347d4f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1db03b2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d12c390 0x56347d4f5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1db03b2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56347d46aea0 con 0x56347d14d070 2024-09-21T06:18:03.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1db03b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56347d45a2d0 con 0x56347d1241b0 2024-09-21T06:18:03.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1db03b2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56347d45e0e0 con 0x56347d12c390 2024-09-21T06:18:03.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1da2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d14d070 0x56347d4f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1da37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d1241b0 0x56347d4f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1da2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d14d070 0x56347d4f46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34800/0 (socket says 172.21.15.3:34800) 2024-09-21T06:18:03.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1da37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d1241b0 0x56347d4f4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40280/0 (socket says 172.21.15.3:40280) 2024-09-21T06:18:03.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1da37fe640 1 -- 172.21.15.3:0/2714370182 learned_addr learned my addr 172.21.15.3:0/2714370182 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:03.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/2714370182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d12c390 0x56347d4f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1da37fe640 1 -- 172.21.15.3:0/2714370182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d12c390 msgr2=0x56347d4f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:03.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1da37fe640 1 --2- 172.21.15.3:0/2714370182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d12c390 0x56347d4f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1da37fe640 1 -- 172.21.15.3:0/2714370182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d14d070 msgr2=0x56347d4f46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1da0ff9640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1d8c0b6700 0x7f1d8c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1da37fe640 1 --2- 172.21.15.3:0/2714370182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d14d070 0x56347d4f46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1da37fe640 1 -- 172.21.15.3:0/2714370182 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347d1b8650 con 0x56347d1241b0 2024-09-21T06:18:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.667+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/2714370182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d14d070 0x56347d4f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.668+0000 7f1da3fff640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1d8c0b6700 0x7f1d8c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.668+0000 7f1da37fe640 1 --2- 172.21.15.3:0/2714370182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d1241b0 0x56347d4f4be0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1d94006010 tx=0x7f1d94005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=686b308988d41799 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.668+0000 7f1da0ff9640 1 -- 172.21.15.3:0/3159670816 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f1d9c0e7d00 con 0x7f1da410ddf0 2024-09-21T06:18:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.668+0000 7f1d89ffb640 1 -- 172.21.15.3:0/2714370182 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94035040 con 0x56347d1241b0 2024-09-21T06:18:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.668+0000 7f1d89ffb640 1 -- 172.21.15.3:0/2714370182 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d940317a0 con 0x56347d1241b0 2024-09-21T06:18:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.668+0000 7f1d89ffb640 1 -- 172.21.15.3:0/2714370182 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94004a00 con 0x56347d1241b0 2024-09-21T06:18:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.669+0000 7f1db03b2740 1 -- 172.21.15.3:0/2714370182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d1241b0 msgr2=0x56347d4f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.669+0000 7f1db03b2740 1 --2- 172.21.15.3:0/2714370182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d1241b0 0x56347d4f4be0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1d94006010 tx=0x7f1d94005e20 comp rx=0 tx=0).stop 2024-09-21T06:18:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.669+0000 7f1da3fff640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1d8c0b6700 0x7f1d8c0b8c10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0e5540 tx=0x7f1d9c0766b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.669+0000 7f1db03b2740 1 -- 172.21.15.3:0/2714370182 shutdown_connections 2024-09-21T06:18:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.669+0000 7f1db03b2740 1 --2- 172.21.15.3:0/2714370182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d12c390 0x56347d4f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.669+0000 7f1db03b2740 1 --2- 172.21.15.3:0/2714370182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d1241b0 0x56347d4f4be0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.669+0000 7f1db03b2740 1 --2- 172.21.15.3:0/2714370182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d14d070 0x56347d4f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.669+0000 7f1db03b2740 1 -- 172.21.15.3:0/2714370182 >> 172.21.15.3:0/2714370182 conn(0x56347d141460 msgr2=0x56347d4f2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.669+0000 7f1db03b2740 1 -- 172.21.15.3:0/2714370182 shutdown_connections 2024-09-21T06:18:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.669+0000 7f1db03b2740 1 -- 172.21.15.3:0/2714370182 wait complete. 2024-09-21T06:18:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.669+0000 7f1db03b2740 1 -- start start 2024-09-21T06:18:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.670+0000 7f1db03b2740 1 Processor -- start 2024-09-21T06:18:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.670+0000 7f1db03b2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d4ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.670+0000 7f1db03b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d12c390 0x56347d5002e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.670+0000 7f1db03b2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d14d070 0x56347d500840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.670+0000 7f1db03b2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56347d2bb700 con 0x56347d14d070 2024-09-21T06:18:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.670+0000 7f1db03b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56347d3f5160 con 0x56347d12c390 2024-09-21T06:18:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.670+0000 7f1db03b2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56347d502e20 con 0x56347d1241b0 2024-09-21T06:18:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.670+0000 7f1db03b2740 10 client.0 _ll_register_callbacks cb 0x56347d40e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.670+0000 7f1da37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d12c390 0x56347d5002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.670+0000 7f1da2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d14d070 0x56347d500840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.670+0000 7f1da37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d12c390 0x56347d5002e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40292/0 (socket says 172.21.15.3:40292) 2024-09-21T06:18:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.670+0000 7f1da2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d14d070 0x56347d500840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34802/0 (socket says 172.21.15.3:34802) 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.670+0000 7f1da37fe640 1 -- 172.21.15.3:0/3516426587 learned_addr learned my addr 172.21.15.3:0/3516426587 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/3516426587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d4ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1da37fe640 1 -- 172.21.15.3:0/3516426587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 msgr2=0x56347d4ffda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1da37fe640 1 --2- 172.21.15.3:0/3516426587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d4ffda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1da37fe640 1 -- 172.21.15.3:0/3516426587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d14d070 msgr2=0x56347d500840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1da37fe640 1 --2- 172.21.15.3:0/3516426587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d14d070 0x56347d500840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1da37fe640 1 -- 172.21.15.3:0/3516426587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347d422c90 con 0x56347d12c390 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/3516426587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d4ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1da37fe640 1 --2- 172.21.15.3:0/3516426587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d12c390 0x56347d5002e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005f70 tx=0x7f1d940042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b2df36f5edbd0db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1d82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1d82ffd640 1 -- 172.21.15.3:0/3516426587 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94035040 con 0x56347d12c390 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1db03b2740 1 -- 172.21.15.3:0/3516426587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d940317a0 con 0x56347d12c390 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1d82ffd640 1 -- 172.21.15.3:0/3516426587 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f1d94003800 con 0x56347d12c390 2024-09-21T06:18:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1d82ffd640 1 -- 172.21.15.3:0/3516426587 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1d94059050 con 0x56347d12c390 2024-09-21T06:18:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1db03b2740 10 client.26039 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.671+0000 7f1db03b2740 1 -- 172.21.15.3:0/3516426587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56347d50eaa0 con 0x56347d12c390 2024-09-21T06:18:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.672+0000 7f1d82ffd640 1 -- 172.21.15.3:0/3516426587 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d940317a0 con 0x56347d12c390 2024-09-21T06:18:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.672+0000 7f1d82ffd640 1 -- 172.21.15.3:0/3516426587 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1d94004770 con 0x56347d12c390 2024-09-21T06:18:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.672+0000 7f1db03b2740 10 client.26039 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.672+0000 7f1db03b2740 10 client.26039 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:18:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.672+0000 7f1db03b2740 20 client.26039 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.672+0000 7f1db03b2740 1 --2- 172.21.15.3:0/3516426587 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56347d10b9f0 0x56347d511e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.672+0000 7f1db03b2740 4 client.26039 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:03.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.672+0000 7f1db03b2740 1 -- 172.21.15.3:0/3516426587 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56347d46cdf0 con 0x56347d10b9f0 2024-09-21T06:18:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.672+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/3516426587 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56347d10b9f0 0x56347d511e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.673+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/3516426587 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56347d10b9f0 0x56347d511e00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fcfcbe2df66458ef server_cookie=b69c6cd454eafb6e in_seq=0 out_seq=0 2024-09-21T06:18:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.673+0000 7f1d82ffd640 10 client.26039 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:18:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.673+0000 7f1d82ffd640 1 -- 172.21.15.3:0/3516426587 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56347d46cdf0 con 0x56347d10b9f0 2024-09-21T06:18:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.673+0000 7f1d82ffd640 10 client.26039 handle_command_reply: tid=0 2024-09-21T06:18:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.673+0000 7f1db03b2740 1 client.26039 shutdown 2024-09-21T06:18:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.674+0000 7f1db03b2740 1 -- 172.21.15.3:0/3516426587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d12c390 msgr2=0x56347d5002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.674+0000 7f1db03b2740 1 --2- 172.21.15.3:0/3516426587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d12c390 0x56347d5002e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005f70 tx=0x7f1d940042f0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.674+0000 7f1db03b2740 1 -- 172.21.15.3:0/3516426587 shutdown_connections 2024-09-21T06:18:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.674+0000 7f1db03b2740 1 --2- 172.21.15.3:0/3516426587 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56347d10b9f0 0x56347d511e00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.674+0000 7f1db03b2740 1 --2- 172.21.15.3:0/3516426587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d14d070 0x56347d500840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.674+0000 7f1d82ffd640 0 client.26039 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:18:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.674+0000 7f1db03b2740 1 --2- 172.21.15.3:0/3516426587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d12c390 0x56347d5002e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.674+0000 7f1db03b2740 1 --2- 172.21.15.3:0/3516426587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d4ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.674+0000 7f1db03b2740 1 -- 172.21.15.3:0/3516426587 >> 172.21.15.3:0/3516426587 conn(0x56347d141460 msgr2=0x56347d4f7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.674+0000 7f1db03b2740 1 -- 172.21.15.3:0/3516426587 shutdown_connections 2024-09-21T06:18:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.674+0000 7f1db03b2740 1 -- 172.21.15.3:0/3516426587 wait complete. 2024-09-21T06:18:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.674+0000 7f1db03b2740 20 client.26039 trim_cache size 0 max 16384 2024-09-21T06:18:03.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.687+0000 7f1db03b2740 1 Processor -- start 2024-09-21T06:18:03.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.687+0000 7f1db03b2740 1 -- start start 2024-09-21T06:18:03.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.687+0000 7f1db03b2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d10b9f0 0x56347d500260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.687+0000 7f1db03b2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d4fcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.687+0000 7f1db03b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d12c390 0x56347d4fd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.687+0000 7f1db03b2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b5ce0 con 0x56347d10b9f0 2024-09-21T06:18:03.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.687+0000 7f1db03b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b7880 con 0x56347d12c390 2024-09-21T06:18:03.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.687+0000 7f1db03b2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b63b0 con 0x56347d1241b0 2024-09-21T06:18:03.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.687+0000 7f1da2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d10b9f0 0x56347d500260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.687+0000 7f1da2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d10b9f0 0x56347d500260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34804/0 (socket says 172.21.15.3:34804) 2024-09-21T06:18:03.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.688+0000 7f1da2ffd640 1 -- 172.21.15.3:0/1956864862 learned_addr learned my addr 172.21.15.3:0/1956864862 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:03.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.688+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/1956864862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d12c390 0x56347d4fd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.688+0000 7f1da37fe640 1 --2- 172.21.15.3:0/1956864862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d4fcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.688+0000 7f1da2ffd640 1 -- 172.21.15.3:0/1956864862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 msgr2=0x56347d4fcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:03.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.688+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/1956864862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d4fcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.688+0000 7f1da2ffd640 1 -- 172.21.15.3:0/1956864862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d12c390 msgr2=0x56347d4fd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:03.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.688+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/1956864862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d12c390 0x56347d4fd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.688+0000 7f1da2ffd640 1 -- 172.21.15.3:0/1956864862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347d4f6990 con 0x56347d10b9f0 2024-09-21T06:18:03.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.688+0000 7f1da37fe640 1 --2- 172.21.15.3:0/1956864862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d4fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:03.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.688+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/1956864862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d10b9f0 0x56347d500260 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1da410fa20 tx=0x7f1d98005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e06491905399ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:03.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.688+0000 7f1d897fa640 1 -- 172.21.15.3:0/1956864862 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98004310 con 0x56347d10b9f0 2024-09-21T06:18:03.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.688+0000 7f1d897fa640 1 -- 172.21.15.3:0/1956864862 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d980044b0 con 0x56347d10b9f0 2024-09-21T06:18:03.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.688+0000 7f1d897fa640 1 -- 172.21.15.3:0/1956864862 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98005650 con 0x56347d10b9f0 2024-09-21T06:18:03.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.689+0000 7f1db03b2740 1 -- 172.21.15.3:0/1956864862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d10b9f0 msgr2=0x56347d500260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:03.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.689+0000 7f1db03b2740 1 --2- 172.21.15.3:0/1956864862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d10b9f0 0x56347d500260 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1da410fa20 tx=0x7f1d98005f90 comp rx=0 tx=0).stop 2024-09-21T06:18:03.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.689+0000 7f1db03b2740 1 -- 172.21.15.3:0/1956864862 shutdown_connections 2024-09-21T06:18:03.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.689+0000 7f1db03b2740 1 --2- 172.21.15.3:0/1956864862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d4fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.689+0000 7f1db03b2740 1 --2- 172.21.15.3:0/1956864862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d12c390 0x56347d4fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.689+0000 7f1db03b2740 1 --2- 172.21.15.3:0/1956864862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d10b9f0 0x56347d500260 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.689+0000 7f1db03b2740 1 -- 172.21.15.3:0/1956864862 >> 172.21.15.3:0/1956864862 conn(0x56347d141460 msgr2=0x56347d4ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:03.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.689+0000 7f1db03b2740 1 -- 172.21.15.3:0/1956864862 shutdown_connections 2024-09-21T06:18:03.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.689+0000 7f1db03b2740 1 -- 172.21.15.3:0/1956864862 wait complete. 2024-09-21T06:18:03.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.689+0000 7f1db03b2740 1 -- start start 2024-09-21T06:18:03.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1db03b2740 1 Processor -- start 2024-09-21T06:18:03.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1db03b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d10b9f0 0x56347d513980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1db03b2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d513f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1db03b2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d12c390 0x56347d5745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1db03b2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b7150 con 0x56347d12c390 2024-09-21T06:18:03.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1db03b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b6a80 con 0x56347d10b9f0 2024-09-21T06:18:03.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1db03b2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b3cc0 con 0x56347d1241b0 2024-09-21T06:18:03.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1db03b2740 10 client.0 _ll_register_callbacks cb 0x7f1d94040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:03.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1da2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d10b9f0 0x56347d513980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1da37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d513f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1da2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d10b9f0 0x56347d513980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:40320/0 (socket says 172.21.15.3:40320) 2024-09-21T06:18:03.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1da2ffd640 1 -- 172.21.15.3:0/2120846286 learned_addr learned my addr 172.21.15.3:0/2120846286 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:03.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1da37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d513f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:40902/0 (socket says 172.21.15.3:40902) 2024-09-21T06:18:03.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.690+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/2120846286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d12c390 0x56347d5745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.691+0000 7f1da37fe640 1 -- 172.21.15.3:0/2120846286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d10b9f0 msgr2=0x56347d513980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:03.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.691+0000 7f1da37fe640 1 --2- 172.21.15.3:0/2120846286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d10b9f0 0x56347d513980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.691+0000 7f1da37fe640 1 -- 172.21.15.3:0/2120846286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d12c390 msgr2=0x56347d5745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.691+0000 7f1da37fe640 1 --2- 172.21.15.3:0/2120846286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d12c390 0x56347d5745b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.691+0000 7f1da37fe640 1 -- 172.21.15.3:0/2120846286 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347d574c20 con 0x56347d1241b0 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.691+0000 7f1da37fe640 1 --2- 172.21.15.3:0/2120846286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d513f60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1d94004bc0 tx=0x7f1d94004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd4c6ea754cd7c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.691+0000 7f1d82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.691+0000 7f1d82ffd640 1 -- 172.21.15.3:0/2120846286 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94035040 con 0x56347d1241b0 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.691+0000 7f1d82ffd640 1 -- 172.21.15.3:0/2120846286 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f1d94004db0 con 0x56347d1241b0 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.691+0000 7f1db03b2740 1 -- 172.21.15.3:0/2120846286 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d980044b0 con 0x56347d1241b0 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.691+0000 7f1d82ffd640 1 -- 172.21.15.3:0/2120846286 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1d94004fb0 con 0x56347d1241b0 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.691+0000 7f1db03b2740 10 client.26011 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.691+0000 7f1db03b2740 1 -- 172.21.15.3:0/2120846286 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56347d57d100 con 0x56347d1241b0 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.692+0000 7f1d82ffd640 1 -- 172.21.15.3:0/2120846286 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d980044b0 con 0x56347d1241b0 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.692+0000 7f1d82ffd640 1 -- 172.21.15.3:0/2120846286 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1d94032040 con 0x56347d1241b0 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.692+0000 7f1db03b2740 10 client.26011 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.692+0000 7f1db03b2740 10 client.26011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:18:03.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.692+0000 7f1db03b2740 20 client.26011 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:03.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.692+0000 7f1db03b2740 1 --2- 172.21.15.3:0/2120846286 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56347d14d070 0x56347d57f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:03.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.692+0000 7f1db03b2740 4 client.26011 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:18:03.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.692+0000 7f1db03b2740 1 -- 172.21.15.3:0/2120846286 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56347d4688a0 con 0x56347d14d070 2024-09-21T06:18:03.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.692+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/2120846286 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56347d14d070 0x56347d57f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:03.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.692+0000 7f1d82ffd640 10 client.26011 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:18:03.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1da2ffd640 1 --2- 172.21.15.3:0/2120846286 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56347d14d070 0x56347d57f820 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b20291d34560256c server_cookie=85f890eeec06870d in_seq=0 out_seq=0 2024-09-21T06:18:03.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1d82ffd640 1 -- 172.21.15.3:0/2120846286 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56347d4688a0 con 0x56347d14d070 2024-09-21T06:18:03.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1d82ffd640 10 client.26011 handle_command_reply: tid=0 2024-09-21T06:18:03.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1db03b2740 1 client.26011 shutdown 2024-09-21T06:18:03.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1db03b2740 1 -- 172.21.15.3:0/2120846286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 msgr2=0x56347d513f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:03.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1db03b2740 1 --2- 172.21.15.3:0/2120846286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d513f60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1d94004bc0 tx=0x7f1d94004250 comp rx=0 tx=0).stop 2024-09-21T06:18:03.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1db03b2740 1 -- 172.21.15.3:0/2120846286 shutdown_connections 2024-09-21T06:18:03.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1db03b2740 1 --2- 172.21.15.3:0/2120846286 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x56347d14d070 0x56347d57f820 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1db03b2740 1 --2- 172.21.15.3:0/2120846286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56347d1241b0 0x56347d513f60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1db03b2740 1 --2- 172.21.15.3:0/2120846286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56347d12c390 0x56347d5745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1d82ffd640 0 client.26011 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:18:03.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1db03b2740 1 --2- 172.21.15.3:0/2120846286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56347d10b9f0 0x56347d513980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1db03b2740 1 -- 172.21.15.3:0/2120846286 >> 172.21.15.3:0/2120846286 conn(0x56347d141460 msgr2=0x56347d4f56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:03.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1db03b2740 1 -- 172.21.15.3:0/2120846286 shutdown_connections 2024-09-21T06:18:03.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1db03b2740 1 -- 172.21.15.3:0/2120846286 wait complete. 2024-09-21T06:18:03.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.693+0000 7f1db03b2740 20 client.26011 trim_cache size 0 max 16384 2024-09-21T06:18:03.699 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:18:03.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.694+0000 7f1da9e3e640 1 -- 172.21.15.3:0/3159670816 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1d8c0b6700 msgr2=0x7f1d8c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:03.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.694+0000 7f1da9e3e640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1d8c0b6700 0x7f1d8c0b8c10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0e5540 tx=0x7f1d9c0766b0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.694+0000 7f1da9e3e640 1 -- 172.21.15.3:0/3159670816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da410ddf0 msgr2=0x7f1da41ac7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:03.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.694+0000 7f1da9e3e640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da410ddf0 0x7f1da41ac7d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009fd0 tx=0x7f1d9c00ef10 comp rx=0 tx=0).stop 2024-09-21T06:18:03.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.694+0000 7f1da9e3e640 1 -- 172.21.15.3:0/3159670816 shutdown_connections 2024-09-21T06:18:03.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.694+0000 7f1da9e3e640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1d8c0b6700 0x7f1d8c0b8c10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.694+0000 7f1da9e3e640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da4105270 0x7f1da41a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.694+0000 7f1da9e3e640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da410ddf0 0x7f1da41ac7d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.694+0000 7f1da9e3e640 1 --2- 172.21.15.3:0/3159670816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1da4104840 0x7f1da41a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:03.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.694+0000 7f1da9e3e640 1 -- 172.21.15.3:0/3159670816 >> 172.21.15.3:0/3159670816 conn(0x7f1da4079730 msgr2=0x7f1da41006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:03.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.694+0000 7f1da9e3e640 1 -- 172.21.15.3:0/3159670816 shutdown_connections 2024-09-21T06:18:03.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:03.694+0000 7f1da9e3e640 1 -- 172.21.15.3:0/3159670816 wait complete. 2024-09-21T06:18:03.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:03 smithi137 ceph-mon[34464]: pgmap v416: 97 pgs: 97 active+clean; 53 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.4 MiB/s wr, 407 op/s 2024-09-21T06:18:03.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:03 smithi204 ceph-mon[34659]: pgmap v416: 97 pgs: 97 active+clean; 53 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.4 MiB/s wr, 407 op/s 2024-09-21T06:18:03.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 238755 13.58 MB/sec warmup 96 sec latency 39.761 ms 2024-09-21T06:18:04.866 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:18:04.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 241629 13.62 MB/sec warmup 97 sec latency 12.661 ms 2024-09-21T06:18:05.139 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:05.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:05 smithi003 ceph-mon[31227]: pgmap v417: 97 pgs: 97 active+clean; 53 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 385 op/s 2024-09-21T06:18:05.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.761+0000 7f8cbd9c2640 1 -- 172.21.15.3:0/940540654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb8107b80 msgr2=0x7f8cb8107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.761+0000 7f8cbd9c2640 1 --2- 172.21.15.3:0/940540654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb8107b80 0x7f8cb8107f90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009900 tx=0x7f8cac02a410 comp rx=0 tx=0).stop 2024-09-21T06:18:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.761+0000 7f8cbd9c2640 1 -- 172.21.15.3:0/940540654 shutdown_connections 2024-09-21T06:18:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.761+0000 7f8cbd9c2640 1 --2- 172.21.15.3:0/940540654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cb81084d0 0x7f8cb810ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.761+0000 7f8cbd9c2640 1 --2- 172.21.15.3:0/940540654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb8107b80 0x7f8cb8107f90 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.761+0000 7f8cbd9c2640 1 --2- 172.21.15.3:0/940540654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cb8102240 0x7f8cb8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.761+0000 7f8cbd9c2640 1 -- 172.21.15.3:0/940540654 >> 172.21.15.3:0/940540654 conn(0x7f8cb8079f90 msgr2=0x7f8cb807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.761+0000 7f8cbd9c2640 1 -- 172.21.15.3:0/940540654 shutdown_connections 2024-09-21T06:18:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.761+0000 7f8cbd9c2640 1 -- 172.21.15.3:0/940540654 wait complete. 2024-09-21T06:18:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.762+0000 7f8cbd9c2640 1 Processor -- start 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.762+0000 7f8cbd9c2640 1 -- start start 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.762+0000 7f8cbd9c2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cb8102240 0x7f8cb819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.762+0000 7f8cbd9c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb8107b80 0x7f8cb819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.762+0000 7f8cbd9c2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cb81084d0 0x7f8cb819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.762+0000 7f8cbd9c2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8111910 con 0x7f8cb8107b80 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.762+0000 7f8cbd9c2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8111790 con 0x7f8cb81084d0 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.762+0000 7f8cbd9c2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8111a90 con 0x7f8cb8102240 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.763+0000 7f8cb67fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb8107b80 0x7f8cb819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.763+0000 7f8cb6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cb8102240 0x7f8cb819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.763+0000 7f8cb67fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb8107b80 0x7f8cb819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41176/0 (socket says 172.21.15.3:41176) 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.763+0000 7f8cb67fc640 1 -- 172.21.15.3:0/4250816794 learned_addr learned my addr 172.21.15.3:0/4250816794 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.763+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cb81084d0 0x7f8cb819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.763+0000 7f8cb6ffd640 1 -- 172.21.15.3:0/4250816794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cb81084d0 msgr2=0x7f8cb819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.763+0000 7f8cb6ffd640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cb81084d0 0x7f8cb819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.763+0000 7f8cb6ffd640 1 -- 172.21.15.3:0/4250816794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb8107b80 msgr2=0x7f8cb819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.763+0000 7f8cb6ffd640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb8107b80 0x7f8cb819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.763+0000 7f8cb6ffd640 1 -- 172.21.15.3:0/4250816794 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb81a04c0 con 0x7f8cb8102240 2024-09-21T06:18:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.763+0000 7f8cb67fc640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb8107b80 0x7f8cb819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.763+0000 7f8cb6ffd640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cb8102240 0x7f8cb819a550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800ea90 tx=0x7f8ca800ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.763+0000 7f8cbc9c0640 1 -- 172.21.15.3:0/4250816794 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca800cc80 con 0x7f8cb8102240 2024-09-21T06:18:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.764+0000 7f8cbd9c2640 1 -- 172.21.15.3:0/4250816794 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb81a07b0 con 0x7f8cb8102240 2024-09-21T06:18:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.764+0000 7f8cbc9c0640 1 -- 172.21.15.3:0/4250816794 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca800ce20 con 0x7f8cb8102240 2024-09-21T06:18:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.764+0000 7f8cbc9c0640 1 -- 172.21.15.3:0/4250816794 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8004a00 con 0x7f8cb8102240 2024-09-21T06:18:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.764+0000 7f8cbd9c2640 1 -- 172.21.15.3:0/4250816794 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb81a6f10 con 0x7f8cb8102240 2024-09-21T06:18:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.766+0000 7f8cc3f36740 1 Processor -- start 2024-09-21T06:18:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.766+0000 7f8cc3f36740 1 -- start start 2024-09-21T06:18:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.766+0000 7f8cc3f36740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947d9070 0x55b994b802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.766+0000 7f8cc3f36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b01b0 0x55b994b80800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.766+0000 7f8cc3f36740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b8390 0x55b994b80d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cc3f36740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b994af6ea0 con 0x55b9947d9070 2024-09-21T06:18:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cc3f36740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b994ae62d0 con 0x55b9947b8390 2024-09-21T06:18:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cc3f36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b994aea0e0 con 0x55b9947b01b0 2024-09-21T06:18:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cb77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947d9070 0x55b994b802c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cb77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947d9070 0x55b994b802c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41184/0 (socket says 172.21.15.3:41184) 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cb77fe640 1 -- 172.21.15.3:0/706579710 learned_addr learned my addr 172.21.15.3:0/706579710 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cbc9c0640 1 -- 172.21.15.3:0/4250816794 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f8ca801d020 con 0x7f8cb8102240 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cb67fc640 1 --2- 172.21.15.3:0/706579710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b8390 0x55b994b80d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cb6ffd640 1 --2- 172.21.15.3:0/706579710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b01b0 0x55b994b80800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cb77fe640 1 -- 172.21.15.3:0/706579710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b01b0 msgr2=0x55b994b80800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/706579710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b01b0 0x55b994b80800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cb77fe640 1 -- 172.21.15.3:0/706579710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b8390 msgr2=0x55b994b80d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/706579710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b8390 0x55b994b80d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cb77fe640 1 -- 172.21.15.3:0/706579710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b994844650 con 0x55b9947d9070 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cb67fc640 1 --2- 172.21.15.3:0/706579710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b8390 0x55b994b80d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cb6ffd640 1 --2- 172.21.15.3:0/706579710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b01b0 0x55b994b80800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.767+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/706579710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947d9070 0x55b994b802c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0003a20 tx=0x7f8cb0003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eedc80efe1cd8e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.768+0000 7f8c9dffb640 1 -- 172.21.15.3:0/706579710 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb001a470 con 0x55b9947d9070 2024-09-21T06:18:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.768+0000 7f8c9dffb640 1 -- 172.21.15.3:0/706579710 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb0015070 con 0x55b9947d9070 2024-09-21T06:18:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.768+0000 7f8c9dffb640 1 -- 172.21.15.3:0/706579710 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00119e0 con 0x55b9947d9070 2024-09-21T06:18:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.768+0000 7f8cbc9c0640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8ca00b6680 0x7f8ca00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.768+0000 7f8cb6ffd640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8ca00b6680 0x7f8ca00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.768+0000 7f8cbc9c0640 1 -- 172.21.15.3:0/4250816794 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8ca80e71e0 con 0x7f8cb8102240 2024-09-21T06:18:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.769+0000 7f8cc3f36740 1 -- 172.21.15.3:0/706579710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947d9070 msgr2=0x55b994b802c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.769+0000 7f8cb6ffd640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8ca00b6680 0x7f8ca00b8b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80acfa0 tx=0x7f8ca80054e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.769+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/706579710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947d9070 0x55b994b802c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0003a20 tx=0x7f8cb0003ee0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.769+0000 7f8cc3f36740 1 -- 172.21.15.3:0/706579710 shutdown_connections 2024-09-21T06:18:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.769+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/706579710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b01b0 0x55b994b80800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.769+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/706579710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b8390 0x55b994b80d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.769+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/706579710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947d9070 0x55b994b802c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.769+0000 7f8cc3f36740 1 -- 172.21.15.3:0/706579710 >> 172.21.15.3:0/706579710 conn(0x55b9947cd460 msgr2=0x55b994b7e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.769+0000 7f8cc3f36740 1 -- 172.21.15.3:0/706579710 shutdown_connections 2024-09-21T06:18:05.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.769+0000 7f8cc3f36740 1 -- 172.21.15.3:0/706579710 wait complete. 2024-09-21T06:18:05.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.769+0000 7f8cc3f36740 1 -- start start 2024-09-21T06:18:05.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.770+0000 7f8cc3f36740 1 Processor -- start 2024-09-21T06:18:05.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.770+0000 7f8cc3f36740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 0x55b994b8fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.770+0000 7f8cc3f36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b8390 0x55b994b90290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.770+0000 7f8cc3f36740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947d9070 0x55b994b9c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.770+0000 7f8cc3f36740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b994947700 con 0x55b9947d9070 2024-09-21T06:18:05.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.770+0000 7f8cc3f36740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b994a81160 con 0x55b9947b01b0 2024-09-21T06:18:05.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.770+0000 7f8cc3f36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b994b92dd0 con 0x55b9947b8390 2024-09-21T06:18:05.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.770+0000 7f8cc3f36740 10 client.0 _ll_register_callbacks cb 0x55b994a9a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:05.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.770+0000 7f8cb77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947d9070 0x55b994b9c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.770+0000 7f8cb77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 0x55b994b8fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.770+0000 7f8cb77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 0x55b994b8fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:34634/0 (socket says 172.21.15.3:34634) 2024-09-21T06:18:05.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.770+0000 7f8cb77fe640 1 -- 172.21.15.3:0/1450448735 learned_addr learned my addr 172.21.15.3:0/1450448735 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:05.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.770+0000 7f8cb67fc640 1 --2- 172.21.15.3:0/1450448735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b8390 0x55b994b90290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8cb77fe640 1 -- 172.21.15.3:0/1450448735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b8390 msgr2=0x55b994b90290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/1450448735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b8390 0x55b994b90290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8cb77fe640 1 -- 172.21.15.3:0/1450448735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947d9070 msgr2=0x55b994b9c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/1450448735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947d9070 0x55b994b9c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8cb77fe640 1 -- 172.21.15.3:0/1450448735 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b994aaec90 con 0x55b9947b01b0 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/1450448735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 0x55b994b8fd50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0011680 tx=0x7f8cb0000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b7326a11c27f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8c96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8c96ffd640 1 -- 172.21.15.3:0/1450448735 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0023040 con 0x55b9947b01b0 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8cc3f36740 1 -- 172.21.15.3:0/1450448735 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb0015070 con 0x55b9947b01b0 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8c96ffd640 1 -- 172.21.15.3:0/1450448735 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8cb0004c40 con 0x55b9947b01b0 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8c96ffd640 1 -- 172.21.15.3:0/1450448735 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8cb0011e50 con 0x55b9947b01b0 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8cc3f36740 10 client.26057 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8cc3f36740 1 -- 172.21.15.3:0/1450448735 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b994b9ea50 con 0x55b9947b01b0 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.771+0000 7f8c96ffd640 1 -- 172.21.15.3:0/1450448735 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb0015070 con 0x55b9947b01b0 2024-09-21T06:18:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.772+0000 7f8c96ffd640 1 -- 172.21.15.3:0/1450448735 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8cb0005280 con 0x55b9947b01b0 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.772+0000 7f8cc3f36740 10 client.26057 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.772+0000 7f8cc3f36740 10 client.26057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.772+0000 7f8cc3f36740 20 client.26057 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.772+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/1450448735 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b9947979f0 0x55b994ba1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.772+0000 7f8cc3f36740 4 client.26057 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.772+0000 7f8cc3f36740 1 -- 172.21.15.3:0/1450448735 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b994af8df0 con 0x55b9947979f0 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.772+0000 7f8cb67fc640 1 --2- 172.21.15.3:0/1450448735 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b9947979f0 0x55b994ba1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.772+0000 7f8c96ffd640 10 client.26057 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.772+0000 7f8cb67fc640 1 --2- 172.21.15.3:0/1450448735 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b9947979f0 0x55b994ba1e10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bc5ea814e366c13 server_cookie=1ce8e421880f757d in_seq=0 out_seq=0 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.773+0000 7f8c96ffd640 1 -- 172.21.15.3:0/1450448735 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b994af8df0 con 0x55b9947979f0 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.773+0000 7f8c96ffd640 10 client.26057 handle_command_reply: tid=0 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.773+0000 7f8cc3f36740 1 client.26057 shutdown 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.773+0000 7f8cc3f36740 1 -- 172.21.15.3:0/1450448735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 msgr2=0x55b994b8fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:05.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.773+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/1450448735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 0x55b994b8fd50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0011680 tx=0x7f8cb0000f40 comp rx=0 tx=0).stop 2024-09-21T06:18:05.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.773+0000 7f8cc3f36740 1 -- 172.21.15.3:0/1450448735 shutdown_connections 2024-09-21T06:18:05.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.773+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/1450448735 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b9947979f0 0x55b994ba1e10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.773+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/1450448735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b8390 0x55b994b90290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.773+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/1450448735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947d9070 0x55b994b9c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.773+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/1450448735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 0x55b994b8fd50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.773+0000 7f8c96ffd640 0 client.26057 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:05.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.773+0000 7f8cc3f36740 1 -- 172.21.15.3:0/1450448735 >> 172.21.15.3:0/1450448735 conn(0x55b9947cd460 msgr2=0x55b994b83450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:05.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.774+0000 7f8cc3f36740 1 -- 172.21.15.3:0/1450448735 shutdown_connections 2024-09-21T06:18:05.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.774+0000 7f8cc3f36740 1 -- 172.21.15.3:0/1450448735 wait complete. 2024-09-21T06:18:05.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.774+0000 7f8cc3f36740 20 client.26057 trim_cache size 0 max 16384 2024-09-21T06:18:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.792+0000 7f8cc3f36740 1 Processor -- start 2024-09-21T06:18:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.792+0000 7f8cc3f36740 1 -- start start 2024-09-21T06:18:05.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.792+0000 7f8cc3f36740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 0x55b994b90210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cc3f36740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947b8390 0x55b994b7d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cc3f36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947d9070 0x55b994b8a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cc3f36740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81ad570 con 0x55b9947b8390 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cc3f36740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81af110 con 0x55b9947b01b0 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cc3f36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81adc40 con 0x55b9947d9070 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cb77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 0x55b994b90210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cb77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 0x55b994b90210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:34638/0 (socket says 172.21.15.3:34638) 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cb77fe640 1 -- 172.21.15.3:0/340545512 learned_addr learned my addr 172.21.15.3:0/340545512 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cb67fc640 1 --2- 172.21.15.3:0/340545512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947b8390 0x55b994b7d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/340545512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947d9070 0x55b994b8a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cb77fe640 1 -- 172.21.15.3:0/340545512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947d9070 msgr2=0x55b994b8a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/340545512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947d9070 0x55b994b8a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cb77fe640 1 -- 172.21.15.3:0/340545512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947b8390 msgr2=0x55b994b7d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/340545512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947b8390 0x55b994b7d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cb77fe640 1 -- 172.21.15.3:0/340545512 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b994b830c0 con 0x55b9947b01b0 2024-09-21T06:18:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.793+0000 7f8cb67fc640 1 --2- 172.21.15.3:0/340545512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947b8390 0x55b994b7d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:05.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.794+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/340545512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 0x55b994b90210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00098d0 tx=0x7f8cb001c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceee265bae7b4db2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:05.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.794+0000 7f8c9d7fa640 1 -- 172.21.15.3:0/340545512 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0005820 con 0x55b9947b01b0 2024-09-21T06:18:05.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.794+0000 7f8c9d7fa640 1 -- 172.21.15.3:0/340545512 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb003a070 con 0x55b9947b01b0 2024-09-21T06:18:05.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.794+0000 7f8c9d7fa640 1 -- 172.21.15.3:0/340545512 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb002cd00 con 0x55b9947b01b0 2024-09-21T06:18:05.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.795+0000 7f8cc3f36740 1 -- 172.21.15.3:0/340545512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 msgr2=0x55b994b90210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:05.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.795+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/340545512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 0x55b994b90210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00098d0 tx=0x7f8cb001c040 comp rx=0 tx=0).stop 2024-09-21T06:18:05.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.795+0000 7f8cc3f36740 1 -- 172.21.15.3:0/340545512 shutdown_connections 2024-09-21T06:18:05.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.795+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/340545512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947d9070 0x55b994b8a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.795+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/340545512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947b8390 0x55b994b7d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.795+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/340545512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947b01b0 0x55b994b90210 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.795+0000 7f8cc3f36740 1 -- 172.21.15.3:0/340545512 >> 172.21.15.3:0/340545512 conn(0x55b9947cd460 msgr2=0x55b994b8fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:05.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.796+0000 7f8cc3f36740 1 -- 172.21.15.3:0/340545512 shutdown_connections 2024-09-21T06:18:05.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.796+0000 7f8cc3f36740 1 -- 172.21.15.3:0/340545512 wait complete. 2024-09-21T06:18:05.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.796+0000 7f8cc3f36740 1 -- start start 2024-09-21T06:18:05.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.796+0000 7f8cc3f36740 1 Processor -- start 2024-09-21T06:18:05.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cc3f36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b01b0 0x55b994b98320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cc3f36740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947b8390 0x55b994b988e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cc3f36740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947d9070 0x55b994b87790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cc3f36740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81ae9e0 con 0x55b9947b8390 2024-09-21T06:18:05.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cc3f36740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81ae310 con 0x55b9947d9070 2024-09-21T06:18:05.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cc3f36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81ab550 con 0x55b9947b01b0 2024-09-21T06:18:05.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cc3f36740 10 client.0 _ll_register_callbacks cb 0x7f8cac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:05.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cb77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b01b0 0x55b994b98320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cb67fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947b8390 0x55b994b988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cb77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b01b0 0x55b994b98320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:59792/0 (socket says 172.21.15.3:59792) 2024-09-21T06:18:05.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cb77fe640 1 -- 172.21.15.3:0/2821107075 learned_addr learned my addr 172.21.15.3:0/2821107075 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cb67fc640 1 --2- 172.21.15.3:0/2821107075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947b8390 0x55b994b988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41204/0 (socket says 172.21.15.3:41204) 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/2821107075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947d9070 0x55b994b87790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cb77fe640 1 -- 172.21.15.3:0/2821107075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947d9070 msgr2=0x55b994b87790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/2821107075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947d9070 0x55b994b87790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cb77fe640 1 -- 172.21.15.3:0/2821107075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947b8390 msgr2=0x55b994b988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/2821107075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947b8390 0x55b994b988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.797+0000 7f8cb77fe640 1 -- 172.21.15.3:0/2821107075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b994b87f20 con 0x55b9947b01b0 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.798+0000 7f8cb77fe640 1 --2- 172.21.15.3:0/2821107075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b01b0 0x55b994b98320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009df0 tx=0x7f8cb00115d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faf1f48b5a17e65e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.798+0000 7f8c96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.798+0000 7f8c96ffd640 1 -- 172.21.15.3:0/2821107075 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb002c200 con 0x55b9947b01b0 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.798+0000 7f8c96ffd640 1 -- 172.21.15.3:0/2821107075 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8cb002c5f0 con 0x55b9947b01b0 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.798+0000 7f8cc3f36740 1 -- 172.21.15.3:0/2821107075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb003a070 con 0x55b9947b01b0 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.798+0000 7f8c96ffd640 1 -- 172.21.15.3:0/2821107075 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8cb002c7f0 con 0x55b9947b01b0 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.798+0000 7f8cc3f36740 10 client.26026 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.798+0000 7f8cc3f36740 1 -- 172.21.15.3:0/2821107075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b994b94c90 con 0x55b9947b01b0 2024-09-21T06:18:05.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.798+0000 7f8c96ffd640 1 -- 172.21.15.3:0/2821107075 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb003a070 con 0x55b9947b01b0 2024-09-21T06:18:05.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.798+0000 7f8c96ffd640 1 -- 172.21.15.3:0/2821107075 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8cb001a8c0 con 0x55b9947b01b0 2024-09-21T06:18:05.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.799+0000 7f8cc3f36740 10 client.26026 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:05.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.799+0000 7f8cc3f36740 10 client.26026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:05.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.799+0000 7f8cc3f36740 20 client.26026 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:05.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.799+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/2821107075 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b99476cda0 0x55b994c131a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:05.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.799+0000 7f8cc3f36740 4 client.26026 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:18:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.799+0000 7f8cc3f36740 1 -- 172.21.15.3:0/2821107075 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b994af48a0 con 0x55b99476cda0 2024-09-21T06:18:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.799+0000 7f8cb67fc640 1 --2- 172.21.15.3:0/2821107075 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b99476cda0 0x55b994c131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.799+0000 7f8c96ffd640 10 client.26026 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.799+0000 7f8cb67fc640 1 --2- 172.21.15.3:0/2821107075 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b99476cda0 0x55b994c131a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7972b1984c0a2467 server_cookie=64d65fe8d90d6262 in_seq=0 out_seq=0 2024-09-21T06:18:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.799+0000 7f8c96ffd640 1 -- 172.21.15.3:0/2821107075 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b994af48a0 con 0x55b99476cda0 2024-09-21T06:18:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8c96ffd640 10 client.26026 handle_command_reply: tid=0 2024-09-21T06:18:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8cc3f36740 1 client.26026 shutdown 2024-09-21T06:18:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8cc3f36740 1 -- 172.21.15.3:0/2821107075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b01b0 msgr2=0x55b994b98320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/2821107075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b01b0 0x55b994b98320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009df0 tx=0x7f8cb00115d0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8cc3f36740 1 -- 172.21.15.3:0/2821107075 shutdown_connections 2024-09-21T06:18:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/2821107075 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b99476cda0 0x55b994c131a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/2821107075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9947d9070 0x55b994b87790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/2821107075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b9947b8390 0x55b994b988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8cc3f36740 1 --2- 172.21.15.3:0/2821107075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9947b01b0 0x55b994b98320 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8cc3f36740 1 -- 172.21.15.3:0/2821107075 >> 172.21.15.3:0/2821107075 conn(0x55b9947cd460 msgr2=0x55b994b8e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:05.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8c96ffd640 0 client.26026 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:05.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8cc3f36740 1 -- 172.21.15.3:0/2821107075 shutdown_connections 2024-09-21T06:18:05.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8cc3f36740 1 -- 172.21.15.3:0/2821107075 wait complete. 2024-09-21T06:18:05.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.800+0000 7f8cc3f36740 20 client.26026 trim_cache size 0 max 16384 2024-09-21T06:18:05.805 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:18:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.801+0000 7f8cbd9c2640 1 -- 172.21.15.3:0/4250816794 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8ca00b6680 msgr2=0x7f8ca00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.801+0000 7f8cbd9c2640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8ca00b6680 0x7f8ca00b8b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80acfa0 tx=0x7f8ca80054e0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.801+0000 7f8cbd9c2640 1 -- 172.21.15.3:0/4250816794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cb8102240 msgr2=0x7f8cb819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.801+0000 7f8cbd9c2640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cb8102240 0x7f8cb819a550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800ea90 tx=0x7f8ca800ef50 comp rx=0 tx=0).stop 2024-09-21T06:18:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.801+0000 7f8cbd9c2640 1 -- 172.21.15.3:0/4250816794 shutdown_connections 2024-09-21T06:18:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.801+0000 7f8cbd9c2640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8ca00b6680 0x7f8ca00b8b90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.801+0000 7f8cbd9c2640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cb81084d0 0x7f8cb819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.801+0000 7f8cbd9c2640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cb8107b80 0x7f8cb819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.801+0000 7f8cbd9c2640 1 --2- 172.21.15.3:0/4250816794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cb8102240 0x7f8cb819a550 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.801+0000 7f8cbd9c2640 1 -- 172.21.15.3:0/4250816794 >> 172.21.15.3:0/4250816794 conn(0x7f8cb8079f90 msgr2=0x7f8cb8103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.801+0000 7f8cbd9c2640 1 -- 172.21.15.3:0/4250816794 shutdown_connections 2024-09-21T06:18:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:05.801+0000 7f8cbd9c2640 1 -- 172.21.15.3:0/4250816794 wait complete. 2024-09-21T06:18:05.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:05 smithi137 ceph-mon[34464]: pgmap v417: 97 pgs: 97 active+clean; 53 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 385 op/s 2024-09-21T06:18:05.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:05 smithi204 ceph-mon[34659]: pgmap v417: 97 pgs: 97 active+clean; 53 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 385 op/s 2024-09-21T06:18:05.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 243848 13.56 MB/sec warmup 98 sec latency 83.633 ms 2024-09-21T06:18:06.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 245694 13.55 MB/sec warmup 99 sec latency 29.403 ms 2024-09-21T06:18:07.592 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:18:07.593 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:18:07.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:07 smithi003 ceph-mon[31227]: pgmap v418: 97 pgs: 97 active+clean; 54 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.8 MiB/s wr, 435 op/s 2024-09-21T06:18:07.856 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:07.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:07 smithi137 ceph-mon[34464]: pgmap v418: 97 pgs: 97 active+clean; 54 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.8 MiB/s wr, 435 op/s 2024-09-21T06:18:07.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:07 smithi204 ceph-mon[34659]: pgmap v418: 97 pgs: 97 active+clean; 54 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.8 MiB/s wr, 435 op/s 2024-09-21T06:18:07.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 248657 13.59 MB/sec warmup 100 sec latency 9.944 ms 2024-09-21T06:18:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.380+0000 7f4fada68640 1 -- 172.21.15.3:0/3657345704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa8108cf0 msgr2=0x7f4fa810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.380+0000 7f4fada68640 1 --2- 172.21.15.3:0/3657345704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa8108cf0 0x7f4fa810f520 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009be0 tx=0x7f4fa002a6c0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.380+0000 7f4fada68640 1 -- 172.21.15.3:0/3657345704 shutdown_connections 2024-09-21T06:18:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.380+0000 7f4fada68640 1 --2- 172.21.15.3:0/3657345704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa8108cf0 0x7f4fa810f520 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.380+0000 7f4fada68640 1 --2- 172.21.15.3:0/3657345704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa81083a0 0x7f4fa81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.380+0000 7f4fada68640 1 --2- 172.21.15.3:0/3657345704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa8102a60 0x7f4fa8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.380+0000 7f4fada68640 1 -- 172.21.15.3:0/3657345704 >> 172.21.15.3:0/3657345704 conn(0x7f4fa8079730 msgr2=0x7f4fa8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.380+0000 7f4fada68640 1 -- 172.21.15.3:0/3657345704 shutdown_connections 2024-09-21T06:18:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.380+0000 7f4fada68640 1 -- 172.21.15.3:0/3657345704 wait complete. 2024-09-21T06:18:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.381+0000 7f4fada68640 1 Processor -- start 2024-09-21T06:18:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.381+0000 7f4fada68640 1 -- start start 2024-09-21T06:18:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fada68640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa8102a60 0x7f4fa819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fada68640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa81083a0 0x7f4fa819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fada68640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa8108cf0 0x7f4fa819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fada68640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8112960 con 0x7f4fa81083a0 2024-09-21T06:18:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fada68640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81127e0 con 0x7f4fa8102a60 2024-09-21T06:18:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fada68640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8112ae0 con 0x7f4fa8108cf0 2024-09-21T06:18:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fa77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa8102a60 0x7f4fa819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fa6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa81083a0 0x7f4fa819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fa77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa8102a60 0x7f4fa819b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:34650/0 (socket says 172.21.15.3:34650) 2024-09-21T06:18:08.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fa77fe640 1 -- 172.21.15.3:0/10088991 learned_addr learned my addr 172.21.15.3:0/10088991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:08.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fa7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa8108cf0 0x7f4fa819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fa77fe640 1 -- 172.21.15.3:0/10088991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa8108cf0 msgr2=0x7f4fa819bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:08.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fa77fe640 1 --2- 172.21.15.3:0/10088991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa8108cf0 0x7f4fa819bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fa77fe640 1 -- 172.21.15.3:0/10088991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa81083a0 msgr2=0x7f4fa819b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:08.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fa77fe640 1 --2- 172.21.15.3:0/10088991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa81083a0 0x7f4fa819b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.382+0000 7f4fa77fe640 1 -- 172.21.15.3:0/10088991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa81a6520 con 0x7f4fa8102a60 2024-09-21T06:18:08.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.383+0000 7f4fa77fe640 1 --2- 172.21.15.3:0/10088991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa8102a60 0x7f4fa819b290 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009c40 tx=0x7f4f9800b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:08.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.383+0000 7f4fa4ff9640 1 -- 172.21.15.3:0/10088991 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98005020 con 0x7f4fa8102a60 2024-09-21T06:18:08.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.383+0000 7f4fada68640 1 -- 172.21.15.3:0/10088991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa81990e0 con 0x7f4fa8102a60 2024-09-21T06:18:08.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.383+0000 7f4fa4ff9640 1 -- 172.21.15.3:0/10088991 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f98005670 con 0x7f4fa8102a60 2024-09-21T06:18:08.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.383+0000 7f4fada68640 1 -- 172.21.15.3:0/10088991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa8199620 con 0x7f4fa8102a60 2024-09-21T06:18:08.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.383+0000 7f4fa4ff9640 1 -- 172.21.15.3:0/10088991 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98019670 con 0x7f4fa8102a60 2024-09-21T06:18:08.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.386+0000 7f4fa4ff9640 1 -- 172.21.15.3:0/10088991 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f4f98019810 con 0x7f4fa8102a60 2024-09-21T06:18:08.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.386+0000 7f4fb3fdc740 1 Processor -- start 2024-09-21T06:18:08.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.387+0000 7f4fb3fdc740 1 -- start start 2024-09-21T06:18:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.387+0000 7f4fb3fdc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484a4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.387+0000 7f4fb3fdc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480d4490 0x562d484a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.387+0000 7f4fb3fdc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480dc670 0x562d484a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.387+0000 7f4fb3fdc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562d4841ae60 con 0x562d480dc670 2024-09-21T06:18:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.387+0000 7f4fb3fdc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d4840a290 con 0x562d480fd350 2024-09-21T06:18:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.387+0000 7f4fb3fdc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562d4840e0a0 con 0x562d480d4490 2024-09-21T06:18:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.387+0000 7f4fa77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480d4490 0x562d484a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.387+0000 7f4fa6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480dc670 0x562d484a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.387+0000 7f4fa7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.387+0000 7f4fa6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480dc670 0x562d484a4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41224/0 (socket says 172.21.15.3:41224) 2024-09-21T06:18:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.387+0000 7f4fa6ffd640 1 -- 172.21.15.3:0/162501381 learned_addr learned my addr 172.21.15.3:0/162501381 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.388+0000 7f4fa7fff640 1 -- 172.21.15.3:0/162501381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480d4490 msgr2=0x562d484a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.388+0000 7f4fa7fff640 1 --2- 172.21.15.3:0/162501381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480d4490 0x562d484a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.388+0000 7f4fa7fff640 1 -- 172.21.15.3:0/162501381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480dc670 msgr2=0x562d484a4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.388+0000 7f4fa7fff640 1 --2- 172.21.15.3:0/162501381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480dc670 0x562d484a4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.388+0000 7f4fa7fff640 1 -- 172.21.15.3:0/162501381 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d48168930 con 0x562d480fd350 2024-09-21T06:18:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.388+0000 7f4fa6ffd640 1 --2- 172.21.15.3:0/162501381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480dc670 0x562d484a4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.388+0000 7f4fa4ff9640 1 --2- 172.21.15.3:0/10088991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f900b66b0 0x7f4f900b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.388+0000 7f4fa7fff640 1 --2- 172.21.15.3:0/10088991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f900b66b0 0x7f4f900b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.388+0000 7f4fa4ff9640 1 -- 172.21.15.3:0/10088991 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f4f980eae10 con 0x7f4fa8102a60 2024-09-21T06:18:08.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.389+0000 7f4fa7fff640 1 --2- 172.21.15.3:0/162501381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484a4430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4fa002abe0 tx=0x7f4fa0030420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ab86750f62eaa40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:08.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.389+0000 7f4f8dffb640 1 -- 172.21.15.3:0/162501381 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0041040 con 0x562d480fd350 2024-09-21T06:18:08.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.389+0000 7f4f8dffb640 1 -- 172.21.15.3:0/162501381 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fa0030e20 con 0x562d480fd350 2024-09-21T06:18:08.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.389+0000 7f4f8dffb640 1 -- 172.21.15.3:0/162501381 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0036040 con 0x562d480fd350 2024-09-21T06:18:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.390+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/162501381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 msgr2=0x562d484a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.390+0000 7f4fa7fff640 1 --2- 172.21.15.3:0/10088991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f900b66b0 0x7f4f900b8bc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x562d484b2860 tx=0x7f4fa003d460 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.390+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/162501381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484a4430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4fa002abe0 tx=0x7f4fa0030420 comp rx=0 tx=0).stop 2024-09-21T06:18:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.390+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/162501381 shutdown_connections 2024-09-21T06:18:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.390+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/162501381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480d4490 0x562d484a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.390+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/162501381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480dc670 0x562d484a4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.390+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/162501381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484a4430 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.390+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/162501381 >> 172.21.15.3:0/162501381 conn(0x562d480f1740 msgr2=0x562d484a2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.390+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/162501381 shutdown_connections 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.390+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/162501381 wait complete. 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.390+0000 7f4fb3fdc740 1 -- start start 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fb3fdc740 1 Processor -- start 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fb3fdc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484b0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fb3fdc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480dc670 0x562d484b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fb3fdc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480fd350 0x562d484bbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fb3fdc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562d4826b890 con 0x562d480d4490 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fb3fdc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d483a4d40 con 0x562d480dc670 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fb3fdc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562d484b2c10 con 0x562d480fd350 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fb3fdc740 10 client.0 _ll_register_callbacks cb 0x562d48408c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fa77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fa6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480dc670 0x562d484b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fa7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480fd350 0x562d484bbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fa77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484b0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41238/0 (socket says 172.21.15.3:41238) 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fa6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480dc670 0x562d484b0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:34662/0 (socket says 172.21.15.3:34662) 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.391+0000 7f4fa77fe640 1 -- 172.21.15.3:0/1651776442 learned_addr learned my addr 172.21.15.3:0/1651776442 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4fa77fe640 1 -- 172.21.15.3:0/1651776442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480fd350 msgr2=0x562d484bbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4fa77fe640 1 --2- 172.21.15.3:0/1651776442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480fd350 0x562d484bbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4fa77fe640 1 -- 172.21.15.3:0/1651776442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480dc670 msgr2=0x562d484b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4fa77fe640 1 --2- 172.21.15.3:0/1651776442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480dc670 0x562d484b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4fa77fe640 1 -- 172.21.15.3:0/1651776442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d484bc470 con 0x562d480d4490 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4fa7fff640 1 --2- 172.21.15.3:0/1651776442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480fd350 0x562d484bbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4fa6ffd640 1 --2- 172.21.15.3:0/1651776442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480dc670 0x562d484b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4fa77fe640 1 --2- 172.21.15.3:0/1651776442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484b0010 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4f980e8600 tx=0x7f4f980ee550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97060b96bfb0051b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4f86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4f86ffd640 1 -- 172.21.15.3:0/1651776442 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f980ad3a0 con 0x562d480d4490 2024-09-21T06:18:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4f86ffd640 1 -- 172.21.15.3:0/1651776442 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f4f980affb0 con 0x562d480d4490 2024-09-21T06:18:08.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1651776442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fa0030e20 con 0x562d480d4490 2024-09-21T06:18:08.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4f86ffd640 1 -- 172.21.15.3:0/1651776442 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f980b04c0 con 0x562d480d4490 2024-09-21T06:18:08.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4fb3fdc740 10 client.16500 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:08.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.392+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1651776442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562d484beda0 con 0x562d480d4490 2024-09-21T06:18:08.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.393+0000 7f4f86ffd640 1 -- 172.21.15.3:0/1651776442 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fa0030e20 con 0x562d480d4490 2024-09-21T06:18:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.393+0000 7f4f86ffd640 1 -- 172.21.15.3:0/1651776442 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4f980eeba0 con 0x562d480d4490 2024-09-21T06:18:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.393+0000 7f4fb3fdc740 10 client.16500 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.393+0000 7f4fb3fdc740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.393+0000 7f4fb3fdc740 20 client.16500 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.393+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/1651776442 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562d480bbcd0 0x562d484c2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.393+0000 7f4fb3fdc740 4 client.16500 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.393+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1651776442 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d4841cdb0 con 0x562d480bbcd0 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.393+0000 7f4fa6ffd640 1 --2- 172.21.15.3:0/1651776442 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562d480bbcd0 0x562d484c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.394+0000 7f4fa6ffd640 1 --2- 172.21.15.3:0/1651776442 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562d480bbcd0 0x562d484c2270 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d0c2b603fb75b41 server_cookie=ff57acc42cc4565b in_seq=0 out_seq=0 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.394+0000 7f4f86ffd640 10 client.16500 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4f86ffd640 1 -- 172.21.15.3:0/1651776442 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d4841cdb0 con 0x562d480bbcd0 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4f86ffd640 10 client.16500 handle_command_reply: tid=0 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4fb3fdc740 1 client.16500 shutdown 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1651776442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 msgr2=0x562d484b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/1651776442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484b0010 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4f980e8600 tx=0x7f4f980ee550 comp rx=0 tx=0).stop 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1651776442 shutdown_connections 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/1651776442 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562d480bbcd0 0x562d484c2270 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/1651776442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480fd350 0x562d484bbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/1651776442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480dc670 0x562d484b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/1651776442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484b0010 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4f86ffd640 0 client.16500 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1651776442 >> 172.21.15.3:0/1651776442 conn(0x562d480f1740 msgr2=0x562d484a75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1651776442 shutdown_connections 2024-09-21T06:18:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1651776442 wait complete. 2024-09-21T06:18:08.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.395+0000 7f4fb3fdc740 20 client.16500 trim_cache size 0 max 16384 2024-09-21T06:18:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.414+0000 7f4fb3fdc740 1 Processor -- start 2024-09-21T06:18:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fb3fdc740 1 -- start start 2024-09-21T06:18:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fb3fdc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fb3fdc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480dc670 0x562d484aa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fb3fdc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fb3fdc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9400e650 con 0x562d480d4490 2024-09-21T06:18:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fb3fdc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4f940101f0 con 0x562d480fd350 2024-09-21T06:18:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fb3fdc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9400ed20 con 0x562d480dc670 2024-09-21T06:18:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fa6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480dc670 0x562d484aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fa6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480dc670 0x562d484aa550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:59826/0 (socket says 172.21.15.3:59826) 2024-09-21T06:18:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fa6ffd640 1 -- 172.21.15.3:0/2352616437 learned_addr learned my addr 172.21.15.3:0/2352616437 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fa77fe640 1 --2- 172.21.15.3:0/2352616437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fa7fff640 1 --2- 172.21.15.3:0/2352616437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fa6ffd640 1 -- 172.21.15.3:0/2352616437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 msgr2=0x562d484ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fa6ffd640 1 --2- 172.21.15.3:0/2352616437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fa6ffd640 1 -- 172.21.15.3:0/2352616437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 msgr2=0x562d484bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fa6ffd640 1 --2- 172.21.15.3:0/2352616437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484bbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.415+0000 7f4fa6ffd640 1 -- 172.21.15.3:0/2352616437 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4849c520 con 0x562d480dc670 2024-09-21T06:18:08.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.416+0000 7f4fa6ffd640 1 --2- 172.21.15.3:0/2352616437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480dc670 0x562d484aa550 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x562d484c2a10 tx=0x7f4f9c009d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1f33ba847d8e9df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:08.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.417+0000 7f4f8d7fa640 1 -- 172.21.15.3:0/2352616437 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c004220 con 0x562d480dc670 2024-09-21T06:18:08.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.417+0000 7f4f8d7fa640 1 -- 172.21.15.3:0/2352616437 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f9c0043c0 con 0x562d480dc670 2024-09-21T06:18:08.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.417+0000 7f4f8d7fa640 1 -- 172.21.15.3:0/2352616437 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c010950 con 0x562d480dc670 2024-09-21T06:18:08.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.418+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/2352616437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480dc670 msgr2=0x562d484aa550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:08.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.418+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/2352616437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480dc670 0x562d484aa550 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x562d484c2a10 tx=0x7f4f9c009d50 comp rx=0 tx=0).stop 2024-09-21T06:18:08.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.418+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/2352616437 shutdown_connections 2024-09-21T06:18:08.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.418+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/2352616437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480dc670 0x562d484aa550 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.418+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/2352616437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.418+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/2352616437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.418+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/2352616437 >> 172.21.15.3:0/2352616437 conn(0x562d480f1740 msgr2=0x562d484ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:08.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.418+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/2352616437 shutdown_connections 2024-09-21T06:18:08.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.418+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/2352616437 wait complete. 2024-09-21T06:18:08.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.419+0000 7f4fb3fdc740 1 -- start start 2024-09-21T06:18:08.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.419+0000 7f4fb3fdc740 1 Processor -- start 2024-09-21T06:18:08.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fb3fdc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fb3fdc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480dc670 0x562d484b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fb3fdc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fb3fdc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9400fac0 con 0x562d480d4490 2024-09-21T06:18:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fb3fdc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9400f3f0 con 0x562d480fd350 2024-09-21T06:18:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fb3fdc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9400c630 con 0x562d480dc670 2024-09-21T06:18:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fa77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fa7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fa77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41268/0 (socket says 172.21.15.3:41268) 2024-09-21T06:18:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fa77fe640 1 -- 172.21.15.3:0/1173892438 learned_addr learned my addr 172.21.15.3:0/1173892438 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:08.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fa6ffd640 1 --2- 172.21.15.3:0/1173892438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480dc670 0x562d484b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fb3fdc740 10 client.0 _ll_register_callbacks cb 0x7f4f980e8a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fa7fff640 1 -- 172.21.15.3:0/1173892438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480dc670 msgr2=0x562d484b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fa7fff640 1 --2- 172.21.15.3:0/1173892438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480dc670 0x562d484b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fa7fff640 1 -- 172.21.15.3:0/1173892438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 msgr2=0x562d484b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fa7fff640 1 --2- 172.21.15.3:0/1173892438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.420+0000 7f4fa7fff640 1 -- 172.21.15.3:0/1173892438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4852bcf0 con 0x562d480fd350 2024-09-21T06:18:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.421+0000 7f4fa7fff640 1 --2- 172.21.15.3:0/1173892438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484b49b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0034040 tx=0x7f4fa004e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a56dd6580671b451 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.421+0000 7f4f86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:18:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.421+0000 7f4f86ffd640 1 -- 172.21.15.3:0/1173892438 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0036040 con 0x562d480fd350 2024-09-21T06:18:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.421+0000 7f4f86ffd640 1 -- 172.21.15.3:0/1173892438 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f4fa004e070 con 0x562d480fd350 2024-09-21T06:18:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.421+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1173892438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f9c0043c0 con 0x562d480fd350 2024-09-21T06:18:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.421+0000 7f4f86ffd640 1 -- 172.21.15.3:0/1173892438 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4fa0009800 con 0x562d480fd350 2024-09-21T06:18:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.421+0000 7f4fb3fdc740 10 client.26081 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.421+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1173892438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562d4852cf50 con 0x562d480fd350 2024-09-21T06:18:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.421+0000 7f4f86ffd640 1 -- 172.21.15.3:0/1173892438 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f9c0043c0 con 0x562d480fd350 2024-09-21T06:18:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.421+0000 7f4f86ffd640 1 -- 172.21.15.3:0/1173892438 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4fa005eaa0 con 0x562d480fd350 2024-09-21T06:18:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.422+0000 7f4fb3fdc740 10 client.26081 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.422+0000 7f4fb3fdc740 10 client.26081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.422+0000 7f4fb3fdc740 20 client.26081 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.422+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/1173892438 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562d48091080 0x562d4852f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.422+0000 7f4fb3fdc740 4 client.26081 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:18:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.422+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1173892438 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562d48418860 con 0x562d48091080 2024-09-21T06:18:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.422+0000 7f4fa77fe640 1 --2- 172.21.15.3:0/1173892438 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562d48091080 0x562d4852f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.422+0000 7f4f86ffd640 10 client.26081 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.422+0000 7f4fa77fe640 1 --2- 172.21.15.3:0/1173892438 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562d48091080 0x562d4852f370 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e2c993f6b5121a4 server_cookie=debf58f3cdd80781 in_seq=0 out_seq=0 2024-09-21T06:18:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.426+0000 7f4f86ffd640 1 -- 172.21.15.3:0/1173892438 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562d48418860 con 0x562d48091080 2024-09-21T06:18:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.426+0000 7f4f86ffd640 10 client.26081 handle_command_reply: tid=0 2024-09-21T06:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.426+0000 7f4fb3fdc740 1 client.26081 shutdown 2024-09-21T06:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.426+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1173892438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 msgr2=0x562d484b49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.426+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/1173892438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484b49b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0034040 tx=0x7f4fa004e000 comp rx=0 tx=0).stop 2024-09-21T06:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.426+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1173892438 shutdown_connections 2024-09-21T06:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.426+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/1173892438 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562d48091080 0x562d4852f370 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.426+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/1173892438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d480dc670 0x562d484b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.426+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/1173892438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d480fd350 0x562d484b49b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.426+0000 7f4fb3fdc740 1 --2- 172.21.15.3:0/1173892438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562d480d4490 0x562d484b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.426+0000 7f4f86ffd640 0 client.26081 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1173892438 >> 172.21.15.3:0/1173892438 conn(0x562d480f1740 msgr2=0x562d484a5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1173892438 shutdown_connections 2024-09-21T06:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fb3fdc740 1 -- 172.21.15.3:0/1173892438 wait complete. 2024-09-21T06:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fb3fdc740 20 client.26081 trim_cache size 0 max 16384 2024-09-21T06:18:08.430 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:18:08.430 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:18:08.430 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "adba1d94-e33f-4efd-a020-63ebe07f6992", 2024-09-21T06:18:08.430 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:18:08.430 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:18:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fada68640 1 -- 172.21.15.3:0/10088991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f900b66b0 msgr2=0x7f4f900b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fada68640 1 --2- 172.21.15.3:0/10088991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f900b66b0 0x7f4f900b8bc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x562d484b2860 tx=0x7f4fa003d460 comp rx=0 tx=0).stop 2024-09-21T06:18:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fada68640 1 -- 172.21.15.3:0/10088991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa8102a60 msgr2=0x7f4fa819b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fada68640 1 --2- 172.21.15.3:0/10088991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa8102a60 0x7f4fa819b290 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009c40 tx=0x7f4f9800b690 comp rx=0 tx=0).stop 2024-09-21T06:18:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fada68640 1 -- 172.21.15.3:0/10088991 shutdown_connections 2024-09-21T06:18:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fada68640 1 --2- 172.21.15.3:0/10088991 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4f900b66b0 0x7f4f900b8bc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fada68640 1 --2- 172.21.15.3:0/10088991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa8108cf0 0x7f4fa819bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fada68640 1 --2- 172.21.15.3:0/10088991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fa81083a0 0x7f4fa819b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fada68640 1 --2- 172.21.15.3:0/10088991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa8102a60 0x7f4fa819b290 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.427+0000 7f4fada68640 1 -- 172.21.15.3:0/10088991 >> 172.21.15.3:0/10088991 conn(0x7f4fa8079730 msgr2=0x7f4fa8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.428+0000 7f4fada68640 1 -- 172.21.15.3:0/10088991 shutdown_connections 2024-09-21T06:18:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:08.428+0000 7f4fada68640 1 -- 172.21.15.3:0/10088991 wait complete. 2024-09-21T06:18:08.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 250783 13.53 MB/sec warmup 101 sec latency 56.857 ms 2024-09-21T06:18:09.460 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:18:09.726 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:09 smithi003 ceph-mon[31227]: pgmap v419: 97 pgs: 97 active+clean; 59 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 425 op/s 2024-09-21T06:18:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:09 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:18:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:09 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:09 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:18:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:09 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:18:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:09 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:18:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:09 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:09.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:09 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:09.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:09 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:09 smithi137 ceph-mon[34464]: pgmap v419: 97 pgs: 97 active+clean; 59 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 425 op/s 2024-09-21T06:18:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:09 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:18:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:09 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:09 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:18:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:09 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:18:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:09 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:18:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:09 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:09 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:09 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:09 smithi204 ceph-mon[34659]: pgmap v419: 97 pgs: 97 active+clean; 59 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 425 op/s 2024-09-21T06:18:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:09 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:18:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:09 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:09 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:18:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:09 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:18:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:09 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:18:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:09 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:09 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:09 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:09.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 252660 13.52 MB/sec warmup 102 sec latency 17.673 ms 2024-09-21T06:18:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.237+0000 7f5f6f510640 1 -- 172.21.15.3:0/2592181214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f6810de10 msgr2=0x7f5f68110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.237+0000 7f5f6f510640 1 --2- 172.21.15.3:0/2592181214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f6810de10 0x7f5f68110300 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009ca0 tx=0x7f5f6002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.237+0000 7f5f6f510640 1 -- 172.21.15.3:0/2592181214 shutdown_connections 2024-09-21T06:18:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.237+0000 7f5f6f510640 1 --2- 172.21.15.3:0/2592181214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f6810de10 0x7f5f68110300 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.237+0000 7f5f6f510640 1 --2- 172.21.15.3:0/2592181214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f68105300 0x7f5f6810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.237+0000 7f5f6f510640 1 --2- 172.21.15.3:0/2592181214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f681049b0 0x7f5f68104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.237+0000 7f5f6f510640 1 -- 172.21.15.3:0/2592181214 >> 172.21.15.3:0/2592181214 conn(0x7f5f68079730 msgr2=0x7f5f68079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.237+0000 7f5f6f510640 1 -- 172.21.15.3:0/2592181214 shutdown_connections 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.237+0000 7f5f6f510640 1 -- 172.21.15.3:0/2592181214 wait complete. 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.238+0000 7f5f6f510640 1 Processor -- start 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.238+0000 7f5f6f510640 1 -- start start 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.238+0000 7f5f6f510640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f681049b0 0x7f5f6819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.238+0000 7f5f6f510640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f68105300 0x7f5f6819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.238+0000 7f5f6f510640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f6810de10 0x7f5f6819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.238+0000 7f5f6f510640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68113a90 con 0x7f5f68105300 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.238+0000 7f5f6f510640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68113910 con 0x7f5f681049b0 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.238+0000 7f5f6f510640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68113c10 con 0x7f5f6810de10 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.238+0000 7f5f6caab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f68105300 0x7f5f6819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.238+0000 7f5f6d2ac640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f681049b0 0x7f5f6819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.238+0000 7f5f6caab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f68105300 0x7f5f6819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41276/0 (socket says 172.21.15.3:41276) 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.238+0000 7f5f6caab640 1 -- 172.21.15.3:0/3275196824 learned_addr learned my addr 172.21.15.3:0/3275196824 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.238+0000 7f5f6daad640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f6810de10 0x7f5f6819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.239+0000 7f5f6d2ac640 1 -- 172.21.15.3:0/3275196824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f6810de10 msgr2=0x7f5f6819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.239+0000 7f5f6d2ac640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f6810de10 0x7f5f6819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.239+0000 7f5f6d2ac640 1 -- 172.21.15.3:0/3275196824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f68105300 msgr2=0x7f5f6819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.239+0000 7f5f6d2ac640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f68105300 0x7f5f6819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.239+0000 7f5f6d2ac640 1 -- 172.21.15.3:0/3275196824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f681a03d0 con 0x7f5f681049b0 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.239+0000 7f5f6caab640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f68105300 0x7f5f6819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.239+0000 7f5f6daad640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f6810de10 0x7f5f6819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.239+0000 7f5f6d2ac640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f681049b0 0x7f5f6819a460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400e9c0 tx=0x7f5f6400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.239+0000 7f5f5e7fc640 1 -- 172.21.15.3:0/3275196824 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400ccc0 con 0x7f5f681049b0 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.239+0000 7f5f6f510640 1 -- 172.21.15.3:0/3275196824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f681a06c0 con 0x7f5f681049b0 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.239+0000 7f5f5e7fc640 1 -- 172.21.15.3:0/3275196824 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f640035f0 con 0x7f5f681049b0 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.239+0000 7f5f5e7fc640 1 -- 172.21.15.3:0/3275196824 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400ce60 con 0x7f5f681049b0 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.239+0000 7f5f6f510640 1 -- 172.21.15.3:0/3275196824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f681a6e20 con 0x7f5f681049b0 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.242+0000 7f5f75a84740 1 Processor -- start 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.242+0000 7f5f5e7fc640 1 -- 172.21.15.3:0/3275196824 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f5f64004000 con 0x7f5f681049b0 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.242+0000 7f5f75a84740 1 -- start start 2024-09-21T06:18:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f75a84740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ad5070 0x55d9a7e7c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f75a84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e7cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f75a84740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7ab4390 0x55d9a7e7d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f75a84740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d9a7df2e50 con 0x55d9a7ad5070 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f75a84740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d9a7de2280 con 0x55d9a7ab4390 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f75a84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d9a7de6090 con 0x55d9a7aac1b0 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f6daad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ad5070 0x55d9a7e7c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f6daad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ad5070 0x55d9a7e7c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41290/0 (socket says 172.21.15.3:41290) 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f6daad640 1 -- 172.21.15.3:0/683056782 learned_addr learned my addr 172.21.15.3:0/683056782 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f6d2ac640 1 --2- 172.21.15.3:0/683056782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e7cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f6caab640 1 --2- 172.21.15.3:0/683056782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7ab4390 0x55d9a7e7d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f6daad640 1 -- 172.21.15.3:0/683056782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7aac1b0 msgr2=0x55d9a7e7cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f6daad640 1 --2- 172.21.15.3:0/683056782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e7cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f6daad640 1 -- 172.21.15.3:0/683056782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7ab4390 msgr2=0x55d9a7e7d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f6daad640 1 --2- 172.21.15.3:0/683056782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7ab4390 0x55d9a7e7d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f6daad640 1 -- 172.21.15.3:0/683056782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a7b40650 con 0x55d9a7ad5070 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f5e7fc640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5f540b6680 0x7f5f540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f6d2ac640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5f540b6680 0x7f5f540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.243+0000 7f5f6daad640 1 --2- 172.21.15.3:0/683056782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ad5070 0x55d9a7e7c650 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009470 tx=0x7f5f600094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cc05d913fa1083 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.244+0000 7f5f5e7fc640 1 -- 172.21.15.3:0/3275196824 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5f640e7b50 con 0x7f5f681049b0 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.244+0000 7f5f537fe640 1 -- 172.21.15.3:0/683056782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60031cf0 con 0x55d9a7ad5070 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.244+0000 7f5f537fe640 1 -- 172.21.15.3:0/683056782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f60036040 con 0x55d9a7ad5070 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.244+0000 7f5f537fe640 1 -- 172.21.15.3:0/683056782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f600042a0 con 0x55d9a7ad5070 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.245+0000 7f5f75a84740 1 -- 172.21.15.3:0/683056782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ad5070 msgr2=0x55d9a7e7c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.245+0000 7f5f75a84740 1 --2- 172.21.15.3:0/683056782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ad5070 0x55d9a7e7c650 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009470 tx=0x7f5f600094a0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.245+0000 7f5f6d2ac640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5f540b6680 0x7f5f540b8b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55d9a7e7b2d0 tx=0x7f5f640eb600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.245+0000 7f5f75a84740 1 -- 172.21.15.3:0/683056782 shutdown_connections 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.245+0000 7f5f75a84740 1 --2- 172.21.15.3:0/683056782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e7cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.245+0000 7f5f75a84740 1 --2- 172.21.15.3:0/683056782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7ab4390 0x55d9a7e7d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.245+0000 7f5f75a84740 1 --2- 172.21.15.3:0/683056782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ad5070 0x55d9a7e7c650 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.245+0000 7f5f75a84740 1 -- 172.21.15.3:0/683056782 >> 172.21.15.3:0/683056782 conn(0x55d9a7ac9460 msgr2=0x55d9a7e7a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.245+0000 7f5f75a84740 1 -- 172.21.15.3:0/683056782 shutdown_connections 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.245+0000 7f5f75a84740 1 -- 172.21.15.3:0/683056782 wait complete. 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.245+0000 7f5f75a84740 1 -- start start 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.245+0000 7f5f75a84740 1 Processor -- start 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f75a84740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e8c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f75a84740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ab4390 0x55d9a7e8c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f75a84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7ad5070 0x55d9a7e8cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f75a84740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d9a7c43700 con 0x55d9a7ab4390 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f75a84740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d9a7d7d110 con 0x55d9a7aac1b0 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f75a84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d9a7e8f130 con 0x55d9a7ad5070 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f75a84740 10 client.0 _ll_register_callbacks cb 0x55d9a7d96990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f6caab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ab4390 0x55d9a7e8c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f6daad640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e8c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f6caab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ab4390 0x55d9a7e8c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41298/0 (socket says 172.21.15.3:41298) 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f6caab640 1 -- 172.21.15.3:0/2806719560 learned_addr learned my addr 172.21.15.3:0/2806719560 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f6daad640 1 --2- 172.21.15.3:0/2806719560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7ad5070 0x55d9a7e8cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f6caab640 1 -- 172.21.15.3:0/2806719560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7ad5070 msgr2=0x55d9a7e8cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f6caab640 1 --2- 172.21.15.3:0/2806719560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7ad5070 0x55d9a7e8cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f6caab640 1 -- 172.21.15.3:0/2806719560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7aac1b0 msgr2=0x55d9a7e8c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f6caab640 1 --2- 172.21.15.3:0/2806719560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e8c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f6caab640 1 -- 172.21.15.3:0/2806719560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a7daac40 con 0x55d9a7ab4390 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.246+0000 7f5f6daad640 1 --2- 172.21.15.3:0/2806719560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7ad5070 0x55d9a7e8cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f6caab640 1 --2- 172.21.15.3:0/2806719560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ab4390 0x55d9a7e8c5f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5f58012d20 tx=0x7f5f5800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211363787b81cde9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f50ff9640 1 -- 172.21.15.3:0/2806719560 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800fe20 con 0x55d9a7ab4390 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f50ff9640 1 -- 172.21.15.3:0/2806719560 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5f58005020 con 0x55d9a7ab4390 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f75a84740 1 -- 172.21.15.3:0/2806719560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f60036040 con 0x55d9a7ab4390 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f50ff9640 1 -- 172.21.15.3:0/2806719560 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5f58005220 con 0x55d9a7ab4390 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f75a84740 10 client.16518 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f75a84740 1 -- 172.21.15.3:0/2806719560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d9a7e9ae50 con 0x55d9a7ab4390 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f50ff9640 1 -- 172.21.15.3:0/2806719560 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f60036040 con 0x55d9a7ab4390 2024-09-21T06:18:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f50ff9640 1 -- 172.21.15.3:0/2806719560 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5f5801c470 con 0x55d9a7ab4390 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f75a84740 10 client.16518 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f75a84740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f75a84740 20 client.16518 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f75a84740 1 --2- 172.21.15.3:0/2806719560 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d9a7a939f0 0x55d9a7e9e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f75a84740 4 client.16518 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.247+0000 7f5f75a84740 1 -- 172.21.15.3:0/2806719560 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9a7df4da0 con 0x55d9a7a939f0 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.248+0000 7f5f6daad640 1 --2- 172.21.15.3:0/2806719560 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d9a7a939f0 0x55d9a7e9e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.248+0000 7f5f50ff9640 10 client.16518 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.248+0000 7f5f6daad640 1 --2- 172.21.15.3:0/2806719560 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d9a7a939f0 0x55d9a7e9e400 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22d924cc3099ae03 server_cookie=8247dc24ee6ea13a in_seq=0 out_seq=0 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.250+0000 7f5f50ff9640 1 -- 172.21.15.3:0/2806719560 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9a7df4da0 con 0x55d9a7a939f0 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.250+0000 7f5f50ff9640 10 client.16518 handle_command_reply: tid=0 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.251+0000 7f5f75a84740 1 client.16518 shutdown 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.251+0000 7f5f75a84740 1 -- 172.21.15.3:0/2806719560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ab4390 msgr2=0x55d9a7e8c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.251+0000 7f5f75a84740 1 --2- 172.21.15.3:0/2806719560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ab4390 0x55d9a7e8c5f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5f58012d20 tx=0x7f5f5800f790 comp rx=0 tx=0).stop 2024-09-21T06:18:10.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.251+0000 7f5f75a84740 1 -- 172.21.15.3:0/2806719560 shutdown_connections 2024-09-21T06:18:10.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.251+0000 7f5f75a84740 1 --2- 172.21.15.3:0/2806719560 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d9a7a939f0 0x55d9a7e9e400 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.251+0000 7f5f75a84740 1 --2- 172.21.15.3:0/2806719560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7ad5070 0x55d9a7e8cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.251+0000 7f5f50ff9640 0 client.16518 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:10.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.251+0000 7f5f75a84740 1 --2- 172.21.15.3:0/2806719560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ab4390 0x55d9a7e8c5f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.251+0000 7f5f75a84740 1 --2- 172.21.15.3:0/2806719560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e8c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.251+0000 7f5f75a84740 1 -- 172.21.15.3:0/2806719560 >> 172.21.15.3:0/2806719560 conn(0x55d9a7ac9460 msgr2=0x55d9a7e78b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:10.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.251+0000 7f5f75a84740 1 -- 172.21.15.3:0/2806719560 shutdown_connections 2024-09-21T06:18:10.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.251+0000 7f5f75a84740 1 -- 172.21.15.3:0/2806719560 wait complete. 2024-09-21T06:18:10.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.251+0000 7f5f75a84740 20 client.16518 trim_cache size 0 max 16384 2024-09-21T06:18:10.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.270+0000 7f5f75a84740 1 Processor -- start 2024-09-21T06:18:10.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.270+0000 7f5f75a84740 1 -- start start 2024-09-21T06:18:10.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.270+0000 7f5f75a84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7a939f0 0x55d9a7e87860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.270+0000 7f5f75a84740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e7a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.270+0000 7f5f75a84740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7ab4390 0x55d9a7e7dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.270+0000 7f5f75a84740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681ad410 con 0x55d9a7aac1b0 2024-09-21T06:18:10.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.270+0000 7f5f75a84740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681aefb0 con 0x55d9a7ab4390 2024-09-21T06:18:10.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.270+0000 7f5f75a84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681adae0 con 0x55d9a7a939f0 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f6caab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e7a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f6daad640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7a939f0 0x55d9a7e87860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f6caab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e7a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41304/0 (socket says 172.21.15.3:41304) 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f6caab640 1 -- 172.21.15.3:0/1908787993 learned_addr learned my addr 172.21.15.3:0/1908787993 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f6daad640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7a939f0 0x55d9a7e87860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:59882/0 (socket says 172.21.15.3:59882) 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f6daad640 1 --2- 172.21.15.3:0/1908787993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7ab4390 0x55d9a7e7dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f6daad640 1 -- 172.21.15.3:0/1908787993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7ab4390 msgr2=0x55d9a7e7dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f6daad640 1 --2- 172.21.15.3:0/1908787993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7ab4390 0x55d9a7e7dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f6daad640 1 -- 172.21.15.3:0/1908787993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7aac1b0 msgr2=0x55d9a7e7a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f6daad640 1 --2- 172.21.15.3:0/1908787993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e7a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f6daad640 1 -- 172.21.15.3:0/1908787993 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a7e98360 con 0x55d9a7a939f0 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f6caab640 1 --2- 172.21.15.3:0/1908787993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e7a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f6daad640 1 --2- 172.21.15.3:0/1908787993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7a939f0 0x55d9a7e87860 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009bf0 tx=0x7f5f60005c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92c2d8a7a132b900 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:10.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f52ffd640 1 -- 172.21.15.3:0/1908787993 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60031b70 con 0x55d9a7a939f0 2024-09-21T06:18:10.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f52ffd640 1 -- 172.21.15.3:0/1908787993 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f60036070 con 0x55d9a7a939f0 2024-09-21T06:18:10.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.271+0000 7f5f52ffd640 1 -- 172.21.15.3:0/1908787993 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f600042a0 con 0x55d9a7a939f0 2024-09-21T06:18:10.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.272+0000 7f5f75a84740 1 -- 172.21.15.3:0/1908787993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7a939f0 msgr2=0x55d9a7e87860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.272+0000 7f5f75a84740 1 --2- 172.21.15.3:0/1908787993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7a939f0 0x55d9a7e87860 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009bf0 tx=0x7f5f60005c30 comp rx=0 tx=0).stop 2024-09-21T06:18:10.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.272+0000 7f5f75a84740 1 -- 172.21.15.3:0/1908787993 shutdown_connections 2024-09-21T06:18:10.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.272+0000 7f5f75a84740 1 --2- 172.21.15.3:0/1908787993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7ab4390 0x55d9a7e7dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.272+0000 7f5f75a84740 1 --2- 172.21.15.3:0/1908787993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e7a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.272+0000 7f5f75a84740 1 --2- 172.21.15.3:0/1908787993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7a939f0 0x55d9a7e87860 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.272+0000 7f5f75a84740 1 -- 172.21.15.3:0/1908787993 >> 172.21.15.3:0/1908787993 conn(0x55d9a7ac9460 msgr2=0x55d9a7e7d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.272+0000 7f5f75a84740 1 -- 172.21.15.3:0/1908787993 shutdown_connections 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.272+0000 7f5f75a84740 1 -- 172.21.15.3:0/1908787993 wait complete. 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.273+0000 7f5f75a84740 1 -- start start 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.273+0000 7f5f75a84740 1 Processor -- start 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.273+0000 7f5f75a84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7a939f0 0x55d9a7e902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.273+0000 7f5f75a84740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e90830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.273+0000 7f5f75a84740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ab4390 0x55d9a7e90d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.273+0000 7f5f75a84740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681ae880 con 0x55d9a7ab4390 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.273+0000 7f5f75a84740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681ae1b0 con 0x55d9a7aac1b0 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.273+0000 7f5f75a84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681ab410 con 0x55d9a7a939f0 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.273+0000 7f5f75a84740 10 client.0 _ll_register_callbacks cb 0x7f5f58009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.273+0000 7f5f6daad640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7a939f0 0x55d9a7e902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f6caab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e90830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f6daad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ab4390 0x55d9a7e90d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f6caab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e90830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:34726/0 (socket says 172.21.15.3:34726) 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f6caab640 1 -- 172.21.15.3:0/3520854371 learned_addr learned my addr 172.21.15.3:0/3520854371 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f6caab640 1 -- 172.21.15.3:0/3520854371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7a939f0 msgr2=0x55d9a7e902f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f6caab640 1 --2- 172.21.15.3:0/3520854371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7a939f0 0x55d9a7e902f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f6caab640 1 -- 172.21.15.3:0/3520854371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ab4390 msgr2=0x55d9a7e90d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f6caab640 1 --2- 172.21.15.3:0/3520854371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ab4390 0x55d9a7e90d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f6caab640 1 -- 172.21.15.3:0/3520854371 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a7f07dc0 con 0x55d9a7aac1b0 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f6caab640 1 --2- 172.21.15.3:0/3520854371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e90830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5f5801c9e0 tx=0x7f5f5800fcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68a82aacce744ef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f50ff9640 1 -- 172.21.15.3:0/3520854371 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58029040 con 0x55d9a7aac1b0 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f50ff9640 1 -- 172.21.15.3:0/3520854371 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5f58021020 con 0x55d9a7aac1b0 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.274+0000 7f5f75a84740 1 -- 172.21.15.3:0/3520854371 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f60036070 con 0x55d9a7aac1b0 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.275+0000 7f5f50ff9640 1 -- 172.21.15.3:0/3520854371 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5f5801c470 con 0x55d9a7aac1b0 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.275+0000 7f5f75a84740 10 client.26099 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:10.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.275+0000 7f5f75a84740 1 -- 172.21.15.3:0/3520854371 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d9a7f08fb0 con 0x55d9a7aac1b0 2024-09-21T06:18:10.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.275+0000 7f5f50ff9640 1 -- 172.21.15.3:0/3520854371 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f60036070 con 0x55d9a7aac1b0 2024-09-21T06:18:10.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.275+0000 7f5f50ff9640 1 -- 172.21.15.3:0/3520854371 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5f5802c030 con 0x55d9a7aac1b0 2024-09-21T06:18:10.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.275+0000 7f5f75a84740 10 client.26099 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:10.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.275+0000 7f5f75a84740 10 client.26099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:10.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.275+0000 7f5f75a84740 20 client.26099 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:10.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.275+0000 7f5f75a84740 1 --2- 172.21.15.3:0/3520854371 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d9a7ad5070 0x55d9a7f0b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.275+0000 7f5f75a84740 4 client.26099 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:18:10.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.275+0000 7f5f75a84740 1 -- 172.21.15.3:0/3520854371 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d9a7df0850 con 0x55d9a7ad5070 2024-09-21T06:18:10.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.275+0000 7f5f6daad640 1 --2- 172.21.15.3:0/3520854371 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d9a7ad5070 0x55d9a7f0b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f50ff9640 10 client.26099 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:10.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f6daad640 1 --2- 172.21.15.3:0/3520854371 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d9a7ad5070 0x55d9a7f0b7a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec8d766cc4ab940 server_cookie=39dd55af7cd451e2 in_seq=0 out_seq=0 2024-09-21T06:18:10.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f50ff9640 1 -- 172.21.15.3:0/3520854371 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x55d9a7df0850 con 0x55d9a7ad5070 2024-09-21T06:18:10.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f50ff9640 10 client.26099 handle_command_reply: tid=0 2024-09-21T06:18:10.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f75a84740 1 client.26099 shutdown 2024-09-21T06:18:10.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f75a84740 1 -- 172.21.15.3:0/3520854371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7aac1b0 msgr2=0x55d9a7e90830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f75a84740 1 --2- 172.21.15.3:0/3520854371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e90830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5f5801c9e0 tx=0x7f5f5800fcb0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f75a84740 1 -- 172.21.15.3:0/3520854371 shutdown_connections 2024-09-21T06:18:10.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f75a84740 1 --2- 172.21.15.3:0/3520854371 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d9a7ad5070 0x55d9a7f0b7a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f75a84740 1 --2- 172.21.15.3:0/3520854371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d9a7ab4390 0x55d9a7e90d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f75a84740 1 --2- 172.21.15.3:0/3520854371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9a7aac1b0 0x55d9a7e90830 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f50ff9640 0 client.26099 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:10.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f75a84740 1 --2- 172.21.15.3:0/3520854371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9a7a939f0 0x55d9a7e902f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f75a84740 1 -- 172.21.15.3:0/3520854371 >> 172.21.15.3:0/3520854371 conn(0x55d9a7ac9460 msgr2=0x55d9a7e7d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:10.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f75a84740 1 -- 172.21.15.3:0/3520854371 shutdown_connections 2024-09-21T06:18:10.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f75a84740 1 -- 172.21.15.3:0/3520854371 wait complete. 2024-09-21T06:18:10.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.276+0000 7f5f75a84740 20 client.26099 trim_cache size 0 max 16384 2024-09-21T06:18:10.282 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "adba1d94-e33f-4efd-a020-63ebe07f6992", 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:10.284 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:10.285 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:18:10.286 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:18:10.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.277+0000 7f5f6f510640 1 -- 172.21.15.3:0/3275196824 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5f540b6680 msgr2=0x7f5f540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.277+0000 7f5f6f510640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5f540b6680 0x7f5f540b8b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55d9a7e7b2d0 tx=0x7f5f640eb600 comp rx=0 tx=0).stop 2024-09-21T06:18:10.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.277+0000 7f5f6f510640 1 -- 172.21.15.3:0/3275196824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f681049b0 msgr2=0x7f5f6819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.277+0000 7f5f6f510640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f681049b0 0x7f5f6819a460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400e9c0 tx=0x7f5f6400ee80 comp rx=0 tx=0).stop 2024-09-21T06:18:10.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.277+0000 7f5f6f510640 1 -- 172.21.15.3:0/3275196824 shutdown_connections 2024-09-21T06:18:10.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.277+0000 7f5f6f510640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f5f540b6680 0x7f5f540b8b90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.277+0000 7f5f6f510640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f6810de10 0x7f5f6819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.277+0000 7f5f6f510640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f68105300 0x7f5f6819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.277+0000 7f5f6f510640 1 --2- 172.21.15.3:0/3275196824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f681049b0 0x7f5f6819a460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.277+0000 7f5f6f510640 1 -- 172.21.15.3:0/3275196824 >> 172.21.15.3:0/3275196824 conn(0x7f5f68079730 msgr2=0x7f5f680ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:10.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.277+0000 7f5f6f510640 1 -- 172.21.15.3:0/3275196824 shutdown_connections 2024-09-21T06:18:10.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:10.277+0000 7f5f6f510640 1 -- 172.21.15.3:0/3275196824 wait complete. 2024-09-21T06:18:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.769+0000 7f892bc25640 1 -- 172.21.15.3:0/4087131321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8924109d80 msgr2=0x7f89241105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.769+0000 7f892bc25640 1 --2- 172.21.15.3:0/4087131321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8924109d80 0x7f89241105b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f891c009ca0 tx=0x7f891c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.769+0000 7f892bc25640 1 -- 172.21.15.3:0/4087131321 shutdown_connections 2024-09-21T06:18:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.769+0000 7f892bc25640 1 --2- 172.21.15.3:0/4087131321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8924109430 0x7f8924109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.769+0000 7f892bc25640 1 --2- 172.21.15.3:0/4087131321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8924109d80 0x7f89241105b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.769+0000 7f892bc25640 1 --2- 172.21.15.3:0/4087131321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8924103b00 0x7f8924103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.769+0000 7f892bc25640 1 -- 172.21.15.3:0/4087131321 >> 172.21.15.3:0/4087131321 conn(0x7f89240ff500 msgr2=0x7f8924101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.769+0000 7f892bc25640 1 -- 172.21.15.3:0/4087131321 shutdown_connections 2024-09-21T06:18:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.769+0000 7f892bc25640 1 -- 172.21.15.3:0/4087131321 wait complete. 2024-09-21T06:18:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.770+0000 7f892bc25640 1 Processor -- start 2024-09-21T06:18:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.770+0000 7f892bc25640 1 -- start start 2024-09-21T06:18:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.770+0000 7f892bc25640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8924103b00 0x7f892419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.770+0000 7f892bc25640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8924109430 0x7f892419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.770+0000 7f892bc25640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8924109d80 0x7f89241a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.770+0000 7f892bc25640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8924113a40 con 0x7f8924109d80 2024-09-21T06:18:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.770+0000 7f892bc25640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89241138c0 con 0x7f8924103b00 2024-09-21T06:18:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.770+0000 7f892bc25640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8924113bc0 con 0x7f8924109430 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.770+0000 7f892a1c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8924109d80 0x7f89241a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f89291c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8924109430 0x7f892419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f89299c1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8924103b00 0x7f892419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f89291c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8924109430 0x7f892419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:59906/0 (socket says 172.21.15.3:59906) 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f89291c0640 1 -- 172.21.15.3:0/2842273948 learned_addr learned my addr 172.21.15.3:0/2842273948 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f892a1c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8924109d80 0x7f89241a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41326/0 (socket says 172.21.15.3:41326) 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f892a1c2640 1 -- 172.21.15.3:0/2842273948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8924109430 msgr2=0x7f892419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f892a1c2640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8924109430 0x7f892419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f892a1c2640 1 -- 172.21.15.3:0/2842273948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8924103b00 msgr2=0x7f892419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f892a1c2640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8924103b00 0x7f892419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f892a1c2640 1 -- 172.21.15.3:0/2842273948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89241a2590 con 0x7f8924109d80 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f89299c1640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8924103b00 0x7f892419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f89291c0640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8924109430 0x7f892419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f892a1c2640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8924109d80 0x7f89241a1e90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f891c02ade0 tx=0x7f891c0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f891affd640 1 -- 172.21.15.3:0/2842273948 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c003b70 con 0x7f8924109d80 2024-09-21T06:18:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f892bc25640 1 -- 172.21.15.3:0/2842273948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89241a2820 con 0x7f8924109d80 2024-09-21T06:18:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f891affd640 1 -- 172.21.15.3:0/2842273948 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f891c003d10 con 0x7f8924109d80 2024-09-21T06:18:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f891affd640 1 -- 172.21.15.3:0/2842273948 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c02f9e0 con 0x7f8924109d80 2024-09-21T06:18:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.771+0000 7f892bc25640 1 -- 172.21.15.3:0/2842273948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89241a9010 con 0x7f8924109d80 2024-09-21T06:18:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.774+0000 7f8932c72740 1 Processor -- start 2024-09-21T06:18:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.774+0000 7f891affd640 1 -- 172.21.15.3:0/2842273948 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f891c02fb80 con 0x7f8924109d80 2024-09-21T06:18:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.775+0000 7f8932c72740 1 -- start start 2024-09-21T06:18:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.775+0000 7f8932c72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c2d1b0 0x560a92fd29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.775+0000 7f8932c72740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 0x560a92fd2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.775+0000 7f8932c72740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c0c4d0 0x560a92fd3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.775+0000 7f8932c72740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560a92f476f0 con 0x560a92c042f0 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.775+0000 7f8932c72740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560a92f36b20 con 0x560a92c0c4d0 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.775+0000 7f8932c72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560a92f3a930 con 0x560a92c2d1b0 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.775+0000 7f89299c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 0x560a92fd2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.775+0000 7f89299c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 0x560a92fd2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41342/0 (socket says 172.21.15.3:41342) 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.775+0000 7f89299c1640 1 -- 172.21.15.3:0/304837510 learned_addr learned my addr 172.21.15.3:0/304837510 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.775+0000 7f89291c0640 1 --2- 172.21.15.3:0/304837510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c0c4d0 0x560a92fd3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.775+0000 7f89291c0640 1 --2- 172.21.15.3:0/304837510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c2d1b0 0x560a92fd29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.776+0000 7f89299c1640 1 -- 172.21.15.3:0/304837510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c2d1b0 msgr2=0x560a92fd29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.776+0000 7f89299c1640 1 --2- 172.21.15.3:0/304837510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c2d1b0 0x560a92fd29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.776+0000 7f891affd640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89100b6680 0x7f89100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.776+0000 7f89299c1640 1 -- 172.21.15.3:0/304837510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c0c4d0 msgr2=0x560a92fd3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.776+0000 7f89299c1640 1 --2- 172.21.15.3:0/304837510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c0c4d0 0x560a92fd3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.776+0000 7f89299c1640 1 -- 172.21.15.3:0/304837510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a92efc530 con 0x560a92c042f0 2024-09-21T06:18:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.776+0000 7f89291c0640 1 --2- 172.21.15.3:0/304837510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c2d1b0 0x560a92fd29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.776+0000 7f892a1c2640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89100b6680 0x7f89100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.776+0000 7f89299c1640 1 --2- 172.21.15.3:0/304837510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 0x560a92fd2ef0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f89200115f0 tx=0x7f8920011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389eb246931a5941 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.776+0000 7f891affd640 1 -- 172.21.15.3:0/2842273948 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f891c108790 con 0x7f8924109d80 2024-09-21T06:18:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.776+0000 7f890ffff640 1 -- 172.21.15.3:0/304837510 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892001e070 con 0x560a92c042f0 2024-09-21T06:18:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.776+0000 7f890ffff640 1 -- 172.21.15.3:0/304837510 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f892000faf0 con 0x560a92c042f0 2024-09-21T06:18:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.776+0000 7f890ffff640 1 -- 172.21.15.3:0/304837510 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920019ac0 con 0x560a92c042f0 2024-09-21T06:18:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.777+0000 7f8932c72740 1 -- 172.21.15.3:0/304837510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 msgr2=0x560a92fd2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.777+0000 7f8932c72740 1 --2- 172.21.15.3:0/304837510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 0x560a92fd2ef0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f89200115f0 tx=0x7f8920011ab0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.777+0000 7f892a1c2640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89100b6680 0x7f89100b8b90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f891c0ce5b0 tx=0x7f891c005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.777+0000 7f8932c72740 1 -- 172.21.15.3:0/304837510 shutdown_connections 2024-09-21T06:18:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.777+0000 7f8932c72740 1 --2- 172.21.15.3:0/304837510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c0c4d0 0x560a92fd3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.777+0000 7f8932c72740 1 --2- 172.21.15.3:0/304837510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 0x560a92fd2ef0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.777+0000 7f8932c72740 1 --2- 172.21.15.3:0/304837510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c2d1b0 0x560a92fd29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.777+0000 7f8932c72740 1 -- 172.21.15.3:0/304837510 >> 172.21.15.3:0/304837510 conn(0x560a92c21300 msgr2=0x560a92fd08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.777+0000 7f8932c72740 1 -- 172.21.15.3:0/304837510 shutdown_connections 2024-09-21T06:18:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.777+0000 7f8932c72740 1 -- 172.21.15.3:0/304837510 wait complete. 2024-09-21T06:18:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.778+0000 7f8932c72740 1 -- start start 2024-09-21T06:18:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.778+0000 7f8932c72740 1 Processor -- start 2024-09-21T06:18:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.778+0000 7f8932c72740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c042f0 0x560a92fde020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.778+0000 7f8932c72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c0c4d0 0x560a92fde560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.778+0000 7f8932c72740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c2d1b0 0x560a92fea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.778+0000 7f8932c72740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560a92e6eab0 con 0x560a92c2d1b0 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.778+0000 7f8932c72740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560a92edc730 con 0x560a92c042f0 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.778+0000 7f8932c72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560a92fe1130 con 0x560a92c0c4d0 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.778+0000 7f8932c72740 10 client.0 _ll_register_callbacks cb 0x560a92f306b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.778+0000 7f89291c0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c2d1b0 0x560a92fea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.778+0000 7f89291c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c042f0 0x560a92fde020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.778+0000 7f89291c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c042f0 0x560a92fde020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:34774/0 (socket says 172.21.15.3:34774) 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.778+0000 7f89291c0640 1 -- 172.21.15.3:0/2443433690 learned_addr learned my addr 172.21.15.3:0/2443433690 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f89299c1640 1 --2- 172.21.15.3:0/2443433690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c0c4d0 0x560a92fde560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f89291c0640 1 -- 172.21.15.3:0/2443433690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c0c4d0 msgr2=0x560a92fde560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f89291c0640 1 --2- 172.21.15.3:0/2443433690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c0c4d0 0x560a92fde560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f89291c0640 1 -- 172.21.15.3:0/2443433690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c2d1b0 msgr2=0x560a92fea420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f89291c0640 1 --2- 172.21.15.3:0/2443433690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c2d1b0 0x560a92fea420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f89291c0640 1 -- 172.21.15.3:0/2443433690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a92c98790 con 0x560a92c042f0 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f89299c1640 1 --2- 172.21.15.3:0/2443433690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c0c4d0 0x560a92fde560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f89291c0640 1 --2- 172.21.15.3:0/2443433690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c042f0 0x560a92fde020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8914012ef0 tx=0x7f8914005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab11991974710a89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f890d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f890d7fa640 1 -- 172.21.15.3:0/2443433690 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891401b040 con 0x560a92c042f0 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f890d7fa640 1 -- 172.21.15.3:0/2443433690 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8914004460 con 0x560a92c042f0 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f8932c72740 1 -- 172.21.15.3:0/2443433690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f892000faf0 con 0x560a92c042f0 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f890d7fa640 1 -- 172.21.15.3:0/2443433690 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8914005020 con 0x560a92c042f0 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f8932c72740 10 client.26111 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.779+0000 7f8932c72740 1 -- 172.21.15.3:0/2443433690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a92d11620 con 0x560a92c042f0 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.780+0000 7f890d7fa640 1 -- 172.21.15.3:0/2443433690 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f892000faf0 con 0x560a92c042f0 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.780+0000 7f890d7fa640 1 -- 172.21.15.3:0/2443433690 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8914004b80 con 0x560a92c042f0 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.780+0000 7f8932c72740 10 client.26111 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.780+0000 7f8932c72740 10 client.26111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.780+0000 7f8932c72740 20 client.26111 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.780+0000 7f8932c72740 1 --2- 172.21.15.3:0/2443433690 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a92bebd80 0x560a92ff0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.780+0000 7f8932c72740 4 client.26111 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.780+0000 7f8932c72740 1 -- 172.21.15.3:0/2443433690 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a92f49640 con 0x560a92bebd80 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.780+0000 7f89299c1640 1 --2- 172.21.15.3:0/2443433690 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a92bebd80 0x560a92ff0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.781+0000 7f89299c1640 1 --2- 172.21.15.3:0/2443433690 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a92bebd80 0x560a92ff0180 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a31eda8ff0c79f6 server_cookie=5f14c1721afeeaba in_seq=0 out_seq=0 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.781+0000 7f890d7fa640 10 client.26111 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f890d7fa640 1 -- 172.21.15.3:0/2443433690 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560a92f49640 con 0x560a92bebd80 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f890d7fa640 10 client.26111 handle_command_reply: tid=0 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f8932c72740 1 client.26111 shutdown 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f8932c72740 1 -- 172.21.15.3:0/2443433690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c042f0 msgr2=0x560a92fde020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f8932c72740 1 --2- 172.21.15.3:0/2443433690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c042f0 0x560a92fde020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8914012ef0 tx=0x7f8914005c70 comp rx=0 tx=0).stop 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f8932c72740 1 -- 172.21.15.3:0/2443433690 shutdown_connections 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f8932c72740 1 --2- 172.21.15.3:0/2443433690 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a92bebd80 0x560a92ff0180 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f8932c72740 1 --2- 172.21.15.3:0/2443433690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c0c4d0 0x560a92fde560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f8932c72740 1 --2- 172.21.15.3:0/2443433690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c2d1b0 0x560a92fea420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f8932c72740 1 --2- 172.21.15.3:0/2443433690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c042f0 0x560a92fde020 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f8932c72740 1 -- 172.21.15.3:0/2443433690 >> 172.21.15.3:0/2443433690 conn(0x560a92c21300 msgr2=0x560a92fd39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f890d7fa640 0 client.26111 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f8932c72740 1 -- 172.21.15.3:0/2443433690 shutdown_connections 2024-09-21T06:18:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f8932c72740 1 -- 172.21.15.3:0/2443433690 wait complete. 2024-09-21T06:18:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.782+0000 7f8932c72740 20 client.26111 trim_cache size 0 max 16384 2024-09-21T06:18:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.802+0000 7f8932c72740 1 Processor -- start 2024-09-21T06:18:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.802+0000 7f8932c72740 1 -- start start 2024-09-21T06:18:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f8932c72740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 0x560a92fd43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f8932c72740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c0c4d0 0x560a92fde500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f8932c72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c2d1b0 0x560a92fdfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f8932c72740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f89241b18d0 con 0x560a92c042f0 2024-09-21T06:18:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f8932c72740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89241b3470 con 0x560a92c0c4d0 2024-09-21T06:18:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f8932c72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f89241b1fa0 con 0x560a92c2d1b0 2024-09-21T06:18:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f89291c0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 0x560a92fd43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f89291c0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 0x560a92fd43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41370/0 (socket says 172.21.15.3:41370) 2024-09-21T06:18:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f89291c0640 1 -- 172.21.15.3:0/655866595 learned_addr learned my addr 172.21.15.3:0/655866595 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f89299c1640 1 --2- 172.21.15.3:0/655866595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c0c4d0 0x560a92fde500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f89291c0640 1 --2- 172.21.15.3:0/655866595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c2d1b0 0x560a92fdfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f89291c0640 1 -- 172.21.15.3:0/655866595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c2d1b0 msgr2=0x560a92fdfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f89291c0640 1 --2- 172.21.15.3:0/655866595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c2d1b0 0x560a92fdfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f89291c0640 1 -- 172.21.15.3:0/655866595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c0c4d0 msgr2=0x560a92fde500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f89291c0640 1 --2- 172.21.15.3:0/655866595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c0c4d0 0x560a92fde500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f89291c0640 1 -- 172.21.15.3:0/655866595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a92fd3790 con 0x560a92c042f0 2024-09-21T06:18:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.803+0000 7f89291c0640 1 --2- 172.21.15.3:0/655866595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 0x560a92fd43a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8914006010 tx=0x7f891400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5d627930641b6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.804+0000 7f890f7fe640 1 -- 172.21.15.3:0/655866595 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914016020 con 0x560a92c042f0 2024-09-21T06:18:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.804+0000 7f890f7fe640 1 -- 172.21.15.3:0/655866595 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89140129e0 con 0x560a92c042f0 2024-09-21T06:18:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.804+0000 7f890f7fe640 1 -- 172.21.15.3:0/655866595 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914031070 con 0x560a92c042f0 2024-09-21T06:18:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.804+0000 7f8932c72740 1 -- 172.21.15.3:0/655866595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 msgr2=0x560a92fd43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.805+0000 7f8932c72740 1 --2- 172.21.15.3:0/655866595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 0x560a92fd43a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8914006010 tx=0x7f891400b6a0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.805+0000 7f8932c72740 1 -- 172.21.15.3:0/655866595 shutdown_connections 2024-09-21T06:18:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.805+0000 7f8932c72740 1 --2- 172.21.15.3:0/655866595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c2d1b0 0x560a92fdfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.805+0000 7f8932c72740 1 --2- 172.21.15.3:0/655866595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c0c4d0 0x560a92fde500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.805+0000 7f8932c72740 1 --2- 172.21.15.3:0/655866595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c042f0 0x560a92fd43a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.805+0000 7f8932c72740 1 -- 172.21.15.3:0/655866595 >> 172.21.15.3:0/655866595 conn(0x560a92c21300 msgr2=0x560a92fdf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.805+0000 7f8932c72740 1 -- 172.21.15.3:0/655866595 shutdown_connections 2024-09-21T06:18:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.805+0000 7f8932c72740 1 -- 172.21.15.3:0/655866595 wait complete. 2024-09-21T06:18:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.805+0000 7f8932c72740 1 -- start start 2024-09-21T06:18:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.805+0000 7f8932c72740 1 Processor -- start 2024-09-21T06:18:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f8932c72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c042f0 0x560a92fe3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f8932c72740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c0c4d0 0x560a92fe3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f8932c72740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c2d1b0 0x560a93058b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f8932c72740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f89241b2d40 con 0x560a92c0c4d0 2024-09-21T06:18:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f8932c72740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89241b2670 con 0x560a92c2d1b0 2024-09-21T06:18:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f8932c72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f89241af8b0 con 0x560a92c042f0 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f8932c72740 10 client.0 _ll_register_callbacks cb 0x7f8914012660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f89299c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c0c4d0 0x560a92fe3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f89299c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c0c4d0 0x560a92fe3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41372/0 (socket says 172.21.15.3:41372) 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f89299c1640 1 -- 172.21.15.3:0/163915393 learned_addr learned my addr 172.21.15.3:0/163915393 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f89291c0640 1 --2- 172.21.15.3:0/163915393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c042f0 0x560a92fe3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f89291c0640 1 --2- 172.21.15.3:0/163915393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c2d1b0 0x560a93058b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f89299c1640 1 -- 172.21.15.3:0/163915393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c042f0 msgr2=0x560a92fe3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f89299c1640 1 --2- 172.21.15.3:0/163915393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c042f0 0x560a92fe3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f89299c1640 1 -- 172.21.15.3:0/163915393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c2d1b0 msgr2=0x560a93058b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f89299c1640 1 --2- 172.21.15.3:0/163915393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c2d1b0 0x560a93058b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.806+0000 7f89299c1640 1 -- 172.21.15.3:0/163915393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a930590c0 con 0x560a92c0c4d0 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f89299c1640 1 --2- 172.21.15.3:0/163915393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c0c4d0 0x560a92fe3f50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8920009250 tx=0x7f8920005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e584bc10449e4a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f890d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f890d7fa640 1 -- 172.21.15.3:0/163915393 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892001e040 con 0x560a92c0c4d0 2024-09-21T06:18:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f890d7fa640 1 -- 172.21.15.3:0/163915393 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8920009cc0 con 0x560a92c0c4d0 2024-09-21T06:18:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f8932c72740 1 -- 172.21.15.3:0/163915393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89140129e0 con 0x560a92c0c4d0 2024-09-21T06:18:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f890d7fa640 1 -- 172.21.15.3:0/163915393 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f89200050b0 con 0x560a92c0c4d0 2024-09-21T06:18:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f8932c72740 10 client.16542 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f8932c72740 1 -- 172.21.15.3:0/163915393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a9305a2b0 con 0x560a92c0c4d0 2024-09-21T06:18:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f890d7fa640 1 -- 172.21.15.3:0/163915393 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89140129e0 con 0x560a92c0c4d0 2024-09-21T06:18:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f890d7fa640 1 -- 172.21.15.3:0/163915393 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8920019710 con 0x560a92c0c4d0 2024-09-21T06:18:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f8932c72740 10 client.16542 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f8932c72740 10 client.16542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f8932c72740 20 client.16542 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.807+0000 7f8932c72740 1 --2- 172.21.15.3:0/163915393 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a92bc11c0 0x560a9305c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.808+0000 7f8932c72740 4 client.16542 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:18:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.808+0000 7f8932c72740 1 -- 172.21.15.3:0/163915393 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a92f450f0 con 0x560a92bc11c0 2024-09-21T06:18:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.808+0000 7f89291c0640 1 --2- 172.21.15.3:0/163915393 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a92bc11c0 0x560a9305c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.808+0000 7f890d7fa640 10 client.16542 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.808+0000 7f89291c0640 1 --2- 172.21.15.3:0/163915393 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a92bc11c0 0x560a9305c910 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab85f0071ec1813 server_cookie=cb1c2e9a22022ab in_seq=0 out_seq=0 2024-09-21T06:18:10.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 255429 13.55 MB/sec warmup 103 sec latency 67.219 ms 2024-09-21T06:18:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.994+0000 7f890d7fa640 1 -- 172.21.15.3:0/163915393 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a92f450f0 con 0x560a92bc11c0 2024-09-21T06:18:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.994+0000 7f890d7fa640 10 client.16542 handle_command_reply: tid=0 2024-09-21T06:18:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.995+0000 7f8932c72740 1 client.16542 shutdown 2024-09-21T06:18:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.995+0000 7f8932c72740 1 -- 172.21.15.3:0/163915393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c0c4d0 msgr2=0x560a92fe3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.995+0000 7f8932c72740 1 --2- 172.21.15.3:0/163915393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c0c4d0 0x560a92fe3f50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8920009250 tx=0x7f8920005850 comp rx=0 tx=0).stop 2024-09-21T06:18:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.995+0000 7f8932c72740 1 -- 172.21.15.3:0/163915393 shutdown_connections 2024-09-21T06:18:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.995+0000 7f8932c72740 1 --2- 172.21.15.3:0/163915393 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a92bc11c0 0x560a9305c910 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.995+0000 7f8932c72740 1 --2- 172.21.15.3:0/163915393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a92c2d1b0 0x560a93058b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.995+0000 7f890d7fa640 0 client.16542 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.995+0000 7f8932c72740 1 --2- 172.21.15.3:0/163915393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a92c0c4d0 0x560a92fe3f50 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.995+0000 7f8932c72740 1 --2- 172.21.15.3:0/163915393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a92c042f0 0x560a92fe3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.995+0000 7f8932c72740 1 -- 172.21.15.3:0/163915393 >> 172.21.15.3:0/163915393 conn(0x560a92c21300 msgr2=0x560a92fd3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.995+0000 7f8932c72740 1 -- 172.21.15.3:0/163915393 shutdown_connections 2024-09-21T06:18:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.995+0000 7f8932c72740 1 -- 172.21.15.3:0/163915393 wait complete. 2024-09-21T06:18:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.995+0000 7f8932c72740 20 client.16542 trim_cache size 0 max 16384 2024-09-21T06:18:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:18:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:18:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:18:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:18:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.996+0000 7f892bc25640 1 -- 172.21.15.3:0/2842273948 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89100b6680 msgr2=0x7f89100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.996+0000 7f892bc25640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89100b6680 0x7f89100b8b90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f891c0ce5b0 tx=0x7f891c005c30 comp rx=0 tx=0).stop 2024-09-21T06:18:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.996+0000 7f892bc25640 1 -- 172.21.15.3:0/2842273948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8924109d80 msgr2=0x7f89241a1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.996+0000 7f892bc25640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8924109d80 0x7f89241a1e90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f891c02ade0 tx=0x7f891c0316f0 comp rx=0 tx=0).stop 2024-09-21T06:18:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.996+0000 7f892bc25640 1 -- 172.21.15.3:0/2842273948 shutdown_connections 2024-09-21T06:18:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.996+0000 7f892bc25640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89100b6680 0x7f89100b8b90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.996+0000 7f892bc25640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8924109430 0x7f892419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.996+0000 7f892bc25640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8924109d80 0x7f89241a1e90 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.996+0000 7f892bc25640 1 --2- 172.21.15.3:0/2842273948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8924103b00 0x7f892419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.996+0000 7f892bc25640 1 -- 172.21.15.3:0/2842273948 >> 172.21.15.3:0/2842273948 conn(0x7f89240ff500 msgr2=0x7f89241069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.996+0000 7f892bc25640 1 -- 172.21.15.3:0/2842273948 shutdown_connections 2024-09-21T06:18:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:10.996+0000 7f892bc25640 1 -- 172.21.15.3:0/2842273948 wait complete. 2024-09-21T06:18:11.289 INFO:tasks.cephfs.filesystem:scrub status for tag:adba1d94-e33f-4efd-a020-63ebe07f6992 - {'path': '/', 'tag': 'adba1d94-e33f-4efd-a020-63ebe07f6992', 'options': 'recursive,force'} 2024-09-21T06:18:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:11 smithi003 ceph-mon[31227]: pgmap v420: 97 pgs: 97 active+clean; 62 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-09-21T06:18:11.696 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:11 smithi137 ceph-mon[34464]: pgmap v420: 97 pgs: 97 active+clean; 62 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-09-21T06:18:11.705 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:11 smithi204 ceph-mon[34659]: pgmap v420: 97 pgs: 97 active+clean; 62 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-09-21T06:18:11.949 INFO:tasks.workunit.client.0.smithi003.stdout: 1 257247 13.49 MB/sec warmup 104 sec latency 19.496 ms 2024-09-21T06:18:12.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 259425 13.49 MB/sec warmup 105 sec latency 60.879 ms 2024-09-21T06:18:13.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:13 smithi003 ceph-mon[31227]: pgmap v421: 97 pgs: 97 active+clean; 62 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.5 MiB/s wr, 377 op/s 2024-09-21T06:18:13.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:13 smithi137 ceph-mon[34464]: pgmap v421: 97 pgs: 97 active+clean; 62 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.5 MiB/s wr, 377 op/s 2024-09-21T06:18:13.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:13 smithi204 ceph-mon[34659]: pgmap v421: 97 pgs: 97 active+clean; 62 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.5 MiB/s wr, 377 op/s 2024-09-21T06:18:13.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 262632 13.53 MB/sec warmup 106 sec latency 17.010 ms 2024-09-21T06:18:14.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 264604 13.47 MB/sec warmup 107 sec latency 15.002 ms 2024-09-21T06:18:15.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:15 smithi003 ceph-mon[31227]: pgmap v422: 97 pgs: 97 active+clean; 62 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 321 op/s 2024-09-21T06:18:15.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:15 smithi137 ceph-mon[34464]: pgmap v422: 97 pgs: 97 active+clean; 62 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 321 op/s 2024-09-21T06:18:15.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:15 smithi204 ceph-mon[34659]: pgmap v422: 97 pgs: 97 active+clean; 62 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 321 op/s 2024-09-21T06:18:15.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 266499 13.47 MB/sec warmup 108 sec latency 69.438 ms 2024-09-21T06:18:16.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:16 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:18:16.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:16 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:18:16.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:16 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:18:16.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 269701 13.51 MB/sec warmup 109 sec latency 14.529 ms 2024-09-21T06:18:17.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:17 smithi003 ceph-mon[31227]: pgmap v423: 97 pgs: 97 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 425 op/s 2024-09-21T06:18:17.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:17 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:18:17.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:17 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:18:17.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:17 smithi137 ceph-mon[34464]: pgmap v423: 97 pgs: 97 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 425 op/s 2024-09-21T06:18:17.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:17 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:18:17.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:17 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:18:17.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:17 smithi204 ceph-mon[34659]: pgmap v423: 97 pgs: 97 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 425 op/s 2024-09-21T06:18:17.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:17 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:18:17.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:17 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:18:17.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 271493 13.45 MB/sec warmup 110 sec latency 17.263 ms 2024-09-21T06:18:18.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 273697 13.45 MB/sec warmup 111 sec latency 13.280 ms 2024-09-21T06:18:19.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:19 smithi003 ceph-mon[31227]: pgmap v424: 97 pgs: 97 active+clean; 57 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 401 op/s 2024-09-21T06:18:19.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:19 smithi137 ceph-mon[34464]: pgmap v424: 97 pgs: 97 active+clean; 57 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 401 op/s 2024-09-21T06:18:19.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:19 smithi204 ceph-mon[34659]: pgmap v424: 97 pgs: 97 active+clean; 57 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 401 op/s 2024-09-21T06:18:19.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 276916 13.50 MB/sec warmup 112 sec latency 15.468 ms 2024-09-21T06:18:20.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 278388 13.43 MB/sec warmup 113 sec latency 94.287 ms 2024-09-21T06:18:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:21 smithi003 ceph-mon[31227]: pgmap v425: 97 pgs: 97 active+clean; 61 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-09-21T06:18:21.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:21 smithi137 ceph-mon[34464]: pgmap v425: 97 pgs: 97 active+clean; 61 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-09-21T06:18:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:21 smithi204 ceph-mon[34659]: pgmap v425: 97 pgs: 97 active+clean; 61 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-09-21T06:18:21.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 280779 13.44 MB/sec warmup 114 sec latency 9.391 ms 2024-09-21T06:18:22.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 284000 13.48 MB/sec warmup 115 sec latency 18.254 ms 2024-09-21T06:18:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:23 smithi003 ceph-mon[31227]: pgmap v426: 97 pgs: 97 active+clean; 64 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 398 op/s 2024-09-21T06:18:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:23 smithi137 ceph-mon[34464]: pgmap v426: 97 pgs: 97 active+clean; 64 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 398 op/s 2024-09-21T06:18:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:23 smithi204 ceph-mon[34659]: pgmap v426: 97 pgs: 97 active+clean; 64 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 398 op/s 2024-09-21T06:18:23.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 286170 13.43 MB/sec warmup 116 sec latency 18.909 ms 2024-09-21T06:18:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.103+0000 7ff7ee4ec640 1 -- 172.21.15.3:0/1308636593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e8109d80 msgr2=0x7ff7e81105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.103+0000 7ff7ee4ec640 1 --2- 172.21.15.3:0/1308636593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e8109d80 0x7ff7e81105b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009ca0 tx=0x7ff7e002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.103+0000 7ff7ee4ec640 1 -- 172.21.15.3:0/1308636593 shutdown_connections 2024-09-21T06:18:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.103+0000 7ff7ee4ec640 1 --2- 172.21.15.3:0/1308636593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e8109d80 0x7ff7e81105b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.103+0000 7ff7ee4ec640 1 --2- 172.21.15.3:0/1308636593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7e8109430 0x7ff7e8109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.103+0000 7ff7ee4ec640 1 --2- 172.21.15.3:0/1308636593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7e8103b00 0x7ff7e8103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.103+0000 7ff7ee4ec640 1 -- 172.21.15.3:0/1308636593 >> 172.21.15.3:0/1308636593 conn(0x7ff7e80ff500 msgr2=0x7ff7e8101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.103+0000 7ff7ee4ec640 1 -- 172.21.15.3:0/1308636593 shutdown_connections 2024-09-21T06:18:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.103+0000 7ff7ee4ec640 1 -- 172.21.15.3:0/1308636593 wait complete. 2024-09-21T06:18:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.104+0000 7ff7ee4ec640 1 Processor -- start 2024-09-21T06:18:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.104+0000 7ff7ee4ec640 1 -- start start 2024-09-21T06:18:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7ee4ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7e8103b00 0x7ff7e819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7ee4ec640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e8109430 0x7ff7e819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7ee4ec640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7e8109d80 0x7ff7e819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7ee4ec640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8113a40 con 0x7ff7e8109430 2024-09-21T06:18:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7ee4ec640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81138c0 con 0x7ff7e8103b00 2024-09-21T06:18:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7ee4ec640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8113bc0 con 0x7ff7e8109d80 2024-09-21T06:18:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e8109430 0x7ff7e819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7e8103b00 0x7ff7e819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7eca89640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7e8109d80 0x7ff7e819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e8109430 0x7ff7e819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36888/0 (socket says 172.21.15.3:36888) 2024-09-21T06:18:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7e8103b00 0x7ff7e819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:43856/0 (socket says 172.21.15.3:43856) 2024-09-21T06:18:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7e77fe640 1 -- 172.21.15.3:0/4153066615 learned_addr learned my addr 172.21.15.3:0/4153066615 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7e77fe640 1 -- 172.21.15.3:0/4153066615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7e8109d80 msgr2=0x7ff7e819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/4153066615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7e8109d80 0x7ff7e819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7e77fe640 1 -- 172.21.15.3:0/4153066615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7e8103b00 msgr2=0x7ff7e819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/4153066615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7e8103b00 0x7ff7e819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7e77fe640 1 -- 172.21.15.3:0/4153066615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e81a03f0 con 0x7ff7e8109430 2024-09-21T06:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/4153066615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e8109430 0x7ff7e819aa60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00e9f0 tx=0x7ff7dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7e57fa640 1 -- 172.21.15.3:0/4153066615 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00ccc0 con 0x7ff7e8109430 2024-09-21T06:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.105+0000 7ff7ee4ec640 1 -- 172.21.15.3:0/4153066615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e81a06e0 con 0x7ff7e8109430 2024-09-21T06:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.106+0000 7ff7e57fa640 1 -- 172.21.15.3:0/4153066615 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7dc0108e0 con 0x7ff7e8109430 2024-09-21T06:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.106+0000 7ff7e57fa640 1 -- 172.21.15.3:0/4153066615 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc018860 con 0x7ff7e8109430 2024-09-21T06:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.106+0000 7ff7ee4ec640 1 -- 172.21.15.3:0/4153066615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e81a6e70 con 0x7ff7e8109430 2024-09-21T06:18:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.108+0000 7ff7e57fa640 1 -- 172.21.15.3:0/4153066615 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7ff7dc018a00 con 0x7ff7e8109430 2024-09-21T06:18:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.108+0000 7ff7f545a740 1 Processor -- start 2024-09-21T06:18:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7f545a740 1 -- start start 2024-09-21T06:18:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7f545a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff528f1b0 0x558ff56349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7f545a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 0x558ff5634ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7f545a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff526e4d0 0x558ff5635430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7f545a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558ff55a96f0 con 0x558ff52662f0 2024-09-21T06:18:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7f545a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558ff5598b20 con 0x558ff526e4d0 2024-09-21T06:18:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7f545a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558ff559c930 con 0x558ff528f1b0 2024-09-21T06:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 0x558ff5634ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 0x558ff5634ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36894/0 (socket says 172.21.15.3:36894) 2024-09-21T06:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7e7fff640 1 -- 172.21.15.3:0/3687155144 learned_addr learned my addr 172.21.15.3:0/3687155144 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/3687155144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff526e4d0 0x558ff5635430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7e57fa640 1 --2- 172.21.15.3:0/4153066615 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7d00b6680 0x7ff7d00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7eca89640 1 --2- 172.21.15.3:0/3687155144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff528f1b0 0x558ff56349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7eca89640 1 --2- 172.21.15.3:0/4153066615 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7d00b6680 0x7ff7d00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7e7fff640 1 -- 172.21.15.3:0/3687155144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff528f1b0 msgr2=0x558ff56349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/3687155144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff528f1b0 0x558ff56349b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7e7fff640 1 -- 172.21.15.3:0/3687155144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff526e4d0 msgr2=0x558ff5635430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/3687155144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff526e4d0 0x558ff5635430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7e7fff640 1 -- 172.21.15.3:0/3687155144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff555e530 con 0x558ff52662f0 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.109+0000 7ff7e57fa640 1 -- 172.21.15.3:0/4153066615 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff7dc0e7990 con 0x7ff7e8109430 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.110+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/3687155144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 0x558ff5634ef0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009cf0 tx=0x7ff7d8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc49d15633a3682 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.110+0000 7ff7eca89640 1 --2- 172.21.15.3:0/4153066615 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7d00b6680 0x7ff7d00b8b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0006010 tx=0x7ff7e00095c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.110+0000 7ff7ce7fc640 1 -- 172.21.15.3:0/3687155144 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8004230 con 0x558ff52662f0 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.110+0000 7ff7ce7fc640 1 -- 172.21.15.3:0/3687155144 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d80043d0 con 0x558ff52662f0 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.110+0000 7ff7ce7fc640 1 -- 172.21.15.3:0/3687155144 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8019880 con 0x558ff52662f0 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.111+0000 7ff7f545a740 1 -- 172.21.15.3:0/3687155144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 msgr2=0x558ff5634ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.111+0000 7ff7f545a740 1 --2- 172.21.15.3:0/3687155144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 0x558ff5634ef0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009cf0 tx=0x7ff7d8011810 comp rx=0 tx=0).stop 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.111+0000 7ff7f545a740 1 -- 172.21.15.3:0/3687155144 shutdown_connections 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.111+0000 7ff7f545a740 1 --2- 172.21.15.3:0/3687155144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff526e4d0 0x558ff5635430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.111+0000 7ff7f545a740 1 --2- 172.21.15.3:0/3687155144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 0x558ff5634ef0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.111+0000 7ff7f545a740 1 --2- 172.21.15.3:0/3687155144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff528f1b0 0x558ff56349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.111+0000 7ff7f545a740 1 -- 172.21.15.3:0/3687155144 >> 172.21.15.3:0/3687155144 conn(0x558ff5283300 msgr2=0x558ff56328a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.111+0000 7ff7f545a740 1 -- 172.21.15.3:0/3687155144 shutdown_connections 2024-09-21T06:18:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.111+0000 7ff7f545a740 1 -- 172.21.15.3:0/3687155144 wait complete. 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.111+0000 7ff7f545a740 1 -- start start 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.112+0000 7ff7f545a740 1 Processor -- start 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.112+0000 7ff7f545a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 0x558ff5640000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.112+0000 7ff7f545a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff526e4d0 0x558ff5640540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.112+0000 7ff7f545a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff528f1b0 0x558ff564c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.112+0000 7ff7f545a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558ff54d0ab0 con 0x558ff52662f0 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.112+0000 7ff7f545a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558ff553e730 con 0x558ff528f1b0 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.112+0000 7ff7f545a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558ff5643130 con 0x558ff526e4d0 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.112+0000 7ff7f545a740 10 client.0 _ll_register_callbacks cb 0x558ff55926b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.112+0000 7ff7e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 0x558ff5640000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.112+0000 7ff7e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 0x558ff5640000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36896/0 (socket says 172.21.15.3:36896) 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.112+0000 7ff7e7fff640 1 -- 172.21.15.3:0/496314185 learned_addr learned my addr 172.21.15.3:0/496314185 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.112+0000 7ff7e77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff526e4d0 0x558ff5640540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.112+0000 7ff7eca89640 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff528f1b0 0x558ff564c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7e7fff640 1 -- 172.21.15.3:0/496314185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff526e4d0 msgr2=0x558ff5640540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff526e4d0 0x558ff5640540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7e7fff640 1 -- 172.21.15.3:0/496314185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff528f1b0 msgr2=0x558ff564c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff528f1b0 0x558ff564c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7e7fff640 1 -- 172.21.15.3:0/496314185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff52fa790 con 0x558ff52662f0 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7eca89640 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff528f1b0 0x558ff564c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff526e4d0 0x558ff5640540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 0x558ff5640000 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009560 tx=0x7ff7d8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8118836d5c591478 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7c77fe640 1 -- 172.21.15.3:0/496314185 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8027070 con 0x558ff52662f0 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7c77fe640 1 -- 172.21.15.3:0/496314185 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff7d8005240 con 0x558ff52662f0 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7f545a740 1 -- 172.21.15.3:0/496314185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d80043d0 con 0x558ff52662f0 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7c77fe640 1 -- 172.21.15.3:0/496314185 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7d8033050 con 0x558ff52662f0 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7f545a740 10 client.16554 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7f545a740 1 -- 172.21.15.3:0/496314185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ff5373620 con 0x558ff52662f0 2024-09-21T06:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.113+0000 7ff7c77fe640 1 -- 172.21.15.3:0/496314185 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d80043d0 con 0x558ff52662f0 2024-09-21T06:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.114+0000 7ff7c77fe640 1 -- 172.21.15.3:0/496314185 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff7d8004b70 con 0x558ff52662f0 2024-09-21T06:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.114+0000 7ff7f545a740 10 client.16554 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.114+0000 7ff7f545a740 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.114+0000 7ff7f545a740 20 client.16554 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.114+0000 7ff7f545a740 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558ff524dd80 0x558ff5652060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.114+0000 7ff7f545a740 4 client.16554 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.114+0000 7ff7f545a740 1 -- 172.21.15.3:0/496314185 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ff55ab640 con 0x558ff524dd80 2024-09-21T06:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.114+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558ff524dd80 0x558ff5652060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.114+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558ff524dd80 0x558ff5652060 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7cf3e858ef3a00 server_cookie=1080f9d1c51eff49 in_seq=0 out_seq=0 2024-09-21T06:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.114+0000 7ff7c77fe640 10 client.16554 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.115+0000 7ff7c77fe640 1 -- 172.21.15.3:0/496314185 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558ff55ab640 con 0x558ff524dd80 2024-09-21T06:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7c77fe640 10 client.16554 handle_command_reply: tid=0 2024-09-21T06:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7f545a740 1 client.16554 shutdown 2024-09-21T06:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7f545a740 1 -- 172.21.15.3:0/496314185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 msgr2=0x558ff5640000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7f545a740 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 0x558ff5640000 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009560 tx=0x7ff7d8005dc0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7f545a740 1 -- 172.21.15.3:0/496314185 shutdown_connections 2024-09-21T06:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7f545a740 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558ff524dd80 0x558ff5652060 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7c77fe640 0 client.16554 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7f545a740 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff526e4d0 0x558ff5640540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7f545a740 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff528f1b0 0x558ff564c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7f545a740 1 --2- 172.21.15.3:0/496314185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff52662f0 0x558ff5640000 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7f545a740 1 -- 172.21.15.3:0/496314185 >> 172.21.15.3:0/496314185 conn(0x558ff5283300 msgr2=0x558ff56359e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7f545a740 1 -- 172.21.15.3:0/496314185 shutdown_connections 2024-09-21T06:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7f545a740 1 -- 172.21.15.3:0/496314185 wait complete. 2024-09-21T06:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.116+0000 7ff7f545a740 20 client.16554 trim_cache size 0 max 16384 2024-09-21T06:18:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.132+0000 7ff7f545a740 1 Processor -- start 2024-09-21T06:18:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.132+0000 7ff7f545a740 1 -- start start 2024-09-21T06:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.132+0000 7ff7f545a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff524dd80 0x558ff564c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.132+0000 7ff7f545a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff52662f0 0x558ff5641c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.132+0000 7ff7f545a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff526e4d0 0x558ff5648790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.132+0000 7ff7f545a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81af770 con 0x558ff526e4d0 2024-09-21T06:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.132+0000 7ff7f545a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81b1310 con 0x558ff52662f0 2024-09-21T06:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.132+0000 7ff7f545a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81afe40 con 0x558ff524dd80 2024-09-21T06:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.133+0000 7ff7e7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff524dd80 0x558ff564c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.133+0000 7ff7eca89640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff526e4d0 0x558ff5648790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.133+0000 7ff7e7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff524dd80 0x558ff564c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41104/0 (socket says 172.21.15.3:41104) 2024-09-21T06:18:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.133+0000 7ff7e77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff52662f0 0x558ff5641c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.133+0000 7ff7e7fff640 1 -- 172.21.15.3:0/1956969648 learned_addr learned my addr 172.21.15.3:0/1956969648 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.133+0000 7ff7e7fff640 1 -- 172.21.15.3:0/1956969648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff52662f0 msgr2=0x558ff5641c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.133+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/1956969648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff52662f0 0x558ff5641c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.133+0000 7ff7e7fff640 1 -- 172.21.15.3:0/1956969648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff526e4d0 msgr2=0x558ff5648790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.133+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/1956969648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff526e4d0 0x558ff5648790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.133+0000 7ff7e7fff640 1 -- 172.21.15.3:0/1956969648 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff5636cb0 con 0x558ff524dd80 2024-09-21T06:18:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.133+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/1956969648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff524dd80 0x558ff564c350 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8003c60 tx=0x7ff7d8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9fbc36d17ca1078 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.133+0000 7ff7cdffb640 1 -- 172.21.15.3:0/1956969648 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d803c070 con 0x558ff524dd80 2024-09-21T06:18:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.134+0000 7ff7cdffb640 1 -- 172.21.15.3:0/1956969648 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d80038a0 con 0x558ff524dd80 2024-09-21T06:18:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.134+0000 7ff7cdffb640 1 -- 172.21.15.3:0/1956969648 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8027030 con 0x558ff524dd80 2024-09-21T06:18:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.134+0000 7ff7f545a740 1 -- 172.21.15.3:0/1956969648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff524dd80 msgr2=0x558ff564c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.134+0000 7ff7f545a740 1 --2- 172.21.15.3:0/1956969648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff524dd80 0x558ff564c350 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8003c60 tx=0x7ff7d8005f90 comp rx=0 tx=0).stop 2024-09-21T06:18:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.134+0000 7ff7f545a740 1 -- 172.21.15.3:0/1956969648 shutdown_connections 2024-09-21T06:18:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.134+0000 7ff7f545a740 1 --2- 172.21.15.3:0/1956969648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff526e4d0 0x558ff5648790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.134+0000 7ff7f545a740 1 --2- 172.21.15.3:0/1956969648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff52662f0 0x558ff5641c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.134+0000 7ff7f545a740 1 --2- 172.21.15.3:0/1956969648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff524dd80 0x558ff564c350 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.135+0000 7ff7f545a740 1 -- 172.21.15.3:0/1956969648 >> 172.21.15.3:0/1956969648 conn(0x558ff5283300 msgr2=0x558ff5641520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.135+0000 7ff7f545a740 1 -- 172.21.15.3:0/1956969648 shutdown_connections 2024-09-21T06:18:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.135+0000 7ff7f545a740 1 -- 172.21.15.3:0/1956969648 wait complete. 2024-09-21T06:18:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.135+0000 7ff7f545a740 1 -- start start 2024-09-21T06:18:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.135+0000 7ff7f545a740 1 Processor -- start 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.135+0000 7ff7f545a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff524dd80 0x558ff56b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.135+0000 7ff7f545a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff52662f0 0x558ff56b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.135+0000 7ff7f545a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff526e4d0 0x558ff56b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.135+0000 7ff7f545a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81b0be0 con 0x558ff526e4d0 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.135+0000 7ff7f545a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81b0510 con 0x558ff524dd80 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.135+0000 7ff7f545a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81ad750 con 0x558ff52662f0 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7f545a740 10 client.0 _ll_register_callbacks cb 0x7ff7dc0a9f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff524dd80 0x558ff56b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7e77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff52662f0 0x558ff56b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7eca89640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff526e4d0 0x558ff56b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7e77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff52662f0 0x558ff56b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41118/0 (socket says 172.21.15.3:41118) 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7eca89640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff526e4d0 0x558ff56b8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36910/0 (socket says 172.21.15.3:36910) 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff524dd80 0x558ff56b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:43874/0 (socket says 172.21.15.3:43874) 2024-09-21T06:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7e77fe640 1 -- 172.21.15.3:0/1914975856 learned_addr learned my addr 172.21.15.3:0/1914975856 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7e77fe640 1 -- 172.21.15.3:0/1914975856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff524dd80 msgr2=0x558ff56b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/1914975856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff524dd80 0x558ff56b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7e77fe640 1 -- 172.21.15.3:0/1914975856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff526e4d0 msgr2=0x558ff56b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/1914975856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff526e4d0 0x558ff56b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7e77fe640 1 -- 172.21.15.3:0/1914975856 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff56b90f0 con 0x558ff52662f0 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/1914975856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff524dd80 0x558ff56b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/1914975856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff52662f0 0x558ff56b3a40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0a9df0 tx=0x7ff7dc0a9040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69029f34323b54fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7c77fe640 1 -- 172.21.15.3:0/1914975856 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc0e5550 con 0x558ff52662f0 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7c77fe640 1 -- 172.21.15.3:0/1914975856 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff7dc0accd0 con 0x558ff52662f0 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.136+0000 7ff7f545a740 1 -- 172.21.15.3:0/1914975856 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d80038a0 con 0x558ff52662f0 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.137+0000 7ff7c77fe640 1 -- 172.21.15.3:0/1914975856 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7dc0ad380 con 0x558ff52662f0 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.137+0000 7ff7f545a740 10 client.26098 fetch_fsmap learned FSMap version 29 2024-09-21T06:18:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.137+0000 7ff7f545a740 1 -- 172.21.15.3:0/1914975856 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ff56c04e0 con 0x558ff52662f0 2024-09-21T06:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.137+0000 7ff7c77fe640 1 -- 172.21.15.3:0/1914975856 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d80038a0 con 0x558ff52662f0 2024-09-21T06:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.137+0000 7ff7c77fe640 1 -- 172.21.15.3:0/1914975856 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff7dc0ac740 con 0x558ff52662f0 2024-09-21T06:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.137+0000 7ff7f545a740 10 client.26098 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.137+0000 7ff7f545a740 10 client.26098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.137+0000 7ff7f545a740 20 client.26098 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.137+0000 7ff7f545a740 1 --2- 172.21.15.3:0/1914975856 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558ff528f1b0 0x558ff56c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.137+0000 7ff7f545a740 4 client.26098 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.137+0000 7ff7f545a740 1 -- 172.21.15.3:0/1914975856 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ff55a70f0 con 0x558ff528f1b0 2024-09-21T06:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.137+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/1914975856 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558ff528f1b0 0x558ff56c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.138+0000 7ff7c77fe640 10 client.26098 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.138+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/1914975856 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558ff528f1b0 0x558ff56c2cb0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a01d55c14b77593 server_cookie=371f1b795a0dd976 in_seq=0 out_seq=0 2024-09-21T06:18:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.281+0000 7ff7c77fe640 1 -- 172.21.15.3:0/1914975856 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ff55a70f0 con 0x558ff528f1b0 2024-09-21T06:18:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.281+0000 7ff7c77fe640 10 client.26098 handle_command_reply: tid=0 2024-09-21T06:18:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.281+0000 7ff7f545a740 1 client.26098 shutdown 2024-09-21T06:18:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.282+0000 7ff7f545a740 1 -- 172.21.15.3:0/1914975856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff52662f0 msgr2=0x558ff56b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.282+0000 7ff7f545a740 1 --2- 172.21.15.3:0/1914975856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff52662f0 0x558ff56b3a40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0a9df0 tx=0x7ff7dc0a9040 comp rx=0 tx=0).stop 2024-09-21T06:18:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.282+0000 7ff7f545a740 1 -- 172.21.15.3:0/1914975856 shutdown_connections 2024-09-21T06:18:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.282+0000 7ff7f545a740 1 --2- 172.21.15.3:0/1914975856 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558ff528f1b0 0x558ff56c2cb0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.282+0000 7ff7f545a740 1 --2- 172.21.15.3:0/1914975856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ff52662f0 0x558ff56b3a40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.282+0000 7ff7f545a740 1 --2- 172.21.15.3:0/1914975856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558ff526e4d0 0x558ff56b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.282+0000 7ff7f545a740 1 --2- 172.21.15.3:0/1914975856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ff524dd80 0x558ff56b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.282+0000 7ff7f545a740 1 -- 172.21.15.3:0/1914975856 >> 172.21.15.3:0/1914975856 conn(0x558ff5283300 msgr2=0x558ff5632880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.282+0000 7ff7c77fe640 0 client.26098 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.282+0000 7ff7f545a740 1 -- 172.21.15.3:0/1914975856 shutdown_connections 2024-09-21T06:18:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.282+0000 7ff7f545a740 1 -- 172.21.15.3:0/1914975856 wait complete. 2024-09-21T06:18:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.282+0000 7ff7f545a740 20 client.26098 trim_cache size 0 max 16384 2024-09-21T06:18:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:18:24.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:18:24.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:18:24.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:18:24.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.283+0000 7ff7ee4ec640 1 -- 172.21.15.3:0/4153066615 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7d00b6680 msgr2=0x7ff7d00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:24.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.283+0000 7ff7ee4ec640 1 --2- 172.21.15.3:0/4153066615 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7d00b6680 0x7ff7d00b8b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0006010 tx=0x7ff7e00095c0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.283+0000 7ff7ee4ec640 1 -- 172.21.15.3:0/4153066615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e8109430 msgr2=0x7ff7e819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.283+0000 7ff7ee4ec640 1 --2- 172.21.15.3:0/4153066615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e8109430 0x7ff7e819aa60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00e9f0 tx=0x7ff7dc00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.283+0000 7ff7ee4ec640 1 -- 172.21.15.3:0/4153066615 shutdown_connections 2024-09-21T06:18:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.283+0000 7ff7ee4ec640 1 --2- 172.21.15.3:0/4153066615 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7d00b6680 0x7ff7d00b8b90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.283+0000 7ff7ee4ec640 1 --2- 172.21.15.3:0/4153066615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7e8109d80 0x7ff7e819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.283+0000 7ff7ee4ec640 1 --2- 172.21.15.3:0/4153066615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e8109430 0x7ff7e819aa60 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.283+0000 7ff7ee4ec640 1 --2- 172.21.15.3:0/4153066615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7e8103b00 0x7ff7e819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.283+0000 7ff7ee4ec640 1 -- 172.21.15.3:0/4153066615 >> 172.21.15.3:0/4153066615 conn(0x7ff7e80ff500 msgr2=0x7ff7e81069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.283+0000 7ff7ee4ec640 1 -- 172.21.15.3:0/4153066615 shutdown_connections 2024-09-21T06:18:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:24.283+0000 7ff7ee4ec640 1 -- 172.21.15.3:0/4153066615 wait complete. 2024-09-21T06:18:24.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:18:24.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:18:24.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:18:24.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 287772 13.42 MB/sec warmup 117 sec latency 66.380 ms 2024-09-21T06:18:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:25 smithi003 ceph-mon[31227]: pgmap v427: 97 pgs: 97 active+clean; 65 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 359 op/s 2024-09-21T06:18:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:18:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:18:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:18:25.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:25 smithi137 ceph-mon[34464]: pgmap v427: 97 pgs: 97 active+clean; 65 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 359 op/s 2024-09-21T06:18:25.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:18:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:18:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:18:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:25 smithi204 ceph-mon[34659]: pgmap v427: 97 pgs: 97 active+clean; 65 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 359 op/s 2024-09-21T06:18:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:18:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:18:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:18:25.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 290528 13.42 MB/sec warmup 118 sec latency 91.436 ms 2024-09-21T06:18:26.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 292159 13.37 MB/sec warmup 119 sec latency 13.922 ms 2024-09-21T06:18:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:27 smithi003 ceph-mon[31227]: pgmap v428: 97 pgs: 97 active+clean; 65 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 411 op/s 2024-09-21T06:18:27.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:27 smithi137 ceph-mon[34464]: pgmap v428: 97 pgs: 97 active+clean; 65 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 411 op/s 2024-09-21T06:18:27.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:27 smithi204 ceph-mon[34659]: pgmap v428: 97 pgs: 97 active+clean; 65 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 411 op/s 2024-09-21T06:18:28.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 297920 17.42 MB/sec execute 1 sec latency 17.668 ms 2024-09-21T06:18:29.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:29 smithi003 ceph-mon[31227]: pgmap v429: 97 pgs: 97 active+clean; 62 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-09-21T06:18:29.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:29 smithi137 ceph-mon[34464]: pgmap v429: 97 pgs: 97 active+clean; 62 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-09-21T06:18:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:29 smithi204 ceph-mon[34659]: pgmap v429: 97 pgs: 97 active+clean; 62 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-09-21T06:18:29.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 299479 12.12 MB/sec execute 2 sec latency 18.201 ms 2024-09-21T06:18:30.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 301907 12.75 MB/sec execute 3 sec latency 52.790 ms 2024-09-21T06:18:31.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:31 smithi003 ceph-mon[31227]: pgmap v430: 97 pgs: 97 active+clean; 60 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 423 op/s 2024-09-21T06:18:31.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:31 smithi137 ceph-mon[34464]: pgmap v430: 97 pgs: 97 active+clean; 60 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 423 op/s 2024-09-21T06:18:31.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:31 smithi204 ceph-mon[34659]: pgmap v430: 97 pgs: 97 active+clean; 60 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 423 op/s 2024-09-21T06:18:31.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 304288 12.32 MB/sec execute 4 sec latency 96.558 ms 2024-09-21T06:18:32.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 306058 11.67 MB/sec execute 5 sec latency 16.947 ms 2024-09-21T06:18:33.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:33 smithi003 ceph-mon[31227]: pgmap v431: 97 pgs: 97 active+clean; 65 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-09-21T06:18:33.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:33 smithi137 ceph-mon[34464]: pgmap v431: 97 pgs: 97 active+clean; 65 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-09-21T06:18:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:33 smithi204 ceph-mon[34659]: pgmap v431: 97 pgs: 97 active+clean; 65 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-09-21T06:18:33.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 309004 12.75 MB/sec execute 6 sec latency 15.662 ms 2024-09-21T06:18:34.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 311974 13.00 MB/sec execute 7 sec latency 13.871 ms 2024-09-21T06:18:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:35 smithi003 ceph-mon[31227]: pgmap v432: 97 pgs: 97 active+clean; 65 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 358 op/s 2024-09-21T06:18:35.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:35 smithi137 ceph-mon[34464]: pgmap v432: 97 pgs: 97 active+clean; 65 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 358 op/s 2024-09-21T06:18:35.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:35 smithi204 ceph-mon[34659]: pgmap v432: 97 pgs: 97 active+clean; 65 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 358 op/s 2024-09-21T06:18:35.950 INFO:tasks.workunit.client.0.smithi003.stdout: 1 313823 12.61 MB/sec execute 8 sec latency 16.234 ms 2024-09-21T06:18:36.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 316103 12.75 MB/sec execute 9 sec latency 38.824 ms 2024-09-21T06:18:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.384+0000 7fb9a57c6640 1 -- 172.21.15.3:0/193720307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a0105250 msgr2=0x7fb9a010d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.384+0000 7fb9a57c6640 1 --2- 172.21.15.3:0/193720307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a0105250 0x7fb9a010d790 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb988009900 tx=0x7fb98802a410 comp rx=0 tx=0).stop 2024-09-21T06:18:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.385+0000 7fb9a57c6640 1 -- 172.21.15.3:0/193720307 shutdown_connections 2024-09-21T06:18:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.385+0000 7fb9a57c6640 1 --2- 172.21.15.3:0/193720307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9a010dd60 0x7fb9a0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.385+0000 7fb9a57c6640 1 --2- 172.21.15.3:0/193720307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a0105250 0x7fb9a010d790 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.385+0000 7fb9a57c6640 1 --2- 172.21.15.3:0/193720307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9a0104900 0x7fb9a0104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.385+0000 7fb9a57c6640 1 -- 172.21.15.3:0/193720307 >> 172.21.15.3:0/193720307 conn(0x7fb9a007a740 msgr2=0x7fb9a007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.385+0000 7fb9a57c6640 1 -- 172.21.15.3:0/193720307 shutdown_connections 2024-09-21T06:18:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.385+0000 7fb9a57c6640 1 -- 172.21.15.3:0/193720307 wait complete. 2024-09-21T06:18:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.385+0000 7fb9a57c6640 1 Processor -- start 2024-09-21T06:18:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.385+0000 7fb9a57c6640 1 -- start start 2024-09-21T06:18:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb9a57c6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9a0104900 0x7fb9a019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb9a57c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9a0105250 0x7fb9a019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb9a57c6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a010dd60 0x7fb9a019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb9a57c6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01139c0 con 0x7fb9a010dd60 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb9a57c6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0113840 con 0x7fb9a0104900 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb9a57c6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0113b40 con 0x7fb9a0105250 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb99f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a010dd60 0x7fb9a019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb99f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a010dd60 0x7fb9a019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57440/0 (socket says 172.21.15.3:57440) 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb99effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9a0104900 0x7fb9a019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb99f7fe640 1 -- 172.21.15.3:0/2546932471 learned_addr learned my addr 172.21.15.3:0/2546932471 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9a0105250 0x7fb9a019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb99f7fe640 1 -- 172.21.15.3:0/2546932471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9a0105250 msgr2=0x7fb9a019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb99f7fe640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9a0105250 0x7fb9a019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb99f7fe640 1 -- 172.21.15.3:0/2546932471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9a0104900 msgr2=0x7fb9a019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb99f7fe640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9a0104900 0x7fb9a019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb99f7fe640 1 -- 172.21.15.3:0/2546932471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a01a0410 con 0x7fb9a010dd60 2024-09-21T06:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9a0105250 0x7fb9a019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.386+0000 7fb99f7fe640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a010dd60 0x7fb9a019fd10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb99000e970 tx=0x7fb99000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.387+0000 7fb997fff640 1 -- 172.21.15.3:0/2546932471 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99000ccc0 con 0x7fb9a010dd60 2024-09-21T06:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.387+0000 7fb9a57c6640 1 -- 172.21.15.3:0/2546932471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a01a0700 con 0x7fb9a010dd60 2024-09-21T06:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.387+0000 7fb997fff640 1 -- 172.21.15.3:0/2546932471 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb99000ce60 con 0x7fb9a010dd60 2024-09-21T06:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.387+0000 7fb997fff640 1 -- 172.21.15.3:0/2546932471 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990010430 con 0x7fb9a010dd60 2024-09-21T06:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.387+0000 7fb9a57c6640 1 -- 172.21.15.3:0/2546932471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a01a6e90 con 0x7fb9a010dd60 2024-09-21T06:18:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb997fff640 1 -- 172.21.15.3:0/2546932471 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fb990004f90 con 0x7fb9a010dd60 2024-09-21T06:18:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb9ac818740 1 Processor -- start 2024-09-21T06:18:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb9ac818740 1 -- start start 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb9ac818740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4dc1b0 0x562baf8819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb9ac818740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4b32f0 0x562baf881ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb9ac818740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4bb4d0 0x562baf882430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb9ac818740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562baf7f66f0 con 0x562baf4b32f0 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb9ac818740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562baf7e5b20 con 0x562baf4dc1b0 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb9ac818740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562baf7e9930 con 0x562baf4bb4d0 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb99effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4b32f0 0x562baf881ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb99e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4dc1b0 0x562baf8819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb99effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4b32f0 0x562baf881ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57446/0 (socket says 172.21.15.3:57446) 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb99effd640 1 -- 172.21.15.3:0/3632203674 learned_addr learned my addr 172.21.15.3:0/3632203674 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.390+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/3632203674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4dc1b0 0x562baf8819b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49900/0 (socket says 172.21.15.3:49900) 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/3632203674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4bb4d0 0x562baf882430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb997fff640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb98c0b6680 0x7fb98c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb99effd640 1 -- 172.21.15.3:0/3632203674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4bb4d0 msgr2=0x562baf882430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb99effd640 1 --2- 172.21.15.3:0/3632203674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4bb4d0 0x562baf882430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb99effd640 1 -- 172.21.15.3:0/3632203674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4dc1b0 msgr2=0x562baf8819b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb99effd640 1 --2- 172.21.15.3:0/3632203674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4dc1b0 0x562baf8819b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb99effd640 1 -- 172.21.15.3:0/3632203674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562baf7ab530 con 0x562baf4b32f0 2024-09-21T06:18:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb99f7fe640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb98c0b6680 0x7fb98c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/3632203674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4bb4d0 0x562baf882430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb997fff640 1 -- 172.21.15.3:0/2546932471 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb9900e7ac0 con 0x7fb9a010dd60 2024-09-21T06:18:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb99effd640 1 --2- 172.21.15.3:0/3632203674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4b32f0 0x562baf881ef0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb9980114f0 tx=0x7fb9980119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac80e071d01df517 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb9957fa640 1 -- 172.21.15.3:0/3632203674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99801e070 con 0x562baf4b32f0 2024-09-21T06:18:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb9957fa640 1 -- 172.21.15.3:0/3632203674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb99800fad0 con 0x562baf4b32f0 2024-09-21T06:18:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.391+0000 7fb9957fa640 1 -- 172.21.15.3:0/3632203674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998019aa0 con 0x562baf4b32f0 2024-09-21T06:18:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.392+0000 7fb9ac818740 1 -- 172.21.15.3:0/3632203674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4b32f0 msgr2=0x562baf881ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.392+0000 7fb99f7fe640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb98c0b6680 0x7fb98c0b8b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb9900aabc0 tx=0x7fb990079500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.392+0000 7fb9ac818740 1 --2- 172.21.15.3:0/3632203674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4b32f0 0x562baf881ef0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb9980114f0 tx=0x7fb9980119b0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.392+0000 7fb9ac818740 1 -- 172.21.15.3:0/3632203674 shutdown_connections 2024-09-21T06:18:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.392+0000 7fb9ac818740 1 --2- 172.21.15.3:0/3632203674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4bb4d0 0x562baf882430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.392+0000 7fb9ac818740 1 --2- 172.21.15.3:0/3632203674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4b32f0 0x562baf881ef0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.392+0000 7fb9ac818740 1 --2- 172.21.15.3:0/3632203674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4dc1b0 0x562baf8819b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.392+0000 7fb9ac818740 1 -- 172.21.15.3:0/3632203674 >> 172.21.15.3:0/3632203674 conn(0x562baf4d0300 msgr2=0x562baf87f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.392+0000 7fb9ac818740 1 -- 172.21.15.3:0/3632203674 shutdown_connections 2024-09-21T06:18:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.392+0000 7fb9ac818740 1 -- 172.21.15.3:0/3632203674 wait complete. 2024-09-21T06:18:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb9ac818740 1 -- start start 2024-09-21T06:18:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb9ac818740 1 Processor -- start 2024-09-21T06:18:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb9ac818740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4b32f0 0x562baf88d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb9ac818740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf88d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb9ac818740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4dc1b0 0x562baf899420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb9ac818740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562baf71dab0 con 0x562baf4bb4d0 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb9ac818740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562baf78b730 con 0x562baf4dc1b0 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb9ac818740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562baf890130 con 0x562baf4b32f0 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb99e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4b32f0 0x562baf88d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb9ac818740 10 client.0 _ll_register_callbacks cb 0x562baf7df6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb99e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4b32f0 0x562baf88d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:42832/0 (socket says 172.21.15.3:42832) 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb99e7fc640 1 -- 172.21.15.3:0/3945188391 learned_addr learned my addr 172.21.15.3:0/3945188391 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb99effd640 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf88d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.393+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4dc1b0 0x562baf899420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb99effd640 1 -- 172.21.15.3:0/3945188391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4b32f0 msgr2=0x562baf88d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb99effd640 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4b32f0 0x562baf88d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb99effd640 1 -- 172.21.15.3:0/3945188391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4dc1b0 msgr2=0x562baf899420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb99effd640 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4dc1b0 0x562baf899420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb99effd640 1 -- 172.21.15.3:0/3945188391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562baf547790 con 0x562baf4bb4d0 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4b32f0 0x562baf88d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:37.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb99effd640 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf88d5e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb998000f80 tx=0x7fb998019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945d46291ddfa694 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb9867fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb9867fc640 1 -- 172.21.15.3:0/3945188391 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99801e040 con 0x562baf4bb4d0 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb9867fc640 1 -- 172.21.15.3:0/3945188391 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb998016040 con 0x562baf4bb4d0 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb9ac818740 1 -- 172.21.15.3:0/3945188391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb99800fad0 con 0x562baf4bb4d0 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb9867fc640 1 -- 172.21.15.3:0/3945188391 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb998019410 con 0x562baf4bb4d0 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb9ac818740 10 client.16572 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb9ac818740 1 -- 172.21.15.3:0/3945188391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562baf5c0620 con 0x562baf4bb4d0 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.394+0000 7fb9867fc640 1 -- 172.21.15.3:0/3945188391 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb99800fad0 con 0x562baf4bb4d0 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.395+0000 7fb9867fc640 1 -- 172.21.15.3:0/3945188391 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb998005040 con 0x562baf4bb4d0 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.395+0000 7fb9ac818740 10 client.16572 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.395+0000 7fb9ac818740 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.395+0000 7fb9ac818740 20 client.16572 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.395+0000 7fb9ac818740 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562baf49ad80 0x562baf89f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.395+0000 7fb9ac818740 4 client.16572 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:37.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.395+0000 7fb9ac818740 1 -- 172.21.15.3:0/3945188391 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562baf7f8640 con 0x562baf49ad80 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.395+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562baf49ad80 0x562baf89f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.395+0000 7fb9867fc640 10 client.16572 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.395+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562baf49ad80 0x562baf89f070 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65781609c832355 server_cookie=5e0c8658185f8627 in_seq=0 out_seq=0 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9867fc640 1 -- 172.21.15.3:0/3945188391 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562baf7f8640 con 0x562baf49ad80 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9867fc640 10 client.16572 handle_command_reply: tid=0 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9ac818740 1 client.16572 shutdown 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9ac818740 1 -- 172.21.15.3:0/3945188391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 msgr2=0x562baf88d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9ac818740 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf88d5e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb998000f80 tx=0x7fb998019600 comp rx=0 tx=0).stop 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9ac818740 1 -- 172.21.15.3:0/3945188391 shutdown_connections 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9ac818740 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562baf49ad80 0x562baf89f070 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9ac818740 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4dc1b0 0x562baf899420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9ac818740 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf88d5e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9ac818740 1 --2- 172.21.15.3:0/3945188391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4b32f0 0x562baf88d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9867fc640 0 client.16572 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9ac818740 1 -- 172.21.15.3:0/3945188391 >> 172.21.15.3:0/3945188391 conn(0x562baf4d0300 msgr2=0x562baf8829e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9ac818740 1 -- 172.21.15.3:0/3945188391 shutdown_connections 2024-09-21T06:18:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9ac818740 1 -- 172.21.15.3:0/3945188391 wait complete. 2024-09-21T06:18:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.397+0000 7fb9ac818740 20 client.16572 trim_cache size 0 max 16384 2024-09-21T06:18:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.412+0000 7fb9ac818740 1 Processor -- start 2024-09-21T06:18:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.412+0000 7fb9ac818740 1 -- start start 2024-09-21T06:18:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb9ac818740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4b32f0 0x562baf8830f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb9ac818740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf899320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb9ac818740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4dc1b0 0x562baf88a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb9ac818740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01af7e0 con 0x562baf4bb4d0 2024-09-21T06:18:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb9ac818740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01b1350 con 0x562baf4b32f0 2024-09-21T06:18:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb9ac818740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01afeb0 con 0x562baf4dc1b0 2024-09-21T06:18:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb99effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf899320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb99effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf899320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57468/0 (socket says 172.21.15.3:57468) 2024-09-21T06:18:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb99effd640 1 -- 172.21.15.3:0/1553028429 learned_addr learned my addr 172.21.15.3:0/1553028429 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/1553028429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4b32f0 0x562baf8830f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/1553028429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4dc1b0 0x562baf88a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb99effd640 1 -- 172.21.15.3:0/1553028429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4dc1b0 msgr2=0x562baf88a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb99effd640 1 --2- 172.21.15.3:0/1553028429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4dc1b0 0x562baf88a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb99effd640 1 -- 172.21.15.3:0/1553028429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4b32f0 msgr2=0x562baf8830f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb99effd640 1 --2- 172.21.15.3:0/1553028429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4b32f0 0x562baf8830f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb99effd640 1 -- 172.21.15.3:0/1553028429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562baf89a1c0 con 0x562baf4bb4d0 2024-09-21T06:18:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb99effd640 1 --2- 172.21.15.3:0/1553028429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf899320 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb9980195d0 tx=0x7fb9980119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a281110505225d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.413+0000 7fb994ff9640 1 -- 172.21.15.3:0/1553028429 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99801e040 con 0x562baf4bb4d0 2024-09-21T06:18:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.414+0000 7fb994ff9640 1 -- 172.21.15.3:0/1553028429 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb998003e20 con 0x562baf4bb4d0 2024-09-21T06:18:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.414+0000 7fb994ff9640 1 -- 172.21.15.3:0/1553028429 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99800fa90 con 0x562baf4bb4d0 2024-09-21T06:18:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.414+0000 7fb9ac818740 1 -- 172.21.15.3:0/1553028429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 msgr2=0x562baf899320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.414+0000 7fb9ac818740 1 --2- 172.21.15.3:0/1553028429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf899320 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb9980195d0 tx=0x7fb9980119b0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.414+0000 7fb9ac818740 1 -- 172.21.15.3:0/1553028429 shutdown_connections 2024-09-21T06:18:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.414+0000 7fb9ac818740 1 --2- 172.21.15.3:0/1553028429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4dc1b0 0x562baf88a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.414+0000 7fb9ac818740 1 --2- 172.21.15.3:0/1553028429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf899320 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.414+0000 7fb9ac818740 1 --2- 172.21.15.3:0/1553028429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4b32f0 0x562baf8830f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.414+0000 7fb9ac818740 1 -- 172.21.15.3:0/1553028429 >> 172.21.15.3:0/1553028429 conn(0x562baf4d0300 msgr2=0x562baf8829e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.415+0000 7fb9ac818740 1 -- 172.21.15.3:0/1553028429 shutdown_connections 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.415+0000 7fb9ac818740 1 -- 172.21.15.3:0/1553028429 wait complete. 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.415+0000 7fb9ac818740 1 -- start start 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.415+0000 7fb9ac818740 1 Processor -- start 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.415+0000 7fb9ac818740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4b32f0 0x562baf8927c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.415+0000 7fb9ac818740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf892da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.415+0000 7fb9ac818740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4dc1b0 0x562baf9078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.415+0000 7fb9ac818740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01b0c50 con 0x562baf4bb4d0 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.415+0000 7fb9ac818740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01b0580 con 0x562baf4b32f0 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.415+0000 7fb9ac818740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01ad7c0 con 0x562baf4dc1b0 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb9ac818740 10 client.0 _ll_register_callbacks cb 0x7fb99801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb99effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf892da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb99effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf892da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57476/0 (socket says 172.21.15.3:57476) 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb99effd640 1 -- 172.21.15.3:0/2016399866 learned_addr learned my addr 172.21.15.3:0/2016399866 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4b32f0 0x562baf8927c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4dc1b0 0x562baf9078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb99effd640 1 -- 172.21.15.3:0/2016399866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4dc1b0 msgr2=0x562baf9078a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb99effd640 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4dc1b0 0x562baf9078a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb99effd640 1 -- 172.21.15.3:0/2016399866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4b32f0 msgr2=0x562baf8927c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb99effd640 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4b32f0 0x562baf8927c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb99effd640 1 -- 172.21.15.3:0/2016399866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562baf907de0 con 0x562baf4bb4d0 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4b32f0 0x562baf8927c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb99effd640 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf892da0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb998005b40 tx=0x7fb9980052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57471c889e297914 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb9867fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb9867fc640 1 -- 172.21.15.3:0/2016399866 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99801e030 con 0x562baf4bb4d0 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb9ac818740 1 -- 172.21.15.3:0/2016399866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb998003e20 con 0x562baf4bb4d0 2024-09-21T06:18:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb9867fc640 1 -- 172.21.15.3:0/2016399866 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb998003970 con 0x562baf4bb4d0 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb9867fc640 1 -- 172.21.15.3:0/2016399866 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb99800fac0 con 0x562baf4bb4d0 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb9ac818740 10 client.16578 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.416+0000 7fb9ac818740 1 -- 172.21.15.3:0/2016399866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562baf908fa0 con 0x562baf4bb4d0 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.417+0000 7fb9867fc640 1 -- 172.21.15.3:0/2016399866 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb998003e20 con 0x562baf4bb4d0 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.417+0000 7fb9867fc640 1 -- 172.21.15.3:0/2016399866 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb99803d070 con 0x562baf4bb4d0 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.417+0000 7fb9ac818740 10 client.16578 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.417+0000 7fb9ac818740 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.417+0000 7fb9ac818740 20 client.16578 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.417+0000 7fb9ac818740 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562baf4701c0 0x562baf90b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.417+0000 7fb9ac818740 4 client.16578 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.417+0000 7fb9ac818740 1 -- 172.21.15.3:0/2016399866 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562baf7f40f0 con 0x562baf4701c0 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.417+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562baf4701c0 0x562baf90b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.417+0000 7fb99e7fc640 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562baf4701c0 0x562baf90b5f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51129b4874d85637 server_cookie=a4a48a5e4a60cf3e in_seq=0 out_seq=0 2024-09-21T06:18:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.417+0000 7fb9867fc640 10 client.16578 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9867fc640 1 -- 172.21.15.3:0/2016399866 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562baf7f40f0 con 0x562baf4701c0 2024-09-21T06:18:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9867fc640 10 client.16578 handle_command_reply: tid=0 2024-09-21T06:18:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9ac818740 1 client.16578 shutdown 2024-09-21T06:18:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9ac818740 1 -- 172.21.15.3:0/2016399866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 msgr2=0x562baf892da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9ac818740 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf892da0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb998005b40 tx=0x7fb9980052f0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9ac818740 1 -- 172.21.15.3:0/2016399866 shutdown_connections 2024-09-21T06:18:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9ac818740 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562baf4701c0 0x562baf90b5f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9ac818740 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562baf4dc1b0 0x562baf9078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9ac818740 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562baf4bb4d0 0x562baf892da0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9867fc640 0 client.16578 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9ac818740 1 --2- 172.21.15.3:0/2016399866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562baf4b32f0 0x562baf8927c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9ac818740 1 -- 172.21.15.3:0/2016399866 >> 172.21.15.3:0/2016399866 conn(0x562baf4d0300 msgr2=0x562baf87f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9ac818740 1 -- 172.21.15.3:0/2016399866 shutdown_connections 2024-09-21T06:18:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9ac818740 1 -- 172.21.15.3:0/2016399866 wait complete. 2024-09-21T06:18:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.545+0000 7fb9ac818740 20 client.16578 trim_cache size 0 max 16384 2024-09-21T06:18:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:18:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:18:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:18:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:18:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.546+0000 7fb9a57c6640 1 -- 172.21.15.3:0/2546932471 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb98c0b6680 msgr2=0x7fb98c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.546+0000 7fb9a57c6640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb98c0b6680 0x7fb98c0b8b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb9900aabc0 tx=0x7fb990079500 comp rx=0 tx=0).stop 2024-09-21T06:18:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.546+0000 7fb9a57c6640 1 -- 172.21.15.3:0/2546932471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a010dd60 msgr2=0x7fb9a019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.546+0000 7fb9a57c6640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a010dd60 0x7fb9a019fd10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb99000e970 tx=0x7fb99000ed70 comp rx=0 tx=0).stop 2024-09-21T06:18:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.546+0000 7fb9a57c6640 1 -- 172.21.15.3:0/2546932471 shutdown_connections 2024-09-21T06:18:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.546+0000 7fb9a57c6640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb98c0b6680 0x7fb98c0b8b90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.546+0000 7fb9a57c6640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9a0105250 0x7fb9a019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.546+0000 7fb9a57c6640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a010dd60 0x7fb9a019fd10 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.546+0000 7fb9a57c6640 1 --2- 172.21.15.3:0/2546932471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9a0104900 0x7fb9a019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.546+0000 7fb9a57c6640 1 -- 172.21.15.3:0/2546932471 >> 172.21.15.3:0/2546932471 conn(0x7fb9a007a740 msgr2=0x7fb9a0107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.546+0000 7fb9a57c6640 1 -- 172.21.15.3:0/2546932471 shutdown_connections 2024-09-21T06:18:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:37.546+0000 7fb9a57c6640 1 -- 172.21.15.3:0/2546932471 wait complete. 2024-09-21T06:18:37.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:37 smithi003 ceph-mon[31227]: pgmap v433: 97 pgs: 97 active+clean; 69 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 420 op/s 2024-09-21T06:18:37.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:37 smithi003 ceph-mon[31227]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:18:37.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:37 smithi137 ceph-mon[34464]: pgmap v433: 97 pgs: 97 active+clean; 69 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 420 op/s 2024-09-21T06:18:37.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:37 smithi137 ceph-mon[34464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:18:37.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:37 smithi204 ceph-mon[34659]: pgmap v433: 97 pgs: 97 active+clean; 69 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 420 op/s 2024-09-21T06:18:37.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:37 smithi204 ceph-mon[34659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:18:37.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 318635 12.73 MB/sec execute 10 sec latency 48.876 ms 2024-09-21T06:18:38.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 320354 12.35 MB/sec execute 11 sec latency 13.710 ms 2024-09-21T06:18:39.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:39 smithi003 ceph-mon[31227]: pgmap v434: 97 pgs: 97 active+clean; 72 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 353 op/s 2024-09-21T06:18:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:39 smithi137 ceph-mon[34464]: pgmap v434: 97 pgs: 97 active+clean; 72 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 353 op/s 2024-09-21T06:18:39.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:39 smithi204 ceph-mon[34659]: pgmap v434: 97 pgs: 97 active+clean; 72 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 353 op/s 2024-09-21T06:18:39.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 323187 12.75 MB/sec execute 12 sec latency 15.032 ms 2024-09-21T06:18:40.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 326312 13.11 MB/sec execute 13 sec latency 11.346 ms 2024-09-21T06:18:41.290 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:18:41.574 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:41.589 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:41 smithi003 ceph-mon[31227]: pgmap v435: 97 pgs: 97 active+clean; 73 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 412 op/s 2024-09-21T06:18:41.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:41 smithi137 ceph-mon[34464]: pgmap v435: 97 pgs: 97 active+clean; 73 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 412 op/s 2024-09-21T06:18:41.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:41 smithi204 ceph-mon[34659]: pgmap v435: 97 pgs: 97 active+clean; 73 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 412 op/s 2024-09-21T06:18:41.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 327492 12.44 MB/sec execute 14 sec latency 96.984 ms 2024-09-21T06:18:42.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.115+0000 7fb812264640 1 -- 172.21.15.3:0/1557361794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1048c0 msgr2=0x7fb80c104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:42.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.115+0000 7fb812264640 1 --2- 172.21.15.3:0/1557361794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1048c0 0x7fb80c104cd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009900 tx=0x7fb7fc02a410 comp rx=0 tx=0).stop 2024-09-21T06:18:42.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.116+0000 7fb812264640 1 -- 172.21.15.3:0/1557361794 shutdown_connections 2024-09-21T06:18:42.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.116+0000 7fb812264640 1 --2- 172.21.15.3:0/1557361794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb80c10de70 0x7fb80c110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.116+0000 7fb812264640 1 --2- 172.21.15.3:0/1557361794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb80c1052a0 0x7fb80c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.116+0000 7fb812264640 1 --2- 172.21.15.3:0/1557361794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1048c0 0x7fb80c104cd0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.116+0000 7fb812264640 1 -- 172.21.15.3:0/1557361794 >> 172.21.15.3:0/1557361794 conn(0x7fb80c0797b0 msgr2=0x7fb80c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:42.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.116+0000 7fb812264640 1 -- 172.21.15.3:0/1557361794 shutdown_connections 2024-09-21T06:18:42.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.116+0000 7fb812264640 1 -- 172.21.15.3:0/1557361794 wait complete. 2024-09-21T06:18:42.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.117+0000 7fb812264640 1 Processor -- start 2024-09-21T06:18:42.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.117+0000 7fb812264640 1 -- start start 2024-09-21T06:18:42.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.117+0000 7fb812264640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb80c1048c0 0x7fb80c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.117+0000 7fb812264640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb80c1052a0 0x7fb80c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.117+0000 7fb812264640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c10de70 0x7fb80c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.117+0000 7fb812264640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c113b00 con 0x7fb80c10de70 2024-09-21T06:18:42.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.117+0000 7fb812264640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c113980 con 0x7fb80c1048c0 2024-09-21T06:18:42.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.117+0000 7fb812264640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c113c80 con 0x7fb80c1052a0 2024-09-21T06:18:42.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.117+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb80c1048c0 0x7fb80c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.117+0000 7fb810801640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c10de70 0x7fb80c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.117+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb80c1048c0 0x7fb80c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49958/0 (socket says 172.21.15.3:49958) 2024-09-21T06:18:42.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.117+0000 7fb80bfff640 1 -- 172.21.15.3:0/4000308922 learned_addr learned my addr 172.21.15.3:0/4000308922 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:42.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.117+0000 7fb8037fe640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb80c1052a0 0x7fb80c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.118+0000 7fb810801640 1 -- 172.21.15.3:0/4000308922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb80c1052a0 msgr2=0x7fb80c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:42.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.118+0000 7fb810801640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb80c1052a0 0x7fb80c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.118+0000 7fb810801640 1 -- 172.21.15.3:0/4000308922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb80c1048c0 msgr2=0x7fb80c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:42.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.118+0000 7fb810801640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb80c1048c0 0x7fb80c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.118+0000 7fb810801640 1 -- 172.21.15.3:0/4000308922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c1a0440 con 0x7fb80c10de70 2024-09-21T06:18:42.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.118+0000 7fb80bfff640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb80c1048c0 0x7fb80c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.118+0000 7fb810801640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c10de70 0x7fb80c19fd10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb804009fd0 tx=0x7fb80400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.118+0000 7fb809ffb640 1 -- 172.21.15.3:0/4000308922 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80400ce20 con 0x7fb80c10de70 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.118+0000 7fb812264640 1 -- 172.21.15.3:0/4000308922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80c1a0730 con 0x7fb80c10de70 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.118+0000 7fb809ffb640 1 -- 172.21.15.3:0/4000308922 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb804010430 con 0x7fb80c10de70 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.118+0000 7fb809ffb640 1 -- 172.21.15.3:0/4000308922 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80400f620 con 0x7fb80c10de70 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.118+0000 7fb812264640 1 -- 172.21.15.3:0/4000308922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80c1a6e90 con 0x7fb80c10de70 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.121+0000 7fb809ffb640 1 -- 172.21.15.3:0/4000308922 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fb8040108e0 con 0x7fb80c10de70 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.121+0000 7fb8187d8740 1 Processor -- start 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.121+0000 7fb8187d8740 1 -- start start 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8187d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5ecf070 0x5578a6276650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8187d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ea61b0 0x5578a6276b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8187d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5eae390 0x5578a62770d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8187d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5578a61ece50 con 0x5578a5ea61b0 2024-09-21T06:18:42.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8187d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5578a61dc280 con 0x5578a5ecf070 2024-09-21T06:18:42.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8187d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5578a61e0090 con 0x5578a5eae390 2024-09-21T06:18:42.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5ecf070 0x5578a6276650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5ecf070 0x5578a6276650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49970/0 (socket says 172.21.15.3:49970) 2024-09-21T06:18:42.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8037fe640 1 -- 172.21.15.3:0/301187393 learned_addr learned my addr 172.21.15.3:0/301187393 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:42.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8037fe640 1 --2- 172.21.15.3:0/301187393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5eae390 0x5578a62770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb80bfff640 1 --2- 172.21.15.3:0/301187393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ea61b0 0x5578a6276b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8037fe640 1 -- 172.21.15.3:0/301187393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5eae390 msgr2=0x5578a62770d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:42.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8037fe640 1 --2- 172.21.15.3:0/301187393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5eae390 0x5578a62770d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8037fe640 1 -- 172.21.15.3:0/301187393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ea61b0 msgr2=0x5578a6276b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:42.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8037fe640 1 --2- 172.21.15.3:0/301187393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ea61b0 0x5578a6276b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.122+0000 7fb8037fe640 1 -- 172.21.15.3:0/301187393 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a5f3a650 con 0x5578a5ecf070 2024-09-21T06:18:42.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.123+0000 7fb809ffb640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb7f80b6680 0x7fb7f80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.123+0000 7fb8037fe640 1 --2- 172.21.15.3:0/301187393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5ecf070 0x5578a6276650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4011710 tx=0x7fb7f4005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3058359d1d26ef6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:42.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.123+0000 7fb80bfff640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb7f80b6680 0x7fb7f80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.123+0000 7fb809ffb640 1 -- 172.21.15.3:0/4000308922 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb8040e79f0 con 0x7fb80c10de70 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.123+0000 7fb8027fc640 1 -- 172.21.15.3:0/301187393 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400f840 con 0x5578a5ecf070 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.123+0000 7fb8027fc640 1 -- 172.21.15.3:0/301187393 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f400fe90 con 0x5578a5ecf070 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.123+0000 7fb8027fc640 1 -- 172.21.15.3:0/301187393 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f401fca0 con 0x5578a5ecf070 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.124+0000 7fb8187d8740 1 -- 172.21.15.3:0/301187393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5ecf070 msgr2=0x5578a6276650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.124+0000 7fb80bfff640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb7f80b6680 0x7fb7f80b8b90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5578a62752d0 tx=0x7fb7fc005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.124+0000 7fb8187d8740 1 --2- 172.21.15.3:0/301187393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5ecf070 0x5578a6276650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4011710 tx=0x7fb7f4005850 comp rx=0 tx=0).stop 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.124+0000 7fb8187d8740 1 -- 172.21.15.3:0/301187393 shutdown_connections 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.124+0000 7fb8187d8740 1 --2- 172.21.15.3:0/301187393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5eae390 0x5578a62770d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.124+0000 7fb8187d8740 1 --2- 172.21.15.3:0/301187393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ea61b0 0x5578a6276b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.124+0000 7fb8187d8740 1 --2- 172.21.15.3:0/301187393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5ecf070 0x5578a6276650 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.124+0000 7fb8187d8740 1 -- 172.21.15.3:0/301187393 >> 172.21.15.3:0/301187393 conn(0x5578a5ec3460 msgr2=0x5578a6274540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.125+0000 7fb8187d8740 1 -- 172.21.15.3:0/301187393 shutdown_connections 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.125+0000 7fb8187d8740 1 -- 172.21.15.3:0/301187393 wait complete. 2024-09-21T06:18:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.125+0000 7fb8187d8740 1 -- start start 2024-09-21T06:18:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.125+0000 7fb8187d8740 1 Processor -- start 2024-09-21T06:18:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.125+0000 7fb8187d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5ea61b0 0x5578a6281d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.125+0000 7fb8187d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5eae390 0x5578a62822b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.125+0000 7fb8187d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ecf070 0x5578a6292560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb8187d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5578a603d700 con 0x5578a5ecf070 2024-09-21T06:18:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb8187d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5578a6177110 con 0x5578a5ea61b0 2024-09-21T06:18:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb8187d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5578a6289240 con 0x5578a5eae390 2024-09-21T06:18:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5ea61b0 0x5578a6281d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb8037fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5eae390 0x5578a62822b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb8037fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5eae390 0x5578a62822b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:42890/0 (socket says 172.21.15.3:42890) 2024-09-21T06:18:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb8037fe640 1 -- 172.21.15.3:0/1230065486 learned_addr learned my addr 172.21.15.3:0/1230065486 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5ea61b0 0x5578a6281d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49980/0 (socket says 172.21.15.3:49980) 2024-09-21T06:18:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb810801640 1 --2- 172.21.15.3:0/1230065486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ecf070 0x5578a6292560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb8187d8740 10 client.0 _ll_register_callbacks cb 0x5578a6190990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb8037fe640 1 -- 172.21.15.3:0/1230065486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5ea61b0 msgr2=0x5578a6281d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb8037fe640 1 --2- 172.21.15.3:0/1230065486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5ea61b0 0x5578a6281d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb8037fe640 1 -- 172.21.15.3:0/1230065486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ecf070 msgr2=0x5578a6292560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb8037fe640 1 --2- 172.21.15.3:0/1230065486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ecf070 0x5578a6292560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb8037fe640 1 -- 172.21.15.3:0/1230065486 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a61a4c40 con 0x5578a5eae390 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb8037fe640 1 --2- 172.21.15.3:0/1230065486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5eae390 0x5578a62822b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb7f401f940 tx=0x7fb7f4005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f49c09df0e2bc481 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb7eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.126+0000 7fb7eb7fe640 1 -- 172.21.15.3:0/1230065486 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f402e060 con 0x5578a5eae390 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.127+0000 7fb8187d8740 1 -- 172.21.15.3:0/1230065486 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f400fe90 con 0x5578a5eae390 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.127+0000 7fb7eb7fe640 1 -- 172.21.15.3:0/1230065486 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb7f401fd70 con 0x5578a5eae390 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.127+0000 7fb7eb7fe640 1 -- 172.21.15.3:0/1230065486 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb7f4011020 con 0x5578a5eae390 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.127+0000 7fb8187d8740 10 client.26128 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:42.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.127+0000 7fb8187d8740 1 -- 172.21.15.3:0/1230065486 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578a6294f40 con 0x5578a5eae390 2024-09-21T06:18:42.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.127+0000 7fb7eb7fe640 1 -- 172.21.15.3:0/1230065486 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f400fe90 con 0x5578a5eae390 2024-09-21T06:18:42.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.127+0000 7fb7eb7fe640 1 -- 172.21.15.3:0/1230065486 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb7f4028030 con 0x5578a5eae390 2024-09-21T06:18:42.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.127+0000 7fb8187d8740 10 client.26128 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:42.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.127+0000 7fb8187d8740 10 client.26128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:42.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.127+0000 7fb8187d8740 20 client.26128 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:42.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.127+0000 7fb8187d8740 1 --2- 172.21.15.3:0/1230065486 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5578a5e8d9f0 0x5578a62982a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.127+0000 7fb8187d8740 4 client.26128 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:42.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.127+0000 7fb8187d8740 1 -- 172.21.15.3:0/1230065486 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578a61eeda0 con 0x5578a5e8d9f0 2024-09-21T06:18:42.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.128+0000 7fb80bfff640 1 --2- 172.21.15.3:0/1230065486 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5578a5e8d9f0 0x5578a62982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.128+0000 7fb7eb7fe640 10 client.26128 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:42.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.128+0000 7fb80bfff640 1 --2- 172.21.15.3:0/1230065486 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5578a5e8d9f0 0x5578a62982a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b99757c017ac7ce server_cookie=c4e8ce680cee3c88 in_seq=0 out_seq=0 2024-09-21T06:18:42.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb7eb7fe640 1 -- 172.21.15.3:0/1230065486 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5578a61eeda0 con 0x5578a5e8d9f0 2024-09-21T06:18:42.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb7eb7fe640 10 client.26128 handle_command_reply: tid=0 2024-09-21T06:18:42.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb8187d8740 1 client.26128 shutdown 2024-09-21T06:18:42.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb8187d8740 1 -- 172.21.15.3:0/1230065486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5eae390 msgr2=0x5578a62822b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:42.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb8187d8740 1 --2- 172.21.15.3:0/1230065486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5eae390 0x5578a62822b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb7f401f940 tx=0x7fb7f4005760 comp rx=0 tx=0).stop 2024-09-21T06:18:42.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb8187d8740 1 -- 172.21.15.3:0/1230065486 shutdown_connections 2024-09-21T06:18:42.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb8187d8740 1 --2- 172.21.15.3:0/1230065486 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5578a5e8d9f0 0x5578a62982a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb8187d8740 1 --2- 172.21.15.3:0/1230065486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5eae390 0x5578a62822b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb8187d8740 1 --2- 172.21.15.3:0/1230065486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ecf070 0x5578a6292560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb8187d8740 1 --2- 172.21.15.3:0/1230065486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5ea61b0 0x5578a6281d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb8187d8740 1 -- 172.21.15.3:0/1230065486 >> 172.21.15.3:0/1230065486 conn(0x5578a5ec3460 msgr2=0x5578a62797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:42.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb7eb7fe640 0 client.26128 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:42.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb8187d8740 1 -- 172.21.15.3:0/1230065486 shutdown_connections 2024-09-21T06:18:42.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.129+0000 7fb8187d8740 1 -- 172.21.15.3:0/1230065486 wait complete. 2024-09-21T06:18:42.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.130+0000 7fb8187d8740 20 client.26128 trim_cache size 0 max 16384 2024-09-21T06:18:42.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8187d8740 1 Processor -- start 2024-09-21T06:18:42.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8187d8740 1 -- start start 2024-09-21T06:18:42.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8187d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5ea61b0 0x5578a62839a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8187d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5eae390 0x5578a6292460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8187d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ecf070 0x5578a627ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8187d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1ad490 con 0x5578a5ecf070 2024-09-21T06:18:42.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8187d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1af000 con 0x5578a5eae390 2024-09-21T06:18:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8187d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1adb60 con 0x5578a5ea61b0 2024-09-21T06:18:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb810801640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ecf070 0x5578a627ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5eae390 0x5578a6292460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb810801640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ecf070 0x5578a627ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57530/0 (socket says 172.21.15.3:57530) 2024-09-21T06:18:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5ea61b0 0x5578a62839a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb810801640 1 -- 172.21.15.3:0/1040681186 learned_addr learned my addr 172.21.15.3:0/1040681186 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8037fe640 1 -- 172.21.15.3:0/1040681186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5ea61b0 msgr2=0x5578a62839a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8037fe640 1 --2- 172.21.15.3:0/1040681186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5ea61b0 0x5578a62839a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8037fe640 1 -- 172.21.15.3:0/1040681186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ecf070 msgr2=0x5578a627ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8037fe640 1 --2- 172.21.15.3:0/1040681186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ecf070 0x5578a627ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb8037fe640 1 -- 172.21.15.3:0/1040681186 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a6283030 con 0x5578a5eae390 2024-09-21T06:18:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.150+0000 7fb80bfff640 1 --2- 172.21.15.3:0/1040681186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5ea61b0 0x5578a62839a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:42.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.151+0000 7fb8037fe640 1 --2- 172.21.15.3:0/1040681186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5eae390 0x5578a6292460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009730 tx=0x7fb7f401f8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a93e8b116d40e25a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.151+0000 7fb801ffb640 1 -- 172.21.15.3:0/1040681186 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f402e040 con 0x5578a5eae390 2024-09-21T06:18:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.151+0000 7fb801ffb640 1 -- 172.21.15.3:0/1040681186 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f401f970 con 0x5578a5eae390 2024-09-21T06:18:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.151+0000 7fb801ffb640 1 -- 172.21.15.3:0/1040681186 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400f9f0 con 0x5578a5eae390 2024-09-21T06:18:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.152+0000 7fb8187d8740 1 -- 172.21.15.3:0/1040681186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5eae390 msgr2=0x5578a6292460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.152+0000 7fb8187d8740 1 --2- 172.21.15.3:0/1040681186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5eae390 0x5578a6292460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009730 tx=0x7fb7f401f8b0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.152+0000 7fb8187d8740 1 -- 172.21.15.3:0/1040681186 shutdown_connections 2024-09-21T06:18:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.152+0000 7fb8187d8740 1 --2- 172.21.15.3:0/1040681186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ecf070 0x5578a627ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.152+0000 7fb8187d8740 1 --2- 172.21.15.3:0/1040681186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5eae390 0x5578a6292460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.152+0000 7fb8187d8740 1 --2- 172.21.15.3:0/1040681186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5ea61b0 0x5578a62839a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.152+0000 7fb8187d8740 1 -- 172.21.15.3:0/1040681186 >> 172.21.15.3:0/1040681186 conn(0x5578a5ec3460 msgr2=0x5578a6283290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.152+0000 7fb8187d8740 1 -- 172.21.15.3:0/1040681186 shutdown_connections 2024-09-21T06:18:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.152+0000 7fb8187d8740 1 -- 172.21.15.3:0/1040681186 wait complete. 2024-09-21T06:18:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.152+0000 7fb8187d8740 1 -- start start 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.153+0000 7fb8187d8740 1 Processor -- start 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.153+0000 7fb8187d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ea61b0 0x5578a6289c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.153+0000 7fb8187d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5eae390 0x5578a628a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.153+0000 7fb8187d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5ecf070 0x5578a6300b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.153+0000 7fb8187d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1ae900 con 0x5578a5ea61b0 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.153+0000 7fb8037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5eae390 0x5578a628a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.153+0000 7fb8187d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1ae230 con 0x5578a5eae390 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.153+0000 7fb8037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5eae390 0x5578a628a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:49996/0 (socket says 172.21.15.3:49996) 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.153+0000 7fb8187d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1ab470 con 0x5578a5ecf070 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.153+0000 7fb8037fe640 1 -- 172.21.15.3:0/3419409181 learned_addr learned my addr 172.21.15.3:0/3419409181 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.153+0000 7fb810801640 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5ecf070 0x5578a6300b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.153+0000 7fb80bfff640 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ea61b0 0x5578a6289c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb8187d8740 10 client.0 _ll_register_callbacks cb 0x7fb7f4009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb810801640 1 -- 172.21.15.3:0/3419409181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5eae390 msgr2=0x5578a628a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb810801640 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5eae390 0x5578a628a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb810801640 1 -- 172.21.15.3:0/3419409181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ea61b0 msgr2=0x5578a6289c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb810801640 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ea61b0 0x5578a6289c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb810801640 1 -- 172.21.15.3:0/3419409181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a6301130 con 0x5578a5ecf070 2024-09-21T06:18:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb8037fe640 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5eae390 0x5578a628a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb80bfff640 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ea61b0 0x5578a6289c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb810801640 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5ecf070 0x5578a6300b40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb804011fd0 tx=0x7fb8040ad6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc44ade990dbd1f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb7eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:18:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb7eb7fe640 1 -- 172.21.15.3:0/3419409181 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8040adf80 con 0x5578a5ecf070 2024-09-21T06:18:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb7eb7fe640 1 -- 172.21.15.3:0/3419409181 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb8040a91d0 con 0x5578a5ecf070 2024-09-21T06:18:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb8187d8740 1 -- 172.21.15.3:0/3419409181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f401f970 con 0x5578a5ecf070 2024-09-21T06:18:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb7eb7fe640 1 -- 172.21.15.3:0/3419409181 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb8040b2360 con 0x5578a5ecf070 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb8187d8740 10 client.26134 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.154+0000 7fb8187d8740 1 -- 172.21.15.3:0/3419409181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578a63023e0 con 0x5578a5ecf070 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.155+0000 7fb7eb7fe640 1 -- 172.21.15.3:0/3419409181 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f401f970 con 0x5578a5ecf070 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.155+0000 7fb7eb7fe640 1 -- 172.21.15.3:0/3419409181 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb80401f020 con 0x5578a5ecf070 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.155+0000 7fb8187d8740 10 client.26134 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.155+0000 7fb8187d8740 10 client.26134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.155+0000 7fb8187d8740 20 client.26134 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.155+0000 7fb8187d8740 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5578a5e62da0 0x5578a6304b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.155+0000 7fb8187d8740 4 client.26134 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.155+0000 7fb8187d8740 1 -- 172.21.15.3:0/3419409181 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578a61ea850 con 0x5578a5e62da0 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.155+0000 7fb80bfff640 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5578a5e62da0 0x5578a6304b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.155+0000 7fb7eb7fe640 10 client.26134 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.156+0000 7fb80bfff640 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5578a5e62da0 0x5578a6304b70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10a8a58a9d760c46 server_cookie=58bc5ff30b8e76dc in_seq=0 out_seq=0 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.156+0000 7fb7eb7fe640 1 -- 172.21.15.3:0/3419409181 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x5578a61ea850 con 0x5578a5e62da0 2024-09-21T06:18:42.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.156+0000 7fb7eb7fe640 10 client.26134 handle_command_reply: tid=0 2024-09-21T06:18:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.157+0000 7fb8187d8740 1 client.26134 shutdown 2024-09-21T06:18:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.157+0000 7fb8187d8740 1 -- 172.21.15.3:0/3419409181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5ecf070 msgr2=0x5578a6300b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.157+0000 7fb8187d8740 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5ecf070 0x5578a6300b40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb804011fd0 tx=0x7fb8040ad6b0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.157+0000 7fb8187d8740 1 -- 172.21.15.3:0/3419409181 shutdown_connections 2024-09-21T06:18:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.157+0000 7fb8187d8740 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5578a5e62da0 0x5578a6304b70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.157+0000 7fb8187d8740 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a5ecf070 0x5578a6300b40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.157+0000 7fb8187d8740 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578a5eae390 0x5578a628a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.157+0000 7fb7eb7fe640 0 client.26134 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.157+0000 7fb8187d8740 1 --2- 172.21.15.3:0/3419409181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578a5ea61b0 0x5578a6289c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.157+0000 7fb8187d8740 1 -- 172.21.15.3:0/3419409181 >> 172.21.15.3:0/3419409181 conn(0x5578a5ec3460 msgr2=0x5578a6277620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.157+0000 7fb8187d8740 1 -- 172.21.15.3:0/3419409181 shutdown_connections 2024-09-21T06:18:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.157+0000 7fb8187d8740 1 -- 172.21.15.3:0/3419409181 wait complete. 2024-09-21T06:18:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.157+0000 7fb8187d8740 20 client.26134 trim_cache size 0 max 16384 2024-09-21T06:18:42.165 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:18:42.165 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:18:42.165 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:18:42.165 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:18:42.165 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:18:42.165 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:18:42.165 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:42.165 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:42.165 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:42.165 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:42.165 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:42.165 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:42.166 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:18:42.166 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:18:42.166 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:42.166 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:42.166 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:42.166 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:42.166 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:42.166 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:42.166 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:18:42.166 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:18:42.166 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:42.166 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:42.166 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:42.167 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:18:42.168 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:18:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.158+0000 7fb812264640 1 -- 172.21.15.3:0/4000308922 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb7f80b6680 msgr2=0x7fb7f80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.158+0000 7fb812264640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb7f80b6680 0x7fb7f80b8b90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5578a62752d0 tx=0x7fb7fc005a20 comp rx=0 tx=0).stop 2024-09-21T06:18:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.158+0000 7fb812264640 1 -- 172.21.15.3:0/4000308922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c10de70 msgr2=0x7fb80c19fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.158+0000 7fb812264640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c10de70 0x7fb80c19fd10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb804009fd0 tx=0x7fb80400ef10 comp rx=0 tx=0).stop 2024-09-21T06:18:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.158+0000 7fb812264640 1 -- 172.21.15.3:0/4000308922 shutdown_connections 2024-09-21T06:18:42.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.158+0000 7fb812264640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb7f80b6680 0x7fb7f80b8b90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.158+0000 7fb812264640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb80c1052a0 0x7fb80c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.158+0000 7fb812264640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c10de70 0x7fb80c19fd10 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.158+0000 7fb812264640 1 --2- 172.21.15.3:0/4000308922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb80c1048c0 0x7fb80c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:42.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.158+0000 7fb812264640 1 -- 172.21.15.3:0/4000308922 >> 172.21.15.3:0/4000308922 conn(0x7fb80c0797b0 msgr2=0x7fb80c1079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:42.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.158+0000 7fb812264640 1 -- 172.21.15.3:0/4000308922 shutdown_connections 2024-09-21T06:18:42.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:42.158+0000 7fb812264640 1 -- 172.21.15.3:0/4000308922 wait complete. 2024-09-21T06:18:42.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 330253 12.75 MB/sec execute 15 sec latency 13.208 ms 2024-09-21T06:18:42.985 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:18:42.985 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:18:43.273 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:43.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:43 smithi003 ceph-mon[31227]: pgmap v436: 97 pgs: 97 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 414 op/s 2024-09-21T06:18:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.854+0000 7f03e838b640 1 -- 172.21.15.3:0/3507115981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03e0105300 msgr2=0x7f03e010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.854+0000 7f03e838b640 1 --2- 172.21.15.3:0/3507115981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03e0105300 0x7f03e010d850 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009900 tx=0x7f03d002a410 comp rx=0 tx=0).stop 2024-09-21T06:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.854+0000 7f03e838b640 1 -- 172.21.15.3:0/3507115981 shutdown_connections 2024-09-21T06:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.854+0000 7f03e838b640 1 --2- 172.21.15.3:0/3507115981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03e010de10 0x7f03e0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.854+0000 7f03e838b640 1 --2- 172.21.15.3:0/3507115981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03e0105300 0x7f03e010d850 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.854+0000 7f03e838b640 1 --2- 172.21.15.3:0/3507115981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03e01049b0 0x7f03e0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.854+0000 7f03e838b640 1 -- 172.21.15.3:0/3507115981 >> 172.21.15.3:0/3507115981 conn(0x7f03e0079730 msgr2=0x7f03e0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.855+0000 7f03e838b640 1 -- 172.21.15.3:0/3507115981 shutdown_connections 2024-09-21T06:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.855+0000 7f03e838b640 1 -- 172.21.15.3:0/3507115981 wait complete. 2024-09-21T06:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.855+0000 7f03e838b640 1 Processor -- start 2024-09-21T06:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.855+0000 7f03e838b640 1 -- start start 2024-09-21T06:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e838b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03e01049b0 0x7f03e019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e838b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03e0105300 0x7f03e019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e838b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03e010de10 0x7f03e019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e838b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f03e0113a70 con 0x7f03e010de10 2024-09-21T06:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e838b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01138f0 con 0x7f03e01049b0 2024-09-21T06:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e838b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f03e0113bf0 con 0x7f03e0105300 2024-09-21T06:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e6127640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03e01049b0 0x7f03e019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e6127640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03e01049b0 0x7f03e019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:50002/0 (socket says 172.21.15.3:50002) 2024-09-21T06:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e6928640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03e010de10 0x7f03e019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e6127640 1 -- 172.21.15.3:0/1667378769 learned_addr learned my addr 172.21.15.3:0/1667378769 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e5926640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03e0105300 0x7f03e019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e6127640 1 -- 172.21.15.3:0/1667378769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03e0105300 msgr2=0x7f03e019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e6127640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03e0105300 0x7f03e019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e6127640 1 -- 172.21.15.3:0/1667378769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03e010de10 msgr2=0x7f03e019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e6127640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03e010de10 0x7f03e019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e6127640 1 -- 172.21.15.3:0/1667378769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e01a0410 con 0x7f03e01049b0 2024-09-21T06:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.856+0000 7f03e6928640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03e010de10 0x7f03e019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.857+0000 7f03e6127640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03e01049b0 0x7f03e019a4a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00e9c0 tx=0x7f03dc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.857+0000 7f03d77fe640 1 -- 172.21.15.3:0/1667378769 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00cd80 con 0x7f03e01049b0 2024-09-21T06:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.857+0000 7f03d77fe640 1 -- 172.21.15.3:0/1667378769 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03dc0035f0 con 0x7f03e01049b0 2024-09-21T06:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.857+0000 7f03e838b640 1 -- 172.21.15.3:0/1667378769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e01a0700 con 0x7f03e01049b0 2024-09-21T06:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.857+0000 7f03d77fe640 1 -- 172.21.15.3:0/1667378769 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc010690 con 0x7f03e01049b0 2024-09-21T06:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.857+0000 7f03e838b640 1 -- 172.21.15.3:0/1667378769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e01a6e60 con 0x7f03e01049b0 2024-09-21T06:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.858+0000 7f03e838b640 1 -- 172.21.15.3:0/1667378769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e0104dc0 con 0x7f03e01049b0 2024-09-21T06:18:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.860+0000 7f03d77fe640 1 -- 172.21.15.3:0/1667378769 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f03dc004000 con 0x7f03e01049b0 2024-09-21T06:18:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.860+0000 7f03d77fe640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f03cc0b6680 0x7f03cc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.860+0000 7f03e5926640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f03cc0b6680 0x7f03cc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.860+0000 7f03d77fe640 1 -- 172.21.15.3:0/1667378769 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f03dc0e7d00 con 0x7f03e01049b0 2024-09-21T06:18:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.861+0000 7f03e5926640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f03cc0b6680 0x7f03cc0b8b90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009230 tx=0x7f03d0031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:43.863+0000 7f03d77fe640 1 -- 172.21.15.3:0/1667378769 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f03dc0ae290 con 0x7f03e01049b0 2024-09-21T06:18:43.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:43 smithi137 ceph-mon[34464]: pgmap v436: 97 pgs: 97 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 414 op/s 2024-09-21T06:18:43.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:43 smithi204 ceph-mon[34659]: pgmap v436: 97 pgs: 97 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 414 op/s 2024-09-21T06:18:43.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 332895 12.74 MB/sec execute 16 sec latency 24.737 ms 2024-09-21T06:18:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.104+0000 7f03e838b640 1 -- 172.21.15.3:0/1667378769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03e019b5d0 con 0x7f03e01049b0 2024-09-21T06:18:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.104+0000 7f03d77fe640 1 -- 172.21.15.3:0/1667378769 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11963 (secure 0 0 0) 0x7f03dc0b2670 con 0x7f03e01049b0 2024-09-21T06:18:44.107 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:18:44.107 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"btime":"2024-09-21T06:18:36:459109+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:18:35.916019+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24670,24674,24679,24691,24704]},"id":1}]} 2024-09-21T06:18:44.107 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 30 2024-09-21T06:18:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.107+0000 7f03e838b640 1 -- 172.21.15.3:0/1667378769 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f03cc0b6680 msgr2=0x7f03cc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.107+0000 7f03e838b640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f03cc0b6680 0x7f03cc0b8b90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009230 tx=0x7f03d0031400 comp rx=0 tx=0).stop 2024-09-21T06:18:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.107+0000 7f03e838b640 1 -- 172.21.15.3:0/1667378769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03e01049b0 msgr2=0x7f03e019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.107+0000 7f03e838b640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03e01049b0 0x7f03e019a4a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00e9c0 tx=0x7f03dc00ee80 comp rx=0 tx=0).stop 2024-09-21T06:18:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.107+0000 7f03e838b640 1 -- 172.21.15.3:0/1667378769 shutdown_connections 2024-09-21T06:18:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.107+0000 7f03e838b640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f03cc0b6680 0x7f03cc0b8b90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.107+0000 7f03e838b640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03e0105300 0x7f03e019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.107+0000 7f03e838b640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03e010de10 0x7f03e019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.107+0000 7f03e838b640 1 --2- 172.21.15.3:0/1667378769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03e01049b0 0x7f03e019a4a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:44.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.107+0000 7f03e838b640 1 -- 172.21.15.3:0/1667378769 >> 172.21.15.3:0/1667378769 conn(0x7f03e0079730 msgr2=0x7f03e00ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:44.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.107+0000 7f03e838b640 1 -- 172.21.15.3:0/1667378769 shutdown_connections 2024-09-21T06:18:44.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:44.107+0000 7f03e838b640 1 -- 172.21.15.3:0/1667378769 wait complete. 2024-09-21T06:18:44.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:44 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1667378769' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:18:44.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:44 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1667378769' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:18:44.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:44 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1667378769' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:18:44.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 334679 12.50 MB/sec execute 17 sec latency 15.987 ms 2024-09-21T06:18:45.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:18:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:45.815 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:45 smithi003 ceph-mon[31227]: pgmap v437: 97 pgs: 97 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-09-21T06:18:45.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:45 smithi137 ceph-mon[34464]: pgmap v437: 97 pgs: 97 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-09-21T06:18:45.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:45 smithi204 ceph-mon[34659]: pgmap v437: 97 pgs: 97 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-09-21T06:18:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.912+0000 7f8d5c246640 1 -- 172.21.15.3:0/264660960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d54109e20 msgr2=0x7f8d54110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.912+0000 7f8d5c246640 1 --2- 172.21.15.3:0/264660960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d54109e20 0x7f8d54110650 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009ca0 tx=0x7f8d4c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.912+0000 7f8d5c246640 1 -- 172.21.15.3:0/264660960 shutdown_connections 2024-09-21T06:18:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.912+0000 7f8d5c246640 1 --2- 172.21.15.3:0/264660960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d54109e20 0x7f8d54110650 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.912+0000 7f8d5c246640 1 --2- 172.21.15.3:0/264660960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d541094d0 0x7f8d541098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.912+0000 7f8d5c246640 1 --2- 172.21.15.3:0/264660960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54103b90 0x7f8d54103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.912+0000 7f8d5c246640 1 -- 172.21.15.3:0/264660960 >> 172.21.15.3:0/264660960 conn(0x7f8d540ff5e0 msgr2=0x7f8d54101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.912+0000 7f8d5c246640 1 -- 172.21.15.3:0/264660960 shutdown_connections 2024-09-21T06:18:45.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.912+0000 7f8d5c246640 1 -- 172.21.15.3:0/264660960 wait complete. 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d5c246640 1 Processor -- start 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d5c246640 1 -- start start 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d5c246640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d54103b90 0x7f8d5419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d5c246640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d541094d0 0x7f8d5419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d5c246640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54109e20 0x7f8d541a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d5c246640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8d54113ae0 con 0x7f8d54103b90 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d5c246640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8d54113960 con 0x7f8d541094d0 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d5c246640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d54113c60 con 0x7f8d54109e20 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d59fe2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d54103b90 0x7f8d5419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d59fe2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d54103b90 0x7f8d5419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33390/0 (socket says 172.21.15.3:33390) 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d59fe2640 1 -- 172.21.15.3:0/279983796 learned_addr learned my addr 172.21.15.3:0/279983796 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54109e20 0x7f8d541a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d597e1640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d541094d0 0x7f8d5419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d59fe2640 1 -- 172.21.15.3:0/279983796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54109e20 msgr2=0x7f8d541a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d59fe2640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54109e20 0x7f8d541a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d59fe2640 1 -- 172.21.15.3:0/279983796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d541094d0 msgr2=0x7f8d5419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d59fe2640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d541094d0 0x7f8d5419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.913+0000 7f8d59fe2640 1 -- 172.21.15.3:0/279983796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d541a25e0 con 0x7f8d54103b90 2024-09-21T06:18:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.914+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54109e20 0x7f8d541a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.914+0000 7f8d59fe2640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d54103b90 0x7f8d5419c670 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000e9f0 tx=0x7f8d5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.914+0000 7f8d4affd640 1 -- 172.21.15.3:0/279983796 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5000cd80 con 0x7f8d54103b90 2024-09-21T06:18:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.914+0000 7f8d5c246640 1 -- 172.21.15.3:0/279983796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d541a28d0 con 0x7f8d54103b90 2024-09-21T06:18:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.914+0000 7f8d4affd640 1 -- 172.21.15.3:0/279983796 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d500108e0 con 0x7f8d54103b90 2024-09-21T06:18:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.914+0000 7f8d4affd640 1 -- 172.21.15.3:0/279983796 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50018980 con 0x7f8d54103b90 2024-09-21T06:18:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.914+0000 7f8d5c246640 1 -- 172.21.15.3:0/279983796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d541a9030 con 0x7f8d54103b90 2024-09-21T06:18:45.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.917+0000 7f8d627ba740 1 Processor -- start 2024-09-21T06:18:45.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.917+0000 7f8d627ba740 1 -- start start 2024-09-21T06:18:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.918+0000 7f8d4affd640 1 -- 172.21.15.3:0/279983796 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f8d50018b20 con 0x7f8d54103b90 2024-09-21T06:18:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.918+0000 7f8d627ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0afc070 0x5556f0ea32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.918+0000 7f8d627ba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0ad31b0 0x5556f0ea3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.918+0000 7f8d627ba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0adb390 0x5556f0ea3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.918+0000 7f8d627ba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5556f0e19ea0 con 0x5556f0ad31b0 2024-09-21T06:18:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.918+0000 7f8d627ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5556f0e092d0 con 0x5556f0afc070 2024-09-21T06:18:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.918+0000 7f8d627ba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5556f0e0d0e0 con 0x5556f0adb390 2024-09-21T06:18:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.918+0000 7f8d5a7e3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0afc070 0x5556f0ea32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.918+0000 7f8d597e1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0adb390 0x5556f0ea3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.918+0000 7f8d597e1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0adb390 0x5556f0ea3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46512/0 (socket says 172.21.15.3:46512) 2024-09-21T06:18:45.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.918+0000 7f8d597e1640 1 -- 172.21.15.3:0/2880291717 learned_addr learned my addr 172.21.15.3:0/2880291717 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.919+0000 7f8d5a7e3640 1 -- 172.21.15.3:0/2880291717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0adb390 msgr2=0x5556f0ea3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.919+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/2880291717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0adb390 0x5556f0ea3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.919+0000 7f8d5a7e3640 1 -- 172.21.15.3:0/2880291717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0ad31b0 msgr2=0x5556f0ea3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.919+0000 7f8d59fe2640 1 --2- 172.21.15.3:0/2880291717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0ad31b0 0x5556f0ea3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.919+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/2880291717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0ad31b0 0x5556f0ea3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.919+0000 7f8d5a7e3640 1 -- 172.21.15.3:0/2880291717 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556f0b67650 con 0x5556f0afc070 2024-09-21T06:18:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.919+0000 7f8d59fe2640 1 --2- 172.21.15.3:0/2880291717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0ad31b0 0x5556f0ea3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:18:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.919+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/2880291717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0afc070 0x5556f0ea32c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c005e30 tx=0x7f8d4c02f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a01e01a3a3a1c8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.920+0000 7f8d3ffff640 1 -- 172.21.15.3:0/2880291717 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c031650 con 0x5556f0afc070 2024-09-21T06:18:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.920+0000 7f8d3ffff640 1 -- 172.21.15.3:0/2880291717 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d4c036040 con 0x5556f0afc070 2024-09-21T06:18:45.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.920+0000 7f8d3ffff640 1 -- 172.21.15.3:0/2880291717 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c0043a0 con 0x5556f0afc070 2024-09-21T06:18:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.920+0000 7f8d4affd640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8d400b6680 0x7f8d400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.921+0000 7f8d59fe2640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8d400b6680 0x7f8d400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.921+0000 7f8d627ba740 1 -- 172.21.15.3:0/2880291717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0afc070 msgr2=0x5556f0ea32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:45.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.921+0000 7f8d627ba740 1 --2- 172.21.15.3:0/2880291717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0afc070 0x5556f0ea32c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c005e30 tx=0x7f8d4c02f310 comp rx=0 tx=0).stop 2024-09-21T06:18:45.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.921+0000 7f8d4affd640 1 -- 172.21.15.3:0/279983796 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8d500e7a50 con 0x7f8d54103b90 2024-09-21T06:18:45.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.921+0000 7f8d627ba740 1 -- 172.21.15.3:0/2880291717 shutdown_connections 2024-09-21T06:18:45.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.921+0000 7f8d627ba740 1 --2- 172.21.15.3:0/2880291717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0adb390 0x5556f0ea3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.921+0000 7f8d627ba740 1 --2- 172.21.15.3:0/2880291717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0ad31b0 0x5556f0ea3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.921+0000 7f8d627ba740 1 --2- 172.21.15.3:0/2880291717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0afc070 0x5556f0ea32c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.921+0000 7f8d627ba740 1 -- 172.21.15.3:0/2880291717 >> 172.21.15.3:0/2880291717 conn(0x5556f0af0460 msgr2=0x5556f0ea11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:45.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.921+0000 7f8d627ba740 1 -- 172.21.15.3:0/2880291717 shutdown_connections 2024-09-21T06:18:45.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.921+0000 7f8d627ba740 1 -- 172.21.15.3:0/2880291717 wait complete. 2024-09-21T06:18:45.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.921+0000 7f8d59fe2640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8d400b6680 0x7f8d400b8b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8d500e5290 tx=0x7f8d5001f000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:45.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.922+0000 7f8d627ba740 1 -- start start 2024-09-21T06:18:45.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.922+0000 7f8d627ba740 1 Processor -- start 2024-09-21T06:18:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d627ba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0ad31b0 0x5556f0eae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d627ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 0x5556f0eaee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d627ba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0afc070 0x5556f0ebabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d627ba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5556f0c6a700 con 0x5556f0afc070 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d627ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5556f0da4160 con 0x5556f0adb390 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d627ba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5556f0eb18c0 con 0x5556f0ad31b0 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d597e1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 0x5556f0eaee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d5a7e3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0ad31b0 0x5556f0eae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d597e1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 0x5556f0eaee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:48222/0 (socket says 172.21.15.3:48222) 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/3329100473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0afc070 0x5556f0ebabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d597e1640 1 -- 172.21.15.3:0/3329100473 learned_addr learned my addr 172.21.15.3:0/3329100473 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d5a7e3640 1 -- 172.21.15.3:0/3329100473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 msgr2=0x5556f0eaee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/3329100473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 0x5556f0eaee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d5a7e3640 1 -- 172.21.15.3:0/3329100473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0afc070 msgr2=0x5556f0ebabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/3329100473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0afc070 0x5556f0ebabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d627ba740 10 client.0 _ll_register_callbacks cb 0x5556f0dbd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d5a7e3640 1 -- 172.21.15.3:0/3329100473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556f0dd1c90 con 0x5556f0ad31b0 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.923+0000 7f8d597e1640 1 --2- 172.21.15.3:0/3329100473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 0x5556f0eaee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:45.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.924+0000 7f8d627ba740 1 -- 172.21.15.3:0/3329100473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d4c036040 con 0x5556f0ad31b0 2024-09-21T06:18:45.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.924+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/3329100473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0ad31b0 0x5556f0eae910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c006010 tx=0x7f8d4c009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d30b09c0a731bc1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:45.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.924+0000 7f8d3d7fa640 10 client.26152 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:18:45.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.924+0000 7f8d3d7fa640 1 -- 172.21.15.3:0/3329100473 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c059060 con 0x5556f0ad31b0 2024-09-21T06:18:45.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.924+0000 7f8d3d7fa640 1 -- 172.21.15.3:0/3329100473 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8d4c003ce0 con 0x5556f0ad31b0 2024-09-21T06:18:45.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.924+0000 7f8d3d7fa640 1 -- 172.21.15.3:0/3329100473 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d4c003ee0 con 0x5556f0ad31b0 2024-09-21T06:18:45.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.924+0000 7f8d627ba740 10 client.26152 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:45.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.925+0000 7f8d627ba740 1 -- 172.21.15.3:0/3329100473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5556f0ebd2f0 con 0x5556f0ad31b0 2024-09-21T06:18:45.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.925+0000 7f8d3d7fa640 1 -- 172.21.15.3:0/3329100473 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f8d4c04e030 con 0x5556f0ad31b0 2024-09-21T06:18:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.925+0000 7f8d627ba740 10 client.26152 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.925+0000 7f8d627ba740 10 client.26152 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:18:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.925+0000 7f8d627ba740 20 client.26152 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.925+0000 7f8d627ba740 1 --2- 172.21.15.3:0/3329100473 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5556f0aba9f0 0x5556f0ec0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.925+0000 7f8d627ba740 4 client.26152 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.925+0000 7f8d627ba740 1 -- 172.21.15.3:0/3329100473 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556f0e1bdf0 con 0x5556f0aba9f0 2024-09-21T06:18:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.926+0000 7f8d597e1640 1 --2- 172.21.15.3:0/3329100473 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5556f0aba9f0 0x5556f0ec0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.926+0000 7f8d597e1640 1 --2- 172.21.15.3:0/3329100473 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5556f0aba9f0 0x5556f0ec0650 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4197b11a157ac965 server_cookie=4f6c7580f57c525b in_seq=0 out_seq=0 2024-09-21T06:18:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.926+0000 7f8d3d7fa640 10 client.26152 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:18:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d3d7fa640 1 -- 172.21.15.3:0/3329100473 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5556f0e1bdf0 con 0x5556f0aba9f0 2024-09-21T06:18:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d3d7fa640 10 client.26152 handle_command_reply: tid=0 2024-09-21T06:18:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d627ba740 1 client.26152 shutdown 2024-09-21T06:18:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d627ba740 1 -- 172.21.15.3:0/3329100473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0ad31b0 msgr2=0x5556f0eae910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:45.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d627ba740 1 --2- 172.21.15.3:0/3329100473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0ad31b0 0x5556f0eae910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c006010 tx=0x7f8d4c009450 comp rx=0 tx=0).stop 2024-09-21T06:18:45.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d627ba740 1 -- 172.21.15.3:0/3329100473 shutdown_connections 2024-09-21T06:18:45.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d627ba740 1 --2- 172.21.15.3:0/3329100473 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5556f0aba9f0 0x5556f0ec0650 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d627ba740 1 --2- 172.21.15.3:0/3329100473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0afc070 0x5556f0ebabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d627ba740 1 --2- 172.21.15.3:0/3329100473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 0x5556f0eaee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d627ba740 1 --2- 172.21.15.3:0/3329100473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0ad31b0 0x5556f0eae910 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d3d7fa640 0 client.26152 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:18:45.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d627ba740 1 -- 172.21.15.3:0/3329100473 >> 172.21.15.3:0/3329100473 conn(0x5556f0af0460 msgr2=0x5556f0ea64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:45.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d627ba740 1 -- 172.21.15.3:0/3329100473 shutdown_connections 2024-09-21T06:18:45.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d627ba740 1 -- 172.21.15.3:0/3329100473 wait complete. 2024-09-21T06:18:45.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.927+0000 7f8d627ba740 20 client.26152 trim_cache size 0 max 16384 2024-09-21T06:18:45.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.942+0000 7f8d627ba740 1 Processor -- start 2024-09-21T06:18:45.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.942+0000 7f8d627ba740 1 -- start start 2024-09-21T06:18:45.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.943+0000 7f8d627ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0ad31b0 0x5556f0eaedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.943+0000 7f8d627ba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0adb390 0x5556f0ebab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.943+0000 7f8d627ba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0afc070 0x5556f0eabb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.943+0000 7f8d627ba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541b17b0 con 0x5556f0adb390 2024-09-21T06:18:45.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.943+0000 7f8d627ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541b3320 con 0x5556f0ad31b0 2024-09-21T06:18:45.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.943+0000 7f8d627ba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541b1e80 con 0x5556f0afc070 2024-09-21T06:18:45.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.943+0000 7f8d597e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0adb390 0x5556f0ebab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.943+0000 7f8d597e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0adb390 0x5556f0ebab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33432/0 (socket says 172.21.15.3:33432) 2024-09-21T06:18:45.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.943+0000 7f8d597e1640 1 -- 172.21.15.3:0/2596616911 learned_addr learned my addr 172.21.15.3:0/2596616911 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:45.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.943+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/2596616911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0ad31b0 0x5556f0eaedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.943+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/2596616911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0afc070 0x5556f0eabb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.944+0000 7f8d597e1640 1 -- 172.21.15.3:0/2596616911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0afc070 msgr2=0x5556f0eabb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:45.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.944+0000 7f8d597e1640 1 --2- 172.21.15.3:0/2596616911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0afc070 0x5556f0eabb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.944+0000 7f8d597e1640 1 -- 172.21.15.3:0/2596616911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0ad31b0 msgr2=0x5556f0eaedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:45.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.944+0000 7f8d597e1640 1 --2- 172.21.15.3:0/2596616911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0ad31b0 0x5556f0eaedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.944+0000 7f8d597e1640 1 -- 172.21.15.3:0/2596616911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556f0ea6120 con 0x5556f0adb390 2024-09-21T06:18:45.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.944+0000 7f8d597e1640 1 --2- 172.21.15.3:0/2596616911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0adb390 0x5556f0ebab20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x5556f0e19f30 tx=0x7f8d44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4eff43bb7811f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:45.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.944+0000 7f8d3f7fe640 1 -- 172.21.15.3:0/2596616911 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44004230 con 0x5556f0adb390 2024-09-21T06:18:45.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.944+0000 7f8d3f7fe640 1 -- 172.21.15.3:0/2596616911 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d440043d0 con 0x5556f0adb390 2024-09-21T06:18:45.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.944+0000 7f8d3f7fe640 1 -- 172.21.15.3:0/2596616911 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d440035f0 con 0x5556f0adb390 2024-09-21T06:18:45.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.945+0000 7f8d627ba740 1 -- 172.21.15.3:0/2596616911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0adb390 msgr2=0x5556f0ebab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:45.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.945+0000 7f8d627ba740 1 --2- 172.21.15.3:0/2596616911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0adb390 0x5556f0ebab20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x5556f0e19f30 tx=0x7f8d44000bc0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.945+0000 7f8d627ba740 1 -- 172.21.15.3:0/2596616911 shutdown_connections 2024-09-21T06:18:45.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.945+0000 7f8d627ba740 1 --2- 172.21.15.3:0/2596616911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0afc070 0x5556f0eabb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.945+0000 7f8d627ba740 1 --2- 172.21.15.3:0/2596616911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0adb390 0x5556f0ebab20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.945+0000 7f8d627ba740 1 --2- 172.21.15.3:0/2596616911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0ad31b0 0x5556f0eaedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.945+0000 7f8d627ba740 1 -- 172.21.15.3:0/2596616911 >> 172.21.15.3:0/2596616911 conn(0x5556f0af0460 msgr2=0x5556f0eae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:45.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.945+0000 7f8d627ba740 1 -- 172.21.15.3:0/2596616911 shutdown_connections 2024-09-21T06:18:45.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.945+0000 7f8d627ba740 1 -- 172.21.15.3:0/2596616911 wait complete. 2024-09-21T06:18:45.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.946+0000 7f8d627ba740 1 -- start start 2024-09-21T06:18:45.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.946+0000 7f8d627ba740 1 Processor -- start 2024-09-21T06:18:45.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.946+0000 7f8d627ba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0ad31b0 0x5556f0ec1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.946+0000 7f8d627ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 0x5556f0ec2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.946+0000 7f8d627ba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0afc070 0x5556f0f29dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.946+0000 7f8d627ba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541b2c20 con 0x5556f0ad31b0 2024-09-21T06:18:45.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.946+0000 7f8d627ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541b2550 con 0x5556f0adb390 2024-09-21T06:18:45.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.946+0000 7f8d627ba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541af790 con 0x5556f0afc070 2024-09-21T06:18:45.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.946+0000 7f8d627ba740 10 client.0 _ll_register_callbacks cb 0x7f8d440097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:45.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.946+0000 7f8d597e1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 0x5556f0ec2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.946+0000 7f8d597e1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 0x5556f0ec2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:48244/0 (socket says 172.21.15.3:48244) 2024-09-21T06:18:45.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.946+0000 7f8d597e1640 1 -- 172.21.15.3:0/3353982515 learned_addr learned my addr 172.21.15.3:0/3353982515 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:45.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.946+0000 7f8d5a7e3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0ad31b0 0x5556f0ec1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.947+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/3353982515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0afc070 0x5556f0f29dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.947+0000 7f8d5a7e3640 1 -- 172.21.15.3:0/3353982515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0afc070 msgr2=0x5556f0f29dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:45.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.947+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/3353982515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0afc070 0x5556f0f29dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.947+0000 7f8d5a7e3640 1 -- 172.21.15.3:0/3353982515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 msgr2=0x5556f0ec2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:45.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.947+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/3353982515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 0x5556f0ec2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.947+0000 7f8d5a7e3640 1 -- 172.21.15.3:0/3353982515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556f0f2a310 con 0x5556f0ad31b0 2024-09-21T06:18:45.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.947+0000 7f8d597e1640 1 --2- 172.21.15.3:0/3353982515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 0x5556f0ec2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:45.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.947+0000 7f8d5a7e3640 1 --2- 172.21.15.3:0/3353982515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0ad31b0 0x5556f0ec1a80 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c000f80 tx=0x7f8d4c04d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74b10da455525ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:45.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.947+0000 7f8d3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.947+0000 7f8d3d7fa640 1 -- 172.21.15.3:0/3353982515 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c004b40 con 0x5556f0ad31b0 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.947+0000 7f8d3d7fa640 1 -- 172.21.15.3:0/3353982515 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8d4c004ce0 con 0x5556f0ad31b0 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.947+0000 7f8d627ba740 1 -- 172.21.15.3:0/3353982515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d440043d0 con 0x5556f0ad31b0 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.948+0000 7f8d3d7fa640 1 -- 172.21.15.3:0/3353982515 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d4c06f050 con 0x5556f0ad31b0 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.948+0000 7f8d627ba740 10 client.16617 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.948+0000 7f8d627ba740 1 -- 172.21.15.3:0/3353982515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5556f0f2b460 con 0x5556f0ad31b0 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.948+0000 7f8d3d7fa640 1 -- 172.21.15.3:0/3353982515 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d440043d0 con 0x5556f0ad31b0 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.948+0000 7f8d3d7fa640 1 -- 172.21.15.3:0/3353982515 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f8d4c04e070 con 0x5556f0ad31b0 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.948+0000 7f8d627ba740 10 client.16617 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.948+0000 7f8d627ba740 10 client.16617 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.948+0000 7f8d627ba740 20 client.16617 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.948+0000 7f8d627ba740 1 --2- 172.21.15.3:0/3353982515 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5556f0a8fda0 0x5556f0f2dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.948+0000 7f8d627ba740 4 client.16617 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.948+0000 7f8d627ba740 1 -- 172.21.15.3:0/3353982515 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556f0eaa4c0 con 0x5556f0a8fda0 2024-09-21T06:18:45.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.948+0000 7f8d597e1640 1 --2- 172.21.15.3:0/3353982515 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5556f0a8fda0 0x5556f0f2dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:45.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d3d7fa640 10 client.16617 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:18:45.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d597e1640 1 --2- 172.21.15.3:0/3353982515 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5556f0a8fda0 0x5556f0f2dae0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d681d4e4f17c25b7 server_cookie=4e8bf07b50840d5e in_seq=0 out_seq=0 2024-09-21T06:18:45.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d3d7fa640 1 -- 172.21.15.3:0/3353982515 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556f0eaa4c0 con 0x5556f0a8fda0 2024-09-21T06:18:45.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d3d7fa640 10 client.16617 handle_command_reply: tid=0 2024-09-21T06:18:45.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d627ba740 1 client.16617 shutdown 2024-09-21T06:18:45.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 337364 12.75 MB/sec execute 18 sec latency 25.149 ms 2024-09-21T06:18:45.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d627ba740 1 -- 172.21.15.3:0/3353982515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0ad31b0 msgr2=0x5556f0ec1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:45.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d627ba740 1 --2- 172.21.15.3:0/3353982515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0ad31b0 0x5556f0ec1a80 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c000f80 tx=0x7f8d4c04d000 comp rx=0 tx=0).stop 2024-09-21T06:18:45.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d627ba740 1 -- 172.21.15.3:0/3353982515 shutdown_connections 2024-09-21T06:18:45.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d627ba740 1 --2- 172.21.15.3:0/3353982515 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5556f0a8fda0 0x5556f0f2dae0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d627ba740 1 --2- 172.21.15.3:0/3353982515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556f0afc070 0x5556f0f29dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d627ba740 1 --2- 172.21.15.3:0/3353982515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556f0adb390 0x5556f0ec2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d627ba740 1 --2- 172.21.15.3:0/3353982515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556f0ad31b0 0x5556f0ec1a80 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d627ba740 1 -- 172.21.15.3:0/3353982515 >> 172.21.15.3:0/3353982515 conn(0x5556f0af0460 msgr2=0x5556f0ea42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:45.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.949+0000 7f8d3d7fa640 0 client.16617 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:18:45.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d627ba740 1 -- 172.21.15.3:0/3353982515 shutdown_connections 2024-09-21T06:18:45.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d627ba740 1 -- 172.21.15.3:0/3353982515 wait complete. 2024-09-21T06:18:45.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d627ba740 20 client.16617 trim_cache size 0 max 16384 2024-09-21T06:18:45.956 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:18:45.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d5c246640 1 -- 172.21.15.3:0/279983796 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8d400b6680 msgr2=0x7f8d400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:45.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d5c246640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8d400b6680 0x7f8d400b8b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8d500e5290 tx=0x7f8d5001f000 comp rx=0 tx=0).stop 2024-09-21T06:18:45.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d5c246640 1 -- 172.21.15.3:0/279983796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d54103b90 msgr2=0x7f8d5419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:45.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d5c246640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d54103b90 0x7f8d5419c670 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000e9f0 tx=0x7f8d5000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d5c246640 1 -- 172.21.15.3:0/279983796 shutdown_connections 2024-09-21T06:18:45.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d5c246640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8d400b6680 0x7f8d400b8b90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d5c246640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54109e20 0x7f8d541a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d5c246640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d541094d0 0x7f8d5419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d5c246640 1 --2- 172.21.15.3:0/279983796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d54103b90 0x7f8d5419c670 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:45.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d5c246640 1 -- 172.21.15.3:0/279983796 >> 172.21.15.3:0/279983796 conn(0x7f8d540ff5e0 msgr2=0x7f8d54108b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:45.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d5c246640 1 -- 172.21.15.3:0/279983796 shutdown_connections 2024-09-21T06:18:45.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:45.950+0000 7f8d5c246640 1 -- 172.21.15.3:0/279983796 wait complete. 2024-09-21T06:18:46.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:18:46.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 339877 12.70 MB/sec execute 19 sec latency 84.668 ms 2024-09-21T06:18:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:47.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.569+0000 7f9c7a1ee640 1 -- 172.21.15.3:0/490444019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c741049c0 msgr2=0x7f9c74104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:47.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.569+0000 7f9c7a1ee640 1 --2- 172.21.15.3:0/490444019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c741049c0 0x7f9c74104dd0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009900 tx=0x7f9c6402a410 comp rx=0 tx=0).stop 2024-09-21T06:18:47.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.569+0000 7f9c7a1ee640 1 -- 172.21.15.3:0/490444019 shutdown_connections 2024-09-21T06:18:47.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.569+0000 7f9c7a1ee640 1 --2- 172.21.15.3:0/490444019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c74105310 0x7f9c7410d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.569+0000 7f9c7a1ee640 1 --2- 172.21.15.3:0/490444019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c7410de20 0x7f9c74110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.569+0000 7f9c7a1ee640 1 --2- 172.21.15.3:0/490444019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c741049c0 0x7f9c74104dd0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.569+0000 7f9c7a1ee640 1 -- 172.21.15.3:0/490444019 >> 172.21.15.3:0/490444019 conn(0x7f9c74079710 msgr2=0x7f9c74079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:47.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.569+0000 7f9c7a1ee640 1 -- 172.21.15.3:0/490444019 shutdown_connections 2024-09-21T06:18:47.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.569+0000 7f9c7a1ee640 1 -- 172.21.15.3:0/490444019 wait complete. 2024-09-21T06:18:47.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.570+0000 7f9c7a1ee640 1 Processor -- start 2024-09-21T06:18:47.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.570+0000 7f9c7a1ee640 1 -- start start 2024-09-21T06:18:47.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.570+0000 7f9c7a1ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c741049c0 0x7f9c7419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.570+0000 7f9c7a1ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c74105310 0x7f9c7419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.570+0000 7f9c7a1ee640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c7410de20 0x7f9c7419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.570+0000 7f9c7a1ee640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9c74113aa0 con 0x7f9c74105310 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.570+0000 7f9c7a1ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9c74113920 con 0x7f9c741049c0 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.570+0000 7f9c7a1ee640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c74113c20 con 0x7f9c7410de20 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.570+0000 7f9c72ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c74105310 0x7f9c7419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c737fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c741049c0 0x7f9c7419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c72ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c74105310 0x7f9c7419ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33446/0 (socket says 172.21.15.3:33446) 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c72ffd640 1 -- 172.21.15.3:0/887602368 learned_addr learned my addr 172.21.15.3:0/887602368 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c737fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c741049c0 0x7f9c7419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:48260/0 (socket says 172.21.15.3:48260) 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c73fff640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c7410de20 0x7f9c7419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c72ffd640 1 -- 172.21.15.3:0/887602368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c7410de20 msgr2=0x7f9c7419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c72ffd640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c7410de20 0x7f9c7419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c72ffd640 1 -- 172.21.15.3:0/887602368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c741049c0 msgr2=0x7f9c7419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c72ffd640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c741049c0 0x7f9c7419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c72ffd640 1 -- 172.21.15.3:0/887602368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c741a04e0 con 0x7f9c74105310 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c737fe640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c741049c0 0x7f9c7419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c72ffd640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c74105310 0x7f9c7419ab10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800e9c0 tx=0x7f9c6800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c70ff9640 1 -- 172.21.15.3:0/887602368 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6800ccc0 con 0x7f9c74105310 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c7a1ee640 1 -- 172.21.15.3:0/887602368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c741a07d0 con 0x7f9c74105310 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c70ff9640 1 -- 172.21.15.3:0/887602368 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c680054d0 con 0x7f9c74105310 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c70ff9640 1 -- 172.21.15.3:0/887602368 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68010430 con 0x7f9c74105310 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.571+0000 7f9c7a1ee640 1 -- 172.21.15.3:0/887602368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c741a6f30 con 0x7f9c74105310 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.574+0000 7f9c70ff9640 1 -- 172.21.15.3:0/887602368 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f9c680105d0 con 0x7f9c74105310 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.575+0000 7f9c80762740 1 Processor -- start 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.575+0000 7f9c80762740 1 -- start start 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.575+0000 7f9c80762740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565110898070 0x565110c3f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.575+0000 7f9c80762740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56511086f1b0 0x565110c3fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.575+0000 7f9c80762740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 0x565110c40120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.575+0000 7f9c80762740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x565110bb5ea0 con 0x565110877390 2024-09-21T06:18:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.575+0000 7f9c80762740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565110ba52d0 con 0x56511086f1b0 2024-09-21T06:18:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.575+0000 7f9c80762740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x565110ba90e0 con 0x565110898070 2024-09-21T06:18:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.575+0000 7f9c73fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565110898070 0x565110c3f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.575+0000 7f9c73fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565110898070 0x565110c3f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46570/0 (socket says 172.21.15.3:46570) 2024-09-21T06:18:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.575+0000 7f9c73fff640 1 -- 172.21.15.3:0/1532784773 learned_addr learned my addr 172.21.15.3:0/1532784773 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.575+0000 7f9c72ffd640 1 --2- 172.21.15.3:0/1532784773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 0x565110c40120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.575+0000 7f9c737fe640 1 --2- 172.21.15.3:0/1532784773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56511086f1b0 0x565110c3fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.576+0000 7f9c70ff9640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9c5c0b6680 0x7f9c5c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.576+0000 7f9c737fe640 1 -- 172.21.15.3:0/1532784773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565110898070 msgr2=0x565110c3f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.576+0000 7f9c737fe640 1 --2- 172.21.15.3:0/1532784773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565110898070 0x565110c3f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.576+0000 7f9c737fe640 1 -- 172.21.15.3:0/1532784773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 msgr2=0x565110c40120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.576+0000 7f9c737fe640 1 --2- 172.21.15.3:0/1532784773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 0x565110c40120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.576+0000 7f9c737fe640 1 -- 172.21.15.3:0/1532784773 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565110903650 con 0x56511086f1b0 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.576+0000 7f9c73fff640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9c5c0b6680 0x7f9c5c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.576+0000 7f9c70ff9640 1 -- 172.21.15.3:0/887602368 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f9c680e7a90 con 0x7f9c74105310 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.576+0000 7f9c737fe640 1 --2- 172.21.15.3:0/1532784773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56511086f1b0 0x565110c3fbe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009230 tx=0x7f9c64005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75246690fd039951 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.576+0000 7f9c73fff640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9c5c0b6680 0x7f9c5c0b8b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c010610 tx=0x7f9c6c004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.576+0000 7f9c59ffb640 1 -- 172.21.15.3:0/1532784773 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64040070 con 0x56511086f1b0 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.576+0000 7f9c59ffb640 1 -- 172.21.15.3:0/1532784773 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c64031720 con 0x56511086f1b0 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.576+0000 7f9c59ffb640 1 -- 172.21.15.3:0/1532784773 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64035040 con 0x56511086f1b0 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.577+0000 7f9c80762740 1 -- 172.21.15.3:0/1532784773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56511086f1b0 msgr2=0x565110c3fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.577+0000 7f9c80762740 1 --2- 172.21.15.3:0/1532784773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56511086f1b0 0x565110c3fbe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009230 tx=0x7f9c64005e20 comp rx=0 tx=0).stop 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.577+0000 7f9c80762740 1 -- 172.21.15.3:0/1532784773 shutdown_connections 2024-09-21T06:18:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.577+0000 7f9c80762740 1 --2- 172.21.15.3:0/1532784773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 0x565110c40120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.577+0000 7f9c80762740 1 --2- 172.21.15.3:0/1532784773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56511086f1b0 0x565110c3fbe0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.577+0000 7f9c80762740 1 --2- 172.21.15.3:0/1532784773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565110898070 0x565110c3f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.577+0000 7f9c80762740 1 -- 172.21.15.3:0/1532784773 >> 172.21.15.3:0/1532784773 conn(0x56511088c460 msgr2=0x565110c3d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.577+0000 7f9c80762740 1 -- 172.21.15.3:0/1532784773 shutdown_connections 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.577+0000 7f9c80762740 1 -- 172.21.15.3:0/1532784773 wait complete. 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.578+0000 7f9c80762740 1 -- start start 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.578+0000 7f9c80762740 1 Processor -- start 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c80762740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 0x565110c4adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c80762740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 0x565110c4b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c80762740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565110898070 0x565110c57110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c80762740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x565110a06700 con 0x565110877390 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c80762740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565110b40160 con 0x565110898070 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c80762740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x565110c4de20 con 0x56511086f1b0 2024-09-21T06:18:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c80762740 10 client.0 _ll_register_callbacks cb 0x565110b599e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c72ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 0x565110c4b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c737fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 0x565110c4adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c737fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 0x565110c4adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46578/0 (socket says 172.21.15.3:46578) 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c737fe640 1 -- 172.21.15.3:0/1079541214 learned_addr learned my addr 172.21.15.3:0/1079541214 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c73fff640 1 --2- 172.21.15.3:0/1079541214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565110898070 0x565110c57110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c72ffd640 1 -- 172.21.15.3:0/1079541214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 msgr2=0x565110c4adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c72ffd640 1 --2- 172.21.15.3:0/1079541214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 0x565110c4adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c72ffd640 1 -- 172.21.15.3:0/1079541214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565110898070 msgr2=0x565110c57110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c72ffd640 1 --2- 172.21.15.3:0/1079541214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565110898070 0x565110c57110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c72ffd640 1 -- 172.21.15.3:0/1079541214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565110b6dc90 con 0x565110877390 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c737fe640 1 --2- 172.21.15.3:0/1079541214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 0x565110c4adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c72ffd640 1 --2- 172.21.15.3:0/1079541214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 0x565110c4b300 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9c680e5220 tx=0x7f9c680eb1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48264bd9f3e3cb9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:47.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.579+0000 7f9c52ffd640 1 -- 172.21.15.3:0/1079541214 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c680ad270 con 0x565110877390 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c80762740 1 -- 172.21.15.3:0/1079541214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c64031720 con 0x565110877390 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c52ffd640 1 -- 172.21.15.3:0/1079541214 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f9c68005000 con 0x565110877390 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c52ffd640 1 -- 172.21.15.3:0/1079541214 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9c6800ce60 con 0x565110877390 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c80762740 10 client.16629 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c80762740 1 -- 172.21.15.3:0/1079541214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565110c59b90 con 0x565110877390 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c52ffd640 1 -- 172.21.15.3:0/1079541214 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c64031720 con 0x565110877390 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c52ffd640 1 -- 172.21.15.3:0/1079541214 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f9c680ac5f0 con 0x565110877390 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c80762740 10 client.16629 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c80762740 10 client.16629 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c80762740 20 client.16629 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c80762740 1 --2- 172.21.15.3:0/1079541214 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5651108569f0 0x565110c5cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c80762740 4 client.16629 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c80762740 1 -- 172.21.15.3:0/1079541214 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565110bb7df0 con 0x5651108569f0 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.580+0000 7f9c737fe640 1 --2- 172.21.15.3:0/1079541214 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5651108569f0 0x565110c5cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.581+0000 7f9c52ffd640 10 client.16629 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.581+0000 7f9c737fe640 1 --2- 172.21.15.3:0/1079541214 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5651108569f0 0x565110c5cfb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=66e4fa96ea6d6670 server_cookie=851d115ada4aa7be in_seq=0 out_seq=0 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.582+0000 7f9c52ffd640 1 -- 172.21.15.3:0/1079541214 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565110bb7df0 con 0x5651108569f0 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.582+0000 7f9c52ffd640 10 client.16629 handle_command_reply: tid=0 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.583+0000 7f9c80762740 1 client.16629 shutdown 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.583+0000 7f9c80762740 1 -- 172.21.15.3:0/1079541214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 msgr2=0x565110c4b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.583+0000 7f9c80762740 1 --2- 172.21.15.3:0/1079541214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 0x565110c4b300 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9c680e5220 tx=0x7f9c680eb1d0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.583+0000 7f9c80762740 1 -- 172.21.15.3:0/1079541214 shutdown_connections 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.583+0000 7f9c80762740 1 --2- 172.21.15.3:0/1079541214 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5651108569f0 0x565110c5cfb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.583+0000 7f9c80762740 1 --2- 172.21.15.3:0/1079541214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565110898070 0x565110c57110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.583+0000 7f9c80762740 1 --2- 172.21.15.3:0/1079541214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 0x565110c4b300 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.583+0000 7f9c52ffd640 0 client.16629 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.583+0000 7f9c80762740 1 --2- 172.21.15.3:0/1079541214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 0x565110c4adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.583+0000 7f9c80762740 1 -- 172.21.15.3:0/1079541214 >> 172.21.15.3:0/1079541214 conn(0x56511088c460 msgr2=0x565110c42830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.583+0000 7f9c80762740 1 -- 172.21.15.3:0/1079541214 shutdown_connections 2024-09-21T06:18:47.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.583+0000 7f9c80762740 1 -- 172.21.15.3:0/1079541214 wait complete. 2024-09-21T06:18:47.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.583+0000 7f9c80762740 20 client.16629 trim_cache size 0 max 16384 2024-09-21T06:18:47.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.598+0000 7f9c80762740 1 Processor -- start 2024-09-21T06:18:47.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c80762740 1 -- start start 2024-09-21T06:18:47.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c80762740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 0x565110c4b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c80762740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 0x565110c57010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c80762740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565110898070 0x565110c47fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c80762740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741af700 con 0x565110877390 2024-09-21T06:18:47.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c80762740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741b1270 con 0x565110898070 2024-09-21T06:18:47.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c80762740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741afdd0 con 0x56511086f1b0 2024-09-21T06:18:47.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c737fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 0x565110c4b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c737fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 0x565110c4b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46584/0 (socket says 172.21.15.3:46584) 2024-09-21T06:18:47.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c737fe640 1 -- 172.21.15.3:0/3206638656 learned_addr learned my addr 172.21.15.3:0/3206638656 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:47.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c73fff640 1 --2- 172.21.15.3:0/3206638656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565110898070 0x565110c47fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c72ffd640 1 --2- 172.21.15.3:0/3206638656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 0x565110c57010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c737fe640 1 -- 172.21.15.3:0/3206638656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565110898070 msgr2=0x565110c47fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c737fe640 1 --2- 172.21.15.3:0/3206638656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565110898070 0x565110c47fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c737fe640 1 -- 172.21.15.3:0/3206638656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 msgr2=0x565110c57010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c737fe640 1 --2- 172.21.15.3:0/3206638656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 0x565110c57010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.599+0000 7f9c737fe640 1 -- 172.21.15.3:0/3206638656 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565110c3de80 con 0x56511086f1b0 2024-09-21T06:18:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.600+0000 7f9c737fe640 1 --2- 172.21.15.3:0/3206638656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 0x565110c4b280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009900 tx=0x7f9c64009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a06b18532c6a32bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.600+0000 7f9c597fa640 1 -- 172.21.15.3:0/3206638656 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64035040 con 0x56511086f1b0 2024-09-21T06:18:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.600+0000 7f9c597fa640 1 -- 172.21.15.3:0/3206638656 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c64031700 con 0x56511086f1b0 2024-09-21T06:18:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.600+0000 7f9c597fa640 1 -- 172.21.15.3:0/3206638656 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64040040 con 0x56511086f1b0 2024-09-21T06:18:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.601+0000 7f9c80762740 1 -- 172.21.15.3:0/3206638656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 msgr2=0x565110c4b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.601+0000 7f9c80762740 1 --2- 172.21.15.3:0/3206638656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 0x565110c4b280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009900 tx=0x7f9c64009260 comp rx=0 tx=0).stop 2024-09-21T06:18:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.601+0000 7f9c80762740 1 -- 172.21.15.3:0/3206638656 shutdown_connections 2024-09-21T06:18:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.601+0000 7f9c80762740 1 --2- 172.21.15.3:0/3206638656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565110898070 0x565110c47fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.601+0000 7f9c80762740 1 --2- 172.21.15.3:0/3206638656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110877390 0x565110c57010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.601+0000 7f9c80762740 1 --2- 172.21.15.3:0/3206638656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56511086f1b0 0x565110c4b280 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.601+0000 7f9c80762740 1 -- 172.21.15.3:0/3206638656 >> 172.21.15.3:0/3206638656 conn(0x56511088c460 msgr2=0x565110c4ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.601+0000 7f9c80762740 1 -- 172.21.15.3:0/3206638656 shutdown_connections 2024-09-21T06:18:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.601+0000 7f9c80762740 1 -- 172.21.15.3:0/3206638656 wait complete. 2024-09-21T06:18:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.601+0000 7f9c80762740 1 -- start start 2024-09-21T06:18:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c80762740 1 Processor -- start 2024-09-21T06:18:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c80762740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56511086f1b0 0x565110c4ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c80762740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565110877390 0x565110c4f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c80762740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110898070 0x565110c4f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c80762740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741b0b70 con 0x565110898070 2024-09-21T06:18:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c80762740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741b04a0 con 0x56511086f1b0 2024-09-21T06:18:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c80762740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741ad6e0 con 0x565110877390 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c80762740 10 client.0 _ll_register_callbacks cb 0x7f9c680aa060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c73fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110898070 0x565110c4f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c737fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56511086f1b0 0x565110c4ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c73fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110898070 0x565110c4f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33482/0 (socket says 172.21.15.3:33482) 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c73fff640 1 -- 172.21.15.3:0/1161125223 learned_addr learned my addr 172.21.15.3:0/1161125223 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c72ffd640 1 --2- 172.21.15.3:0/1161125223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565110877390 0x565110c4f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c73fff640 1 -- 172.21.15.3:0/1161125223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565110877390 msgr2=0x565110c4f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c73fff640 1 --2- 172.21.15.3:0/1161125223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565110877390 0x565110c4f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c73fff640 1 -- 172.21.15.3:0/1161125223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56511086f1b0 msgr2=0x565110c4ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c73fff640 1 --2- 172.21.15.3:0/1161125223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56511086f1b0 0x565110c4ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c73fff640 1 -- 172.21.15.3:0/1161125223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565110cc6970 con 0x565110898070 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.602+0000 7f9c72ffd640 1 --2- 172.21.15.3:0/1161125223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565110877390 0x565110c4f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.603+0000 7f9c73fff640 1 --2- 172.21.15.3:0/1161125223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110898070 0x565110c4f9f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00fed0 tx=0x7f9c6c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9ac72318a83df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.603+0000 7f9c52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.603+0000 7f9c52ffd640 1 -- 172.21.15.3:0/1161125223 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c004b20 con 0x565110898070 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.603+0000 7f9c52ffd640 1 -- 172.21.15.3:0/1161125223 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f9c6c004cc0 con 0x565110898070 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.603+0000 7f9c80762740 1 -- 172.21.15.3:0/1161125223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c64031700 con 0x565110898070 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.603+0000 7f9c52ffd640 1 -- 172.21.15.3:0/1161125223 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9c6c005020 con 0x565110898070 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.603+0000 7f9c80762740 10 client.16635 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.603+0000 7f9c80762740 1 -- 172.21.15.3:0/1161125223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565110cc7b10 con 0x565110898070 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.603+0000 7f9c52ffd640 1 -- 172.21.15.3:0/1161125223 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c64031700 con 0x565110898070 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.604+0000 7f9c52ffd640 1 -- 172.21.15.3:0/1161125223 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f9c6c02a450 con 0x565110898070 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.604+0000 7f9c80762740 10 client.16635 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.604+0000 7f9c80762740 10 client.16635 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.604+0000 7f9c80762740 20 client.16635 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.604+0000 7f9c80762740 1 --2- 172.21.15.3:0/1161125223 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56511082bda0 0x565110cca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.604+0000 7f9c80762740 4 client.16635 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:18:47.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.604+0000 7f9c80762740 1 -- 172.21.15.3:0/1161125223 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565110bb38a0 con 0x56511082bda0 2024-09-21T06:18:47.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.604+0000 7f9c737fe640 1 --2- 172.21.15.3:0/1161125223 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56511082bda0 0x565110cca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:47.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.604+0000 7f9c52ffd640 10 client.16635 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:18:47.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.604+0000 7f9c737fe640 1 --2- 172.21.15.3:0/1161125223 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56511082bda0 0x565110cca300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bf8f1e7b9bfb1587 server_cookie=ac15b0ff57daa610 in_seq=0 out_seq=0 2024-09-21T06:18:47.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.606+0000 7f9c52ffd640 1 -- 172.21.15.3:0/1161125223 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565110bb38a0 con 0x56511082bda0 2024-09-21T06:18:47.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.606+0000 7f9c52ffd640 10 client.16635 handle_command_reply: tid=0 2024-09-21T06:18:47.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.606+0000 7f9c80762740 1 client.16635 shutdown 2024-09-21T06:18:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.606+0000 7f9c80762740 1 -- 172.21.15.3:0/1161125223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110898070 msgr2=0x565110c4f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.606+0000 7f9c80762740 1 --2- 172.21.15.3:0/1161125223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110898070 0x565110c4f9f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00fed0 tx=0x7f9c6c004ab0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.606+0000 7f9c80762740 1 -- 172.21.15.3:0/1161125223 shutdown_connections 2024-09-21T06:18:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.606+0000 7f9c80762740 1 --2- 172.21.15.3:0/1161125223 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x56511082bda0 0x565110cca300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.606+0000 7f9c80762740 1 --2- 172.21.15.3:0/1161125223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565110877390 0x565110c4f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.606+0000 7f9c52ffd640 0 client.16635 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:18:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.606+0000 7f9c80762740 1 --2- 172.21.15.3:0/1161125223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565110898070 0x565110c4f9f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.606+0000 7f9c80762740 1 --2- 172.21.15.3:0/1161125223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56511086f1b0 0x565110c4ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.606+0000 7f9c80762740 1 -- 172.21.15.3:0/1161125223 >> 172.21.15.3:0/1161125223 conn(0x56511088c460 msgr2=0x565110c40630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.607+0000 7f9c80762740 1 -- 172.21.15.3:0/1161125223 shutdown_connections 2024-09-21T06:18:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.607+0000 7f9c80762740 1 -- 172.21.15.3:0/1161125223 wait complete. 2024-09-21T06:18:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.607+0000 7f9c80762740 20 client.16635 trim_cache size 0 max 16384 2024-09-21T06:18:47.615 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:18:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.607+0000 7f9c7a1ee640 1 -- 172.21.15.3:0/887602368 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9c5c0b6680 msgr2=0x7f9c5c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.607+0000 7f9c7a1ee640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9c5c0b6680 0x7f9c5c0b8b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c010610 tx=0x7f9c6c004a40 comp rx=0 tx=0).stop 2024-09-21T06:18:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.607+0000 7f9c7a1ee640 1 -- 172.21.15.3:0/887602368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c74105310 msgr2=0x7f9c7419ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.607+0000 7f9c7a1ee640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c74105310 0x7f9c7419ab10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800e9c0 tx=0x7f9c6800ee80 comp rx=0 tx=0).stop 2024-09-21T06:18:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.607+0000 7f9c7a1ee640 1 -- 172.21.15.3:0/887602368 shutdown_connections 2024-09-21T06:18:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.607+0000 7f9c7a1ee640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9c5c0b6680 0x7f9c5c0b8b90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.607+0000 7f9c7a1ee640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c7410de20 0x7f9c7419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.607+0000 7f9c7a1ee640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c74105310 0x7f9c7419ab10 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.607+0000 7f9c7a1ee640 1 --2- 172.21.15.3:0/887602368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c741049c0 0x7f9c7419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.607+0000 7f9c7a1ee640 1 -- 172.21.15.3:0/887602368 >> 172.21.15.3:0/887602368 conn(0x7f9c74079710 msgr2=0x7f9c74107940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.608+0000 7f9c7a1ee640 1 -- 172.21.15.3:0/887602368 shutdown_connections 2024-09-21T06:18:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:47.608+0000 7f9c7a1ee640 1 -- 172.21.15.3:0/887602368 wait complete. 2024-09-21T06:18:47.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:47 smithi003 ceph-mon[31227]: pgmap v438: 97 pgs: 97 active+clean; 66 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 397 op/s 2024-09-21T06:18:47.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:47 smithi137 ceph-mon[34464]: pgmap v438: 97 pgs: 97 active+clean; 66 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 397 op/s 2024-09-21T06:18:47.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:47 smithi204 ceph-mon[34659]: pgmap v438: 97 pgs: 97 active+clean; 66 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 397 op/s 2024-09-21T06:18:47.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 341258 12.45 MB/sec execute 20 sec latency 42.212 ms 2024-09-21T06:18:48.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:18:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:48.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 344313 12.73 MB/sec execute 21 sec latency 12.911 ms 2024-09-21T06:18:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.458+0000 7fae73b21640 1 -- 172.21.15.3:0/4054192805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae6c108d10 msgr2=0x7fae6c10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.458+0000 7fae73b21640 1 --2- 172.21.15.3:0/4054192805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae6c108d10 0x7fae6c10f540 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fae64009ca0 tx=0x7fae6402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.458+0000 7fae73b21640 1 -- 172.21.15.3:0/4054192805 shutdown_connections 2024-09-21T06:18:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.458+0000 7fae73b21640 1 --2- 172.21.15.3:0/4054192805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae6c108d10 0x7fae6c10f540 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.458+0000 7fae73b21640 1 --2- 172.21.15.3:0/4054192805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae6c1083c0 0x7fae6c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.458+0000 7fae73b21640 1 --2- 172.21.15.3:0/4054192805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae6c102ad0 0x7fae6c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.458+0000 7fae73b21640 1 -- 172.21.15.3:0/4054192805 >> 172.21.15.3:0/4054192805 conn(0x7fae6c0796d0 msgr2=0x7fae6c079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.458+0000 7fae73b21640 1 -- 172.21.15.3:0/4054192805 shutdown_connections 2024-09-21T06:18:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.458+0000 7fae73b21640 1 -- 172.21.15.3:0/4054192805 wait complete. 2024-09-21T06:18:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.458+0000 7fae73b21640 1 Processor -- start 2024-09-21T06:18:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.458+0000 7fae73b21640 1 -- start start 2024-09-21T06:18:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae73b21640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae6c102ad0 0x7fae6c199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae73b21640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae6c1083c0 0x7fae6c1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae73b21640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae6c108d10 0x7fae6c199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae73b21640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1129d0 con 0x7fae6c102ad0 2024-09-21T06:18:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae73b21640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c112850 con 0x7fae6c108d10 2024-09-21T06:18:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae73b21640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c112b50 con 0x7fae6c1083c0 2024-09-21T06:18:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae718bd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae6c102ad0 0x7fae6c199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae718bd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae6c102ad0 0x7fae6c199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33500/0 (socket says 172.21.15.3:33500) 2024-09-21T06:18:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae718bd640 1 -- 172.21.15.3:0/705943769 learned_addr learned my addr 172.21.15.3:0/705943769 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae720be640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae6c108d10 0x7fae6c199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae710bc640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae6c1083c0 0x7fae6c1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae718bd640 1 -- 172.21.15.3:0/705943769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae6c1083c0 msgr2=0x7fae6c1995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae718bd640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae6c1083c0 0x7fae6c1995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae718bd640 1 -- 172.21.15.3:0/705943769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae6c108d10 msgr2=0x7fae6c199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae718bd640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae6c108d10 0x7fae6c199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae718bd640 1 -- 172.21.15.3:0/705943769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c1a8740 con 0x7fae6c102ad0 2024-09-21T06:18:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.459+0000 7fae718bd640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae6c102ad0 0x7fae6c199090 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fae6800d950 tx=0x7fae6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.460+0000 7fae62ffd640 1 -- 172.21.15.3:0/705943769 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68017070 con 0x7fae6c102ad0 2024-09-21T06:18:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.460+0000 7fae62ffd640 1 -- 172.21.15.3:0/705943769 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae68009c30 con 0x7fae6c102ad0 2024-09-21T06:18:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.460+0000 7fae73b21640 1 -- 172.21.15.3:0/705943769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6c1a8a30 con 0x7fae6c102ad0 2024-09-21T06:18:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.460+0000 7fae62ffd640 1 -- 172.21.15.3:0/705943769 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6800f620 con 0x7fae6c102ad0 2024-09-21T06:18:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.460+0000 7fae73b21640 1 -- 172.21.15.3:0/705943769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6c1a8f70 con 0x7fae6c102ad0 2024-09-21T06:18:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.462+0000 7fae7a095740 1 Processor -- start 2024-09-21T06:18:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.463+0000 7fae7a095740 1 -- start start 2024-09-21T06:18:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.463+0000 7fae62ffd640 1 -- 172.21.15.3:0/705943769 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fae68010460 con 0x7fae6c102ad0 2024-09-21T06:18:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.463+0000 7fae7a095740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce7b4070 0x55f8ceb5b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.463+0000 7fae7a095740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb5bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.463+0000 7fae7a095740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb5c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.463+0000 7fae7a095740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f8cead1ea0 con 0x55f8ce793390 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.463+0000 7fae7a095740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f8ceac12d0 con 0x55f8ce78b1b0 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.463+0000 7fae7a095740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f8ceac50e0 con 0x55f8ce7b4070 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.463+0000 7fae710bc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb5c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.463+0000 7fae718bd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb5bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.463+0000 7fae710bc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb5c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33512/0 (socket says 172.21.15.3:33512) 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.463+0000 7fae710bc640 1 -- 172.21.15.3:0/3326536048 learned_addr learned my addr 172.21.15.3:0/3326536048 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.463+0000 7fae720be640 1 --2- 172.21.15.3:0/3326536048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce7b4070 0x55f8ceb5b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae710bc640 1 -- 172.21.15.3:0/3326536048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce7b4070 msgr2=0x55f8ceb5b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae710bc640 1 --2- 172.21.15.3:0/3326536048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce7b4070 0x55f8ceb5b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae710bc640 1 -- 172.21.15.3:0/3326536048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce78b1b0 msgr2=0x55f8ceb5bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae710bc640 1 --2- 172.21.15.3:0/3326536048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb5bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae710bc640 1 -- 172.21.15.3:0/3326536048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ce81f650 con 0x55f8ce793390 2024-09-21T06:18:49.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae720be640 1 --2- 172.21.15.3:0/3326536048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce7b4070 0x55f8ceb5b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae718bd640 1 --2- 172.21.15.3:0/3326536048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb5bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae62ffd640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fae580b6680 0x7fae580b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae720be640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fae580b6680 0x7fae580b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae710bc640 1 --2- 172.21.15.3:0/3326536048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb5c120 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009c20 tx=0x7fae5c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c043939228310a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae62ffd640 1 -- 172.21.15.3:0/705943769 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fae680e72a0 con 0x7fae6c102ad0 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae57fff640 1 -- 172.21.15.3:0/3326536048 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c004230 con 0x55f8ce793390 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae57fff640 1 -- 172.21.15.3:0/3326536048 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5c0043d0 con 0x55f8ce793390 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.464+0000 7fae57fff640 1 -- 172.21.15.3:0/3326536048 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c004cc0 con 0x55f8ce793390 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.465+0000 7fae7a095740 1 -- 172.21.15.3:0/3326536048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 msgr2=0x55f8ceb5c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.465+0000 7fae720be640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fae580b6680 0x7fae580b8b90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fae64006010 tx=0x7fae64009480 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.465+0000 7fae7a095740 1 --2- 172.21.15.3:0/3326536048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb5c120 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009c20 tx=0x7fae5c011810 comp rx=0 tx=0).stop 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.465+0000 7fae7a095740 1 -- 172.21.15.3:0/3326536048 shutdown_connections 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.466+0000 7fae7a095740 1 --2- 172.21.15.3:0/3326536048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb5c120 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.466+0000 7fae7a095740 1 --2- 172.21.15.3:0/3326536048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb5bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.466+0000 7fae7a095740 1 --2- 172.21.15.3:0/3326536048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce7b4070 0x55f8ceb5b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.466+0000 7fae7a095740 1 -- 172.21.15.3:0/3326536048 >> 172.21.15.3:0/3326536048 conn(0x55f8ce7a8460 msgr2=0x55f8ceb59590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.466+0000 7fae7a095740 1 -- 172.21.15.3:0/3326536048 shutdown_connections 2024-09-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.466+0000 7fae7a095740 1 -- 172.21.15.3:0/3326536048 wait complete. 2024-09-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.466+0000 7fae7a095740 1 -- start start 2024-09-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.466+0000 7fae7a095740 1 Processor -- start 2024-09-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae7a095740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb66d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae7a095740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb67280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae7a095740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce7b4070 0x55f8ceb73170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae7a095740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f8ce922700 con 0x55f8ce793390 2024-09-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae7a095740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f8cea5c160 con 0x55f8ce7b4070 2024-09-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae7a095740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f8ceb69e80 con 0x55f8ce78b1b0 2024-09-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae7a095740 10 client.0 _ll_register_callbacks cb 0x55f8cea759e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:49.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae710bc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb67280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae710bc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb67280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33524/0 (socket says 172.21.15.3:33524) 2024-09-21T06:18:49.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae710bc640 1 -- 172.21.15.3:0/2738403835 learned_addr learned my addr 172.21.15.3:0/2738403835 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:49.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae720be640 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce7b4070 0x55f8ceb73170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae718bd640 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb66d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae710bc640 1 -- 172.21.15.3:0/2738403835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce78b1b0 msgr2=0x55f8ceb66d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:49.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae710bc640 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb66d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae710bc640 1 -- 172.21.15.3:0/2738403835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce7b4070 msgr2=0x55f8ceb73170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:49.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae710bc640 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce7b4070 0x55f8ceb73170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae710bc640 1 -- 172.21.15.3:0/2738403835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8cea89c90 con 0x55f8ce793390 2024-09-21T06:18:49.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.467+0000 7fae718bd640 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb66d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:49.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.468+0000 7fae710bc640 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb67280 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fae5c005890 tx=0x7fae5c019ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375fcb7786cfb324 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:49.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.468+0000 7fae557fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:49.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.468+0000 7fae557fa640 1 -- 172.21.15.3:0/2738403835 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c028020 con 0x55f8ce793390 2024-09-21T06:18:49.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.468+0000 7fae557fa640 1 -- 172.21.15.3:0/2738403835 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fae5c024620 con 0x55f8ce793390 2024-09-21T06:18:49.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.468+0000 7fae7a095740 1 -- 172.21.15.3:0/2738403835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae5c0043d0 con 0x55f8ce793390 2024-09-21T06:18:49.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.468+0000 7fae557fa640 1 -- 172.21.15.3:0/2738403835 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fae5c033050 con 0x55f8ce793390 2024-09-21T06:18:49.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.468+0000 7fae7a095740 10 client.16647 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:49.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.468+0000 7fae7a095740 1 -- 172.21.15.3:0/2738403835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f8ceb75b70 con 0x55f8ce793390 2024-09-21T06:18:49.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.468+0000 7fae557fa640 1 -- 172.21.15.3:0/2738403835 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5c0043d0 con 0x55f8ce793390 2024-09-21T06:18:49.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.468+0000 7fae557fa640 1 -- 172.21.15.3:0/2738403835 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fae5c018070 con 0x55f8ce793390 2024-09-21T06:18:49.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.468+0000 7fae7a095740 10 client.16647 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:49.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.468+0000 7fae7a095740 10 client.16647 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:18:49.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.469+0000 7fae7a095740 20 client.16647 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:49.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.469+0000 7fae7a095740 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55f8ce7729f0 0x55f8ceb78ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.469+0000 7fae7a095740 4 client.16647 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:49.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.469+0000 7fae7a095740 1 -- 172.21.15.3:0/2738403835 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8cead3df0 con 0x55f8ce7729f0 2024-09-21T06:18:49.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.469+0000 7fae718bd640 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55f8ce7729f0 0x55f8ceb78ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.469+0000 7fae718bd640 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55f8ce7729f0 0x55f8ceb78ed0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1db5c4ab9a2f3dd9 server_cookie=6007ba98b6b9a454 in_seq=0 out_seq=0 2024-09-21T06:18:49.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.469+0000 7fae557fa640 10 client.16647 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:18:49.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae557fa640 1 -- 172.21.15.3:0/2738403835 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8cead3df0 con 0x55f8ce7729f0 2024-09-21T06:18:49.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae557fa640 10 client.16647 handle_command_reply: tid=0 2024-09-21T06:18:49.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae7a095740 1 client.16647 shutdown 2024-09-21T06:18:49.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae7a095740 1 -- 172.21.15.3:0/2738403835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 msgr2=0x55f8ceb67280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:49.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae7a095740 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb67280 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fae5c005890 tx=0x7fae5c019ba0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae7a095740 1 -- 172.21.15.3:0/2738403835 shutdown_connections 2024-09-21T06:18:49.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae7a095740 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55f8ce7729f0 0x55f8ceb78ed0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae7a095740 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce7b4070 0x55f8ceb73170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae557fa640 0 client.16647 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:18:49.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae7a095740 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb67280 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae7a095740 1 --2- 172.21.15.3:0/2738403835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb66d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae7a095740 1 -- 172.21.15.3:0/2738403835 >> 172.21.15.3:0/2738403835 conn(0x55f8ce7a8460 msgr2=0x55f8ceb5e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:49.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae7a095740 1 -- 172.21.15.3:0/2738403835 shutdown_connections 2024-09-21T06:18:49.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae7a095740 1 -- 172.21.15.3:0/2738403835 wait complete. 2024-09-21T06:18:49.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.470+0000 7fae7a095740 20 client.16647 trim_cache size 0 max 16384 2024-09-21T06:18:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.484+0000 7fae7a095740 1 Processor -- start 2024-09-21T06:18:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.484+0000 7fae7a095740 1 -- start start 2024-09-21T06:18:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.484+0000 7fae7a095740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb5d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.484+0000 7fae7a095740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb68970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.484+0000 7fae7a095740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce7b4070 0x55f8ceb58bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.484+0000 7fae7a095740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b1870 con 0x55f8ce793390 2024-09-21T06:18:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.484+0000 7fae7a095740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b33e0 con 0x55f8ce78b1b0 2024-09-21T06:18:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.484+0000 7fae7a095740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b1f40 con 0x55f8ce7b4070 2024-09-21T06:18:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.484+0000 7fae710bc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb68970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.484+0000 7fae710bc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb68970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33528/0 (socket says 172.21.15.3:33528) 2024-09-21T06:18:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.484+0000 7fae710bc640 1 -- 172.21.15.3:0/1575460713 learned_addr learned my addr 172.21.15.3:0/1575460713 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.484+0000 7fae718bd640 1 --2- 172.21.15.3:0/1575460713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb5d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.485+0000 7fae720be640 1 --2- 172.21.15.3:0/1575460713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce7b4070 0x55f8ceb58bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.485+0000 7fae710bc640 1 -- 172.21.15.3:0/1575460713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce7b4070 msgr2=0x55f8ceb58bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.485+0000 7fae710bc640 1 --2- 172.21.15.3:0/1575460713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce7b4070 0x55f8ceb58bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.485+0000 7fae710bc640 1 -- 172.21.15.3:0/1575460713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce78b1b0 msgr2=0x55f8ceb5d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.485+0000 7fae710bc640 1 --2- 172.21.15.3:0/1575460713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb5d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.485+0000 7fae710bc640 1 -- 172.21.15.3:0/1575460713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ceb68000 con 0x55f8ce793390 2024-09-21T06:18:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.485+0000 7fae720be640 1 --2- 172.21.15.3:0/1575460713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce7b4070 0x55f8ceb58bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:18:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.485+0000 7fae710bc640 1 --2- 172.21.15.3:0/1575460713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb68970 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fae5c012fd0 tx=0x7fae5c009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29510716684a046 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.485+0000 7fae577fe640 1 -- 172.21.15.3:0/1575460713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c028030 con 0x55f8ce793390 2024-09-21T06:18:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.485+0000 7fae577fe640 1 -- 172.21.15.3:0/1575460713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5c01f040 con 0x55f8ce793390 2024-09-21T06:18:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.485+0000 7fae577fe640 1 -- 172.21.15.3:0/1575460713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c018e70 con 0x55f8ce793390 2024-09-21T06:18:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.486+0000 7fae7a095740 1 -- 172.21.15.3:0/1575460713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 msgr2=0x55f8ceb68970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.486+0000 7fae7a095740 1 --2- 172.21.15.3:0/1575460713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb68970 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fae5c012fd0 tx=0x7fae5c009640 comp rx=0 tx=0).stop 2024-09-21T06:18:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.486+0000 7fae7a095740 1 -- 172.21.15.3:0/1575460713 shutdown_connections 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.486+0000 7fae7a095740 1 --2- 172.21.15.3:0/1575460713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce7b4070 0x55f8ceb58bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.486+0000 7fae7a095740 1 --2- 172.21.15.3:0/1575460713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb68970 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.486+0000 7fae7a095740 1 --2- 172.21.15.3:0/1575460713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb5d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.486+0000 7fae7a095740 1 -- 172.21.15.3:0/1575460713 >> 172.21.15.3:0/1575460713 conn(0x55f8ce7a8460 msgr2=0x55f8ceb68260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.486+0000 7fae7a095740 1 -- 172.21.15.3:0/1575460713 shutdown_connections 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.486+0000 7fae7a095740 1 -- 172.21.15.3:0/1575460713 wait complete. 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.487+0000 7fae7a095740 1 -- start start 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.487+0000 7fae7a095740 1 Processor -- start 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.487+0000 7fae7a095740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb6f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae7a095740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb6abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae7a095740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce7b4070 0x55f8ceb6b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae7a095740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b2ce0 con 0x55f8ce793390 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae7a095740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b2610 con 0x55f8ce7b4070 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae7a095740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1af850 con 0x55f8ce78b1b0 2024-09-21T06:18:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae7a095740 10 client.0 _ll_register_callbacks cb 0x7fae680eacc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae718bd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb6f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae718bd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb6f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46662/0 (socket says 172.21.15.3:46662) 2024-09-21T06:18:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae718bd640 1 -- 172.21.15.3:0/302303703 learned_addr learned my addr 172.21.15.3:0/302303703 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae710bc640 1 --2- 172.21.15.3:0/302303703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb6abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae720be640 1 --2- 172.21.15.3:0/302303703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce7b4070 0x55f8ceb6b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae718bd640 1 -- 172.21.15.3:0/302303703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce7b4070 msgr2=0x55f8ceb6b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae718bd640 1 --2- 172.21.15.3:0/302303703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce7b4070 0x55f8ceb6b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae718bd640 1 -- 172.21.15.3:0/302303703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 msgr2=0x55f8ceb6abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae718bd640 1 --2- 172.21.15.3:0/302303703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb6abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.488+0000 7fae718bd640 1 -- 172.21.15.3:0/302303703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ceb6b6e0 con 0x55f8ce78b1b0 2024-09-21T06:18:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.489+0000 7fae718bd640 1 --2- 172.21.15.3:0/302303703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb6f700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fae680ad170 tx=0x7fae68005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c9a36d22ea2f5ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.489+0000 7fae557fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:18:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.489+0000 7fae557fa640 1 -- 172.21.15.3:0/302303703 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68020040 con 0x55f8ce78b1b0 2024-09-21T06:18:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.489+0000 7fae557fa640 1 -- 172.21.15.3:0/302303703 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fae680a8d30 con 0x55f8ce78b1b0 2024-09-21T06:18:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.489+0000 7fae7a095740 1 -- 172.21.15.3:0/302303703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae5c01f040 con 0x55f8ce78b1b0 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.490+0000 7fae557fa640 1 -- 172.21.15.3:0/302303703 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fae680acb30 con 0x55f8ce78b1b0 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.490+0000 7fae7a095740 10 client.26191 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.490+0000 7fae7a095740 1 -- 172.21.15.3:0/302303703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f8cebe34a0 con 0x55f8ce78b1b0 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.490+0000 7fae557fa640 1 -- 172.21.15.3:0/302303703 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5c01f040 con 0x55f8ce78b1b0 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.490+0000 7fae557fa640 1 -- 172.21.15.3:0/302303703 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fae680d0ab0 con 0x55f8ce78b1b0 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.491+0000 7fae7a095740 10 client.26191 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.491+0000 7fae7a095740 10 client.26191 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.491+0000 7fae7a095740 20 client.26191 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.491+0000 7fae7a095740 1 --2- 172.21.15.3:0/302303703 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55f8ce747da0 0x55f8cebe5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.491+0000 7fae7a095740 4 client.26191 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.491+0000 7fae7a095740 1 -- 172.21.15.3:0/302303703 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8ceb67630 con 0x55f8ce747da0 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.491+0000 7fae710bc640 1 --2- 172.21.15.3:0/302303703 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55f8ce747da0 0x55f8cebe5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.491+0000 7fae557fa640 10 client.26191 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.491+0000 7fae710bc640 1 --2- 172.21.15.3:0/302303703 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55f8ce747da0 0x55f8cebe5bc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69bbdc21b1f6882 server_cookie=a549ea0c9a24c3d9 in_seq=0 out_seq=0 2024-09-21T06:18:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae557fa640 1 -- 172.21.15.3:0/302303703 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8ceb67630 con 0x55f8ce747da0 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae557fa640 10 client.26191 handle_command_reply: tid=0 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae7a095740 1 client.26191 shutdown 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae7a095740 1 -- 172.21.15.3:0/302303703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce78b1b0 msgr2=0x55f8ceb6f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae7a095740 1 --2- 172.21.15.3:0/302303703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb6f700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fae680ad170 tx=0x7fae68005880 comp rx=0 tx=0).stop 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae7a095740 1 -- 172.21.15.3:0/302303703 shutdown_connections 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae7a095740 1 --2- 172.21.15.3:0/302303703 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55f8ce747da0 0x55f8cebe5bc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae7a095740 1 --2- 172.21.15.3:0/302303703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ce7b4070 0x55f8ceb6b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae7a095740 1 --2- 172.21.15.3:0/302303703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ce793390 0x55f8ceb6abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae7a095740 1 --2- 172.21.15.3:0/302303703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8ce78b1b0 0x55f8ceb6f700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae7a095740 1 -- 172.21.15.3:0/302303703 >> 172.21.15.3:0/302303703 conn(0x55f8ce7a8460 msgr2=0x55f8ceb5ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae557fa640 0 client.26191 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae7a095740 1 -- 172.21.15.3:0/302303703 shutdown_connections 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae7a095740 1 -- 172.21.15.3:0/302303703 wait complete. 2024-09-21T06:18:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.492+0000 7fae7a095740 20 client.26191 trim_cache size 0 max 16384 2024-09-21T06:18:49.498 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:18:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.493+0000 7fae73b21640 1 -- 172.21.15.3:0/705943769 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fae580b6680 msgr2=0x7fae580b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.493+0000 7fae73b21640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fae580b6680 0x7fae580b8b90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fae64006010 tx=0x7fae64009480 comp rx=0 tx=0).stop 2024-09-21T06:18:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.493+0000 7fae73b21640 1 -- 172.21.15.3:0/705943769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae6c102ad0 msgr2=0x7fae6c199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.493+0000 7fae73b21640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae6c102ad0 0x7fae6c199090 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fae6800d950 tx=0x7fae6800de10 comp rx=0 tx=0).stop 2024-09-21T06:18:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.493+0000 7fae73b21640 1 -- 172.21.15.3:0/705943769 shutdown_connections 2024-09-21T06:18:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.493+0000 7fae73b21640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fae580b6680 0x7fae580b8b90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.493+0000 7fae73b21640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae6c1083c0 0x7fae6c1995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.493+0000 7fae73b21640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae6c108d10 0x7fae6c199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.493+0000 7fae73b21640 1 --2- 172.21.15.3:0/705943769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae6c102ad0 0x7fae6c199090 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.493+0000 7fae73b21640 1 -- 172.21.15.3:0/705943769 >> 172.21.15.3:0/705943769 conn(0x7fae6c0796d0 msgr2=0x7fae6c106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.493+0000 7fae73b21640 1 -- 172.21.15.3:0/705943769 shutdown_connections 2024-09-21T06:18:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:49.493+0000 7fae73b21640 1 -- 172.21.15.3:0/705943769 wait complete. 2024-09-21T06:18:49.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:49 smithi003 ceph-mon[31227]: pgmap v439: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 381 op/s 2024-09-21T06:18:49.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:49 smithi137 ceph-mon[34464]: pgmap v439: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 381 op/s 2024-09-21T06:18:49.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:49 smithi204 ceph-mon[34659]: pgmap v439: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 381 op/s 2024-09-21T06:18:49.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 346508 12.49 MB/sec execute 22 sec latency 43.427 ms 2024-09-21T06:18:50.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:18:50.528 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.651+0000 7fe525269640 1 -- 172.21.15.3:0/2572241907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe520109400 msgr2=0x7fe520109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.651+0000 7fe525269640 1 --2- 172.21.15.3:0/2572241907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe520109400 0x7fe520109810 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe508009900 tx=0x7fe50802a410 comp rx=0 tx=0).stop 2024-09-21T06:18:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.651+0000 7fe525269640 1 -- 172.21.15.3:0/2572241907 shutdown_connections 2024-09-21T06:18:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.651+0000 7fe525269640 1 --2- 172.21.15.3:0/2572241907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe520109d50 0x7fe520110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.651+0000 7fe525269640 1 --2- 172.21.15.3:0/2572241907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe520109400 0x7fe520109810 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.651+0000 7fe525269640 1 --2- 172.21.15.3:0/2572241907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe520103ad0 0x7fe520103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.651+0000 7fe525269640 1 -- 172.21.15.3:0/2572241907 >> 172.21.15.3:0/2572241907 conn(0x7fe5200ff510 msgr2=0x7fe520101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.651+0000 7fe525269640 1 -- 172.21.15.3:0/2572241907 shutdown_connections 2024-09-21T06:18:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.651+0000 7fe525269640 1 -- 172.21.15.3:0/2572241907 wait complete. 2024-09-21T06:18:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.652+0000 7fe525269640 1 Processor -- start 2024-09-21T06:18:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.652+0000 7fe525269640 1 -- start start 2024-09-21T06:18:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.652+0000 7fe525269640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe520103ad0 0x7fe52019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.652+0000 7fe525269640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe520109400 0x7fe52019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.652+0000 7fe525269640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe520109d50 0x7fe52019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.652+0000 7fe525269640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe520113a10 con 0x7fe520109d50 2024-09-21T06:18:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.652+0000 7fe525269640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe520113890 con 0x7fe520103ad0 2024-09-21T06:18:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.652+0000 7fe525269640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe520113b90 con 0x7fe520109400 2024-09-21T06:18:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.652+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe520109d50 0x7fe52019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.652+0000 7fe51effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe520103ad0 0x7fe52019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.652+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe520109d50 0x7fe52019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33566/0 (socket says 172.21.15.3:33566) 2024-09-21T06:18:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.652+0000 7fe51f7fe640 1 -- 172.21.15.3:0/3367730809 learned_addr learned my addr 172.21.15.3:0/3367730809 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.652+0000 7fe517fff640 1 --2- 172.21.15.3:0/3367730809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe520109400 0x7fe52019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.653+0000 7fe51f7fe640 1 -- 172.21.15.3:0/3367730809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe520109400 msgr2=0x7fe52019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.653+0000 7fe51f7fe640 1 --2- 172.21.15.3:0/3367730809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe520109400 0x7fe52019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.653+0000 7fe51f7fe640 1 -- 172.21.15.3:0/3367730809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe520103ad0 msgr2=0x7fe52019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.653+0000 7fe51f7fe640 1 --2- 172.21.15.3:0/3367730809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe520103ad0 0x7fe52019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.653+0000 7fe51f7fe640 1 -- 172.21.15.3:0/3367730809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5201a03a0 con 0x7fe520109d50 2024-09-21T06:18:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.653+0000 7fe51f7fe640 1 --2- 172.21.15.3:0/3367730809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe520109d50 0x7fe52019fca0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe51800efd0 tx=0x7fe51800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.653+0000 7fe51cff9640 1 -- 172.21.15.3:0/3367730809 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518003b70 con 0x7fe520109d50 2024-09-21T06:18:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.653+0000 7fe51cff9640 1 -- 172.21.15.3:0/3367730809 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe518003d10 con 0x7fe520109d50 2024-09-21T06:18:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.653+0000 7fe525269640 1 -- 172.21.15.3:0/3367730809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5201a0690 con 0x7fe520109d50 2024-09-21T06:18:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.653+0000 7fe51cff9640 1 -- 172.21.15.3:0/3367730809 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180108a0 con 0x7fe520109d50 2024-09-21T06:18:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.653+0000 7fe525269640 1 -- 172.21.15.3:0/3367730809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5201a6e20 con 0x7fe520109d50 2024-09-21T06:18:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.656+0000 7fe52c1d7740 1 Processor -- start 2024-09-21T06:18:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.656+0000 7fe51cff9640 1 -- 172.21.15.3:0/3367730809 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fe518010a60 con 0x7fe520109d50 2024-09-21T06:18:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.656+0000 7fe52c1d7740 1 -- start start 2024-09-21T06:18:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe52c1d7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6271b0 0x55d15b9cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe52c1d7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe52c1d7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6064d0 0x55d15b9cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe52c1d7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d15b9416f0 con 0x55d15b6064d0 2024-09-21T06:18:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe52c1d7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d15b930b20 con 0x55d15b6271b0 2024-09-21T06:18:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe52c1d7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d15b934930 con 0x55d15b5fe2f0 2024-09-21T06:18:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe51effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe517fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6271b0 0x55d15b9cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe51effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46688/0 (socket says 172.21.15.3:46688) 2024-09-21T06:18:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe51effd640 1 -- 172.21.15.3:0/2937625796 learned_addr learned my addr 172.21.15.3:0/2937625796 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe517fff640 1 --2- 172.21.15.3:0/2937625796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6064d0 0x55d15b9cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe51effd640 1 -- 172.21.15.3:0/2937625796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6271b0 msgr2=0x55d15b9cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe51effd640 1 --2- 172.21.15.3:0/2937625796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6271b0 0x55d15b9cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe51effd640 1 -- 172.21.15.3:0/2937625796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6064d0 msgr2=0x55d15b9cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe51effd640 1 --2- 172.21.15.3:0/2937625796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6064d0 0x55d15b9cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe51effd640 1 -- 172.21.15.3:0/2937625796 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d15b8f6530 con 0x55d15b5fe2f0 2024-09-21T06:18:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe517fff640 1 --2- 172.21.15.3:0/2937625796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6064d0 0x55d15b9cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe51cff9640 1 --2- 172.21.15.3:0/3367730809 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe50c0b6680 0x7fe50c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe51effd640 1 --2- 172.21.15.3:0/2937625796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9ccef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe510009cf0 tx=0x7fe510011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4339e2080d66b9de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe517fff640 1 --2- 172.21.15.3:0/3367730809 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe50c0b6680 0x7fe50c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.657+0000 7fe5157fa640 1 -- 172.21.15.3:0/2937625796 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510004230 con 0x55d15b5fe2f0 2024-09-21T06:18:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.658+0000 7fe51cff9640 1 -- 172.21.15.3:0/3367730809 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe5180e7b00 con 0x7fe520109d50 2024-09-21T06:18:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.658+0000 7fe5157fa640 1 -- 172.21.15.3:0/2937625796 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5100043d0 con 0x55d15b5fe2f0 2024-09-21T06:18:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.658+0000 7fe5157fa640 1 -- 172.21.15.3:0/2937625796 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5100197e0 con 0x55d15b5fe2f0 2024-09-21T06:18:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.658+0000 7fe52c1d7740 1 -- 172.21.15.3:0/2937625796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 msgr2=0x55d15b9ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.658+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/2937625796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9ccef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe510009cf0 tx=0x7fe510011810 comp rx=0 tx=0).stop 2024-09-21T06:18:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.658+0000 7fe517fff640 1 --2- 172.21.15.3:0/3367730809 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe50c0b6680 0x7fe50c0b8b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe508009230 tx=0x7fe508031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 -- 172.21.15.3:0/2937625796 shutdown_connections 2024-09-21T06:18:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/2937625796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9ccef0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/2937625796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6064d0 0x55d15b9cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/2937625796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6271b0 0x55d15b9cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 -- 172.21.15.3:0/2937625796 >> 172.21.15.3:0/2937625796 conn(0x55d15b61b300 msgr2=0x55d15b9ca8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 -- 172.21.15.3:0/2937625796 shutdown_connections 2024-09-21T06:18:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 -- 172.21.15.3:0/2937625796 wait complete. 2024-09-21T06:18:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 -- start start 2024-09-21T06:18:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 Processor -- start 2024-09-21T06:18:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b5fe2f0 0x55d15b9dc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 0x55d15b9dc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b6271b0 0x55d15b9e8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d15b868ab0 con 0x55d15b5fe2f0 2024-09-21T06:18:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d15b8d6730 con 0x55d15b6064d0 2024-09-21T06:18:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d15b9df510 con 0x55d15b6271b0 2024-09-21T06:18:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.659+0000 7fe52c1d7740 10 client.0 _ll_register_callbacks cb 0x55d15b92a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.660+0000 7fe51effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b5fe2f0 0x55d15b9dc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.660+0000 7fe517fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 0x55d15b9dc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.660+0000 7fe51effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b5fe2f0 0x55d15b9dc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33586/0 (socket says 172.21.15.3:33586) 2024-09-21T06:18:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.660+0000 7fe51effd640 1 -- 172.21.15.3:0/3310107717 learned_addr learned my addr 172.21.15.3:0/3310107717 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.660+0000 7fe51f7fe640 1 --2- 172.21.15.3:0/3310107717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b6271b0 0x55d15b9e8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.660+0000 7fe517fff640 1 -- 172.21.15.3:0/3310107717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b6271b0 msgr2=0x55d15b9e8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.660+0000 7fe517fff640 1 --2- 172.21.15.3:0/3310107717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b6271b0 0x55d15b9e8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.660+0000 7fe517fff640 1 -- 172.21.15.3:0/3310107717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b5fe2f0 msgr2=0x55d15b9dc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.660+0000 7fe517fff640 1 --2- 172.21.15.3:0/3310107717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b5fe2f0 0x55d15b9dc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.660+0000 7fe517fff640 1 -- 172.21.15.3:0/3310107717 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d15b692790 con 0x55d15b6064d0 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.660+0000 7fe51effd640 1 --2- 172.21.15.3:0/3310107717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b5fe2f0 0x55d15b9dc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.660+0000 7fe517fff640 1 --2- 172.21.15.3:0/3310107717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 0x55d15b9dc970 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe50802a930 tx=0x7fe50802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c6c158ec6483035 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.660+0000 7fe5067fc640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.661+0000 7fe5067fc640 1 -- 172.21.15.3:0/3310107717 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50804e070 con 0x55d15b6064d0 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.661+0000 7fe5067fc640 1 -- 172.21.15.3:0/3310107717 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe508005240 con 0x55d15b6064d0 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.661+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3310107717 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5100043d0 con 0x55d15b6064d0 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.661+0000 7fe5067fc640 1 -- 172.21.15.3:0/3310107717 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe50805a050 con 0x55d15b6064d0 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.661+0000 7fe52c1d7740 10 client.26243 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.661+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3310107717 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d15b70b620 con 0x55d15b6064d0 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.661+0000 7fe5067fc640 1 -- 172.21.15.3:0/3310107717 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5100043d0 con 0x55d15b6064d0 2024-09-21T06:18:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.661+0000 7fe5067fc640 1 -- 172.21.15.3:0/3310107717 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fe50803d5c0 con 0x55d15b6064d0 2024-09-21T06:18:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.661+0000 7fe52c1d7740 10 client.26243 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.661+0000 7fe52c1d7740 10 client.26243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.662+0000 7fe52c1d7740 20 client.26243 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.662+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/3310107717 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d15b5e5d80 0x55d15b9ee560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.662+0000 7fe52c1d7740 4 client.26243 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.662+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3310107717 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d15b943640 con 0x55d15b5e5d80 2024-09-21T06:18:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.662+0000 7fe51effd640 1 --2- 172.21.15.3:0/3310107717 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d15b5e5d80 0x55d15b9ee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.662+0000 7fe5067fc640 10 client.26243 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.662+0000 7fe51effd640 1 --2- 172.21.15.3:0/3310107717 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d15b5e5d80 0x55d15b9ee560 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a95ede70825125 server_cookie=3558fc9bf4ff1d1 in_seq=0 out_seq=0 2024-09-21T06:18:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.663+0000 7fe5067fc640 1 -- 172.21.15.3:0/3310107717 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d15b943640 con 0x55d15b5e5d80 2024-09-21T06:18:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.663+0000 7fe5067fc640 10 client.26243 handle_command_reply: tid=0 2024-09-21T06:18:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.663+0000 7fe52c1d7740 1 client.26243 shutdown 2024-09-21T06:18:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.664+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3310107717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 msgr2=0x55d15b9dc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.664+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/3310107717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 0x55d15b9dc970 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe50802a930 tx=0x7fe50802a410 comp rx=0 tx=0).stop 2024-09-21T06:18:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.664+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3310107717 shutdown_connections 2024-09-21T06:18:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.664+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/3310107717 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d15b5e5d80 0x55d15b9ee560 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.664+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/3310107717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b6271b0 0x55d15b9e8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.664+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/3310107717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 0x55d15b9dc970 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.664+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/3310107717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b5fe2f0 0x55d15b9dc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.664+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3310107717 >> 172.21.15.3:0/3310107717 conn(0x55d15b61b300 msgr2=0x55d15b9c8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.664+0000 7fe5067fc640 0 client.26243 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.664+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3310107717 shutdown_connections 2024-09-21T06:18:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.664+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3310107717 wait complete. 2024-09-21T06:18:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.664+0000 7fe52c1d7740 20 client.26243 trim_cache size 0 max 16384 2024-09-21T06:18:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.680+0000 7fe52c1d7740 1 Processor -- start 2024-09-21T06:18:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.680+0000 7fe52c1d7740 1 -- start start 2024-09-21T06:18:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.680+0000 7fe52c1d7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9dc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe52c1d7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 0x55d15b9e8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe52c1d7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6271b0 0x55d15b9d9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe52c1d7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201ad440 con 0x55d15b6271b0 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe52c1d7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201aefb0 con 0x55d15b6064d0 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe52c1d7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201adb10 con 0x55d15b5fe2f0 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe51effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe51effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9dc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46702/0 (socket says 172.21.15.3:46702) 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe51effd640 1 -- 172.21.15.3:0/1535050537 learned_addr learned my addr 172.21.15.3:0/1535050537 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe517fff640 1 --2- 172.21.15.3:0/1535050537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 0x55d15b9e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe51f7fe640 1 --2- 172.21.15.3:0/1535050537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6271b0 0x55d15b9d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe51effd640 1 -- 172.21.15.3:0/1535050537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 msgr2=0x55d15b9e8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe51effd640 1 --2- 172.21.15.3:0/1535050537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 0x55d15b9e8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe51effd640 1 -- 172.21.15.3:0/1535050537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6271b0 msgr2=0x55d15b9d9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe51effd640 1 --2- 172.21.15.3:0/1535050537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6271b0 0x55d15b9d9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe51effd640 1 -- 172.21.15.3:0/1535050537 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d15b9cecb0 con 0x55d15b5fe2f0 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.681+0000 7fe51effd640 1 --2- 172.21.15.3:0/1535050537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9dc910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe510009250 tx=0x7fe510019420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=640316337d427438 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.682+0000 7fe514ff9640 1 -- 172.21.15.3:0/1535050537 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5100037f0 con 0x55d15b5fe2f0 2024-09-21T06:18:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.682+0000 7fe514ff9640 1 -- 172.21.15.3:0/1535050537 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5100054f0 con 0x55d15b5fe2f0 2024-09-21T06:18:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.682+0000 7fe514ff9640 1 -- 172.21.15.3:0/1535050537 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5100048b0 con 0x55d15b5fe2f0 2024-09-21T06:18:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.683+0000 7fe52c1d7740 1 -- 172.21.15.3:0/1535050537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 msgr2=0x55d15b9dc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.683+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/1535050537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9dc910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe510009250 tx=0x7fe510019420 comp rx=0 tx=0).stop 2024-09-21T06:18:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.683+0000 7fe52c1d7740 1 -- 172.21.15.3:0/1535050537 shutdown_connections 2024-09-21T06:18:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.683+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/1535050537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6271b0 0x55d15b9d9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.683+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/1535050537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 0x55d15b9e8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.683+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/1535050537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9dc910 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.683+0000 7fe52c1d7740 1 -- 172.21.15.3:0/1535050537 >> 172.21.15.3:0/1535050537 conn(0x55d15b61b300 msgr2=0x55d15b9dac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.683+0000 7fe52c1d7740 1 -- 172.21.15.3:0/1535050537 shutdown_connections 2024-09-21T06:18:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.683+0000 7fe52c1d7740 1 -- 172.21.15.3:0/1535050537 wait complete. 2024-09-21T06:18:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.683+0000 7fe52c1d7740 1 -- start start 2024-09-21T06:18:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe52c1d7740 1 Processor -- start 2024-09-21T06:18:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe52c1d7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9d44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe52c1d7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 0x55d15b9d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe52c1d7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6271b0 0x55d15b9e3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe52c1d7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201ae8b0 con 0x55d15b6271b0 2024-09-21T06:18:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe52c1d7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201ae1e0 con 0x55d15b6064d0 2024-09-21T06:18:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe52c1d7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201ab420 con 0x55d15b5fe2f0 2024-09-21T06:18:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe52c1d7740 10 client.0 _ll_register_callbacks cb 0x7fe508009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6271b0 0x55d15b9e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe51effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9d44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6271b0 0x55d15b9e3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33608/0 (socket says 172.21.15.3:33608) 2024-09-21T06:18:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe51f7fe640 1 -- 172.21.15.3:0/3951237748 learned_addr learned my addr 172.21.15.3:0/3951237748 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe517fff640 1 --2- 172.21.15.3:0/3951237748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 0x55d15b9d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe51effd640 1 -- 172.21.15.3:0/3951237748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 msgr2=0x55d15b9d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe51effd640 1 --2- 172.21.15.3:0/3951237748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 0x55d15b9d4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe51effd640 1 -- 172.21.15.3:0/3951237748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6271b0 msgr2=0x55d15b9e3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe51effd640 1 --2- 172.21.15.3:0/3951237748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6271b0 0x55d15b9e3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe51effd640 1 -- 172.21.15.3:0/3951237748 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d15b9d52f0 con 0x55d15b5fe2f0 2024-09-21T06:18:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.684+0000 7fe51f7fe640 1 --2- 172.21.15.3:0/3951237748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6271b0 0x55d15b9e3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe51effd640 1 --2- 172.21.15.3:0/3951237748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9d44c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe510000c00 tx=0x7fe510009cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67d5e220768e423 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe5067fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:18:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe5067fc640 1 -- 172.21.15.3:0/3951237748 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510005040 con 0x55d15b5fe2f0 2024-09-21T06:18:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe5067fc640 1 -- 172.21.15.3:0/3951237748 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe5100051e0 con 0x55d15b5fe2f0 2024-09-21T06:18:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3951237748 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5100054f0 con 0x55d15b5fe2f0 2024-09-21T06:18:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe5067fc640 1 -- 172.21.15.3:0/3951237748 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5100035f0 con 0x55d15b5fe2f0 2024-09-21T06:18:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe52c1d7740 10 client.26209 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3951237748 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d15b9e4d90 con 0x55d15b5fe2f0 2024-09-21T06:18:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe5067fc640 1 -- 172.21.15.3:0/3951237748 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5100054f0 con 0x55d15b5fe2f0 2024-09-21T06:18:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe5067fc640 1 -- 172.21.15.3:0/3951237748 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fe51002b020 con 0x55d15b5fe2f0 2024-09-21T06:18:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe52c1d7740 10 client.26209 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe52c1d7740 10 client.26209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe52c1d7740 20 client.26209 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.685+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/3951237748 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d15b5bb1c0 0x55d15ba56f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.686+0000 7fe52c1d7740 4 client.26209 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:18:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.686+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3951237748 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d15b93f0f0 con 0x55d15b5bb1c0 2024-09-21T06:18:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.686+0000 7fe517fff640 1 --2- 172.21.15.3:0/3951237748 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d15b5bb1c0 0x55d15ba56f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.686+0000 7fe5067fc640 10 client.26209 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.686+0000 7fe517fff640 1 --2- 172.21.15.3:0/3951237748 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d15b5bb1c0 0x55d15ba56f60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b3898c73495d5e server_cookie=5c0d8ee30c0adba3 in_seq=0 out_seq=0 2024-09-21T06:18:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe5067fc640 1 -- 172.21.15.3:0/3951237748 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d15b93f0f0 con 0x55d15b5bb1c0 2024-09-21T06:18:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe5067fc640 10 client.26209 handle_command_reply: tid=0 2024-09-21T06:18:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe52c1d7740 1 client.26209 shutdown 2024-09-21T06:18:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3951237748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 msgr2=0x55d15b9d44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/3951237748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9d44c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe510000c00 tx=0x7fe510009cb0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3951237748 shutdown_connections 2024-09-21T06:18:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/3951237748 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d15b5bb1c0 0x55d15ba56f60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/3951237748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d15b6271b0 0x55d15b9e3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/3951237748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d15b6064d0 0x55d15b9d4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe5067fc640 0 client.26209 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe52c1d7740 1 --2- 172.21.15.3:0/3951237748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d15b5fe2f0 0x55d15b9d44c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3951237748 >> 172.21.15.3:0/3951237748 conn(0x55d15b61b300 msgr2=0x55d15b9dd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3951237748 shutdown_connections 2024-09-21T06:18:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe52c1d7740 1 -- 172.21.15.3:0/3951237748 wait complete. 2024-09-21T06:18:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe52c1d7740 20 client.26209 trim_cache size 0 max 16384 2024-09-21T06:18:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:18:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:18:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:18:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:18:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe525269640 1 -- 172.21.15.3:0/3367730809 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe50c0b6680 msgr2=0x7fe50c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe525269640 1 --2- 172.21.15.3:0/3367730809 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe50c0b6680 0x7fe50c0b8b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe508009230 tx=0x7fe508031400 comp rx=0 tx=0).stop 2024-09-21T06:18:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe525269640 1 -- 172.21.15.3:0/3367730809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe520109d50 msgr2=0x7fe52019fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe525269640 1 --2- 172.21.15.3:0/3367730809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe520109d50 0x7fe52019fca0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe51800efd0 tx=0x7fe51800c6a0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe525269640 1 -- 172.21.15.3:0/3367730809 shutdown_connections 2024-09-21T06:18:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe525269640 1 --2- 172.21.15.3:0/3367730809 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe50c0b6680 0x7fe50c0b8b90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe525269640 1 --2- 172.21.15.3:0/3367730809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe520109400 0x7fe52019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe525269640 1 --2- 172.21.15.3:0/3367730809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe520109d50 0x7fe52019fca0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe525269640 1 --2- 172.21.15.3:0/3367730809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe520103ad0 0x7fe52019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe525269640 1 -- 172.21.15.3:0/3367730809 >> 172.21.15.3:0/3367730809 conn(0x7fe5200ff510 msgr2=0x7fe520101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe525269640 1 -- 172.21.15.3:0/3367730809 shutdown_connections 2024-09-21T06:18:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:18:50.819+0000 7fe525269640 1 -- 172.21.15.3:0/3367730809 wait complete. 2024-09-21T06:18:50.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 348068 12.47 MB/sec execute 23 sec latency 61.167 ms 2024-09-21T06:18:51.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.109+0000 7f6b04ac4640 1 -- 172.21.15.3:0/1450615889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b001094a0 msgr2=0x7f6b001098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.109+0000 7f6b04ac4640 1 --2- 172.21.15.3:0/1450615889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b001094a0 0x7f6b001098b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009900 tx=0x7f6ae802a410 comp rx=0 tx=0).stop 2024-09-21T06:18:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.109+0000 7f6b04ac4640 1 -- 172.21.15.3:0/1450615889 shutdown_connections 2024-09-21T06:18:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.109+0000 7f6b04ac4640 1 --2- 172.21.15.3:0/1450615889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b00109df0 0x7f6b00110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.109+0000 7f6b04ac4640 1 --2- 172.21.15.3:0/1450615889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b001094a0 0x7f6b001098b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.109+0000 7f6b04ac4640 1 --2- 172.21.15.3:0/1450615889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00103b60 0x7f6b00103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.109+0000 7f6b04ac4640 1 -- 172.21.15.3:0/1450615889 >> 172.21.15.3:0/1450615889 conn(0x7f6b0007b850 msgr2=0x7f6b0007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.109+0000 7f6b04ac4640 1 -- 172.21.15.3:0/1450615889 shutdown_connections 2024-09-21T06:18:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.109+0000 7f6b04ac4640 1 -- 172.21.15.3:0/1450615889 wait complete. 2024-09-21T06:18:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.110+0000 7f6b04ac4640 1 Processor -- start 2024-09-21T06:18:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.110+0000 7f6b04ac4640 1 -- start start 2024-09-21T06:18:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.110+0000 7f6b04ac4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00103b60 0x7f6b00075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.110+0000 7f6b04ac4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b001094a0 0x7f6b00075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.110+0000 7f6b04ac4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b00109df0 0x7f6b00076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.110+0000 7f6b04ac4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6b00113ab0 con 0x7f6b001094a0 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6b04ac4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6b00113930 con 0x7f6b00109df0 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6b04ac4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b00113c30 con 0x7f6b00103b60 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6af7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b001094a0 0x7f6b00075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6afe59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00103b60 0x7f6b00075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6af7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b001094a0 0x7f6b00075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33618/0 (socket says 172.21.15.3:33618) 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6afe59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00103b60 0x7f6b00075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46722/0 (socket says 172.21.15.3:46722) 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6afed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b00109df0 0x7f6b00076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6af7fff640 1 -- 172.21.15.3:0/1407563058 learned_addr learned my addr 172.21.15.3:0/1407563058 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6af7fff640 1 -- 172.21.15.3:0/1407563058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00103b60 msgr2=0x7f6b00075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6af7fff640 1 --2- 172.21.15.3:0/1407563058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00103b60 0x7f6b00075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6af7fff640 1 -- 172.21.15.3:0/1407563058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b00109df0 msgr2=0x7f6b00076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6af7fff640 1 --2- 172.21.15.3:0/1407563058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b00109df0 0x7f6b00076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6af7fff640 1 -- 172.21.15.3:0/1407563058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b001a8840 con 0x7f6b001094a0 2024-09-21T06:18:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6afe59c640 1 --2- 172.21.15.3:0/1407563058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00103b60 0x7f6b00075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6af7fff640 1 --2- 172.21.15.3:0/1407563058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b001094a0 0x7f6b00075c50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6ae802a930 tx=0x7f6ae8031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6af77fe640 1 -- 172.21.15.3:0/1407563058 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae803f030 con 0x7f6b001094a0 2024-09-21T06:18:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.111+0000 7f6b04ac4640 1 -- 172.21.15.3:0/1407563058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b001a8ad0 con 0x7f6b001094a0 2024-09-21T06:18:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.112+0000 7f6af77fe640 1 -- 172.21.15.3:0/1407563058 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae8004490 con 0x7f6b001094a0 2024-09-21T06:18:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.112+0000 7f6af77fe640 1 -- 172.21.15.3:0/1407563058 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8035070 con 0x7f6b001094a0 2024-09-21T06:18:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.112+0000 7f6b04ac4640 1 -- 172.21.15.3:0/1407563058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b001a9000 con 0x7f6b001094a0 2024-09-21T06:18:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.114+0000 7f6b0b038740 1 Processor -- start 2024-09-21T06:18:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.115+0000 7f6b0b038740 1 -- start start 2024-09-21T06:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.115+0000 7f6b0b038740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb215d070 0x562bb25046a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.115+0000 7f6b0b038740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb21341b0 0x562bb2504be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.115+0000 7f6b0b038740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb213c390 0x562bb2505120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.115+0000 7f6b0b038740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562bb247aea0 con 0x562bb215d070 2024-09-21T06:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.115+0000 7f6b0b038740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562bb246a2d0 con 0x562bb21341b0 2024-09-21T06:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.115+0000 7f6b0b038740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562bb246e0e0 con 0x562bb213c390 2024-09-21T06:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.115+0000 7f6afed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb215d070 0x562bb25046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.115+0000 7f6afe59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb21341b0 0x562bb2504be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.115+0000 7f6afed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb215d070 0x562bb25046a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33634/0 (socket says 172.21.15.3:33634) 2024-09-21T06:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.115+0000 7f6afed9d640 1 -- 172.21.15.3:0/1051845190 learned_addr learned my addr 172.21.15.3:0/1051845190 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.116+0000 7f6afed9d640 1 -- 172.21.15.3:0/1051845190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb213c390 msgr2=0x562bb2505120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.116+0000 7f6afed9d640 1 --2- 172.21.15.3:0/1051845190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb213c390 0x562bb2505120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.116+0000 7f6afed9d640 1 -- 172.21.15.3:0/1051845190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb21341b0 msgr2=0x562bb2504be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.116+0000 7f6afed9d640 1 --2- 172.21.15.3:0/1051845190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb21341b0 0x562bb2504be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.116+0000 7f6afed9d640 1 -- 172.21.15.3:0/1051845190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb21c8650 con 0x562bb215d070 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.116+0000 7f6af77fe640 1 -- 172.21.15.3:0/1407563058 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f6ae8042030 con 0x7f6b001094a0 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.116+0000 7f6afed9d640 1 --2- 172.21.15.3:0/1051845190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb215d070 0x562bb25046a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009f50 tx=0x7f6af80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9aa2efaa13e7f22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.116+0000 7f6af4ff9640 1 -- 172.21.15.3:0/1051845190 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8004d20 con 0x562bb215d070 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.116+0000 7f6af4ff9640 1 -- 172.21.15.3:0/1051845190 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6af8004050 con 0x562bb215d070 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.116+0000 7f6af4ff9640 1 -- 172.21.15.3:0/1051845190 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8005220 con 0x562bb215d070 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.117+0000 7f6af77fe640 1 --2- 172.21.15.3:0/1407563058 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6aec0b6680 0x7f6aec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.117+0000 7f6afe59c640 1 --2- 172.21.15.3:0/1407563058 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6aec0b6680 0x7f6aec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.117+0000 7f6af77fe640 1 -- 172.21.15.3:0/1407563058 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f6ae8107a00 con 0x7f6b001094a0 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.117+0000 7f6b0b038740 1 -- 172.21.15.3:0/1051845190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb215d070 msgr2=0x562bb25046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.117+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1051845190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb215d070 0x562bb25046a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009f50 tx=0x7f6af80118d0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.117+0000 7f6b0b038740 1 -- 172.21.15.3:0/1051845190 shutdown_connections 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.117+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1051845190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb213c390 0x562bb2505120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.117+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1051845190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb21341b0 0x562bb2504be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.117+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1051845190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb215d070 0x562bb25046a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.117+0000 7f6b0b038740 1 -- 172.21.15.3:0/1051845190 >> 172.21.15.3:0/1051845190 conn(0x562bb2151460 msgr2=0x562bb2502590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.117+0000 7f6afe59c640 1 --2- 172.21.15.3:0/1407563058 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6aec0b6680 0x7f6aec0b8b90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x562bb2503320 tx=0x7f6af0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.117+0000 7f6b0b038740 1 -- 172.21.15.3:0/1051845190 shutdown_connections 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.118+0000 7f6b0b038740 1 -- 172.21.15.3:0/1051845190 wait complete. 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.118+0000 7f6b0b038740 1 -- start start 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.118+0000 7f6b0b038740 1 Processor -- start 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.118+0000 7f6b0b038740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb21341b0 0x562bb250fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.118+0000 7f6b0b038740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb213c390 0x562bb2510370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.118+0000 7f6b0b038740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb215d070 0x562bb251c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.118+0000 7f6b0b038740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562bb22cb700 con 0x562bb213c390 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.118+0000 7f6b0b038740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562bb2405160 con 0x562bb215d070 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.118+0000 7f6b0b038740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562bb2512e80 con 0x562bb21341b0 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6b0b038740 10 client.0 _ll_register_callbacks cb 0x562bb241e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6afe59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb21341b0 0x562bb250fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6afed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb215d070 0x562bb251c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6afe59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb21341b0 0x562bb250fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46736/0 (socket says 172.21.15.3:46736) 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6afe59c640 1 -- 172.21.15.3:0/4104312270 learned_addr learned my addr 172.21.15.3:0/4104312270 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6af7fff640 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb213c390 0x562bb2510370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6afe59c640 1 -- 172.21.15.3:0/4104312270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb215d070 msgr2=0x562bb251c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6afe59c640 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb215d070 0x562bb251c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6afe59c640 1 -- 172.21.15.3:0/4104312270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb213c390 msgr2=0x562bb2510370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6afe59c640 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb213c390 0x562bb2510370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6afe59c640 1 -- 172.21.15.3:0/4104312270 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb2432c90 con 0x562bb21341b0 2024-09-21T06:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6afed9d640 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb215d070 0x562bb251c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6af7fff640 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb213c390 0x562bb2510370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6afe59c640 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb21341b0 0x562bb250fe30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009f70 tx=0x7f6af0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb425b34b5dd9ba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.119+0000 7f6ae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6ae5ffb640 1 -- 172.21.15.3:0/4104312270 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af000f8b0 con 0x562bb21341b0 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6ae5ffb640 1 -- 172.21.15.3:0/4104312270 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f6af000fa50 con 0x562bb21341b0 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6b0b038740 1 -- 172.21.15.3:0/4104312270 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6af8004050 con 0x562bb21341b0 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6ae5ffb640 1 -- 172.21.15.3:0/4104312270 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6af000fc50 con 0x562bb21341b0 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6b0b038740 10 client.26218 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6b0b038740 1 -- 172.21.15.3:0/4104312270 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562bb251ec70 con 0x562bb21341b0 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6ae5ffb640 1 -- 172.21.15.3:0/4104312270 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6af8004050 con 0x562bb21341b0 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6ae5ffb640 1 -- 172.21.15.3:0/4104312270 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f6af0027070 con 0x562bb21341b0 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6b0b038740 10 client.26218 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6b0b038740 10 client.26218 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6b0b038740 20 client.26218 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6b0b038740 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562bb211b9f0 0x562bb25220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6b0b038740 4 client.26218 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.120+0000 7f6b0b038740 1 -- 172.21.15.3:0/4104312270 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bb247cdf0 con 0x562bb211b9f0 2024-09-21T06:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.121+0000 7f6af7fff640 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562bb211b9f0 0x562bb25220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.121+0000 7f6ae5ffb640 10 client.26218 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.121+0000 7f6af7fff640 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562bb211b9f0 0x562bb25220e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b586c3a36f13ff27 server_cookie=43a9f68b084bb12f in_seq=0 out_seq=0 2024-09-21T06:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6ae5ffb640 1 -- 172.21.15.3:0/4104312270 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562bb247cdf0 con 0x562bb211b9f0 2024-09-21T06:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6ae5ffb640 10 client.26218 handle_command_reply: tid=0 2024-09-21T06:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6b0b038740 1 client.26218 shutdown 2024-09-21T06:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6b0b038740 1 -- 172.21.15.3:0/4104312270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb21341b0 msgr2=0x562bb250fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6b0b038740 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb21341b0 0x562bb250fe30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009f70 tx=0x7f6af0004730 comp rx=0 tx=0).stop 2024-09-21T06:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6b0b038740 1 -- 172.21.15.3:0/4104312270 shutdown_connections 2024-09-21T06:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6b0b038740 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562bb211b9f0 0x562bb25220e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6b0b038740 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb215d070 0x562bb251c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6b0b038740 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb213c390 0x562bb2510370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6b0b038740 1 --2- 172.21.15.3:0/4104312270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb21341b0 0x562bb250fe30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6ae5ffb640 0 client.26218 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:18:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6b0b038740 1 -- 172.21.15.3:0/4104312270 >> 172.21.15.3:0/4104312270 conn(0x562bb2151460 msgr2=0x562bb2507880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6b0b038740 1 -- 172.21.15.3:0/4104312270 shutdown_connections 2024-09-21T06:18:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.122+0000 7f6b0b038740 1 -- 172.21.15.3:0/4104312270 wait complete. 2024-09-21T06:18:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.123+0000 7f6b0b038740 20 client.26218 trim_cache size 0 max 16384 2024-09-21T06:18:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.136+0000 7f6b0b038740 1 Processor -- start 2024-09-21T06:18:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.136+0000 7f6b0b038740 1 -- start start 2024-09-21T06:18:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.136+0000 7f6b0b038740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb21341b0 0x562bb25077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.136+0000 7f6b0b038740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb213c390 0x562bb2507ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.136+0000 7f6b0b038740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb215d070 0x562bb250d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6b0b038740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6b001b19c0 con 0x562bb21341b0 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6b0b038740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6b001b3530 con 0x562bb215d070 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6b0b038740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b001b2090 con 0x562bb213c390 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6afe59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb21341b0 0x562bb25077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6afe59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb21341b0 0x562bb25077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33662/0 (socket says 172.21.15.3:33662) 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6afe59c640 1 -- 172.21.15.3:0/1826906571 learned_addr learned my addr 172.21.15.3:0/1826906571 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6afed9d640 1 --2- 172.21.15.3:0/1826906571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb215d070 0x562bb250d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6af7fff640 1 --2- 172.21.15.3:0/1826906571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb213c390 0x562bb2507ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6afe59c640 1 -- 172.21.15.3:0/1826906571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb213c390 msgr2=0x562bb2507ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6afe59c640 1 --2- 172.21.15.3:0/1826906571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb213c390 0x562bb2507ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6afe59c640 1 -- 172.21.15.3:0/1826906571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb215d070 msgr2=0x562bb250d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6afe59c640 1 --2- 172.21.15.3:0/1826906571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb215d070 0x562bb250d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6afe59c640 1 -- 172.21.15.3:0/1826906571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb2506bc0 con 0x562bb21341b0 2024-09-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6af7fff640 1 --2- 172.21.15.3:0/1826906571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb213c390 0x562bb2507ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6afe59c640 1 --2- 172.21.15.3:0/1826906571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb21341b0 0x562bb25077a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6af00058c0 tx=0x7f6af00045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6760a264d50a10d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6ae7fff640 1 -- 172.21.15.3:0/1826906571 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0036030 con 0x562bb21341b0 2024-09-21T06:18:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6ae7fff640 1 -- 172.21.15.3:0/1826906571 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6af000f8b0 con 0x562bb21341b0 2024-09-21T06:18:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.137+0000 7f6ae7fff640 1 -- 172.21.15.3:0/1826906571 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0031480 con 0x562bb21341b0 2024-09-21T06:18:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.138+0000 7f6b0b038740 1 -- 172.21.15.3:0/1826906571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb21341b0 msgr2=0x562bb25077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.138+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1826906571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb21341b0 0x562bb25077a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6af00058c0 tx=0x7f6af00045c0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.138+0000 7f6b0b038740 1 -- 172.21.15.3:0/1826906571 shutdown_connections 2024-09-21T06:18:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.138+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1826906571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb213c390 0x562bb2507ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.138+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1826906571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb215d070 0x562bb250d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.138+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1826906571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb21341b0 0x562bb25077a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.138+0000 7f6b0b038740 1 -- 172.21.15.3:0/1826906571 >> 172.21.15.3:0/1826906571 conn(0x562bb2151460 msgr2=0x562bb250e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.138+0000 7f6b0b038740 1 -- 172.21.15.3:0/1826906571 shutdown_connections 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.138+0000 7f6b0b038740 1 -- 172.21.15.3:0/1826906571 wait complete. 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6b0b038740 1 -- start start 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6b0b038740 1 Processor -- start 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6b0b038740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb21341b0 0x562bb2518a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6b0b038740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb213c390 0x562bb2513f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6b0b038740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb215d070 0x562bb2514660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6b0b038740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6b001b2e30 con 0x562bb213c390 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6b0b038740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6b001b2760 con 0x562bb21341b0 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6b0b038740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b001af9a0 con 0x562bb215d070 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6b0b038740 10 client.0 _ll_register_callbacks cb 0x7f6af0009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6afe59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb21341b0 0x562bb2518a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6af7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb213c390 0x562bb2513f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6afe59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb21341b0 0x562bb2518a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:48448/0 (socket says 172.21.15.3:48448) 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6afe59c640 1 -- 172.21.15.3:0/1432471270 learned_addr learned my addr 172.21.15.3:0/1432471270 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.139+0000 7f6afed9d640 1 --2- 172.21.15.3:0/1432471270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb215d070 0x562bb2514660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.140+0000 7f6af7fff640 1 -- 172.21.15.3:0/1432471270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb215d070 msgr2=0x562bb2514660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.140+0000 7f6af7fff640 1 --2- 172.21.15.3:0/1432471270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb215d070 0x562bb2514660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.140+0000 7f6af7fff640 1 -- 172.21.15.3:0/1432471270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb21341b0 msgr2=0x562bb2518a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.140+0000 7f6af7fff640 1 --2- 172.21.15.3:0/1432471270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb21341b0 0x562bb2518a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.140+0000 7f6af7fff640 1 -- 172.21.15.3:0/1432471270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb258bae0 con 0x562bb213c390 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.140+0000 7f6afed9d640 1 --2- 172.21.15.3:0/1432471270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb215d070 0x562bb2514660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.140+0000 7f6af7fff640 1 --2- 172.21.15.3:0/1432471270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb213c390 0x562bb2513f70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8105190 tx=0x7f6ae802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6391b13c55d09db8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.140+0000 7f6ae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.140+0000 7f6ae5ffb640 1 -- 172.21.15.3:0/1432471270 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8049040 con 0x562bb213c390 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.140+0000 7f6ae5ffb640 1 -- 172.21.15.3:0/1432471270 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f6ae80c9bb0 con 0x562bb213c390 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.140+0000 7f6b0b038740 1 -- 172.21.15.3:0/1432471270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6af000f8b0 con 0x562bb213c390 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.140+0000 7f6ae5ffb640 1 -- 172.21.15.3:0/1432471270 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ae80d2370 con 0x562bb213c390 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.141+0000 7f6b0b038740 10 client.16689 fetch_fsmap learned FSMap version 30 2024-09-21T06:18:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.141+0000 7f6b0b038740 1 -- 172.21.15.3:0/1432471270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562bb258cc30 con 0x562bb213c390 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.141+0000 7f6ae5ffb640 1 -- 172.21.15.3:0/1432471270 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6af000f8b0 con 0x562bb213c390 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.141+0000 7f6ae5ffb640 1 -- 172.21.15.3:0/1432471270 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f6ae80d6e20 con 0x562bb213c390 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.141+0000 7f6b0b038740 10 client.16689 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.141+0000 7f6b0b038740 10 client.16689 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.141+0000 7f6b0b038740 20 client.16689 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.141+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1432471270 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562bb20f0da0 0x562bb258f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.141+0000 7f6b0b038740 4 client.16689 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.141+0000 7f6b0b038740 1 -- 172.21.15.3:0/1432471270 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562bb24788a0 con 0x562bb20f0da0 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.141+0000 7f6afe59c640 1 --2- 172.21.15.3:0/1432471270 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562bb20f0da0 0x562bb258f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6ae5ffb640 10 client.16689 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6afe59c640 1 --2- 172.21.15.3:0/1432471270 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562bb20f0da0 0x562bb258f3f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dde7a93d3094d863 server_cookie=dea44ac5edc46419 in_seq=0 out_seq=0 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6ae5ffb640 1 -- 172.21.15.3:0/1432471270 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562bb24788a0 con 0x562bb20f0da0 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6ae5ffb640 10 client.16689 handle_command_reply: tid=0 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6b0b038740 1 client.16689 shutdown 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6b0b038740 1 -- 172.21.15.3:0/1432471270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb213c390 msgr2=0x562bb2513f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1432471270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb213c390 0x562bb2513f70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8105190 tx=0x7f6ae802a410 comp rx=0 tx=0).stop 2024-09-21T06:18:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6b0b038740 1 -- 172.21.15.3:0/1432471270 shutdown_connections 2024-09-21T06:18:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1432471270 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562bb20f0da0 0x562bb258f3f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1432471270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bb215d070 0x562bb2514660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1432471270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bb213c390 0x562bb2513f70 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6b0b038740 1 --2- 172.21.15.3:0/1432471270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bb21341b0 0x562bb2518a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6b0b038740 1 -- 172.21.15.3:0/1432471270 >> 172.21.15.3:0/1432471270 conn(0x562bb2151460 msgr2=0x562bb25077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6ae5ffb640 0 client.16689 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:18:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6b0b038740 1 -- 172.21.15.3:0/1432471270 shutdown_connections 2024-09-21T06:18:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6b0b038740 1 -- 172.21.15.3:0/1432471270 wait complete. 2024-09-21T06:18:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.142+0000 7f6b0b038740 20 client.16689 trim_cache size 0 max 16384 2024-09-21T06:18:51.149 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:18:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.143+0000 7f6b04ac4640 1 -- 172.21.15.3:0/1407563058 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6aec0b6680 msgr2=0x7f6aec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.143+0000 7f6b04ac4640 1 --2- 172.21.15.3:0/1407563058 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6aec0b6680 0x7f6aec0b8b90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x562bb2503320 tx=0x7f6af0005f90 comp rx=0 tx=0).stop 2024-09-21T06:18:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.143+0000 7f6b04ac4640 1 -- 172.21.15.3:0/1407563058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b001094a0 msgr2=0x7f6b00075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.143+0000 7f6b04ac4640 1 --2- 172.21.15.3:0/1407563058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b001094a0 0x7f6b00075c50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6ae802a930 tx=0x7f6ae8031a20 comp rx=0 tx=0).stop 2024-09-21T06:18:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.143+0000 7f6b04ac4640 1 -- 172.21.15.3:0/1407563058 shutdown_connections 2024-09-21T06:18:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.143+0000 7f6b04ac4640 1 --2- 172.21.15.3:0/1407563058 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6aec0b6680 0x7f6aec0b8b90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.143+0000 7f6b04ac4640 1 --2- 172.21.15.3:0/1407563058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b00109df0 0x7f6b00076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.143+0000 7f6b04ac4640 1 --2- 172.21.15.3:0/1407563058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b001094a0 0x7f6b00075c50 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.143+0000 7f6b04ac4640 1 --2- 172.21.15.3:0/1407563058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00103b60 0x7f6b00075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:51.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.143+0000 7f6b04ac4640 1 -- 172.21.15.3:0/1407563058 >> 172.21.15.3:0/1407563058 conn(0x7f6b0007b850 msgr2=0x7f6b00105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:51.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.143+0000 7f6b04ac4640 1 -- 172.21.15.3:0/1407563058 shutdown_connections 2024-09-21T06:18:51.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:51.143+0000 7f6b04ac4640 1 -- 172.21.15.3:0/1407563058 wait complete. 2024-09-21T06:18:51.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:51 smithi003 ceph-mon[31227]: pgmap v440: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 412 op/s 2024-09-21T06:18:51.884 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:18:51.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:51 smithi137 ceph-mon[34464]: pgmap v440: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 412 op/s 2024-09-21T06:18:51.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:51 smithi204 ceph-mon[34659]: pgmap v440: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 412 op/s 2024-09-21T06:18:51.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 350582 12.47 MB/sec execute 24 sec latency 59.392 ms 2024-09-21T06:18:52.175 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:52.555 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:52 smithi003 ceph-mon[31227]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:18:52.555 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:52 smithi003 ceph-mon[31227]: pgmap v441: 97 pgs: 97 active+clean; 67 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 352 op/s 2024-09-21T06:18:52.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.784+0000 7fccdfc71640 1 -- 172.21.15.3:0/2692727154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd81051a0 msgr2=0x7fccd81055b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:52.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.784+0000 7fccdfc71640 1 --2- 172.21.15.3:0/2692727154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd81051a0 0x7fccd81055b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009900 tx=0x7fccc802a410 comp rx=0 tx=0).stop 2024-09-21T06:18:52.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.784+0000 7fccdfc71640 1 -- 172.21.15.3:0/2692727154 shutdown_connections 2024-09-21T06:18:52.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.784+0000 7fccdfc71640 1 --2- 172.21.15.3:0/2692727154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccd8105af0 0x7fccd810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.784+0000 7fccdfc71640 1 --2- 172.21.15.3:0/2692727154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd81051a0 0x7fccd81055b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.784+0000 7fccdfc71640 1 --2- 172.21.15.3:0/2692727154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8103190 0x7fccd81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.784+0000 7fccdfc71640 1 -- 172.21.15.3:0/2692727154 >> 172.21.15.3:0/2692727154 conn(0x7fccd8077cc0 msgr2=0x7fccd8078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:52.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.784+0000 7fccdfc71640 1 -- 172.21.15.3:0/2692727154 shutdown_connections 2024-09-21T06:18:52.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.784+0000 7fccdfc71640 1 -- 172.21.15.3:0/2692727154 wait complete. 2024-09-21T06:18:52.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.785+0000 7fccdfc71640 1 Processor -- start 2024-09-21T06:18:52.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.785+0000 7fccdfc71640 1 -- start start 2024-09-21T06:18:52.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.785+0000 7fccdfc71640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8103190 0x7fccd819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.785+0000 7fccdfc71640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccd81051a0 0x7fccd819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.785+0000 7fccdfc71640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd8105af0 0x7fccd819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.785+0000 7fccdfc71640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81129f0 con 0x7fccd8105af0 2024-09-21T06:18:52.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.785+0000 7fccdfc71640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8112870 con 0x7fccd81051a0 2024-09-21T06:18:52.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.785+0000 7fccdfc71640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8112b70 con 0x7fccd8103190 2024-09-21T06:18:52.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.785+0000 7fccde20e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd8105af0 0x7fccd819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.785+0000 7fccde20e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd8105af0 0x7fccd819bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33682/0 (socket says 172.21.15.3:33682) 2024-09-21T06:18:52.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.785+0000 7fccde20e640 1 -- 172.21.15.3:0/3156427969 learned_addr learned my addr 172.21.15.3:0/3156427969 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:52.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.785+0000 7fccdd20c640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccd81051a0 0x7fccd819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.785+0000 7fccdda0d640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8103190 0x7fccd819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.786+0000 7fccde20e640 1 -- 172.21.15.3:0/3156427969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8103190 msgr2=0x7fccd819b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.786+0000 7fccde20e640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8103190 0x7fccd819b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.786+0000 7fccde20e640 1 -- 172.21.15.3:0/3156427969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccd81051a0 msgr2=0x7fccd819b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.786+0000 7fccde20e640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccd81051a0 0x7fccd819b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.786+0000 7fccde20e640 1 -- 172.21.15.3:0/3156427969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd81a6590 con 0x7fccd8105af0 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.786+0000 7fccde20e640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd8105af0 0x7fccd819bd00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fccd000eb10 tx=0x7fccd000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.786+0000 7fccceffd640 1 -- 172.21.15.3:0/3156427969 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0004b20 con 0x7fccd8105af0 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.786+0000 7fccceffd640 1 -- 172.21.15.3:0/3156427969 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd0004cc0 con 0x7fccd8105af0 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.786+0000 7fccdfc71640 1 -- 172.21.15.3:0/3156427969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd81a6880 con 0x7fccd8105af0 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.786+0000 7fccceffd640 1 -- 172.21.15.3:0/3156427969 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0010670 con 0x7fccd8105af0 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.786+0000 7fccdfc71640 1 -- 172.21.15.3:0/3156427969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd81a6d90 con 0x7fccd8105af0 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.788+0000 7fcce61e5740 1 Processor -- start 2024-09-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.789+0000 7fccceffd640 1 -- 172.21.15.3:0/3156427969 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fccd0005020 con 0x7fccd8105af0 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.789+0000 7fcce61e5740 1 -- start start 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.789+0000 7fcce61e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfdd070 0x55f03e3846a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.789+0000 7fcce61e5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfb41b0 0x55f03e384be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.789+0000 7fcce61e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfbc390 0x55f03e385120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.789+0000 7fcce61e5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f03e2faea0 con 0x55f03dfb41b0 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.789+0000 7fcce61e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f03e2ea2d0 con 0x55f03dfbc390 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.789+0000 7fcce61e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f03e2ee0e0 con 0x55f03dfdd070 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.789+0000 7fccdda0d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfb41b0 0x55f03e384be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.789+0000 7fccdda0d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfb41b0 0x55f03e384be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33686/0 (socket says 172.21.15.3:33686) 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.789+0000 7fccdda0d640 1 -- 172.21.15.3:0/1281345537 learned_addr learned my addr 172.21.15.3:0/1281345537 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccdd20c640 1 --2- 172.21.15.3:0/1281345537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfbc390 0x55f03e385120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccdd20c640 1 --2- 172.21.15.3:0/1281345537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfdd070 0x55f03e3846a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccdda0d640 1 -- 172.21.15.3:0/1281345537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfdd070 msgr2=0x55f03e3846a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccdda0d640 1 --2- 172.21.15.3:0/1281345537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfdd070 0x55f03e3846a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccdda0d640 1 -- 172.21.15.3:0/1281345537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfbc390 msgr2=0x55f03e385120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccdda0d640 1 --2- 172.21.15.3:0/1281345537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfbc390 0x55f03e385120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccdda0d640 1 -- 172.21.15.3:0/1281345537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03e048650 con 0x55f03dfb41b0 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccdd20c640 1 --2- 172.21.15.3:0/1281345537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfdd070 0x55f03e3846a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccceffd640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fccc40b6700 0x7fccc40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccdd20c640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fccc40b6700 0x7fccc40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccdda0d640 1 --2- 172.21.15.3:0/1281345537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfb41b0 0x55f03e384be0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009ca0 tx=0x7fccd4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106c7299bdfdff5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccceffd640 1 -- 172.21.15.3:0/3156427969 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fccd00e7c30 con 0x7fccd8105af0 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccc3fff640 1 -- 172.21.15.3:0/1281345537 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4005220 con 0x55f03dfb41b0 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccc3fff640 1 -- 172.21.15.3:0/1281345537 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd4004c50 con 0x55f03dfb41b0 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.790+0000 7fccc3fff640 1 -- 172.21.15.3:0/1281345537 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4019760 con 0x55f03dfb41b0 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.791+0000 7fcce61e5740 1 -- 172.21.15.3:0/1281345537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfb41b0 msgr2=0x55f03e384be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.791+0000 7fcce61e5740 1 --2- 172.21.15.3:0/1281345537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfb41b0 0x55f03e384be0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009ca0 tx=0x7fccd4011810 comp rx=0 tx=0).stop 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.791+0000 7fccdd20c640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fccc40b6700 0x7fccc40b8c10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009230 tx=0x7fccc8041040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.791+0000 7fcce61e5740 1 -- 172.21.15.3:0/1281345537 shutdown_connections 2024-09-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.791+0000 7fcce61e5740 1 --2- 172.21.15.3:0/1281345537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfbc390 0x55f03e385120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.791+0000 7fcce61e5740 1 --2- 172.21.15.3:0/1281345537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfb41b0 0x55f03e384be0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.791+0000 7fcce61e5740 1 --2- 172.21.15.3:0/1281345537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfdd070 0x55f03e3846a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.791+0000 7fcce61e5740 1 -- 172.21.15.3:0/1281345537 >> 172.21.15.3:0/1281345537 conn(0x55f03dfd1460 msgr2=0x55f03e382590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:52.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.791+0000 7fcce61e5740 1 -- 172.21.15.3:0/1281345537 shutdown_connections 2024-09-21T06:18:52.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.791+0000 7fcce61e5740 1 -- 172.21.15.3:0/1281345537 wait complete. 2024-09-21T06:18:52.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.792+0000 7fcce61e5740 1 -- start start 2024-09-21T06:18:52.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.792+0000 7fcce61e5740 1 Processor -- start 2024-09-21T06:18:52.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.792+0000 7fcce61e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfb41b0 0x55f03e38fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.792+0000 7fcce61e5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 0x55f03e390240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.792+0000 7fcce61e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfdd070 0x55f03e39c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.792+0000 7fcce61e5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f03e14b700 con 0x55f03dfbc390 2024-09-21T06:18:52.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.792+0000 7fcce61e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f03e285160 con 0x55f03dfdd070 2024-09-21T06:18:52.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.792+0000 7fcce61e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f03e392e80 con 0x55f03dfb41b0 2024-09-21T06:18:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fcce61e5740 10 client.0 _ll_register_callbacks cb 0x55f03e29e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccdd20c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 0x55f03e390240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccdd20c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 0x55f03e390240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33688/0 (socket says 172.21.15.3:33688) 2024-09-21T06:18:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccdd20c640 1 -- 172.21.15.3:0/1989415760 learned_addr learned my addr 172.21.15.3:0/1989415760 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccde20e640 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfdd070 0x55f03e39c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccdda0d640 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfb41b0 0x55f03e38fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccdd20c640 1 -- 172.21.15.3:0/1989415760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfb41b0 msgr2=0x55f03e38fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccdd20c640 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfb41b0 0x55f03e38fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccdd20c640 1 -- 172.21.15.3:0/1989415760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfdd070 msgr2=0x55f03e39c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccdd20c640 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfdd070 0x55f03e39c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccdd20c640 1 -- 172.21.15.3:0/1989415760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f03e2b2c90 con 0x55f03dfbc390 2024-09-21T06:18:52.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccde20e640 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfdd070 0x55f03e39c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccdd20c640 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 0x55f03e390240 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fccc8033000 tx=0x7fccc8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e165ee0050448556 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccc17fa640 1 -- 172.21.15.3:0/1989415760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8050070 con 0x55f03dfbc390 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fccc17fa640 1 -- 172.21.15.3:0/1989415760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fccc8004450 con 0x55f03dfbc390 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.793+0000 7fcce61e5740 1 -- 172.21.15.3:0/1989415760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd4004c50 con 0x55f03dfbc390 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.794+0000 7fccc17fa640 1 -- 172.21.15.3:0/1989415760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fccc805c050 con 0x55f03dfbc390 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.794+0000 7fcce61e5740 10 client.16701 fetch_fsmap learned FSMap version 31 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.794+0000 7fcce61e5740 1 -- 172.21.15.3:0/1989415760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f03e39eb30 con 0x55f03dfbc390 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.794+0000 7fccc17fa640 1 -- 172.21.15.3:0/1989415760 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd4004c50 con 0x55f03dfbc390 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.794+0000 7fccc17fa640 1 -- 172.21.15.3:0/1989415760 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fccc80035f0 con 0x55f03dfbc390 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.794+0000 7fcce61e5740 10 client.16701 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.794+0000 7fcce61e5740 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.794+0000 7fcce61e5740 20 client.16701 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.794+0000 7fcce61e5740 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f03df9b9f0 0x55f03e3a2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.794+0000 7fcce61e5740 4 client.16701 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.794+0000 7fcce61e5740 1 -- 172.21.15.3:0/1989415760 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f03e2fcdf0 con 0x55f03df9b9f0 2024-09-21T06:18:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.794+0000 7fccdda0d640 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f03df9b9f0 0x55f03e3a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.795+0000 7fccc17fa640 10 client.16701 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.795+0000 7fccdda0d640 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f03df9b9f0 0x55f03e3a2010 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4750453db0a3cc06 server_cookie=2ec13ef867161097 in_seq=0 out_seq=0 2024-09-21T06:18:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fccc17fa640 1 -- 172.21.15.3:0/1989415760 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f03e2fcdf0 con 0x55f03df9b9f0 2024-09-21T06:18:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fccc17fa640 10 client.16701 handle_command_reply: tid=0 2024-09-21T06:18:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fcce61e5740 1 client.16701 shutdown 2024-09-21T06:18:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fcce61e5740 1 -- 172.21.15.3:0/1989415760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 msgr2=0x55f03e390240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fcce61e5740 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 0x55f03e390240 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fccc8033000 tx=0x7fccc8009260 comp rx=0 tx=0).stop 2024-09-21T06:18:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fcce61e5740 1 -- 172.21.15.3:0/1989415760 shutdown_connections 2024-09-21T06:18:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fcce61e5740 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f03df9b9f0 0x55f03e3a2010 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fcce61e5740 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfdd070 0x55f03e39c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fcce61e5740 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 0x55f03e390240 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fccc17fa640 0 client.16701 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fcce61e5740 1 --2- 172.21.15.3:0/1989415760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfb41b0 0x55f03e38fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fcce61e5740 1 -- 172.21.15.3:0/1989415760 >> 172.21.15.3:0/1989415760 conn(0x55f03dfd1460 msgr2=0x55f03e387830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fcce61e5740 1 -- 172.21.15.3:0/1989415760 shutdown_connections 2024-09-21T06:18:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fcce61e5740 1 -- 172.21.15.3:0/1989415760 wait complete. 2024-09-21T06:18:52.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.796+0000 7fcce61e5740 20 client.16701 trim_cache size 0 max 16384 2024-09-21T06:18:52.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.811+0000 7fcce61e5740 1 Processor -- start 2024-09-21T06:18:52.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.811+0000 7fcce61e5740 1 -- start start 2024-09-21T06:18:52.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.811+0000 7fcce61e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfb41b0 0x55f03e3901c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.811+0000 7fcce61e5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 0x55f03e39c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.811+0000 7fcce61e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfdd070 0x55f03e38cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.811+0000 7fcce61e5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81af610 con 0x55f03dfbc390 2024-09-21T06:18:52.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.811+0000 7fcce61e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81b1150 con 0x55f03dfdd070 2024-09-21T06:18:52.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.811+0000 7fcce61e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81afce0 con 0x55f03dfb41b0 2024-09-21T06:18:52.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.811+0000 7fccdd20c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 0x55f03e39c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.811+0000 7fccdd20c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 0x55f03e39c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33696/0 (socket says 172.21.15.3:33696) 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.811+0000 7fccdd20c640 1 -- 172.21.15.3:0/6481863 learned_addr learned my addr 172.21.15.3:0/6481863 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.811+0000 7fccde20e640 1 --2- 172.21.15.3:0/6481863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfdd070 0x55f03e38cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.811+0000 7fccdda0d640 1 --2- 172.21.15.3:0/6481863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfb41b0 0x55f03e3901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.812+0000 7fccde20e640 1 -- 172.21.15.3:0/6481863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfb41b0 msgr2=0x55f03e3901c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.812+0000 7fccde20e640 1 --2- 172.21.15.3:0/6481863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfb41b0 0x55f03e3901c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.812+0000 7fccde20e640 1 -- 172.21.15.3:0/6481863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 msgr2=0x55f03e39c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.812+0000 7fccde20e640 1 --2- 172.21.15.3:0/6481863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 0x55f03e39c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.812+0000 7fccde20e640 1 -- 172.21.15.3:0/6481863 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03e390fc0 con 0x55f03dfdd070 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.812+0000 7fccdd20c640 1 --2- 172.21.15.3:0/6481863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 0x55f03e39c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.812+0000 7fccde20e640 1 --2- 172.21.15.3:0/6481863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfdd070 0x55f03e38cef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fccd00e53c0 tx=0x7fccd0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75666f5eda9b8f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.812+0000 7fccc37fe640 1 -- 172.21.15.3:0/6481863 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd00ad230 con 0x55f03dfdd070 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.812+0000 7fccc37fe640 1 -- 172.21.15.3:0/6481863 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd00ad5d0 con 0x55f03dfdd070 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.812+0000 7fccc37fe640 1 -- 172.21.15.3:0/6481863 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd00a9e80 con 0x55f03dfdd070 2024-09-21T06:18:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.813+0000 7fcce61e5740 1 -- 172.21.15.3:0/6481863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfdd070 msgr2=0x55f03e38cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:52.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.813+0000 7fcce61e5740 1 --2- 172.21.15.3:0/6481863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfdd070 0x55f03e38cef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fccd00e53c0 tx=0x7fccd0004a40 comp rx=0 tx=0).stop 2024-09-21T06:18:52.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.813+0000 7fcce61e5740 1 -- 172.21.15.3:0/6481863 shutdown_connections 2024-09-21T06:18:52.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.813+0000 7fcce61e5740 1 --2- 172.21.15.3:0/6481863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfdd070 0x55f03e38cef0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.813+0000 7fcce61e5740 1 --2- 172.21.15.3:0/6481863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfbc390 0x55f03e39c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.813+0000 7fcce61e5740 1 --2- 172.21.15.3:0/6481863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfb41b0 0x55f03e3901c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.813+0000 7fcce61e5740 1 -- 172.21.15.3:0/6481863 >> 172.21.15.3:0/6481863 conn(0x55f03dfd1460 msgr2=0x55f03e38fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:52.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.813+0000 7fcce61e5740 1 -- 172.21.15.3:0/6481863 shutdown_connections 2024-09-21T06:18:52.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.813+0000 7fcce61e5740 1 -- 172.21.15.3:0/6481863 wait complete. 2024-09-21T06:18:52.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fcce61e5740 1 -- start start 2024-09-21T06:18:52.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fcce61e5740 1 Processor -- start 2024-09-21T06:18:52.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fcce61e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfb41b0 0x55f03e408420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fcce61e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfbc390 0x55f03e3980f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fcce61e5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfdd070 0x55f03e398800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fcce61e5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81b0a80 con 0x55f03dfdd070 2024-09-21T06:18:52.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fcce61e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81b03b0 con 0x55f03dfb41b0 2024-09-21T06:18:52.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fcce61e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81ad5f0 con 0x55f03dfbc390 2024-09-21T06:18:52.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fcce61e5740 10 client.0 _ll_register_callbacks cb 0x7fccc8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:52.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fccdda0d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfb41b0 0x55f03e408420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fccde20e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfdd070 0x55f03e398800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fccdd20c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfbc390 0x55f03e3980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fccdd20c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfbc390 0x55f03e3980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46810/0 (socket says 172.21.15.3:46810) 2024-09-21T06:18:52.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fccde20e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfdd070 0x55f03e398800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33708/0 (socket says 172.21.15.3:33708) 2024-09-21T06:18:52.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.814+0000 7fccdd20c640 1 -- 172.21.15.3:0/3584855495 learned_addr learned my addr 172.21.15.3:0/3584855495 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:52.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.815+0000 7fccdda0d640 1 -- 172.21.15.3:0/3584855495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfbc390 msgr2=0x55f03e3980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:52.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.815+0000 7fccdda0d640 1 --2- 172.21.15.3:0/3584855495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfbc390 0x55f03e3980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.815+0000 7fccdda0d640 1 -- 172.21.15.3:0/3584855495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfdd070 msgr2=0x55f03e398800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.815+0000 7fccdda0d640 1 --2- 172.21.15.3:0/3584855495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfdd070 0x55f03e398800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.815+0000 7fccdda0d640 1 -- 172.21.15.3:0/3584855495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f03e4058f0 con 0x55f03dfb41b0 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.815+0000 7fccdd20c640 1 --2- 172.21.15.3:0/3584855495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfbc390 0x55f03e3980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.815+0000 7fccdda0d640 1 --2- 172.21.15.3:0/3584855495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfb41b0 0x55f03e408420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009f50 tx=0x7fccd4009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77152302bd086a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.815+0000 7fccc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.815+0000 7fccc17fa640 1 -- 172.21.15.3:0/3584855495 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4011d10 con 0x55f03dfb41b0 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.815+0000 7fccc17fa640 1 -- 172.21.15.3:0/3584855495 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fccd4004250 con 0x55f03dfb41b0 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.815+0000 7fcce61e5740 1 -- 172.21.15.3:0/3584855495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd00ad5d0 con 0x55f03dfb41b0 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.816+0000 7fccc17fa640 1 -- 172.21.15.3:0/3584855495 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fccd403c050 con 0x55f03dfb41b0 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.816+0000 7fcce61e5740 10 client.26285 fetch_fsmap learned FSMap version 31 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.816+0000 7fcce61e5740 1 -- 172.21.15.3:0/3584855495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f03e410ff0 con 0x55f03dfb41b0 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.816+0000 7fccc17fa640 1 -- 172.21.15.3:0/3584855495 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd00ad5d0 con 0x55f03dfb41b0 2024-09-21T06:18:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.816+0000 7fccc17fa640 1 -- 172.21.15.3:0/3584855495 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fccd402d420 con 0x55f03dfb41b0 2024-09-21T06:18:52.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.816+0000 7fcce61e5740 10 client.26285 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:18:52.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.816+0000 7fcce61e5740 10 client.26285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:52.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.816+0000 7fcce61e5740 20 client.26285 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:52.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.816+0000 7fcce61e5740 1 --2- 172.21.15.3:0/3584855495 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f03df70da0 0x55f03e4135e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:52.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.816+0000 7fcce61e5740 4 client.26285 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:18:52.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.816+0000 7fcce61e5740 1 -- 172.21.15.3:0/3584855495 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f03e2f88a0 con 0x55f03df70da0 2024-09-21T06:18:52.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.816+0000 7fccdd20c640 1 --2- 172.21.15.3:0/3584855495 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f03df70da0 0x55f03e4135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:52.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.817+0000 7fccc17fa640 10 client.26285 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:52.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.817+0000 7fccdd20c640 1 --2- 172.21.15.3:0/3584855495 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f03df70da0 0x55f03e4135e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3242669b01b3b0a8 server_cookie=6fcc8a207bd049d1 in_seq=0 out_seq=0 2024-09-21T06:18:52.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fccc17fa640 1 -- 172.21.15.3:0/3584855495 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f03e2f88a0 con 0x55f03df70da0 2024-09-21T06:18:52.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fccc17fa640 10 client.26285 handle_command_reply: tid=0 2024-09-21T06:18:52.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fcce61e5740 1 client.26285 shutdown 2024-09-21T06:18:52.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fcce61e5740 1 -- 172.21.15.3:0/3584855495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfb41b0 msgr2=0x55f03e408420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:52.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fcce61e5740 1 --2- 172.21.15.3:0/3584855495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfb41b0 0x55f03e408420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009f50 tx=0x7fccd4009620 comp rx=0 tx=0).stop 2024-09-21T06:18:52.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fcce61e5740 1 -- 172.21.15.3:0/3584855495 shutdown_connections 2024-09-21T06:18:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fcce61e5740 1 --2- 172.21.15.3:0/3584855495 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f03df70da0 0x55f03e4135e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fcce61e5740 1 --2- 172.21.15.3:0/3584855495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f03dfbc390 0x55f03e3980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fcce61e5740 1 --2- 172.21.15.3:0/3584855495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f03dfdd070 0x55f03e398800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fcce61e5740 1 --2- 172.21.15.3:0/3584855495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f03dfb41b0 0x55f03e408420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fccc17fa640 0 client.26285 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fcce61e5740 1 -- 172.21.15.3:0/3584855495 >> 172.21.15.3:0/3584855495 conn(0x55f03dfd1460 msgr2=0x55f03e391190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fcce61e5740 1 -- 172.21.15.3:0/3584855495 shutdown_connections 2024-09-21T06:18:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fcce61e5740 1 -- 172.21.15.3:0/3584855495 wait complete. 2024-09-21T06:18:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.819+0000 7fcce61e5740 20 client.26285 trim_cache size 0 max 16384 2024-09-21T06:18:52.825 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:18:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.820+0000 7fccdfc71640 1 -- 172.21.15.3:0/3156427969 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fccc40b6700 msgr2=0x7fccc40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.820+0000 7fccdfc71640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fccc40b6700 0x7fccc40b8c10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009230 tx=0x7fccc8041040 comp rx=0 tx=0).stop 2024-09-21T06:18:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.820+0000 7fccdfc71640 1 -- 172.21.15.3:0/3156427969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd8105af0 msgr2=0x7fccd819bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.820+0000 7fccdfc71640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd8105af0 0x7fccd819bd00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fccd000eb10 tx=0x7fccd000c2d0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.820+0000 7fccdfc71640 1 -- 172.21.15.3:0/3156427969 shutdown_connections 2024-09-21T06:18:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.820+0000 7fccdfc71640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fccc40b6700 0x7fccc40b8c10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.820+0000 7fccdfc71640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd8105af0 0x7fccd819bd00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.820+0000 7fccdfc71640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccd81051a0 0x7fccd819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.820+0000 7fccdfc71640 1 --2- 172.21.15.3:0/3156427969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8103190 0x7fccd819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.820+0000 7fccdfc71640 1 -- 172.21.15.3:0/3156427969 >> 172.21.15.3:0/3156427969 conn(0x7fccd8077cc0 msgr2=0x7fccd8106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.820+0000 7fccdfc71640 1 -- 172.21.15.3:0/3156427969 shutdown_connections 2024-09-21T06:18:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:52.820+0000 7fccdfc71640 1 -- 172.21.15.3:0/3156427969 wait complete. 2024-09-21T06:18:52.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:52 smithi137 ceph-mon[34464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:18:52.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:52 smithi137 ceph-mon[34464]: pgmap v441: 97 pgs: 97 active+clean; 67 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 352 op/s 2024-09-21T06:18:52.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:52 smithi204 ceph-mon[34659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:18:52.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:52 smithi204 ceph-mon[34659]: pgmap v441: 97 pgs: 97 active+clean; 67 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 352 op/s 2024-09-21T06:18:52.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 352065 12.27 MB/sec execute 25 sec latency 18.382 ms 2024-09-21T06:18:53.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 355008 12.49 MB/sec execute 26 sec latency 13.296 ms 2024-09-21T06:18:54.682 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:18:54.682 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:18:54.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 357632 12.50 MB/sec execute 27 sec latency 12.336 ms 2024-09-21T06:18:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:55.076 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:54 smithi003 ceph-mon[31227]: pgmap v442: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 294 op/s 2024-09-21T06:18:55.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:54 smithi137 ceph-mon[34464]: pgmap v442: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 294 op/s 2024-09-21T06:18:55.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:54 smithi204 ceph-mon[34659]: pgmap v442: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 294 op/s 2024-09-21T06:18:55.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.491+0000 7f794a894640 1 -- 172.21.15.3:0/978956980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7944102ae0 msgr2=0x7f7944102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:55.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.491+0000 7f794a894640 1 --2- 172.21.15.3:0/978956980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7944102ae0 0x7f7944102ef0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7934009900 tx=0x7f793402a410 comp rx=0 tx=0).stop 2024-09-21T06:18:55.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.491+0000 7f794a894640 1 -- 172.21.15.3:0/978956980 shutdown_connections 2024-09-21T06:18:55.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.491+0000 7f794a894640 1 --2- 172.21.15.3:0/978956980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79441083d0 0x7f79441087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.491+0000 7f794a894640 1 --2- 172.21.15.3:0/978956980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7944108d20 0x7f794410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.491+0000 7f794a894640 1 --2- 172.21.15.3:0/978956980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7944102ae0 0x7f7944102ef0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.491+0000 7f794a894640 1 -- 172.21.15.3:0/978956980 >> 172.21.15.3:0/978956980 conn(0x7f7944079760 msgr2=0x7f7944079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.492+0000 7f794a894640 1 -- 172.21.15.3:0/978956980 shutdown_connections 2024-09-21T06:18:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.492+0000 7f794a894640 1 -- 172.21.15.3:0/978956980 wait complete. 2024-09-21T06:18:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.492+0000 7f794a894640 1 Processor -- start 2024-09-21T06:18:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.492+0000 7f794a894640 1 -- start start 2024-09-21T06:18:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f794a894640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944102ae0 0x7f794419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f794a894640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79441083d0 0x7f794419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f794a894640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7944108d20 0x7f794419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f794a894640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7944112990 con 0x7f7944108d20 2024-09-21T06:18:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f794a894640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7944112810 con 0x7f79441083d0 2024-09-21T06:18:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f794a894640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7944112b10 con 0x7f7944102ae0 2024-09-21T06:18:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f7948e31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7944108d20 0x7f794419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f7948e31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7944108d20 0x7f794419bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44352/0 (socket says 172.21.15.3:44352) 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f7943fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944102ae0 0x7f794419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f7948e31640 1 -- 172.21.15.3:0/2837295924 learned_addr learned my addr 172.21.15.3:0/2837295924 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f79437fe640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79441083d0 0x7f794419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f79437fe640 1 -- 172.21.15.3:0/2837295924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944102ae0 msgr2=0x7f794419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f79437fe640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944102ae0 0x7f794419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f79437fe640 1 -- 172.21.15.3:0/2837295924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7944108d20 msgr2=0x7f794419bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f79437fe640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7944108d20 0x7f794419bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f79437fe640 1 -- 172.21.15.3:0/2837295924 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79441a65b0 con 0x7f79441083d0 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f7948e31640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7944108d20 0x7f794419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.493+0000 7f7943fff640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944102ae0 0x7f794419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.494+0000 7f79437fe640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79441083d0 0x7f794419b860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f793800e9c0 tx=0x7f793800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.494+0000 7f79417fa640 1 -- 172.21.15.3:0/2837295924 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793800ccc0 con 0x7f79441083d0 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.494+0000 7f794a894640 1 -- 172.21.15.3:0/2837295924 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79441a68a0 con 0x7f79441083d0 2024-09-21T06:18:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.494+0000 7f794a894640 1 -- 172.21.15.3:0/2837295924 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79441a6de0 con 0x7f79441083d0 2024-09-21T06:18:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.494+0000 7f79417fa640 1 -- 172.21.15.3:0/2837295924 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79380044d0 con 0x7f79441083d0 2024-09-21T06:18:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.494+0000 7f79417fa640 1 -- 172.21.15.3:0/2837295924 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793801c070 con 0x7f79441083d0 2024-09-21T06:18:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.496+0000 7f7950e08740 1 Processor -- start 2024-09-21T06:18:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.496+0000 7f7950e08740 1 -- start start 2024-09-21T06:18:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7950e08740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e47350 0x5589c61ee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7950e08740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e1e490 0x5589c61ee970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7950e08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e26670 0x5589c61eeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7950e08740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5589c6164e60 con 0x5589c5e1e490 2024-09-21T06:18:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7950e08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5589c6154290 con 0x5589c5e26670 2024-09-21T06:18:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7950e08740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5589c61580a0 con 0x5589c5e47350 2024-09-21T06:18:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7948e31640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e47350 0x5589c61ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7948e31640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e47350 0x5589c61ee430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:55074/0 (socket says 172.21.15.3:55074) 2024-09-21T06:18:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7948e31640 1 -- 172.21.15.3:0/1724887847 learned_addr learned my addr 172.21.15.3:0/1724887847 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7943fff640 1 --2- 172.21.15.3:0/1724887847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e1e490 0x5589c61ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f79417fa640 1 -- 172.21.15.3:0/2837295924 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f7938004020 con 0x7f79441083d0 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7948e31640 1 -- 172.21.15.3:0/1724887847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e26670 msgr2=0x5589c61eeeb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7948e31640 1 --2- 172.21.15.3:0/1724887847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e26670 0x5589c61eeeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7948e31640 1 -- 172.21.15.3:0/1724887847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e1e490 msgr2=0x5589c61ee970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7948e31640 1 --2- 172.21.15.3:0/1724887847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e1e490 0x5589c61ee970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.497+0000 7f7948e31640 1 -- 172.21.15.3:0/1724887847 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589c5eb2930 con 0x5589c5e47350 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.498+0000 7f7948e31640 1 --2- 172.21.15.3:0/1724887847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e47350 0x5589c61ee430 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f793c011850 tx=0x7f793c011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb0ad4ef3c55b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.498+0000 7f792a7fc640 1 -- 172.21.15.3:0/1724887847 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c011330 con 0x5589c5e47350 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.498+0000 7f792a7fc640 1 -- 172.21.15.3:0/1724887847 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f793c00fe50 con 0x5589c5e47350 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.498+0000 7f792a7fc640 1 -- 172.21.15.3:0/1724887847 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c0193e0 con 0x5589c5e47350 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.498+0000 7f79417fa640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f792c0b66b0 0x7f792c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.499+0000 7f7943fff640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f792c0b66b0 0x7f792c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.499+0000 7f79417fa640 1 -- 172.21.15.3:0/2837295924 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f79380e74d0 con 0x7f79441083d0 2024-09-21T06:18:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.499+0000 7f7950e08740 1 -- 172.21.15.3:0/1724887847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e47350 msgr2=0x5589c61ee430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.499+0000 7f7950e08740 1 --2- 172.21.15.3:0/1724887847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e47350 0x5589c61ee430 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f793c011850 tx=0x7f793c011d10 comp rx=0 tx=0).stop 2024-09-21T06:18:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.499+0000 7f7950e08740 1 -- 172.21.15.3:0/1724887847 shutdown_connections 2024-09-21T06:18:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.499+0000 7f7950e08740 1 --2- 172.21.15.3:0/1724887847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e26670 0x5589c61eeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.499+0000 7f7950e08740 1 --2- 172.21.15.3:0/1724887847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e1e490 0x5589c61ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.499+0000 7f7950e08740 1 --2- 172.21.15.3:0/1724887847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e47350 0x5589c61ee430 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.499+0000 7f7950e08740 1 -- 172.21.15.3:0/1724887847 >> 172.21.15.3:0/1724887847 conn(0x5589c5e3b740 msgr2=0x5589c61ec320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.499+0000 7f7950e08740 1 -- 172.21.15.3:0/1724887847 shutdown_connections 2024-09-21T06:18:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.499+0000 7f7950e08740 1 -- 172.21.15.3:0/1724887847 wait complete. 2024-09-21T06:18:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.499+0000 7f7943fff640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f792c0b66b0 0x7f792c0b8bc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7934000c00 tx=0x7f7934031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7950e08740 1 -- start start 2024-09-21T06:18:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7950e08740 1 Processor -- start 2024-09-21T06:18:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7950e08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e1e490 0x5589c61f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7950e08740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e26670 0x5589c61fa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7950e08740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e47350 0x5589c6205f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7950e08740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5589c5fb5890 con 0x5589c5e47350 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7950e08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5589c60eed40 con 0x5589c5e1e490 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7950e08740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5589c61fcc10 con 0x5589c5e26670 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7950e08740 10 client.0 _ll_register_callbacks cb 0x5589c6152c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7948e31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e47350 0x5589c6205f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7943fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e1e490 0x5589c61f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f79437fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e26670 0x5589c61fa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7948e31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e47350 0x5589c6205f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44370/0 (socket says 172.21.15.3:44370) 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7948e31640 1 -- 172.21.15.3:0/3922763193 learned_addr learned my addr 172.21.15.3:0/3922763193 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.500+0000 7f7943fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e1e490 0x5589c61f9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:45854/0 (socket says 172.21.15.3:45854) 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.501+0000 7f7948e31640 1 -- 172.21.15.3:0/3922763193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e26670 msgr2=0x5589c61fa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.501+0000 7f7948e31640 1 --2- 172.21.15.3:0/3922763193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e26670 0x5589c61fa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.501+0000 7f7948e31640 1 -- 172.21.15.3:0/3922763193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e1e490 msgr2=0x5589c61f9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.501+0000 7f7948e31640 1 --2- 172.21.15.3:0/3922763193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e1e490 0x5589c61f9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.501+0000 7f7948e31640 1 -- 172.21.15.3:0/3922763193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589c62064a0 con 0x5589c5e47350 2024-09-21T06:18:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.501+0000 7f7943fff640 1 --2- 172.21.15.3:0/3922763193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e1e490 0x5589c61f9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.501+0000 7f7948e31640 1 --2- 172.21.15.3:0/3922763193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e47350 0x5589c6205f60 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f793c009a70 tx=0x7f793c010fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88854e98adc31929 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.501+0000 7f79237fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.501+0000 7f79237fe640 1 -- 172.21.15.3:0/3922763193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c01f040 con 0x5589c5e47350 2024-09-21T06:18:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.501+0000 7f79237fe640 1 -- 172.21.15.3:0/3922763193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f793c004a40 con 0x5589c5e47350 2024-09-21T06:18:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.501+0000 7f7950e08740 1 -- 172.21.15.3:0/3922763193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f793c00fe50 con 0x5589c5e47350 2024-09-21T06:18:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.502+0000 7f79237fe640 1 -- 172.21.15.3:0/3922763193 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f793c032050 con 0x5589c5e47350 2024-09-21T06:18:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.502+0000 7f7950e08740 10 client.16719 fetch_fsmap learned FSMap version 31 2024-09-21T06:18:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.502+0000 7f7950e08740 1 -- 172.21.15.3:0/3922763193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5589c6208ce0 con 0x5589c5e47350 2024-09-21T06:18:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.502+0000 7f79237fe640 1 -- 172.21.15.3:0/3922763193 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f793c00fe50 con 0x5589c5e47350 2024-09-21T06:18:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.502+0000 7f79237fe640 1 -- 172.21.15.3:0/3922763193 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f793c018070 con 0x5589c5e47350 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.502+0000 7f7950e08740 10 client.16719 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.502+0000 7f7950e08740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.502+0000 7f7950e08740 20 client.16719 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.502+0000 7f7950e08740 1 --2- 172.21.15.3:0/3922763193 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5589c5e05cd0 0x5589c620c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.502+0000 7f7950e08740 4 client.16719 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.502+0000 7f7950e08740 1 -- 172.21.15.3:0/3922763193 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589c6166db0 con 0x5589c5e05cd0 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.502+0000 7f7943fff640 1 --2- 172.21.15.3:0/3922763193 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5589c5e05cd0 0x5589c620c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.503+0000 7f7943fff640 1 --2- 172.21.15.3:0/3922763193 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5589c5e05cd0 0x5589c620c0f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8edf54bb94eccecb server_cookie=64af70a8e15e79ba in_seq=0 out_seq=0 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.503+0000 7f79237fe640 10 client.16719 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f79237fe640 1 -- 172.21.15.3:0/3922763193 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589c6166db0 con 0x5589c5e05cd0 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f79237fe640 10 client.16719 handle_command_reply: tid=0 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f7950e08740 1 client.16719 shutdown 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f7950e08740 1 -- 172.21.15.3:0/3922763193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e47350 msgr2=0x5589c6205f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f7950e08740 1 --2- 172.21.15.3:0/3922763193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e47350 0x5589c6205f60 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f793c009a70 tx=0x7f793c010fe0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f7950e08740 1 -- 172.21.15.3:0/3922763193 shutdown_connections 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f7950e08740 1 --2- 172.21.15.3:0/3922763193 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5589c5e05cd0 0x5589c620c0f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f7950e08740 1 --2- 172.21.15.3:0/3922763193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e26670 0x5589c61fa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f7950e08740 1 --2- 172.21.15.3:0/3922763193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e47350 0x5589c6205f60 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f7950e08740 1 --2- 172.21.15.3:0/3922763193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e1e490 0x5589c61f9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f79237fe640 0 client.16719 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f7950e08740 1 -- 172.21.15.3:0/3922763193 >> 172.21.15.3:0/3922763193 conn(0x5589c5e3b740 msgr2=0x5589c61f15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f7950e08740 1 -- 172.21.15.3:0/3922763193 shutdown_connections 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.504+0000 7f7950e08740 1 -- 172.21.15.3:0/3922763193 wait complete. 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.505+0000 7f7950e08740 20 client.16719 trim_cache size 0 max 16384 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.523+0000 7f7950e08740 1 Processor -- start 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.523+0000 7f7950e08740 1 -- start start 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.523+0000 7f7950e08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e3b740 0x5589c6205e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.523+0000 7f7950e08740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e1e490 0x5589c61f4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.523+0000 7f7950e08740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e26670 0x5589c61f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.523+0000 7f7950e08740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f793000e730 con 0x5589c5e1e490 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.523+0000 7f7950e08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7930010270 con 0x5589c5e3b740 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.523+0000 7f7950e08740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f793000ee00 con 0x5589c5e26670 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f79437fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e1e490 0x5589c61f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f7943fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e3b740 0x5589c6205e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f79437fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e1e490 0x5589c61f4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44380/0 (socket says 172.21.15.3:44380) 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f79437fe640 1 -- 172.21.15.3:0/2784727378 learned_addr learned my addr 172.21.15.3:0/2784727378 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f7948e31640 1 --2- 172.21.15.3:0/2784727378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e26670 0x5589c61f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f7948e31640 1 -- 172.21.15.3:0/2784727378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e3b740 msgr2=0x5589c6205e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f7948e31640 1 --2- 172.21.15.3:0/2784727378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e3b740 0x5589c6205e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f7948e31640 1 -- 172.21.15.3:0/2784727378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e1e490 msgr2=0x5589c61f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f7948e31640 1 --2- 172.21.15.3:0/2784727378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e1e490 0x5589c61f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f7948e31640 1 -- 172.21.15.3:0/2784727378 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589c61fa600 con 0x5589c5e26670 2024-09-21T06:18:55.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f7943fff640 1 --2- 172.21.15.3:0/2784727378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e3b740 0x5589c6205e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f79437fe640 1 --2- 172.21.15.3:0/2784727378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e1e490 0x5589c61f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f7948e31640 1 --2- 172.21.15.3:0/2784727378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e26670 0x5589c61f6ec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f793c005470 tx=0x7f793c005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2a2baa4e88f271f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f7929ffb640 1 -- 172.21.15.3:0/2784727378 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c03c070 con 0x5589c5e26670 2024-09-21T06:18:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f7929ffb640 1 -- 172.21.15.3:0/2784727378 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f793c029070 con 0x5589c5e26670 2024-09-21T06:18:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.524+0000 7f7929ffb640 1 -- 172.21.15.3:0/2784727378 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c00fb20 con 0x5589c5e26670 2024-09-21T06:18:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.525+0000 7f7950e08740 1 -- 172.21.15.3:0/2784727378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e26670 msgr2=0x5589c61f6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.525+0000 7f7950e08740 1 --2- 172.21.15.3:0/2784727378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e26670 0x5589c61f6ec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f793c005470 tx=0x7f793c005cf0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.525+0000 7f7950e08740 1 -- 172.21.15.3:0/2784727378 shutdown_connections 2024-09-21T06:18:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.525+0000 7f7950e08740 1 --2- 172.21.15.3:0/2784727378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e26670 0x5589c61f6ec0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.525+0000 7f7950e08740 1 --2- 172.21.15.3:0/2784727378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e1e490 0x5589c61f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.525+0000 7f7950e08740 1 --2- 172.21.15.3:0/2784727378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e3b740 0x5589c6205e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.525+0000 7f7950e08740 1 -- 172.21.15.3:0/2784727378 >> 172.21.15.3:0/2784727378 conn(0x5589c61ec2d0 msgr2=0x5589c61f8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.525+0000 7f7950e08740 1 -- 172.21.15.3:0/2784727378 shutdown_connections 2024-09-21T06:18:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.525+0000 7f7950e08740 1 -- 172.21.15.3:0/2784727378 wait complete. 2024-09-21T06:18:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.526+0000 7f7950e08740 1 -- start start 2024-09-21T06:18:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.526+0000 7f7950e08740 1 Processor -- start 2024-09-21T06:18:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.526+0000 7f7950e08740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e1e490 0x5589c61fe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.526+0000 7f7950e08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e26670 0x5589c61fed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.526+0000 7f7950e08740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e3b740 0x5589c626f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.526+0000 7f7950e08740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f793000fba0 con 0x5589c5e3b740 2024-09-21T06:18:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.526+0000 7f7950e08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f793000f4d0 con 0x5589c5e26670 2024-09-21T06:18:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.526+0000 7f7950e08740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f793000c710 con 0x5589c5e1e490 2024-09-21T06:18:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.526+0000 7f7950e08740 10 client.0 _ll_register_callbacks cb 0x7f793c0115e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7948e31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e3b740 0x5589c626f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7943fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e1e490 0x5589c61fe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7948e31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e3b740 0x5589c626f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44384/0 (socket says 172.21.15.3:44384) 2024-09-21T06:18:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7943fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e1e490 0x5589c61fe750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:55110/0 (socket says 172.21.15.3:55110) 2024-09-21T06:18:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f79437fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e26670 0x5589c61fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7948e31640 1 -- 172.21.15.3:0/312218689 learned_addr learned my addr 172.21.15.3:0/312218689 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7948e31640 1 -- 172.21.15.3:0/312218689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e1e490 msgr2=0x5589c61fe750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7948e31640 1 --2- 172.21.15.3:0/312218689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e1e490 0x5589c61fe750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7948e31640 1 -- 172.21.15.3:0/312218689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e26670 msgr2=0x5589c61fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7948e31640 1 --2- 172.21.15.3:0/312218689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e26670 0x5589c61fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7948e31640 1 -- 172.21.15.3:0/312218689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589c6270030 con 0x5589c5e3b740 2024-09-21T06:18:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7943fff640 1 --2- 172.21.15.3:0/312218689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e1e490 0x5589c61fe750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7948e31640 1 --2- 172.21.15.3:0/312218689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e3b740 0x5589c626f9c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f793c009560 tx=0x7f793c005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae771f26e362b66c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f79237fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f79237fe640 1 -- 172.21.15.3:0/312218689 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c03c070 con 0x5589c5e3b740 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7950e08740 1 -- 172.21.15.3:0/312218689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f793c029070 con 0x5589c5e3b740 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f79237fe640 1 -- 172.21.15.3:0/312218689 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f793c010640 con 0x5589c5e3b740 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f79237fe640 1 -- 172.21.15.3:0/312218689 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f793c005660 con 0x5589c5e3b740 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.527+0000 7f7950e08740 10 client.16725 fetch_fsmap learned FSMap version 31 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.528+0000 7f7950e08740 1 -- 172.21.15.3:0/312218689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5589c6277370 con 0x5589c5e3b740 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.528+0000 7f79237fe640 1 -- 172.21.15.3:0/312218689 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f793c029070 con 0x5589c5e3b740 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.528+0000 7f79237fe640 1 -- 172.21.15.3:0/312218689 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f793c018070 con 0x5589c5e3b740 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.528+0000 7f7950e08740 10 client.16725 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.528+0000 7f7950e08740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.528+0000 7f7950e08740 20 client.16725 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.528+0000 7f7950e08740 1 --2- 172.21.15.3:0/312218689 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5589c5e47350 0x5589c6279ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.528+0000 7f7950e08740 4 client.16725 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:18:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.528+0000 7f7950e08740 1 -- 172.21.15.3:0/312218689 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5589c6162860 con 0x5589c5e47350 2024-09-21T06:18:55.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.528+0000 7f7943fff640 1 --2- 172.21.15.3:0/312218689 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5589c5e47350 0x5589c6279ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:55.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.529+0000 7f79237fe640 10 client.16725 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:55.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.529+0000 7f7943fff640 1 --2- 172.21.15.3:0/312218689 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5589c5e47350 0x5589c6279ae0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d20708b44f9b36b server_cookie=7d00c330061bc457 in_seq=0 out_seq=0 2024-09-21T06:18:55.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.530+0000 7f79237fe640 1 -- 172.21.15.3:0/312218689 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5589c6162860 con 0x5589c5e47350 2024-09-21T06:18:55.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.530+0000 7f79237fe640 10 client.16725 handle_command_reply: tid=0 2024-09-21T06:18:55.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.530+0000 7f7950e08740 1 client.16725 shutdown 2024-09-21T06:18:55.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.530+0000 7f7950e08740 1 -- 172.21.15.3:0/312218689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e3b740 msgr2=0x5589c626f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:55.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.530+0000 7f7950e08740 1 --2- 172.21.15.3:0/312218689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e3b740 0x5589c626f9c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f793c009560 tx=0x7f793c005ab0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.556 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:18:55.557 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:18:55.557 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "1296f5f3-c779-4eac-ba35-9ebe6d75c09a", 2024-09-21T06:18:55.557 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:18:55.557 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:18:55.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.531+0000 7f7950e08740 1 -- 172.21.15.3:0/312218689 shutdown_connections 2024-09-21T06:18:55.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.531+0000 7f7950e08740 1 --2- 172.21.15.3:0/312218689 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5589c5e47350 0x5589c6279ae0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.531+0000 7f7950e08740 1 --2- 172.21.15.3:0/312218689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589c5e3b740 0x5589c626f9c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.531+0000 7f7950e08740 1 --2- 172.21.15.3:0/312218689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589c5e26670 0x5589c61fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.531+0000 7f79237fe640 0 client.16725 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:55.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.531+0000 7f7950e08740 1 --2- 172.21.15.3:0/312218689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5589c5e1e490 0x5589c61fe750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.531+0000 7f7950e08740 1 -- 172.21.15.3:0/312218689 >> 172.21.15.3:0/312218689 conn(0x5589c61ec2d0 msgr2=0x5589c61ef320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:55.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.531+0000 7f7950e08740 1 -- 172.21.15.3:0/312218689 shutdown_connections 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.531+0000 7f7950e08740 1 -- 172.21.15.3:0/312218689 wait complete. 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.531+0000 7f7950e08740 20 client.16725 trim_cache size 0 max 16384 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.531+0000 7f794a894640 1 -- 172.21.15.3:0/2837295924 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f792c0b66b0 msgr2=0x7f792c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.531+0000 7f794a894640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f792c0b66b0 0x7f792c0b8bc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7934000c00 tx=0x7f7934031400 comp rx=0 tx=0).stop 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.531+0000 7f794a894640 1 -- 172.21.15.3:0/2837295924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79441083d0 msgr2=0x7f794419b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.532+0000 7f794a894640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79441083d0 0x7f794419b860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f793800e9c0 tx=0x7f793800ee80 comp rx=0 tx=0).stop 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.532+0000 7f794a894640 1 -- 172.21.15.3:0/2837295924 shutdown_connections 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.532+0000 7f794a894640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f792c0b66b0 0x7f792c0b8bc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.532+0000 7f794a894640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7944108d20 0x7f794419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.532+0000 7f794a894640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79441083d0 0x7f794419b860 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.532+0000 7f794a894640 1 --2- 172.21.15.3:0/2837295924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944102ae0 0x7f794419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.532+0000 7f794a894640 1 -- 172.21.15.3:0/2837295924 >> 172.21.15.3:0/2837295924 conn(0x7f7944079760 msgr2=0x7f79441014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.532+0000 7f794a894640 1 -- 172.21.15.3:0/2837295924 shutdown_connections 2024-09-21T06:18:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:55.532+0000 7f794a894640 1 -- 172.21.15.3:0/2837295924 wait complete. 2024-09-21T06:18:55.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 359124 12.31 MB/sec execute 28 sec latency 47.557 ms 2024-09-21T06:18:56.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:55 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:18:56.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:55 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:56.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:55 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:18:56.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:55 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:56.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:55 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:18:56.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:55 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:56.351 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:18:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:56.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 362006 12.52 MB/sec execute 29 sec latency 51.971 ms 2024-09-21T06:18:57.085 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:56 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:18:57.085 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:56 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:57.086 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:56 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:18:57.086 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:56 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:18:57.086 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:56 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:57.086 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:56 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:57.086 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:56 smithi003 ceph-mon[31227]: pgmap v443: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-21T06:18:57.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:56 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:18:57.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:56 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:57.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:56 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:18:57.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:56 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:18:57.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:56 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:57.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:56 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:57.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:56 smithi137 ceph-mon[34464]: pgmap v443: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-21T06:18:57.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:56 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:18:57.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:56 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:57.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:56 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:18:57.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:56 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:18:57.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:56 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:57.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:56 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:57.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:56 smithi204 ceph-mon[34659]: pgmap v443: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-21T06:18:57.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.184+0000 7fdf7efc1640 1 -- 172.21.15.3:0/3542359353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf781083a0 msgr2=0x7fdf781087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:57.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.184+0000 7fdf7efc1640 1 --2- 172.21.15.3:0/3542359353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf781083a0 0x7fdf781087b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009900 tx=0x7fdf6402a410 comp rx=0 tx=0).stop 2024-09-21T06:18:57.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.184+0000 7fdf7efc1640 1 -- 172.21.15.3:0/3542359353 shutdown_connections 2024-09-21T06:18:57.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.184+0000 7fdf7efc1640 1 --2- 172.21.15.3:0/3542359353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf78108cf0 0x7fdf7810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.184+0000 7fdf7efc1640 1 --2- 172.21.15.3:0/3542359353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf781083a0 0x7fdf781087b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.184+0000 7fdf7efc1640 1 --2- 172.21.15.3:0/3542359353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf78102a60 0x7fdf78102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.184+0000 7fdf7efc1640 1 -- 172.21.15.3:0/3542359353 >> 172.21.15.3:0/3542359353 conn(0x7fdf78079730 msgr2=0x7fdf78079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:57.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.184+0000 7fdf7efc1640 1 -- 172.21.15.3:0/3542359353 shutdown_connections 2024-09-21T06:18:57.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.184+0000 7fdf7efc1640 1 -- 172.21.15.3:0/3542359353 wait complete. 2024-09-21T06:18:57.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.185+0000 7fdf7efc1640 1 Processor -- start 2024-09-21T06:18:57.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.185+0000 7fdf7efc1640 1 -- start start 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.185+0000 7fdf7efc1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf78102a60 0x7fdf7819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.185+0000 7fdf7efc1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf781083a0 0x7fdf7819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.185+0000 7fdf7efc1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf78108cf0 0x7fdf7819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.185+0000 7fdf7efc1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf78113a80 con 0x7fdf78102a60 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.185+0000 7fdf7efc1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdf78113900 con 0x7fdf78108cf0 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.185+0000 7fdf7efc1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf78113c00 con 0x7fdf781083a0 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.185+0000 7fdf7cd5d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf78102a60 0x7fdf7819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.185+0000 7fdf7cd5d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf78102a60 0x7fdf7819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44394/0 (socket says 172.21.15.3:44394) 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.185+0000 7fdf7cd5d640 1 -- 172.21.15.3:0/917964637 learned_addr learned my addr 172.21.15.3:0/917964637 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.185+0000 7fdf77fff640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf781083a0 0x7fdf7819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.185+0000 7fdf7d55e640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf78108cf0 0x7fdf7819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.186+0000 7fdf7cd5d640 1 -- 172.21.15.3:0/917964637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf781083a0 msgr2=0x7fdf7819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.186+0000 7fdf7cd5d640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf781083a0 0x7fdf7819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.186+0000 7fdf7cd5d640 1 -- 172.21.15.3:0/917964637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf78108cf0 msgr2=0x7fdf7819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:57.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.186+0000 7fdf7cd5d640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf78108cf0 0x7fdf7819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.186+0000 7fdf7cd5d640 1 -- 172.21.15.3:0/917964637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf781a0440 con 0x7fdf78102a60 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.186+0000 7fdf7cd5d640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf78102a60 0x7fdf7819a4d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00e9f0 tx=0x7fdf6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.186+0000 7fdf75ffb640 1 -- 172.21.15.3:0/917964637 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c00cd80 con 0x7fdf78102a60 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.186+0000 7fdf7efc1640 1 -- 172.21.15.3:0/917964637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf781a0730 con 0x7fdf78102a60 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.186+0000 7fdf75ffb640 1 -- 172.21.15.3:0/917964637 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf6c0054d0 con 0x7fdf78102a60 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.186+0000 7fdf75ffb640 1 -- 172.21.15.3:0/917964637 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c010690 con 0x7fdf78102a60 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.187+0000 7fdf7efc1640 1 -- 172.21.15.3:0/917964637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf781a6e90 con 0x7fdf78102a60 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.189+0000 7fdf75ffb640 1 -- 172.21.15.3:0/917964637 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fdf6c004e90 con 0x7fdf78102a60 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.189+0000 7fdf85535740 1 Processor -- start 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.190+0000 7fdf85535740 1 -- start start 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.190+0000 7fdf85535740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed157070 0x5583ed4fe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.190+0000 7fdf85535740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed12e1b0 0x5583ed4fe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.190+0000 7fdf85535740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed136390 0x5583ed4fecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.190+0000 7fdf85535740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5583ed474e50 con 0x5583ed157070 2024-09-21T06:18:57.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.190+0000 7fdf85535740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5583ed464280 con 0x5583ed136390 2024-09-21T06:18:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.190+0000 7fdf85535740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5583ed468090 con 0x5583ed12e1b0 2024-09-21T06:18:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.190+0000 7fdf7d55e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed157070 0x5583ed4fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.190+0000 7fdf7cd5d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed12e1b0 0x5583ed4fe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.190+0000 7fdf77fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed136390 0x5583ed4fecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.190+0000 7fdf7d55e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed157070 0x5583ed4fe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44406/0 (socket says 172.21.15.3:44406) 2024-09-21T06:18:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.190+0000 7fdf7d55e640 1 -- 172.21.15.3:0/2000445426 learned_addr learned my addr 172.21.15.3:0/2000445426 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.190+0000 7fdf75ffb640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdf680b6680 0x7fdf680b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.191+0000 7fdf7d55e640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdf680b6680 0x7fdf680b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.191+0000 7fdf77fff640 1 -- 172.21.15.3:0/2000445426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed12e1b0 msgr2=0x5583ed4fe7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.191+0000 7fdf77fff640 1 --2- 172.21.15.3:0/2000445426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed12e1b0 0x5583ed4fe7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.191+0000 7fdf77fff640 1 -- 172.21.15.3:0/2000445426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed157070 msgr2=0x5583ed4fe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.191+0000 7fdf77fff640 1 --2- 172.21.15.3:0/2000445426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed157070 0x5583ed4fe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.191+0000 7fdf77fff640 1 -- 172.21.15.3:0/2000445426 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583ed1c2650 con 0x5583ed136390 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.191+0000 7fdf75ffb640 1 -- 172.21.15.3:0/917964637 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fdf6c0e7ab0 con 0x7fdf78102a60 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.191+0000 7fdf7cd5d640 1 --2- 172.21.15.3:0/2000445426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed12e1b0 0x5583ed4fe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.191+0000 7fdf77fff640 1 --2- 172.21.15.3:0/2000445426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed136390 0x5583ed4fecf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006010 tx=0x7fdf64032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49515c93d3e58ef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.191+0000 7fdf62ffd640 1 -- 172.21.15.3:0/2000445426 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64040070 con 0x5583ed136390 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.191+0000 7fdf62ffd640 1 -- 172.21.15.3:0/2000445426 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf64004030 con 0x5583ed136390 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.191+0000 7fdf7d55e640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdf680b6680 0x7fdf680b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdf70011a00 tx=0x7fdf70004ab0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.191+0000 7fdf62ffd640 1 -- 172.21.15.3:0/2000445426 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64035040 con 0x5583ed136390 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.192+0000 7fdf85535740 1 -- 172.21.15.3:0/2000445426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed136390 msgr2=0x5583ed4fecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.192+0000 7fdf85535740 1 --2- 172.21.15.3:0/2000445426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed136390 0x5583ed4fecf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006010 tx=0x7fdf64032b20 comp rx=0 tx=0).stop 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.192+0000 7fdf85535740 1 -- 172.21.15.3:0/2000445426 shutdown_connections 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.192+0000 7fdf85535740 1 --2- 172.21.15.3:0/2000445426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed12e1b0 0x5583ed4fe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.192+0000 7fdf85535740 1 --2- 172.21.15.3:0/2000445426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed136390 0x5583ed4fecf0 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006010 tx=0x7fdf64032b20 comp rx=0 tx=0).stop 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.192+0000 7fdf85535740 1 --2- 172.21.15.3:0/2000445426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed157070 0x5583ed4fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.192+0000 7fdf85535740 1 -- 172.21.15.3:0/2000445426 >> 172.21.15.3:0/2000445426 conn(0x5583ed14b460 msgr2=0x5583ed4fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:57.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.193+0000 7fdf85535740 1 -- 172.21.15.3:0/2000445426 shutdown_connections 2024-09-21T06:18:57.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.193+0000 7fdf85535740 1 -- 172.21.15.3:0/2000445426 wait complete. 2024-09-21T06:18:57.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf85535740 1 -- start start 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf85535740 1 Processor -- start 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf85535740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 0x5583ed509a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf85535740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed157070 0x5583ed509f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf85535740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed1159f0 0x5583ed51a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf85535740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5583ed2c5700 con 0x5583ed12e1b0 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf85535740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5583ed3ff110 con 0x5583ed157070 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf85535740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5583ed510e00 con 0x5583ed1159f0 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf85535740 10 client.0 _ll_register_callbacks cb 0x5583ed418990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf77fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed157070 0x5583ed509f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf7d55e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed1159f0 0x5583ed51a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf77fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed157070 0x5583ed509f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:45906/0 (socket says 172.21.15.3:45906) 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf77fff640 1 -- 172.21.15.3:0/814958317 learned_addr learned my addr 172.21.15.3:0/814958317 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.194+0000 7fdf7cd5d640 1 --2- 172.21.15.3:0/814958317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 0x5583ed509a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.195+0000 7fdf77fff640 1 -- 172.21.15.3:0/814958317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed1159f0 msgr2=0x5583ed51a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:57.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.195+0000 7fdf77fff640 1 --2- 172.21.15.3:0/814958317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed1159f0 0x5583ed51a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.195+0000 7fdf77fff640 1 -- 172.21.15.3:0/814958317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 msgr2=0x5583ed509a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.195+0000 7fdf77fff640 1 --2- 172.21.15.3:0/814958317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 0x5583ed509a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.195+0000 7fdf77fff640 1 -- 172.21.15.3:0/814958317 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583ed42cc40 con 0x5583ed157070 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.195+0000 7fdf7d55e640 1 --2- 172.21.15.3:0/814958317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed1159f0 0x5583ed51a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.195+0000 7fdf77fff640 1 --2- 172.21.15.3:0/814958317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed157070 0x5583ed509f70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdf6402a930 tx=0x7fdf6402abb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=577c794ffc106176 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.195+0000 7fdf57fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.195+0000 7fdf57fff640 1 -- 172.21.15.3:0/814958317 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64052060 con 0x5583ed157070 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.195+0000 7fdf57fff640 1 -- 172.21.15.3:0/814958317 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fdf640051e0 con 0x5583ed157070 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.195+0000 7fdf85535740 1 -- 172.21.15.3:0/814958317 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf64004030 con 0x5583ed157070 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.195+0000 7fdf57fff640 1 -- 172.21.15.3:0/814958317 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdf64035070 con 0x5583ed157070 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.195+0000 7fdf85535740 10 client.26312 fetch_fsmap learned FSMap version 31 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.196+0000 7fdf85535740 1 -- 172.21.15.3:0/814958317 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5583ed51cbb0 con 0x5583ed157070 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.196+0000 7fdf57fff640 1 -- 172.21.15.3:0/814958317 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf64004030 con 0x5583ed157070 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.196+0000 7fdf57fff640 1 -- 172.21.15.3:0/814958317 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fdf64004ab0 con 0x5583ed157070 2024-09-21T06:18:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.196+0000 7fdf85535740 10 client.26312 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.196+0000 7fdf85535740 10 client.26312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.196+0000 7fdf85535740 20 client.26312 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.196+0000 7fdf85535740 1 --2- 172.21.15.3:0/814958317 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5583ed0eada0 0x5583ed51ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.196+0000 7fdf85535740 4 client.26312 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.196+0000 7fdf85535740 1 -- 172.21.15.3:0/814958317 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583ed476da0 con 0x5583ed0eada0 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.196+0000 7fdf7cd5d640 1 --2- 172.21.15.3:0/814958317 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5583ed0eada0 0x5583ed51ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.197+0000 7fdf57fff640 10 client.26312 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.197+0000 7fdf7cd5d640 1 --2- 172.21.15.3:0/814958317 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5583ed0eada0 0x5583ed51ff10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b126a1caf9f86895 server_cookie=29c8ac48c42b949d in_seq=0 out_seq=0 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.197+0000 7fdf57fff640 1 -- 172.21.15.3:0/814958317 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5583ed476da0 con 0x5583ed0eada0 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.197+0000 7fdf57fff640 10 client.26312 handle_command_reply: tid=0 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.197+0000 7fdf85535740 1 client.26312 shutdown 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.198+0000 7fdf85535740 1 -- 172.21.15.3:0/814958317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed157070 msgr2=0x5583ed509f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.198+0000 7fdf85535740 1 --2- 172.21.15.3:0/814958317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed157070 0x5583ed509f70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdf6402a930 tx=0x7fdf6402abb0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.198+0000 7fdf85535740 1 -- 172.21.15.3:0/814958317 shutdown_connections 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.198+0000 7fdf85535740 1 --2- 172.21.15.3:0/814958317 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5583ed0eada0 0x5583ed51ff10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.198+0000 7fdf85535740 1 --2- 172.21.15.3:0/814958317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed1159f0 0x5583ed51a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.198+0000 7fdf85535740 1 --2- 172.21.15.3:0/814958317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed157070 0x5583ed509f70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.198+0000 7fdf57fff640 0 client.26312 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.198+0000 7fdf85535740 1 --2- 172.21.15.3:0/814958317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 0x5583ed509a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.198+0000 7fdf85535740 1 -- 172.21.15.3:0/814958317 >> 172.21.15.3:0/814958317 conn(0x5583ed14b460 msgr2=0x5583ed501450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.198+0000 7fdf85535740 1 -- 172.21.15.3:0/814958317 shutdown_connections 2024-09-21T06:18:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.198+0000 7fdf85535740 1 -- 172.21.15.3:0/814958317 wait complete. 2024-09-21T06:18:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.198+0000 7fdf85535740 20 client.26312 trim_cache size 0 max 16384 2024-09-21T06:18:57.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.216+0000 7fdf85535740 1 Processor -- start 2024-09-21T06:18:57.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf85535740 1 -- start start 2024-09-21T06:18:57.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf85535740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed0eada0 0x5583ed4fc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf85535740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed1159f0 0x5583ed509a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf85535740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 0x5583ed51a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf85535740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf781ad3d0 con 0x5583ed12e1b0 2024-09-21T06:18:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf85535740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdf781aef10 con 0x5583ed0eada0 2024-09-21T06:18:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf85535740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf781adaa0 con 0x5583ed1159f0 2024-09-21T06:18:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf77fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed1159f0 0x5583ed509a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf7cd5d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed0eada0 0x5583ed4fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf7d55e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 0x5583ed51a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf77fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed1159f0 0x5583ed509a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:55154/0 (socket says 172.21.15.3:55154) 2024-09-21T06:18:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf77fff640 1 -- 172.21.15.3:0/1493402096 learned_addr learned my addr 172.21.15.3:0/1493402096 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf77fff640 1 -- 172.21.15.3:0/1493402096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed0eada0 msgr2=0x5583ed4fc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf77fff640 1 --2- 172.21.15.3:0/1493402096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed0eada0 0x5583ed4fc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf77fff640 1 -- 172.21.15.3:0/1493402096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 msgr2=0x5583ed51a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf77fff640 1 --2- 172.21.15.3:0/1493402096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 0x5583ed51a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.217+0000 7fdf77fff640 1 -- 172.21.15.3:0/1493402096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583ed508990 con 0x5583ed1159f0 2024-09-21T06:18:57.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.218+0000 7fdf77fff640 1 --2- 172.21.15.3:0/1493402096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed1159f0 0x5583ed509a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdf64005f70 tx=0x7fdf6402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ba285f4e788dee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:57.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.218+0000 7fdf627fc640 1 -- 172.21.15.3:0/1493402096 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64052040 con 0x5583ed1159f0 2024-09-21T06:18:57.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.218+0000 7fdf627fc640 1 -- 172.21.15.3:0/1493402096 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf640036f0 con 0x5583ed1159f0 2024-09-21T06:18:57.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.218+0000 7fdf627fc640 1 -- 172.21.15.3:0/1493402096 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64040040 con 0x5583ed1159f0 2024-09-21T06:18:57.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.219+0000 7fdf85535740 1 -- 172.21.15.3:0/1493402096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed1159f0 msgr2=0x5583ed509a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:57.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.219+0000 7fdf85535740 1 --2- 172.21.15.3:0/1493402096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed1159f0 0x5583ed509a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdf64005f70 tx=0x7fdf6402a410 comp rx=0 tx=0).stop 2024-09-21T06:18:57.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.219+0000 7fdf85535740 1 -- 172.21.15.3:0/1493402096 shutdown_connections 2024-09-21T06:18:57.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.219+0000 7fdf85535740 1 --2- 172.21.15.3:0/1493402096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed1159f0 0x5583ed509a30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.219+0000 7fdf85535740 1 --2- 172.21.15.3:0/1493402096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 0x5583ed51a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.219+0000 7fdf85535740 1 --2- 172.21.15.3:0/1493402096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed0eada0 0x5583ed4fc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.219+0000 7fdf85535740 1 -- 172.21.15.3:0/1493402096 >> 172.21.15.3:0/1493402096 conn(0x5583ed14b460 msgr2=0x5583ed4fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:57.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.219+0000 7fdf85535740 1 -- 172.21.15.3:0/1493402096 shutdown_connections 2024-09-21T06:18:57.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.219+0000 7fdf85535740 1 -- 172.21.15.3:0/1493402096 wait complete. 2024-09-21T06:18:57.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.219+0000 7fdf85535740 1 -- start start 2024-09-21T06:18:57.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.220+0000 7fdf85535740 1 Processor -- start 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.220+0000 7fdf85535740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed0eada0 0x5583ed5211d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.220+0000 7fdf85535740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed1159f0 0x5583ed521710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.220+0000 7fdf85535740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 0x5583ed589690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.220+0000 7fdf85535740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf781ae840 con 0x5583ed12e1b0 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.220+0000 7fdf85535740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdf781ae170 con 0x5583ed1159f0 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.220+0000 7fdf85535740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf781ab3b0 con 0x5583ed0eada0 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.220+0000 7fdf85535740 10 client.0 _ll_register_callbacks cb 0x7fdf70009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.220+0000 7fdf77fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed1159f0 0x5583ed521710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.220+0000 7fdf77fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed1159f0 0x5583ed521710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:45922/0 (socket says 172.21.15.3:45922) 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.220+0000 7fdf77fff640 1 -- 172.21.15.3:0/1343424297 learned_addr learned my addr 172.21.15.3:0/1343424297 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.220+0000 7fdf7d55e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 0x5583ed589690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.220+0000 7fdf7cd5d640 1 --2- 172.21.15.3:0/1343424297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed0eada0 0x5583ed5211d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf77fff640 1 -- 172.21.15.3:0/1343424297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed0eada0 msgr2=0x5583ed5211d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf77fff640 1 --2- 172.21.15.3:0/1343424297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed0eada0 0x5583ed5211d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf77fff640 1 -- 172.21.15.3:0/1343424297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 msgr2=0x5583ed589690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf77fff640 1 --2- 172.21.15.3:0/1343424297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 0x5583ed589690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf77fff640 1 -- 172.21.15.3:0/1343424297 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583ed521f30 con 0x5583ed1159f0 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf7cd5d640 1 --2- 172.21.15.3:0/1343424297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed0eada0 0x5583ed5211d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf77fff640 1 --2- 172.21.15.3:0/1343424297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed1159f0 0x5583ed521710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdf6402a930 tx=0x7fdf64000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f14a8b76e4edff0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf57fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf57fff640 1 -- 172.21.15.3:0/1343424297 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64040070 con 0x5583ed1159f0 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf57fff640 1 -- 172.21.15.3:0/1343424297 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fdf64031b10 con 0x5583ed1159f0 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf85535740 1 -- 172.21.15.3:0/1343424297 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf640036f0 con 0x5583ed1159f0 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf57fff640 1 -- 172.21.15.3:0/1343424297 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdf6406f050 con 0x5583ed1159f0 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf85535740 10 client.26318 fetch_fsmap learned FSMap version 31 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.221+0000 7fdf85535740 1 -- 172.21.15.3:0/1343424297 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5583ed521f30 con 0x5583ed1159f0 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.222+0000 7fdf57fff640 1 -- 172.21.15.3:0/1343424297 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf640036f0 con 0x5583ed1159f0 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.222+0000 7fdf57fff640 1 -- 172.21.15.3:0/1343424297 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fdf6405f020 con 0x5583ed1159f0 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.222+0000 7fdf85535740 10 client.26318 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.222+0000 7fdf85535740 10 client.26318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.222+0000 7fdf85535740 20 client.26318 populate_metadata read hostname 'smithi003' 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.222+0000 7fdf85535740 1 --2- 172.21.15.3:0/1343424297 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5583ed157070 0x5583ed58cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.222+0000 7fdf85535740 4 client.26318 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.222+0000 7fdf85535740 1 -- 172.21.15.3:0/1343424297 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5583ed472850 con 0x5583ed157070 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.222+0000 7fdf7cd5d640 1 --2- 172.21.15.3:0/1343424297 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5583ed157070 0x5583ed58cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf57fff640 10 client.26318 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf7cd5d640 1 --2- 172.21.15.3:0/1343424297 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5583ed157070 0x5583ed58cf00 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea4dfa09a1a23c1 server_cookie=d6f862d0be60f069 in_seq=0 out_seq=0 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf57fff640 1 -- 172.21.15.3:0/1343424297 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x5583ed472850 con 0x5583ed157070 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf57fff640 10 client.26318 handle_command_reply: tid=0 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf85535740 1 client.26318 shutdown 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf85535740 1 -- 172.21.15.3:0/1343424297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed1159f0 msgr2=0x5583ed521710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf85535740 1 --2- 172.21.15.3:0/1343424297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed1159f0 0x5583ed521710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdf6402a930 tx=0x7fdf64000bc0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf85535740 1 -- 172.21.15.3:0/1343424297 shutdown_connections 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf85535740 1 --2- 172.21.15.3:0/1343424297 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5583ed157070 0x5583ed58cf00 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf85535740 1 --2- 172.21.15.3:0/1343424297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5583ed12e1b0 0x5583ed589690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf85535740 1 --2- 172.21.15.3:0/1343424297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ed1159f0 0x5583ed521710 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf85535740 1 --2- 172.21.15.3:0/1343424297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583ed0eada0 0x5583ed5211d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf57fff640 0 client.26318 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:18:57.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf85535740 1 -- 172.21.15.3:0/1343424297 >> 172.21.15.3:0/1343424297 conn(0x5583ed14b460 msgr2=0x5583ed501370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:57.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf85535740 1 -- 172.21.15.3:0/1343424297 shutdown_connections 2024-09-21T06:18:57.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf85535740 1 -- 172.21.15.3:0/1343424297 wait complete. 2024-09-21T06:18:57.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.223+0000 7fdf85535740 20 client.26318 trim_cache size 0 max 16384 2024-09-21T06:18:57.231 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:18:57.231 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:18:57.231 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "1296f5f3-c779-4eac-ba35-9ebe6d75c09a", 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:18:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:18:57.236 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:18:57.236 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:18:57.236 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:18:57.236 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:18:57.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.224+0000 7fdf7efc1640 1 -- 172.21.15.3:0/917964637 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdf680b6680 msgr2=0x7fdf680b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:57.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.224+0000 7fdf7efc1640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdf680b6680 0x7fdf680b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdf70011a00 tx=0x7fdf70004ab0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.224+0000 7fdf7efc1640 1 -- 172.21.15.3:0/917964637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf78102a60 msgr2=0x7fdf7819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:57.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.224+0000 7fdf7efc1640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf78102a60 0x7fdf7819a4d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00e9f0 tx=0x7fdf6c00edf0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.224+0000 7fdf7efc1640 1 -- 172.21.15.3:0/917964637 shutdown_connections 2024-09-21T06:18:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.224+0000 7fdf7efc1640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdf680b6680 0x7fdf680b8b90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.224+0000 7fdf7efc1640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf781083a0 0x7fdf7819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.224+0000 7fdf7efc1640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf78108cf0 0x7fdf7819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.224+0000 7fdf7efc1640 1 --2- 172.21.15.3:0/917964637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf78102a60 0x7fdf7819a4d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.224+0000 7fdf7efc1640 1 -- 172.21.15.3:0/917964637 >> 172.21.15.3:0/917964637 conn(0x7fdf78079730 msgr2=0x7fdf78105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.224+0000 7fdf7efc1640 1 -- 172.21.15.3:0/917964637 shutdown_connections 2024-09-21T06:18:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:18:57.224+0000 7fdf7efc1640 1 -- 172.21.15.3:0/917964637 wait complete. 2024-09-21T06:18:57.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 364126 12.34 MB/sec execute 30 sec latency 42.603 ms 2024-09-21T06:18:58.139 INFO:tasks.cephfs.filesystem:scrub status for tag:1296f5f3-c779-4eac-ba35-9ebe6d75c09a - {'path': '/', 'tag': '1296f5f3-c779-4eac-ba35-9ebe6d75c09a', 'options': 'recursive,force'} 2024-09-21T06:18:58.951 INFO:tasks.workunit.client.0.smithi003.stdout: 1 366006 12.34 MB/sec execute 31 sec latency 17.109 ms 2024-09-21T06:18:59.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:18:58 smithi003 ceph-mon[31227]: pgmap v444: 97 pgs: 97 active+clean; 67 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-09-21T06:18:59.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:18:58 smithi137 ceph-mon[34464]: pgmap v444: 97 pgs: 97 active+clean; 67 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-09-21T06:18:59.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:18:58 smithi204 ceph-mon[34659]: pgmap v444: 97 pgs: 97 active+clean; 67 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-09-21T06:18:59.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 369148 12.54 MB/sec execute 32 sec latency 13.862 ms 2024-09-21T06:19:00.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 370134 12.23 MB/sec execute 33 sec latency 30.208 ms 2024-09-21T06:19:01.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:00 smithi003 ceph-mon[31227]: pgmap v445: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 415 op/s 2024-09-21T06:19:01.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:00 smithi137 ceph-mon[34464]: pgmap v445: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 415 op/s 2024-09-21T06:19:01.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:00 smithi204 ceph-mon[34659]: pgmap v445: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 415 op/s 2024-09-21T06:19:01.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 372732 12.36 MB/sec execute 34 sec latency 109.270 ms 2024-09-21T06:19:02.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:01 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:19:02.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:01 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:19:02.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:01 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:19:02.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:01 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:19:02.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:01 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:19:02.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:01 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:19:02.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:01 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:19:02.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:01 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:19:02.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:01 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:19:02.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 375209 12.33 MB/sec execute 35 sec latency 25.353 ms 2024-09-21T06:19:03.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:02 smithi003 ceph-mon[31227]: pgmap v446: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 415 op/s 2024-09-21T06:19:03.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:02 smithi137 ceph-mon[34464]: pgmap v446: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 415 op/s 2024-09-21T06:19:03.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:02 smithi204 ceph-mon[34659]: pgmap v446: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 415 op/s 2024-09-21T06:19:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.924+0000 7fda8e8f2640 1 -- 172.21.15.3:0/2696964916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda881029e0 msgr2=0x7fda88102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.924+0000 7fda8e8f2640 1 --2- 172.21.15.3:0/2696964916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda881029e0 0x7fda88102df0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fda78009900 tx=0x7fda7802a410 comp rx=0 tx=0).stop 2024-09-21T06:19:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.924+0000 7fda8e8f2640 1 -- 172.21.15.3:0/2696964916 shutdown_connections 2024-09-21T06:19:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.924+0000 7fda8e8f2640 1 --2- 172.21.15.3:0/2696964916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda88108c60 0x7fda8810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.924+0000 7fda8e8f2640 1 --2- 172.21.15.3:0/2696964916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda88108310 0x7fda88108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.924+0000 7fda8e8f2640 1 --2- 172.21.15.3:0/2696964916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda881029e0 0x7fda88102df0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.924+0000 7fda8e8f2640 1 -- 172.21.15.3:0/2696964916 >> 172.21.15.3:0/2696964916 conn(0x7fda880fe3e0 msgr2=0x7fda88100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.924+0000 7fda8e8f2640 1 -- 172.21.15.3:0/2696964916 shutdown_connections 2024-09-21T06:19:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.924+0000 7fda8e8f2640 1 -- 172.21.15.3:0/2696964916 wait complete. 2024-09-21T06:19:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.925+0000 7fda8e8f2640 1 Processor -- start 2024-09-21T06:19:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.925+0000 7fda8e8f2640 1 -- start start 2024-09-21T06:19:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.925+0000 7fda8e8f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda881029e0 0x7fda88199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.925+0000 7fda8e8f2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda88108310 0x7fda88199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.925+0000 7fda8e8f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda88108c60 0x7fda8819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.925+0000 7fda8e8f2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fda88112920 con 0x7fda881029e0 2024-09-21T06:19:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.925+0000 7fda8e8f2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fda881127a0 con 0x7fda88108c60 2024-09-21T06:19:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.925+0000 7fda8e8f2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda88112aa0 con 0x7fda88108310 2024-09-21T06:19:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.925+0000 7fda87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda881029e0 0x7fda88199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda881029e0 0x7fda88199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44442/0 (socket says 172.21.15.3:44442) 2024-09-21T06:19:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda87fff640 1 -- 172.21.15.3:0/3751283501 learned_addr learned my addr 172.21.15.3:0/3751283501 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda8ce8f640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda88108c60 0x7fda8819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda877fe640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda88108310 0x7fda88199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda8ce8f640 1 -- 172.21.15.3:0/3751283501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda88108310 msgr2=0x7fda88199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda8ce8f640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda88108310 0x7fda88199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda8ce8f640 1 -- 172.21.15.3:0/3751283501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda881029e0 msgr2=0x7fda88199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda8ce8f640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda881029e0 0x7fda88199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda8ce8f640 1 -- 172.21.15.3:0/3751283501 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8819f2b0 con 0x7fda88108c60 2024-09-21T06:19:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda87fff640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda881029e0 0x7fda88199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda8ce8f640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda88108c60 0x7fda8819ebb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda80009fd0 tx=0x7fda8000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda857fa640 1 -- 172.21.15.3:0/3751283501 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8000ce20 con 0x7fda88108c60 2024-09-21T06:19:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda8e8f2640 1 -- 172.21.15.3:0/3751283501 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8819f5a0 con 0x7fda88108c60 2024-09-21T06:19:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda857fa640 1 -- 172.21.15.3:0/3751283501 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda80005020 con 0x7fda88108c60 2024-09-21T06:19:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda857fa640 1 -- 172.21.15.3:0/3751283501 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80010690 con 0x7fda88108c60 2024-09-21T06:19:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.926+0000 7fda8e8f2640 1 -- 172.21.15.3:0/3751283501 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda881a5d30 con 0x7fda88108c60 2024-09-21T06:19:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.929+0000 7fda9585e740 1 Processor -- start 2024-09-21T06:19:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.929+0000 7fda857fa640 1 -- 172.21.15.3:0/3751283501 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fda800054d0 con 0x7fda88108c60 2024-09-21T06:19:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.929+0000 7fda9585e740 1 -- start start 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.929+0000 7fda9585e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 0x55f347dedbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.929+0000 7fda9585e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a1d2f0 0x55f347dee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.929+0000 7fda9585e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a254d0 0x55f347dee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.929+0000 7fda9585e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f347d606f0 con 0x55f347a254d0 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.929+0000 7fda9585e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f347d4fb20 con 0x55f347a461b0 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.929+0000 7fda9585e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f347d53930 con 0x55f347a1d2f0 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda87fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a1d2f0 0x55f347dee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda87fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a1d2f0 0x55f347dee100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:55196/0 (socket says 172.21.15.3:55196) 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda87fff640 1 -- 172.21.15.3:0/2972277976 learned_addr learned my addr 172.21.15.3:0/2972277976 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda877fe640 1 --2- 172.21.15.3:0/2972277976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 0x55f347dedbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda877fe640 1 --2- 172.21.15.3:0/2972277976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a254d0 0x55f347dee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda87fff640 1 -- 172.21.15.3:0/2972277976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 msgr2=0x55f347dedbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda87fff640 1 --2- 172.21.15.3:0/2972277976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 0x55f347dedbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda87fff640 1 -- 172.21.15.3:0/2972277976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a254d0 msgr2=0x55f347dee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda87fff640 1 --2- 172.21.15.3:0/2972277976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a254d0 0x55f347dee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda87fff640 1 -- 172.21.15.3:0/2972277976 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f347d15530 con 0x55f347a1d2f0 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda857fa640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fda700b6680 0x7fda700b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda877fe640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fda700b6680 0x7fda700b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda87fff640 1 --2- 172.21.15.3:0/2972277976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a1d2f0 0x55f347dee100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda78009a20 tx=0x7fda78005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6adeaa1ad592f58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda857fa640 1 -- 172.21.15.3:0/3751283501 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fda800e7dc0 con 0x7fda88108c60 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.930+0000 7fda6e7fc640 1 -- 172.21.15.3:0/2972277976 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78035040 con 0x55f347a1d2f0 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.931+0000 7fda6e7fc640 1 -- 172.21.15.3:0/2972277976 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda78031820 con 0x55f347a1d2f0 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.931+0000 7fda6e7fc640 1 -- 172.21.15.3:0/2972277976 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda780049e0 con 0x55f347a1d2f0 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.931+0000 7fda9585e740 1 -- 172.21.15.3:0/2972277976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a1d2f0 msgr2=0x55f347dee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.931+0000 7fda877fe640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fda700b6680 0x7fda700b8b90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0058c0 tx=0x7fda7c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.931+0000 7fda9585e740 1 --2- 172.21.15.3:0/2972277976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a1d2f0 0x55f347dee100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda78009a20 tx=0x7fda78005ed0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.931+0000 7fda9585e740 1 -- 172.21.15.3:0/2972277976 shutdown_connections 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.931+0000 7fda9585e740 1 --2- 172.21.15.3:0/2972277976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a1d2f0 0x55f347dee100 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.931+0000 7fda9585e740 1 --2- 172.21.15.3:0/2972277976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a254d0 0x55f347dee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.931+0000 7fda9585e740 1 --2- 172.21.15.3:0/2972277976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 0x55f347dedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.931+0000 7fda9585e740 1 -- 172.21.15.3:0/2972277976 >> 172.21.15.3:0/2972277976 conn(0x55f347a3a300 msgr2=0x55f347debab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.932+0000 7fda9585e740 1 -- 172.21.15.3:0/2972277976 shutdown_connections 2024-09-21T06:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.932+0000 7fda9585e740 1 -- 172.21.15.3:0/2972277976 wait complete. 2024-09-21T06:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.932+0000 7fda9585e740 1 -- start start 2024-09-21T06:19:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.932+0000 7fda9585e740 1 Processor -- start 2024-09-21T06:19:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.932+0000 7fda9585e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a1d2f0 0x55f347df92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda9585e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347df97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda9585e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a461b0 0x55f347e055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda9585e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f347c87ab0 con 0x55f347a461b0 2024-09-21T06:19:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda9585e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f347cf5730 con 0x55f347a1d2f0 2024-09-21T06:19:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda9585e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f347dfc2e0 con 0x55f347a254d0 2024-09-21T06:19:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda9585e740 10 client.0 _ll_register_callbacks cb 0x55f347d496b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347df97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda87fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a1d2f0 0x55f347df92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda8ce8f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a461b0 0x55f347e055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda87fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a1d2f0 0x55f347df92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:45960/0 (socket says 172.21.15.3:45960) 2024-09-21T06:19:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347df97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:55210/0 (socket says 172.21.15.3:55210) 2024-09-21T06:19:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda87fff640 1 -- 172.21.15.3:0/3096382094 learned_addr learned my addr 172.21.15.3:0/3096382094 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda8ce8f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a461b0 0x55f347e055d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44450/0 (socket says 172.21.15.3:44450) 2024-09-21T06:19:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda8ce8f640 1 -- 172.21.15.3:0/3096382094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 msgr2=0x55f347df97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda8ce8f640 1 --2- 172.21.15.3:0/3096382094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347df97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda8ce8f640 1 -- 172.21.15.3:0/3096382094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a1d2f0 msgr2=0x55f347df92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda8ce8f640 1 --2- 172.21.15.3:0/3096382094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a1d2f0 0x55f347df92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda8ce8f640 1 -- 172.21.15.3:0/3096382094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f347ab1790 con 0x55f347a461b0 2024-09-21T06:19:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda87fff640 1 --2- 172.21.15.3:0/3096382094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a1d2f0 0x55f347df92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda8ce8f640 1 --2- 172.21.15.3:0/3096382094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a461b0 0x55f347e055d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fda800e5550 tx=0x7fda800a93c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd40a85c0ae0d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda677fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda677fe640 1 -- 172.21.15.3:0/3096382094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda800d12d0 con 0x55f347a461b0 2024-09-21T06:19:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda677fe640 1 -- 172.21.15.3:0/3096382094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fda800aceb0 con 0x55f347a461b0 2024-09-21T06:19:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.933+0000 7fda9585e740 1 -- 172.21.15.3:0/3096382094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda78031820 con 0x55f347a461b0 2024-09-21T06:19:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.934+0000 7fda677fe640 1 -- 172.21.15.3:0/3096382094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fda800d51e0 con 0x55f347a461b0 2024-09-21T06:19:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.934+0000 7fda9585e740 10 client.16755 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.934+0000 7fda9585e740 1 -- 172.21.15.3:0/3096382094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f347b2a620 con 0x55f347a461b0 2024-09-21T06:19:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.934+0000 7fda677fe640 1 -- 172.21.15.3:0/3096382094 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda78031820 con 0x55f347a461b0 2024-09-21T06:19:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.934+0000 7fda677fe640 1 -- 172.21.15.3:0/3096382094 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fda800b5ab0 con 0x55f347a461b0 2024-09-21T06:19:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.934+0000 7fda9585e740 10 client.16755 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.934+0000 7fda9585e740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.934+0000 7fda9585e740 20 client.16755 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.934+0000 7fda9585e740 1 --2- 172.21.15.3:0/3096382094 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f347a04d80 0x55f347e0b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.934+0000 7fda9585e740 4 client.16755 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.934+0000 7fda9585e740 1 -- 172.21.15.3:0/3096382094 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f347d62640 con 0x55f347a04d80 2024-09-21T06:19:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.934+0000 7fda87fff640 1 --2- 172.21.15.3:0/3096382094 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f347a04d80 0x55f347e0b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.935+0000 7fda677fe640 10 client.16755 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.935+0000 7fda87fff640 1 --2- 172.21.15.3:0/3096382094 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f347a04d80 0x55f347e0b370 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d01038a571b07640 server_cookie=245c52ee41206b64 in_seq=0 out_seq=0 2024-09-21T06:19:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda677fe640 1 -- 172.21.15.3:0/3096382094 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f347d62640 con 0x55f347a04d80 2024-09-21T06:19:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda677fe640 10 client.16755 handle_command_reply: tid=0 2024-09-21T06:19:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda9585e740 1 client.16755 shutdown 2024-09-21T06:19:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda9585e740 1 -- 172.21.15.3:0/3096382094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a461b0 msgr2=0x55f347e055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda9585e740 1 --2- 172.21.15.3:0/3096382094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a461b0 0x55f347e055d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fda800e5550 tx=0x7fda800a93c0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda9585e740 1 -- 172.21.15.3:0/3096382094 shutdown_connections 2024-09-21T06:19:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda9585e740 1 --2- 172.21.15.3:0/3096382094 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f347a04d80 0x55f347e0b370 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda9585e740 1 --2- 172.21.15.3:0/3096382094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347df97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda9585e740 1 --2- 172.21.15.3:0/3096382094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a461b0 0x55f347e055d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda677fe640 0 client.16755 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda9585e740 1 --2- 172.21.15.3:0/3096382094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a1d2f0 0x55f347df92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda9585e740 1 -- 172.21.15.3:0/3096382094 >> 172.21.15.3:0/3096382094 conn(0x55f347a3a300 msgr2=0x55f347deebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda9585e740 1 -- 172.21.15.3:0/3096382094 shutdown_connections 2024-09-21T06:19:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda9585e740 1 -- 172.21.15.3:0/3096382094 wait complete. 2024-09-21T06:19:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.936+0000 7fda9585e740 20 client.16755 trim_cache size 0 max 16384 2024-09-21T06:19:03.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 377042 12.27 MB/sec execute 36 sec latency 19.197 ms 2024-09-21T06:19:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.951+0000 7fda9585e740 1 Processor -- start 2024-09-21T06:19:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda9585e740 1 -- start start 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda9585e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 0x55f347df9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda9585e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347dfaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda9585e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 0x55f347e054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda9585e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fda881ad430 con 0x55f347a1d2f0 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda9585e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fda881aef70 con 0x55f347a461b0 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda9585e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda881adb00 con 0x55f347a254d0 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 0x55f347df9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 0x55f347df9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44456/0 (socket says 172.21.15.3:44456) 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda87fff640 1 -- 172.21.15.3:0/3240522115 learned_addr learned my addr 172.21.15.3:0/3240522115 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda877fe640 1 --2- 172.21.15.3:0/3240522115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347dfaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda8ce8f640 1 --2- 172.21.15.3:0/3240522115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 0x55f347e054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda87fff640 1 -- 172.21.15.3:0/3240522115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 msgr2=0x55f347dfaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda87fff640 1 --2- 172.21.15.3:0/3240522115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347dfaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda87fff640 1 -- 172.21.15.3:0/3240522115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 msgr2=0x55f347e054d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda87fff640 1 --2- 172.21.15.3:0/3240522115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 0x55f347e054d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda87fff640 1 -- 172.21.15.3:0/3240522115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f347dec840 con 0x55f347a1d2f0 2024-09-21T06:19:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.952+0000 7fda877fe640 1 --2- 172.21.15.3:0/3240522115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347dfaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.953+0000 7fda87fff640 1 --2- 172.21.15.3:0/3240522115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 0x55f347df9780 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fda78009a20 tx=0x7fda780054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447b26fad08cc0c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.953+0000 7fda6dffb640 1 -- 172.21.15.3:0/3240522115 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78035040 con 0x55f347a1d2f0 2024-09-21T06:19:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.953+0000 7fda6dffb640 1 -- 172.21.15.3:0/3240522115 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda780317a0 con 0x55f347a1d2f0 2024-09-21T06:19:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.953+0000 7fda6dffb640 1 -- 172.21.15.3:0/3240522115 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda780038e0 con 0x55f347a1d2f0 2024-09-21T06:19:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.954+0000 7fda9585e740 1 -- 172.21.15.3:0/3240522115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 msgr2=0x55f347df9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.954+0000 7fda9585e740 1 --2- 172.21.15.3:0/3240522115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 0x55f347df9780 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fda78009a20 tx=0x7fda780054e0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.954+0000 7fda9585e740 1 -- 172.21.15.3:0/3240522115 shutdown_connections 2024-09-21T06:19:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.954+0000 7fda9585e740 1 --2- 172.21.15.3:0/3240522115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347dfaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.954+0000 7fda9585e740 1 --2- 172.21.15.3:0/3240522115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 0x55f347e054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.954+0000 7fda9585e740 1 --2- 172.21.15.3:0/3240522115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 0x55f347df9780 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.954+0000 7fda9585e740 1 -- 172.21.15.3:0/3240522115 >> 172.21.15.3:0/3240522115 conn(0x55f347a3a300 msgr2=0x55f347dfa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.954+0000 7fda9585e740 1 -- 172.21.15.3:0/3240522115 shutdown_connections 2024-09-21T06:19:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.954+0000 7fda9585e740 1 -- 172.21.15.3:0/3240522115 wait complete. 2024-09-21T06:19:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.954+0000 7fda9585e740 1 -- start start 2024-09-21T06:19:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.954+0000 7fda9585e740 1 Processor -- start 2024-09-21T06:19:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda9585e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 0x55f347dfcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda9585e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347dfd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda9585e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 0x55f347e73d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda9585e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fda881ae8a0 con 0x55f347a1d2f0 2024-09-21T06:19:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda9585e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fda881ae1d0 con 0x55f347a461b0 2024-09-21T06:19:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda9585e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda881ab410 con 0x55f347a254d0 2024-09-21T06:19:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda9585e740 10 client.0 _ll_register_callbacks cb 0x7fda800ad5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 0x55f347dfcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 0x55f347dfcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44462/0 (socket says 172.21.15.3:44462) 2024-09-21T06:19:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda87fff640 1 -- 172.21.15.3:0/2590361369 learned_addr learned my addr 172.21.15.3:0/2590361369 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda8ce8f640 1 --2- 172.21.15.3:0/2590361369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 0x55f347e73d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda877fe640 1 --2- 172.21.15.3:0/2590361369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347dfd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda87fff640 1 -- 172.21.15.3:0/2590361369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 msgr2=0x55f347dfd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda87fff640 1 --2- 172.21.15.3:0/2590361369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347dfd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda87fff640 1 -- 172.21.15.3:0/2590361369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 msgr2=0x55f347e73d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda87fff640 1 --2- 172.21.15.3:0/2590361369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 0x55f347e73d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.955+0000 7fda87fff640 1 -- 172.21.15.3:0/2590361369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f347e74280 con 0x55f347a1d2f0 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda87fff640 1 --2- 172.21.15.3:0/2590361369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 0x55f347dfcd30 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fda780042a0 tx=0x7fda780042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27eb56e3f77e7627 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda677fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda677fe640 1 -- 172.21.15.3:0/2590361369 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78031be0 con 0x55f347a1d2f0 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda9585e740 1 -- 172.21.15.3:0/2590361369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda780317a0 con 0x55f347a1d2f0 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda677fe640 1 -- 172.21.15.3:0/2590361369 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fda78031d80 con 0x55f347a1d2f0 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda677fe640 1 -- 172.21.15.3:0/2590361369 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fda780035f0 con 0x55f347a1d2f0 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda9585e740 10 client.16761 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda9585e740 1 -- 172.21.15.3:0/2590361369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f347e754f0 con 0x55f347a1d2f0 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda677fe640 1 -- 172.21.15.3:0/2590361369 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda780317a0 con 0x55f347a1d2f0 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda677fe640 1 -- 172.21.15.3:0/2590361369 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fda78050070 con 0x55f347a1d2f0 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda9585e740 10 client.16761 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda9585e740 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda9585e740 20 client.16761 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda9585e740 1 --2- 172.21.15.3:0/2590361369 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f3479da1c0 0x55f347e77830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda9585e740 4 client.16761 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:19:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.956+0000 7fda9585e740 1 -- 172.21.15.3:0/2590361369 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f347d5e0f0 con 0x55f3479da1c0 2024-09-21T06:19:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.957+0000 7fda877fe640 1 --2- 172.21.15.3:0/2590361369 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f3479da1c0 0x55f347e77830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.957+0000 7fda677fe640 10 client.16761 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:03.957+0000 7fda877fe640 1 --2- 172.21.15.3:0/2590361369 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f3479da1c0 0x55f347e77830 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f94381f1990ca10 server_cookie=de4d4bdbd02cf61c in_seq=0 out_seq=0 2024-09-21T06:19:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.089+0000 7fda677fe640 1 -- 172.21.15.3:0/2590361369 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f347d5e0f0 con 0x55f3479da1c0 2024-09-21T06:19:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.089+0000 7fda677fe640 10 client.16761 handle_command_reply: tid=0 2024-09-21T06:19:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.089+0000 7fda9585e740 1 client.16761 shutdown 2024-09-21T06:19:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.090+0000 7fda9585e740 1 -- 172.21.15.3:0/2590361369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 msgr2=0x55f347dfcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.090+0000 7fda9585e740 1 --2- 172.21.15.3:0/2590361369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 0x55f347dfcd30 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fda780042a0 tx=0x7fda780042d0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.090+0000 7fda9585e740 1 -- 172.21.15.3:0/2590361369 shutdown_connections 2024-09-21T06:19:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.090+0000 7fda9585e740 1 --2- 172.21.15.3:0/2590361369 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f3479da1c0 0x55f347e77830 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.090+0000 7fda9585e740 1 --2- 172.21.15.3:0/2590361369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f347a254d0 0x55f347dfd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.090+0000 7fda677fe640 0 client.16761 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.090+0000 7fda9585e740 1 --2- 172.21.15.3:0/2590361369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f347a461b0 0x55f347e73d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.090+0000 7fda9585e740 1 --2- 172.21.15.3:0/2590361369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f347a1d2f0 0x55f347dfcd30 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.090+0000 7fda9585e740 1 -- 172.21.15.3:0/2590361369 >> 172.21.15.3:0/2590361369 conn(0x55f347a3a300 msgr2=0x55f347deba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.090+0000 7fda9585e740 1 -- 172.21.15.3:0/2590361369 shutdown_connections 2024-09-21T06:19:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.090+0000 7fda9585e740 1 -- 172.21.15.3:0/2590361369 wait complete. 2024-09-21T06:19:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.090+0000 7fda9585e740 20 client.16761 trim_cache size 0 max 16384 2024-09-21T06:19:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:19:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:19:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:19:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:19:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.091+0000 7fda8e8f2640 1 -- 172.21.15.3:0/3751283501 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fda700b6680 msgr2=0x7fda700b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.091+0000 7fda8e8f2640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fda700b6680 0x7fda700b8b90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0058c0 tx=0x7fda7c005f90 comp rx=0 tx=0).stop 2024-09-21T06:19:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.091+0000 7fda8e8f2640 1 -- 172.21.15.3:0/3751283501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda88108c60 msgr2=0x7fda8819ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.091+0000 7fda8e8f2640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda88108c60 0x7fda8819ebb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda80009fd0 tx=0x7fda8000ef10 comp rx=0 tx=0).stop 2024-09-21T06:19:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.091+0000 7fda8e8f2640 1 -- 172.21.15.3:0/3751283501 shutdown_connections 2024-09-21T06:19:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.091+0000 7fda8e8f2640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fda700b6680 0x7fda700b8b90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.091+0000 7fda8e8f2640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda88108310 0x7fda88199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.091+0000 7fda8e8f2640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda88108c60 0x7fda8819ebb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.091+0000 7fda8e8f2640 1 --2- 172.21.15.3:0/3751283501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda881029e0 0x7fda88199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.091+0000 7fda8e8f2640 1 -- 172.21.15.3:0/3751283501 >> 172.21.15.3:0/3751283501 conn(0x7fda880fe3e0 msgr2=0x7fda88107a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.091+0000 7fda8e8f2640 1 -- 172.21.15.3:0/3751283501 shutdown_connections 2024-09-21T06:19:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:04.091+0000 7fda8e8f2640 1 -- 172.21.15.3:0/3751283501 wait complete. 2024-09-21T06:19:04.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 379803 12.40 MB/sec execute 37 sec latency 75.231 ms 2024-09-21T06:19:05.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:04 smithi003 ceph-mon[31227]: pgmap v447: 97 pgs: 97 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-09-21T06:19:05.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:04 smithi137 ceph-mon[34464]: pgmap v447: 97 pgs: 97 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-09-21T06:19:05.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:04 smithi204 ceph-mon[34659]: pgmap v447: 97 pgs: 97 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-09-21T06:19:05.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 382146 12.29 MB/sec execute 38 sec latency 22.281 ms 2024-09-21T06:19:06.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 383724 12.26 MB/sec execute 39 sec latency 58.649 ms 2024-09-21T06:19:07.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:06 smithi003 ceph-mon[31227]: pgmap v448: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 455 op/s 2024-09-21T06:19:07.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:06 smithi137 ceph-mon[34464]: pgmap v448: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 455 op/s 2024-09-21T06:19:07.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:06 smithi204 ceph-mon[34659]: pgmap v448: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 455 op/s 2024-09-21T06:19:07.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 386897 12.42 MB/sec execute 40 sec latency 43.679 ms 2024-09-21T06:19:08.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 389113 12.30 MB/sec execute 41 sec latency 24.276 ms 2024-09-21T06:19:09.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:08 smithi003 ceph-mon[31227]: pgmap v449: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 424 op/s 2024-09-21T06:19:09.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:08 smithi137 ceph-mon[34464]: pgmap v449: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 424 op/s 2024-09-21T06:19:09.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:08 smithi204 ceph-mon[34659]: pgmap v449: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 424 op/s 2024-09-21T06:19:09.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 390942 12.30 MB/sec execute 42 sec latency 17.586 ms 2024-09-21T06:19:10.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 394042 12.45 MB/sec execute 43 sec latency 12.357 ms 2024-09-21T06:19:11.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:10 smithi003 ceph-mon[31227]: pgmap v450: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 418 op/s 2024-09-21T06:19:11.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:10 smithi137 ceph-mon[34464]: pgmap v450: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 418 op/s 2024-09-21T06:19:11.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:10 smithi204 ceph-mon[34659]: pgmap v450: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 418 op/s 2024-09-21T06:19:11.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 396343 12.35 MB/sec execute 44 sec latency 44.427 ms 2024-09-21T06:19:12.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 397993 12.33 MB/sec execute 45 sec latency 49.518 ms 2024-09-21T06:19:13.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:12 smithi003 ceph-mon[31227]: pgmap v451: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-09-21T06:19:13.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:12 smithi137 ceph-mon[34464]: pgmap v451: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-09-21T06:19:13.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:12 smithi204 ceph-mon[34659]: pgmap v451: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-09-21T06:19:13.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 401110 12.46 MB/sec execute 46 sec latency 9.731 ms 2024-09-21T06:19:14.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 403300 12.36 MB/sec execute 47 sec latency 17.824 ms 2024-09-21T06:19:15.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:14 smithi003 ceph-mon[31227]: pgmap v452: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.6 MiB/s wr, 409 op/s 2024-09-21T06:19:15.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:14 smithi137 ceph-mon[34464]: pgmap v452: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.6 MiB/s wr, 409 op/s 2024-09-21T06:19:15.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:14 smithi204 ceph-mon[34659]: pgmap v452: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.6 MiB/s wr, 409 op/s 2024-09-21T06:19:15.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 405133 12.36 MB/sec execute 48 sec latency 22.476 ms 2024-09-21T06:19:16.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 408189 12.48 MB/sec execute 49 sec latency 12.885 ms 2024-09-21T06:19:17.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:16 smithi003 ceph-mon[31227]: pgmap v453: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 447 op/s 2024-09-21T06:19:17.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:16 smithi137 ceph-mon[34464]: pgmap v453: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 447 op/s 2024-09-21T06:19:17.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:16 smithi204 ceph-mon[34659]: pgmap v453: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 447 op/s 2024-09-21T06:19:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.192+0000 7f0ba7119640 1 -- 172.21.15.3:0/4080206093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba01082e0 msgr2=0x7f0ba01086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.192+0000 7f0ba7119640 1 --2- 172.21.15.3:0/4080206093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba01082e0 0x7f0ba01086f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009900 tx=0x7f0b8c02a410 comp rx=0 tx=0).stop 2024-09-21T06:19:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.192+0000 7f0ba7119640 1 -- 172.21.15.3:0/4080206093 shutdown_connections 2024-09-21T06:19:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.192+0000 7f0ba7119640 1 --2- 172.21.15.3:0/4080206093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba0108c30 0x7f0ba010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.192+0000 7f0ba7119640 1 --2- 172.21.15.3:0/4080206093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba01082e0 0x7f0ba01086f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.192+0000 7f0ba7119640 1 --2- 172.21.15.3:0/4080206093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba01029b0 0x7f0ba0102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.192+0000 7f0ba7119640 1 -- 172.21.15.3:0/4080206093 >> 172.21.15.3:0/4080206093 conn(0x7f0ba007a740 msgr2=0x7f0ba007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.192+0000 7f0ba7119640 1 -- 172.21.15.3:0/4080206093 shutdown_connections 2024-09-21T06:19:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.192+0000 7f0ba7119640 1 -- 172.21.15.3:0/4080206093 wait complete. 2024-09-21T06:19:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0ba7119640 1 Processor -- start 2024-09-21T06:19:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0ba7119640 1 -- start start 2024-09-21T06:19:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0ba7119640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba01029b0 0x7f0ba019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0ba7119640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba01082e0 0x7f0ba019b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0ba7119640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba0108c30 0x7f0ba019bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0ba7119640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01128f0 con 0x7f0ba01082e0 2024-09-21T06:19:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0ba7119640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0112770 con 0x7f0ba0108c30 2024-09-21T06:19:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0ba7119640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0112a70 con 0x7f0ba01029b0 2024-09-21T06:19:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0b9ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba01082e0 0x7f0ba019b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0b9ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba01082e0 0x7f0ba019b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49654/0 (socket says 172.21.15.3:49654) 2024-09-21T06:19:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0ba4eb5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba01029b0 0x7f0ba019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0b9ffff640 1 -- 172.21.15.3:0/533438906 learned_addr learned my addr 172.21.15.3:0/533438906 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0ba56b6640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba0108c30 0x7f0ba019bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0b9ffff640 1 -- 172.21.15.3:0/533438906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba01029b0 msgr2=0x7f0ba019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0b9ffff640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba01029b0 0x7f0ba019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.193+0000 7f0b9ffff640 1 -- 172.21.15.3:0/533438906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba0108c30 msgr2=0x7f0ba019bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.194+0000 7f0b9ffff640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba0108c30 0x7f0ba019bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.194+0000 7f0b9ffff640 1 -- 172.21.15.3:0/533438906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba01a6590 con 0x7f0ba01082e0 2024-09-21T06:19:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.194+0000 7f0ba56b6640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba0108c30 0x7f0ba019bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.194+0000 7f0b9ffff640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba01082e0 0x7f0ba019b840 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c02a930 tx=0x7f0b8c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.194+0000 7f0b9dffb640 1 -- 172.21.15.3:0/533438906 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c03f030 con 0x7f0ba01082e0 2024-09-21T06:19:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.194+0000 7f0b9dffb640 1 -- 172.21.15.3:0/533438906 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b8c004490 con 0x7f0ba01082e0 2024-09-21T06:19:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.194+0000 7f0ba7119640 1 -- 172.21.15.3:0/533438906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba01a6820 con 0x7f0ba01082e0 2024-09-21T06:19:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.194+0000 7f0b9dffb640 1 -- 172.21.15.3:0/533438906 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c035070 con 0x7f0ba01082e0 2024-09-21T06:19:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.194+0000 7f0ba7119640 1 -- 172.21.15.3:0/533438906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba01a6d10 con 0x7f0ba01082e0 2024-09-21T06:19:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.197+0000 7f0bae085740 1 Processor -- start 2024-09-21T06:19:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.197+0000 7f0bae085740 1 -- start start 2024-09-21T06:19:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0bae085740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x56234198d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0bae085740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x56234198def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0bae085740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x56234198e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0bae085740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5623419026f0 con 0x5623415e81b0 2024-09-21T06:19:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0bae085740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5623418f1b20 con 0x5623415bf2f0 2024-09-21T06:19:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0bae085740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5623418f5930 con 0x5623415c74d0 2024-09-21T06:19:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0ba4eb5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x56234198def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0ba56b6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x56234198d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0ba4eb5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x56234198def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:39852/0 (socket says 172.21.15.3:39852) 2024-09-21T06:19:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0ba4eb5640 1 -- 172.21.15.3:0/1202176883 learned_addr learned my addr 172.21.15.3:0/1202176883 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0ba56b6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x56234198d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49658/0 (socket says 172.21.15.3:49658) 2024-09-21T06:19:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0b9dffb640 1 -- 172.21.15.3:0/533438906 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f0b8c042030 con 0x7f0ba01082e0 2024-09-21T06:19:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0b9ffff640 1 --2- 172.21.15.3:0/1202176883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x56234198e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0ba4eb5640 1 -- 172.21.15.3:0/1202176883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 msgr2=0x56234198e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0ba4eb5640 1 --2- 172.21.15.3:0/1202176883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x56234198e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0ba4eb5640 1 -- 172.21.15.3:0/1202176883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 msgr2=0x56234198d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0ba4eb5640 1 --2- 172.21.15.3:0/1202176883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x56234198d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0ba4eb5640 1 -- 172.21.15.3:0/1202176883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623418b7530 con 0x5623415bf2f0 2024-09-21T06:19:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.198+0000 7f0b9ffff640 1 --2- 172.21.15.3:0/1202176883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x56234198e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:19:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.199+0000 7f0ba4eb5640 1 --2- 172.21.15.3:0/1202176883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x56234198def0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009cc0 tx=0x7f0b94011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab60b32685a3399c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.199+0000 7f0b8affd640 1 -- 172.21.15.3:0/1202176883 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94005220 con 0x5623415bf2f0 2024-09-21T06:19:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.199+0000 7f0b8affd640 1 -- 172.21.15.3:0/1202176883 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b94004c50 con 0x5623415bf2f0 2024-09-21T06:19:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.199+0000 7f0b8affd640 1 -- 172.21.15.3:0/1202176883 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94019760 con 0x5623415bf2f0 2024-09-21T06:19:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.199+0000 7f0b9dffb640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0b900b6700 0x7f0b900b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0ba56b6640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0b900b6700 0x7f0b900b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0b9dffb640 1 -- 172.21.15.3:0/533438906 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f0b8c107a20 con 0x7f0ba01082e0 2024-09-21T06:19:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0bae085740 1 -- 172.21.15.3:0/1202176883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 msgr2=0x56234198def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0bae085740 1 --2- 172.21.15.3:0/1202176883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x56234198def0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009cc0 tx=0x7f0b94011810 comp rx=0 tx=0).stop 2024-09-21T06:19:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0bae085740 1 -- 172.21.15.3:0/1202176883 shutdown_connections 2024-09-21T06:19:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0bae085740 1 --2- 172.21.15.3:0/1202176883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x56234198e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0bae085740 1 --2- 172.21.15.3:0/1202176883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x56234198def0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0bae085740 1 --2- 172.21.15.3:0/1202176883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x56234198d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0bae085740 1 -- 172.21.15.3:0/1202176883 >> 172.21.15.3:0/1202176883 conn(0x5623415dc300 msgr2=0x56234198b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0bae085740 1 -- 172.21.15.3:0/1202176883 shutdown_connections 2024-09-21T06:19:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0ba56b6640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0b900b6700 0x7f0b900b8c10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009fd0 tx=0x7f0b98009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0bae085740 1 -- 172.21.15.3:0/1202176883 wait complete. 2024-09-21T06:19:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0bae085740 1 -- start start 2024-09-21T06:19:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.200+0000 7f0bae085740 1 Processor -- start 2024-09-21T06:19:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0bae085740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x5623419990b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0bae085740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x5623419995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0bae085740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x5623419a5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0bae085740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562341829ab0 con 0x5623415e81b0 2024-09-21T06:19:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0bae085740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562341897730 con 0x5623415bf2f0 2024-09-21T06:19:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0bae085740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56234199c130 con 0x5623415c74d0 2024-09-21T06:19:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0bae085740 10 client.0 _ll_register_callbacks cb 0x5623418eb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0ba56b6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x5623419a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0ba4eb5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x5623419990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0b9ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x5623419995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0ba4eb5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x5623419990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:39862/0 (socket says 172.21.15.3:39862) 2024-09-21T06:19:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0ba56b6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x5623419a5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49672/0 (socket says 172.21.15.3:49672) 2024-09-21T06:19:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0ba4eb5640 1 -- 172.21.15.3:0/3228580867 learned_addr learned my addr 172.21.15.3:0/3228580867 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0ba4eb5640 1 -- 172.21.15.3:0/3228580867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 msgr2=0x5623419995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0ba4eb5640 1 --2- 172.21.15.3:0/3228580867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x5623419995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0ba4eb5640 1 -- 172.21.15.3:0/3228580867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 msgr2=0x5623419a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0ba4eb5640 1 --2- 172.21.15.3:0/3228580867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x5623419a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0ba4eb5640 1 -- 172.21.15.3:0/3228580867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562341653790 con 0x5623415bf2f0 2024-09-21T06:19:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0b9ffff640 1 --2- 172.21.15.3:0/3228580867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x5623419995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.201+0000 7f0ba56b6640 1 --2- 172.21.15.3:0/3228580867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x5623419a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.202+0000 7f0ba4eb5640 1 --2- 172.21.15.3:0/3228580867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x5623419990b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b94000c00 tx=0x7f0b94005030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa37ed69d2c7531 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.202+0000 7f0b7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:19:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.202+0000 7f0b7ffff640 1 -- 172.21.15.3:0/3228580867 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94028070 con 0x5623415bf2f0 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.202+0000 7f0bae085740 1 -- 172.21.15.3:0/3228580867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b94004c50 con 0x5623415bf2f0 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.202+0000 7f0b7ffff640 1 -- 172.21.15.3:0/3228580867 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f0b940194a0 con 0x5623415bf2f0 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.202+0000 7f0b7ffff640 1 -- 172.21.15.3:0/3228580867 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0b94034050 con 0x5623415bf2f0 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.202+0000 7f0bae085740 10 client.26348 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.202+0000 7f0bae085740 1 -- 172.21.15.3:0/3228580867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5623416cc620 con 0x5623415bf2f0 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.202+0000 7f0b7ffff640 1 -- 172.21.15.3:0/3228580867 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b94004c50 con 0x5623415bf2f0 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.203+0000 7f0b7ffff640 1 -- 172.21.15.3:0/3228580867 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0b94024430 con 0x5623415bf2f0 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.203+0000 7f0bae085740 10 client.26348 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.203+0000 7f0bae085740 10 client.26348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.203+0000 7f0bae085740 20 client.26348 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.203+0000 7f0bae085740 1 --2- 172.21.15.3:0/3228580867 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5623415a6d80 0x5623419ab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.203+0000 7f0bae085740 4 client.26348 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.203+0000 7f0bae085740 1 -- 172.21.15.3:0/3228580867 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562341904640 con 0x5623415a6d80 2024-09-21T06:19:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.203+0000 7f0b9ffff640 1 --2- 172.21.15.3:0/3228580867 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5623415a6d80 0x5623419ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.203+0000 7f0b9ffff640 1 --2- 172.21.15.3:0/3228580867 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5623415a6d80 0x5623419ab000 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff7ed27bc062240e server_cookie=a45075512b4d1226 in_seq=0 out_seq=0 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.203+0000 7f0b7ffff640 10 client.26348 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0b7ffff640 1 -- 172.21.15.3:0/3228580867 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562341904640 con 0x5623415a6d80 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0b7ffff640 10 client.26348 handle_command_reply: tid=0 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0bae085740 1 client.26348 shutdown 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0bae085740 1 -- 172.21.15.3:0/3228580867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 msgr2=0x5623419990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0bae085740 1 --2- 172.21.15.3:0/3228580867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x5623419990b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b94000c00 tx=0x7f0b94005030 comp rx=0 tx=0).stop 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0bae085740 1 -- 172.21.15.3:0/3228580867 shutdown_connections 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0bae085740 1 --2- 172.21.15.3:0/3228580867 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5623415a6d80 0x5623419ab000 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0bae085740 1 --2- 172.21.15.3:0/3228580867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x5623419995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0bae085740 1 --2- 172.21.15.3:0/3228580867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x5623419a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0bae085740 1 --2- 172.21.15.3:0/3228580867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x5623419990b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0b7ffff640 0 client.26348 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0bae085740 1 -- 172.21.15.3:0/3228580867 >> 172.21.15.3:0/3228580867 conn(0x5623415dc300 msgr2=0x56234198e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0bae085740 1 -- 172.21.15.3:0/3228580867 shutdown_connections 2024-09-21T06:19:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0bae085740 1 -- 172.21.15.3:0/3228580867 wait complete. 2024-09-21T06:19:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.204+0000 7f0bae085740 20 client.26348 trim_cache size 0 max 16384 2024-09-21T06:19:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.220+0000 7f0bae085740 1 Processor -- start 2024-09-21T06:19:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.220+0000 7f0bae085740 1 -- start start 2024-09-21T06:19:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0bae085740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x5623419931f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0bae085740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x562341999590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0bae085740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x5623419947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0bae085740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01af750 con 0x5623415e81b0 2024-09-21T06:19:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0bae085740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01b1290 con 0x5623415bf2f0 2024-09-21T06:19:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0bae085740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01afe20 con 0x5623415c74d0 2024-09-21T06:19:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0ba56b6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x5623419947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0ba4eb5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x5623419931f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0b9ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x562341999590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0ba56b6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x5623419947f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49684/0 (socket says 172.21.15.3:49684) 2024-09-21T06:19:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0ba56b6640 1 -- 172.21.15.3:0/3621090144 learned_addr learned my addr 172.21.15.3:0/3621090144 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0b9ffff640 1 -- 172.21.15.3:0/3621090144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 msgr2=0x5623419931f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0b9ffff640 1 --2- 172.21.15.3:0/3621090144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x5623419931f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0b9ffff640 1 -- 172.21.15.3:0/3621090144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 msgr2=0x5623419947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0b9ffff640 1 --2- 172.21.15.3:0/3621090144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x5623419947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0b9ffff640 1 -- 172.21.15.3:0/3621090144 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562341997590 con 0x5623415c74d0 2024-09-21T06:19:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.221+0000 7f0b9ffff640 1 --2- 172.21.15.3:0/3621090144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x562341999590 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c105150 tx=0x7f0b8c0f1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90b4a1b842fa88cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.222+0000 7f0b8a7fc640 1 -- 172.21.15.3:0/3621090144 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c049040 con 0x5623415c74d0 2024-09-21T06:19:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.222+0000 7f0b8a7fc640 1 -- 172.21.15.3:0/3621090144 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b8c107e50 con 0x5623415c74d0 2024-09-21T06:19:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.222+0000 7f0b8a7fc640 1 -- 172.21.15.3:0/3621090144 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c0d6de0 con 0x5623415c74d0 2024-09-21T06:19:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.222+0000 7f0bae085740 1 -- 172.21.15.3:0/3621090144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 msgr2=0x562341999590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.222+0000 7f0bae085740 1 --2- 172.21.15.3:0/3621090144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x562341999590 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c105150 tx=0x7f0b8c0f1040 comp rx=0 tx=0).stop 2024-09-21T06:19:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.223+0000 7f0bae085740 1 -- 172.21.15.3:0/3621090144 shutdown_connections 2024-09-21T06:19:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.223+0000 7f0bae085740 1 --2- 172.21.15.3:0/3621090144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415c74d0 0x562341999590 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.223+0000 7f0bae085740 1 --2- 172.21.15.3:0/3621090144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x5623419947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.223+0000 7f0bae085740 1 --2- 172.21.15.3:0/3621090144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415bf2f0 0x5623419931f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.223+0000 7f0bae085740 1 -- 172.21.15.3:0/3621090144 >> 172.21.15.3:0/3621090144 conn(0x5623415dc300 msgr2=0x562341992ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.223+0000 7f0bae085740 1 -- 172.21.15.3:0/3621090144 shutdown_connections 2024-09-21T06:19:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.223+0000 7f0bae085740 1 -- 172.21.15.3:0/3621090144 wait complete. 2024-09-21T06:19:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.223+0000 7f0bae085740 1 -- start start 2024-09-21T06:19:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.223+0000 7f0bae085740 1 Processor -- start 2024-09-21T06:19:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.223+0000 7f0bae085740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415bf2f0 0x56234199cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.223+0000 7f0bae085740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415c74d0 0x56234199d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.223+0000 7f0bae085740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x562341a138d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0bae085740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01b0bc0 con 0x5623415e81b0 2024-09-21T06:19:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0bae085740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01b04f0 con 0x5623415c74d0 2024-09-21T06:19:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0bae085740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01ad730 con 0x5623415bf2f0 2024-09-21T06:19:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0bae085740 10 client.0 _ll_register_callbacks cb 0x7f0b98009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0ba56b6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x562341a138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0ba4eb5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415bf2f0 0x56234199cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0ba56b6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x562341a138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49692/0 (socket says 172.21.15.3:49692) 2024-09-21T06:19:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0b9ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415c74d0 0x56234199d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0ba56b6640 1 -- 172.21.15.3:0/3099181027 learned_addr learned my addr 172.21.15.3:0/3099181027 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0ba56b6640 1 -- 172.21.15.3:0/3099181027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415bf2f0 msgr2=0x56234199cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0ba56b6640 1 --2- 172.21.15.3:0/3099181027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415bf2f0 0x56234199cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0ba56b6640 1 -- 172.21.15.3:0/3099181027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415c74d0 msgr2=0x56234199d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0ba56b6640 1 --2- 172.21.15.3:0/3099181027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415c74d0 0x56234199d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0ba56b6640 1 -- 172.21.15.3:0/3099181027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562341a13e10 con 0x5623415e81b0 2024-09-21T06:19:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0b9ffff640 1 --2- 172.21.15.3:0/3099181027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415c74d0 0x56234199d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0ba56b6640 1 --2- 172.21.15.3:0/3099181027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x562341a138d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800fec0 tx=0x7f0b98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b787bb57821f062 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0b7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0b7ffff640 1 -- 172.21.15.3:0/3099181027 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9802f060 con 0x5623415e81b0 2024-09-21T06:19:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0b7ffff640 1 -- 172.21.15.3:0/3099181027 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f0b9801f6d0 con 0x5623415e81b0 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.224+0000 7f0bae085740 1 -- 172.21.15.3:0/3099181027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b8c107e50 con 0x5623415e81b0 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.225+0000 7f0b7ffff640 1 -- 172.21.15.3:0/3099181027 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0b9801f8d0 con 0x5623415e81b0 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.225+0000 7f0bae085740 10 client.16779 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.225+0000 7f0bae085740 1 -- 172.21.15.3:0/3099181027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562341a15080 con 0x5623415e81b0 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.225+0000 7f0b7ffff640 1 -- 172.21.15.3:0/3099181027 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b8c107e50 con 0x5623415e81b0 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.225+0000 7f0b7ffff640 1 -- 172.21.15.3:0/3099181027 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0b9802aa60 con 0x5623415e81b0 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.225+0000 7f0bae085740 10 client.16779 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.225+0000 7f0bae085740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.225+0000 7f0bae085740 20 client.16779 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.225+0000 7f0bae085740 1 --2- 172.21.15.3:0/3099181027 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56234157c1c0 0x562341a178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.225+0000 7f0bae085740 4 client.16779 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.225+0000 7f0bae085740 1 -- 172.21.15.3:0/3099181027 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56234198bb10 con 0x56234157c1c0 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.225+0000 7f0ba4eb5640 1 --2- 172.21.15.3:0/3099181027 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56234157c1c0 0x562341a178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.226+0000 7f0b7ffff640 10 client.16779 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.226+0000 7f0ba4eb5640 1 --2- 172.21.15.3:0/3099181027 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56234157c1c0 0x562341a178a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b9db2002dac13a0 server_cookie=92f3b1f3ae836d2a in_seq=0 out_seq=0 2024-09-21T06:19:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.358+0000 7f0b7ffff640 1 -- 172.21.15.3:0/3099181027 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56234198bb10 con 0x56234157c1c0 2024-09-21T06:19:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.358+0000 7f0b7ffff640 10 client.16779 handle_command_reply: tid=0 2024-09-21T06:19:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.358+0000 7f0bae085740 1 client.16779 shutdown 2024-09-21T06:19:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.358+0000 7f0bae085740 1 -- 172.21.15.3:0/3099181027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 msgr2=0x562341a138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.358+0000 7f0bae085740 1 --2- 172.21.15.3:0/3099181027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x562341a138d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800fec0 tx=0x7f0b98004a40 comp rx=0 tx=0).stop 2024-09-21T06:19:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.358+0000 7f0bae085740 1 -- 172.21.15.3:0/3099181027 shutdown_connections 2024-09-21T06:19:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.358+0000 7f0bae085740 1 --2- 172.21.15.3:0/3099181027 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56234157c1c0 0x562341a178a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.358+0000 7f0bae085740 1 --2- 172.21.15.3:0/3099181027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623415e81b0 0x562341a138d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.358+0000 7f0bae085740 1 --2- 172.21.15.3:0/3099181027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623415c74d0 0x56234199d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.358+0000 7f0bae085740 1 --2- 172.21.15.3:0/3099181027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623415bf2f0 0x56234199cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.358+0000 7f0b7ffff640 0 client.16779 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.358+0000 7f0bae085740 1 -- 172.21.15.3:0/3099181027 >> 172.21.15.3:0/3099181027 conn(0x5623415dc300 msgr2=0x56234199a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.358+0000 7f0bae085740 1 -- 172.21.15.3:0/3099181027 shutdown_connections 2024-09-21T06:19:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.359+0000 7f0bae085740 1 -- 172.21.15.3:0/3099181027 wait complete. 2024-09-21T06:19:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.359+0000 7f0bae085740 20 client.16779 trim_cache size 0 max 16384 2024-09-21T06:19:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:19:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:19:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:19:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:19:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.360+0000 7f0ba7119640 1 -- 172.21.15.3:0/533438906 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0b900b6700 msgr2=0x7f0b900b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.360+0000 7f0ba7119640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0b900b6700 0x7f0b900b8c10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009fd0 tx=0x7f0b98009500 comp rx=0 tx=0).stop 2024-09-21T06:19:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.360+0000 7f0ba7119640 1 -- 172.21.15.3:0/533438906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba01082e0 msgr2=0x7f0ba019b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.360+0000 7f0ba7119640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba01082e0 0x7f0ba019b840 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c02a930 tx=0x7f0b8c031a20 comp rx=0 tx=0).stop 2024-09-21T06:19:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.360+0000 7f0ba7119640 1 -- 172.21.15.3:0/533438906 shutdown_connections 2024-09-21T06:19:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.360+0000 7f0ba7119640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0b900b6700 0x7f0b900b8c10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.360+0000 7f0ba7119640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba0108c30 0x7f0ba019bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.360+0000 7f0ba7119640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba01082e0 0x7f0ba019b840 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.360+0000 7f0ba7119640 1 --2- 172.21.15.3:0/533438906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba01029b0 0x7f0ba019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.360+0000 7f0ba7119640 1 -- 172.21.15.3:0/533438906 >> 172.21.15.3:0/533438906 conn(0x7f0ba007a740 msgr2=0x7f0ba0106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.360+0000 7f0ba7119640 1 -- 172.21.15.3:0/533438906 shutdown_connections 2024-09-21T06:19:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:17.360+0000 7f0ba7119640 1 -- 172.21.15.3:0/533438906 wait complete. 2024-09-21T06:19:17.952 INFO:tasks.workunit.client.0.smithi003.stdout: 1 409458 12.34 MB/sec execute 50 sec latency 71.981 ms 2024-09-21T06:19:18.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 411886 12.37 MB/sec execute 51 sec latency 29.059 ms 2024-09-21T06:19:19.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:18 smithi003 ceph-mon[31227]: pgmap v454: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 358 op/s 2024-09-21T06:19:19.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:18 smithi137 ceph-mon[34464]: pgmap v454: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 358 op/s 2024-09-21T06:19:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:18 smithi204 ceph-mon[34659]: pgmap v454: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 358 op/s 2024-09-21T06:19:19.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 415012 12.47 MB/sec execute 52 sec latency 17.809 ms 2024-09-21T06:19:20.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 416744 12.37 MB/sec execute 53 sec latency 17.885 ms 2024-09-21T06:19:21.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:20 smithi003 ceph-mon[31227]: pgmap v455: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.1 MiB/s wr, 393 op/s 2024-09-21T06:19:21.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:20 smithi137 ceph-mon[34464]: pgmap v455: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.1 MiB/s wr, 393 op/s 2024-09-21T06:19:21.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:20 smithi204 ceph-mon[34659]: pgmap v455: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.1 MiB/s wr, 393 op/s 2024-09-21T06:19:21.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 419154 12.39 MB/sec execute 54 sec latency 10.541 ms 2024-09-21T06:19:22.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 422159 12.49 MB/sec execute 55 sec latency 50.205 ms 2024-09-21T06:19:23.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:22 smithi003 ceph-mon[31227]: pgmap v456: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 408 op/s 2024-09-21T06:19:23.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:22 smithi137 ceph-mon[34464]: pgmap v456: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 408 op/s 2024-09-21T06:19:23.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:22 smithi204 ceph-mon[34659]: pgmap v456: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 408 op/s 2024-09-21T06:19:23.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 423827 12.39 MB/sec execute 56 sec latency 18.226 ms 2024-09-21T06:19:24.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 426140 12.41 MB/sec execute 57 sec latency 15.293 ms 2024-09-21T06:19:25.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:24 smithi003 ceph-mon[31227]: pgmap v457: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 401 op/s 2024-09-21T06:19:25.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:24 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:19:25.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:24 smithi137 ceph-mon[34464]: pgmap v457: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 401 op/s 2024-09-21T06:19:25.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:24 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:19:25.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:24 smithi204 ceph-mon[34659]: pgmap v457: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 401 op/s 2024-09-21T06:19:25.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:24 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:19:25.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 429292 12.50 MB/sec execute 58 sec latency 15.658 ms 2024-09-21T06:19:26.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:19:26.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:19:26.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:19:26.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:19:26.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:19:26.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:19:26.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:19:26.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:19:26.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:19:26.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 430926 12.41 MB/sec execute 59 sec latency 17.798 ms 2024-09-21T06:19:27.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:27 smithi003 ceph-mon[31227]: pgmap v458: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.0 MiB/s wr, 403 op/s 2024-09-21T06:19:27.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:27 smithi137 ceph-mon[34464]: pgmap v458: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.0 MiB/s wr, 403 op/s 2024-09-21T06:19:27.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:27 smithi204 ceph-mon[34659]: pgmap v458: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.0 MiB/s wr, 403 op/s 2024-09-21T06:19:27.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 433194 12.43 MB/sec execute 60 sec latency 42.891 ms 2024-09-21T06:19:28.141 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:19:28.429 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:28.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 435951 12.44 MB/sec execute 61 sec latency 28.391 ms 2024-09-21T06:19:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.096+0000 7f8ed0bc3640 1 -- 172.21.15.3:0/804146026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ecc108cf0 msgr2=0x7f8ecc10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.096+0000 7f8ed0bc3640 1 --2- 172.21.15.3:0/804146026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ecc108cf0 0x7f8ecc10f520 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009900 tx=0x7f8ec402a410 comp rx=0 tx=0).stop 2024-09-21T06:19:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.096+0000 7f8ed0bc3640 1 -- 172.21.15.3:0/804146026 shutdown_connections 2024-09-21T06:19:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.096+0000 7f8ed0bc3640 1 --2- 172.21.15.3:0/804146026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ecc108cf0 0x7f8ecc10f520 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.096+0000 7f8ed0bc3640 1 --2- 172.21.15.3:0/804146026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ecc1083a0 0x7f8ecc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.096+0000 7f8ed0bc3640 1 --2- 172.21.15.3:0/804146026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc102a60 0x7f8ecc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.096+0000 7f8ed0bc3640 1 -- 172.21.15.3:0/804146026 >> 172.21.15.3:0/804146026 conn(0x7f8ecc079730 msgr2=0x7f8ecc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:29.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.096+0000 7f8ed0bc3640 1 -- 172.21.15.3:0/804146026 shutdown_connections 2024-09-21T06:19:29.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.096+0000 7f8ed0bc3640 1 -- 172.21.15.3:0/804146026 wait complete. 2024-09-21T06:19:29.099 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:29 smithi003 ceph-mon[31227]: pgmap v459: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 385 op/s 2024-09-21T06:19:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.097+0000 7f8ed0bc3640 1 Processor -- start 2024-09-21T06:19:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.097+0000 7f8ed0bc3640 1 -- start start 2024-09-21T06:19:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8ed0bc3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc102a60 0x7f8ecc10b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8ed0bc3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ecc1083a0 0x7f8ecc10bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8ed0bc3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ecc108cf0 0x7f8ecc10c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8ed0bc3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc112960 con 0x7f8ecc108cf0 2024-09-21T06:19:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8ed0bc3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1127e0 con 0x7f8ecc1083a0 2024-09-21T06:19:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8ed0bc3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc112ae0 con 0x7f8ecc102a60 2024-09-21T06:19:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8ec9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ecc1083a0 0x7f8ecc10bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8eca59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc102a60 0x7f8ecc10b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8eca59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc102a60 0x7f8ecc10b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:45666/0 (socket says 172.21.15.3:45666) 2024-09-21T06:19:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8eca59c640 1 -- 172.21.15.3:0/2673008081 learned_addr learned my addr 172.21.15.3:0/2673008081 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8ec9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ecc1083a0 0x7f8ecc10bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:59960/0 (socket says 172.21.15.3:59960) 2024-09-21T06:19:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8ecad9d640 1 --2- 172.21.15.3:0/2673008081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ecc108cf0 0x7f8ecc10c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8eca59c640 1 -- 172.21.15.3:0/2673008081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ecc1083a0 msgr2=0x7f8ecc10bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8eca59c640 1 --2- 172.21.15.3:0/2673008081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ecc1083a0 0x7f8ecc10bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8eca59c640 1 -- 172.21.15.3:0/2673008081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ecc108cf0 msgr2=0x7f8ecc10c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8eca59c640 1 --2- 172.21.15.3:0/2673008081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ecc108cf0 0x7f8ecc10c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.098+0000 7f8eca59c640 1 -- 172.21.15.3:0/2673008081 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc1acaa0 con 0x7f8ecc102a60 2024-09-21T06:19:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.099+0000 7f8eca59c640 1 --2- 172.21.15.3:0/2673008081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc102a60 0x7f8ecc10b850 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00eb90 tx=0x7f8ebc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.099+0000 7f8ec37fe640 1 -- 172.21.15.3:0/2673008081 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc004d20 con 0x7f8ecc102a60 2024-09-21T06:19:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.099+0000 7f8ed0bc3640 1 -- 172.21.15.3:0/2673008081 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ecc1acd90 con 0x7f8ecc102a60 2024-09-21T06:19:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.099+0000 7f8ec37fe640 1 -- 172.21.15.3:0/2673008081 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ebc00ee90 con 0x7f8ecc102a60 2024-09-21T06:19:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.099+0000 7f8ec37fe640 1 -- 172.21.15.3:0/2673008081 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc010670 con 0x7f8ecc102a60 2024-09-21T06:19:29.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.099+0000 7f8ed0bc3640 1 -- 172.21.15.3:0/2673008081 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ecc1ad220 con 0x7f8ecc102a60 2024-09-21T06:19:29.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.102+0000 7f8ec37fe640 1 -- 172.21.15.3:0/2673008081 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f8ebc010870 con 0x7f8ecc102a60 2024-09-21T06:19:29.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.102+0000 7f8ed7137740 1 Processor -- start 2024-09-21T06:19:29.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.102+0000 7f8ed7137740 1 -- start start 2024-09-21T06:19:29.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.102+0000 7f8ed7137740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662ae4070 0x55a662e8b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.102+0000 7f8ed7137740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 0x55a662e8bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.102+0000 7f8ed7137740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e8c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.102+0000 7f8ed7137740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a662e01e50 con 0x55a662abb1b0 2024-09-21T06:19:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.102+0000 7f8ed7137740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a662df1280 con 0x55a662ac3390 2024-09-21T06:19:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.102+0000 7f8ed7137740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a662df5090 con 0x55a662ae4070 2024-09-21T06:19:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.102+0000 7f8eca59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 0x55a662e8bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.102+0000 7f8ecad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662ae4070 0x55a662e8b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.103+0000 7f8ec9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e8c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.103+0000 7f8eca59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 0x55a662e8bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57248/0 (socket says 172.21.15.3:57248) 2024-09-21T06:19:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.103+0000 7f8ecad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662ae4070 0x55a662e8b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:45672/0 (socket says 172.21.15.3:45672) 2024-09-21T06:19:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.103+0000 7f8eca59c640 1 -- 172.21.15.3:0/2102880771 learned_addr learned my addr 172.21.15.3:0/2102880771 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.103+0000 7f8ec37fe640 1 --2- 172.21.15.3:0/2673008081 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8eb80b6680 0x7f8eb80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.103+0000 7f8ec9d9b640 1 -- 172.21.15.3:0/2102880771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662ae4070 msgr2=0x55a662e8b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:29.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.103+0000 7f8ec9d9b640 1 --2- 172.21.15.3:0/2102880771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662ae4070 0x55a662e8b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.103+0000 7f8ec9d9b640 1 -- 172.21.15.3:0/2102880771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 msgr2=0x55a662e8bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:29.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.103+0000 7f8ec9d9b640 1 --2- 172.21.15.3:0/2102880771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 0x55a662e8bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.103+0000 7f8ec9d9b640 1 -- 172.21.15.3:0/2102880771 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a662b4f650 con 0x55a662ac3390 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.103+0000 7f8eca59c640 1 --2- 172.21.15.3:0/2102880771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 0x55a662e8bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.103+0000 7f8ec37fe640 1 -- 172.21.15.3:0/2673008081 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8ebc0e7d70 con 0x7f8ecc102a60 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.103+0000 7f8ecad9d640 1 --2- 172.21.15.3:0/2673008081 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8eb80b6680 0x7f8eb80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.104+0000 7f8ec9d9b640 1 --2- 172.21.15.3:0/2102880771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e8c0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc06aaa0 tx=0x7f8eb4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffb0e8c1704224d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.104+0000 7f8ec0ff9640 1 -- 172.21.15.3:0/2102880771 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4004050 con 0x55a662ac3390 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.104+0000 7f8ec0ff9640 1 -- 172.21.15.3:0/2102880771 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eb40054d0 con 0x55a662ac3390 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.104+0000 7f8ec0ff9640 1 -- 172.21.15.3:0/2102880771 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4004c80 con 0x55a662ac3390 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.105+0000 7f8ed7137740 1 -- 172.21.15.3:0/2102880771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 msgr2=0x55a662e8c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.105+0000 7f8ed7137740 1 --2- 172.21.15.3:0/2102880771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e8c0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc06aaa0 tx=0x7f8eb4011810 comp rx=0 tx=0).stop 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.105+0000 7f8ecad9d640 1 --2- 172.21.15.3:0/2673008081 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8eb80b6680 0x7f8eb80b8b90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4006010 tx=0x7f8ec4009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.105+0000 7f8ed7137740 1 -- 172.21.15.3:0/2102880771 shutdown_connections 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.105+0000 7f8ed7137740 1 --2- 172.21.15.3:0/2102880771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e8c0d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.105+0000 7f8ed7137740 1 --2- 172.21.15.3:0/2102880771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 0x55a662e8bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.105+0000 7f8ed7137740 1 --2- 172.21.15.3:0/2102880771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662ae4070 0x55a662e8b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.105+0000 7f8ed7137740 1 -- 172.21.15.3:0/2102880771 >> 172.21.15.3:0/2102880771 conn(0x55a662ad8460 msgr2=0x55a662e89540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.105+0000 7f8ed7137740 1 -- 172.21.15.3:0/2102880771 shutdown_connections 2024-09-21T06:19:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.105+0000 7f8ed7137740 1 -- 172.21.15.3:0/2102880771 wait complete. 2024-09-21T06:19:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8ed7137740 1 -- start start 2024-09-21T06:19:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8ed7137740 1 Processor -- start 2024-09-21T06:19:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8ed7137740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662abb1b0 0x55a662e96d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8ed7137740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662ac3390 0x55a662e97270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8ed7137740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662ae4070 0x55a662ea3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8ed7137740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a662c52700 con 0x55a662ae4070 2024-09-21T06:19:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8ed7137740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a662d8c110 con 0x55a662abb1b0 2024-09-21T06:19:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8ed7137740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a662e99e30 con 0x55a662ac3390 2024-09-21T06:19:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8ed7137740 10 client.0 _ll_register_callbacks cb 0x55a662da5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8ecad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662ae4070 0x55a662ea3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8eca59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662abb1b0 0x55a662e96d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8ec9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662ac3390 0x55a662e97270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8ecad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662ae4070 0x55a662ea3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57256/0 (socket says 172.21.15.3:57256) 2024-09-21T06:19:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8ecad9d640 1 -- 172.21.15.3:0/4146190445 learned_addr learned my addr 172.21.15.3:0/4146190445 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.106+0000 7f8eca59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662abb1b0 0x55a662e96d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:59974/0 (socket says 172.21.15.3:59974) 2024-09-21T06:19:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8ecad9d640 1 -- 172.21.15.3:0/4146190445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662ac3390 msgr2=0x55a662e97270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8ecad9d640 1 --2- 172.21.15.3:0/4146190445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662ac3390 0x55a662e97270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8ecad9d640 1 -- 172.21.15.3:0/4146190445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662abb1b0 msgr2=0x55a662e96d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8ecad9d640 1 --2- 172.21.15.3:0/4146190445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662abb1b0 0x55a662e96d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8ecad9d640 1 -- 172.21.15.3:0/4146190445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a662db9c40 con 0x55a662ae4070 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8eca59c640 1 --2- 172.21.15.3:0/4146190445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662abb1b0 0x55a662e96d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8ecad9d640 1 --2- 172.21.15.3:0/4146190445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662ae4070 0x55a662ea3120 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4033000 tx=0x7f8ec402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be72cc37cbbab58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8eb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8eb1ffb640 1 -- 172.21.15.3:0/4146190445 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec404e060 con 0x55a662ae4070 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8eb1ffb640 1 -- 172.21.15.3:0/4146190445 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8ec4005030 con 0x55a662ae4070 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8ed7137740 1 -- 172.21.15.3:0/4146190445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eb40054d0 con 0x55a662ae4070 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8eb1ffb640 1 -- 172.21.15.3:0/4146190445 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8ec4035040 con 0x55a662ae4070 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8ed7137740 10 client.16791 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.107+0000 7f8ed7137740 1 -- 172.21.15.3:0/4146190445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a662ea5bc0 con 0x55a662ae4070 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.108+0000 7f8eb1ffb640 1 -- 172.21.15.3:0/4146190445 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eb40054d0 con 0x55a662ae4070 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.108+0000 7f8eb1ffb640 1 -- 172.21.15.3:0/4146190445 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8ec40035f0 con 0x55a662ae4070 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.108+0000 7f8ed7137740 10 client.16791 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.108+0000 7f8ed7137740 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.108+0000 7f8ed7137740 20 client.16791 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.108+0000 7f8ed7137740 1 --2- 172.21.15.3:0/4146190445 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a662aa29f0 0x55a662ea9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.108+0000 7f8ed7137740 4 client.16791 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.108+0000 7f8ed7137740 1 -- 172.21.15.3:0/4146190445 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a662e03da0 con 0x55a662aa29f0 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.108+0000 7f8eca59c640 1 --2- 172.21.15.3:0/4146190445 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a662aa29f0 0x55a662ea9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.109+0000 7f8eb1ffb640 10 client.16791 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.109+0000 7f8eca59c640 1 --2- 172.21.15.3:0/4146190445 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a662aa29f0 0x55a662ea9120 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c399844a1bf98a server_cookie=52be72af5fbe50b4 in_seq=0 out_seq=0 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8eb1ffb640 1 -- 172.21.15.3:0/4146190445 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a662e03da0 con 0x55a662aa29f0 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8eb1ffb640 10 client.16791 handle_command_reply: tid=0 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8ed7137740 1 client.16791 shutdown 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8ed7137740 1 -- 172.21.15.3:0/4146190445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662ae4070 msgr2=0x55a662ea3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8ed7137740 1 --2- 172.21.15.3:0/4146190445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662ae4070 0x55a662ea3120 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4033000 tx=0x7f8ec402a410 comp rx=0 tx=0).stop 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8ed7137740 1 -- 172.21.15.3:0/4146190445 shutdown_connections 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8ed7137740 1 --2- 172.21.15.3:0/4146190445 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a662aa29f0 0x55a662ea9120 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8ed7137740 1 --2- 172.21.15.3:0/4146190445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662ac3390 0x55a662e97270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8ed7137740 1 --2- 172.21.15.3:0/4146190445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662ae4070 0x55a662ea3120 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8eb1ffb640 0 client.16791 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8ed7137740 1 --2- 172.21.15.3:0/4146190445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662abb1b0 0x55a662e96d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8ed7137740 1 -- 172.21.15.3:0/4146190445 >> 172.21.15.3:0/4146190445 conn(0x55a662ad8460 msgr2=0x55a662e8e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8ed7137740 1 -- 172.21.15.3:0/4146190445 shutdown_connections 2024-09-21T06:19:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8ed7137740 1 -- 172.21.15.3:0/4146190445 wait complete. 2024-09-21T06:19:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.110+0000 7f8ed7137740 20 client.16791 trim_cache size 0 max 16384 2024-09-21T06:19:29.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.129+0000 7f8ed7137740 1 Processor -- start 2024-09-21T06:19:29.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.129+0000 7f8ed7137740 1 -- start start 2024-09-21T06:19:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.129+0000 7f8ed7137740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662aa29f0 0x55a662e8cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.129+0000 7f8ed7137740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662abb1b0 0x55a662e88b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.129+0000 7f8ed7137740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e890b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.129+0000 7f8ed7137740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b5b40 con 0x55a662aa29f0 2024-09-21T06:19:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.129+0000 7f8ed7137740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b7680 con 0x55a662ac3390 2024-09-21T06:19:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.129+0000 7f8ed7137740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b6210 con 0x55a662abb1b0 2024-09-21T06:19:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.129+0000 7f8eca59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662aa29f0 0x55a662e8cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.129+0000 7f8eca59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662aa29f0 0x55a662e8cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57262/0 (socket says 172.21.15.3:57262) 2024-09-21T06:19:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.129+0000 7f8eca59c640 1 -- 172.21.15.3:0/1068842741 learned_addr learned my addr 172.21.15.3:0/1068842741 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.129+0000 7f8ec9d9b640 1 --2- 172.21.15.3:0/1068842741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662abb1b0 0x55a662e88b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.130+0000 7f8ecad9d640 1 --2- 172.21.15.3:0/1068842741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e890b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.130+0000 7f8eca59c640 1 -- 172.21.15.3:0/1068842741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662abb1b0 msgr2=0x55a662e88b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.130+0000 7f8eca59c640 1 --2- 172.21.15.3:0/1068842741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662abb1b0 0x55a662e88b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.130+0000 7f8eca59c640 1 -- 172.21.15.3:0/1068842741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 msgr2=0x55a662e890b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.130+0000 7f8eca59c640 1 --2- 172.21.15.3:0/1068842741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e890b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.130+0000 7f8eca59c640 1 -- 172.21.15.3:0/1068842741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a662e95230 con 0x55a662aa29f0 2024-09-21T06:19:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.130+0000 7f8eca59c640 1 --2- 172.21.15.3:0/1068842741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662aa29f0 0x55a662e8cd90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00cba0 tx=0x7f8ebc0ea280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16debe1e97e4d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.130+0000 7f8eb3fff640 1 -- 172.21.15.3:0/1068842741 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc0b2070 con 0x55a662aa29f0 2024-09-21T06:19:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.130+0000 7f8eb3fff640 1 -- 172.21.15.3:0/1068842741 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ebc0ad710 con 0x55a662aa29f0 2024-09-21T06:19:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.130+0000 7f8eb3fff640 1 -- 172.21.15.3:0/1068842741 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc0b26e0 con 0x55a662aa29f0 2024-09-21T06:19:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.131+0000 7f8ed7137740 1 -- 172.21.15.3:0/1068842741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662aa29f0 msgr2=0x55a662e8cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.131+0000 7f8ed7137740 1 --2- 172.21.15.3:0/1068842741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662aa29f0 0x55a662e8cd90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00cba0 tx=0x7f8ebc0ea280 comp rx=0 tx=0).stop 2024-09-21T06:19:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.131+0000 7f8ed7137740 1 -- 172.21.15.3:0/1068842741 shutdown_connections 2024-09-21T06:19:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.131+0000 7f8ed7137740 1 --2- 172.21.15.3:0/1068842741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662abb1b0 0x55a662e88b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.131+0000 7f8ed7137740 1 --2- 172.21.15.3:0/1068842741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e890b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.131+0000 7f8ed7137740 1 --2- 172.21.15.3:0/1068842741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662aa29f0 0x55a662e8cd90 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.131+0000 7f8ed7137740 1 -- 172.21.15.3:0/1068842741 >> 172.21.15.3:0/1068842741 conn(0x55a662ad8460 msgr2=0x55a662e8c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:29.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.131+0000 7f8ed7137740 1 -- 172.21.15.3:0/1068842741 shutdown_connections 2024-09-21T06:19:29.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.131+0000 7f8ed7137740 1 -- 172.21.15.3:0/1068842741 wait complete. 2024-09-21T06:19:29.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ed7137740 1 -- start start 2024-09-21T06:19:29.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ed7137740 1 Processor -- start 2024-09-21T06:19:29.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ed7137740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662aa29f0 0x55a662e9b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ed7137740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 0x55a662e9b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ed7137740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e9bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ed7137740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b6fb0 con 0x55a662abb1b0 2024-09-21T06:19:29.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ed7137740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b68e0 con 0x55a662ac3390 2024-09-21T06:19:29.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ed7137740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b3b20 con 0x55a662aa29f0 2024-09-21T06:19:29.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ed7137740 10 client.0 _ll_register_callbacks cb 0x7f8ec4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:29.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ec9d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 0x55a662e9b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ec9d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 0x55a662e9b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57264/0 (socket says 172.21.15.3:57264) 2024-09-21T06:19:29.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8eca59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662aa29f0 0x55a662e9b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ec9d9b640 1 -- 172.21.15.3:0/2433258381 learned_addr learned my addr 172.21.15.3:0/2433258381 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:29.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ecad9d640 1 --2- 172.21.15.3:0/2433258381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e9bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ec9d9b640 1 -- 172.21.15.3:0/2433258381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662aa29f0 msgr2=0x55a662e9b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:29.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ec9d9b640 1 --2- 172.21.15.3:0/2433258381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662aa29f0 0x55a662e9b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ec9d9b640 1 -- 172.21.15.3:0/2433258381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 msgr2=0x55a662e9bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:29.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ec9d9b640 1 --2- 172.21.15.3:0/2433258381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e9bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ec9d9b640 1 -- 172.21.15.3:0/2433258381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a662f12c10 con 0x55a662abb1b0 2024-09-21T06:19:29.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.132+0000 7f8ecad9d640 1 --2- 172.21.15.3:0/2433258381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e9bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:29.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.133+0000 7f8ec9d9b640 1 --2- 172.21.15.3:0/2433258381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 0x55a662e9b6b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009b40 tx=0x7f8eb40096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7f51fd61c42664 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.133+0000 7f8eb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.133+0000 7f8eb1ffb640 1 -- 172.21.15.3:0/2433258381 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb40037f0 con 0x55a662abb1b0 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.133+0000 7f8eb1ffb640 1 -- 172.21.15.3:0/2433258381 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8eb4003990 con 0x55a662abb1b0 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.133+0000 7f8ed7137740 1 -- 172.21.15.3:0/2433258381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ebc0ad710 con 0x55a662abb1b0 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.133+0000 7f8eb1ffb640 1 -- 172.21.15.3:0/2433258381 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8eb4003bb0 con 0x55a662abb1b0 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.133+0000 7f8ed7137740 10 client.16797 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.133+0000 7f8ed7137740 1 -- 172.21.15.3:0/2433258381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a662f13e80 con 0x55a662abb1b0 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.133+0000 7f8eb1ffb640 1 -- 172.21.15.3:0/2433258381 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ebc0ad710 con 0x55a662abb1b0 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.134+0000 7f8eb1ffb640 1 -- 172.21.15.3:0/2433258381 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8eb402b070 con 0x55a662abb1b0 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.134+0000 7f8ed7137740 10 client.16797 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.134+0000 7f8ed7137740 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.134+0000 7f8ed7137740 20 client.16797 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.134+0000 7f8ed7137740 1 --2- 172.21.15.3:0/2433258381 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a662ae4070 0x55a662f16370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.134+0000 7f8ed7137740 4 client.16797 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:19:29.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.134+0000 7f8ed7137740 1 -- 172.21.15.3:0/2433258381 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a662dff850 con 0x55a662ae4070 2024-09-21T06:19:29.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.134+0000 7f8eca59c640 1 --2- 172.21.15.3:0/2433258381 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a662ae4070 0x55a662f16370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:29.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.134+0000 7f8eca59c640 1 --2- 172.21.15.3:0/2433258381 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a662ae4070 0x55a662f16370 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=358920882549b986 server_cookie=63e68c7c5a89be8f in_seq=0 out_seq=0 2024-09-21T06:19:29.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.134+0000 7f8eb1ffb640 10 client.16797 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:29.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.134+0000 7f8eb1ffb640 1 -- 172.21.15.3:0/2433258381 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x55a662dff850 con 0x55a662ae4070 2024-09-21T06:19:29.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.134+0000 7f8eb1ffb640 10 client.16797 handle_command_reply: tid=0 2024-09-21T06:19:29.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.134+0000 7f8ed7137740 1 client.16797 shutdown 2024-09-21T06:19:29.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.135+0000 7f8ed7137740 1 -- 172.21.15.3:0/2433258381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 msgr2=0x55a662e9b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:29.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.135+0000 7f8ed7137740 1 --2- 172.21.15.3:0/2433258381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 0x55a662e9b6b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009b40 tx=0x7f8eb40096e0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.135+0000 7f8ed7137740 1 -- 172.21.15.3:0/2433258381 shutdown_connections 2024-09-21T06:19:29.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.135+0000 7f8ed7137740 1 --2- 172.21.15.3:0/2433258381 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a662ae4070 0x55a662f16370 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.135+0000 7f8ed7137740 1 --2- 172.21.15.3:0/2433258381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a662ac3390 0x55a662e9bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.135+0000 7f8ed7137740 1 --2- 172.21.15.3:0/2433258381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a662abb1b0 0x55a662e9b6b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.135+0000 7f8ed7137740 1 --2- 172.21.15.3:0/2433258381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a662aa29f0 0x55a662e9b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.135+0000 7f8ed7137740 1 -- 172.21.15.3:0/2433258381 >> 172.21.15.3:0/2433258381 conn(0x55a662ad8460 msgr2=0x55a662e8c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:29.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.135+0000 7f8eb1ffb640 0 client.16797 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:29.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.135+0000 7f8ed7137740 1 -- 172.21.15.3:0/2433258381 shutdown_connections 2024-09-21T06:19:29.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.135+0000 7f8ed7137740 1 -- 172.21.15.3:0/2433258381 wait complete. 2024-09-21T06:19:29.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.135+0000 7f8ed7137740 20 client.16797 trim_cache size 0 max 16384 2024-09-21T06:19:29.146 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:19:29.146 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:19:29.146 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:19:29.146 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:19:29.146 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:19:29.147 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:19:29.147 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:29.147 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:29.147 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:29.147 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:29.147 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:29.147 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:29.147 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:19:29.147 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:19:29.147 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:29.147 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:19:29.148 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:29.149 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:29.150 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:29.151 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:19:29.151 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:19:29.151 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:19:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.136+0000 7f8ed0bc3640 1 -- 172.21.15.3:0/2673008081 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8eb80b6680 msgr2=0x7f8eb80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.136+0000 7f8ed0bc3640 1 --2- 172.21.15.3:0/2673008081 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8eb80b6680 0x7f8eb80b8b90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4006010 tx=0x7f8ec4009260 comp rx=0 tx=0).stop 2024-09-21T06:19:29.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.136+0000 7f8ed0bc3640 1 -- 172.21.15.3:0/2673008081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc102a60 msgr2=0x7f8ecc10b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:29.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.136+0000 7f8ed0bc3640 1 --2- 172.21.15.3:0/2673008081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc102a60 0x7f8ecc10b850 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00eb90 tx=0x7f8ebc00c6a0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.136+0000 7f8ed0bc3640 1 -- 172.21.15.3:0/2673008081 shutdown_connections 2024-09-21T06:19:29.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.136+0000 7f8ed0bc3640 1 --2- 172.21.15.3:0/2673008081 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8eb80b6680 0x7f8eb80b8b90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.136+0000 7f8ed0bc3640 1 --2- 172.21.15.3:0/2673008081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ecc108cf0 0x7f8ecc10c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.136+0000 7f8ed0bc3640 1 --2- 172.21.15.3:0/2673008081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ecc1083a0 0x7f8ecc10bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.136+0000 7f8ed0bc3640 1 --2- 172.21.15.3:0/2673008081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc102a60 0x7f8ecc10b850 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:29.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.136+0000 7f8ed0bc3640 1 -- 172.21.15.3:0/2673008081 >> 172.21.15.3:0/2673008081 conn(0x7f8ecc079730 msgr2=0x7f8ecc1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:29.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.136+0000 7f8ed0bc3640 1 -- 172.21.15.3:0/2673008081 shutdown_connections 2024-09-21T06:19:29.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:29.136+0000 7f8ed0bc3640 1 -- 172.21.15.3:0/2673008081 wait complete. 2024-09-21T06:19:29.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:29 smithi137 ceph-mon[34464]: pgmap v459: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 385 op/s 2024-09-21T06:19:29.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:29 smithi204 ceph-mon[34659]: pgmap v459: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 385 op/s 2024-09-21T06:19:29.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 437449 12.37 MB/sec execute 62 sec latency 18.396 ms 2024-09-21T06:19:30.046 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:19:30.046 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:19:30.335 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.466+0000 7fcbe1989640 1 -- 172.21.15.3:0/2102775269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbdc109440 msgr2=0x7fcbdc109850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.466+0000 7fcbe1989640 1 --2- 172.21.15.3:0/2102775269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbdc109440 0x7fcbdc109850 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009900 tx=0x7fcbd002a410 comp rx=0 tx=0).stop 2024-09-21T06:19:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.466+0000 7fcbe1989640 1 -- 172.21.15.3:0/2102775269 shutdown_connections 2024-09-21T06:19:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.466+0000 7fcbe1989640 1 --2- 172.21.15.3:0/2102775269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbdc109440 0x7fcbdc109850 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.466+0000 7fcbe1989640 1 --2- 172.21.15.3:0/2102775269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc109d90 0x7fcbdc1105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.466+0000 7fcbe1989640 1 --2- 172.21.15.3:0/2102775269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbdc103b10 0x7fcbdc103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.466+0000 7fcbe1989640 1 -- 172.21.15.3:0/2102775269 >> 172.21.15.3:0/2102775269 conn(0x7fcbdc0ff510 msgr2=0x7fcbdc101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.466+0000 7fcbe1989640 1 -- 172.21.15.3:0/2102775269 shutdown_connections 2024-09-21T06:19:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.466+0000 7fcbe1989640 1 -- 172.21.15.3:0/2102775269 wait complete. 2024-09-21T06:19:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.467+0000 7fcbe1989640 1 Processor -- start 2024-09-21T06:19:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.467+0000 7fcbe1989640 1 -- start start 2024-09-21T06:19:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbe1989640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc103b10 0x7fcbdc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbe1989640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbdc109440 0x7fcbdc19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbe1989640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbdc109d90 0x7fcbdc19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbe1989640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc113a50 con 0x7fcbdc109440 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbe1989640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc1138d0 con 0x7fcbdc103b10 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbe1989640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc113bd0 con 0x7fcbdc109d90 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbda7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbdc109440 0x7fcbdc19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbda7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbdc109440 0x7fcbdc19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57274/0 (socket says 172.21.15.3:57274) 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbdaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc103b10 0x7fcbdc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbda7fc640 1 -- 172.21.15.3:0/2416524748 learned_addr learned my addr 172.21.15.3:0/2416524748 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbdb7fe640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbdc109d90 0x7fcbdc19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbda7fc640 1 -- 172.21.15.3:0/2416524748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbdc109d90 msgr2=0x7fcbdc19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbda7fc640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbdc109d90 0x7fcbdc19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbda7fc640 1 -- 172.21.15.3:0/2416524748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc103b10 msgr2=0x7fcbdc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbda7fc640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc103b10 0x7fcbdc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbda7fc640 1 -- 172.21.15.3:0/2416524748 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbdc1a0420 con 0x7fcbdc109440 2024-09-21T06:19:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.468+0000 7fcbdaffd640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc103b10 0x7fcbdc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.469+0000 7fcbda7fc640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbdc109440 0x7fcbdc19aa90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcbd002a930 tx=0x7fcbd0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.469+0000 7fcbe0987640 1 -- 172.21.15.3:0/2416524748 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd003f030 con 0x7fcbdc109440 2024-09-21T06:19:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.469+0000 7fcbe0987640 1 -- 172.21.15.3:0/2416524748 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbd0004490 con 0x7fcbdc109440 2024-09-21T06:19:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.469+0000 7fcbe1989640 1 -- 172.21.15.3:0/2416524748 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbdc1a06b0 con 0x7fcbdc109440 2024-09-21T06:19:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.469+0000 7fcbe0987640 1 -- 172.21.15.3:0/2416524748 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0035070 con 0x7fcbdc109440 2024-09-21T06:19:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.469+0000 7fcbe1989640 1 -- 172.21.15.3:0/2416524748 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbdc1a6ea0 con 0x7fcbdc109440 2024-09-21T06:19:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.472+0000 7fcbe0987640 1 -- 172.21.15.3:0/2416524748 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fcbd0042030 con 0x7fcbdc109440 2024-09-21T06:19:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.472+0000 7fcbe89d0740 1 Processor -- start 2024-09-21T06:19:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.472+0000 7fcbe89d0740 1 -- start start 2024-09-21T06:19:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbe0987640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcbc40b6680 0x7fcbc40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbe89d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dee1b0 0x55a7671939b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbe89d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dc52f0 0x55a767193ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbe89d0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a767194430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbe89d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a7671086f0 con 0x55a766dc52f0 2024-09-21T06:19:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbe89d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7670f7b20 con 0x55a766dee1b0 2024-09-21T06:19:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbe89d0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a7670fb930 con 0x55a766dcd4d0 2024-09-21T06:19:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbdb7fe640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcbc40b6680 0x7fcbc40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbe0987640 1 -- 172.21.15.3:0/2416524748 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcbd0107a50 con 0x7fcbdc109440 2024-09-21T06:19:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbdaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dee1b0 0x55a7671939b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbda7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dc52f0 0x55a767193ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbdb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a767194430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbdaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dee1b0 0x55a7671939b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:60004/0 (socket says 172.21.15.3:60004) 2024-09-21T06:19:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbdaffd640 1 -- 172.21.15.3:0/3824647885 learned_addr learned my addr 172.21.15.3:0/3824647885 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbdaffd640 1 -- 172.21.15.3:0/3824647885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 msgr2=0x55a767194430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbdaffd640 1 --2- 172.21.15.3:0/3824647885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a767194430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbdaffd640 1 -- 172.21.15.3:0/3824647885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dc52f0 msgr2=0x55a767193ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbdaffd640 1 --2- 172.21.15.3:0/3824647885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dc52f0 0x55a767193ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbdaffd640 1 -- 172.21.15.3:0/3824647885 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7670bd530 con 0x55a766dee1b0 2024-09-21T06:19:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.473+0000 7fcbdb7fe640 1 --2- 172.21.15.3:0/3824647885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a767194430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.474+0000 7fcbdb7fe640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcbc40b6680 0x7fcbc40b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009fd0 tx=0x7fcbd4009d80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.474+0000 7fcbdaffd640 1 --2- 172.21.15.3:0/3824647885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dee1b0 0x55a7671939b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc011620 tx=0x7fcbcc011ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da24c2245dfc3c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.474+0000 7fcbc1ffb640 1 -- 172.21.15.3:0/3824647885 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc01e070 con 0x55a766dee1b0 2024-09-21T06:19:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.474+0000 7fcbc1ffb640 1 -- 172.21.15.3:0/3824647885 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbcc00fb10 con 0x55a766dee1b0 2024-09-21T06:19:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.474+0000 7fcbc1ffb640 1 -- 172.21.15.3:0/3824647885 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc019bb0 con 0x55a766dee1b0 2024-09-21T06:19:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.475+0000 7fcbe89d0740 1 -- 172.21.15.3:0/3824647885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dee1b0 msgr2=0x55a7671939b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.475+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/3824647885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dee1b0 0x55a7671939b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc011620 tx=0x7fcbcc011ae0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.475+0000 7fcbe89d0740 1 -- 172.21.15.3:0/3824647885 shutdown_connections 2024-09-21T06:19:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.475+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/3824647885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a767194430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.475+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/3824647885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dc52f0 0x55a767193ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.475+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/3824647885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dee1b0 0x55a7671939b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.475+0000 7fcbe89d0740 1 -- 172.21.15.3:0/3824647885 >> 172.21.15.3:0/3824647885 conn(0x55a766de2300 msgr2=0x55a7671918a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.475+0000 7fcbe89d0740 1 -- 172.21.15.3:0/3824647885 shutdown_connections 2024-09-21T06:19:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.475+0000 7fcbe89d0740 1 -- 172.21.15.3:0/3824647885 wait complete. 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.476+0000 7fcbe89d0740 1 -- start start 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.476+0000 7fcbe89d0740 1 Processor -- start 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.476+0000 7fcbe89d0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dc52f0 0x55a76719f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.476+0000 7fcbe89d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dcd4d0 0x55a76719f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.476+0000 7fcbe89d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dee1b0 0x55a7671ab420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.476+0000 7fcbe89d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a76702fab0 con 0x55a766dcd4d0 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.476+0000 7fcbe89d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a76709d730 con 0x55a766dee1b0 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.476+0000 7fcbe89d0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a7671a2130 con 0x55a766dc52f0 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.476+0000 7fcbe89d0740 10 client.0 _ll_register_callbacks cb 0x55a7670f16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbda7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dcd4d0 0x55a76719f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbda7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dcd4d0 0x55a76719f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57304/0 (socket says 172.21.15.3:57304) 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbda7fc640 1 -- 172.21.15.3:0/2349127125 learned_addr learned my addr 172.21.15.3:0/2349127125 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbdb7fe640 1 --2- 172.21.15.3:0/2349127125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dee1b0 0x55a7671ab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbdaffd640 1 --2- 172.21.15.3:0/2349127125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dc52f0 0x55a76719f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbda7fc640 1 -- 172.21.15.3:0/2349127125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dc52f0 msgr2=0x55a76719f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbda7fc640 1 --2- 172.21.15.3:0/2349127125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dc52f0 0x55a76719f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbda7fc640 1 -- 172.21.15.3:0/2349127125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dee1b0 msgr2=0x55a7671ab420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbda7fc640 1 --2- 172.21.15.3:0/2349127125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dee1b0 0x55a7671ab420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbda7fc640 1 -- 172.21.15.3:0/2349127125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a766e59790 con 0x55a766dcd4d0 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbda7fc640 1 --2- 172.21.15.3:0/2349127125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dcd4d0 0x55a76719f590 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0031440 tx=0x7fcbd0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409962a3a9d1769f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbbaffd640 1 -- 172.21.15.3:0/2349127125 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0049070 con 0x55a766dcd4d0 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbbaffd640 1 -- 172.21.15.3:0/2349127125 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcbd00d6e10 con 0x55a766dcd4d0 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbe89d0740 1 -- 172.21.15.3:0/2349127125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbcc00fb10 con 0x55a766dcd4d0 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbbaffd640 1 -- 172.21.15.3:0/2349127125 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcbd00b8050 con 0x55a766dcd4d0 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbe89d0740 10 client.16809 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.477+0000 7fcbe89d0740 1 -- 172.21.15.3:0/2349127125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a766ed2620 con 0x55a766dcd4d0 2024-09-21T06:19:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.478+0000 7fcbbaffd640 1 -- 172.21.15.3:0/2349127125 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbd0105600 con 0x55a766dcd4d0 2024-09-21T06:19:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.479+0000 7fcbbaffd640 1 -- 172.21.15.3:0/2349127125 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fcbd00cebb0 con 0x55a766dcd4d0 2024-09-21T06:19:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.479+0000 7fcbe89d0740 10 client.16809 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.479+0000 7fcbe89d0740 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.479+0000 7fcbe89d0740 20 client.16809 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.479+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/2349127125 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a766dacd80 0x55a7671b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.479+0000 7fcbe89d0740 4 client.16809 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.479+0000 7fcbe89d0740 1 -- 172.21.15.3:0/2349127125 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcbcc00fb10 con 0x55a766dacd80 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.479+0000 7fcbdaffd640 1 --2- 172.21.15.3:0/2349127125 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a766dacd80 0x55a7671b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.479+0000 7fcbbaffd640 10 client.16809 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.479+0000 7fcbdaffd640 1 --2- 172.21.15.3:0/2349127125 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a766dacd80 0x55a7671b11e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f82a3eb95a5989b server_cookie=febbd91642671b6f in_seq=0 out_seq=0 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbbaffd640 1 -- 172.21.15.3:0/2349127125 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fcbcc00fb10 con 0x55a766dacd80 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbbaffd640 10 client.16809 handle_command_reply: tid=0 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbe89d0740 1 client.16809 shutdown 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbe89d0740 1 -- 172.21.15.3:0/2349127125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dcd4d0 msgr2=0x55a76719f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/2349127125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dcd4d0 0x55a76719f590 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0031440 tx=0x7fcbd0003f60 comp rx=0 tx=0).stop 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbe89d0740 1 -- 172.21.15.3:0/2349127125 shutdown_connections 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/2349127125 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a766dacd80 0x55a7671b11e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/2349127125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dee1b0 0x55a7671ab420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/2349127125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dcd4d0 0x55a76719f590 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbbaffd640 0 client.16809 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/2349127125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dc52f0 0x55a76719f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbe89d0740 1 -- 172.21.15.3:0/2349127125 >> 172.21.15.3:0/2349127125 conn(0x55a766de2300 msgr2=0x55a7671949e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbe89d0740 1 -- 172.21.15.3:0/2349127125 shutdown_connections 2024-09-21T06:19:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbe89d0740 1 -- 172.21.15.3:0/2349127125 wait complete. 2024-09-21T06:19:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.481+0000 7fcbe89d0740 20 client.16809 trim_cache size 0 max 16384 2024-09-21T06:19:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.496+0000 7fcbe89d0740 1 Processor -- start 2024-09-21T06:19:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.496+0000 7fcbe89d0740 1 -- start start 2024-09-21T06:19:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbe89d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 0x55a7671953a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbe89d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dc52f0 0x55a7671905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbe89d0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a76719c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbe89d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc1af6e0 con 0x55a766dacd80 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbe89d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc1b1220 con 0x55a766dc52f0 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbe89d0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc1afdb0 con 0x55a766dcd4d0 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbdaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 0x55a7671953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbdaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 0x55a7671953a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57314/0 (socket says 172.21.15.3:57314) 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbdb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a76719c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbdaffd640 1 -- 172.21.15.3:0/834445903 learned_addr learned my addr 172.21.15.3:0/834445903 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbda7fc640 1 --2- 172.21.15.3:0/834445903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dc52f0 0x55a7671905b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbdaffd640 1 -- 172.21.15.3:0/834445903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 msgr2=0x55a76719c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbdaffd640 1 --2- 172.21.15.3:0/834445903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a76719c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbdaffd640 1 -- 172.21.15.3:0/834445903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dc52f0 msgr2=0x55a7671905b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbdaffd640 1 --2- 172.21.15.3:0/834445903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dc52f0 0x55a7671905b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbdaffd640 1 -- 172.21.15.3:0/834445903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a76718bb40 con 0x55a766dacd80 2024-09-21T06:19:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbdb7fe640 1 --2- 172.21.15.3:0/834445903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a76719c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.497+0000 7fcbdaffd640 1 --2- 172.21.15.3:0/834445903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 0x55a7671953a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0196e0 tx=0x7fcbcc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9994f1259e08eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.498+0000 7fcbc17fa640 1 -- 172.21.15.3:0/834445903 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc01e040 con 0x55a766dacd80 2024-09-21T06:19:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.498+0000 7fcbc17fa640 1 -- 172.21.15.3:0/834445903 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbcc00fb10 con 0x55a766dacd80 2024-09-21T06:19:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.498+0000 7fcbc17fa640 1 -- 172.21.15.3:0/834445903 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc019840 con 0x55a766dacd80 2024-09-21T06:19:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.498+0000 7fcbe89d0740 1 -- 172.21.15.3:0/834445903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 msgr2=0x55a7671953a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.498+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/834445903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 0x55a7671953a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0196e0 tx=0x7fcbcc019710 comp rx=0 tx=0).stop 2024-09-21T06:19:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.499+0000 7fcbe89d0740 1 -- 172.21.15.3:0/834445903 shutdown_connections 2024-09-21T06:19:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.499+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/834445903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a76719c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.499+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/834445903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dc52f0 0x55a7671905b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.499+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/834445903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 0x55a7671953a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.499+0000 7fcbe89d0740 1 -- 172.21.15.3:0/834445903 >> 172.21.15.3:0/834445903 conn(0x55a766de2300 msgr2=0x55a76718fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.499+0000 7fcbe89d0740 1 -- 172.21.15.3:0/834445903 shutdown_connections 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.499+0000 7fcbe89d0740 1 -- 172.21.15.3:0/834445903 wait complete. 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.499+0000 7fcbe89d0740 1 -- start start 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.499+0000 7fcbe89d0740 1 Processor -- start 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbe89d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 0x55a7671a7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbe89d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dc52f0 0x55a7671a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbe89d0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a7671a5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbe89d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc1b0b50 con 0x55a766dacd80 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbe89d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc1b0480 con 0x55a766dc52f0 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbe89d0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc1ad6c0 con 0x55a766dcd4d0 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbe89d0740 10 client.0 _ll_register_callbacks cb 0x7fcbd0009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbda7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dc52f0 0x55a7671a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbdaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 0x55a7671a7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbdaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 0x55a7671a7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57328/0 (socket says 172.21.15.3:57328) 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbdaffd640 1 -- 172.21.15.3:0/1744113062 learned_addr learned my addr 172.21.15.3:0/1744113062 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbdb7fe640 1 --2- 172.21.15.3:0/1744113062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a7671a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbdaffd640 1 -- 172.21.15.3:0/1744113062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 msgr2=0x55a7671a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.500+0000 7fcbdaffd640 1 --2- 172.21.15.3:0/1744113062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a7671a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.501+0000 7fcbdaffd640 1 -- 172.21.15.3:0/1744113062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dc52f0 msgr2=0x55a7671a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.501+0000 7fcbdaffd640 1 --2- 172.21.15.3:0/1744113062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dc52f0 0x55a7671a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.501+0000 7fcbdaffd640 1 -- 172.21.15.3:0/1744113062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7671a5950 con 0x55a766dacd80 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.501+0000 7fcbdaffd640 1 --2- 172.21.15.3:0/1744113062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 0x55a7671a7990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009d70 tx=0x7fcbcc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa853f869a53df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.501+0000 7fcbbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.501+0000 7fcbbaffd640 1 -- 172.21.15.3:0/1744113062 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc036070 con 0x55a766dacd80 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.501+0000 7fcbbaffd640 1 -- 172.21.15.3:0/1744113062 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcbcc00f660 con 0x55a766dacd80 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.501+0000 7fcbe89d0740 1 -- 172.21.15.3:0/1744113062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbcc00fb10 con 0x55a766dacd80 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.501+0000 7fcbbaffd640 1 -- 172.21.15.3:0/1744113062 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcbcc01e070 con 0x55a766dacd80 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.501+0000 7fcbe89d0740 10 client.16815 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.501+0000 7fcbe89d0740 1 -- 172.21.15.3:0/1744113062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a76721b930 con 0x55a766dacd80 2024-09-21T06:19:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.502+0000 7fcbbaffd640 1 -- 172.21.15.3:0/1744113062 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbcc00fb10 con 0x55a766dacd80 2024-09-21T06:19:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.502+0000 7fcbbaffd640 1 -- 172.21.15.3:0/1744113062 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fcbcc019980 con 0x55a766dacd80 2024-09-21T06:19:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.502+0000 7fcbe89d0740 10 client.16815 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.502+0000 7fcbe89d0740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.502+0000 7fcbe89d0740 20 client.16815 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.502+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/1744113062 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a766dee1b0 0x55a76721ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.502+0000 7fcbe89d0740 4 client.16815 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:19:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.502+0000 7fcbe89d0740 1 -- 172.21.15.3:0/1744113062 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a76710a640 con 0x55a766dee1b0 2024-09-21T06:19:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.502+0000 7fcbda7fc640 1 --2- 172.21.15.3:0/1744113062 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a766dee1b0 0x55a76721ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.503+0000 7fcbbaffd640 10 client.16815 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.503+0000 7fcbda7fc640 1 --2- 172.21.15.3:0/1744113062 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a766dee1b0 0x55a76721ddc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee23f5603cf8d428 server_cookie=e9aa00a5097c6083 in_seq=0 out_seq=0 2024-09-21T06:19:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.640+0000 7fcbbaffd640 1 -- 172.21.15.3:0/1744113062 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a76710a640 con 0x55a766dee1b0 2024-09-21T06:19:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.640+0000 7fcbbaffd640 10 client.16815 handle_command_reply: tid=0 2024-09-21T06:19:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.640+0000 7fcbe89d0740 1 client.16815 shutdown 2024-09-21T06:19:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.640+0000 7fcbe89d0740 1 -- 172.21.15.3:0/1744113062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 msgr2=0x55a7671a7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.640+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/1744113062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 0x55a7671a7990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009d70 tx=0x7fcbcc009f60 comp rx=0 tx=0).stop 2024-09-21T06:19:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.640+0000 7fcbe89d0740 1 -- 172.21.15.3:0/1744113062 shutdown_connections 2024-09-21T06:19:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.640+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/1744113062 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a766dee1b0 0x55a76721ddc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.640+0000 7fcbbaffd640 0 client.16815 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.641+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/1744113062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a766dcd4d0 0x55a7671a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.641+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/1744113062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a766dc52f0 0x55a7671a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.641+0000 7fcbe89d0740 1 --2- 172.21.15.3:0/1744113062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766dacd80 0x55a7671a7990 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.641+0000 7fcbe89d0740 1 -- 172.21.15.3:0/1744113062 >> 172.21.15.3:0/1744113062 conn(0x55a766de2300 msgr2=0x55a767190630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.641+0000 7fcbe89d0740 1 -- 172.21.15.3:0/1744113062 shutdown_connections 2024-09-21T06:19:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.641+0000 7fcbe89d0740 1 -- 172.21.15.3:0/1744113062 wait complete. 2024-09-21T06:19:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.641+0000 7fcbe89d0740 20 client.16815 trim_cache size 0 max 16384 2024-09-21T06:19:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:19:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:19:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:19:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:19:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.642+0000 7fcbe1989640 1 -- 172.21.15.3:0/2416524748 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcbc40b6680 msgr2=0x7fcbc40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.642+0000 7fcbe1989640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcbc40b6680 0x7fcbc40b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009fd0 tx=0x7fcbd4009d80 comp rx=0 tx=0).stop 2024-09-21T06:19:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.642+0000 7fcbe1989640 1 -- 172.21.15.3:0/2416524748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbdc109440 msgr2=0x7fcbdc19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.642+0000 7fcbe1989640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbdc109440 0x7fcbdc19aa90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcbd002a930 tx=0x7fcbd0031a20 comp rx=0 tx=0).stop 2024-09-21T06:19:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.642+0000 7fcbe1989640 1 -- 172.21.15.3:0/2416524748 shutdown_connections 2024-09-21T06:19:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.642+0000 7fcbe1989640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcbc40b6680 0x7fcbc40b8b90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.642+0000 7fcbe1989640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbdc109d90 0x7fcbdc19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.642+0000 7fcbe1989640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbdc109440 0x7fcbdc19aa90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.642+0000 7fcbe1989640 1 --2- 172.21.15.3:0/2416524748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc103b10 0x7fcbdc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.642+0000 7fcbe1989640 1 -- 172.21.15.3:0/2416524748 >> 172.21.15.3:0/2416524748 conn(0x7fcbdc0ff510 msgr2=0x7fcbdc101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.643+0000 7fcbe1989640 1 -- 172.21.15.3:0/2416524748 shutdown_connections 2024-09-21T06:19:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:30.643+0000 7fcbe1989640 1 -- 172.21.15.3:0/2416524748 wait complete. 2024-09-21T06:19:30.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.864+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/122000891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7e4102650 msgr2=0x7fd7e4102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:30.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.864+0000 7fd7ebf2c640 1 --2- 172.21.15.3:0/122000891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7e4102650 0x7fd7e4102a60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009900 tx=0x7fd7e002a480 comp rx=0 tx=0).stop 2024-09-21T06:19:30.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.864+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/122000891 shutdown_connections 2024-09-21T06:19:30.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.864+0000 7fd7ebf2c640 1 --2- 172.21.15.3:0/122000891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e4110200 0x7fd7e4112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.864+0000 7fd7ebf2c640 1 --2- 172.21.15.3:0/122000891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7e4102fa0 0x7fd7e410fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.864+0000 7fd7ebf2c640 1 --2- 172.21.15.3:0/122000891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7e4102650 0x7fd7e4102a60 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.864+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/122000891 >> 172.21.15.3:0/122000891 conn(0x7fd7e4079770 msgr2=0x7fd7e4079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:30.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.864+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/122000891 shutdown_connections 2024-09-21T06:19:30.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.864+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/122000891 wait complete. 2024-09-21T06:19:30.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ebf2c640 1 Processor -- start 2024-09-21T06:19:30.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ebf2c640 1 -- start start 2024-09-21T06:19:30.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ebf2c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7e4102650 0x7fd7e419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ebf2c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7e4102fa0 0x7fd7e419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ebf2c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e4110200 0x7fd7e41a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ebf2c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e4115cf0 con 0x7fd7e4102fa0 2024-09-21T06:19:30.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ebf2c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e4115b70 con 0x7fd7e4102650 2024-09-21T06:19:30.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ebf2c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e4115e70 con 0x7fd7e4110200 2024-09-21T06:19:30.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7e94c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7e4102fa0 0x7fd7e419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ea4c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e4110200 0x7fd7e41a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7e94c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7e4102fa0 0x7fd7e419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57334/0 (socket says 172.21.15.3:57334) 2024-09-21T06:19:30.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7e94c7640 1 -- 172.21.15.3:0/2235370459 learned_addr learned my addr 172.21.15.3:0/2235370459 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7e9cc8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7e4102650 0x7fd7e419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ea4c9640 1 -- 172.21.15.3:0/2235370459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7e4102650 msgr2=0x7fd7e419c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ea4c9640 1 --2- 172.21.15.3:0/2235370459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7e4102650 0x7fd7e419c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ea4c9640 1 -- 172.21.15.3:0/2235370459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7e4102fa0 msgr2=0x7fd7e419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ea4c9640 1 --2- 172.21.15.3:0/2235370459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7e4102fa0 0x7fd7e419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7ea4c9640 1 -- 172.21.15.3:0/2235370459 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e41a2610 con 0x7fd7e4110200 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.865+0000 7fd7e94c7640 1 --2- 172.21.15.3:0/2235370459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7e4102fa0 0x7fd7e419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.866+0000 7fd7ea4c9640 1 --2- 172.21.15.3:0/2235370459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e4110200 0x7fd7e41a1ee0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009fd0 tx=0x7fd7dc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.866+0000 7fd7daffd640 1 -- 172.21.15.3:0/2235370459 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc00ce20 con 0x7fd7e4110200 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.866+0000 7fd7daffd640 1 -- 172.21.15.3:0/2235370459 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7dc005020 con 0x7fd7e4110200 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.866+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/2235370459 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e41a2900 con 0x7fd7e4110200 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.866+0000 7fd7daffd640 1 -- 172.21.15.3:0/2235370459 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc010690 con 0x7fd7e4110200 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.866+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/2235370459 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e41a9060 con 0x7fd7e4110200 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.867+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/2235370459 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e406c260 con 0x7fd7e4110200 2024-09-21T06:19:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.869+0000 7fd7daffd640 1 -- 172.21.15.3:0/2235370459 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fd7dc0054d0 con 0x7fd7e4110200 2024-09-21T06:19:30.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.869+0000 7fd7daffd640 1 --2- 172.21.15.3:0/2235370459 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd7d00b6680 0x7fd7d00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:30.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.870+0000 7fd7e9cc8640 1 --2- 172.21.15.3:0/2235370459 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd7d00b6680 0x7fd7d00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:30.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.870+0000 7fd7daffd640 1 -- 172.21.15.3:0/2235370459 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fd7dc0e7de0 con 0x7fd7e4110200 2024-09-21T06:19:30.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.870+0000 7fd7e9cc8640 1 --2- 172.21.15.3:0/2235370459 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd7d00b6680 0x7fd7d00b8b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0031440 tx=0x7fd7e0009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:30.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:30.873+0000 7fd7daffd640 1 -- 172.21.15.3:0/2235370459 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd7dc0ae310 con 0x7fd7e4110200 2024-09-21T06:19:30.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 440107 12.44 MB/sec execute 63 sec latency 81.534 ms 2024-09-21T06:19:31.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.106+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/2235370459 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7e4102a60 con 0x7fd7e4110200 2024-09-21T06:19:31.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.106+0000 7fd7daffd640 1 -- 172.21.15.3:0/2235370459 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11962 (secure 0 0 0) 0x7fd7e4102a60 con 0x7fd7e4110200 2024-09-21T06:19:31.109 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:19:31.109 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"btime":"2024-09-21T06:18:51:489472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:18:50.916258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:19:31.109 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 31 2024-09-21T06:19:31.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.109+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/2235370459 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd7d00b6680 msgr2=0x7fd7d00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:31.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.109+0000 7fd7ebf2c640 1 --2- 172.21.15.3:0/2235370459 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd7d00b6680 0x7fd7d00b8b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0031440 tx=0x7fd7e0009560 comp rx=0 tx=0).stop 2024-09-21T06:19:31.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.109+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/2235370459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e4110200 msgr2=0x7fd7e41a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:31.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.109+0000 7fd7ebf2c640 1 --2- 172.21.15.3:0/2235370459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e4110200 0x7fd7e41a1ee0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009fd0 tx=0x7fd7dc00ef10 comp rx=0 tx=0).stop 2024-09-21T06:19:31.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.109+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/2235370459 shutdown_connections 2024-09-21T06:19:31.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.109+0000 7fd7ebf2c640 1 --2- 172.21.15.3:0/2235370459 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd7d00b6680 0x7fd7d00b8b90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:31.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.109+0000 7fd7ebf2c640 1 --2- 172.21.15.3:0/2235370459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e4110200 0x7fd7e41a1ee0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:31.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.109+0000 7fd7ebf2c640 1 --2- 172.21.15.3:0/2235370459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7e4102fa0 0x7fd7e419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:31.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.109+0000 7fd7ebf2c640 1 --2- 172.21.15.3:0/2235370459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7e4102650 0x7fd7e419c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:31.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.109+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/2235370459 >> 172.21.15.3:0/2235370459 conn(0x7fd7e4079770 msgr2=0x7fd7e4108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:31.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.109+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/2235370459 shutdown_connections 2024-09-21T06:19:31.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:31.109+0000 7fd7ebf2c640 1 -- 172.21.15.3:0/2235370459 wait complete. 2024-09-21T06:19:31.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:31 smithi003 ceph-mon[31227]: pgmap v460: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 398 op/s 2024-09-21T06:19:31.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:31 smithi137 ceph-mon[34464]: pgmap v460: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 398 op/s 2024-09-21T06:19:31.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:31 smithi204 ceph-mon[34659]: pgmap v460: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 398 op/s 2024-09-21T06:19:31.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 442311 12.36 MB/sec execute 64 sec latency 18.991 ms 2024-09-21T06:19:32.205 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:19:32.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:32 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2235370459' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:19:32.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:32 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2235370459' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:19:32.492 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:32.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:32 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2235370459' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:19:32.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 443906 12.36 MB/sec execute 65 sec latency 75.676 ms 2024-09-21T06:19:33.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.024+0000 7f745e367640 1 -- 172.21.15.3:0/3865293633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7458108cf0 msgr2=0x7f745810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.024+0000 7f745e367640 1 --2- 172.21.15.3:0/3865293633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7458108cf0 0x7f745810f520 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7450009ca0 tx=0x7f745002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.024+0000 7f745e367640 1 -- 172.21.15.3:0/3865293633 shutdown_connections 2024-09-21T06:19:33.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.024+0000 7f745e367640 1 --2- 172.21.15.3:0/3865293633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7458108cf0 0x7f745810f520 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.024+0000 7f745e367640 1 --2- 172.21.15.3:0/3865293633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74581083a0 0x7f74581087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.024+0000 7f745e367640 1 --2- 172.21.15.3:0/3865293633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7458102a60 0x7f7458102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.024+0000 7f745e367640 1 -- 172.21.15.3:0/3865293633 >> 172.21.15.3:0/3865293633 conn(0x7f7458079730 msgr2=0x7f7458079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.024+0000 7f745e367640 1 -- 172.21.15.3:0/3865293633 shutdown_connections 2024-09-21T06:19:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.024+0000 7f745e367640 1 -- 172.21.15.3:0/3865293633 wait complete. 2024-09-21T06:19:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.025+0000 7f745e367640 1 Processor -- start 2024-09-21T06:19:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.025+0000 7f745e367640 1 -- start start 2024-09-21T06:19:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.025+0000 7f745e367640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7458102a60 0x7f7458199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.025+0000 7f745e367640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74581083a0 0x7f74581995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.025+0000 7f745e367640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7458108cf0 0x7f7458199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.025+0000 7f745e367640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7458112960 con 0x7f7458102a60 2024-09-21T06:19:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.025+0000 7f745e367640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f74581127e0 con 0x7f7458108cf0 2024-09-21T06:19:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.025+0000 7f745e367640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7458112ae0 con 0x7f74581083a0 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f7457fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7458102a60 0x7f7458199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f7457fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7458102a60 0x7f7458199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57354/0 (socket says 172.21.15.3:57354) 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f7457fff640 1 -- 172.21.15.3:0/2237582131 learned_addr learned my addr 172.21.15.3:0/2237582131 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f74577fe640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74581083a0 0x7f74581995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f745c904640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7458108cf0 0x7f7458199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f7457fff640 1 -- 172.21.15.3:0/2237582131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74581083a0 msgr2=0x7f74581995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f7457fff640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74581083a0 0x7f74581995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f7457fff640 1 -- 172.21.15.3:0/2237582131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7458108cf0 msgr2=0x7f7458199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f7457fff640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7458108cf0 0x7f7458199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f7457fff640 1 -- 172.21.15.3:0/2237582131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74581a8730 con 0x7f7458102a60 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f7457fff640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7458102a60 0x7f7458199080 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f744800e9f0 tx=0x7f744800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f74557fa640 1 -- 172.21.15.3:0/2237582131 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744800cd80 con 0x7f7458102a60 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f745e367640 1 -- 172.21.15.3:0/2237582131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74581a8a20 con 0x7f7458102a60 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f74557fa640 1 -- 172.21.15.3:0/2237582131 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74480054d0 con 0x7f7458102a60 2024-09-21T06:19:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f74557fa640 1 -- 172.21.15.3:0/2237582131 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448010690 con 0x7f7458102a60 2024-09-21T06:19:33.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.026+0000 7f745e367640 1 -- 172.21.15.3:0/2237582131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74581a8f10 con 0x7f7458102a60 2024-09-21T06:19:33.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.029+0000 7f74557fa640 1 -- 172.21.15.3:0/2237582131 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f7448004e90 con 0x7f7458102a60 2024-09-21T06:19:33.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f74648db740 1 Processor -- start 2024-09-21T06:19:33.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f74648db740 1 -- start start 2024-09-21T06:19:33.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f74648db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fd8070 0x5580c837f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f74648db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 0x5580c837f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f74648db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7fb7390 0x5580c837fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f74648db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5580c82f5ea0 con 0x5580c7faf1b0 2024-09-21T06:19:33.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f74648db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5580c82e52d0 con 0x5580c7fd8070 2024-09-21T06:19:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f74648db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5580c82e90e0 con 0x5580c7fb7390 2024-09-21T06:19:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f74557fa640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74400b6680 0x7f74400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f7457fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 0x5580c837f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f7457fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 0x5580c837f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57362/0 (socket says 172.21.15.3:57362) 2024-09-21T06:19:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f7457fff640 1 -- 172.21.15.3:0/215321258 learned_addr learned my addr 172.21.15.3:0/215321258 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f745c904640 1 --2- 172.21.15.3:0/215321258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fd8070 0x5580c837f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f745c904640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74400b6680 0x7f74400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.030+0000 7f74577fe640 1 --2- 172.21.15.3:0/215321258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7fb7390 0x5580c837fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.031+0000 7f74557fa640 1 -- 172.21.15.3:0/2237582131 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f74480e7ab0 con 0x7f7458102a60 2024-09-21T06:19:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.031+0000 7f7457fff640 1 -- 172.21.15.3:0/215321258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7fb7390 msgr2=0x5580c837fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.031+0000 7f7457fff640 1 --2- 172.21.15.3:0/215321258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7fb7390 0x5580c837fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.031+0000 7f7457fff640 1 -- 172.21.15.3:0/215321258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fd8070 msgr2=0x5580c837f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.031+0000 7f7457fff640 1 --2- 172.21.15.3:0/215321258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fd8070 0x5580c837f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.031+0000 7f7457fff640 1 -- 172.21.15.3:0/215321258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c8043650 con 0x5580c7faf1b0 2024-09-21T06:19:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.031+0000 7f745c904640 1 --2- 172.21.15.3:0/215321258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fd8070 0x5580c837f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.031+0000 7f74577fe640 1 --2- 172.21.15.3:0/215321258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7fb7390 0x5580c837fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:19:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.031+0000 7f7457fff640 1 --2- 172.21.15.3:0/215321258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 0x5580c837f800 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f74480a9f10 tx=0x7f7448004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ae006b4686998e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.031+0000 7f745c904640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74400b6680 0x7f74400b8b90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7450009a60 tx=0x7f74500095c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.031+0000 7f743e7fc640 1 -- 172.21.15.3:0/215321258 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74480ad0b0 con 0x5580c7faf1b0 2024-09-21T06:19:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.031+0000 7f743e7fc640 1 -- 172.21.15.3:0/215321258 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74480ad450 con 0x5580c7faf1b0 2024-09-21T06:19:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.031+0000 7f743e7fc640 1 -- 172.21.15.3:0/215321258 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448005180 con 0x5580c7faf1b0 2024-09-21T06:19:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.032+0000 7f74648db740 1 -- 172.21.15.3:0/215321258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 msgr2=0x5580c837f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.032+0000 7f74648db740 1 --2- 172.21.15.3:0/215321258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 0x5580c837f800 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f74480a9f10 tx=0x7f7448004a90 comp rx=0 tx=0).stop 2024-09-21T06:19:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.032+0000 7f74648db740 1 -- 172.21.15.3:0/215321258 shutdown_connections 2024-09-21T06:19:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.032+0000 7f74648db740 1 --2- 172.21.15.3:0/215321258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7fb7390 0x5580c837fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.032+0000 7f74648db740 1 --2- 172.21.15.3:0/215321258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 0x5580c837f800 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.032+0000 7f74648db740 1 --2- 172.21.15.3:0/215321258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fd8070 0x5580c837f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.032+0000 7f74648db740 1 -- 172.21.15.3:0/215321258 >> 172.21.15.3:0/215321258 conn(0x5580c7fcc460 msgr2=0x5580c837d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.032+0000 7f74648db740 1 -- 172.21.15.3:0/215321258 shutdown_connections 2024-09-21T06:19:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.032+0000 7f74648db740 1 -- 172.21.15.3:0/215321258 wait complete. 2024-09-21T06:19:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.033+0000 7f74648db740 1 -- start start 2024-09-21T06:19:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.033+0000 7f74648db740 1 Processor -- start 2024-09-21T06:19:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.033+0000 7f74648db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7faf1b0 0x5580c838a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.033+0000 7f74648db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c838aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.033+0000 7f74648db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7fd8070 0x5580c8396d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.033+0000 7f74648db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5580c8146700 con 0x5580c7fd8070 2024-09-21T06:19:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.033+0000 7f74648db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5580c8280160 con 0x5580c7fb7390 2024-09-21T06:19:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.033+0000 7f74648db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5580c838d9e0 con 0x5580c7faf1b0 2024-09-21T06:19:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.033+0000 7f74648db740 10 client.0 _ll_register_callbacks cb 0x5580c82999e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f74577fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c838aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f745c904640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7fd8070 0x5580c8396d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f74577fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c838aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:60090/0 (socket says 172.21.15.3:60090) 2024-09-21T06:19:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f74577fe640 1 -- 172.21.15.3:0/32897809 learned_addr learned my addr 172.21.15.3:0/32897809 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f7457fff640 1 --2- 172.21.15.3:0/32897809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7faf1b0 0x5580c838a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f74577fe640 1 -- 172.21.15.3:0/32897809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7faf1b0 msgr2=0x5580c838a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f74577fe640 1 --2- 172.21.15.3:0/32897809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7faf1b0 0x5580c838a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f74577fe640 1 -- 172.21.15.3:0/32897809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7fd8070 msgr2=0x5580c8396d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f74577fe640 1 --2- 172.21.15.3:0/32897809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7fd8070 0x5580c8396d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f74577fe640 1 -- 172.21.15.3:0/32897809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c82adc90 con 0x5580c7fb7390 2024-09-21T06:19:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f74577fe640 1 --2- 172.21.15.3:0/32897809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c838aee0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f744c012700 tx=0x7f744c012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ef40eedb6323754 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f74377fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f74377fe640 1 -- 172.21.15.3:0/32897809 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c00f920 con 0x5580c7fb7390 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f74648db740 1 -- 172.21.15.3:0/32897809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74480ad450 con 0x5580c7fb7390 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.034+0000 7f74377fe640 1 -- 172.21.15.3:0/32897809 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f744c00fac0 con 0x5580c7fb7390 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.035+0000 7f74377fe640 1 -- 172.21.15.3:0/32897809 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f744c00fcc0 con 0x5580c7fb7390 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.035+0000 7f74648db740 10 client.26408 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.035+0000 7f74648db740 1 -- 172.21.15.3:0/32897809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5580c83997d0 con 0x5580c7fb7390 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.035+0000 7f74377fe640 1 -- 172.21.15.3:0/32897809 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74480ad450 con 0x5580c7fb7390 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.035+0000 7f74377fe640 1 -- 172.21.15.3:0/32897809 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f744c010430 con 0x5580c7fb7390 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.035+0000 7f74648db740 10 client.26408 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.035+0000 7f74648db740 10 client.26408 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.035+0000 7f74648db740 20 client.26408 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.035+0000 7f74648db740 1 --2- 172.21.15.3:0/32897809 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5580c7f969f0 0x5580c839cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.035+0000 7f74648db740 4 client.26408 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:33.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.035+0000 7f74648db740 1 -- 172.21.15.3:0/32897809 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580c82f7df0 con 0x5580c7f969f0 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.035+0000 7f7457fff640 1 --2- 172.21.15.3:0/32897809 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5580c7f969f0 0x5580c839cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.036+0000 7f7457fff640 1 --2- 172.21.15.3:0/32897809 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5580c7f969f0 0x5580c839cc10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fe5d4ada444949b6 server_cookie=4f4356e05eed564c in_seq=0 out_seq=0 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.036+0000 7f74377fe640 10 client.26408 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.036+0000 7f74377fe640 1 -- 172.21.15.3:0/32897809 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5580c82f7df0 con 0x5580c7f969f0 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.036+0000 7f74377fe640 10 client.26408 handle_command_reply: tid=0 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.037+0000 7f74648db740 1 client.26408 shutdown 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.037+0000 7f74648db740 1 -- 172.21.15.3:0/32897809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 msgr2=0x5580c838aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.037+0000 7f74648db740 1 --2- 172.21.15.3:0/32897809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c838aee0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f744c012700 tx=0x7f744c012bc0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.037+0000 7f74648db740 1 -- 172.21.15.3:0/32897809 shutdown_connections 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.037+0000 7f74648db740 1 --2- 172.21.15.3:0/32897809 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5580c7f969f0 0x5580c839cc10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.037+0000 7f74648db740 1 --2- 172.21.15.3:0/32897809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7fd8070 0x5580c8396d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.037+0000 7f74377fe640 0 client.26408 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.037+0000 7f74648db740 1 --2- 172.21.15.3:0/32897809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c838aee0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.037+0000 7f74648db740 1 --2- 172.21.15.3:0/32897809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7faf1b0 0x5580c838a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.037+0000 7f74648db740 1 -- 172.21.15.3:0/32897809 >> 172.21.15.3:0/32897809 conn(0x5580c7fcc460 msgr2=0x5580c8382450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.037+0000 7f74648db740 1 -- 172.21.15.3:0/32897809 shutdown_connections 2024-09-21T06:19:33.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.037+0000 7f74648db740 1 -- 172.21.15.3:0/32897809 wait complete. 2024-09-21T06:19:33.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.037+0000 7f74648db740 20 client.26408 trim_cache size 0 max 16384 2024-09-21T06:19:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.051+0000 7f74648db740 1 Processor -- start 2024-09-21T06:19:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.051+0000 7f74648db740 1 -- start start 2024-09-21T06:19:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.051+0000 7f74648db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7faf1b0 0x5580c8380a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.051+0000 7f74648db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c837c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.051+0000 7f74648db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7fd8070 0x5580c837cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.051+0000 7f74648db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f74581b17f0 con 0x5580c7fd8070 2024-09-21T06:19:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.051+0000 7f74648db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f74581b3330 con 0x5580c7fb7390 2024-09-21T06:19:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.051+0000 7f74648db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74581b1ec0 con 0x5580c7faf1b0 2024-09-21T06:19:33.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f7457fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7faf1b0 0x5580c8380a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f74577fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c837c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f745c904640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7fd8070 0x5580c837cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f7457fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7faf1b0 0x5580c8380a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:45840/0 (socket says 172.21.15.3:45840) 2024-09-21T06:19:33.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f7457fff640 1 -- 172.21.15.3:0/27707785 learned_addr learned my addr 172.21.15.3:0/27707785 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:33.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f74577fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c837c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:60092/0 (socket says 172.21.15.3:60092) 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f745c904640 1 -- 172.21.15.3:0/27707785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7faf1b0 msgr2=0x5580c8380a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f745c904640 1 --2- 172.21.15.3:0/27707785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7faf1b0 0x5580c8380a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f745c904640 1 -- 172.21.15.3:0/27707785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 msgr2=0x5580c837c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f745c904640 1 --2- 172.21.15.3:0/27707785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c837c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f745c904640 1 -- 172.21.15.3:0/27707785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c8377460 con 0x5580c7fd8070 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f745c904640 1 --2- 172.21.15.3:0/27707785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7fd8070 0x5580c837cca0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7450005e30 tx=0x7f7450036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d3bd9f353edb9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f743dffb640 1 -- 172.21.15.3:0/27707785 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450056070 con 0x5580c7fd8070 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f743dffb640 1 -- 172.21.15.3:0/27707785 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7450036070 con 0x5580c7fd8070 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.052+0000 7f743dffb640 1 -- 172.21.15.3:0/27707785 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745002f660 con 0x5580c7fd8070 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.053+0000 7f74648db740 1 -- 172.21.15.3:0/27707785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7fd8070 msgr2=0x5580c837cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.053+0000 7f74648db740 1 --2- 172.21.15.3:0/27707785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7fd8070 0x5580c837cca0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7450005e30 tx=0x7f7450036000 comp rx=0 tx=0).stop 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.053+0000 7f74648db740 1 -- 172.21.15.3:0/27707785 shutdown_connections 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.053+0000 7f74648db740 1 --2- 172.21.15.3:0/27707785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7fd8070 0x5580c837cca0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.053+0000 7f74648db740 1 --2- 172.21.15.3:0/27707785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c837c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.053+0000 7f74648db740 1 --2- 172.21.15.3:0/27707785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7faf1b0 0x5580c8380a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.053+0000 7f74648db740 1 -- 172.21.15.3:0/27707785 >> 172.21.15.3:0/27707785 conn(0x5580c7fcc460 msgr2=0x5580c83802f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.053+0000 7f74648db740 1 -- 172.21.15.3:0/27707785 shutdown_connections 2024-09-21T06:19:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.053+0000 7f74648db740 1 -- 172.21.15.3:0/27707785 wait complete. 2024-09-21T06:19:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.053+0000 7f74648db740 1 -- start start 2024-09-21T06:19:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.054+0000 7f74648db740 1 Processor -- start 2024-09-21T06:19:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.054+0000 7f74648db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 0x5580c8393540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.054+0000 7f74648db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c838e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.054+0000 7f74648db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7fd8070 0x5580c838f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.054+0000 7f74648db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f74581b2c60 con 0x5580c7faf1b0 2024-09-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.054+0000 7f74648db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f74581b2590 con 0x5580c7fb7390 2024-09-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.054+0000 7f74648db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74581af7d0 con 0x5580c7fd8070 2024-09-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.054+0000 7f7457fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 0x5580c8393540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.054+0000 7f7457fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 0x5580c8393540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57392/0 (socket says 172.21.15.3:57392) 2024-09-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.054+0000 7f7457fff640 1 -- 172.21.15.3:0/2155484899 learned_addr learned my addr 172.21.15.3:0/2155484899 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.054+0000 7f74648db740 10 client.0 _ll_register_callbacks cb 0x7f744c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.054+0000 7f74577fe640 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c838e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.055+0000 7f745c904640 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7fd8070 0x5580c838f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.055+0000 7f7457fff640 1 -- 172.21.15.3:0/2155484899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7fd8070 msgr2=0x5580c838f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.055+0000 7f7457fff640 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7fd8070 0x5580c838f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.055+0000 7f7457fff640 1 -- 172.21.15.3:0/2155484899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 msgr2=0x5580c838e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.055+0000 7f7457fff640 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c838e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.055+0000 7f7457fff640 1 -- 172.21.15.3:0/2155484899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c838f7b0 con 0x5580c7faf1b0 2024-09-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.055+0000 7f745c904640 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7fd8070 0x5580c838f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.055+0000 7f7457fff640 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 0x5580c8393540 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f744800c6a0 tx=0x7f74480e7db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b106e921b54e4af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.055+0000 7f74377fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.055+0000 7f74377fe640 1 -- 172.21.15.3:0/2155484899 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448098030 con 0x5580c7faf1b0 2024-09-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.055+0000 7f74377fe640 1 -- 172.21.15.3:0/2155484899 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f74480e54c0 con 0x5580c7faf1b0 2024-09-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.055+0000 7f74648db740 1 -- 172.21.15.3:0/2155484899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7450036070 con 0x5580c7faf1b0 2024-09-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.056+0000 7f74377fe640 1 -- 172.21.15.3:0/2155484899 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f74480a90b0 con 0x5580c7faf1b0 2024-09-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.056+0000 7f74648db740 10 client.16839 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.056+0000 7f74648db740 1 -- 172.21.15.3:0/2155484899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5580c8407710 con 0x5580c7faf1b0 2024-09-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.056+0000 7f74377fe640 1 -- 172.21.15.3:0/2155484899 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7450036070 con 0x5580c7faf1b0 2024-09-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.056+0000 7f74377fe640 1 -- 172.21.15.3:0/2155484899 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f74480a9bb0 con 0x5580c7faf1b0 2024-09-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.056+0000 7f74648db740 10 client.16839 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.056+0000 7f74648db740 10 client.16839 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.056+0000 7f74648db740 20 client.16839 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.056+0000 7f74648db740 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5580c7f6bda0 0x5580c84099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.056+0000 7f74648db740 4 client.16839 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.056+0000 7f74648db740 1 -- 172.21.15.3:0/2155484899 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580c82f38a0 con 0x5580c7f6bda0 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.056+0000 7f74577fe640 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5580c7f6bda0 0x5580c84099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.057+0000 7f74377fe640 10 client.16839 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.057+0000 7f74577fe640 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5580c7f6bda0 0x5580c84099b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4b1182b128f1adc3 server_cookie=18e31816e6f916d4 in_seq=0 out_seq=0 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.057+0000 7f74377fe640 1 -- 172.21.15.3:0/2155484899 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580c82f38a0 con 0x5580c7f6bda0 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.057+0000 7f74377fe640 10 client.16839 handle_command_reply: tid=0 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.057+0000 7f74648db740 1 client.16839 shutdown 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.057+0000 7f74648db740 1 -- 172.21.15.3:0/2155484899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 msgr2=0x5580c8393540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.057+0000 7f74648db740 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 0x5580c8393540 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f744800c6a0 tx=0x7f74480e7db0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.057+0000 7f74648db740 1 -- 172.21.15.3:0/2155484899 shutdown_connections 2024-09-21T06:19:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.057+0000 7f74648db740 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5580c7f6bda0 0x5580c84099b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.057+0000 7f74648db740 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580c7fd8070 0x5580c838f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.057+0000 7f74377fe640 0 client.16839 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:19:33.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.057+0000 7f74648db740 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c7fb7390 0x5580c838e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.057+0000 7f74648db740 1 --2- 172.21.15.3:0/2155484899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5580c7faf1b0 0x5580c8393540 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.058+0000 7f74648db740 1 -- 172.21.15.3:0/2155484899 >> 172.21.15.3:0/2155484899 conn(0x5580c7fcc460 msgr2=0x5580c8380cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:33.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.058+0000 7f74648db740 1 -- 172.21.15.3:0/2155484899 shutdown_connections 2024-09-21T06:19:33.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.058+0000 7f74648db740 1 -- 172.21.15.3:0/2155484899 wait complete. 2024-09-21T06:19:33.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.058+0000 7f74648db740 20 client.16839 trim_cache size 0 max 16384 2024-09-21T06:19:33.064 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.058+0000 7f745e367640 1 -- 172.21.15.3:0/2237582131 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74400b6680 msgr2=0x7f74400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.058+0000 7f745e367640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74400b6680 0x7f74400b8b90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7450009a60 tx=0x7f74500095c0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.058+0000 7f745e367640 1 -- 172.21.15.3:0/2237582131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7458102a60 msgr2=0x7f7458199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.058+0000 7f745e367640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7458102a60 0x7f7458199080 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f744800e9f0 tx=0x7f744800edf0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.059+0000 7f745e367640 1 -- 172.21.15.3:0/2237582131 shutdown_connections 2024-09-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.059+0000 7f745e367640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74400b6680 0x7f74400b8b90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.059+0000 7f745e367640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74581083a0 0x7f74581995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.059+0000 7f745e367640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7458108cf0 0x7f7458199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.059+0000 7f745e367640 1 --2- 172.21.15.3:0/2237582131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7458102a60 0x7f7458199080 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.059+0000 7f745e367640 1 -- 172.21.15.3:0/2237582131 >> 172.21.15.3:0/2237582131 conn(0x7f7458079730 msgr2=0x7f7458104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.059+0000 7f745e367640 1 -- 172.21.15.3:0/2237582131 shutdown_connections 2024-09-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:33.059+0000 7f745e367640 1 -- 172.21.15.3:0/2237582131 wait complete. 2024-09-21T06:19:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:33 smithi003 ceph-mon[31227]: pgmap v461: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 377 op/s 2024-09-21T06:19:33.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:33 smithi137 ceph-mon[34464]: pgmap v461: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 377 op/s 2024-09-21T06:19:33.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:33 smithi204 ceph-mon[34659]: pgmap v461: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 377 op/s 2024-09-21T06:19:33.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 447010 12.44 MB/sec execute 66 sec latency 13.236 ms 2024-09-21T06:19:34.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:19:34.440 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:34.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 448455 12.35 MB/sec execute 67 sec latency 37.650 ms 2024-09-21T06:19:35.132 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:35 smithi003 ceph-mon[31227]: pgmap v462: 97 pgs: 97 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 349 op/s 2024-09-21T06:19:35.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.130+0000 7f591d3a6640 1 -- 172.21.15.3:0/321726742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5918108cf0 msgr2=0x7f591810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:35.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.130+0000 7f591d3a6640 1 --2- 172.21.15.3:0/321726742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5918108cf0 0x7f591810f520 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5910009ca0 tx=0x7f591002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.130+0000 7f591d3a6640 1 -- 172.21.15.3:0/321726742 shutdown_connections 2024-09-21T06:19:35.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.130+0000 7f591d3a6640 1 --2- 172.21.15.3:0/321726742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5918108cf0 0x7f591810f520 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.130+0000 7f591d3a6640 1 --2- 172.21.15.3:0/321726742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59181083a0 0x7f59181087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.130+0000 7f591d3a6640 1 --2- 172.21.15.3:0/321726742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918102a60 0x7f5918102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.130+0000 7f591d3a6640 1 -- 172.21.15.3:0/321726742 >> 172.21.15.3:0/321726742 conn(0x7f5918079730 msgr2=0x7f5918079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:35.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.130+0000 7f591d3a6640 1 -- 172.21.15.3:0/321726742 shutdown_connections 2024-09-21T06:19:35.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.130+0000 7f591d3a6640 1 -- 172.21.15.3:0/321726742 wait complete. 2024-09-21T06:19:35.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f591d3a6640 1 Processor -- start 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f591d3a6640 1 -- start start 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f591d3a6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918102a60 0x7f59180ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f591d3a6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59181083a0 0x7f59180ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f591d3a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5918108cf0 0x7f59181000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f591d3a6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f59181129b0 con 0x7f5918102a60 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f591d3a6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5918112830 con 0x7f5918108cf0 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f591d3a6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5918112b30 con 0x7f59181083a0 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f5916ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918102a60 0x7f59180ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f5916ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918102a60 0x7f59180ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57410/0 (socket says 172.21.15.3:57410) 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f5916ffd640 1 -- 172.21.15.3:0/2156482630 learned_addr learned my addr 172.21.15.3:0/2156482630 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f59177fe640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5918108cf0 0x7f59181000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f590ffff640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59181083a0 0x7f59180ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f5916ffd640 1 -- 172.21.15.3:0/2156482630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59181083a0 msgr2=0x7f59180ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:35.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f5916ffd640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59181083a0 0x7f59180ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f5916ffd640 1 -- 172.21.15.3:0/2156482630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5918108cf0 msgr2=0x7f59181000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f5916ffd640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5918108cf0 0x7f59181000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.131+0000 7f5916ffd640 1 -- 172.21.15.3:0/2156482630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59181a8710 con 0x7f5918102a60 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.132+0000 7f5916ffd640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918102a60 0x7f59180ff670 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f590800e9f0 tx=0x7f590800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.132+0000 7f5914ff9640 1 -- 172.21.15.3:0/2156482630 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590800cd80 con 0x7f5918102a60 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.132+0000 7f591d3a6640 1 -- 172.21.15.3:0/2156482630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59181a8a00 con 0x7f5918102a60 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.132+0000 7f5914ff9640 1 -- 172.21.15.3:0/2156482630 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59080108e0 con 0x7f5918102a60 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.132+0000 7f5914ff9640 1 -- 172.21.15.3:0/2156482630 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908018980 con 0x7f5918102a60 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.132+0000 7f591d3a6640 1 -- 172.21.15.3:0/2156482630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59181a8f40 con 0x7f5918102a60 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.134+0000 7f592391a740 1 Processor -- start 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f592391a740 1 -- start start 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f592391a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb716070 0x55e4cbabd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f592391a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbabd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f592391a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbabdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f592391a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e4cba33ea0 con 0x55e4cb716070 2024-09-21T06:19:35.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f592391a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e4cba232d0 con 0x55e4cb6f5390 2024-09-21T06:19:35.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f592391a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e4cba270e0 con 0x55e4cb6ed1b0 2024-09-21T06:19:35.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f59177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb716070 0x55e4cbabd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f59177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb716070 0x55e4cbabd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57424/0 (socket says 172.21.15.3:57424) 2024-09-21T06:19:35.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f59177fe640 1 -- 172.21.15.3:0/2596270378 learned_addr learned my addr 172.21.15.3:0/2596270378 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:35.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f590ffff640 1 --2- 172.21.15.3:0/2596270378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbabdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f590ffff640 1 -- 172.21.15.3:0/2596270378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb6ed1b0 msgr2=0x55e4cbabd800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:19:35.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f590ffff640 1 --2- 172.21.15.3:0/2596270378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbabd800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f590ffff640 1 -- 172.21.15.3:0/2596270378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb716070 msgr2=0x55e4cbabd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:35.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f590ffff640 1 --2- 172.21.15.3:0/2596270378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb716070 0x55e4cbabd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.135+0000 7f590ffff640 1 -- 172.21.15.3:0/2596270378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4cb781650 con 0x55e4cb6f5390 2024-09-21T06:19:35.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.136+0000 7f59177fe640 1 --2- 172.21.15.3:0/2596270378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb716070 0x55e4cbabd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:35.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.136+0000 7f5914ff9640 1 -- 172.21.15.3:0/2156482630 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f5908010430 con 0x7f5918102a60 2024-09-21T06:19:35.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.136+0000 7f590ffff640 1 --2- 172.21.15.3:0/2596270378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbabdd40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f591806aaa0 tx=0x7f5900011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71fb78b4a176fed6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:35.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.136+0000 7f590d7fa640 1 -- 172.21.15.3:0/2596270378 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900004050 con 0x55e4cb6f5390 2024-09-21T06:19:35.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.136+0000 7f590d7fa640 1 -- 172.21.15.3:0/2596270378 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59000054d0 con 0x55e4cb6f5390 2024-09-21T06:19:35.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.136+0000 7f590d7fa640 1 -- 172.21.15.3:0/2596270378 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900004bc0 con 0x55e4cb6f5390 2024-09-21T06:19:35.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.137+0000 7f5914ff9640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f59040b6680 0x7f59040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.137+0000 7f59177fe640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f59040b6680 0x7f59040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.137+0000 7f5914ff9640 1 -- 172.21.15.3:0/2156482630 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f59080e7a10 con 0x7f5918102a60 2024-09-21T06:19:35.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.137+0000 7f592391a740 1 -- 172.21.15.3:0/2596270378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 msgr2=0x55e4cbabdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:35.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.137+0000 7f592391a740 1 --2- 172.21.15.3:0/2596270378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbabdd40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f591806aaa0 tx=0x7f5900011810 comp rx=0 tx=0).stop 2024-09-21T06:19:35.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.137+0000 7f592391a740 1 -- 172.21.15.3:0/2596270378 shutdown_connections 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.137+0000 7f592391a740 1 --2- 172.21.15.3:0/2596270378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbabd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.137+0000 7f592391a740 1 --2- 172.21.15.3:0/2596270378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbabdd40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.137+0000 7f592391a740 1 --2- 172.21.15.3:0/2596270378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb716070 0x55e4cbabd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.137+0000 7f592391a740 1 -- 172.21.15.3:0/2596270378 >> 172.21.15.3:0/2596270378 conn(0x55e4cb70a460 msgr2=0x55e4cbabb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.137+0000 7f592391a740 1 -- 172.21.15.3:0/2596270378 shutdown_connections 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.137+0000 7f592391a740 1 -- 172.21.15.3:0/2596270378 wait complete. 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.137+0000 7f59177fe640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f59040b6680 0x7f59040b8b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f591002aeb0 tx=0x7f5910005d80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f592391a740 1 -- start start 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f592391a740 1 Processor -- start 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f592391a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbac8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f592391a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb6f5390 0x55e4cbac8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f592391a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb716070 0x55e4cbad4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f592391a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e4cb884700 con 0x55e4cb716070 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f592391a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e4cb9be160 con 0x55e4cb6ed1b0 2024-09-21T06:19:35.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f592391a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e4cbacbaa0 con 0x55e4cb6f5390 2024-09-21T06:19:35.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f592391a740 10 client.0 _ll_register_callbacks cb 0x55e4cb9d79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:35.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f59177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb716070 0x55e4cbad4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f5916ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbac8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f5916ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbac8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:60142/0 (socket says 172.21.15.3:60142) 2024-09-21T06:19:35.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f59177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb716070 0x55e4cbad4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57434/0 (socket says 172.21.15.3:57434) 2024-09-21T06:19:35.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f5916ffd640 1 -- 172.21.15.3:0/4215295791 learned_addr learned my addr 172.21.15.3:0/4215295791 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:35.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.138+0000 7f590ffff640 1 --2- 172.21.15.3:0/4215295791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb6f5390 0x55e4cbac8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f5916ffd640 1 -- 172.21.15.3:0/4215295791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb6f5390 msgr2=0x55e4cbac8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:35.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f5916ffd640 1 --2- 172.21.15.3:0/4215295791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb6f5390 0x55e4cbac8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f5916ffd640 1 -- 172.21.15.3:0/4215295791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb716070 msgr2=0x55e4cbad4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:35.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f5916ffd640 1 --2- 172.21.15.3:0/4215295791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb716070 0x55e4cbad4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f5916ffd640 1 -- 172.21.15.3:0/4215295791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4cb9ebc90 con 0x55e4cb6ed1b0 2024-09-21T06:19:35.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f5916ffd640 1 --2- 172.21.15.3:0/4215295791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbac8a20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f59080e5250 tx=0x7f59080ad2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e538f5e26853dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:35.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f58fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:19:35.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f58fe7fc640 1 -- 172.21.15.3:0/4215295791 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59080aedd0 con 0x55e4cb6ed1b0 2024-09-21T06:19:35.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f58fe7fc640 1 -- 172.21.15.3:0/4215295791 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5908018b20 con 0x55e4cb6ed1b0 2024-09-21T06:19:35.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f592391a740 1 -- 172.21.15.3:0/4215295791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59000054d0 con 0x55e4cb6ed1b0 2024-09-21T06:19:35.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f58fe7fc640 1 -- 172.21.15.3:0/4215295791 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5908010720 con 0x55e4cb6ed1b0 2024-09-21T06:19:35.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f592391a740 10 client.26426 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:35.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f592391a740 1 -- 172.21.15.3:0/4215295791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e4cbad7850 con 0x55e4cb6ed1b0 2024-09-21T06:19:35.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.139+0000 7f58fe7fc640 1 -- 172.21.15.3:0/4215295791 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59000054d0 con 0x55e4cb6ed1b0 2024-09-21T06:19:35.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.140+0000 7f58fe7fc640 1 -- 172.21.15.3:0/4215295791 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f59080ac620 con 0x55e4cb6ed1b0 2024-09-21T06:19:35.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.140+0000 7f592391a740 10 client.26426 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:35.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.140+0000 7f592391a740 10 client.26426 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:19:35.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.140+0000 7f592391a740 20 client.26426 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.140+0000 7f592391a740 1 --2- 172.21.15.3:0/4215295791 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55e4cb6d49f0 0x55e4cbadad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.140+0000 7f592391a740 4 client.26426 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.140+0000 7f592391a740 1 -- 172.21.15.3:0/4215295791 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4cba35df0 con 0x55e4cb6d49f0 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.140+0000 7f590ffff640 1 --2- 172.21.15.3:0/4215295791 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55e4cb6d49f0 0x55e4cbadad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.140+0000 7f58fe7fc640 10 client.26426 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.140+0000 7f590ffff640 1 --2- 172.21.15.3:0/4215295791 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55e4cb6d49f0 0x55e4cbadad20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=628141f91e212a43 server_cookie=ccd91a184f1f7732 in_seq=0 out_seq=0 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.141+0000 7f58fe7fc640 1 -- 172.21.15.3:0/4215295791 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e4cba35df0 con 0x55e4cb6d49f0 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.141+0000 7f58fe7fc640 10 client.26426 handle_command_reply: tid=0 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.141+0000 7f592391a740 1 client.26426 shutdown 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.142+0000 7f592391a740 1 -- 172.21.15.3:0/4215295791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6ed1b0 msgr2=0x55e4cbac8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.142+0000 7f592391a740 1 --2- 172.21.15.3:0/4215295791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbac8a20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f59080e5250 tx=0x7f59080ad2a0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.142+0000 7f592391a740 1 -- 172.21.15.3:0/4215295791 shutdown_connections 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.142+0000 7f592391a740 1 --2- 172.21.15.3:0/4215295791 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55e4cb6d49f0 0x55e4cbadad20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.142+0000 7f592391a740 1 --2- 172.21.15.3:0/4215295791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb6f5390 0x55e4cbac8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.142+0000 7f592391a740 1 --2- 172.21.15.3:0/4215295791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb716070 0x55e4cbad4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.142+0000 7f58fe7fc640 0 client.26426 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:19:35.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.142+0000 7f592391a740 1 --2- 172.21.15.3:0/4215295791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbac8a20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.142+0000 7f592391a740 1 -- 172.21.15.3:0/4215295791 >> 172.21.15.3:0/4215295791 conn(0x55e4cb70a460 msgr2=0x55e4cbac04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:35.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.142+0000 7f592391a740 1 -- 172.21.15.3:0/4215295791 shutdown_connections 2024-09-21T06:19:35.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.142+0000 7f592391a740 1 -- 172.21.15.3:0/4215295791 wait complete. 2024-09-21T06:19:35.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.142+0000 7f592391a740 20 client.26426 trim_cache size 0 max 16384 2024-09-21T06:19:35.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.157+0000 7f592391a740 1 Processor -- start 2024-09-21T06:19:35.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.158+0000 7f592391a740 1 -- start start 2024-09-21T06:19:35.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.158+0000 7f592391a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbac0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.158+0000 7f592391a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbad4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.158+0000 7f592391a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb716070 0x55e4cbabaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.158+0000 7f592391a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b1950 con 0x55e4cb6ed1b0 2024-09-21T06:19:35.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.158+0000 7f592391a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b3490 con 0x55e4cb6f5390 2024-09-21T06:19:35.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.158+0000 7f592391a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b2020 con 0x55e4cb716070 2024-09-21T06:19:35.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.158+0000 7f5916ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbac0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.158+0000 7f5916ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbac0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57450/0 (socket says 172.21.15.3:57450) 2024-09-21T06:19:35.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.158+0000 7f5916ffd640 1 -- 172.21.15.3:0/2183314819 learned_addr learned my addr 172.21.15.3:0/2183314819 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:35.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.158+0000 7f590ffff640 1 --2- 172.21.15.3:0/2183314819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbad4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.158+0000 7f59177fe640 1 --2- 172.21.15.3:0/2183314819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb716070 0x55e4cbabaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.159+0000 7f590ffff640 1 -- 172.21.15.3:0/2183314819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb716070 msgr2=0x55e4cbabaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:35.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.159+0000 7f590ffff640 1 --2- 172.21.15.3:0/2183314819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb716070 0x55e4cbabaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.159+0000 7f590ffff640 1 -- 172.21.15.3:0/2183314819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 msgr2=0x55e4cbac0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:35.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.159+0000 7f590ffff640 1 --2- 172.21.15.3:0/2183314819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbac0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.159+0000 7f590ffff640 1 -- 172.21.15.3:0/2183314819 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4cbabf5b0 con 0x55e4cb6f5390 2024-09-21T06:19:35.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.159+0000 7f5916ffd640 1 --2- 172.21.15.3:0/2183314819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbac0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:35.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.159+0000 7f590ffff640 1 --2- 172.21.15.3:0/2183314819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbad4c90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5900004650 tx=0x7f59000096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aa24454dbe0f532 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:35.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.159+0000 7f590cff9640 1 -- 172.21.15.3:0/2183314819 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900004020 con 0x55e4cb6f5390 2024-09-21T06:19:35.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.159+0000 7f590cff9640 1 -- 172.21.15.3:0/2183314819 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59000041c0 con 0x55e4cb6f5390 2024-09-21T06:19:35.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.159+0000 7f590cff9640 1 -- 172.21.15.3:0/2183314819 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590002e690 con 0x55e4cb6f5390 2024-09-21T06:19:35.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.160+0000 7f592391a740 1 -- 172.21.15.3:0/2183314819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 msgr2=0x55e4cbad4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:35.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.160+0000 7f592391a740 1 --2- 172.21.15.3:0/2183314819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbad4c90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5900004650 tx=0x7f59000096a0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.160+0000 7f592391a740 1 -- 172.21.15.3:0/2183314819 shutdown_connections 2024-09-21T06:19:35.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.160+0000 7f592391a740 1 --2- 172.21.15.3:0/2183314819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb716070 0x55e4cbabaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.160+0000 7f592391a740 1 --2- 172.21.15.3:0/2183314819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbad4c90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.160+0000 7f592391a740 1 --2- 172.21.15.3:0/2183314819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbac0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.160+0000 7f592391a740 1 -- 172.21.15.3:0/2183314819 >> 172.21.15.3:0/2183314819 conn(0x55e4cb70a460 msgr2=0x55e4cbac03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.160+0000 7f592391a740 1 -- 172.21.15.3:0/2183314819 shutdown_connections 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.160+0000 7f592391a740 1 -- 172.21.15.3:0/2183314819 wait complete. 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f592391a740 1 -- start start 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f592391a740 1 Processor -- start 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f592391a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbacd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f592391a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbacd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f592391a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb716070 0x55e4cbb44690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f592391a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b2dc0 con 0x55e4cb6ed1b0 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f592391a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b26f0 con 0x55e4cb6f5390 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f592391a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59181af930 con 0x55e4cb716070 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f592391a740 10 client.0 _ll_register_callbacks cb 0x7f5908009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:35.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f5916ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbacd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f590ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbacd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f5916ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbacd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57466/0 (socket says 172.21.15.3:57466) 2024-09-21T06:19:35.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f5916ffd640 1 -- 172.21.15.3:0/1978645401 learned_addr learned my addr 172.21.15.3:0/1978645401 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:35.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.161+0000 7f59177fe640 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb716070 0x55e4cbb44690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f590ffff640 1 -- 172.21.15.3:0/1978645401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb716070 msgr2=0x55e4cbb44690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:35.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f590ffff640 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb716070 0x55e4cbb44690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f590ffff640 1 -- 172.21.15.3:0/1978645401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 msgr2=0x55e4cbacd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:35.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f590ffff640 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbacd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f590ffff640 1 -- 172.21.15.3:0/1978645401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4cbb44c00 con 0x55e4cb6f5390 2024-09-21T06:19:35.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f5916ffd640 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbacd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:35.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f59177fe640 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb716070 0x55e4cbb44690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:19:35.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f590ffff640 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbacd960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5900009730 tx=0x7f5900005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c05e1c27439f6e15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:35.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f58fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:19:35.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f58fe7fc640 1 -- 172.21.15.3:0/1978645401 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590003b020 con 0x55e4cb6f5390 2024-09-21T06:19:35.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f58fe7fc640 1 -- 172.21.15.3:0/1978645401 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f59000037f0 con 0x55e4cb6f5390 2024-09-21T06:19:35.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f592391a740 1 -- 172.21.15.3:0/1978645401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59000041c0 con 0x55e4cb6f5390 2024-09-21T06:19:35.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f58fe7fc640 1 -- 172.21.15.3:0/1978645401 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5900046050 con 0x55e4cb6f5390 2024-09-21T06:19:35.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f592391a740 10 client.26432 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:35.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.162+0000 7f592391a740 1 -- 172.21.15.3:0/1978645401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e4cbb45e80 con 0x55e4cb6f5390 2024-09-21T06:19:35.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.163+0000 7f58fe7fc640 1 -- 172.21.15.3:0/1978645401 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59000041c0 con 0x55e4cb6f5390 2024-09-21T06:19:35.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.163+0000 7f58fe7fc640 1 -- 172.21.15.3:0/1978645401 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5900036070 con 0x55e4cb6f5390 2024-09-21T06:19:35.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.163+0000 7f592391a740 10 client.26432 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:35.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.163+0000 7f592391a740 10 client.26432 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:19:35.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.163+0000 7f592391a740 20 client.26432 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:35.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.163+0000 7f592391a740 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55e4cb6a9da0 0x55e4cbb48350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:35.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.163+0000 7f592391a740 4 client.26432 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:19:35.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.163+0000 7f592391a740 1 -- 172.21.15.3:0/1978645401 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4cba318a0 con 0x55e4cb6a9da0 2024-09-21T06:19:35.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.163+0000 7f5916ffd640 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55e4cb6a9da0 0x55e4cbb48350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:35.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f58fe7fc640 10 client.26432 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:19:35.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f5916ffd640 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55e4cb6a9da0 0x55e4cbb48350 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d7d61ee775b8d34d server_cookie=57723b983b85a3e1 in_seq=0 out_seq=0 2024-09-21T06:19:35.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f58fe7fc640 1 -- 172.21.15.3:0/1978645401 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4cba318a0 con 0x55e4cb6a9da0 2024-09-21T06:19:35.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f58fe7fc640 10 client.26432 handle_command_reply: tid=0 2024-09-21T06:19:35.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f592391a740 1 client.26432 shutdown 2024-09-21T06:19:35.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f592391a740 1 -- 172.21.15.3:0/1978645401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 msgr2=0x55e4cbacd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:35.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f592391a740 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbacd960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5900009730 tx=0x7f5900005bb0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f592391a740 1 -- 172.21.15.3:0/1978645401 shutdown_connections 2024-09-21T06:19:35.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f592391a740 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55e4cb6a9da0 0x55e4cbb48350 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f592391a740 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4cb716070 0x55e4cbb44690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f592391a740 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cb6f5390 0x55e4cbacd960 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f58fe7fc640 0 client.26432 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:19:35.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f592391a740 1 --2- 172.21.15.3:0/1978645401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4cb6ed1b0 0x55e4cbacd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f592391a740 1 -- 172.21.15.3:0/1978645401 >> 172.21.15.3:0/1978645401 conn(0x55e4cb70a460 msgr2=0x55e4cbac9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:35.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f592391a740 1 -- 172.21.15.3:0/1978645401 shutdown_connections 2024-09-21T06:19:35.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f592391a740 1 -- 172.21.15.3:0/1978645401 wait complete. 2024-09-21T06:19:35.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.164+0000 7f592391a740 20 client.26432 trim_cache size 0 max 16384 2024-09-21T06:19:35.172 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:19:35.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.165+0000 7f591d3a6640 1 -- 172.21.15.3:0/2156482630 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f59040b6680 msgr2=0x7f59040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:35.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.165+0000 7f591d3a6640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f59040b6680 0x7f59040b8b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f591002aeb0 tx=0x7f5910005d80 comp rx=0 tx=0).stop 2024-09-21T06:19:35.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.165+0000 7f591d3a6640 1 -- 172.21.15.3:0/2156482630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918102a60 msgr2=0x7f59180ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:35.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.165+0000 7f591d3a6640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918102a60 0x7f59180ff670 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f590800e9f0 tx=0x7f590800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.165+0000 7f591d3a6640 1 -- 172.21.15.3:0/2156482630 shutdown_connections 2024-09-21T06:19:35.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.165+0000 7f591d3a6640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f59040b6680 0x7f59040b8b90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.165+0000 7f591d3a6640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59181083a0 0x7f59180ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.165+0000 7f591d3a6640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5918108cf0 0x7f59181000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.165+0000 7f591d3a6640 1 --2- 172.21.15.3:0/2156482630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918102a60 0x7f59180ff670 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:35.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.165+0000 7f591d3a6640 1 -- 172.21.15.3:0/2156482630 >> 172.21.15.3:0/2156482630 conn(0x7f5918079730 msgr2=0x7f591806b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:35.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.165+0000 7f591d3a6640 1 -- 172.21.15.3:0/2156482630 shutdown_connections 2024-09-21T06:19:35.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:35.165+0000 7f591d3a6640 1 -- 172.21.15.3:0/2156482630 wait complete. 2024-09-21T06:19:35.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:35 smithi137 ceph-mon[34464]: pgmap v462: 97 pgs: 97 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 349 op/s 2024-09-21T06:19:35.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:35 smithi204 ceph-mon[34659]: pgmap v462: 97 pgs: 97 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 349 op/s 2024-09-21T06:19:35.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 450887 12.37 MB/sec execute 68 sec latency 15.838 ms 2024-09-21T06:19:36.068 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:19:36.341 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.879+0000 7f46227eb640 1 -- 172.21.15.3:0/656739018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c102fa0 msgr2=0x7f461c10fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.879+0000 7f46227eb640 1 --2- 172.21.15.3:0/656739018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c102fa0 0x7f461c10fcc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4610009900 tx=0x7f461002a480 comp rx=0 tx=0).stop 2024-09-21T06:19:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.879+0000 7f46227eb640 1 -- 172.21.15.3:0/656739018 shutdown_connections 2024-09-21T06:19:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.879+0000 7f46227eb640 1 --2- 172.21.15.3:0/656739018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f461c110200 0x7f461c112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.879+0000 7f46227eb640 1 --2- 172.21.15.3:0/656739018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c102fa0 0x7f461c10fcc0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.879+0000 7f46227eb640 1 --2- 172.21.15.3:0/656739018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f461c102650 0x7f461c102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.879+0000 7f46227eb640 1 -- 172.21.15.3:0/656739018 >> 172.21.15.3:0/656739018 conn(0x7f461c0797b0 msgr2=0x7f461c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.879+0000 7f46227eb640 1 -- 172.21.15.3:0/656739018 shutdown_connections 2024-09-21T06:19:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.879+0000 7f46227eb640 1 -- 172.21.15.3:0/656739018 wait complete. 2024-09-21T06:19:36.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.880+0000 7f46227eb640 1 Processor -- start 2024-09-21T06:19:36.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.880+0000 7f46227eb640 1 -- start start 2024-09-21T06:19:36.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.880+0000 7f46227eb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f461c102650 0x7f461c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.880+0000 7f46227eb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f461c102fa0 0x7f461c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.880+0000 7f46227eb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c110200 0x7f461c1a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.880+0000 7f46227eb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f461c115ca0 con 0x7f461c110200 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.880+0000 7f46227eb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f461c115b20 con 0x7f461c102fa0 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.880+0000 7f46227eb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f461c115e20 con 0x7f461c102650 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.880+0000 7f4620d88640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c110200 0x7f461c1a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.880+0000 7f461bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f461c102650 0x7f461c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.880+0000 7f461b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f461c102fa0 0x7f461c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.880+0000 7f4620d88640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c110200 0x7f461c1a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36252/0 (socket says 172.21.15.3:36252) 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.881+0000 7f4620d88640 1 -- 172.21.15.3:0/2695320478 learned_addr learned my addr 172.21.15.3:0/2695320478 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.880+0000 7f461bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f461c102650 0x7f461c19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:39720/0 (socket says 172.21.15.3:39720) 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.881+0000 7f4620d88640 1 -- 172.21.15.3:0/2695320478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f461c102650 msgr2=0x7f461c19c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.881+0000 7f4620d88640 1 --2- 172.21.15.3:0/2695320478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f461c102650 0x7f461c19c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.881+0000 7f4620d88640 1 -- 172.21.15.3:0/2695320478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f461c102fa0 msgr2=0x7f461c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.881+0000 7f4620d88640 1 --2- 172.21.15.3:0/2695320478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f461c102fa0 0x7f461c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.881+0000 7f4620d88640 1 -- 172.21.15.3:0/2695320478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461c1a25c0 con 0x7f461c110200 2024-09-21T06:19:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.881+0000 7f461bfff640 1 --2- 172.21.15.3:0/2695320478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f461c102650 0x7f461c19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.881+0000 7f4620d88640 1 --2- 172.21.15.3:0/2695320478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c110200 0x7f461c1a1e90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f461400eb10 tx=0x7f461400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.881+0000 7f46197fa640 1 -- 172.21.15.3:0/2695320478 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614004b20 con 0x7f461c110200 2024-09-21T06:19:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.881+0000 7f46197fa640 1 -- 172.21.15.3:0/2695320478 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4614004cc0 con 0x7f461c110200 2024-09-21T06:19:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.881+0000 7f46227eb640 1 -- 172.21.15.3:0/2695320478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461c1a28b0 con 0x7f461c110200 2024-09-21T06:19:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.881+0000 7f46197fa640 1 -- 172.21.15.3:0/2695320478 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614010670 con 0x7f461c110200 2024-09-21T06:19:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.881+0000 7f46227eb640 1 -- 172.21.15.3:0/2695320478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f461c1a9010 con 0x7f461c110200 2024-09-21T06:19:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.884+0000 7f4628d5f740 1 Processor -- start 2024-09-21T06:19:36.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.884+0000 7f46197fa640 1 -- 172.21.15.3:0/2695320478 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f4614005020 con 0x7f461c110200 2024-09-21T06:19:36.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f4628d5f740 1 -- start start 2024-09-21T06:19:36.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f4628d5f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ade070 0x55d070e856a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f4628d5f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070ab51b0 0x55d070e85be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f4628d5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070abd390 0x55d070e86120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f4628d5f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d070dfbea0 con 0x55d070ab51b0 2024-09-21T06:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f4628d5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d070deb2d0 con 0x55d070abd390 2024-09-21T06:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f4628d5f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d070def0e0 con 0x55d070ade070 2024-09-21T06:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f461b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ade070 0x55d070e856a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f461bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070ab51b0 0x55d070e85be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f461b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ade070 0x55d070e856a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:39728/0 (socket says 172.21.15.3:39728) 2024-09-21T06:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f461b7fe640 1 -- 172.21.15.3:0/501195718 learned_addr learned my addr 172.21.15.3:0/501195718 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f461b7fe640 1 --2- 172.21.15.3:0/501195718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070abd390 0x55d070e86120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f461b7fe640 1 -- 172.21.15.3:0/501195718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070abd390 msgr2=0x55d070e86120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f461b7fe640 1 --2- 172.21.15.3:0/501195718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070abd390 0x55d070e86120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f461b7fe640 1 -- 172.21.15.3:0/501195718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070ab51b0 msgr2=0x55d070e85be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f461b7fe640 1 --2- 172.21.15.3:0/501195718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070ab51b0 0x55d070e85be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f461b7fe640 1 -- 172.21.15.3:0/501195718 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d070b49650 con 0x55d070ade070 2024-09-21T06:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.885+0000 7f46197fa640 1 --2- 172.21.15.3:0/2695320478 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f46040b6680 0x7f46040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.886+0000 7f461bfff640 1 --2- 172.21.15.3:0/501195718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070ab51b0 0x55d070e85be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.886+0000 7f461b7fe640 1 --2- 172.21.15.3:0/501195718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ade070 0x55d070e856a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4610009900 tx=0x7f4610009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc4ba307424d1711 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.886+0000 7f46197fa640 1 -- 172.21.15.3:0/2695320478 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f46140e7c80 con 0x7f461c110200 2024-09-21T06:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.886+0000 7f461bfff640 1 --2- 172.21.15.3:0/2695320478 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f46040b6680 0x7f46040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.886+0000 7f46027fc640 1 -- 172.21.15.3:0/501195718 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610035030 con 0x55d070ade070 2024-09-21T06:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.886+0000 7f46027fc640 1 -- 172.21.15.3:0/501195718 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4610004030 con 0x55d070ade070 2024-09-21T06:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.886+0000 7f46027fc640 1 -- 172.21.15.3:0/501195718 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610003970 con 0x55d070ade070 2024-09-21T06:19:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.887+0000 7f4628d5f740 1 -- 172.21.15.3:0/501195718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ade070 msgr2=0x55d070e856a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.887+0000 7f4628d5f740 1 --2- 172.21.15.3:0/501195718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ade070 0x55d070e856a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4610009900 tx=0x7f4610009560 comp rx=0 tx=0).stop 2024-09-21T06:19:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.887+0000 7f461bfff640 1 --2- 172.21.15.3:0/2695320478 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f46040b6680 0x7f46040b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f460c009fd0 tx=0x7f460c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.887+0000 7f4628d5f740 1 -- 172.21.15.3:0/501195718 shutdown_connections 2024-09-21T06:19:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.887+0000 7f4628d5f740 1 --2- 172.21.15.3:0/501195718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070abd390 0x55d070e86120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.887+0000 7f4628d5f740 1 --2- 172.21.15.3:0/501195718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070ab51b0 0x55d070e85be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.887+0000 7f4628d5f740 1 --2- 172.21.15.3:0/501195718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ade070 0x55d070e856a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.887+0000 7f4628d5f740 1 -- 172.21.15.3:0/501195718 >> 172.21.15.3:0/501195718 conn(0x55d070ad2460 msgr2=0x55d070e83590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:36.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.887+0000 7f4628d5f740 1 -- 172.21.15.3:0/501195718 shutdown_connections 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.887+0000 7f4628d5f740 1 -- 172.21.15.3:0/501195718 wait complete. 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.887+0000 7f4628d5f740 1 -- start start 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.888+0000 7f4628d5f740 1 Processor -- start 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.888+0000 7f4628d5f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ab51b0 0x55d070e90dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.888+0000 7f4628d5f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070e91300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.888+0000 7f4628d5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ade070 0x55d070ea59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.888+0000 7f4628d5f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d070c4c700 con 0x55d070abd390 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.888+0000 7f4628d5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d070d86160 con 0x55d070ade070 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.888+0000 7f4628d5f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d070e9c630 con 0x55d070ab51b0 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.888+0000 7f4628d5f740 10 client.0 _ll_register_callbacks cb 0x55d070d9f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.888+0000 7f461b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070e91300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.888+0000 7f461bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ab51b0 0x55d070e90dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.888+0000 7f461b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070e91300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36266/0 (socket says 172.21.15.3:36266) 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.888+0000 7f461b7fe640 1 -- 172.21.15.3:0/438375302 learned_addr learned my addr 172.21.15.3:0/438375302 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.889+0000 7f4620d88640 1 --2- 172.21.15.3:0/438375302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ade070 0x55d070ea59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.889+0000 7f461bfff640 1 -- 172.21.15.3:0/438375302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ade070 msgr2=0x55d070ea59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.889+0000 7f461bfff640 1 --2- 172.21.15.3:0/438375302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ade070 0x55d070ea59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.889+0000 7f461bfff640 1 -- 172.21.15.3:0/438375302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 msgr2=0x55d070e91300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.889+0000 7f461bfff640 1 --2- 172.21.15.3:0/438375302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070e91300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.889+0000 7f461bfff640 1 -- 172.21.15.3:0/438375302 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d070db3c90 con 0x55d070ab51b0 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.889+0000 7f461b7fe640 1 --2- 172.21.15.3:0/438375302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070e91300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.889+0000 7f461bfff640 1 --2- 172.21.15.3:0/438375302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ab51b0 0x55d070e90dc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f460c00f730 tx=0x7f460c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d3d993ed93be5a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.889+0000 7f45fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.889+0000 7f45fb7fe640 1 -- 172.21.15.3:0/438375302 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c004320 con 0x55d070ab51b0 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.889+0000 7f45fb7fe640 1 -- 172.21.15.3:0/438375302 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f460c0035f0 con 0x55d070ab51b0 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.889+0000 7f4628d5f740 1 -- 172.21.15.3:0/438375302 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4610004030 con 0x55d070ab51b0 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.890+0000 7f45fb7fe640 1 -- 172.21.15.3:0/438375302 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f460c0044c0 con 0x55d070ab51b0 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.890+0000 7f4628d5f740 10 client.26398 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.890+0000 7f4628d5f740 1 -- 172.21.15.3:0/438375302 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d070ea8460 con 0x55d070ab51b0 2024-09-21T06:19:36.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.890+0000 7f45fb7fe640 1 -- 172.21.15.3:0/438375302 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4610004030 con 0x55d070ab51b0 2024-09-21T06:19:36.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.890+0000 7f45fb7fe640 1 -- 172.21.15.3:0/438375302 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f460c027020 con 0x55d070ab51b0 2024-09-21T06:19:36.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.890+0000 7f4628d5f740 10 client.26398 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:36.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.890+0000 7f4628d5f740 10 client.26398 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:19:36.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.890+0000 7f4628d5f740 20 client.26398 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:36.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.890+0000 7f4628d5f740 1 --2- 172.21.15.3:0/438375302 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d070a9c9f0 0x55d070eaba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.890+0000 7f4628d5f740 4 client.26398 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:36.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.890+0000 7f4628d5f740 1 -- 172.21.15.3:0/438375302 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d070dfddf0 con 0x55d070a9c9f0 2024-09-21T06:19:36.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.890+0000 7f461b7fe640 1 --2- 172.21.15.3:0/438375302 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d070a9c9f0 0x55d070eaba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.891+0000 7f461b7fe640 1 --2- 172.21.15.3:0/438375302 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d070a9c9f0 0x55d070eaba00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92f315c1de68f25b server_cookie=839a9c65a8bb7860 in_seq=0 out_seq=0 2024-09-21T06:19:36.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.891+0000 7f45fb7fe640 10 client.26398 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:19:36.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.891+0000 7f45fb7fe640 1 -- 172.21.15.3:0/438375302 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d070dfddf0 con 0x55d070a9c9f0 2024-09-21T06:19:36.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.891+0000 7f45fb7fe640 10 client.26398 handle_command_reply: tid=0 2024-09-21T06:19:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.892+0000 7f4628d5f740 1 client.26398 shutdown 2024-09-21T06:19:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.892+0000 7f4628d5f740 1 -- 172.21.15.3:0/438375302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ab51b0 msgr2=0x55d070e90dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.892+0000 7f4628d5f740 1 --2- 172.21.15.3:0/438375302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ab51b0 0x55d070e90dc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f460c00f730 tx=0x7f460c004730 comp rx=0 tx=0).stop 2024-09-21T06:19:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.892+0000 7f4628d5f740 1 -- 172.21.15.3:0/438375302 shutdown_connections 2024-09-21T06:19:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.892+0000 7f4628d5f740 1 --2- 172.21.15.3:0/438375302 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d070a9c9f0 0x55d070eaba00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.892+0000 7f4628d5f740 1 --2- 172.21.15.3:0/438375302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ade070 0x55d070ea59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.892+0000 7f4628d5f740 1 --2- 172.21.15.3:0/438375302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070e91300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.892+0000 7f4628d5f740 1 --2- 172.21.15.3:0/438375302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ab51b0 0x55d070e90dc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.892+0000 7f4628d5f740 1 -- 172.21.15.3:0/438375302 >> 172.21.15.3:0/438375302 conn(0x55d070ad2460 msgr2=0x55d070e88830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.892+0000 7f45fb7fe640 0 client.26398 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:19:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.892+0000 7f4628d5f740 1 -- 172.21.15.3:0/438375302 shutdown_connections 2024-09-21T06:19:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.892+0000 7f4628d5f740 1 -- 172.21.15.3:0/438375302 wait complete. 2024-09-21T06:19:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.892+0000 7f4628d5f740 20 client.26398 trim_cache size 0 max 16384 2024-09-21T06:19:36.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.906+0000 7f4628d5f740 1 Processor -- start 2024-09-21T06:19:36.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.906+0000 7f4628d5f740 1 -- start start 2024-09-21T06:19:36.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.906+0000 7f4628d5f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ad2460 0x55d070e8fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.906+0000 7f4628d5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ab51b0 0x55d070e929f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.906+0000 7f4628d5f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070e82bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.906+0000 7f4628d5f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f461c1ad3a0 con 0x55d070abd390 2024-09-21T06:19:36.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.906+0000 7f4628d5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f461c1aeee0 con 0x55d070ab51b0 2024-09-21T06:19:36.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.906+0000 7f4628d5f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f461c1ada70 con 0x55d070ad2460 2024-09-21T06:19:36.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.906+0000 7f4620d88640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070e82bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f461b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ab51b0 0x55d070e929f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f461bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ad2460 0x55d070e8fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f4620d88640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070e82bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36272/0 (socket says 172.21.15.3:36272) 2024-09-21T06:19:36.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f4620d88640 1 -- 172.21.15.3:0/1094372915 learned_addr learned my addr 172.21.15.3:0/1094372915 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:36.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f461b7fe640 1 -- 172.21.15.3:0/1094372915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ad2460 msgr2=0x55d070e8fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:36.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f461b7fe640 1 --2- 172.21.15.3:0/1094372915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ad2460 0x55d070e8fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f461b7fe640 1 -- 172.21.15.3:0/1094372915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 msgr2=0x55d070e82bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:36.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f461b7fe640 1 --2- 172.21.15.3:0/1094372915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070e82bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f461b7fe640 1 -- 172.21.15.3:0/1094372915 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d070e8b9d0 con 0x55d070ab51b0 2024-09-21T06:19:36.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f4620d88640 1 --2- 172.21.15.3:0/1094372915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070e82bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:36.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f461b7fe640 1 --2- 172.21.15.3:0/1094372915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ab51b0 0x55d070e929f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55d070eac160 tx=0x7f4610005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b413c9da2e171d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:36.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f4601ffb640 1 -- 172.21.15.3:0/1094372915 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610035030 con 0x55d070ab51b0 2024-09-21T06:19:36.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f4601ffb640 1 -- 172.21.15.3:0/1094372915 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4610005500 con 0x55d070ab51b0 2024-09-21T06:19:36.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.907+0000 7f4601ffb640 1 -- 172.21.15.3:0/1094372915 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46100049b0 con 0x55d070ab51b0 2024-09-21T06:19:36.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.908+0000 7f4628d5f740 1 -- 172.21.15.3:0/1094372915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ab51b0 msgr2=0x55d070e929f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:36.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.908+0000 7f4628d5f740 1 --2- 172.21.15.3:0/1094372915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ab51b0 0x55d070e929f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55d070eac160 tx=0x7f4610005e70 comp rx=0 tx=0).stop 2024-09-21T06:19:36.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.908+0000 7f4628d5f740 1 -- 172.21.15.3:0/1094372915 shutdown_connections 2024-09-21T06:19:36.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.908+0000 7f4628d5f740 1 --2- 172.21.15.3:0/1094372915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070e82bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.908+0000 7f4628d5f740 1 --2- 172.21.15.3:0/1094372915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ab51b0 0x55d070e929f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.908+0000 7f4628d5f740 1 --2- 172.21.15.3:0/1094372915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ad2460 0x55d070e8fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.908+0000 7f4628d5f740 1 -- 172.21.15.3:0/1094372915 >> 172.21.15.3:0/1094372915 conn(0x55d070eabcf0 msgr2=0x55d070e922e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.909+0000 7f4628d5f740 1 -- 172.21.15.3:0/1094372915 shutdown_connections 2024-09-21T06:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.909+0000 7f4628d5f740 1 -- 172.21.15.3:0/1094372915 wait complete. 2024-09-21T06:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.909+0000 7f4628d5f740 1 -- start start 2024-09-21T06:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.909+0000 7f4628d5f740 1 Processor -- start 2024-09-21T06:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.909+0000 7f4628d5f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ab51b0 0x55d070f0e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.909+0000 7f4628d5f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070f0eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.909+0000 7f4628d5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ad2460 0x55d070ea1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.909+0000 7f4628d5f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f461c1ae810 con 0x55d070abd390 2024-09-21T06:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.909+0000 7f4628d5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f461c1ae140 con 0x55d070ad2460 2024-09-21T06:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.909+0000 7f4628d5f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f461c1ab3c0 con 0x55d070ab51b0 2024-09-21T06:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.909+0000 7f4628d5f740 10 client.0 _ll_register_callbacks cb 0x7f460c009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f4620d88640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ad2460 0x55d070ea1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f461bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ab51b0 0x55d070f0e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f4620d88640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ad2460 0x55d070ea1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:59042/0 (socket says 172.21.15.3:59042) 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f461b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070f0eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f461bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ab51b0 0x55d070f0e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:39768/0 (socket says 172.21.15.3:39768) 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f4620d88640 1 -- 172.21.15.3:0/2500769124 learned_addr learned my addr 172.21.15.3:0/2500769124 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f4620d88640 1 -- 172.21.15.3:0/2500769124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ab51b0 msgr2=0x55d070f0e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f4620d88640 1 --2- 172.21.15.3:0/2500769124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ab51b0 0x55d070f0e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f4620d88640 1 -- 172.21.15.3:0/2500769124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 msgr2=0x55d070f0eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f4620d88640 1 --2- 172.21.15.3:0/2500769124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070f0eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f4620d88640 1 -- 172.21.15.3:0/2500769124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d070ea1da0 con 0x55d070ad2460 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f4620d88640 1 --2- 172.21.15.3:0/2500769124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ad2460 0x55d070ea1730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f46140049d0 tx=0x7f46140a9a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7861b152ca3a4c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f45fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f45fb7fe640 1 -- 172.21.15.3:0/2500769124 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46140a9280 con 0x55d070ad2460 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f45fb7fe640 1 -- 172.21.15.3:0/2500769124 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f46140b72a0 con 0x55d070ad2460 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.910+0000 7f4628d5f740 1 -- 172.21.15.3:0/2500769124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4610005500 con 0x55d070ad2460 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.911+0000 7f45fb7fe640 1 -- 172.21.15.3:0/2500769124 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f46140b23e0 con 0x55d070ad2460 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.911+0000 7f4628d5f740 10 client.26450 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.911+0000 7f4628d5f740 1 -- 172.21.15.3:0/2500769124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d070f15e20 con 0x55d070ad2460 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.911+0000 7f45fb7fe640 1 -- 172.21.15.3:0/2500769124 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4610005500 con 0x55d070ad2460 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.911+0000 7f45fb7fe640 1 -- 172.21.15.3:0/2500769124 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f46140e5b60 con 0x55d070ad2460 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.911+0000 7f4628d5f740 10 client.26450 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.911+0000 7f4628d5f740 10 client.26450 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.911+0000 7f4628d5f740 20 client.26450 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.911+0000 7f4628d5f740 1 --2- 172.21.15.3:0/2500769124 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d070ade070 0x55d070f18630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.911+0000 7f4628d5f740 4 client.26450 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.911+0000 7f4628d5f740 1 -- 172.21.15.3:0/2500769124 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d070df98a0 con 0x55d070ade070 2024-09-21T06:19:36.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.911+0000 7f461bfff640 1 --2- 172.21.15.3:0/2500769124 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d070ade070 0x55d070f18630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:36.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f45fb7fe640 10 client.26450 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:19:36.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f461bfff640 1 --2- 172.21.15.3:0/2500769124 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d070ade070 0x55d070f18630 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7538bf2bec6967b server_cookie=41a4ef10e003d633 in_seq=0 out_seq=0 2024-09-21T06:19:36.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f45fb7fe640 1 -- 172.21.15.3:0/2500769124 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d070df98a0 con 0x55d070ade070 2024-09-21T06:19:36.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f45fb7fe640 10 client.26450 handle_command_reply: tid=0 2024-09-21T06:19:36.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f4628d5f740 1 client.26450 shutdown 2024-09-21T06:19:36.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f4628d5f740 1 -- 172.21.15.3:0/2500769124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ad2460 msgr2=0x55d070ea1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:36.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f4628d5f740 1 --2- 172.21.15.3:0/2500769124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ad2460 0x55d070ea1730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f46140049d0 tx=0x7f46140a9a40 comp rx=0 tx=0).stop 2024-09-21T06:19:36.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f4628d5f740 1 -- 172.21.15.3:0/2500769124 shutdown_connections 2024-09-21T06:19:36.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f4628d5f740 1 --2- 172.21.15.3:0/2500769124 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55d070ade070 0x55d070f18630 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f4628d5f740 1 --2- 172.21.15.3:0/2500769124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d070ad2460 0x55d070ea1730 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f4628d5f740 1 --2- 172.21.15.3:0/2500769124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d070abd390 0x55d070f0eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f45fb7fe640 0 client.26450 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:19:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f4628d5f740 1 --2- 172.21.15.3:0/2500769124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d070ab51b0 0x55d070f0e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f4628d5f740 1 -- 172.21.15.3:0/2500769124 >> 172.21.15.3:0/2500769124 conn(0x55d070eabcf0 msgr2=0x55d070e92250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f4628d5f740 1 -- 172.21.15.3:0/2500769124 shutdown_connections 2024-09-21T06:19:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f4628d5f740 1 -- 172.21.15.3:0/2500769124 wait complete. 2024-09-21T06:19:36.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.912+0000 7f4628d5f740 20 client.26450 trim_cache size 0 max 16384 2024-09-21T06:19:36.919 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.913+0000 7f46227eb640 1 -- 172.21.15.3:0/2695320478 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f46040b6680 msgr2=0x7f46040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.913+0000 7f46227eb640 1 --2- 172.21.15.3:0/2695320478 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f46040b6680 0x7f46040b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f460c009fd0 tx=0x7f460c005f90 comp rx=0 tx=0).stop 2024-09-21T06:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.913+0000 7f46227eb640 1 -- 172.21.15.3:0/2695320478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c110200 msgr2=0x7f461c1a1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.913+0000 7f46227eb640 1 --2- 172.21.15.3:0/2695320478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c110200 0x7f461c1a1e90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f461400eb10 tx=0x7f461400c2d0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.913+0000 7f46227eb640 1 -- 172.21.15.3:0/2695320478 shutdown_connections 2024-09-21T06:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.913+0000 7f46227eb640 1 --2- 172.21.15.3:0/2695320478 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f46040b6680 0x7f46040b8b90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.913+0000 7f46227eb640 1 --2- 172.21.15.3:0/2695320478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c110200 0x7f461c1a1e90 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.913+0000 7f46227eb640 1 --2- 172.21.15.3:0/2695320478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f461c102fa0 0x7f461c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.913+0000 7f46227eb640 1 --2- 172.21.15.3:0/2695320478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f461c102650 0x7f461c19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.913+0000 7f46227eb640 1 -- 172.21.15.3:0/2695320478 >> 172.21.15.3:0/2695320478 conn(0x7f461c0797b0 msgr2=0x7f461c108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.913+0000 7f46227eb640 1 -- 172.21.15.3:0/2695320478 shutdown_connections 2024-09-21T06:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:36.913+0000 7f46227eb640 1 -- 172.21.15.3:0/2695320478 wait complete. 2024-09-21T06:19:36.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 453515 12.38 MB/sec execute 69 sec latency 41.000 ms 2024-09-21T06:19:37.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:37 smithi137 ceph-mon[34464]: pgmap v463: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-21T06:19:37.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:37 smithi204 ceph-mon[34659]: pgmap v463: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-21T06:19:37.522 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:37 smithi003 ceph-mon[31227]: pgmap v463: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-21T06:19:37.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 454909 12.30 MB/sec execute 70 sec latency 112.076 ms 2024-09-21T06:19:37.967 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:19:38.258 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:38.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.761+0000 7fcebebe4640 1 -- 172.21.15.3:0/2248483083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fceb8103b60 msgr2=0x7fceb8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:38.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.761+0000 7fcebebe4640 1 --2- 172.21.15.3:0/2248483083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fceb8103b60 0x7fceb8103f70 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fceac009900 tx=0x7fceac02a410 comp rx=0 tx=0).stop 2024-09-21T06:19:38.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.761+0000 7fcebebe4640 1 -- 172.21.15.3:0/2248483083 shutdown_connections 2024-09-21T06:19:38.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.761+0000 7fcebebe4640 1 --2- 172.21.15.3:0/2248483083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fceb81094a0 0x7fceb81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.761+0000 7fcebebe4640 1 --2- 172.21.15.3:0/2248483083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb8109df0 0x7fceb8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.761+0000 7fcebebe4640 1 --2- 172.21.15.3:0/2248483083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fceb8103b60 0x7fceb8103f70 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.761+0000 7fcebebe4640 1 -- 172.21.15.3:0/2248483083 >> 172.21.15.3:0/2248483083 conn(0x7fceb807b850 msgr2=0x7fceb807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.761+0000 7fcebebe4640 1 -- 172.21.15.3:0/2248483083 shutdown_connections 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.761+0000 7fcebebe4640 1 -- 172.21.15.3:0/2248483083 wait complete. 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.762+0000 7fcebebe4640 1 Processor -- start 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.762+0000 7fcebebe4640 1 -- start start 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.762+0000 7fcebebe4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb8103b60 0x7fceb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.762+0000 7fcebebe4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fceb81094a0 0x7fceb8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.762+0000 7fcebebe4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fceb8109df0 0x7fceb8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.762+0000 7fcebebe4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fceb8113ab0 con 0x7fceb81094a0 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.762+0000 7fcebebe4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fceb8113930 con 0x7fceb8103b60 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.762+0000 7fcebebe4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fceb8113c30 con 0x7fceb8109df0 2024-09-21T06:19:38.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fceb7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fceb81094a0 0x7fceb8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fcebc980640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb8103b60 0x7fceb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fceb7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fceb81094a0 0x7fceb8075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36300/0 (socket says 172.21.15.3:36300) 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fceb7fff640 1 -- 172.21.15.3:0/4079371707 learned_addr learned my addr 172.21.15.3:0/4079371707 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fcebd181640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fceb8109df0 0x7fceb8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fceb7fff640 1 -- 172.21.15.3:0/4079371707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fceb8109df0 msgr2=0x7fceb8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fceb7fff640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fceb8109df0 0x7fceb8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fceb7fff640 1 -- 172.21.15.3:0/4079371707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb8103b60 msgr2=0x7fceb8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fceb7fff640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb8103b60 0x7fceb8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fceb7fff640 1 -- 172.21.15.3:0/4079371707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb81a8760 con 0x7fceb81094a0 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fcebc980640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb8103b60 0x7fceb8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fceb7fff640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fceb81094a0 0x7fceb8075c50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fcea400d750 tx=0x7fcea400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fceb5ffb640 1 -- 172.21.15.3:0/4079371707 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4017070 con 0x7fceb81094a0 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fcebebe4640 1 -- 172.21.15.3:0/4079371707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb81a8a50 con 0x7fceb81094a0 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fceb5ffb640 1 -- 172.21.15.3:0/4079371707 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea40054d0 con 0x7fceb81094a0 2024-09-21T06:19:38.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.763+0000 7fceb5ffb640 1 -- 172.21.15.3:0/4079371707 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4004c30 con 0x7fceb81094a0 2024-09-21T06:19:38.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.764+0000 7fcebebe4640 1 -- 172.21.15.3:0/4079371707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb81a8f40 con 0x7fceb81094a0 2024-09-21T06:19:38.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.767+0000 7fcec5158740 1 Processor -- start 2024-09-21T06:19:38.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.767+0000 7fcec5158740 1 -- start start 2024-09-21T06:19:38.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcec5158740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c111ac070 0x555c115536a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcec5158740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c111831b0 0x555c11553be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcec5158740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c1118b390 0x555c11554120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcec5158740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555c114c9ea0 con 0x555c111ac070 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcec5158740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555c114b92d0 con 0x555c1118b390 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcec5158740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555c114bd0e0 con 0x555c111831b0 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fceb5ffb640 1 -- 172.21.15.3:0/4079371707 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fcea4004dd0 con 0x7fceb81094a0 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcebd181640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c111ac070 0x555c115536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcebd181640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c111ac070 0x555c115536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36308/0 (socket says 172.21.15.3:36308) 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcebd181640 1 -- 172.21.15.3:0/1346746264 learned_addr learned my addr 172.21.15.3:0/1346746264 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcebc980640 1 --2- 172.21.15.3:0/1346746264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c111831b0 0x555c11553be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fceb7fff640 1 --2- 172.21.15.3:0/1346746264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c1118b390 0x555c11554120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcebc980640 1 -- 172.21.15.3:0/1346746264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c1118b390 msgr2=0x555c11554120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcebc980640 1 --2- 172.21.15.3:0/1346746264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c1118b390 0x555c11554120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcebc980640 1 -- 172.21.15.3:0/1346746264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c111ac070 msgr2=0x555c115536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcebc980640 1 --2- 172.21.15.3:0/1346746264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c111ac070 0x555c115536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fcebc980640 1 -- 172.21.15.3:0/1346746264 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c11217650 con 0x555c111831b0 2024-09-21T06:19:38.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.768+0000 7fceb7fff640 1 --2- 172.21.15.3:0/1346746264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c1118b390 0x555c11554120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:19:38.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.769+0000 7fcebc980640 1 --2- 172.21.15.3:0/1346746264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c111831b0 0x555c11553be0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fceb8077160 tx=0x7fceac005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a6bc48448b7ff26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:38.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.769+0000 7fceb5ffb640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcea80b6680 0x7fcea80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.769+0000 7fcea2ffd640 1 -- 172.21.15.3:0/1346746264 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac040070 con 0x555c111831b0 2024-09-21T06:19:38.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.769+0000 7fcea2ffd640 1 -- 172.21.15.3:0/1346746264 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceac031820 con 0x555c111831b0 2024-09-21T06:19:38.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.769+0000 7fcea2ffd640 1 -- 172.21.15.3:0/1346746264 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac035040 con 0x555c111831b0 2024-09-21T06:19:38.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.769+0000 7fcebd181640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcea80b6680 0x7fcea80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.769+0000 7fceb5ffb640 1 -- 172.21.15.3:0/4079371707 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcea40e7040 con 0x7fceb81094a0 2024-09-21T06:19:38.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.770+0000 7fcec5158740 1 -- 172.21.15.3:0/1346746264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c111831b0 msgr2=0x555c11553be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:38.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.770+0000 7fcec5158740 1 --2- 172.21.15.3:0/1346746264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c111831b0 0x555c11553be0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fceb8077160 tx=0x7fceac005ed0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.770+0000 7fcebd181640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcea80b6680 0x7fcea80b8b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fceb00116b0 tx=0x7fceb0009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:38.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.770+0000 7fcec5158740 1 -- 172.21.15.3:0/1346746264 shutdown_connections 2024-09-21T06:19:38.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.770+0000 7fcec5158740 1 --2- 172.21.15.3:0/1346746264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c111831b0 0x555c11553be0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.770+0000 7fcec5158740 1 --2- 172.21.15.3:0/1346746264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c1118b390 0x555c11554120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.770+0000 7fcec5158740 1 --2- 172.21.15.3:0/1346746264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c111ac070 0x555c115536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.770+0000 7fcec5158740 1 -- 172.21.15.3:0/1346746264 >> 172.21.15.3:0/1346746264 conn(0x555c111a0460 msgr2=0x555c11551590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:38.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.770+0000 7fcec5158740 1 -- 172.21.15.3:0/1346746264 shutdown_connections 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.770+0000 7fcec5158740 1 -- 172.21.15.3:0/1346746264 wait complete. 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.771+0000 7fcec5158740 1 -- start start 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.771+0000 7fcec5158740 1 Processor -- start 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.771+0000 7fcec5158740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c111831b0 0x555c1155ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.771+0000 7fcec5158740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c1118b390 0x555c1155f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.771+0000 7fcec5158740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c111ac070 0x555c1156b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.771+0000 7fcec5158740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555c1131a700 con 0x555c111831b0 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.771+0000 7fcec5158740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555c11454160 con 0x555c1118b390 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.771+0000 7fcec5158740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555c11561e20 con 0x555c111ac070 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.771+0000 7fcec5158740 10 client.0 _ll_register_callbacks cb 0x555c1146d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.771+0000 7fcebc980640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c111831b0 0x555c1155ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.771+0000 7fceb7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c1118b390 0x555c1155f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.771+0000 7fcebc980640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c111831b0 0x555c1155ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36318/0 (socket says 172.21.15.3:36318) 2024-09-21T06:19:38.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.771+0000 7fcebc980640 1 -- 172.21.15.3:0/3130733329 learned_addr learned my addr 172.21.15.3:0/3130733329 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fcebd181640 1 --2- 172.21.15.3:0/3130733329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c111ac070 0x555c1156b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fceb7fff640 1 -- 172.21.15.3:0/3130733329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c111ac070 msgr2=0x555c1156b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fceb7fff640 1 --2- 172.21.15.3:0/3130733329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c111ac070 0x555c1156b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fceb7fff640 1 -- 172.21.15.3:0/3130733329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c111831b0 msgr2=0x555c1155ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fceb7fff640 1 --2- 172.21.15.3:0/3130733329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c111831b0 0x555c1155ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fceb7fff640 1 -- 172.21.15.3:0/3130733329 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c11481c90 con 0x555c1118b390 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fcebc980640 1 --2- 172.21.15.3:0/3130733329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c111831b0 0x555c1155ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fceb7fff640 1 --2- 172.21.15.3:0/3130733329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c1118b390 0x555c1155f2c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcea40e47d0 tx=0x7fcea40a8f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e79aedc53a8249f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fce97fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fce97fff640 1 -- 172.21.15.3:0/3130733329 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4020070 con 0x555c1118b390 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fce97fff640 1 -- 172.21.15.3:0/3130733329 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcea40e90d0 con 0x555c1118b390 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fcec5158740 1 -- 172.21.15.3:0/3130733329 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceac031820 con 0x555c1118b390 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fce97fff640 1 -- 172.21.15.3:0/3130733329 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcea40aca90 con 0x555c1118b390 2024-09-21T06:19:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fcec5158740 10 client.26462 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.772+0000 7fcec5158740 1 -- 172.21.15.3:0/3130733329 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555c1156db60 con 0x555c1118b390 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.773+0000 7fce97fff640 1 -- 172.21.15.3:0/3130733329 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceac031820 con 0x555c1118b390 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.773+0000 7fce97fff640 1 -- 172.21.15.3:0/3130733329 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fcea4098190 con 0x555c1118b390 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.773+0000 7fcec5158740 10 client.26462 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.773+0000 7fcec5158740 10 client.26462 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.773+0000 7fcec5158740 20 client.26462 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.773+0000 7fcec5158740 1 --2- 172.21.15.3:0/3130733329 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x555c1116a9f0 0x555c115710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.773+0000 7fcec5158740 4 client.26462 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.773+0000 7fcec5158740 1 -- 172.21.15.3:0/3130733329 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c114cbdf0 con 0x555c1116a9f0 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.773+0000 7fcebc980640 1 --2- 172.21.15.3:0/3130733329 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x555c1116a9f0 0x555c115710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.774+0000 7fce97fff640 10 client.26462 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.774+0000 7fcebc980640 1 --2- 172.21.15.3:0/3130733329 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x555c1116a9f0 0x555c115710b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79886bd6f09ff60e server_cookie=9adb7db094c6528e in_seq=0 out_seq=0 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.774+0000 7fce97fff640 1 -- 172.21.15.3:0/3130733329 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555c114cbdf0 con 0x555c1116a9f0 2024-09-21T06:19:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fce97fff640 10 client.26462 handle_command_reply: tid=0 2024-09-21T06:19:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fcec5158740 1 client.26462 shutdown 2024-09-21T06:19:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fcec5158740 1 -- 172.21.15.3:0/3130733329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c1118b390 msgr2=0x555c1155f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fcec5158740 1 --2- 172.21.15.3:0/3130733329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c1118b390 0x555c1155f2c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcea40e47d0 tx=0x7fcea40a8f10 comp rx=0 tx=0).stop 2024-09-21T06:19:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fcec5158740 1 -- 172.21.15.3:0/3130733329 shutdown_connections 2024-09-21T06:19:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fcec5158740 1 --2- 172.21.15.3:0/3130733329 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x555c1116a9f0 0x555c115710b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fcec5158740 1 --2- 172.21.15.3:0/3130733329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c111ac070 0x555c1156b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fcec5158740 1 --2- 172.21.15.3:0/3130733329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c1118b390 0x555c1155f2c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fce97fff640 0 client.26462 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:19:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fcec5158740 1 --2- 172.21.15.3:0/3130733329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c111831b0 0x555c1155ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fcec5158740 1 -- 172.21.15.3:0/3130733329 >> 172.21.15.3:0/3130733329 conn(0x555c111a0460 msgr2=0x555c11556880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fcec5158740 1 -- 172.21.15.3:0/3130733329 shutdown_connections 2024-09-21T06:19:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fcec5158740 1 -- 172.21.15.3:0/3130733329 wait complete. 2024-09-21T06:19:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.775+0000 7fcec5158740 20 client.26462 trim_cache size 0 max 16384 2024-09-21T06:19:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fcec5158740 1 Processor -- start 2024-09-21T06:19:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fcec5158740 1 -- start start 2024-09-21T06:19:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fcec5158740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c1116a9f0 0x555c1155f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fcec5158740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c111831b0 0x555c1156b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fcec5158740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c1118b390 0x555c1155bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fcec5158740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fceb81b1800 con 0x555c1118b390 2024-09-21T06:19:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fcec5158740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fceb81b3340 con 0x555c111831b0 2024-09-21T06:19:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fcec5158740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fceb81b1ed0 con 0x555c1116a9f0 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fcebd181640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c1118b390 0x555c1155bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fceb7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c111831b0 0x555c1156b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fcebc980640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c1116a9f0 0x555c1155f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fceb7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c111831b0 0x555c1156b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:59076/0 (socket says 172.21.15.3:59076) 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fcebd181640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c1118b390 0x555c1155bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36324/0 (socket says 172.21.15.3:36324) 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fceb7fff640 1 -- 172.21.15.3:0/3218987791 learned_addr learned my addr 172.21.15.3:0/3218987791 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.789+0000 7fcebc980640 1 --2- 172.21.15.3:0/3218987791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c1116a9f0 0x555c1155f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:39812/0 (socket says 172.21.15.3:39812) 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.790+0000 7fceb7fff640 1 -- 172.21.15.3:0/3218987791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c1116a9f0 msgr2=0x555c1155f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.790+0000 7fceb7fff640 1 --2- 172.21.15.3:0/3218987791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c1116a9f0 0x555c1155f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.790+0000 7fceb7fff640 1 -- 172.21.15.3:0/3218987791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c1118b390 msgr2=0x555c1155bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.790+0000 7fceb7fff640 1 --2- 172.21.15.3:0/3218987791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c1118b390 0x555c1155bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.790+0000 7fceb7fff640 1 -- 172.21.15.3:0/3218987791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1155d280 con 0x555c111831b0 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.790+0000 7fceb7fff640 1 --2- 172.21.15.3:0/3218987791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c111831b0 0x555c1156b010 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcea4004770 tx=0x7fcea40ac600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed832c4ec7bda03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:38.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.790+0000 7fcea27fc640 1 -- 172.21.15.3:0/3218987791 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea40d4020 con 0x555c111831b0 2024-09-21T06:19:38.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.790+0000 7fcea27fc640 1 -- 172.21.15.3:0/3218987791 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea40b19b0 con 0x555c111831b0 2024-09-21T06:19:38.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.790+0000 7fcea27fc640 1 -- 172.21.15.3:0/3218987791 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4020070 con 0x555c111831b0 2024-09-21T06:19:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.791+0000 7fcec5158740 1 -- 172.21.15.3:0/3218987791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c111831b0 msgr2=0x555c1156b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.791+0000 7fcec5158740 1 --2- 172.21.15.3:0/3218987791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c111831b0 0x555c1156b010 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcea4004770 tx=0x7fcea40ac600 comp rx=0 tx=0).stop 2024-09-21T06:19:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.791+0000 7fcec5158740 1 -- 172.21.15.3:0/3218987791 shutdown_connections 2024-09-21T06:19:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.791+0000 7fcec5158740 1 --2- 172.21.15.3:0/3218987791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c1118b390 0x555c1155bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.791+0000 7fcec5158740 1 --2- 172.21.15.3:0/3218987791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c111831b0 0x555c1156b010 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.791+0000 7fcec5158740 1 --2- 172.21.15.3:0/3218987791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c1116a9f0 0x555c1155f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.791+0000 7fcec5158740 1 -- 172.21.15.3:0/3218987791 >> 172.21.15.3:0/3218987791 conn(0x555c111a0460 msgr2=0x555c1155eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.791+0000 7fcec5158740 1 -- 172.21.15.3:0/3218987791 shutdown_connections 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.791+0000 7fcec5158740 1 -- 172.21.15.3:0/3218987791 wait complete. 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.791+0000 7fcec5158740 1 -- start start 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.792+0000 7fcec5158740 1 Processor -- start 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.792+0000 7fcec5158740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c1116a9f0 0x555c115632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.792+0000 7fcec5158740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c111831b0 0x555c11563810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.792+0000 7fcec5158740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c1118b390 0x555c115daa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.792+0000 7fcec5158740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fceb81b2c70 con 0x555c1118b390 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.792+0000 7fcec5158740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fceb81b25a0 con 0x555c111831b0 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.792+0000 7fcec5158740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fceb81af7e0 con 0x555c1116a9f0 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.792+0000 7fcec5158740 10 client.0 _ll_register_callbacks cb 0x7fcea4009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.792+0000 7fcebd181640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c1118b390 0x555c115daa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.792+0000 7fcebd181640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c1118b390 0x555c115daa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36338/0 (socket says 172.21.15.3:36338) 2024-09-21T06:19:38.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.792+0000 7fcebd181640 1 -- 172.21.15.3:0/1441382499 learned_addr learned my addr 172.21.15.3:0/1441382499 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.792+0000 7fceb7fff640 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c111831b0 0x555c11563810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.792+0000 7fcebc980640 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c1116a9f0 0x555c115632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fcebd181640 1 -- 172.21.15.3:0/1441382499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c1116a9f0 msgr2=0x555c115632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fcebd181640 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c1116a9f0 0x555c115632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fcebd181640 1 -- 172.21.15.3:0/1441382499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c111831b0 msgr2=0x555c11563810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fcebd181640 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c111831b0 0x555c11563810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fcebd181640 1 -- 172.21.15.3:0/1441382499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c11564020 con 0x555c1118b390 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fceb7fff640 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c111831b0 0x555c11563810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fcebd181640 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c1118b390 0x555c115daa70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fceb000fb40 tx=0x7fceb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c492b695e98c20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fce97fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fce97fff640 1 -- 172.21.15.3:0/1441382499 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb002e060 con 0x555c1118b390 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fce97fff640 1 -- 172.21.15.3:0/1441382499 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fceb0010840 con 0x555c1118b390 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fcec5158740 1 -- 172.21.15.3:0/1441382499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcea40b19b0 con 0x555c1118b390 2024-09-21T06:19:38.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fce97fff640 1 -- 172.21.15.3:0/1441382499 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fceb0010a40 con 0x555c1118b390 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fcec5158740 10 client.16893 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fcec5158740 1 -- 172.21.15.3:0/1441382499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555c115dbfd0 con 0x555c1118b390 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.793+0000 7fce97fff640 1 -- 172.21.15.3:0/1441382499 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea40b19b0 con 0x555c1118b390 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.794+0000 7fce97fff640 1 -- 172.21.15.3:0/1441382499 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fceb001e420 con 0x555c1118b390 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.794+0000 7fcec5158740 10 client.16893 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.794+0000 7fcec5158740 10 client.16893 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.794+0000 7fcec5158740 20 client.16893 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.794+0000 7fcec5158740 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x555c111ac070 0x555c115de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.794+0000 7fcec5158740 4 client.16893 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.794+0000 7fcec5158740 1 -- 172.21.15.3:0/1441382499 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c114c78a0 con 0x555c111ac070 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.794+0000 7fcebc980640 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x555c111ac070 0x555c115de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.794+0000 7fcebc980640 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x555c111ac070 0x555c115de760 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca20d29cb946568f server_cookie=b20d380c3b835cf1 in_seq=0 out_seq=0 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.794+0000 7fce97fff640 10 client.16893 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:19:38.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fce97fff640 1 -- 172.21.15.3:0/1441382499 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c114c78a0 con 0x555c111ac070 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fce97fff640 10 client.16893 handle_command_reply: tid=0 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fcec5158740 1 client.16893 shutdown 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fcec5158740 1 -- 172.21.15.3:0/1441382499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c1118b390 msgr2=0x555c115daa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fcec5158740 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c1118b390 0x555c115daa70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fceb000fb40 tx=0x7fceb0004a40 comp rx=0 tx=0).stop 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fcec5158740 1 -- 172.21.15.3:0/1441382499 shutdown_connections 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fcec5158740 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x555c111ac070 0x555c115de760 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fcec5158740 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c1118b390 0x555c115daa70 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fcec5158740 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c111831b0 0x555c11563810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fcec5158740 1 --2- 172.21.15.3:0/1441382499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c1116a9f0 0x555c115632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fce97fff640 0 client.16893 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fcec5158740 1 -- 172.21.15.3:0/1441382499 >> 172.21.15.3:0/1441382499 conn(0x555c111a0460 msgr2=0x555c115546d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fcec5158740 1 -- 172.21.15.3:0/1441382499 shutdown_connections 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fcec5158740 1 -- 172.21.15.3:0/1441382499 wait complete. 2024-09-21T06:19:38.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.795+0000 7fcec5158740 20 client.16893 trim_cache size 0 max 16384 2024-09-21T06:19:38.802 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:19:38.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.796+0000 7fcebebe4640 1 -- 172.21.15.3:0/4079371707 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcea80b6680 msgr2=0x7fcea80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:38.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.796+0000 7fcebebe4640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcea80b6680 0x7fcea80b8b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fceb00116b0 tx=0x7fceb0009500 comp rx=0 tx=0).stop 2024-09-21T06:19:38.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.796+0000 7fcebebe4640 1 -- 172.21.15.3:0/4079371707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fceb81094a0 msgr2=0x7fceb8075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:38.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.796+0000 7fcebebe4640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fceb81094a0 0x7fceb8075c50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fcea400d750 tx=0x7fcea400dc10 comp rx=0 tx=0).stop 2024-09-21T06:19:38.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.796+0000 7fcebebe4640 1 -- 172.21.15.3:0/4079371707 shutdown_connections 2024-09-21T06:19:38.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.796+0000 7fcebebe4640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcea80b6680 0x7fcea80b8b90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.796+0000 7fcebebe4640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fceb8109df0 0x7fceb8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.796+0000 7fcebebe4640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fceb81094a0 0x7fceb8075c50 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.796+0000 7fcebebe4640 1 --2- 172.21.15.3:0/4079371707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb8103b60 0x7fceb8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:38.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.796+0000 7fcebebe4640 1 -- 172.21.15.3:0/4079371707 >> 172.21.15.3:0/4079371707 conn(0x7fceb807b850 msgr2=0x7fceb8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:38.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.796+0000 7fcebebe4640 1 -- 172.21.15.3:0/4079371707 shutdown_connections 2024-09-21T06:19:38.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:38.796+0000 7fcebebe4640 1 -- 172.21.15.3:0/4079371707 wait complete. 2024-09-21T06:19:38.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 457846 12.38 MB/sec execute 71 sec latency 34.673 ms 2024-09-21T06:19:39.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:39 smithi137 ceph-mon[34464]: pgmap v464: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 382 op/s 2024-09-21T06:19:39.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:39 smithi204 ceph-mon[34659]: pgmap v464: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 382 op/s 2024-09-21T06:19:39.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:39 smithi003 ceph-mon[31227]: pgmap v464: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 382 op/s 2024-09-21T06:19:39.670 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:19:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:39.953 INFO:tasks.workunit.client.0.smithi003.stdout: 1 459054 12.51 MB/sec execute 72 sec latency 68.713 ms 2024-09-21T06:19:40.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.471+0000 7ff75e486640 1 -- 172.21.15.3:0/2682207280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff758102080 msgr2=0x7ff758102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:40.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.471+0000 7ff75e486640 1 --2- 172.21.15.3:0/2682207280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff758102080 0x7ff758102490 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff748009900 tx=0x7ff74802a410 comp rx=0 tx=0).stop 2024-09-21T06:19:40.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.471+0000 7ff75e486640 1 -- 172.21.15.3:0/2682207280 shutdown_connections 2024-09-21T06:19:40.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.471+0000 7ff75e486640 1 --2- 172.21.15.3:0/2682207280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758109eb0 0x7ff7581106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.471+0000 7ff75e486640 1 --2- 172.21.15.3:0/2682207280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758109560 0x7ff758109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.471+0000 7ff75e486640 1 --2- 172.21.15.3:0/2682207280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff758102080 0x7ff758102490 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.471+0000 7ff75e486640 1 -- 172.21.15.3:0/2682207280 >> 172.21.15.3:0/2682207280 conn(0x7ff758079ec0 msgr2=0x7ff75807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.472+0000 7ff75e486640 1 -- 172.21.15.3:0/2682207280 shutdown_connections 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.472+0000 7ff75e486640 1 -- 172.21.15.3:0/2682207280 wait complete. 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.472+0000 7ff75e486640 1 Processor -- start 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.472+0000 7ff75e486640 1 -- start start 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75e486640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff758102080 0x7ff75819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75e486640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758109560 0x7ff75819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75e486640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758109eb0 0x7ff75819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75e486640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff758113b70 con 0x7ff758102080 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75e486640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581139f0 con 0x7ff758109560 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75e486640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff758113cf0 con 0x7ff758109eb0 2024-09-21T06:19:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff757fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff758102080 0x7ff75819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75ca23640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758109eb0 0x7ff75819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758109560 0x7ff75819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75ca23640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758109eb0 0x7ff75819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:39838/0 (socket says 172.21.15.3:39838) 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff757fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff758102080 0x7ff75819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36360/0 (socket says 172.21.15.3:36360) 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75ca23640 1 -- 172.21.15.3:0/3193422137 learned_addr learned my addr 172.21.15.3:0/3193422137 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75ca23640 1 -- 172.21.15.3:0/3193422137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758109560 msgr2=0x7ff75819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75ca23640 1 --2- 172.21.15.3:0/3193422137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758109560 0x7ff75819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75ca23640 1 -- 172.21.15.3:0/3193422137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff758102080 msgr2=0x7ff75819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75ca23640 1 --2- 172.21.15.3:0/3193422137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff758102080 0x7ff75819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.473+0000 7ff75ca23640 1 -- 172.21.15.3:0/3193422137 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7581a0450 con 0x7ff758109eb0 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.474+0000 7ff75ca23640 1 --2- 172.21.15.3:0/3193422137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758109eb0 0x7ff75819fd20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff75000eb10 tx=0x7ff75000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.474+0000 7ff7557fa640 1 -- 172.21.15.3:0/3193422137 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750004b20 con 0x7ff758109eb0 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.474+0000 7ff75e486640 1 -- 172.21.15.3:0/3193422137 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7581a0740 con 0x7ff758109eb0 2024-09-21T06:19:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.474+0000 7ff7557fa640 1 -- 172.21.15.3:0/3193422137 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff750004cc0 con 0x7ff758109eb0 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.474+0000 7ff7557fa640 1 -- 172.21.15.3:0/3193422137 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750010690 con 0x7ff758109eb0 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.474+0000 7ff75e486640 1 -- 172.21.15.3:0/3193422137 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7581a6ea0 con 0x7ff758109eb0 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff7649fa740 1 Processor -- start 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff7557fa640 1 -- 172.21.15.3:0/3193422137 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7ff750005020 con 0x7ff758109eb0 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff7649fa740 1 -- start start 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff7649fa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c54e070 0x55be3c8f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff7649fa740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c5251b0 0x55be3c8f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff7649fa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c52d390 0x55be3c8f6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff7649fa740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55be3c86bea0 con 0x55be3c5251b0 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff7649fa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55be3c85b2d0 con 0x55be3c54e070 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff7649fa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55be3c85f0e0 con 0x55be3c52d390 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c54e070 0x55be3c8f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff757fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c5251b0 0x55be3c8f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff757fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c5251b0 0x55be3c8f5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36364/0 (socket says 172.21.15.3:36364) 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c54e070 0x55be3c8f56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:59106/0 (socket says 172.21.15.3:59106) 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff757fff640 1 -- 172.21.15.3:0/2069470462 learned_addr learned my addr 172.21.15.3:0/2069470462 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.477+0000 7ff7577fe640 1 --2- 172.21.15.3:0/2069470462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c52d390 0x55be3c8f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.478+0000 7ff7557fa640 1 --2- 172.21.15.3:0/3193422137 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7400b6680 0x7ff7400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.478+0000 7ff7577fe640 1 -- 172.21.15.3:0/2069470462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c52d390 msgr2=0x55be3c8f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.478+0000 7ff7577fe640 1 --2- 172.21.15.3:0/2069470462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c52d390 0x55be3c8f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.478+0000 7ff7577fe640 1 -- 172.21.15.3:0/2069470462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c5251b0 msgr2=0x55be3c8f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.478+0000 7ff7577fe640 1 --2- 172.21.15.3:0/2069470462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c5251b0 0x55be3c8f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.478+0000 7ff7577fe640 1 -- 172.21.15.3:0/2069470462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3c5b9650 con 0x55be3c54e070 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.478+0000 7ff757fff640 1 --2- 172.21.15.3:0/2069470462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c5251b0 0x55be3c8f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.478+0000 7ff75ca23640 1 --2- 172.21.15.3:0/3193422137 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7400b6680 0x7ff7400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.478+0000 7ff7557fa640 1 -- 172.21.15.3:0/3193422137 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff7500e7c90 con 0x7ff758109eb0 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.478+0000 7ff7577fe640 1 --2- 172.21.15.3:0/2069470462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c54e070 0x55be3c8f56a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009730 tx=0x7ff74c0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=699757d777b45f59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.478+0000 7ff73e7fc640 1 -- 172.21.15.3:0/2069470462 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c004050 con 0x55be3c54e070 2024-09-21T06:19:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.478+0000 7ff73e7fc640 1 -- 172.21.15.3:0/2069470462 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff74c0054d0 con 0x55be3c54e070 2024-09-21T06:19:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.478+0000 7ff73e7fc640 1 -- 172.21.15.3:0/2069470462 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c004b70 con 0x55be3c54e070 2024-09-21T06:19:40.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.479+0000 7ff7649fa740 1 -- 172.21.15.3:0/2069470462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c54e070 msgr2=0x55be3c8f56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:40.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.479+0000 7ff75ca23640 1 --2- 172.21.15.3:0/3193422137 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7400b6680 0x7ff7400b8b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff7500e5a20 tx=0x7ff75007ae00 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:40.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.479+0000 7ff7649fa740 1 --2- 172.21.15.3:0/2069470462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c54e070 0x55be3c8f56a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009730 tx=0x7ff74c0096c0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.479+0000 7ff7649fa740 1 -- 172.21.15.3:0/2069470462 shutdown_connections 2024-09-21T06:19:40.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.479+0000 7ff7649fa740 1 --2- 172.21.15.3:0/2069470462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c52d390 0x55be3c8f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.479+0000 7ff7649fa740 1 --2- 172.21.15.3:0/2069470462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c5251b0 0x55be3c8f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.479+0000 7ff7649fa740 1 --2- 172.21.15.3:0/2069470462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c54e070 0x55be3c8f56a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.479+0000 7ff7649fa740 1 -- 172.21.15.3:0/2069470462 >> 172.21.15.3:0/2069470462 conn(0x55be3c542460 msgr2=0x55be3c8f3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:40.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.479+0000 7ff7649fa740 1 -- 172.21.15.3:0/2069470462 shutdown_connections 2024-09-21T06:19:40.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.479+0000 7ff7649fa740 1 -- 172.21.15.3:0/2069470462 wait complete. 2024-09-21T06:19:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.480+0000 7ff7649fa740 1 -- start start 2024-09-21T06:19:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.480+0000 7ff7649fa740 1 Processor -- start 2024-09-21T06:19:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff7649fa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c5251b0 0x55be3c900e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff7649fa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c52d390 0x55be3c9013c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff7649fa740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c54e070 0x55be3c9115b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff7649fa740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55be3c6bc700 con 0x55be3c54e070 2024-09-21T06:19:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff7649fa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55be3c7f6160 con 0x55be3c52d390 2024-09-21T06:19:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff7649fa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55be3c908290 con 0x55be3c5251b0 2024-09-21T06:19:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff7649fa740 10 client.0 _ll_register_callbacks cb 0x55be3c80f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c54e070 0x55be3c9115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c54e070 0x55be3c9115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36378/0 (socket says 172.21.15.3:36378) 2024-09-21T06:19:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff7577fe640 1 -- 172.21.15.3:0/3557437756 learned_addr learned my addr 172.21.15.3:0/3557437756 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff757fff640 1 --2- 172.21.15.3:0/3557437756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c52d390 0x55be3c9013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff7577fe640 1 --2- 172.21.15.3:0/3557437756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c5251b0 0x55be3c900e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff757fff640 1 -- 172.21.15.3:0/3557437756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c5251b0 msgr2=0x55be3c900e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff757fff640 1 --2- 172.21.15.3:0/3557437756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c5251b0 0x55be3c900e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff757fff640 1 -- 172.21.15.3:0/3557437756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c54e070 msgr2=0x55be3c9115b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff757fff640 1 --2- 172.21.15.3:0/3557437756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c54e070 0x55be3c9115b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff757fff640 1 -- 172.21.15.3:0/3557437756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3c823c90 con 0x55be3c52d390 2024-09-21T06:19:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff757fff640 1 --2- 172.21.15.3:0/3557437756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c52d390 0x55be3c9013c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55be3c8f4320 tx=0x7ff748032e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8525f82ceb323a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.481+0000 7ff7377fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:19:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7377fe640 1 -- 172.21.15.3:0/3557437756 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748035040 con 0x55be3c52d390 2024-09-21T06:19:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7377fe640 1 -- 172.21.15.3:0/3557437756 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff7480316a0 con 0x55be3c52d390 2024-09-21T06:19:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7649fa740 1 -- 172.21.15.3:0/3557437756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff74c0054d0 con 0x55be3c52d390 2024-09-21T06:19:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7377fe640 1 -- 172.21.15.3:0/3557437756 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff7480318a0 con 0x55be3c52d390 2024-09-21T06:19:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7649fa740 10 client.26480 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7649fa740 1 -- 172.21.15.3:0/3557437756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55be3c914030 con 0x55be3c52d390 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7377fe640 1 -- 172.21.15.3:0/3557437756 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff74c0054d0 con 0x55be3c52d390 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7377fe640 1 -- 172.21.15.3:0/3557437756 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff748004750 con 0x55be3c52d390 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7649fa740 10 client.26480 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7649fa740 10 client.26480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7649fa740 20 client.26480 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7649fa740 1 --2- 172.21.15.3:0/3557437756 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55be3c50c9f0 0x55be3c917520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7649fa740 4 client.26480 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7649fa740 1 -- 172.21.15.3:0/3557437756 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be3c86ddf0 con 0x55be3c50c9f0 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.482+0000 7ff7577fe640 1 --2- 172.21.15.3:0/3557437756 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55be3c50c9f0 0x55be3c917520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.483+0000 7ff7577fe640 1 --2- 172.21.15.3:0/3557437756 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55be3c50c9f0 0x55be3c917520 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd402cf00b8ede56 server_cookie=63fd6b9a15529753 in_seq=0 out_seq=0 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.483+0000 7ff7377fe640 10 client.26480 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7377fe640 1 -- 172.21.15.3:0/3557437756 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55be3c86ddf0 con 0x55be3c50c9f0 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7377fe640 10 client.26480 handle_command_reply: tid=0 2024-09-21T06:19:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7649fa740 1 client.26480 shutdown 2024-09-21T06:19:40.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7649fa740 1 -- 172.21.15.3:0/3557437756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c52d390 msgr2=0x55be3c9013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:40.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7649fa740 1 --2- 172.21.15.3:0/3557437756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c52d390 0x55be3c9013c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55be3c8f4320 tx=0x7ff748032e20 comp rx=0 tx=0).stop 2024-09-21T06:19:40.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7649fa740 1 -- 172.21.15.3:0/3557437756 shutdown_connections 2024-09-21T06:19:40.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7649fa740 1 --2- 172.21.15.3:0/3557437756 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55be3c50c9f0 0x55be3c917520 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7649fa740 1 --2- 172.21.15.3:0/3557437756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c54e070 0x55be3c9115b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7649fa740 1 --2- 172.21.15.3:0/3557437756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c52d390 0x55be3c9013c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7377fe640 0 client.26480 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:40.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7649fa740 1 --2- 172.21.15.3:0/3557437756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c5251b0 0x55be3c900e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7649fa740 1 -- 172.21.15.3:0/3557437756 >> 172.21.15.3:0/3557437756 conn(0x55be3c542460 msgr2=0x55be3c8f8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:40.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7649fa740 1 -- 172.21.15.3:0/3557437756 shutdown_connections 2024-09-21T06:19:40.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7649fa740 1 -- 172.21.15.3:0/3557437756 wait complete. 2024-09-21T06:19:40.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.484+0000 7ff7649fa740 20 client.26480 trim_cache size 0 max 16384 2024-09-21T06:19:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.498+0000 7ff7649fa740 1 Processor -- start 2024-09-21T06:19:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.498+0000 7ff7649fa740 1 -- start start 2024-09-21T06:19:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff7649fa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c5251b0 0x55be3c901400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff7649fa740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 0x55be3c8f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff7649fa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c54e070 0x55be3c9079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff7649fa740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581ad440 con 0x55be3c52d390 2024-09-21T06:19:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff7649fa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581aef80 con 0x55be3c5251b0 2024-09-21T06:19:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff7649fa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581adb10 con 0x55be3c54e070 2024-09-21T06:19:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff757fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 0x55be3c8f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff757fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 0x55be3c8f8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36386/0 (socket says 172.21.15.3:36386) 2024-09-21T06:19:40.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff757fff640 1 -- 172.21.15.3:0/644127745 learned_addr learned my addr 172.21.15.3:0/644127745 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:40.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff7577fe640 1 --2- 172.21.15.3:0/644127745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c54e070 0x55be3c9079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff7577fe640 1 --2- 172.21.15.3:0/644127745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c5251b0 0x55be3c901400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff757fff640 1 -- 172.21.15.3:0/644127745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c54e070 msgr2=0x55be3c9079d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:40.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff757fff640 1 --2- 172.21.15.3:0/644127745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c54e070 0x55be3c9079d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff757fff640 1 -- 172.21.15.3:0/644127745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c5251b0 msgr2=0x55be3c901400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:40.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff757fff640 1 --2- 172.21.15.3:0/644127745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c5251b0 0x55be3c901400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff757fff640 1 -- 172.21.15.3:0/644127745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3c8f93f0 con 0x55be3c52d390 2024-09-21T06:19:40.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.499+0000 7ff757fff640 1 --2- 172.21.15.3:0/644127745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 0x55be3c8f8eb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff748031660 tx=0x7ff748031690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c2e818471c5b07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:40.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.500+0000 7ff73dffb640 1 -- 172.21.15.3:0/644127745 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748035030 con 0x55be3c52d390 2024-09-21T06:19:40.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.500+0000 7ff73dffb640 1 -- 172.21.15.3:0/644127745 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7480046c0 con 0x55be3c52d390 2024-09-21T06:19:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.500+0000 7ff73dffb640 1 -- 172.21.15.3:0/644127745 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748004fb0 con 0x55be3c52d390 2024-09-21T06:19:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.500+0000 7ff7649fa740 1 -- 172.21.15.3:0/644127745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 msgr2=0x55be3c8f8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.500+0000 7ff7649fa740 1 --2- 172.21.15.3:0/644127745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 0x55be3c8f8eb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff748031660 tx=0x7ff748031690 comp rx=0 tx=0).stop 2024-09-21T06:19:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.501+0000 7ff7649fa740 1 -- 172.21.15.3:0/644127745 shutdown_connections 2024-09-21T06:19:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.501+0000 7ff7649fa740 1 --2- 172.21.15.3:0/644127745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c54e070 0x55be3c9079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.501+0000 7ff7649fa740 1 --2- 172.21.15.3:0/644127745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 0x55be3c8f8eb0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.501+0000 7ff7649fa740 1 --2- 172.21.15.3:0/644127745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c5251b0 0x55be3c901400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.501+0000 7ff7649fa740 1 -- 172.21.15.3:0/644127745 >> 172.21.15.3:0/644127745 conn(0x55be3c542460 msgr2=0x55be3c8f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.501+0000 7ff7649fa740 1 -- 172.21.15.3:0/644127745 shutdown_connections 2024-09-21T06:19:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.501+0000 7ff7649fa740 1 -- 172.21.15.3:0/644127745 wait complete. 2024-09-21T06:19:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.501+0000 7ff7649fa740 1 -- start start 2024-09-21T06:19:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.501+0000 7ff7649fa740 1 Processor -- start 2024-09-21T06:19:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff7649fa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c5251b0 0x55be3c90d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff7649fa740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 0x55be3c90dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff7649fa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c54e070 0x55be3c904a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff7649fa740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581ae8b0 con 0x55be3c52d390 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff7649fa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581ae1e0 con 0x55be3c5251b0 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff7649fa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581ab420 con 0x55be3c54e070 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff7649fa740 10 client.0 _ll_register_callbacks cb 0x7ff748009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff757fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 0x55be3c90dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff757fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 0x55be3c90dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36402/0 (socket says 172.21.15.3:36402) 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff757fff640 1 -- 172.21.15.3:0/61566721 learned_addr learned my addr 172.21.15.3:0/61566721 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff7577fe640 1 --2- 172.21.15.3:0/61566721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c5251b0 0x55be3c90d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff7577fe640 1 --2- 172.21.15.3:0/61566721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c54e070 0x55be3c904a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff757fff640 1 -- 172.21.15.3:0/61566721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c54e070 msgr2=0x55be3c904a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff757fff640 1 --2- 172.21.15.3:0/61566721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c54e070 0x55be3c904a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff757fff640 1 -- 172.21.15.3:0/61566721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c5251b0 msgr2=0x55be3c90d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff757fff640 1 --2- 172.21.15.3:0/61566721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c5251b0 0x55be3c90d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff757fff640 1 -- 172.21.15.3:0/61566721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3c9051a0 con 0x55be3c52d390 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff757fff640 1 --2- 172.21.15.3:0/61566721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 0x55be3c90dcf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff74802a930 tx=0x7ff74802ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0498eb2394bab2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.502+0000 7ff7377fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7377fe640 1 -- 172.21.15.3:0/61566721 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748035070 con 0x55be3c52d390 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7377fe640 1 -- 172.21.15.3:0/61566721 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7ff748031b10 con 0x55be3c52d390 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7649fa740 1 -- 172.21.15.3:0/61566721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7480046c0 con 0x55be3c52d390 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7377fe640 1 -- 172.21.15.3:0/61566721 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff7480314a0 con 0x55be3c52d390 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7649fa740 10 client.16911 fetch_fsmap learned FSMap version 31 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7649fa740 1 -- 172.21.15.3:0/61566721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55be3c90a060 con 0x55be3c52d390 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7377fe640 1 -- 172.21.15.3:0/61566721 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7480046c0 con 0x55be3c52d390 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7377fe640 1 -- 172.21.15.3:0/61566721 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff7480035f0 con 0x55be3c52d390 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7649fa740 10 client.16911 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:19:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7649fa740 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:40.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7649fa740 20 client.16911 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:40.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7649fa740 1 --2- 172.21.15.3:0/61566721 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55be3c4e1da0 0x55be3c9888f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:40.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7649fa740 4 client.16911 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:19:40.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.503+0000 7ff7649fa740 1 -- 172.21.15.3:0/61566721 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be3c8698a0 con 0x55be3c4e1da0 2024-09-21T06:19:40.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.504+0000 7ff7577fe640 1 --2- 172.21.15.3:0/61566721 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55be3c4e1da0 0x55be3c9888f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.504+0000 7ff7377fe640 10 client.16911 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.504+0000 7ff7577fe640 1 --2- 172.21.15.3:0/61566721 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55be3c4e1da0 0x55be3c9888f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f8f8755d1bee55b server_cookie=6e8a69b0c819d9cc in_seq=0 out_seq=0 2024-09-21T06:19:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7377fe640 1 -- 172.21.15.3:0/61566721 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be3c8698a0 con 0x55be3c4e1da0 2024-09-21T06:19:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7377fe640 10 client.16911 handle_command_reply: tid=0 2024-09-21T06:19:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7649fa740 1 client.16911 shutdown 2024-09-21T06:19:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7649fa740 1 -- 172.21.15.3:0/61566721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 msgr2=0x55be3c90dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7649fa740 1 --2- 172.21.15.3:0/61566721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 0x55be3c90dcf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff74802a930 tx=0x7ff74802ab90 comp rx=0 tx=0).stop 2024-09-21T06:19:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7649fa740 1 -- 172.21.15.3:0/61566721 shutdown_connections 2024-09-21T06:19:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7649fa740 1 --2- 172.21.15.3:0/61566721 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55be3c4e1da0 0x55be3c9888f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7377fe640 0 client.16911 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7649fa740 1 --2- 172.21.15.3:0/61566721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be3c54e070 0x55be3c904a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7649fa740 1 --2- 172.21.15.3:0/61566721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be3c52d390 0x55be3c90dcf0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7649fa740 1 --2- 172.21.15.3:0/61566721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55be3c5251b0 0x55be3c90d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7649fa740 1 -- 172.21.15.3:0/61566721 >> 172.21.15.3:0/61566721 conn(0x55be3c542460 msgr2=0x55be3c902330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7649fa740 1 -- 172.21.15.3:0/61566721 shutdown_connections 2024-09-21T06:19:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7649fa740 1 -- 172.21.15.3:0/61566721 wait complete. 2024-09-21T06:19:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.505+0000 7ff7649fa740 20 client.16911 trim_cache size 0 max 16384 2024-09-21T06:19:40.512 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:19:40.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.506+0000 7ff75e486640 1 -- 172.21.15.3:0/3193422137 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7400b6680 msgr2=0x7ff7400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:40.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.506+0000 7ff75e486640 1 --2- 172.21.15.3:0/3193422137 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7400b6680 0x7ff7400b8b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff7500e5a20 tx=0x7ff75007ae00 comp rx=0 tx=0).stop 2024-09-21T06:19:40.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.506+0000 7ff75e486640 1 -- 172.21.15.3:0/3193422137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758109eb0 msgr2=0x7ff75819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:40.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.506+0000 7ff75e486640 1 --2- 172.21.15.3:0/3193422137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758109eb0 0x7ff75819fd20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff75000eb10 tx=0x7ff75000c2d0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.506+0000 7ff75e486640 1 -- 172.21.15.3:0/3193422137 shutdown_connections 2024-09-21T06:19:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.506+0000 7ff75e486640 1 --2- 172.21.15.3:0/3193422137 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7400b6680 0x7ff7400b8b90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.506+0000 7ff75e486640 1 --2- 172.21.15.3:0/3193422137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758109eb0 0x7ff75819fd20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.506+0000 7ff75e486640 1 --2- 172.21.15.3:0/3193422137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758109560 0x7ff75819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.506+0000 7ff75e486640 1 --2- 172.21.15.3:0/3193422137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff758102080 0x7ff75819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.506+0000 7ff75e486640 1 -- 172.21.15.3:0/3193422137 >> 172.21.15.3:0/3193422137 conn(0x7ff758079ec0 msgr2=0x7ff7581075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.506+0000 7ff75e486640 1 -- 172.21.15.3:0/3193422137 shutdown_connections 2024-09-21T06:19:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:40.506+0000 7ff75e486640 1 -- 172.21.15.3:0/3193422137 wait complete. 2024-09-21T06:19:40.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 462156 12.60 MB/sec execute 73 sec latency 39.144 ms 2024-09-21T06:19:41.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:41 smithi137 ceph-mon[34464]: pgmap v465: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.6 MiB/s wr, 364 op/s 2024-09-21T06:19:41.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:41 smithi204 ceph-mon[34659]: pgmap v465: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.6 MiB/s wr, 364 op/s 2024-09-21T06:19:41.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:41 smithi003 ceph-mon[31227]: pgmap v465: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.6 MiB/s wr, 364 op/s 2024-09-21T06:19:41.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 464670 12.58 MB/sec execute 74 sec latency 23.079 ms 2024-09-21T06:19:42.282 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:19:42.282 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:19:42.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:42 smithi137 ceph-mon[34464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:19:42.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:42 smithi204 ceph-mon[34659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:19:42.553 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:42.600 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:42 smithi003 ceph-mon[31227]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:19:42.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 466235 12.53 MB/sec execute 75 sec latency 50.455 ms 2024-09-21T06:19:43.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.061+0000 7f88beb03640 1 -- 172.21.15.3:0/2218425650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88b81083a0 msgr2=0x7f88b81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.061+0000 7f88beb03640 1 --2- 172.21.15.3:0/2218425650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88b81083a0 0x7f88b81087b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009900 tx=0x7f88a002a410 comp rx=0 tx=0).stop 2024-09-21T06:19:43.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.062+0000 7f88beb03640 1 -- 172.21.15.3:0/2218425650 shutdown_connections 2024-09-21T06:19:43.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.062+0000 7f88beb03640 1 --2- 172.21.15.3:0/2218425650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b8108cf0 0x7f88b810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.062+0000 7f88beb03640 1 --2- 172.21.15.3:0/2218425650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88b81083a0 0x7f88b81087b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.062+0000 7f88beb03640 1 --2- 172.21.15.3:0/2218425650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88b8102a60 0x7f88b8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.062+0000 7f88beb03640 1 -- 172.21.15.3:0/2218425650 >> 172.21.15.3:0/2218425650 conn(0x7f88b8079730 msgr2=0x7f88b8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:43.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.062+0000 7f88beb03640 1 -- 172.21.15.3:0/2218425650 shutdown_connections 2024-09-21T06:19:43.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.062+0000 7f88beb03640 1 -- 172.21.15.3:0/2218425650 wait complete. 2024-09-21T06:19:43.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.062+0000 7f88beb03640 1 Processor -- start 2024-09-21T06:19:43.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.062+0000 7f88beb03640 1 -- start start 2024-09-21T06:19:43.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88beb03640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b8102a60 0x7f88b819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88beb03640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88b81083a0 0x7f88b819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88beb03640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88b8108cf0 0x7f88b819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88beb03640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8112960 con 0x7f88b8108cf0 2024-09-21T06:19:43.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88beb03640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81127e0 con 0x7f88b81083a0 2024-09-21T06:19:43.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88beb03640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8112ae0 con 0x7f88b8102a60 2024-09-21T06:19:43.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88bd0a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88b8108cf0 0x7f88b819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88bc89f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b8102a60 0x7f88b819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88b7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88b81083a0 0x7f88b819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88bd0a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88b8108cf0 0x7f88b819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36420/0 (socket says 172.21.15.3:36420) 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88bd0a0640 1 -- 172.21.15.3:0/3221964389 learned_addr learned my addr 172.21.15.3:0/3221964389 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88bd0a0640 1 -- 172.21.15.3:0/3221964389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b8102a60 msgr2=0x7f88b819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88bd0a0640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b8102a60 0x7f88b819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88bd0a0640 1 -- 172.21.15.3:0/3221964389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88b81083a0 msgr2=0x7f88b819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88bd0a0640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88b81083a0 0x7f88b819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88bd0a0640 1 -- 172.21.15.3:0/3221964389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b81a0470 con 0x7f88b8108cf0 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88bc89f640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b8102a60 0x7f88b819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88b7fff640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88b81083a0 0x7f88b819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.063+0000 7f88bd0a0640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88b8108cf0 0x7f88b819fd40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f88b000e970 tx=0x7f88b000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.064+0000 7f88b5ffb640 1 -- 172.21.15.3:0/3221964389 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b000ccc0 con 0x7f88b8108cf0 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.064+0000 7f88b5ffb640 1 -- 172.21.15.3:0/3221964389 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b000ce60 con 0x7f88b8108cf0 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.064+0000 7f88beb03640 1 -- 172.21.15.3:0/3221964389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b81a0760 con 0x7f88b8108cf0 2024-09-21T06:19:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.064+0000 7f88b5ffb640 1 -- 172.21.15.3:0/3221964389 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0010430 con 0x7f88b8108cf0 2024-09-21T06:19:43.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.064+0000 7f88beb03640 1 -- 172.21.15.3:0/3221964389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b81a6ec0 con 0x7f88b8108cf0 2024-09-21T06:19:43.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.067+0000 7f88c5077740 1 Processor -- start 2024-09-21T06:19:43.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.067+0000 7f88b5ffb640 1 -- 172.21.15.3:0/3221964389 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f88b0004ff0 con 0x7f88b8108cf0 2024-09-21T06:19:43.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.068+0000 7f88c5077740 1 -- start start 2024-09-21T06:19:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.068+0000 7f88c5077740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f95350 0x55911933c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.068+0000 7f88c5077740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x55911933c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.068+0000 7f88c5077740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f74670 0x55911933ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.068+0000 7f88c5077740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5591192b2e60 con 0x559118f74670 2024-09-21T06:19:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.068+0000 7f88c5077740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5591192a2290 con 0x559118f95350 2024-09-21T06:19:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.068+0000 7f88c5077740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5591192a60a0 con 0x559118f6c490 2024-09-21T06:19:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.068+0000 7f88b7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f95350 0x55911933c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.068+0000 7f88bc89f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x55911933c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.068+0000 7f88bc89f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x55911933c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:39910/0 (socket says 172.21.15.3:39910) 2024-09-21T06:19:43.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.068+0000 7f88b7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f95350 0x55911933c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:59170/0 (socket says 172.21.15.3:59170) 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.068+0000 7f88bc89f640 1 -- 172.21.15.3:0/2410602225 learned_addr learned my addr 172.21.15.3:0/2410602225 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.068+0000 7f88b7fff640 1 --2- 172.21.15.3:0/2410602225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f74670 0x55911933ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.069+0000 7f88b5ffb640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f88a40b6700 0x7f88a40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.069+0000 7f88bc89f640 1 -- 172.21.15.3:0/2410602225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f95350 msgr2=0x55911933c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.069+0000 7f88bc89f640 1 --2- 172.21.15.3:0/2410602225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f95350 0x55911933c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.069+0000 7f88bc89f640 1 -- 172.21.15.3:0/2410602225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f74670 msgr2=0x55911933ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.069+0000 7f88bc89f640 1 --2- 172.21.15.3:0/2410602225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f74670 0x55911933ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.069+0000 7f88bc89f640 1 -- 172.21.15.3:0/2410602225 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559119000930 con 0x559118f6c490 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.069+0000 7f88b7fff640 1 --2- 172.21.15.3:0/2410602225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f74670 0x55911933ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.069+0000 7f88bd0a0640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f88a40b6700 0x7f88a40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.069+0000 7f88b5ffb640 1 -- 172.21.15.3:0/3221964389 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f88b00e79f0 con 0x7f88b8108cf0 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.069+0000 7f88bc89f640 1 --2- 172.21.15.3:0/2410602225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x55911933c970 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f88a8005f40 tx=0x7f88a8011d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5770c1ddf9ce0476 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.069+0000 7f88aeffd640 1 -- 172.21.15.3:0/2410602225 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8011330 con 0x559118f6c490 2024-09-21T06:19:43.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.069+0000 7f88aeffd640 1 -- 172.21.15.3:0/2410602225 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a800fe50 con 0x559118f6c490 2024-09-21T06:19:43.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.069+0000 7f88aeffd640 1 -- 172.21.15.3:0/2410602225 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a80193e0 con 0x559118f6c490 2024-09-21T06:19:43.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.070+0000 7f88c5077740 1 -- 172.21.15.3:0/2410602225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 msgr2=0x55911933c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.070+0000 7f88c5077740 1 --2- 172.21.15.3:0/2410602225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x55911933c970 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f88a8005f40 tx=0x7f88a8011d30 comp rx=0 tx=0).stop 2024-09-21T06:19:43.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.070+0000 7f88bd0a0640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f88a40b6700 0x7f88a40b8c10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f88b00e5780 tx=0x7f88b0079430 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:43.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.070+0000 7f88c5077740 1 -- 172.21.15.3:0/2410602225 shutdown_connections 2024-09-21T06:19:43.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.070+0000 7f88c5077740 1 --2- 172.21.15.3:0/2410602225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x55911933c970 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.070+0000 7f88c5077740 1 --2- 172.21.15.3:0/2410602225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f74670 0x55911933ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.070+0000 7f88c5077740 1 --2- 172.21.15.3:0/2410602225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f95350 0x55911933c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.070+0000 7f88c5077740 1 -- 172.21.15.3:0/2410602225 >> 172.21.15.3:0/2410602225 conn(0x559118f89740 msgr2=0x55911933a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:43.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.070+0000 7f88c5077740 1 -- 172.21.15.3:0/2410602225 shutdown_connections 2024-09-21T06:19:43.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.070+0000 7f88c5077740 1 -- 172.21.15.3:0/2410602225 wait complete. 2024-09-21T06:19:43.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.071+0000 7f88c5077740 1 -- start start 2024-09-21T06:19:43.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.071+0000 7f88c5077740 1 Processor -- start 2024-09-21T06:19:43.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.071+0000 7f88c5077740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x55911934c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.071+0000 7f88c5077740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f74670 0x55911934c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.071+0000 7f88c5077740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f95350 0x5591193582a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.071+0000 7f88c5077740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559119103890 con 0x559118f74670 2024-09-21T06:19:43.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.071+0000 7f88c5077740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55911923cd40 con 0x559118f95350 2024-09-21T06:19:43.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.071+0000 7f88c5077740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55911934ef40 con 0x559118f6c490 2024-09-21T06:19:43.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f88c5077740 10 client.0 _ll_register_callbacks cb 0x5591192a0c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:43.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f88bc89f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f74670 0x55911934c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f88bc89f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f74670 0x55911934c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36434/0 (socket says 172.21.15.3:36434) 2024-09-21T06:19:43.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f88b7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f95350 0x5591193582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f88bc89f640 1 -- 172.21.15.3:0/1274596918 learned_addr learned my addr 172.21.15.3:0/1274596918 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:43.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f88b7fff640 1 --2- 172.21.15.3:0/1274596918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x55911934c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f88bc89f640 1 -- 172.21.15.3:0/1274596918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 msgr2=0x55911934c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f88bc89f640 1 --2- 172.21.15.3:0/1274596918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x55911934c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f88bc89f640 1 -- 172.21.15.3:0/1274596918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f95350 msgr2=0x5591193582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f88bc89f640 1 --2- 172.21.15.3:0/1274596918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f95350 0x5591193582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f88bc89f640 1 -- 172.21.15.3:0/1274596918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591193588b0 con 0x559118f74670 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f88bc89f640 1 --2- 172.21.15.3:0/1274596918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f74670 0x55911934c9d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f88a8006010 tx=0x7f88a8010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16d750c2c8bac3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f8897fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f8897fff640 1 -- 172.21.15.3:0/1274596918 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a801f040 con 0x559118f74670 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f8897fff640 1 -- 172.21.15.3:0/1274596918 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f88a8004a40 con 0x559118f74670 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.072+0000 7f88c5077740 1 -- 172.21.15.3:0/1274596918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a800fe50 con 0x559118f74670 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.073+0000 7f8897fff640 1 -- 172.21.15.3:0/1274596918 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f88a8032050 con 0x559118f74670 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.073+0000 7f88c5077740 10 client.16923 fetch_fsmap learned FSMap version 32 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.073+0000 7f88c5077740 1 -- 172.21.15.3:0/1274596918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55911935b1e0 con 0x559118f74670 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.073+0000 7f8897fff640 1 -- 172.21.15.3:0/1274596918 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a800fe50 con 0x559118f74670 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.073+0000 7f8897fff640 1 -- 172.21.15.3:0/1274596918 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f88a8018070 con 0x559118f74670 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.073+0000 7f88c5077740 10 client.16923 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:19:43.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.073+0000 7f88c5077740 10 client.16923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:43.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.073+0000 7f88c5077740 20 client.16923 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:43.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.073+0000 7f88c5077740 1 --2- 172.21.15.3:0/1274596918 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559118f53cd0 0x55911935e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.073+0000 7f88c5077740 4 client.16923 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:43.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.073+0000 7f88c5077740 1 -- 172.21.15.3:0/1274596918 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591192b4db0 con 0x559118f53cd0 2024-09-21T06:19:43.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.073+0000 7f88b7fff640 1 --2- 172.21.15.3:0/1274596918 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559118f53cd0 0x55911935e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.074+0000 7f8897fff640 10 client.16923 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:43.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.074+0000 7f88b7fff640 1 --2- 172.21.15.3:0/1274596918 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559118f53cd0 0x55911935e540 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8f07ee42676b882 server_cookie=652f21945dd020d0 in_seq=0 out_seq=0 2024-09-21T06:19:43.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.074+0000 7f8897fff640 1 -- 172.21.15.3:0/1274596918 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591192b4db0 con 0x559118f53cd0 2024-09-21T06:19:43.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.074+0000 7f8897fff640 10 client.16923 handle_command_reply: tid=0 2024-09-21T06:19:43.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.075+0000 7f88c5077740 1 client.16923 shutdown 2024-09-21T06:19:43.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.075+0000 7f88c5077740 1 -- 172.21.15.3:0/1274596918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f74670 msgr2=0x55911934c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.075+0000 7f88c5077740 1 --2- 172.21.15.3:0/1274596918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f74670 0x55911934c9d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f88a8006010 tx=0x7f88a8010ed0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.075+0000 7f88c5077740 1 -- 172.21.15.3:0/1274596918 shutdown_connections 2024-09-21T06:19:43.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.075+0000 7f88c5077740 1 --2- 172.21.15.3:0/1274596918 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559118f53cd0 0x55911935e540 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.075+0000 7f88c5077740 1 --2- 172.21.15.3:0/1274596918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f95350 0x5591193582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.075+0000 7f88c5077740 1 --2- 172.21.15.3:0/1274596918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f74670 0x55911934c9d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.075+0000 7f8897fff640 0 client.16923 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:43.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.075+0000 7f88c5077740 1 --2- 172.21.15.3:0/1274596918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x55911934c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.075+0000 7f88c5077740 1 -- 172.21.15.3:0/1274596918 >> 172.21.15.3:0/1274596918 conn(0x559118f89740 msgr2=0x559119338850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:43.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.075+0000 7f88c5077740 1 -- 172.21.15.3:0/1274596918 shutdown_connections 2024-09-21T06:19:43.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.075+0000 7f88c5077740 1 -- 172.21.15.3:0/1274596918 wait complete. 2024-09-21T06:19:43.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.075+0000 7f88c5077740 20 client.16923 trim_cache size 0 max 16384 2024-09-21T06:19:43.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.096+0000 7f88c5077740 1 Processor -- start 2024-09-21T06:19:43.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.096+0000 7f88c5077740 1 -- start start 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88c5077740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f53cd0 0x559119347a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88c5077740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f6c490 0x55911934bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88c5077740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f74670 0x55911934de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88c5077740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81ad4b0 con 0x559118f6c490 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88c5077740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81af020 con 0x559118f74670 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88c5077740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81adb80 con 0x559118f53cd0 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88bc89f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f6c490 0x55911934bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88b7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f53cd0 0x559119347a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88bc89f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f6c490 0x55911934bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36444/0 (socket says 172.21.15.3:36444) 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88bc89f640 1 -- 172.21.15.3:0/726135357 learned_addr learned my addr 172.21.15.3:0/726135357 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88b7fff640 1 --2- 172.21.15.3:0/726135357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f74670 0x55911934de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88bc89f640 1 -- 172.21.15.3:0/726135357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f53cd0 msgr2=0x559119347a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88bc89f640 1 --2- 172.21.15.3:0/726135357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f53cd0 0x559119347a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88bc89f640 1 -- 172.21.15.3:0/726135357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f74670 msgr2=0x55911934de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88bc89f640 1 --2- 172.21.15.3:0/726135357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f74670 0x55911934de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88bc89f640 1 -- 172.21.15.3:0/726135357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55911934d510 con 0x559118f6c490 2024-09-21T06:19:43.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.097+0000 7f88b7fff640 1 --2- 172.21.15.3:0/726135357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f53cd0 0x559119347a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:43.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.098+0000 7f88bc89f640 1 --2- 172.21.15.3:0/726135357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f6c490 0x55911934bae0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f88a8005d70 tx=0x7f88a80092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889e1af0657f32b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:43.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.098+0000 7f88ae7fc640 1 -- 172.21.15.3:0/726135357 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a803c070 con 0x559118f6c490 2024-09-21T06:19:43.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.098+0000 7f88ae7fc640 1 -- 172.21.15.3:0/726135357 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a8029070 con 0x559118f6c490 2024-09-21T06:19:43.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.098+0000 7f88ae7fc640 1 -- 172.21.15.3:0/726135357 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8003bd0 con 0x559118f6c490 2024-09-21T06:19:43.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.099+0000 7f88c5077740 1 -- 172.21.15.3:0/726135357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f6c490 msgr2=0x55911934bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.099+0000 7f88c5077740 1 --2- 172.21.15.3:0/726135357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f6c490 0x55911934bae0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f88a8005d70 tx=0x7f88a80092a0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.099+0000 7f88c5077740 1 -- 172.21.15.3:0/726135357 shutdown_connections 2024-09-21T06:19:43.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.099+0000 7f88c5077740 1 --2- 172.21.15.3:0/726135357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f74670 0x55911934de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.099+0000 7f88c5077740 1 --2- 172.21.15.3:0/726135357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f6c490 0x55911934bae0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.099+0000 7f88c5077740 1 --2- 172.21.15.3:0/726135357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f53cd0 0x559119347a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.099+0000 7f88c5077740 1 -- 172.21.15.3:0/726135357 >> 172.21.15.3:0/726135357 conn(0x559118f89740 msgr2=0x55911934d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:43.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.099+0000 7f88c5077740 1 -- 172.21.15.3:0/726135357 shutdown_connections 2024-09-21T06:19:43.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.099+0000 7f88c5077740 1 -- 172.21.15.3:0/726135357 wait complete. 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.099+0000 7f88c5077740 1 -- start start 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.099+0000 7f88c5077740 1 Processor -- start 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88c5077740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f53cd0 0x559119354850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88c5077740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x559119350d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88c5077740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f74670 0x559119351250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88c5077740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81ae950 con 0x559118f53cd0 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88c5077740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81ae280 con 0x559118f74670 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88c5077740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81ab4b0 con 0x559118f6c490 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88c5077740 10 client.0 _ll_register_callbacks cb 0x7f88b0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88b7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f53cd0 0x559119354850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88b7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f53cd0 0x559119354850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36448/0 (socket says 172.21.15.3:36448) 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88b7fff640 1 -- 172.21.15.3:0/1872562435 learned_addr learned my addr 172.21.15.3:0/1872562435 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88bc89f640 1 --2- 172.21.15.3:0/1872562435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x559119350d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88b7fff640 1 --2- 172.21.15.3:0/1872562435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f74670 0x559119351250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88b7fff640 1 -- 172.21.15.3:0/1872562435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 msgr2=0x559119350d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88b7fff640 1 --2- 172.21.15.3:0/1872562435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x559119350d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88b7fff640 1 -- 172.21.15.3:0/1872562435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f74670 msgr2=0x559119351250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.100+0000 7f88b7fff640 1 --2- 172.21.15.3:0/1872562435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f74670 0x559119351250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.101+0000 7f88b7fff640 1 -- 172.21.15.3:0/1872562435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559119351840 con 0x559118f53cd0 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.101+0000 7f88b7fff640 1 --2- 172.21.15.3:0/1872562435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f53cd0 0x559119354850 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f88a0000c00 tx=0x7f88a8011460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7faec8b6d93d751d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.101+0000 7f8897fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.101+0000 7f8897fff640 1 -- 172.21.15.3:0/1872562435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0035040 con 0x559118f53cd0 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.101+0000 7f8897fff640 1 -- 172.21.15.3:0/1872562435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f88a0003ae0 con 0x559118f53cd0 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.101+0000 7f88c5077740 1 -- 172.21.15.3:0/1872562435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a8029070 con 0x559118f53cd0 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.101+0000 7f8897fff640 1 -- 172.21.15.3:0/1872562435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f88a0003ce0 con 0x559118f53cd0 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.101+0000 7f88c5077740 10 client.16929 fetch_fsmap learned FSMap version 32 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.101+0000 7f88c5077740 1 -- 172.21.15.3:0/1872562435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5591193c9bf0 con 0x559118f53cd0 2024-09-21T06:19:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.101+0000 7f8897fff640 1 -- 172.21.15.3:0/1872562435 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a8029070 con 0x559118f53cd0 2024-09-21T06:19:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.102+0000 7f8897fff640 1 -- 172.21.15.3:0/1872562435 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f88a0053070 con 0x559118f53cd0 2024-09-21T06:19:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.102+0000 7f88c5077740 10 client.16929 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:19:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.102+0000 7f88c5077740 10 client.16929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.102+0000 7f88c5077740 20 client.16929 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.102+0000 7f88c5077740 1 --2- 172.21.15.3:0/1872562435 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559118f95350 0x5591193cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.102+0000 7f88c5077740 4 client.16929 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:19:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.102+0000 7f88c5077740 1 -- 172.21.15.3:0/1872562435 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5591192b0860 con 0x559118f95350 2024-09-21T06:19:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.102+0000 7f88bc89f640 1 --2- 172.21.15.3:0/1872562435 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559118f95350 0x5591193cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.102+0000 7f88bc89f640 1 --2- 172.21.15.3:0/1872562435 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559118f95350 0x5591193cc3e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce1abd6b42abe5d4 server_cookie=70a87adcc79456b0 in_seq=0 out_seq=0 2024-09-21T06:19:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.102+0000 7f8897fff640 10 client.16929 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:43.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.105+0000 7f8897fff640 1 -- 172.21.15.3:0/1872562435 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5591192b0860 con 0x559118f95350 2024-09-21T06:19:43.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.105+0000 7f8897fff640 10 client.16929 handle_command_reply: tid=0 2024-09-21T06:19:43.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.105+0000 7f88c5077740 1 client.16929 shutdown 2024-09-21T06:19:43.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.105+0000 7f88c5077740 1 -- 172.21.15.3:0/1872562435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f53cd0 msgr2=0x559119354850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.105+0000 7f88c5077740 1 --2- 172.21.15.3:0/1872562435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f53cd0 0x559119354850 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f88a0000c00 tx=0x7f88a8011460 comp rx=0 tx=0).stop 2024-09-21T06:19:43.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.105+0000 7f88c5077740 1 -- 172.21.15.3:0/1872562435 shutdown_connections 2024-09-21T06:19:43.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.105+0000 7f88c5077740 1 --2- 172.21.15.3:0/1872562435 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559118f95350 0x5591193cc3e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.105+0000 7f88c5077740 1 --2- 172.21.15.3:0/1872562435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559118f6c490 0x559119350d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.105+0000 7f8897fff640 0 client.16929 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:43.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.106+0000 7f88c5077740 1 --2- 172.21.15.3:0/1872562435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559118f74670 0x559119351250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.106+0000 7f88c5077740 1 --2- 172.21.15.3:0/1872562435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559118f53cd0 0x559119354850 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.106+0000 7f88c5077740 1 -- 172.21.15.3:0/1872562435 >> 172.21.15.3:0/1872562435 conn(0x559118f89740 msgr2=0x55911933de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:43.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.106+0000 7f88c5077740 1 -- 172.21.15.3:0/1872562435 shutdown_connections 2024-09-21T06:19:43.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.106+0000 7f88c5077740 1 -- 172.21.15.3:0/1872562435 wait complete. 2024-09-21T06:19:43.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.106+0000 7f88c5077740 20 client.16929 trim_cache size 0 max 16384 2024-09-21T06:19:43.110 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:19:43.110 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:19:43.110 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "fab678ea-0ea8-42c2-b8b6-dd890880c82f", 2024-09-21T06:19:43.110 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:19:43.110 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:19:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.107+0000 7f88beb03640 1 -- 172.21.15.3:0/3221964389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f88a40b6700 msgr2=0x7f88a40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.107+0000 7f88beb03640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f88a40b6700 0x7f88a40b8c10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f88b00e5780 tx=0x7f88b0079430 comp rx=0 tx=0).stop 2024-09-21T06:19:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.107+0000 7f88beb03640 1 -- 172.21.15.3:0/3221964389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88b8108cf0 msgr2=0x7f88b819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.107+0000 7f88beb03640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88b8108cf0 0x7f88b819fd40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f88b000e970 tx=0x7f88b000ed70 comp rx=0 tx=0).stop 2024-09-21T06:19:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.107+0000 7f88beb03640 1 -- 172.21.15.3:0/3221964389 shutdown_connections 2024-09-21T06:19:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.107+0000 7f88beb03640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f88a40b6700 0x7f88a40b8c10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.107+0000 7f88beb03640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88b8108cf0 0x7f88b819fd40 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.107+0000 7f88beb03640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88b81083a0 0x7f88b819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.107+0000 7f88beb03640 1 --2- 172.21.15.3:0/3221964389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b8102a60 0x7f88b819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.107+0000 7f88beb03640 1 -- 172.21.15.3:0/3221964389 >> 172.21.15.3:0/3221964389 conn(0x7f88b8079730 msgr2=0x7f88b806b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.107+0000 7f88beb03640 1 -- 172.21.15.3:0/3221964389 shutdown_connections 2024-09-21T06:19:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:43.107+0000 7f88beb03640 1 -- 172.21.15.3:0/3221964389 wait complete. 2024-09-21T06:19:43.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:43 smithi003 ceph-mon[31227]: pgmap v466: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 423 op/s 2024-09-21T06:19:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:43 smithi137 ceph-mon[34464]: pgmap v466: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 423 op/s 2024-09-21T06:19:43.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:43 smithi204 ceph-mon[34659]: pgmap v466: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 423 op/s 2024-09-21T06:19:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.748+0000 7fe8fdcdc640 1 -- 172.21.15.3:0/4077250986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f810dd60 msgr2=0x7fe8f8110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.748+0000 7fe8fdcdc640 1 --2- 172.21.15.3:0/4077250986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f810dd60 0x7fe8f8110250 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009ca0 tx=0x7fe8f002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.748+0000 7fe8fdcdc640 1 -- 172.21.15.3:0/4077250986 shutdown_connections 2024-09-21T06:19:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.748+0000 7fe8fdcdc640 1 --2- 172.21.15.3:0/4077250986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8f8105250 0x7fe8f810d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.748+0000 7fe8fdcdc640 1 --2- 172.21.15.3:0/4077250986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f810dd60 0x7fe8f8110250 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.748+0000 7fe8fdcdc640 1 --2- 172.21.15.3:0/4077250986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8f8104900 0x7fe8f8104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.748+0000 7fe8fdcdc640 1 -- 172.21.15.3:0/4077250986 >> 172.21.15.3:0/4077250986 conn(0x7fe8f807a740 msgr2=0x7fe8f807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.748+0000 7fe8fdcdc640 1 -- 172.21.15.3:0/4077250986 shutdown_connections 2024-09-21T06:19:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.748+0000 7fe8fdcdc640 1 -- 172.21.15.3:0/4077250986 wait complete. 2024-09-21T06:19:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.748+0000 7fe8fdcdc640 1 Processor -- start 2024-09-21T06:19:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.748+0000 7fe8fdcdc640 1 -- start start 2024-09-21T06:19:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8fdcdc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8f8104900 0x7fe8f819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8fdcdc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8105250 0x7fe8f819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8fdcdc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8f810dd60 0x7fe8f819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8fdcdc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81139c0 con 0x7fe8f8105250 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8fdcdc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8113840 con 0x7fe8f810dd60 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8fdcdc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8113b40 con 0x7fe8f8104900 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8eeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8105250 0x7fe8f819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8eeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8105250 0x7fe8f819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36462/0 (socket says 172.21.15.3:36462) 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8eeffd640 1 -- 172.21.15.3:0/1363412389 learned_addr learned my addr 172.21.15.3:0/1363412389 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8f77fe640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8f8104900 0x7fe8f819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8f7fff640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8f810dd60 0x7fe8f819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8eeffd640 1 -- 172.21.15.3:0/1363412389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8f8104900 msgr2=0x7fe8f819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8eeffd640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8f8104900 0x7fe8f819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8eeffd640 1 -- 172.21.15.3:0/1363412389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8f810dd60 msgr2=0x7fe8f819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8eeffd640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8f810dd60 0x7fe8f819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.749+0000 7fe8eeffd640 1 -- 172.21.15.3:0/1363412389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f81a0370 con 0x7fe8f8105250 2024-09-21T06:19:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.750+0000 7fe8eeffd640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8105250 0x7fe8f819a9e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000e9f0 tx=0x7fe8e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.750+0000 7fe8f57fa640 1 -- 172.21.15.3:0/1363412389 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e000ccc0 con 0x7fe8f8105250 2024-09-21T06:19:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.750+0000 7fe8f57fa640 1 -- 172.21.15.3:0/1363412389 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e00108e0 con 0x7fe8f8105250 2024-09-21T06:19:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.750+0000 7fe8fdcdc640 1 -- 172.21.15.3:0/1363412389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f81a0660 con 0x7fe8f8105250 2024-09-21T06:19:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.750+0000 7fe8f57fa640 1 -- 172.21.15.3:0/1363412389 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0018860 con 0x7fe8f8105250 2024-09-21T06:19:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.750+0000 7fe8fdcdc640 1 -- 172.21.15.3:0/1363412389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f81a6df0 con 0x7fe8f8105250 2024-09-21T06:19:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe8f57fa640 1 -- 172.21.15.3:0/1363412389 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fe8e0018a00 con 0x7fe8f8105250 2024-09-21T06:19:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe904c49740 1 Processor -- start 2024-09-21T06:19:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe904c49740 1 -- start start 2024-09-21T06:19:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe904c49740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320461b0 0x55f1323eb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe904c49740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f13201d2f0 0x55f1323ebef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe904c49740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1320254d0 0x55f1323ec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe904c49740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f1323606f0 con 0x55f1320461b0 2024-09-21T06:19:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe904c49740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f13234fb20 con 0x55f13201d2f0 2024-09-21T06:19:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe904c49740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f132353930 con 0x55f1320254d0 2024-09-21T06:19:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320461b0 0x55f1323eb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320461b0 0x55f1323eb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36476/0 (socket says 172.21.15.3:36476) 2024-09-21T06:19:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe8f7fff640 1 -- 172.21.15.3:0/3539194639 learned_addr learned my addr 172.21.15.3:0/3539194639 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe8f77fe640 1 --2- 172.21.15.3:0/3539194639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f13201d2f0 0x55f1323ebef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.753+0000 7fe8eeffd640 1 --2- 172.21.15.3:0/3539194639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1320254d0 0x55f1323ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.754+0000 7fe8f77fe640 1 -- 172.21.15.3:0/3539194639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1320254d0 msgr2=0x55f1323ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.754+0000 7fe8f77fe640 1 --2- 172.21.15.3:0/3539194639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1320254d0 0x55f1323ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.754+0000 7fe8f77fe640 1 -- 172.21.15.3:0/3539194639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320461b0 msgr2=0x55f1323eb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.754+0000 7fe8f77fe640 1 --2- 172.21.15.3:0/3539194639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320461b0 0x55f1323eb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.754+0000 7fe8f77fe640 1 -- 172.21.15.3:0/3539194639 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f132315530 con 0x55f13201d2f0 2024-09-21T06:19:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.754+0000 7fe8f57fa640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe8e40b6680 0x7fe8e40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.754+0000 7fe8f7fff640 1 --2- 172.21.15.3:0/3539194639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320461b0 0x55f1323eb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.754+0000 7fe8eeffd640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe8e40b6680 0x7fe8e40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.754+0000 7fe8f77fe640 1 --2- 172.21.15.3:0/3539194639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f13201d2f0 0x55f1323ebef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009c50 tx=0x7fe8e8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f96df81c5fc41fba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.754+0000 7fe8f57fa640 1 -- 172.21.15.3:0/1363412389 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe8e00e7910 con 0x7fe8f8105250 2024-09-21T06:19:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.754+0000 7fe8edffb640 1 -- 172.21.15.3:0/3539194639 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8004050 con 0x55f13201d2f0 2024-09-21T06:19:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.754+0000 7fe8edffb640 1 -- 172.21.15.3:0/3539194639 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e80054d0 con 0x55f13201d2f0 2024-09-21T06:19:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.754+0000 7fe8edffb640 1 -- 172.21.15.3:0/3539194639 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8004cd0 con 0x55f13201d2f0 2024-09-21T06:19:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.755+0000 7fe904c49740 1 -- 172.21.15.3:0/3539194639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f13201d2f0 msgr2=0x55f1323ebef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.755+0000 7fe8eeffd640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe8e40b6680 0x7fe8e40b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00e5150 tx=0x7fe8e00ed550 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.755+0000 7fe904c49740 1 --2- 172.21.15.3:0/3539194639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f13201d2f0 0x55f1323ebef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009c50 tx=0x7fe8e8011810 comp rx=0 tx=0).stop 2024-09-21T06:19:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.755+0000 7fe904c49740 1 -- 172.21.15.3:0/3539194639 shutdown_connections 2024-09-21T06:19:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.755+0000 7fe904c49740 1 --2- 172.21.15.3:0/3539194639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1320254d0 0x55f1323ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.755+0000 7fe904c49740 1 --2- 172.21.15.3:0/3539194639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f13201d2f0 0x55f1323ebef0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.755+0000 7fe904c49740 1 --2- 172.21.15.3:0/3539194639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320461b0 0x55f1323eb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.755+0000 7fe904c49740 1 -- 172.21.15.3:0/3539194639 >> 172.21.15.3:0/3539194639 conn(0x55f13203a300 msgr2=0x55f1323e98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.755+0000 7fe904c49740 1 -- 172.21.15.3:0/3539194639 shutdown_connections 2024-09-21T06:19:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.755+0000 7fe904c49740 1 -- 172.21.15.3:0/3539194639 wait complete. 2024-09-21T06:19:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe904c49740 1 -- start start 2024-09-21T06:19:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe904c49740 1 Processor -- start 2024-09-21T06:19:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe904c49740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f13201d2f0 0x55f1323f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe904c49740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 0x55f1323f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe904c49740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320461b0 0x55f132403450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe904c49740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f132287ab0 con 0x55f1320254d0 2024-09-21T06:19:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe904c49740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f1322f5730 con 0x55f1320461b0 2024-09-21T06:19:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe904c49740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f1323fa130 con 0x55f13201d2f0 2024-09-21T06:19:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe8f77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 0x55f1323f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe904c49740 10 client.0 _ll_register_callbacks cb 0x55f1323496b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe8f77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 0x55f1323f75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36484/0 (socket says 172.21.15.3:36484) 2024-09-21T06:19:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe8f77fe640 1 -- 172.21.15.3:0/3151833268 learned_addr learned my addr 172.21.15.3:0/3151833268 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe8f7fff640 1 --2- 172.21.15.3:0/3151833268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f13201d2f0 0x55f1323f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.756+0000 7fe8f7fff640 1 --2- 172.21.15.3:0/3151833268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320461b0 0x55f132403450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.757+0000 7fe8f77fe640 1 -- 172.21.15.3:0/3151833268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f13201d2f0 msgr2=0x55f1323f70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.757+0000 7fe8f77fe640 1 --2- 172.21.15.3:0/3151833268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f13201d2f0 0x55f1323f70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.757+0000 7fe8f77fe640 1 -- 172.21.15.3:0/3151833268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320461b0 msgr2=0x55f132403450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.757+0000 7fe8f77fe640 1 --2- 172.21.15.3:0/3151833268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320461b0 0x55f132403450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.757+0000 7fe8f77fe640 1 -- 172.21.15.3:0/3151833268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1320b1790 con 0x55f1320254d0 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.757+0000 7fe8f77fe640 1 --2- 172.21.15.3:0/3151833268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 0x55f1323f75f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000c00 tx=0x7fe8e8005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a038523f2388a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.757+0000 7fe8d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.757+0000 7fe8d6ffd640 1 -- 172.21.15.3:0/3151833268 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8027070 con 0x55f1320254d0 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.757+0000 7fe904c49740 1 -- 172.21.15.3:0/3151833268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e80054d0 con 0x55f1320254d0 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.757+0000 7fe8d6ffd640 1 -- 172.21.15.3:0/3151833268 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe8e8003800 con 0x55f1320254d0 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.757+0000 7fe8d6ffd640 1 -- 172.21.15.3:0/3151833268 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe8e8033050 con 0x55f1320254d0 2024-09-21T06:19:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.757+0000 7fe904c49740 10 client.16941 fetch_fsmap learned FSMap version 32 2024-09-21T06:19:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.757+0000 7fe904c49740 1 -- 172.21.15.3:0/3151833268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f13212a620 con 0x55f1320254d0 2024-09-21T06:19:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.758+0000 7fe8d6ffd640 1 -- 172.21.15.3:0/3151833268 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e80054d0 con 0x55f1320254d0 2024-09-21T06:19:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.758+0000 7fe8d6ffd640 1 -- 172.21.15.3:0/3151833268 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fe8e8005090 con 0x55f1320254d0 2024-09-21T06:19:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.758+0000 7fe904c49740 10 client.16941 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:19:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.758+0000 7fe904c49740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.758+0000 7fe904c49740 20 client.16941 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.758+0000 7fe904c49740 1 --2- 172.21.15.3:0/3151833268 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f132004d80 0x55f1324090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.758+0000 7fe904c49740 4 client.16941 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.758+0000 7fe904c49740 1 -- 172.21.15.3:0/3151833268 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f132362640 con 0x55f132004d80 2024-09-21T06:19:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.758+0000 7fe8f7fff640 1 --2- 172.21.15.3:0/3151833268 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f132004d80 0x55f1324090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.759+0000 7fe8f7fff640 1 --2- 172.21.15.3:0/3151833268 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f132004d80 0x55f1324090f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee238fe008eafd28 server_cookie=4430ed463f90b4c7 in_seq=0 out_seq=0 2024-09-21T06:19:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.759+0000 7fe8d6ffd640 10 client.16941 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.762+0000 7fe8d6ffd640 1 -- 172.21.15.3:0/3151833268 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f132362640 con 0x55f132004d80 2024-09-21T06:19:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe8d6ffd640 10 client.16941 handle_command_reply: tid=0 2024-09-21T06:19:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe904c49740 1 client.16941 shutdown 2024-09-21T06:19:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe904c49740 1 -- 172.21.15.3:0/3151833268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 msgr2=0x55f1323f75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe904c49740 1 --2- 172.21.15.3:0/3151833268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 0x55f1323f75f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000c00 tx=0x7fe8e8005760 comp rx=0 tx=0).stop 2024-09-21T06:19:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe904c49740 1 -- 172.21.15.3:0/3151833268 shutdown_connections 2024-09-21T06:19:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe904c49740 1 --2- 172.21.15.3:0/3151833268 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f132004d80 0x55f1324090f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe904c49740 1 --2- 172.21.15.3:0/3151833268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320461b0 0x55f132403450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe904c49740 1 --2- 172.21.15.3:0/3151833268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 0x55f1323f75f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe8d6ffd640 0 client.16941 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe904c49740 1 --2- 172.21.15.3:0/3151833268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f13201d2f0 0x55f1323f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe904c49740 1 -- 172.21.15.3:0/3151833268 >> 172.21.15.3:0/3151833268 conn(0x55f13203a300 msgr2=0x55f1323ec9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe904c49740 1 -- 172.21.15.3:0/3151833268 shutdown_connections 2024-09-21T06:19:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe904c49740 1 -- 172.21.15.3:0/3151833268 wait complete. 2024-09-21T06:19:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.763+0000 7fe904c49740 20 client.16941 trim_cache size 0 max 16384 2024-09-21T06:19:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.779+0000 7fe904c49740 1 Processor -- start 2024-09-21T06:19:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.779+0000 7fe904c49740 1 -- start start 2024-09-21T06:19:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.779+0000 7fe904c49740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f13201d2f0 0x55f1323f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.779+0000 7fe904c49740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320254d0 0x55f1323f7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.779+0000 7fe904c49740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1320461b0 0x55f1323f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.779+0000 7fe904c49740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81af6b0 con 0x55f13201d2f0 2024-09-21T06:19:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.779+0000 7fe904c49740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81b1220 con 0x55f1320254d0 2024-09-21T06:19:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.779+0000 7fe904c49740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81afd80 con 0x55f1320461b0 2024-09-21T06:19:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.779+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f13201d2f0 0x55f1323f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.779+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f13201d2f0 0x55f1323f11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36500/0 (socket says 172.21.15.3:36500) 2024-09-21T06:19:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.779+0000 7fe8f7fff640 1 -- 172.21.15.3:0/1418425762 learned_addr learned my addr 172.21.15.3:0/1418425762 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.779+0000 7fe8f77fe640 1 --2- 172.21.15.3:0/1418425762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320254d0 0x55f1323f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.779+0000 7fe8f7fff640 1 --2- 172.21.15.3:0/1418425762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1320461b0 0x55f1323f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.780+0000 7fe8f7fff640 1 -- 172.21.15.3:0/1418425762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1320461b0 msgr2=0x55f1323f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.780+0000 7fe8f7fff640 1 --2- 172.21.15.3:0/1418425762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1320461b0 0x55f1323f27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.780+0000 7fe8f7fff640 1 -- 172.21.15.3:0/1418425762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320254d0 msgr2=0x55f1323f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.780+0000 7fe8f7fff640 1 --2- 172.21.15.3:0/1418425762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320254d0 0x55f1323f7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.780+0000 7fe8f7fff640 1 -- 172.21.15.3:0/1418425762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1323f7f10 con 0x55f13201d2f0 2024-09-21T06:19:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.780+0000 7fe8f7fff640 1 --2- 172.21.15.3:0/1418425762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f13201d2f0 0x55f1323f11f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009600 tx=0x7fe8f002f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91a5b54de5ea6d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.780+0000 7fe8ed7fa640 1 -- 172.21.15.3:0/1418425762 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0046040 con 0x55f13201d2f0 2024-09-21T06:19:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.780+0000 7fe8ed7fa640 1 -- 172.21.15.3:0/1418425762 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8f0036040 con 0x55f13201d2f0 2024-09-21T06:19:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.780+0000 7fe8ed7fa640 1 -- 172.21.15.3:0/1418425762 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0004450 con 0x55f13201d2f0 2024-09-21T06:19:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.781+0000 7fe904c49740 1 -- 172.21.15.3:0/1418425762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f13201d2f0 msgr2=0x55f1323f11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.781+0000 7fe904c49740 1 --2- 172.21.15.3:0/1418425762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f13201d2f0 0x55f1323f11f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009600 tx=0x7fe8f002f410 comp rx=0 tx=0).stop 2024-09-21T06:19:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.781+0000 7fe904c49740 1 -- 172.21.15.3:0/1418425762 shutdown_connections 2024-09-21T06:19:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.781+0000 7fe904c49740 1 --2- 172.21.15.3:0/1418425762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1320461b0 0x55f1323f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.781+0000 7fe904c49740 1 --2- 172.21.15.3:0/1418425762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320254d0 0x55f1323f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.781+0000 7fe904c49740 1 --2- 172.21.15.3:0/1418425762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f13201d2f0 0x55f1323f11f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.781+0000 7fe904c49740 1 -- 172.21.15.3:0/1418425762 >> 172.21.15.3:0/1418425762 conn(0x55f13203a300 msgr2=0x55f1323f0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.781+0000 7fe904c49740 1 -- 172.21.15.3:0/1418425762 shutdown_connections 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.781+0000 7fe904c49740 1 -- 172.21.15.3:0/1418425762 wait complete. 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.781+0000 7fe904c49740 1 -- start start 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe904c49740 1 Processor -- start 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe904c49740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f13201d2f0 0x55f1323facb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe904c49740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 0x55f1323fb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe904c49740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320461b0 0x55f13246b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe904c49740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81b0b50 con 0x55f1320254d0 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe904c49740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81b0480 con 0x55f1320461b0 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe904c49740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81ad690 con 0x55f13201d2f0 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe904c49740 10 client.0 _ll_register_callbacks cb 0x7fe8e0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe8f77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 0x55f1323fb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe8f77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 0x55f1323fb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36508/0 (socket says 172.21.15.3:36508) 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f13201d2f0 0x55f1323facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe8f77fe640 1 -- 172.21.15.3:0/1090794577 learned_addr learned my addr 172.21.15.3:0/1090794577 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe8f7fff640 1 --2- 172.21.15.3:0/1090794577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320461b0 0x55f13246b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe8f77fe640 1 -- 172.21.15.3:0/1090794577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f13201d2f0 msgr2=0x55f1323facb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe8f77fe640 1 --2- 172.21.15.3:0/1090794577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f13201d2f0 0x55f1323facb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe8f77fe640 1 -- 172.21.15.3:0/1090794577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320461b0 msgr2=0x55f13246b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe8f77fe640 1 --2- 172.21.15.3:0/1090794577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320461b0 0x55f13246b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.782+0000 7fe8f77fe640 1 -- 172.21.15.3:0/1090794577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f13246c020 con 0x55f1320254d0 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.783+0000 7fe8f77fe640 1 --2- 172.21.15.3:0/1090794577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 0x55f1323fb270 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009730 tx=0x7fe8e8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37446497f93f97ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.783+0000 7fe8d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.783+0000 7fe8d6ffd640 1 -- 172.21.15.3:0/1090794577 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8027070 con 0x55f1320254d0 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.783+0000 7fe904c49740 1 -- 172.21.15.3:0/1090794577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8f0036040 con 0x55f1320254d0 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.783+0000 7fe8d6ffd640 1 -- 172.21.15.3:0/1090794577 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe8e8023a20 con 0x55f1320254d0 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.783+0000 7fe8d6ffd640 1 -- 172.21.15.3:0/1090794577 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe8e8023c20 con 0x55f1320254d0 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.783+0000 7fe904c49740 10 client.16947 fetch_fsmap learned FSMap version 32 2024-09-21T06:19:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.783+0000 7fe904c49740 1 -- 172.21.15.3:0/1090794577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f132473350 con 0x55f1320254d0 2024-09-21T06:19:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.783+0000 7fe8d6ffd640 1 -- 172.21.15.3:0/1090794577 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8f0036040 con 0x55f1320254d0 2024-09-21T06:19:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.783+0000 7fe8d6ffd640 1 -- 172.21.15.3:0/1090794577 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fe8e8038560 con 0x55f1320254d0 2024-09-21T06:19:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.784+0000 7fe904c49740 10 client.16947 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:19:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.784+0000 7fe904c49740 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.784+0000 7fe904c49740 20 client.16947 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.784+0000 7fe904c49740 1 --2- 172.21.15.3:0/1090794577 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f131fda1c0 0x55f132475aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.784+0000 7fe904c49740 4 client.16947 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:19:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.784+0000 7fe904c49740 1 -- 172.21.15.3:0/1090794577 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f13235e0f0 con 0x55f131fda1c0 2024-09-21T06:19:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.784+0000 7fe8f7fff640 1 --2- 172.21.15.3:0/1090794577 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f131fda1c0 0x55f132475aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.784+0000 7fe8d6ffd640 10 client.16947 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.784+0000 7fe8f7fff640 1 --2- 172.21.15.3:0/1090794577 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f131fda1c0 0x55f132475aa0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf6a250da5a7b195 server_cookie=fee40dff6119a1f2 in_seq=0 out_seq=0 2024-09-21T06:19:43.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 468795 12.55 MB/sec execute 76 sec latency 52.189 ms 2024-09-21T06:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe8d6ffd640 1 -- 172.21.15.3:0/1090794577 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f13235e0f0 con 0x55f131fda1c0 2024-09-21T06:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe8d6ffd640 10 client.16947 handle_command_reply: tid=0 2024-09-21T06:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe904c49740 1 client.16947 shutdown 2024-09-21T06:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe904c49740 1 -- 172.21.15.3:0/1090794577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 msgr2=0x55f1323fb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe904c49740 1 --2- 172.21.15.3:0/1090794577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 0x55f1323fb270 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009730 tx=0x7fe8e8005f10 comp rx=0 tx=0).stop 2024-09-21T06:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe904c49740 1 -- 172.21.15.3:0/1090794577 shutdown_connections 2024-09-21T06:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe904c49740 1 --2- 172.21.15.3:0/1090794577 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f131fda1c0 0x55f132475aa0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe904c49740 1 --2- 172.21.15.3:0/1090794577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1320461b0 0x55f13246b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe8d6ffd640 0 client.16947 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe904c49740 1 --2- 172.21.15.3:0/1090794577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1320254d0 0x55f1323fb270 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe904c49740 1 --2- 172.21.15.3:0/1090794577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f13201d2f0 0x55f1323facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe904c49740 1 -- 172.21.15.3:0/1090794577 >> 172.21.15.3:0/1090794577 conn(0x55f13203a300 msgr2=0x55f1323f0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe904c49740 1 -- 172.21.15.3:0/1090794577 shutdown_connections 2024-09-21T06:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe904c49740 1 -- 172.21.15.3:0/1090794577 wait complete. 2024-09-21T06:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.957+0000 7fe904c49740 20 client.16947 trim_cache size 0 max 16384 2024-09-21T06:19:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:19:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:19:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:19:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.958+0000 7fe8fdcdc640 1 -- 172.21.15.3:0/1363412389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe8e40b6680 msgr2=0x7fe8e40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.958+0000 7fe8fdcdc640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe8e40b6680 0x7fe8e40b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00e5150 tx=0x7fe8e00ed550 comp rx=0 tx=0).stop 2024-09-21T06:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.958+0000 7fe8fdcdc640 1 -- 172.21.15.3:0/1363412389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8105250 msgr2=0x7fe8f819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.958+0000 7fe8fdcdc640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8105250 0x7fe8f819a9e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000e9f0 tx=0x7fe8e000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.958+0000 7fe8fdcdc640 1 -- 172.21.15.3:0/1363412389 shutdown_connections 2024-09-21T06:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.958+0000 7fe8fdcdc640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe8e40b6680 0x7fe8e40b8b90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.958+0000 7fe8fdcdc640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8f810dd60 0x7fe8f819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.958+0000 7fe8fdcdc640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8105250 0x7fe8f819a9e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.958+0000 7fe8fdcdc640 1 --2- 172.21.15.3:0/1363412389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8f8104900 0x7fe8f819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.958+0000 7fe8fdcdc640 1 -- 172.21.15.3:0/1363412389 >> 172.21.15.3:0/1363412389 conn(0x7fe8f807a740 msgr2=0x7fe8f80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.958+0000 7fe8fdcdc640 1 -- 172.21.15.3:0/1363412389 shutdown_connections 2024-09-21T06:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:43.958+0000 7fe8fdcdc640 1 -- 172.21.15.3:0/1363412389 wait complete. 2024-09-21T06:19:43.996 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:19:44.264 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:44.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:44 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:19:44.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:44 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:44.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:44 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:19:44.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:44 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:19:44.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:44 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:44.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:44 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:19:44.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:44 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:44.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:44 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:19:44.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:44 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:19:44.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:44 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:44.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:44 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:19:44.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:44 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:44.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:44 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:19:44.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:44 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:19:44.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:44 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:44.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.783+0000 7fcc0ba0a640 1 -- 172.21.15.3:0/3141551267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc041083a0 msgr2=0x7fcc041087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:44.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.783+0000 7fcc0ba0a640 1 --2- 172.21.15.3:0/3141551267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc041083a0 0x7fcc041087b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009900 tx=0x7fcbf402a410 comp rx=0 tx=0).stop 2024-09-21T06:19:44.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.783+0000 7fcc0ba0a640 1 -- 172.21.15.3:0/3141551267 shutdown_connections 2024-09-21T06:19:44.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.783+0000 7fcc0ba0a640 1 --2- 172.21.15.3:0/3141551267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc04108cf0 0x7fcc0410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.783+0000 7fcc0ba0a640 1 --2- 172.21.15.3:0/3141551267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc041083a0 0x7fcc041087b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.783+0000 7fcc0ba0a640 1 --2- 172.21.15.3:0/3141551267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc04102a60 0x7fcc04102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.783+0000 7fcc0ba0a640 1 -- 172.21.15.3:0/3141551267 >> 172.21.15.3:0/3141551267 conn(0x7fcc04079730 msgr2=0x7fcc04079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:44.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.784+0000 7fcc0ba0a640 1 -- 172.21.15.3:0/3141551267 shutdown_connections 2024-09-21T06:19:44.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.784+0000 7fcc0ba0a640 1 -- 172.21.15.3:0/3141551267 wait complete. 2024-09-21T06:19:44.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.784+0000 7fcc0ba0a640 1 Processor -- start 2024-09-21T06:19:44.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.784+0000 7fcc0ba0a640 1 -- start start 2024-09-21T06:19:44.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc0ba0a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc04102a60 0x7fcc0419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc0ba0a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc041083a0 0x7fcc0419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc0ba0a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc04108cf0 0x7fcc0419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc0ba0a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04112960 con 0x7fcc04108cf0 2024-09-21T06:19:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc0ba0a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041127e0 con 0x7fcc041083a0 2024-09-21T06:19:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc0ba0a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04112ae0 con 0x7fcc04102a60 2024-09-21T06:19:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc09fa7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc04108cf0 0x7fcc0419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc09fa7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc04108cf0 0x7fcc0419fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36530/0 (socket says 172.21.15.3:36530) 2024-09-21T06:19:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc09fa7640 1 -- 172.21.15.3:0/2383681734 learned_addr learned my addr 172.21.15.3:0/2383681734 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc041083a0 0x7fcc0419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc097a6640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc04102a60 0x7fcc0419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc09fa7640 1 -- 172.21.15.3:0/2383681734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc04102a60 msgr2=0x7fcc0419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc09fa7640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc04102a60 0x7fcc0419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc09fa7640 1 -- 172.21.15.3:0/2383681734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc041083a0 msgr2=0x7fcc0419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:44.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc09fa7640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc041083a0 0x7fcc0419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.785+0000 7fcc09fa7640 1 -- 172.21.15.3:0/2383681734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc041a0410 con 0x7fcc04108cf0 2024-09-21T06:19:44.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.786+0000 7fcc09fa7640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc04108cf0 0x7fcc0419fce0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00eaa0 tx=0x7fcbfc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:44.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.786+0000 7fcbfa7fc640 1 -- 172.21.15.3:0/2383681734 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc00cdb0 con 0x7fcc04108cf0 2024-09-21T06:19:44.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.786+0000 7fcbfa7fc640 1 -- 172.21.15.3:0/2383681734 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbfc005020 con 0x7fcc04108cf0 2024-09-21T06:19:44.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.786+0000 7fcc0ba0a640 1 -- 172.21.15.3:0/2383681734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc041a0700 con 0x7fcc04108cf0 2024-09-21T06:19:44.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.786+0000 7fcbfa7fc640 1 -- 172.21.15.3:0/2383681734 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc0116d0 con 0x7fcc04108cf0 2024-09-21T06:19:44.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.786+0000 7fcc0ba0a640 1 -- 172.21.15.3:0/2383681734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc041a6e60 con 0x7fcc04108cf0 2024-09-21T06:19:44.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.789+0000 7fcc11f7e740 1 Processor -- start 2024-09-21T06:19:44.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.789+0000 7fcbfa7fc640 1 -- 172.21.15.3:0/2383681734 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fcbfc0051d0 con 0x7fcc04108cf0 2024-09-21T06:19:44.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.789+0000 7fcc11f7e740 1 -- start start 2024-09-21T06:19:44.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc11f7e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf2690070 0x562bf2a37650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc11f7e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf26671b0 0x562bf2a37b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc11f7e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf266f390 0x562bf2a380d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc11f7e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562bf29ade50 con 0x562bf2690070 2024-09-21T06:19:44.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc11f7e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562bf299d280 con 0x562bf26671b0 2024-09-21T06:19:44.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc11f7e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562bf29a1090 con 0x562bf266f390 2024-09-21T06:19:44.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc08fa5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf2690070 0x562bf2a37650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc097a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf26671b0 0x562bf2a37b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc08fa5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf2690070 0x562bf2a37650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36540/0 (socket says 172.21.15.3:36540) 2024-09-21T06:19:44.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc08fa5640 1 -- 172.21.15.3:0/2111079432 learned_addr learned my addr 172.21.15.3:0/2111079432 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:44.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/2111079432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf266f390 0x562bf2a380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcbfa7fc640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcbf00b6680 0x7fcbf00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc097a6640 1 -- 172.21.15.3:0/2111079432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf266f390 msgr2=0x562bf2a380d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:44.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc097a6640 1 --2- 172.21.15.3:0/2111079432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf266f390 0x562bf2a380d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc097a6640 1 -- 172.21.15.3:0/2111079432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf2690070 msgr2=0x562bf2a37650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:44.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc097a6640 1 --2- 172.21.15.3:0/2111079432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf2690070 0x562bf2a37650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc097a6640 1 -- 172.21.15.3:0/2111079432 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bf26fb650 con 0x562bf26671b0 2024-09-21T06:19:44.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.790+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/2111079432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf2690070 0x562bf2a37650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:44.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.791+0000 7fcbfa7fc640 1 -- 172.21.15.3:0/2383681734 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcbfc0e7830 con 0x7fcc04108cf0 2024-09-21T06:19:44.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.791+0000 7fcc09fa7640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcbf00b6680 0x7fcbf00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.791+0000 7fcc097a6640 1 --2- 172.21.15.3:0/2111079432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf26671b0 0x562bf2a37b90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc00004630 tx=0x7fcc0000bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbb45d8e4e06688b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:44.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.791+0000 7fcbef7fe640 1 -- 172.21.15.3:0/2111079432 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00003bd0 con 0x562bf26671b0 2024-09-21T06:19:44.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.791+0000 7fcbef7fe640 1 -- 172.21.15.3:0/2111079432 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc00015070 con 0x562bf26671b0 2024-09-21T06:19:44.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.791+0000 7fcbef7fe640 1 -- 172.21.15.3:0/2111079432 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00004c00 con 0x562bf26671b0 2024-09-21T06:19:44.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.792+0000 7fcc11f7e740 1 -- 172.21.15.3:0/2111079432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf26671b0 msgr2=0x562bf2a37b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:44.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.792+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/2111079432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf26671b0 0x562bf2a37b90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc00004630 tx=0x7fcc0000bb20 comp rx=0 tx=0).stop 2024-09-21T06:19:44.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.792+0000 7fcc09fa7640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcbf00b6680 0x7fcbf00b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0ad470 tx=0x7fcbfc00cf50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:44.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.792+0000 7fcc11f7e740 1 -- 172.21.15.3:0/2111079432 shutdown_connections 2024-09-21T06:19:44.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.792+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/2111079432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf266f390 0x562bf2a380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.792+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/2111079432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf26671b0 0x562bf2a37b90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.792+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/2111079432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf2690070 0x562bf2a37650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.792+0000 7fcc11f7e740 1 -- 172.21.15.3:0/2111079432 >> 172.21.15.3:0/2111079432 conn(0x562bf2684460 msgr2=0x562bf2a35540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.792+0000 7fcc11f7e740 1 -- 172.21.15.3:0/2111079432 shutdown_connections 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.792+0000 7fcc11f7e740 1 -- 172.21.15.3:0/2111079432 wait complete. 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.792+0000 7fcc11f7e740 1 -- start start 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.792+0000 7fcc11f7e740 1 Processor -- start 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc11f7e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf26671b0 0x562bf2a470f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc11f7e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf266f390 0x562bf2a47630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc11f7e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf2690070 0x562bf2a53490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc11f7e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562bf27fe700 con 0x562bf26671b0 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc11f7e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562bf2938110 con 0x562bf266f390 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc11f7e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562bf2a4a130 con 0x562bf2690070 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc11f7e740 10 client.0 _ll_register_callbacks cb 0x562bf2951990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc08fa5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf26671b0 0x562bf2a470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc097a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf266f390 0x562bf2a47630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc08fa5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf26671b0 0x562bf2a470f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36550/0 (socket says 172.21.15.3:36550) 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc08fa5640 1 -- 172.21.15.3:0/132259026 learned_addr learned my addr 172.21.15.3:0/132259026 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/132259026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf2690070 0x562bf2a53490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc08fa5640 1 -- 172.21.15.3:0/132259026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf2690070 msgr2=0x562bf2a53490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/132259026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf2690070 0x562bf2a53490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc08fa5640 1 -- 172.21.15.3:0/132259026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf266f390 msgr2=0x562bf2a47630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/132259026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf266f390 0x562bf2a47630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc08fa5640 1 -- 172.21.15.3:0/132259026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bf2965c40 con 0x562bf26671b0 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.793+0000 7fcc097a6640 1 --2- 172.21.15.3:0/132259026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf266f390 0x562bf2a47630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.794+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/132259026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf26671b0 0x562bf2a470f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009230 tx=0x7fcbf4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c1b8f597a5cfda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.794+0000 7fcbecff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.794+0000 7fcbecff9640 1 -- 172.21.15.3:0/132259026 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4035030 con 0x562bf26671b0 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.794+0000 7fcbecff9640 1 -- 172.21.15.3:0/132259026 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcbf4005050 con 0x562bf26671b0 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.794+0000 7fcc11f7e740 1 -- 172.21.15.3:0/132259026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc00015070 con 0x562bf26671b0 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.794+0000 7fcbecff9640 1 -- 172.21.15.3:0/132259026 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcbf40044d0 con 0x562bf26671b0 2024-09-21T06:19:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.794+0000 7fcc11f7e740 10 client.16959 fetch_fsmap learned FSMap version 32 2024-09-21T06:19:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.794+0000 7fcc11f7e740 1 -- 172.21.15.3:0/132259026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562bf2a55e50 con 0x562bf26671b0 2024-09-21T06:19:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.794+0000 7fcbecff9640 1 -- 172.21.15.3:0/132259026 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc00015070 con 0x562bf26671b0 2024-09-21T06:19:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.794+0000 7fcbecff9640 1 -- 172.21.15.3:0/132259026 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fcbf40035f0 con 0x562bf26671b0 2024-09-21T06:19:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.795+0000 7fcc11f7e740 10 client.16959 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:19:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.795+0000 7fcc11f7e740 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.795+0000 7fcc11f7e740 20 client.16959 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.795+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/132259026 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562bf264e9f0 0x562bf2a59290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.795+0000 7fcc11f7e740 4 client.16959 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.795+0000 7fcc11f7e740 1 -- 172.21.15.3:0/132259026 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bf29afda0 con 0x562bf264e9f0 2024-09-21T06:19:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.795+0000 7fcc097a6640 1 --2- 172.21.15.3:0/132259026 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562bf264e9f0 0x562bf2a59290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.795+0000 7fcbecff9640 10 client.16959 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.795+0000 7fcc097a6640 1 --2- 172.21.15.3:0/132259026 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562bf264e9f0 0x562bf2a59290 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28eeaf7df8cd66 server_cookie=2ee70aff496a9fc4 in_seq=0 out_seq=0 2024-09-21T06:19:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.796+0000 7fcbecff9640 1 -- 172.21.15.3:0/132259026 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562bf29afda0 con 0x562bf264e9f0 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.796+0000 7fcbecff9640 10 client.16959 handle_command_reply: tid=0 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.796+0000 7fcc11f7e740 1 client.16959 shutdown 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.796+0000 7fcc11f7e740 1 -- 172.21.15.3:0/132259026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf26671b0 msgr2=0x562bf2a470f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.796+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/132259026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf26671b0 0x562bf2a470f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009230 tx=0x7fcbf4031400 comp rx=0 tx=0).stop 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.796+0000 7fcc11f7e740 1 -- 172.21.15.3:0/132259026 shutdown_connections 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.796+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/132259026 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562bf264e9f0 0x562bf2a59290 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.796+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/132259026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf2690070 0x562bf2a53490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.796+0000 7fcbecff9640 0 client.16959 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.796+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/132259026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf266f390 0x562bf2a47630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.796+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/132259026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf26671b0 0x562bf2a470f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.796+0000 7fcc11f7e740 1 -- 172.21.15.3:0/132259026 >> 172.21.15.3:0/132259026 conn(0x562bf2684460 msgr2=0x562bf2a33b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.797+0000 7fcc11f7e740 1 -- 172.21.15.3:0/132259026 shutdown_connections 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.797+0000 7fcc11f7e740 1 -- 172.21.15.3:0/132259026 wait complete. 2024-09-21T06:19:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.797+0000 7fcc11f7e740 20 client.16959 trim_cache size 0 max 16384 2024-09-21T06:19:44.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.816+0000 7fcc11f7e740 1 Processor -- start 2024-09-21T06:19:44.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.816+0000 7fcc11f7e740 1 -- start start 2024-09-21T06:19:44.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.816+0000 7fcc11f7e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf264e9f0 0x562bf2a475b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.816+0000 7fcc11f7e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf26671b0 0x562bf2a53390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.816+0000 7fcc11f7e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf266f390 0x562bf2a442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.816+0000 7fcc11f7e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041ad430 con 0x562bf266f390 2024-09-21T06:19:44.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.816+0000 7fcc11f7e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041aefa0 con 0x562bf26671b0 2024-09-21T06:19:44.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.816+0000 7fcc11f7e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041adb00 con 0x562bf264e9f0 2024-09-21T06:19:44.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.816+0000 7fcc08fa5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf266f390 0x562bf2a442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.816+0000 7fcc08fa5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf266f390 0x562bf2a442e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36554/0 (socket says 172.21.15.3:36554) 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.816+0000 7fcc097a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf26671b0 0x562bf2a53390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.816+0000 7fcc08fa5640 1 -- 172.21.15.3:0/3050069617 learned_addr learned my addr 172.21.15.3:0/3050069617 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.816+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/3050069617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf264e9f0 0x562bf2a475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.817+0000 7fcc08fa5640 1 -- 172.21.15.3:0/3050069617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf264e9f0 msgr2=0x562bf2a475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.817+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/3050069617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf264e9f0 0x562bf2a475b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.817+0000 7fcc08fa5640 1 -- 172.21.15.3:0/3050069617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf26671b0 msgr2=0x562bf2a53390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.817+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/3050069617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf26671b0 0x562bf2a53390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.817+0000 7fcc08fa5640 1 -- 172.21.15.3:0/3050069617 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bf2a46050 con 0x562bf266f390 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.817+0000 7fcc097a6640 1 --2- 172.21.15.3:0/3050069617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf26671b0 0x562bf2a53390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.817+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/3050069617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf266f390 0x562bf2a442e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4032c90 tx=0x7fcbf4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cca727b4b33374 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.817+0000 7fcbeeffd640 1 -- 172.21.15.3:0/3050069617 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf40042d0 con 0x562bf266f390 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.817+0000 7fcbeeffd640 1 -- 172.21.15.3:0/3050069617 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf405b040 con 0x562bf266f390 2024-09-21T06:19:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.817+0000 7fcbeeffd640 1 -- 172.21.15.3:0/3050069617 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4032260 con 0x562bf266f390 2024-09-21T06:19:44.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.818+0000 7fcc11f7e740 1 -- 172.21.15.3:0/3050069617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf266f390 msgr2=0x562bf2a442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:44.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.818+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/3050069617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf266f390 0x562bf2a442e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4032c90 tx=0x7fcbf4009560 comp rx=0 tx=0).stop 2024-09-21T06:19:44.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.818+0000 7fcc11f7e740 1 -- 172.21.15.3:0/3050069617 shutdown_connections 2024-09-21T06:19:44.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.818+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/3050069617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf266f390 0x562bf2a442e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.818+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/3050069617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf26671b0 0x562bf2a53390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.818+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/3050069617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf264e9f0 0x562bf2a475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.818+0000 7fcc11f7e740 1 -- 172.21.15.3:0/3050069617 >> 172.21.15.3:0/3050069617 conn(0x562bf2684460 msgr2=0x562bf2a46ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:44.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.818+0000 7fcc11f7e740 1 -- 172.21.15.3:0/3050069617 shutdown_connections 2024-09-21T06:19:44.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.818+0000 7fcc11f7e740 1 -- 172.21.15.3:0/3050069617 wait complete. 2024-09-21T06:19:44.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc11f7e740 1 -- start start 2024-09-21T06:19:44.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc11f7e740 1 Processor -- start 2024-09-21T06:19:44.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc11f7e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf264e9f0 0x562bf2a4b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc11f7e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf26671b0 0x562bf2a4bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc11f7e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf266f390 0x562bf2ac2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc11f7e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041ae8d0 con 0x562bf264e9f0 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc11f7e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041ae200 con 0x562bf266f390 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc11f7e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041ab410 con 0x562bf26671b0 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc11f7e740 10 client.0 _ll_register_callbacks cb 0x7fcbf40316c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc08fa5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf264e9f0 0x562bf2a4b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc08fa5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf264e9f0 0x562bf2a4b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36558/0 (socket says 172.21.15.3:36558) 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc08fa5640 1 -- 172.21.15.3:0/3014531522 learned_addr learned my addr 172.21.15.3:0/3014531522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf266f390 0x562bf2ac2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.819+0000 7fcc097a6640 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf26671b0 0x562bf2a4bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcc08fa5640 1 -- 172.21.15.3:0/3014531522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf26671b0 msgr2=0x562bf2a4bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf26671b0 0x562bf2a4bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcc08fa5640 1 -- 172.21.15.3:0/3014531522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf266f390 msgr2=0x562bf2ac2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf266f390 0x562bf2ac2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcc08fa5640 1 -- 172.21.15.3:0/3014531522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bf2a4c500 con 0x562bf264e9f0 2024-09-21T06:19:44.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcc097a6640 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf26671b0 0x562bf2a4bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:19:44.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcc08fa5640 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf264e9f0 0x562bf2a4b6f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4031600 tx=0x7fcbf40038f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6a506971e304e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:44.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcbecff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:44.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcbecff9640 1 -- 172.21.15.3:0/3014531522 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4031a10 con 0x562bf264e9f0 2024-09-21T06:19:44.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcc11f7e740 1 -- 172.21.15.3:0/3014531522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbf405b040 con 0x562bf264e9f0 2024-09-21T06:19:44.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcbecff9640 1 -- 172.21.15.3:0/3014531522 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcbf4031bb0 con 0x562bf264e9f0 2024-09-21T06:19:44.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcbecff9640 1 -- 172.21.15.3:0/3014531522 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcbf4031db0 con 0x562bf264e9f0 2024-09-21T06:19:44.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcc11f7e740 10 client.16965 fetch_fsmap learned FSMap version 32 2024-09-21T06:19:44.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcc11f7e740 1 -- 172.21.15.3:0/3014531522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562bf2ac4290 con 0x562bf264e9f0 2024-09-21T06:19:44.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.820+0000 7fcbecff9640 1 -- 172.21.15.3:0/3014531522 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf405b040 con 0x562bf264e9f0 2024-09-21T06:19:44.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.821+0000 7fcbecff9640 1 -- 172.21.15.3:0/3014531522 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fcbf403d5e0 con 0x562bf264e9f0 2024-09-21T06:19:44.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.821+0000 7fcc11f7e740 10 client.16965 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:19:44.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.821+0000 7fcc11f7e740 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:44.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.821+0000 7fcc11f7e740 20 client.16965 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.821+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562bf2690070 0x562bf2ac6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.821+0000 7fcc11f7e740 4 client.16965 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.821+0000 7fcc11f7e740 1 -- 172.21.15.3:0/3014531522 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562bf29ab850 con 0x562bf2690070 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.821+0000 7fcc097a6640 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562bf2690070 0x562bf2ac6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.821+0000 7fcbecff9640 10 client.16965 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.821+0000 7fcc097a6640 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562bf2690070 0x562bf2ac6970 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8561e27f2534ca12 server_cookie=2e08bf53486fbb28 in_seq=0 out_seq=0 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcbecff9640 1 -- 172.21.15.3:0/3014531522 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2564 (crc 0 0 0) 0x562bf29ab850 con 0x562bf2690070 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcbecff9640 10 client.16965 handle_command_reply: tid=0 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcc11f7e740 1 client.16965 shutdown 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcc11f7e740 1 -- 172.21.15.3:0/3014531522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf264e9f0 msgr2=0x562bf2a4b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf264e9f0 0x562bf2a4b6f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4031600 tx=0x7fcbf40038f0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcc11f7e740 1 -- 172.21.15.3:0/3014531522 shutdown_connections 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562bf2690070 0x562bf2ac6970 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bf26671b0 0x562bf2a4bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bf266f390 0x562bf2ac2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcc11f7e740 1 --2- 172.21.15.3:0/3014531522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bf264e9f0 0x562bf2a4b6f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcc11f7e740 1 -- 172.21.15.3:0/3014531522 >> 172.21.15.3:0/3014531522 conn(0x562bf2684460 msgr2=0x562bf2a485a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:44.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcbecff9640 0 client.16965 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:44.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcc11f7e740 1 -- 172.21.15.3:0/3014531522 shutdown_connections 2024-09-21T06:19:44.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.822+0000 7fcc11f7e740 1 -- 172.21.15.3:0/3014531522 wait complete. 2024-09-21T06:19:44.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.823+0000 7fcc11f7e740 20 client.16965 trim_cache size 0 max 16384 2024-09-21T06:19:44.832 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:19:44.832 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "scrub active (1 inodes in the stack)", 2024-09-21T06:19:44.832 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-21T06:19:44.832 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:19:44.832 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "fab678ea-0ea8-42c2-b8b6-dd890880c82f", 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:19:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:44.835 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:44.836 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:44.837 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:44.837 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:19:44.837 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:19:44.837 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:19:44.837 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:19:44.837 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:19:44.837 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:19:44.837 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:19:44.837 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:19:44.837 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:19:44.837 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:19:44.837 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:19:44.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.823+0000 7fcc0ba0a640 1 -- 172.21.15.3:0/2383681734 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcbf00b6680 msgr2=0x7fcbf00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:44.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.823+0000 7fcc0ba0a640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcbf00b6680 0x7fcbf00b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0ad470 tx=0x7fcbfc00cf50 comp rx=0 tx=0).stop 2024-09-21T06:19:44.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.823+0000 7fcc0ba0a640 1 -- 172.21.15.3:0/2383681734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc04108cf0 msgr2=0x7fcc0419fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:44.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.823+0000 7fcc0ba0a640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc04108cf0 0x7fcc0419fce0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00eaa0 tx=0x7fcbfc00ef60 comp rx=0 tx=0).stop 2024-09-21T06:19:44.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.824+0000 7fcc0ba0a640 1 -- 172.21.15.3:0/2383681734 shutdown_connections 2024-09-21T06:19:44.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.824+0000 7fcc0ba0a640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcbf00b6680 0x7fcbf00b8b90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.824+0000 7fcc0ba0a640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc04108cf0 0x7fcc0419fce0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.824+0000 7fcc0ba0a640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc041083a0 0x7fcc0419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.824+0000 7fcc0ba0a640 1 --2- 172.21.15.3:0/2383681734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc04102a60 0x7fcc0419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:44.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.824+0000 7fcc0ba0a640 1 -- 172.21.15.3:0/2383681734 >> 172.21.15.3:0/2383681734 conn(0x7fcc04079730 msgr2=0x7fcc041019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:44.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.824+0000 7fcc0ba0a640 1 -- 172.21.15.3:0/2383681734 shutdown_connections 2024-09-21T06:19:44.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:19:44.824+0000 7fcc0ba0a640 1 -- 172.21.15.3:0/2383681734 wait complete. 2024-09-21T06:19:44.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 470516 12.51 MB/sec execute 77 sec latency 38.707 ms 2024-09-21T06:19:45.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:45 smithi137 ceph-mon[34464]: pgmap v467: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 337 op/s 2024-09-21T06:19:45.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:45 smithi204 ceph-mon[34659]: pgmap v467: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 337 op/s 2024-09-21T06:19:45.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:45 smithi003 ceph-mon[31227]: pgmap v467: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 337 op/s 2024-09-21T06:19:45.692 INFO:tasks.cephfs.filesystem:scrub status for tag:fab678ea-0ea8-42c2-b8b6-dd890880c82f - {'path': '/', 'tag': 'fab678ea-0ea8-42c2-b8b6-dd890880c82f', 'options': 'recursive,force'} 2024-09-21T06:19:45.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 473091 12.53 MB/sec execute 78 sec latency 12.542 ms 2024-09-21T06:19:46.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:46 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:19:46.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:46 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:46.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:46 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:46.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:46 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:19:46.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:46 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:46.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:46 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:46.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:46 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:19:46.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:46 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:46.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:46 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:46.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 476319 12.61 MB/sec execute 79 sec latency 19.560 ms 2024-09-21T06:19:47.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:47 smithi137 ceph-mon[34464]: pgmap v468: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-09-21T06:19:47.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:47 smithi204 ceph-mon[34659]: pgmap v468: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-09-21T06:19:47.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:47 smithi003 ceph-mon[31227]: pgmap v468: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-09-21T06:19:47.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 477896 12.52 MB/sec execute 80 sec latency 102.849 ms 2024-09-21T06:19:48.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 480135 12.54 MB/sec execute 81 sec latency 35.867 ms 2024-09-21T06:19:49.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:49 smithi137 ceph-mon[34464]: pgmap v469: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 350 op/s 2024-09-21T06:19:49.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:49 smithi204 ceph-mon[34659]: pgmap v469: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 350 op/s 2024-09-21T06:19:49.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:49 smithi003 ceph-mon[31227]: pgmap v469: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 350 op/s 2024-09-21T06:19:49.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 483288 12.60 MB/sec execute 82 sec latency 76.913 ms 2024-09-21T06:19:50.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 485603 12.55 MB/sec execute 83 sec latency 13.849 ms 2024-09-21T06:19:51.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:51 smithi137 ceph-mon[34464]: pgmap v470: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 396 op/s 2024-09-21T06:19:51.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:51 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:19:51.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:51 smithi204 ceph-mon[34659]: pgmap v470: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 396 op/s 2024-09-21T06:19:51.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:51 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:19:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:51 smithi003 ceph-mon[31227]: pgmap v470: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 396 op/s 2024-09-21T06:19:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:51 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:19:51.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 487644 12.55 MB/sec execute 84 sec latency 15.101 ms 2024-09-21T06:19:52.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:52 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:19:52.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:52 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:19:52.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:52 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:19:52.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:52 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:19:52.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:52 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:19:52.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:52 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:19:52.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 490551 12.62 MB/sec execute 85 sec latency 115.862 ms 2024-09-21T06:19:53.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:53 smithi137 ceph-mon[34464]: pgmap v471: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 450 op/s 2024-09-21T06:19:53.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:53 smithi204 ceph-mon[34659]: pgmap v471: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 450 op/s 2024-09-21T06:19:53.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:53 smithi003 ceph-mon[31227]: pgmap v471: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 450 op/s 2024-09-21T06:19:53.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 492749 12.56 MB/sec execute 86 sec latency 60.957 ms 2024-09-21T06:19:54.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 494994 12.58 MB/sec execute 87 sec latency 13.685 ms 2024-09-21T06:19:55.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:55 smithi137 ceph-mon[34464]: pgmap v472: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 354 op/s 2024-09-21T06:19:55.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:55 smithi204 ceph-mon[34659]: pgmap v472: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 354 op/s 2024-09-21T06:19:55.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:55 smithi003 ceph-mon[31227]: pgmap v472: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 354 op/s 2024-09-21T06:19:55.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 497791 12.63 MB/sec execute 88 sec latency 63.470 ms 2024-09-21T06:19:56.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 501039 12.68 MB/sec execute 89 sec latency 12.609 ms 2024-09-21T06:19:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.062+0000 7fdac0b12640 1 -- 172.21.15.3:0/464781727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc108c30 msgr2=0x7fdabc10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.062+0000 7fdac0b12640 1 --2- 172.21.15.3:0/464781727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc108c30 0x7fdabc10f460 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009ca0 tx=0x7fdab402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.062+0000 7fdac0b12640 1 -- 172.21.15.3:0/464781727 shutdown_connections 2024-09-21T06:19:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.062+0000 7fdac0b12640 1 --2- 172.21.15.3:0/464781727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdabc1082e0 0x7fdabc1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.062+0000 7fdac0b12640 1 --2- 172.21.15.3:0/464781727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc108c30 0x7fdabc10f460 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.062+0000 7fdac0b12640 1 --2- 172.21.15.3:0/464781727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdabc1029b0 0x7fdabc102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.062+0000 7fdac0b12640 1 -- 172.21.15.3:0/464781727 >> 172.21.15.3:0/464781727 conn(0x7fdabc07a740 msgr2=0x7fdabc07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.062+0000 7fdac0b12640 1 -- 172.21.15.3:0/464781727 shutdown_connections 2024-09-21T06:19:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.062+0000 7fdac0b12640 1 -- 172.21.15.3:0/464781727 wait complete. 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.063+0000 7fdac0b12640 1 Processor -- start 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.063+0000 7fdac0b12640 1 -- start start 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdac0b12640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc1029b0 0x7fdabc10ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdac0b12640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdabc1082e0 0x7fdabc110480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdac0b12640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdabc108c30 0x7fdabc1109c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdac0b12640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc113a10 con 0x7fdabc1029b0 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdac0b12640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc113890 con 0x7fdabc108c30 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdac0b12640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc113b90 con 0x7fdabc1082e0 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdaba59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc1029b0 0x7fdabc10ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdaba59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc1029b0 0x7fdabc10ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55758/0 (socket says 172.21.15.3:55758) 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdaba59c640 1 -- 172.21.15.3:0/1722652038 learned_addr learned my addr 172.21.15.3:0/1722652038 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdabad9d640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdabc108c30 0x7fdabc1109c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdaba59c640 1 -- 172.21.15.3:0/1722652038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdabc1082e0 msgr2=0x7fdabc110480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdab9d9b640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdabc1082e0 0x7fdabc110480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdaba59c640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdabc1082e0 0x7fdabc110480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdaba59c640 1 -- 172.21.15.3:0/1722652038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdabc108c30 msgr2=0x7fdabc1109c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdaba59c640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdabc108c30 0x7fdabc1109c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdaba59c640 1 -- 172.21.15.3:0/1722652038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdabc1a86e0 con 0x7fdabc1029b0 2024-09-21T06:19:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdab9d9b640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdabc1082e0 0x7fdabc110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:19:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.064+0000 7fdaba59c640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc1029b0 0x7fdabc10ff40 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00e9f0 tx=0x7fdaac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.065+0000 7fdaab7fe640 1 -- 172.21.15.3:0/1722652038 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac00cd80 con 0x7fdabc1029b0 2024-09-21T06:19:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.065+0000 7fdaab7fe640 1 -- 172.21.15.3:0/1722652038 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaac0108e0 con 0x7fdabc1029b0 2024-09-21T06:19:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.065+0000 7fdac0b12640 1 -- 172.21.15.3:0/1722652038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdabc1a89d0 con 0x7fdabc1029b0 2024-09-21T06:19:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.065+0000 7fdaab7fe640 1 -- 172.21.15.3:0/1722652038 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac018980 con 0x7fdabc1029b0 2024-09-21T06:19:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.065+0000 7fdac0b12640 1 -- 172.21.15.3:0/1722652038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdabc1a8f20 con 0x7fdabc1029b0 2024-09-21T06:19:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.067+0000 7fdac7a78740 1 Processor -- start 2024-09-21T06:19:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdac7a78740 1 -- start start 2024-09-21T06:19:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdac7a78740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca2e1b0 0x55d1ecdd39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdaab7fe640 1 -- 172.21.15.3:0/1722652038 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fdaac010430 con 0x7fdabc1029b0 2024-09-21T06:19:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdac7a78740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecdd3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdac7a78740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca0d4d0 0x55d1ecdd4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdac7a78740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d1ecd486f0 con 0x55d1eca052f0 2024-09-21T06:19:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdac7a78740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d1ecd37b20 con 0x55d1eca2e1b0 2024-09-21T06:19:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdac7a78740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d1ecd3b930 con 0x55d1eca0d4d0 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdaba59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecdd3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdabad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca2e1b0 0x55d1ecdd39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdaba59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecdd3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55770/0 (socket says 172.21.15.3:55770) 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdaba59c640 1 -- 172.21.15.3:0/3895617755 learned_addr learned my addr 172.21.15.3:0/3895617755 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdab9d9b640 1 --2- 172.21.15.3:0/3895617755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca0d4d0 0x55d1ecdd4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdaba59c640 1 -- 172.21.15.3:0/3895617755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca0d4d0 msgr2=0x55d1ecdd4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdaba59c640 1 --2- 172.21.15.3:0/3895617755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca0d4d0 0x55d1ecdd4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdaba59c640 1 -- 172.21.15.3:0/3895617755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca2e1b0 msgr2=0x55d1ecdd39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdaba59c640 1 --2- 172.21.15.3:0/3895617755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca2e1b0 0x55d1ecdd39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.068+0000 7fdaba59c640 1 -- 172.21.15.3:0/3895617755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1eccfd530 con 0x55d1eca052f0 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.069+0000 7fdaba59c640 1 --2- 172.21.15.3:0/3895617755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecdd3ef0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0e51a0 tx=0x7fdaac0f05d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd38d22ce5fd2b01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.069+0000 7fdaa8ff9640 1 -- 172.21.15.3:0/3895617755 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0f4420 con 0x55d1eca052f0 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.069+0000 7fdaab7fe640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdaa40b66b0 0x7fdaa40b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.069+0000 7fdaa8ff9640 1 -- 172.21.15.3:0/3895617755 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaac0f9070 con 0x55d1eca052f0 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.069+0000 7fdaa8ff9640 1 -- 172.21.15.3:0/3895617755 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0a9280 con 0x55d1eca052f0 2024-09-21T06:19:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.069+0000 7fdabad9d640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdaa40b66b0 0x7fdaa40b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.069+0000 7fdaab7fe640 1 -- 172.21.15.3:0/1722652038 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fdaac0e7aa0 con 0x7fdabc1029b0 2024-09-21T06:19:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.070+0000 7fdac7a78740 1 -- 172.21.15.3:0/3895617755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 msgr2=0x55d1ecdd3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.070+0000 7fdabad9d640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdaa40b66b0 0x7fdaa40b8bc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdab402f500 tx=0x7fdab402f310 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.070+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3895617755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecdd3ef0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0e51a0 tx=0x7fdaac0f05d0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.070+0000 7fdac7a78740 1 -- 172.21.15.3:0/3895617755 shutdown_connections 2024-09-21T06:19:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.070+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3895617755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca0d4d0 0x55d1ecdd4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.070+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3895617755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecdd3ef0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.070+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3895617755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca2e1b0 0x55d1ecdd39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.070+0000 7fdac7a78740 1 -- 172.21.15.3:0/3895617755 >> 172.21.15.3:0/3895617755 conn(0x55d1eca22300 msgr2=0x55d1ecdd18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.070+0000 7fdac7a78740 1 -- 172.21.15.3:0/3895617755 shutdown_connections 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.070+0000 7fdac7a78740 1 -- 172.21.15.3:0/3895617755 wait complete. 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.070+0000 7fdac7a78740 1 -- start start 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.070+0000 7fdac7a78740 1 Processor -- start 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdac7a78740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecddef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdac7a78740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca0d4d0 0x55d1ecddf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdac7a78740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca2e1b0 0x55d1ecddf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdac7a78740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d1ecc6fab0 con 0x55d1eca052f0 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdac7a78740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d1eccdd730 con 0x55d1eca0d4d0 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdac7a78740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d1ecde20d0 con 0x55d1eca2e1b0 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdac7a78740 10 client.0 _ll_register_callbacks cb 0x55d1ecd316b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdaba59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecddef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdaba59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecddef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55784/0 (socket says 172.21.15.3:55784) 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdaba59c640 1 -- 172.21.15.3:0/3910290429 learned_addr learned my addr 172.21.15.3:0/3910290429 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdab9d9b640 1 --2- 172.21.15.3:0/3910290429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca0d4d0 0x55d1ecddf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdabad9d640 1 --2- 172.21.15.3:0/3910290429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca2e1b0 0x55d1ecddf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdaba59c640 1 -- 172.21.15.3:0/3910290429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca2e1b0 msgr2=0x55d1ecddf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdaba59c640 1 --2- 172.21.15.3:0/3910290429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca2e1b0 0x55d1ecddf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdaba59c640 1 -- 172.21.15.3:0/3910290429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca0d4d0 msgr2=0x55d1ecddf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdaba59c640 1 --2- 172.21.15.3:0/3910290429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca0d4d0 0x55d1ecddf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdaba59c640 1 -- 172.21.15.3:0/3910290429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1eca99790 con 0x55d1eca052f0 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdaba59c640 1 --2- 172.21.15.3:0/3910290429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecddef30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0aaba0 tx=0x7fdaac009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e367a545c78a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.071+0000 7fdaa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdaa1ffb640 1 -- 172.21.15.3:0/3910290429 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0acb90 con 0x55d1eca052f0 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdaa1ffb640 1 -- 172.21.15.3:0/3910290429 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fdaac0eb020 con 0x55d1eca052f0 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdac7a78740 1 -- 172.21.15.3:0/3910290429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaac0f9070 con 0x55d1eca052f0 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdaa1ffb640 1 -- 172.21.15.3:0/3910290429 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdaac0acec0 con 0x55d1eca052f0 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdac7a78740 10 client.16977 fetch_fsmap learned FSMap version 32 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdac7a78740 1 -- 172.21.15.3:0/3910290429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1ecb12620 con 0x55d1eca052f0 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdaa1ffb640 1 -- 172.21.15.3:0/3910290429 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaac0f9070 con 0x55d1eca052f0 2024-09-21T06:19:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdaa1ffb640 1 -- 172.21.15.3:0/3910290429 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fdaac0a9ba0 con 0x55d1eca052f0 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdac7a78740 10 client.16977 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdac7a78740 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdac7a78740 20 client.16977 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3910290429 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d1ec9ecd80 0x55d1ecdf0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdac7a78740 4 client.16977 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdac7a78740 1 -- 172.21.15.3:0/3910290429 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1ecd4a640 con 0x55d1ec9ecd80 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.072+0000 7fdab9d9b640 1 --2- 172.21.15.3:0/3910290429 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d1ec9ecd80 0x55d1ecdf0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.073+0000 7fdab9d9b640 1 --2- 172.21.15.3:0/3910290429 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d1ec9ecd80 0x55d1ecdf0e70 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e12685984b831f05 server_cookie=7e2712c7f5e98f75 in_seq=0 out_seq=0 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.073+0000 7fdaa1ffb640 10 client.16977 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdaa1ffb640 1 -- 172.21.15.3:0/3910290429 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d1ecd4a640 con 0x55d1ec9ecd80 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdaa1ffb640 10 client.16977 handle_command_reply: tid=0 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdac7a78740 1 client.16977 shutdown 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdac7a78740 1 -- 172.21.15.3:0/3910290429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 msgr2=0x55d1ecddef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3910290429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecddef30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0aaba0 tx=0x7fdaac009700 comp rx=0 tx=0).stop 2024-09-21T06:19:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdac7a78740 1 -- 172.21.15.3:0/3910290429 shutdown_connections 2024-09-21T06:19:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3910290429 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d1ec9ecd80 0x55d1ecdf0e70 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3910290429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca2e1b0 0x55d1ecddf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3910290429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca0d4d0 0x55d1ecddf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3910290429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecddef30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdaa1ffb640 0 client.16977 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdac7a78740 1 -- 172.21.15.3:0/3910290429 >> 172.21.15.3:0/3910290429 conn(0x55d1eca22300 msgr2=0x55d1ecdd49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdac7a78740 1 -- 172.21.15.3:0/3910290429 shutdown_connections 2024-09-21T06:19:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdac7a78740 1 -- 172.21.15.3:0/3910290429 wait complete. 2024-09-21T06:19:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.074+0000 7fdac7a78740 20 client.16977 trim_cache size 0 max 16384 2024-09-21T06:19:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdac7a78740 1 Processor -- start 2024-09-21T06:19:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdac7a78740 1 -- start start 2024-09-21T06:19:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdac7a78740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca052f0 0x55d1ecdd1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdac7a78740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca0d4d0 0x55d1ecddc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdac7a78740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca2e1b0 0x55d1ecddc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdac7a78740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc1b18d0 con 0x55d1eca2e1b0 2024-09-21T06:19:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdac7a78740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc1b3440 con 0x55d1eca052f0 2024-09-21T06:19:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdac7a78740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc1b1fa0 con 0x55d1eca0d4d0 2024-09-21T06:19:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdabad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca2e1b0 0x55d1ecddc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdab9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca0d4d0 0x55d1ecddc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdab9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca0d4d0 0x55d1ecddc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:50728/0 (socket says 172.21.15.3:50728) 2024-09-21T06:19:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdaba59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca052f0 0x55d1ecdd1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdabad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca2e1b0 0x55d1ecddc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55788/0 (socket says 172.21.15.3:55788) 2024-09-21T06:19:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.090+0000 7fdab9d9b640 1 -- 172.21.15.3:0/3035330344 learned_addr learned my addr 172.21.15.3:0/3035330344 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.091+0000 7fdab9d9b640 1 -- 172.21.15.3:0/3035330344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca052f0 msgr2=0x55d1ecdd1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.091+0000 7fdab9d9b640 1 --2- 172.21.15.3:0/3035330344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca052f0 0x55d1ecdd1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.091+0000 7fdab9d9b640 1 -- 172.21.15.3:0/3035330344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca2e1b0 msgr2=0x55d1ecddc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.091+0000 7fdab9d9b640 1 --2- 172.21.15.3:0/3035330344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca2e1b0 0x55d1ecddc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.091+0000 7fdab9d9b640 1 -- 172.21.15.3:0/3035330344 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ecdd5ee0 con 0x55d1eca0d4d0 2024-09-21T06:19:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.091+0000 7fdab9d9b640 1 --2- 172.21.15.3:0/3035330344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca0d4d0 0x55d1ecddc100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdabc06c9c0 tx=0x7fdab0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddfdea591a662e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.091+0000 7fdaa3fff640 1 -- 172.21.15.3:0/3035330344 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab001e670 con 0x55d1eca0d4d0 2024-09-21T06:19:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.091+0000 7fdaa3fff640 1 -- 172.21.15.3:0/3035330344 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab001ecc0 con 0x55d1eca0d4d0 2024-09-21T06:19:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.091+0000 7fdaa3fff640 1 -- 172.21.15.3:0/3035330344 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab001dcd0 con 0x55d1eca0d4d0 2024-09-21T06:19:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.092+0000 7fdac7a78740 1 -- 172.21.15.3:0/3035330344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca0d4d0 msgr2=0x55d1ecddc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.092+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3035330344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca0d4d0 0x55d1ecddc100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdabc06c9c0 tx=0x7fdab0009f90 comp rx=0 tx=0).stop 2024-09-21T06:19:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.092+0000 7fdac7a78740 1 -- 172.21.15.3:0/3035330344 shutdown_connections 2024-09-21T06:19:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.092+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3035330344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca0d4d0 0x55d1ecddc100 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.092+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3035330344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca2e1b0 0x55d1ecddc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.092+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3035330344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca052f0 0x55d1ecdd1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.092+0000 7fdac7a78740 1 -- 172.21.15.3:0/3035330344 >> 172.21.15.3:0/3035330344 conn(0x55d1eca22300 msgr2=0x55d1ecdd10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.092+0000 7fdac7a78740 1 -- 172.21.15.3:0/3035330344 shutdown_connections 2024-09-21T06:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.092+0000 7fdac7a78740 1 -- 172.21.15.3:0/3035330344 wait complete. 2024-09-21T06:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.092+0000 7fdac7a78740 1 -- start start 2024-09-21T06:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.092+0000 7fdac7a78740 1 Processor -- start 2024-09-21T06:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdac7a78740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecdf3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdac7a78740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca0d4d0 0x55d1ecdf38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdac7a78740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca2e1b0 0x55d1ece595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdac7a78740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc1b2d70 con 0x55d1eca052f0 2024-09-21T06:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdac7a78740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc1b26a0 con 0x55d1eca0d4d0 2024-09-21T06:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdac7a78740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc1af8b0 con 0x55d1eca2e1b0 2024-09-21T06:19:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdac7a78740 10 client.0 _ll_register_callbacks cb 0x7fdab0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdaba59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecdf3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdab9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca0d4d0 0x55d1ecdf38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdaba59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecdf3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55798/0 (socket says 172.21.15.3:55798) 2024-09-21T06:19:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdaba59c640 1 -- 172.21.15.3:0/3886098973 learned_addr learned my addr 172.21.15.3:0/3886098973 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:19:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdabad9d640 1 --2- 172.21.15.3:0/3886098973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca2e1b0 0x55d1ece595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdaba59c640 1 -- 172.21.15.3:0/3886098973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca2e1b0 msgr2=0x55d1ece595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdaba59c640 1 --2- 172.21.15.3:0/3886098973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca2e1b0 0x55d1ece595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdaba59c640 1 -- 172.21.15.3:0/3886098973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca0d4d0 msgr2=0x55d1ecdf38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdaba59c640 1 --2- 172.21.15.3:0/3886098973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca0d4d0 0x55d1ecdf38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdaba59c640 1 -- 172.21.15.3:0/3886098973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ece59b30 con 0x55d1eca052f0 2024-09-21T06:19:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdab9d9b640 1 --2- 172.21.15.3:0/3886098973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca0d4d0 0x55d1ecdf38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdaba59c640 1 --2- 172.21.15.3:0/3886098973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecdf3300 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0a9940 tx=0x7fdaac010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0f5fac5a6948a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.093+0000 7fdaa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:19:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdaa1ffb640 1 -- 172.21.15.3:0/3886098973 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0ad0a0 con 0x55d1eca052f0 2024-09-21T06:19:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdaa1ffb640 1 -- 172.21.15.3:0/3886098973 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fdaac0f0cb0 con 0x55d1eca052f0 2024-09-21T06:19:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdac7a78740 1 -- 172.21.15.3:0/3886098973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab001ecc0 con 0x55d1eca052f0 2024-09-21T06:19:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdaa1ffb640 1 -- 172.21.15.3:0/3886098973 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdaac0ad280 con 0x55d1eca052f0 2024-09-21T06:19:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdac7a78740 10 client.16983 fetch_fsmap learned FSMap version 32 2024-09-21T06:19:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdac7a78740 1 -- 172.21.15.3:0/3886098973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1ece5acf0 con 0x55d1eca052f0 2024-09-21T06:19:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdaa1ffb640 1 -- 172.21.15.3:0/3886098973 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab001ecc0 con 0x55d1eca052f0 2024-09-21T06:19:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdaa1ffb640 1 -- 172.21.15.3:0/3886098973 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fdaac0e54b0 con 0x55d1eca052f0 2024-09-21T06:19:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdac7a78740 10 client.16983 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:19:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdac7a78740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdac7a78740 20 client.16983 populate_metadata read hostname 'smithi003' 2024-09-21T06:19:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3886098973 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d1ec9c21c0 0x55d1ece5d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdac7a78740 4 client.16983 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:19:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdac7a78740 1 -- 172.21.15.3:0/3886098973 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1ecd460f0 con 0x55d1ec9c21c0 2024-09-21T06:19:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.094+0000 7fdab9d9b640 1 --2- 172.21.15.3:0/3886098973 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d1ec9c21c0 0x55d1ece5d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.095+0000 7fdab9d9b640 1 --2- 172.21.15.3:0/3886098973 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d1ec9c21c0 0x55d1ece5d260 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1157aba1e02bd645 server_cookie=2dd2cff51f5b2e1f in_seq=0 out_seq=0 2024-09-21T06:19:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.095+0000 7fdaa1ffb640 10 client.16983 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdaa1ffb640 1 -- 172.21.15.3:0/3886098973 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1ecd460f0 con 0x55d1ec9c21c0 2024-09-21T06:19:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdaa1ffb640 10 client.16983 handle_command_reply: tid=0 2024-09-21T06:19:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdac7a78740 1 client.16983 shutdown 2024-09-21T06:19:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdac7a78740 1 -- 172.21.15.3:0/3886098973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 msgr2=0x55d1ecdf3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3886098973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecdf3300 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0a9940 tx=0x7fdaac010cc0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdac7a78740 1 -- 172.21.15.3:0/3886098973 shutdown_connections 2024-09-21T06:19:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3886098973 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d1ec9c21c0 0x55d1ece5d260 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3886098973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1eca2e1b0 0x55d1ece595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdaa1ffb640 0 client.16983 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:19:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3886098973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d1eca0d4d0 0x55d1ecdf38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdac7a78740 1 --2- 172.21.15.3:0/3886098973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eca052f0 0x55d1ecdf3300 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdac7a78740 1 -- 172.21.15.3:0/3886098973 >> 172.21.15.3:0/3886098973 conn(0x55d1eca22300 msgr2=0x55d1ecdcfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdac7a78740 1 -- 172.21.15.3:0/3886098973 shutdown_connections 2024-09-21T06:19:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdac7a78740 1 -- 172.21.15.3:0/3886098973 wait complete. 2024-09-21T06:19:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.271+0000 7fdac7a78740 20 client.16983 trim_cache size 0 max 16384 2024-09-21T06:19:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:19:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:19:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:19:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:19:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.272+0000 7fdac0b12640 1 -- 172.21.15.3:0/1722652038 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdaa40b66b0 msgr2=0x7fdaa40b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.272+0000 7fdac0b12640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdaa40b66b0 0x7fdaa40b8bc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdab402f500 tx=0x7fdab402f310 comp rx=0 tx=0).stop 2024-09-21T06:19:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.272+0000 7fdac0b12640 1 -- 172.21.15.3:0/1722652038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc1029b0 msgr2=0x7fdabc10ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.272+0000 7fdac0b12640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc1029b0 0x7fdabc10ff40 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00e9f0 tx=0x7fdaac00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.272+0000 7fdac0b12640 1 -- 172.21.15.3:0/1722652038 shutdown_connections 2024-09-21T06:19:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.272+0000 7fdac0b12640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fdaa40b66b0 0x7fdaa40b8bc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.272+0000 7fdac0b12640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdabc1082e0 0x7fdabc110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.272+0000 7fdac0b12640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdabc108c30 0x7fdabc1109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.272+0000 7fdac0b12640 1 --2- 172.21.15.3:0/1722652038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdabc1029b0 0x7fdabc10ff40 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.272+0000 7fdac0b12640 1 -- 172.21.15.3:0/1722652038 >> 172.21.15.3:0/1722652038 conn(0x7fdabc07a740 msgr2=0x7fdabc106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.272+0000 7fdac0b12640 1 -- 172.21.15.3:0/1722652038 shutdown_connections 2024-09-21T06:19:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:19:57.272+0000 7fdac0b12640 1 -- 172.21.15.3:0/1722652038 wait complete. 2024-09-21T06:19:57.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:57 smithi137 ceph-mon[34464]: pgmap v473: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 395 op/s 2024-09-21T06:19:57.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:57 smithi204 ceph-mon[34659]: pgmap v473: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 395 op/s 2024-09-21T06:19:57.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:57 smithi003 ceph-mon[31227]: pgmap v473: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 395 op/s 2024-09-21T06:19:57.954 INFO:tasks.workunit.client.0.smithi003.stdout: 1 502550 12.62 MB/sec execute 90 sec latency 106.637 ms 2024-09-21T06:19:58.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 504852 12.63 MB/sec execute 91 sec latency 35.149 ms 2024-09-21T06:19:59.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:19:59 smithi204 ceph-mon[34659]: pgmap v474: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2024-09-21T06:19:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:19:59 smithi003 ceph-mon[31227]: pgmap v474: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2024-09-21T06:19:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:19:59 smithi137 ceph-mon[34464]: pgmap v474: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2024-09-21T06:19:59.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 507677 12.64 MB/sec execute 92 sec latency 20.082 ms 2024-09-21T06:20:00.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:00 smithi204 ceph-mon[34659]: overall HEALTH_OK 2024-09-21T06:20:00.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:00 smithi003 ceph-mon[31227]: overall HEALTH_OK 2024-09-21T06:20:00.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:00 smithi137 ceph-mon[34464]: overall HEALTH_OK 2024-09-21T06:20:00.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 509062 12.58 MB/sec execute 93 sec latency 24.864 ms 2024-09-21T06:20:01.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:01 smithi204 ceph-mon[34659]: pgmap v475: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 422 op/s 2024-09-21T06:20:01.491 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:01 smithi003 ceph-mon[31227]: pgmap v475: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 422 op/s 2024-09-21T06:20:01.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:01 smithi137 ceph-mon[34464]: pgmap v475: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 422 op/s 2024-09-21T06:20:01.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 511858 12.63 MB/sec execute 94 sec latency 24.511 ms 2024-09-21T06:20:02.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 514581 12.63 MB/sec execute 95 sec latency 12.680 ms 2024-09-21T06:20:03.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:03 smithi003 ceph-mon[31227]: pgmap v476: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-09-21T06:20:03.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:03 smithi137 ceph-mon[34464]: pgmap v476: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-09-21T06:20:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:03 smithi204 ceph-mon[34659]: pgmap v476: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-09-21T06:20:03.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 516248 12.59 MB/sec execute 96 sec latency 65.622 ms 2024-09-21T06:20:04.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 519040 12.64 MB/sec execute 97 sec latency 36.997 ms 2024-09-21T06:20:05.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:05 smithi003 ceph-mon[31227]: pgmap v477: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 3.5 MiB/s wr, 383 op/s 2024-09-21T06:20:05.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:05 smithi137 ceph-mon[34464]: pgmap v477: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 3.5 MiB/s wr, 383 op/s 2024-09-21T06:20:05.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:05 smithi204 ceph-mon[34659]: pgmap v477: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 3.5 MiB/s wr, 383 op/s 2024-09-21T06:20:05.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 522020 12.66 MB/sec execute 98 sec latency 17.180 ms 2024-09-21T06:20:06.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 524305 12.63 MB/sec execute 99 sec latency 16.468 ms 2024-09-21T06:20:07.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:07 smithi003 ceph-mon[31227]: pgmap v478: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-09-21T06:20:07.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:07 smithi137 ceph-mon[34464]: pgmap v478: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-09-21T06:20:07.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:07 smithi204 ceph-mon[34659]: pgmap v478: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-09-21T06:20:07.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 526497 12.64 MB/sec execute 100 sec latency 27.015 ms 2024-09-21T06:20:08.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 529522 12.70 MB/sec execute 101 sec latency 44.562 ms 2024-09-21T06:20:09.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:09 smithi003 ceph-mon[31227]: pgmap v479: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 409 op/s 2024-09-21T06:20:09.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:09 smithi137 ceph-mon[34464]: pgmap v479: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 409 op/s 2024-09-21T06:20:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:09 smithi204 ceph-mon[34659]: pgmap v479: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 409 op/s 2024-09-21T06:20:09.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 531741 12.65 MB/sec execute 102 sec latency 37.607 ms 2024-09-21T06:20:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.374+0000 7f512ebc3640 1 -- 172.21.15.3:0/4089419892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51281029e0 msgr2=0x7f5128102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.374+0000 7f512ebc3640 1 --2- 172.21.15.3:0/4089419892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51281029e0 0x7f5128102df0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f511c009900 tx=0x7f511c02a410 comp rx=0 tx=0).stop 2024-09-21T06:20:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.374+0000 7f512ebc3640 1 -- 172.21.15.3:0/4089419892 shutdown_connections 2024-09-21T06:20:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.374+0000 7f512ebc3640 1 --2- 172.21.15.3:0/4089419892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5128108c60 0x7f512810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.374+0000 7f512ebc3640 1 --2- 172.21.15.3:0/4089419892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5128108310 0x7f5128108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.374+0000 7f512ebc3640 1 --2- 172.21.15.3:0/4089419892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51281029e0 0x7f5128102df0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.374+0000 7f512ebc3640 1 -- 172.21.15.3:0/4089419892 >> 172.21.15.3:0/4089419892 conn(0x7f512807a6e0 msgr2=0x7f512807aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.374+0000 7f512ebc3640 1 -- 172.21.15.3:0/4089419892 shutdown_connections 2024-09-21T06:20:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.374+0000 7f512ebc3640 1 -- 172.21.15.3:0/4089419892 wait complete. 2024-09-21T06:20:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.375+0000 7f512ebc3640 1 Processor -- start 2024-09-21T06:20:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.375+0000 7f512ebc3640 1 -- start start 2024-09-21T06:20:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.375+0000 7f512ebc3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51281029e0 0x7f512819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.375+0000 7f512ebc3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5128108310 0x7f512819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.375+0000 7f512ebc3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5128108c60 0x7f512819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f512ebc3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5128112920 con 0x7f5128108310 2024-09-21T06:20:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f512ebc3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51281127a0 con 0x7f51281029e0 2024-09-21T06:20:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f512ebc3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5128112aa0 con 0x7f5128108c60 2024-09-21T06:20:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f5127fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5128108310 0x7f512819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f5127fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5128108310 0x7f512819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49968/0 (socket says 172.21.15.3:49968) 2024-09-21T06:20:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f5127fff640 1 -- 172.21.15.3:0/1709828045 learned_addr learned my addr 172.21.15.3:0/1709828045 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f512c95f640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51281029e0 0x7f512819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f512d160640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5128108c60 0x7f512819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f5127fff640 1 -- 172.21.15.3:0/1709828045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5128108c60 msgr2=0x7f512819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f5127fff640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5128108c60 0x7f512819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f5127fff640 1 -- 172.21.15.3:0/1709828045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51281029e0 msgr2=0x7f512819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f5127fff640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51281029e0 0x7f512819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f5127fff640 1 -- 172.21.15.3:0/1709828045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51281a03c0 con 0x7f5128108310 2024-09-21T06:20:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f512c95f640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51281029e0 0x7f512819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f5127fff640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5128108310 0x7f512819aa30 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f511400e9f0 tx=0x7f511400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f5125ffb640 1 -- 172.21.15.3:0/1709828045 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511400ccc0 con 0x7f5128108310 2024-09-21T06:20:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f5125ffb640 1 -- 172.21.15.3:0/1709828045 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51140108e0 con 0x7f5128108310 2024-09-21T06:20:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f512ebc3640 1 -- 172.21.15.3:0/1709828045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51281a06b0 con 0x7f5128108310 2024-09-21T06:20:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.376+0000 7f5125ffb640 1 -- 172.21.15.3:0/1709828045 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114018860 con 0x7f5128108310 2024-09-21T06:20:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.377+0000 7f512ebc3640 1 -- 172.21.15.3:0/1709828045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51281a6e40 con 0x7f5128108310 2024-09-21T06:20:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.379+0000 7f5135c0f740 1 Processor -- start 2024-09-21T06:20:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.379+0000 7f5135c0f740 1 -- start start 2024-09-21T06:20:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5135c0f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4cc1b0 0x55f09a8719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5135c0f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a871ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5125ffb640 1 -- 172.21.15.3:0/1709828045 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f5114010430 con 0x7f5128108310 2024-09-21T06:20:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5135c0f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 0x55f09a872430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5135c0f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f09a7e66f0 con 0x55f09a4ab4d0 2024-09-21T06:20:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5135c0f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f09a7d5b20 con 0x55f09a4a32f0 2024-09-21T06:20:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5135c0f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f09a7d9930 con 0x55f09a4cc1b0 2024-09-21T06:20:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5127fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 0x55f09a872430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f512c95f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a871ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5127fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 0x55f09a872430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49984/0 (socket says 172.21.15.3:49984) 2024-09-21T06:20:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5127fff640 1 -- 172.21.15.3:0/934457261 learned_addr learned my addr 172.21.15.3:0/934457261 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f512d160640 1 --2- 172.21.15.3:0/934457261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4cc1b0 0x55f09a8719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5127fff640 1 -- 172.21.15.3:0/934457261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4cc1b0 msgr2=0x55f09a8719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5127fff640 1 --2- 172.21.15.3:0/934457261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4cc1b0 0x55f09a8719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5127fff640 1 -- 172.21.15.3:0/934457261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 msgr2=0x55f09a871ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5127fff640 1 --2- 172.21.15.3:0/934457261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a871ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5127fff640 1 -- 172.21.15.3:0/934457261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f09a79b530 con 0x55f09a4ab4d0 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.380+0000 7f5127fff640 1 --2- 172.21.15.3:0/934457261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 0x55f09a872430 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f51140e5130 tx=0x7f51140f0570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821fdac66a2ae040 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.381+0000 7f5112ffd640 1 -- 172.21.15.3:0/934457261 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140ebe40 con 0x55f09a4ab4d0 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.381+0000 7f5112ffd640 1 -- 172.21.15.3:0/934457261 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51140ee070 con 0x55f09a4ab4d0 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.381+0000 7f5112ffd640 1 -- 172.21.15.3:0/934457261 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140f4710 con 0x55f09a4ab4d0 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.381+0000 7f5125ffb640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51180b6680 0x7f51180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.381+0000 7f512d160640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51180b6680 0x7f51180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.381+0000 7f5125ffb640 1 -- 172.21.15.3:0/1709828045 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f51140e7a30 con 0x7f5128108310 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.381+0000 7f5135c0f740 1 -- 172.21.15.3:0/934457261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 msgr2=0x55f09a872430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.381+0000 7f512d160640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51180b6680 0x7f51180b8b90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5120005fd0 tx=0x7f5120009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.381+0000 7f5135c0f740 1 --2- 172.21.15.3:0/934457261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 0x55f09a872430 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f51140e5130 tx=0x7f51140f0570 comp rx=0 tx=0).stop 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.382+0000 7f5135c0f740 1 -- 172.21.15.3:0/934457261 shutdown_connections 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.382+0000 7f5135c0f740 1 --2- 172.21.15.3:0/934457261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 0x55f09a872430 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.382+0000 7f5135c0f740 1 --2- 172.21.15.3:0/934457261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a871ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.382+0000 7f5135c0f740 1 --2- 172.21.15.3:0/934457261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4cc1b0 0x55f09a8719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.382+0000 7f5135c0f740 1 -- 172.21.15.3:0/934457261 >> 172.21.15.3:0/934457261 conn(0x55f09a4c0300 msgr2=0x55f09a86f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.382+0000 7f5135c0f740 1 -- 172.21.15.3:0/934457261 shutdown_connections 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.382+0000 7f5135c0f740 1 -- 172.21.15.3:0/934457261 wait complete. 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.382+0000 7f5135c0f740 1 -- start start 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.382+0000 7f5135c0f740 1 Processor -- start 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f5135c0f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a87d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f5135c0f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4ab4d0 0x55f09a87d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f5135c0f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 0x55f09a889420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f5135c0f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f09a70dab0 con 0x55f09a4cc1b0 2024-09-21T06:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f5135c0f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f09a77b730 con 0x55f09a4a32f0 2024-09-21T06:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f5135c0f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f09a880130 con 0x55f09a4ab4d0 2024-09-21T06:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f5135c0f740 10 client.0 _ll_register_callbacks cb 0x55f09a7cf6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f512d160640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 0x55f09a889420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f512c95f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a87d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f512d160640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 0x55f09a889420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49986/0 (socket says 172.21.15.3:49986) 2024-09-21T06:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f512d160640 1 -- 172.21.15.3:0/1896813424 learned_addr learned my addr 172.21.15.3:0/1896813424 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f5127fff640 1 --2- 172.21.15.3:0/1896813424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4ab4d0 0x55f09a87d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f512d160640 1 -- 172.21.15.3:0/1896813424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4ab4d0 msgr2=0x55f09a87d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f512d160640 1 --2- 172.21.15.3:0/1896813424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4ab4d0 0x55f09a87d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f512d160640 1 -- 172.21.15.3:0/1896813424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 msgr2=0x55f09a87d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f512d160640 1 --2- 172.21.15.3:0/1896813424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a87d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f512d160640 1 -- 172.21.15.3:0/1896813424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f09a537790 con 0x55f09a4cc1b0 2024-09-21T06:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f512c95f640 1 --2- 172.21.15.3:0/1896813424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a87d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f512d160640 1 --2- 172.21.15.3:0/1896813424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 0x55f09a889420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5120003fd0 tx=0x7f5120004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c3ecc718cdf51a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f5107fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.383+0000 7f5107fff640 1 -- 172.21.15.3:0/1896813424 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512000f8b0 con 0x55f09a4cc1b0 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f5107fff640 1 -- 172.21.15.3:0/1896813424 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f512000fa50 con 0x55f09a4cc1b0 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f5135c0f740 1 -- 172.21.15.3:0/1896813424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51140ee070 con 0x55f09a4cc1b0 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f5107fff640 1 -- 172.21.15.3:0/1896813424 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f512000fc50 con 0x55f09a4cc1b0 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f5135c0f740 10 client.16995 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f5135c0f740 1 -- 172.21.15.3:0/1896813424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f09a5b0620 con 0x55f09a4cc1b0 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f5107fff640 1 -- 172.21.15.3:0/1896813424 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51140ee070 con 0x55f09a4cc1b0 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f5107fff640 1 -- 172.21.15.3:0/1896813424 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f5120026070 con 0x55f09a4cc1b0 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f5135c0f740 10 client.16995 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f5135c0f740 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f5135c0f740 20 client.16995 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f5135c0f740 1 --2- 172.21.15.3:0/1896813424 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f09a48ad80 0x55f09a88f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f5135c0f740 4 client.16995 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f5135c0f740 1 -- 172.21.15.3:0/1896813424 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f09a7e8640 con 0x55f09a48ad80 2024-09-21T06:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.384+0000 7f512c95f640 1 --2- 172.21.15.3:0/1896813424 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f09a48ad80 0x55f09a88f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.385+0000 7f5107fff640 10 client.16995 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.385+0000 7f512c95f640 1 --2- 172.21.15.3:0/1896813424 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f09a48ad80 0x55f09a88f250 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=160fe84784761373 server_cookie=86e56cc064fa470f in_seq=0 out_seq=0 2024-09-21T06:20:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5107fff640 1 -- 172.21.15.3:0/1896813424 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f09a7e8640 con 0x55f09a48ad80 2024-09-21T06:20:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5107fff640 10 client.16995 handle_command_reply: tid=0 2024-09-21T06:20:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5135c0f740 1 client.16995 shutdown 2024-09-21T06:20:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5135c0f740 1 -- 172.21.15.3:0/1896813424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 msgr2=0x55f09a889420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5135c0f740 1 --2- 172.21.15.3:0/1896813424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 0x55f09a889420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5120003fd0 tx=0x7f5120004ab0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5135c0f740 1 -- 172.21.15.3:0/1896813424 shutdown_connections 2024-09-21T06:20:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5135c0f740 1 --2- 172.21.15.3:0/1896813424 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f09a48ad80 0x55f09a88f250 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5135c0f740 1 --2- 172.21.15.3:0/1896813424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4ab4d0 0x55f09a87d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5135c0f740 1 --2- 172.21.15.3:0/1896813424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 0x55f09a889420 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5135c0f740 1 --2- 172.21.15.3:0/1896813424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a87d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5107fff640 0 client.16995 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5135c0f740 1 -- 172.21.15.3:0/1896813424 >> 172.21.15.3:0/1896813424 conn(0x55f09a4c0300 msgr2=0x55f09a8729e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5135c0f740 1 -- 172.21.15.3:0/1896813424 shutdown_connections 2024-09-21T06:20:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5135c0f740 1 -- 172.21.15.3:0/1896813424 wait complete. 2024-09-21T06:20:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.386+0000 7f5135c0f740 20 client.16995 trim_cache size 0 max 16384 2024-09-21T06:20:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f5135c0f740 1 Processor -- start 2024-09-21T06:20:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f5135c0f740 1 -- start start 2024-09-21T06:20:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f5135c0f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4a32f0 0x55f09a87d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f5135c0f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 0x55f09a87ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f5135c0f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4cc1b0 0x55f09a8730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f5135c0f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f51281af770 con 0x55f09a4ab4d0 2024-09-21T06:20:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f5135c0f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51281b12e0 con 0x55f09a4cc1b0 2024-09-21T06:20:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f5135c0f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51281afe40 con 0x55f09a4a32f0 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f5127fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 0x55f09a87ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f5127fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 0x55f09a87ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49990/0 (socket says 172.21.15.3:49990) 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f5127fff640 1 -- 172.21.15.3:0/4024757016 learned_addr learned my addr 172.21.15.3:0/4024757016 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f512c95f640 1 --2- 172.21.15.3:0/4024757016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4a32f0 0x55f09a87d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f512d160640 1 --2- 172.21.15.3:0/4024757016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4cc1b0 0x55f09a8730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f5127fff640 1 -- 172.21.15.3:0/4024757016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4a32f0 msgr2=0x55f09a87d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.402+0000 7f5127fff640 1 --2- 172.21.15.3:0/4024757016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4a32f0 0x55f09a87d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.403+0000 7f5127fff640 1 -- 172.21.15.3:0/4024757016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4cc1b0 msgr2=0x55f09a8730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.403+0000 7f5127fff640 1 --2- 172.21.15.3:0/4024757016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4cc1b0 0x55f09a8730f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.403+0000 7f5127fff640 1 -- 172.21.15.3:0/4024757016 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f09a87b580 con 0x55f09a4ab4d0 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.403+0000 7f512c95f640 1 --2- 172.21.15.3:0/4024757016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4a32f0 0x55f09a87d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.403+0000 7f5127fff640 1 --2- 172.21.15.3:0/4024757016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 0x55f09a87ef80 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f51140e5270 tx=0x7f5114009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd89b5f7c1f057e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.403+0000 7f51127fc640 1 -- 172.21.15.3:0/4024757016 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140a9760 con 0x55f09a4ab4d0 2024-09-21T06:20:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.403+0000 7f51127fc640 1 -- 172.21.15.3:0/4024757016 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51140ee070 con 0x55f09a4ab4d0 2024-09-21T06:20:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.403+0000 7f51127fc640 1 -- 172.21.15.3:0/4024757016 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140a9030 con 0x55f09a4ab4d0 2024-09-21T06:20:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.404+0000 7f5135c0f740 1 -- 172.21.15.3:0/4024757016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 msgr2=0x55f09a87ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.404+0000 7f5135c0f740 1 --2- 172.21.15.3:0/4024757016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 0x55f09a87ef80 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f51140e5270 tx=0x7f5114009740 comp rx=0 tx=0).stop 2024-09-21T06:20:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.404+0000 7f5135c0f740 1 -- 172.21.15.3:0/4024757016 shutdown_connections 2024-09-21T06:20:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.404+0000 7f5135c0f740 1 --2- 172.21.15.3:0/4024757016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4cc1b0 0x55f09a8730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.404+0000 7f5135c0f740 1 --2- 172.21.15.3:0/4024757016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4ab4d0 0x55f09a87ef80 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.404+0000 7f5135c0f740 1 --2- 172.21.15.3:0/4024757016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4a32f0 0x55f09a87d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.404+0000 7f5135c0f740 1 -- 172.21.15.3:0/4024757016 >> 172.21.15.3:0/4024757016 conn(0x55f09a4c0300 msgr2=0x55f09a8729e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.404+0000 7f5135c0f740 1 -- 172.21.15.3:0/4024757016 shutdown_connections 2024-09-21T06:20:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.404+0000 7f5135c0f740 1 -- 172.21.15.3:0/4024757016 wait complete. 2024-09-21T06:20:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.404+0000 7f5135c0f740 1 -- start start 2024-09-21T06:20:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.404+0000 7f5135c0f740 1 Processor -- start 2024-09-21T06:20:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5135c0f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a880ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5135c0f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4ab4d0 0x55f09a8812a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5135c0f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 0x55f09a8f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5135c0f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f51281b0c10 con 0x55f09a4cc1b0 2024-09-21T06:20:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5135c0f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51281b0540 con 0x55f09a4a32f0 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5135c0f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51281ad750 con 0x55f09a4ab4d0 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5135c0f740 10 client.0 _ll_register_callbacks cb 0x7f5120009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f512d160640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 0x55f09a8f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f512c95f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a880ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5127fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4ab4d0 0x55f09a8812a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5127fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4ab4d0 0x55f09a8812a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:48348/0 (socket says 172.21.15.3:48348) 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f512c95f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a880ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:35362/0 (socket says 172.21.15.3:35362) 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f512d160640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 0x55f09a8f1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50006/0 (socket says 172.21.15.3:50006) 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5127fff640 1 -- 172.21.15.3:0/3215471654 learned_addr learned my addr 172.21.15.3:0/3215471654 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5127fff640 1 -- 172.21.15.3:0/3215471654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 msgr2=0x55f09a880ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5127fff640 1 --2- 172.21.15.3:0/3215471654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a880ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5127fff640 1 -- 172.21.15.3:0/3215471654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 msgr2=0x55f09a8f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5127fff640 1 --2- 172.21.15.3:0/3215471654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 0x55f09a8f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f5127fff640 1 -- 172.21.15.3:0/3215471654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f09a8f22f0 con 0x55f09a4ab4d0 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.405+0000 7f512d160640 1 --2- 172.21.15.3:0/3215471654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 0x55f09a8f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5127fff640 1 --2- 172.21.15.3:0/3215471654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4ab4d0 0x55f09a8812a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f51140f05b0 tx=0x7f511400ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86ea2d72d4f51087 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5107fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5107fff640 1 -- 172.21.15.3:0/3215471654 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140ad030 con 0x55f09a4ab4d0 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5107fff640 1 -- 172.21.15.3:0/3215471654 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f51140eb350 con 0x55f09a4ab4d0 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5135c0f740 1 -- 172.21.15.3:0/3215471654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51140ee070 con 0x55f09a4ab4d0 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5107fff640 1 -- 172.21.15.3:0/3215471654 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f51140ad210 con 0x55f09a4ab4d0 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5135c0f740 10 client.26524 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5135c0f740 1 -- 172.21.15.3:0/3215471654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f09a8f9620 con 0x55f09a4ab4d0 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5107fff640 1 -- 172.21.15.3:0/3215471654 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51140ee070 con 0x55f09a4ab4d0 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5107fff640 1 -- 172.21.15.3:0/3215471654 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f51140b6e50 con 0x55f09a4ab4d0 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5135c0f740 10 client.26524 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5135c0f740 10 client.26524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5135c0f740 20 client.26524 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.406+0000 7f5135c0f740 1 --2- 172.21.15.3:0/3215471654 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f09a4601c0 0x55f09a8fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.407+0000 7f5135c0f740 4 client.26524 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:20:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.407+0000 7f5135c0f740 1 -- 172.21.15.3:0/3215471654 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f09a7e40f0 con 0x55f09a4601c0 2024-09-21T06:20:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.407+0000 7f512c95f640 1 --2- 172.21.15.3:0/3215471654 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f09a4601c0 0x55f09a8fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.407+0000 7f5107fff640 10 client.26524 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.407+0000 7f512c95f640 1 --2- 172.21.15.3:0/3215471654 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f09a4601c0 0x55f09a8fb8b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3854e759bb29a64 server_cookie=eaa39e820d6c1a65 in_seq=0 out_seq=0 2024-09-21T06:20:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5107fff640 1 -- 172.21.15.3:0/3215471654 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f09a7e40f0 con 0x55f09a4601c0 2024-09-21T06:20:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5107fff640 10 client.26524 handle_command_reply: tid=0 2024-09-21T06:20:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5135c0f740 1 client.26524 shutdown 2024-09-21T06:20:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5135c0f740 1 -- 172.21.15.3:0/3215471654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4ab4d0 msgr2=0x55f09a8812a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5135c0f740 1 --2- 172.21.15.3:0/3215471654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4ab4d0 0x55f09a8812a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f51140f05b0 tx=0x7f511400ce60 comp rx=0 tx=0).stop 2024-09-21T06:20:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5135c0f740 1 -- 172.21.15.3:0/3215471654 shutdown_connections 2024-09-21T06:20:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5135c0f740 1 --2- 172.21.15.3:0/3215471654 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f09a4601c0 0x55f09a8fb8b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5135c0f740 1 --2- 172.21.15.3:0/3215471654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a4ab4d0 0x55f09a8812a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5107fff640 0 client.26524 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5135c0f740 1 --2- 172.21.15.3:0/3215471654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f09a4cc1b0 0x55f09a8f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5135c0f740 1 --2- 172.21.15.3:0/3215471654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09a4a32f0 0x55f09a880ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5135c0f740 1 -- 172.21.15.3:0/3215471654 >> 172.21.15.3:0/3215471654 conn(0x55f09a4c0300 msgr2=0x55f09a873350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5135c0f740 1 -- 172.21.15.3:0/3215471654 shutdown_connections 2024-09-21T06:20:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.561+0000 7f5135c0f740 1 -- 172.21.15.3:0/3215471654 wait complete. 2024-09-21T06:20:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.562+0000 7f5135c0f740 20 client.26524 trim_cache size 0 max 16384 2024-09-21T06:20:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:20:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:20:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:20:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:20:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.562+0000 7f512ebc3640 1 -- 172.21.15.3:0/1709828045 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51180b6680 msgr2=0x7f51180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.562+0000 7f512ebc3640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51180b6680 0x7f51180b8b90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5120005fd0 tx=0x7f5120009500 comp rx=0 tx=0).stop 2024-09-21T06:20:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.562+0000 7f512ebc3640 1 -- 172.21.15.3:0/1709828045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5128108310 msgr2=0x7f512819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.562+0000 7f512ebc3640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5128108310 0x7f512819aa30 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f511400e9f0 tx=0x7f511400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.562+0000 7f512ebc3640 1 -- 172.21.15.3:0/1709828045 shutdown_connections 2024-09-21T06:20:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.562+0000 7f512ebc3640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51180b6680 0x7f51180b8b90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.562+0000 7f512ebc3640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5128108c60 0x7f512819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.562+0000 7f512ebc3640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5128108310 0x7f512819aa30 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.562+0000 7f512ebc3640 1 --2- 172.21.15.3:0/1709828045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51281029e0 0x7f512819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.562+0000 7f512ebc3640 1 -- 172.21.15.3:0/1709828045 >> 172.21.15.3:0/1709828045 conn(0x7f512807a6e0 msgr2=0x7f51281019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.562+0000 7f512ebc3640 1 -- 172.21.15.3:0/1709828045 shutdown_connections 2024-09-21T06:20:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:10.562+0000 7f512ebc3640 1 -- 172.21.15.3:0/1709828045 wait complete. 2024-09-21T06:20:10.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 533442 12.65 MB/sec execute 103 sec latency 77.038 ms 2024-09-21T06:20:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:11 smithi003 ceph-mon[31227]: pgmap v480: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 475 op/s 2024-09-21T06:20:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:11 smithi137 ceph-mon[34464]: pgmap v480: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 475 op/s 2024-09-21T06:20:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:11 smithi204 ceph-mon[34659]: pgmap v480: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 475 op/s 2024-09-21T06:20:11.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 536590 12.69 MB/sec execute 104 sec latency 12.872 ms 2024-09-21T06:20:12.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 538921 12.66 MB/sec execute 105 sec latency 12.296 ms 2024-09-21T06:20:13.446 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:13 smithi137 ceph-mon[34464]: pgmap v481: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-09-21T06:20:13.455 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:13 smithi204 ceph-mon[34659]: pgmap v481: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-09-21T06:20:13.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:13 smithi003 ceph-mon[31227]: pgmap v481: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-09-21T06:20:13.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 540702 12.65 MB/sec execute 106 sec latency 58.860 ms 2024-09-21T06:20:14.955 INFO:tasks.workunit.client.0.smithi003.stdout: 1 543655 12.70 MB/sec execute 107 sec latency 19.685 ms 2024-09-21T06:20:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:15 smithi003 ceph-mon[31227]: pgmap v482: 97 pgs: 97 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-09-21T06:20:15.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:15 smithi137 ceph-mon[34464]: pgmap v482: 97 pgs: 97 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-09-21T06:20:15.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:15 smithi204 ceph-mon[34659]: pgmap v482: 97 pgs: 97 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-09-21T06:20:15.694 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:20:15.956 INFO:tasks.workunit.client.0.smithi003.stdout: 1 545188 12.64 MB/sec execute 108 sec latency 19.837 ms 2024-09-21T06:20:15.967 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.502+0000 7f69ae98c640 1 -- 172.21.15.3:0/2124394642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a81081b0 msgr2=0x7f69a81085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.502+0000 7f69ae98c640 1 --2- 172.21.15.3:0/2124394642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a81081b0 0x7f69a81085c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6998009900 tx=0x7f699802a410 comp rx=0 tx=0).stop 2024-09-21T06:20:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.502+0000 7f69ae98c640 1 -- 172.21.15.3:0/2124394642 shutdown_connections 2024-09-21T06:20:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.502+0000 7f69ae98c640 1 --2- 172.21.15.3:0/2124394642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69a8108b00 0x7f69a810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.502+0000 7f69ae98c640 1 --2- 172.21.15.3:0/2124394642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a81081b0 0x7f69a81085c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.502+0000 7f69ae98c640 1 --2- 172.21.15.3:0/2124394642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69a8102870 0x7f69a8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.502+0000 7f69ae98c640 1 -- 172.21.15.3:0/2124394642 >> 172.21.15.3:0/2124394642 conn(0x7f69a80fe2c0 msgr2=0x7f69a8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.502+0000 7f69ae98c640 1 -- 172.21.15.3:0/2124394642 shutdown_connections 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.502+0000 7f69ae98c640 1 -- 172.21.15.3:0/2124394642 wait complete. 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.503+0000 7f69ae98c640 1 Processor -- start 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.503+0000 7f69ae98c640 1 -- start start 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.503+0000 7f69ae98c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69a8102870 0x7f69a819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ae98c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a81081b0 0x7f69a819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ae98c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69a8108b00 0x7f69a819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ae98c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8112770 con 0x7f69a81081b0 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ae98c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81125f0 con 0x7f69a8102870 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ae98c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81128f0 con 0x7f69a8108b00 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ad189640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a81081b0 0x7f69a819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ad189640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a81081b0 0x7f69a819a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54630/0 (socket says 172.21.15.3:54630) 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ad189640 1 -- 172.21.15.3:0/867970667 learned_addr learned my addr 172.21.15.3:0/867970667 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ad98a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69a8102870 0x7f69a819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ae18b640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69a8108b00 0x7f69a819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ad189640 1 -- 172.21.15.3:0/867970667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69a8108b00 msgr2=0x7f69a819fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ad189640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69a8108b00 0x7f69a819fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ad189640 1 -- 172.21.15.3:0/867970667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69a8102870 msgr2=0x7f69a819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ad189640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69a8102870 0x7f69a819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ad189640 1 -- 172.21.15.3:0/867970667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a81a0240 con 0x7f69a81081b0 2024-09-21T06:20:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ad98a640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69a8102870 0x7f69a819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.504+0000 7f69ad189640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a81081b0 0x7f69a819a870 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f699802a930 tx=0x7f6998031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.505+0000 7f699effd640 1 -- 172.21.15.3:0/867970667 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699803f030 con 0x7f69a81081b0 2024-09-21T06:20:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.505+0000 7f69ae98c640 1 -- 172.21.15.3:0/867970667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a81a04d0 con 0x7f69a81081b0 2024-09-21T06:20:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.505+0000 7f699effd640 1 -- 172.21.15.3:0/867970667 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69980043d0 con 0x7f69a81081b0 2024-09-21T06:20:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.505+0000 7f699effd640 1 -- 172.21.15.3:0/867970667 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998035070 con 0x7f69a81081b0 2024-09-21T06:20:16.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.505+0000 7f69ae98c640 1 -- 172.21.15.3:0/867970667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a81a6c90 con 0x7f69a81081b0 2024-09-21T06:20:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.507+0000 7f69b6162740 1 Processor -- start 2024-09-21T06:20:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.508+0000 7f69b6162740 1 -- start start 2024-09-21T06:20:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.508+0000 7f69b6162740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba9202070 0x555ba95a9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.508+0000 7f69b6162740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91d91b0 0x555ba95a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.508+0000 7f69b6162740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba91e1390 0x555ba95aa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.508+0000 7f69b6162740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555ba951fe50 con 0x555ba9202070 2024-09-21T06:20:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.508+0000 7f69b6162740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555ba950f280 con 0x555ba91d91b0 2024-09-21T06:20:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.508+0000 7f69b6162740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555ba9513090 con 0x555ba91e1390 2024-09-21T06:20:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.508+0000 7f69ae18b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba9202070 0x555ba95a9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.508+0000 7f69ae18b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba9202070 0x555ba95a9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54642/0 (socket says 172.21.15.3:54642) 2024-09-21T06:20:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.508+0000 7f69ae18b640 1 -- 172.21.15.3:0/270851807 learned_addr learned my addr 172.21.15.3:0/270851807 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.508+0000 7f69ad98a640 1 --2- 172.21.15.3:0/270851807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91d91b0 0x555ba95a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.508+0000 7f699effd640 1 -- 172.21.15.3:0/867970667 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f6998042030 con 0x7f69a81081b0 2024-09-21T06:20:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.509+0000 7f69ad98a640 1 -- 172.21.15.3:0/270851807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba91e1390 msgr2=0x555ba95aa290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:20:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.509+0000 7f69ad98a640 1 --2- 172.21.15.3:0/270851807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba91e1390 0x555ba95aa290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.509+0000 7f69ad98a640 1 -- 172.21.15.3:0/270851807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba9202070 msgr2=0x555ba95a9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.509+0000 7f69ad98a640 1 --2- 172.21.15.3:0/270851807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba9202070 0x555ba95a9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.509+0000 7f69ad98a640 1 -- 172.21.15.3:0/270851807 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ba926d650 con 0x555ba91d91b0 2024-09-21T06:20:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.509+0000 7f69ae18b640 1 --2- 172.21.15.3:0/270851807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba9202070 0x555ba95a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.509+0000 7f69ad98a640 1 --2- 172.21.15.3:0/270851807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91d91b0 0x555ba95a9d50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009fd0 tx=0x7f69a40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f9cb676b4d1a77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.509+0000 7f6993fff640 1 -- 172.21.15.3:0/270851807 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4004030 con 0x555ba91d91b0 2024-09-21T06:20:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.509+0000 7f6993fff640 1 -- 172.21.15.3:0/270851807 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a40054d0 con 0x555ba91d91b0 2024-09-21T06:20:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.509+0000 7f6993fff640 1 -- 172.21.15.3:0/270851807 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4004c40 con 0x555ba91d91b0 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.510+0000 7f699effd640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f69940b6680 0x7f69940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.510+0000 7f69ae18b640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f69940b6680 0x7f69940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.510+0000 7f699effd640 1 -- 172.21.15.3:0/867970667 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f6998107310 con 0x7f69a81081b0 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.510+0000 7f69b6162740 1 -- 172.21.15.3:0/270851807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91d91b0 msgr2=0x555ba95a9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.510+0000 7f69b6162740 1 --2- 172.21.15.3:0/270851807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91d91b0 0x555ba95a9d50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009fd0 tx=0x7f69a40118d0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.510+0000 7f69b6162740 1 -- 172.21.15.3:0/270851807 shutdown_connections 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.510+0000 7f69b6162740 1 --2- 172.21.15.3:0/270851807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba91e1390 0x555ba95aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.510+0000 7f69b6162740 1 --2- 172.21.15.3:0/270851807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91d91b0 0x555ba95a9d50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.510+0000 7f69b6162740 1 --2- 172.21.15.3:0/270851807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba9202070 0x555ba95a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.510+0000 7f69b6162740 1 -- 172.21.15.3:0/270851807 >> 172.21.15.3:0/270851807 conn(0x555ba91f6460 msgr2=0x555ba95a7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.510+0000 7f69ae18b640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f69940b6680 0x7f69940b8b90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f69a0012840 tx=0x7f69a0009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.510+0000 7f69b6162740 1 -- 172.21.15.3:0/270851807 shutdown_connections 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.510+0000 7f69b6162740 1 -- 172.21.15.3:0/270851807 wait complete. 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.511+0000 7f69b6162740 1 -- start start 2024-09-21T06:20:16.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.511+0000 7f69b6162740 1 Processor -- start 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.511+0000 7f69b6162740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.511+0000 7f69b6162740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95b5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.511+0000 7f69b6162740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.511+0000 7f69b6162740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555ba9370700 con 0x555ba91d91b0 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.511+0000 7f69b6162740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555ba94aa110 con 0x555ba91e1390 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.511+0000 7f69b6162740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555ba95bc400 con 0x555ba9202070 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.512+0000 7f69b6162740 10 client.0 _ll_register_callbacks cb 0x555ba94c3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.512+0000 7f69ad98a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.512+0000 7f69ad98a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54658/0 (socket says 172.21.15.3:54658) 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.512+0000 7f69ae18b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.512+0000 7f69ad98a640 1 -- 172.21.15.3:0/680467668 learned_addr learned my addr 172.21.15.3:0/680467668 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.512+0000 7f69ad189640 1 --2- 172.21.15.3:0/680467668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.512+0000 7f69ae18b640 1 -- 172.21.15.3:0/680467668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 msgr2=0x555ba95b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:16.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.512+0000 7f69ae18b640 1 --2- 172.21.15.3:0/680467668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.512+0000 7f69ae18b640 1 -- 172.21.15.3:0/680467668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 msgr2=0x555ba95b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:16.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.512+0000 7f69ae18b640 1 --2- 172.21.15.3:0/680467668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.512+0000 7f69ae18b640 1 -- 172.21.15.3:0/680467668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ba94d7c40 con 0x555ba9202070 2024-09-21T06:20:16.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.512+0000 7f69ae18b640 1 --2- 172.21.15.3:0/680467668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95c5720 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f69a0003ab0 tx=0x7f69a0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74e4fd3f77ca7301 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:16.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.512+0000 7f69917fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:20:16.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.513+0000 7f69917fa640 1 -- 172.21.15.3:0/680467668 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0004250 con 0x555ba9202070 2024-09-21T06:20:16.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.513+0000 7f69b6162740 1 -- 172.21.15.3:0/680467668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69a40054d0 con 0x555ba9202070 2024-09-21T06:20:16.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.513+0000 7f69917fa640 1 -- 172.21.15.3:0/680467668 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f69a00043f0 con 0x555ba9202070 2024-09-21T06:20:16.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.513+0000 7f69917fa640 1 -- 172.21.15.3:0/680467668 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f69a0004630 con 0x555ba9202070 2024-09-21T06:20:16.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.513+0000 7f69b6162740 10 client.26533 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:16.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.513+0000 7f69b6162740 1 -- 172.21.15.3:0/680467668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555ba95c8200 con 0x555ba9202070 2024-09-21T06:20:16.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.513+0000 7f69917fa640 1 -- 172.21.15.3:0/680467668 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a40054d0 con 0x555ba9202070 2024-09-21T06:20:16.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.514+0000 7f69917fa640 1 -- 172.21.15.3:0/680467668 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f69a0027070 con 0x555ba9202070 2024-09-21T06:20:16.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.514+0000 7f69b6162740 10 client.26533 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.514+0000 7f69b6162740 10 client.26533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.514+0000 7f69b6162740 20 client.26533 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.514+0000 7f69b6162740 1 --2- 172.21.15.3:0/680467668 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555ba91c09f0 0x555ba95cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.514+0000 7f69b6162740 4 client.26533 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.514+0000 7f69b6162740 1 -- 172.21.15.3:0/680467668 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ba9521da0 con 0x555ba91c09f0 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.514+0000 7f69ad98a640 1 --2- 172.21.15.3:0/680467668 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555ba91c09f0 0x555ba95cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.514+0000 7f69ad98a640 1 --2- 172.21.15.3:0/680467668 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555ba91c09f0 0x555ba95cb6b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e003ec02c3392e server_cookie=675c54859ff2715b in_seq=0 out_seq=0 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.514+0000 7f69917fa640 10 client.26533 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.515+0000 7f69917fa640 1 -- 172.21.15.3:0/680467668 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555ba9521da0 con 0x555ba91c09f0 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.515+0000 7f69917fa640 10 client.26533 handle_command_reply: tid=0 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.515+0000 7f69b6162740 1 client.26533 shutdown 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.515+0000 7f69b6162740 1 -- 172.21.15.3:0/680467668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 msgr2=0x555ba95c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.515+0000 7f69b6162740 1 --2- 172.21.15.3:0/680467668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95c5720 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f69a0003ab0 tx=0x7f69a0004ab0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.516+0000 7f69b6162740 1 -- 172.21.15.3:0/680467668 shutdown_connections 2024-09-21T06:20:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.516+0000 7f69b6162740 1 --2- 172.21.15.3:0/680467668 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555ba91c09f0 0x555ba95cb6b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.516+0000 7f69b6162740 1 --2- 172.21.15.3:0/680467668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95c5720 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.516+0000 7f69b6162740 1 --2- 172.21.15.3:0/680467668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.516+0000 7f69b6162740 1 --2- 172.21.15.3:0/680467668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.516+0000 7f69917fa640 0 client.26533 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.516+0000 7f69b6162740 1 -- 172.21.15.3:0/680467668 >> 172.21.15.3:0/680467668 conn(0x555ba91f6460 msgr2=0x555ba95ac9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.516+0000 7f69b6162740 1 -- 172.21.15.3:0/680467668 shutdown_connections 2024-09-21T06:20:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.516+0000 7f69b6162740 1 -- 172.21.15.3:0/680467668 wait complete. 2024-09-21T06:20:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.516+0000 7f69b6162740 20 client.26533 trim_cache size 0 max 16384 2024-09-21T06:20:16.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.535+0000 7f69b6162740 1 Processor -- start 2024-09-21T06:20:16.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.535+0000 7f69b6162740 1 -- start start 2024-09-21T06:20:16.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.535+0000 7f69b6162740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.535+0000 7f69b6162740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95ac910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.535+0000 7f69b6162740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.535+0000 7f69b6162740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81ad2c0 con 0x555ba91d91b0 2024-09-21T06:20:16.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.535+0000 7f69b6162740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81aee30 con 0x555ba91e1390 2024-09-21T06:20:16.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.535+0000 7f69b6162740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81ad990 con 0x555ba9202070 2024-09-21T06:20:16.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.535+0000 7f69ad98a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.535+0000 7f69ae18b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.535+0000 7f69ad98a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54672/0 (socket says 172.21.15.3:54672) 2024-09-21T06:20:16.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.535+0000 7f69ad98a640 1 -- 172.21.15.3:0/3797226830 learned_addr learned my addr 172.21.15.3:0/3797226830 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:16.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.535+0000 7f69ad189640 1 --2- 172.21.15.3:0/3797226830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.536+0000 7f69ad98a640 1 -- 172.21.15.3:0/3797226830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 msgr2=0x555ba95ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:16.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.536+0000 7f69ad98a640 1 --2- 172.21.15.3:0/3797226830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.536+0000 7f69ad98a640 1 -- 172.21.15.3:0/3797226830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 msgr2=0x555ba95ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:16.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.536+0000 7f69ad98a640 1 --2- 172.21.15.3:0/3797226830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95ac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.536+0000 7f69ad98a640 1 -- 172.21.15.3:0/3797226830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ba95b3440 con 0x555ba91d91b0 2024-09-21T06:20:16.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.536+0000 7f69ae18b640 1 --2- 172.21.15.3:0/3797226830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:16.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.536+0000 7f69ad189640 1 --2- 172.21.15.3:0/3797226830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:16.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.536+0000 7f69ad98a640 1 --2- 172.21.15.3:0/3797226830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95b5400 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f69a40058c0 tx=0x7f69a4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6a69dd08f16198 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:16.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.536+0000 7f69937fe640 1 -- 172.21.15.3:0/3797226830 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4031070 con 0x555ba91d91b0 2024-09-21T06:20:16.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.536+0000 7f69937fe640 1 -- 172.21.15.3:0/3797226830 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a4009b50 con 0x555ba91d91b0 2024-09-21T06:20:16.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.536+0000 7f69937fe640 1 -- 172.21.15.3:0/3797226830 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a40196b0 con 0x555ba91d91b0 2024-09-21T06:20:16.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.537+0000 7f69b6162740 1 -- 172.21.15.3:0/3797226830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 msgr2=0x555ba95b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:16.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.537+0000 7f69b6162740 1 --2- 172.21.15.3:0/3797226830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95b5400 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f69a40058c0 tx=0x7f69a4009310 comp rx=0 tx=0).stop 2024-09-21T06:20:16.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.537+0000 7f69b6162740 1 -- 172.21.15.3:0/3797226830 shutdown_connections 2024-09-21T06:20:16.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.537+0000 7f69b6162740 1 --2- 172.21.15.3:0/3797226830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.537+0000 7f69b6162740 1 --2- 172.21.15.3:0/3797226830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.537+0000 7f69b6162740 1 --2- 172.21.15.3:0/3797226830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95b5400 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.537+0000 7f69b6162740 1 -- 172.21.15.3:0/3797226830 >> 172.21.15.3:0/3797226830 conn(0x555ba91f6460 msgr2=0x555ba95b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:16.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.537+0000 7f69b6162740 1 -- 172.21.15.3:0/3797226830 shutdown_connections 2024-09-21T06:20:16.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.537+0000 7f69b6162740 1 -- 172.21.15.3:0/3797226830 wait complete. 2024-09-21T06:20:16.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.538+0000 7f69b6162740 1 -- start start 2024-09-21T06:20:16.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.538+0000 7f69b6162740 1 Processor -- start 2024-09-21T06:20:16.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.538+0000 7f69b6162740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95bd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.538+0000 7f69b6162740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95bdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.538+0000 7f69b6162740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95be2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.538+0000 7f69b6162740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81ae760 con 0x555ba91d91b0 2024-09-21T06:20:16.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69b6162740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81ae090 con 0x555ba91e1390 2024-09-21T06:20:16.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69b6162740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81ab2a0 con 0x555ba9202070 2024-09-21T06:20:16.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69b6162740 10 client.0 _ll_register_callbacks cb 0x7f69a0012e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:16.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69ad98a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95bd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69ad98a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95bd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54674/0 (socket says 172.21.15.3:54674) 2024-09-21T06:20:16.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69ad98a640 1 -- 172.21.15.3:0/4113289719 learned_addr learned my addr 172.21.15.3:0/4113289719 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:16.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69ae18b640 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95be2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69ad189640 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95bdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69ad98a640 1 -- 172.21.15.3:0/4113289719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 msgr2=0x555ba95be2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69ad98a640 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95be2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69ad98a640 1 -- 172.21.15.3:0/4113289719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 msgr2=0x555ba95bdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69ad98a640 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95bdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69ad98a640 1 -- 172.21.15.3:0/4113289719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ba96352b0 con 0x555ba91d91b0 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69ad189640 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95bdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69ae18b640 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95be2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69ad98a640 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95bd830 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009250 tx=0x7f69a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37a2263c0e99023 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69917fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69917fa640 1 -- 172.21.15.3:0/4113289719 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4031050 con 0x555ba91d91b0 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69917fa640 1 -- 172.21.15.3:0/4113289719 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f69a402a040 con 0x555ba91d91b0 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.539+0000 7f69b6162740 1 -- 172.21.15.3:0/4113289719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69a4009b50 con 0x555ba91d91b0 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.540+0000 7f69917fa640 1 -- 172.21.15.3:0/4113289719 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f69a400fb90 con 0x555ba91d91b0 2024-09-21T06:20:16.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.540+0000 7f69b6162740 10 client.17019 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.540+0000 7f69b6162740 1 -- 172.21.15.3:0/4113289719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555ba9636520 con 0x555ba91d91b0 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.540+0000 7f69917fa640 1 -- 172.21.15.3:0/4113289719 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a4009b50 con 0x555ba91d91b0 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.540+0000 7f69917fa640 1 -- 172.21.15.3:0/4113289719 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f69a4024410 con 0x555ba91d91b0 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.540+0000 7f69b6162740 10 client.17019 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.540+0000 7f69b6162740 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.540+0000 7f69b6162740 20 client.17019 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.540+0000 7f69b6162740 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555ba9195da0 0x555ba9638a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.540+0000 7f69b6162740 4 client.17019 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.540+0000 7f69b6162740 1 -- 172.21.15.3:0/4113289719 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ba951d850 con 0x555ba9195da0 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.540+0000 7f69ad189640 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555ba9195da0 0x555ba9638a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.541+0000 7f69917fa640 10 client.17019 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.541+0000 7f69ad189640 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555ba9195da0 0x555ba9638a50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f15f8291a7c78b server_cookie=e160983b93d62d04 in_seq=0 out_seq=0 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.541+0000 7f69917fa640 1 -- 172.21.15.3:0/4113289719 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x555ba951d850 con 0x555ba9195da0 2024-09-21T06:20:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.541+0000 7f69917fa640 10 client.17019 handle_command_reply: tid=0 2024-09-21T06:20:16.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.541+0000 7f69b6162740 1 client.17019 shutdown 2024-09-21T06:20:16.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.541+0000 7f69b6162740 1 -- 172.21.15.3:0/4113289719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 msgr2=0x555ba95bd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:16.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.542+0000 7f69b6162740 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95bd830 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009250 tx=0x7f69a4004750 comp rx=0 tx=0).stop 2024-09-21T06:20:16.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.542+0000 7f69b6162740 1 -- 172.21.15.3:0/4113289719 shutdown_connections 2024-09-21T06:20:16.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.542+0000 7f69b6162740 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x555ba9195da0 0x555ba9638a50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.542+0000 7f69b6162740 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ba9202070 0x555ba95be2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.542+0000 7f69b6162740 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ba91e1390 0x555ba95bdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.542+0000 7f69b6162740 1 --2- 172.21.15.3:0/4113289719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555ba91d91b0 0x555ba95bd830 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.542+0000 7f69b6162740 1 -- 172.21.15.3:0/4113289719 >> 172.21.15.3:0/4113289719 conn(0x555ba91f6460 msgr2=0x555ba95b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:16.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.542+0000 7f69917fa640 0 client.17019 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:16.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.542+0000 7f69b6162740 1 -- 172.21.15.3:0/4113289719 shutdown_connections 2024-09-21T06:20:16.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.542+0000 7f69b6162740 1 -- 172.21.15.3:0/4113289719 wait complete. 2024-09-21T06:20:16.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.542+0000 7f69b6162740 20 client.17019 trim_cache size 0 max 16384 2024-09-21T06:20:16.548 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:20:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:20:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:20:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:20:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:20:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:20:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:20:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:20:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:20:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:20:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:20:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:20:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:20:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:20:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:20:16.553 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:20:16.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.543+0000 7f69ae98c640 1 -- 172.21.15.3:0/867970667 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f69940b6680 msgr2=0x7f69940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:16.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.543+0000 7f69ae98c640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f69940b6680 0x7f69940b8b90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f69a0012840 tx=0x7f69a0009500 comp rx=0 tx=0).stop 2024-09-21T06:20:16.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.543+0000 7f69ae98c640 1 -- 172.21.15.3:0/867970667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a81081b0 msgr2=0x7f69a819a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:16.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.543+0000 7f69ae98c640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a81081b0 0x7f69a819a870 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f699802a930 tx=0x7f6998031a20 comp rx=0 tx=0).stop 2024-09-21T06:20:16.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.543+0000 7f69ae98c640 1 -- 172.21.15.3:0/867970667 shutdown_connections 2024-09-21T06:20:16.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.543+0000 7f69ae98c640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f69940b6680 0x7f69940b8b90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.543+0000 7f69ae98c640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69a8108b00 0x7f69a819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.543+0000 7f69ae98c640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a81081b0 0x7f69a819a870 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.543+0000 7f69ae98c640 1 --2- 172.21.15.3:0/867970667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69a8102870 0x7f69a819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:16.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.543+0000 7f69ae98c640 1 -- 172.21.15.3:0/867970667 >> 172.21.15.3:0/867970667 conn(0x7f69a80fe2c0 msgr2=0x7f69a81068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:16.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.543+0000 7f69ae98c640 1 -- 172.21.15.3:0/867970667 shutdown_connections 2024-09-21T06:20:16.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:16.543+0000 7f69ae98c640 1 -- 172.21.15.3:0/867970667 wait complete. 2024-09-21T06:20:16.956 INFO:tasks.workunit.client.0.smithi003.stdout: 1 547501 12.65 MB/sec execute 109 sec latency 39.283 ms 2024-09-21T06:20:17.581 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:17 smithi003 ceph-mon[31227]: pgmap v483: 97 pgs: 97 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 436 op/s 2024-09-21T06:20:17.611 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:20:17.611 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:20:17.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:17 smithi137 ceph-mon[34464]: pgmap v483: 97 pgs: 97 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 436 op/s 2024-09-21T06:20:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:17 smithi204 ceph-mon[34659]: pgmap v483: 97 pgs: 97 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 436 op/s 2024-09-21T06:20:17.882 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:17.956 INFO:tasks.workunit.client.0.smithi003.stdout: 1 550510 12.68 MB/sec execute 110 sec latency 22.168 ms 2024-09-21T06:20:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.558+0000 7fb909cd6640 1 -- 172.21.15.3:0/1707503358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb904106c20 msgr2=0x7fb904110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.558+0000 7fb909cd6640 1 --2- 172.21.15.3:0/1707503358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb904106c20 0x7fb904110670 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009ca0 tx=0x7fb8fc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:20:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.558+0000 7fb909cd6640 1 -- 172.21.15.3:0/1707503358 shutdown_connections 2024-09-21T06:20:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.558+0000 7fb909cd6640 1 --2- 172.21.15.3:0/1707503358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9041062d0 0x7fb9041066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.558+0000 7fb909cd6640 1 --2- 172.21.15.3:0/1707503358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb904106c20 0x7fb904110670 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.558+0000 7fb909cd6640 1 --2- 172.21.15.3:0/1707503358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb904101ff0 0x7fb904102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.558+0000 7fb909cd6640 1 -- 172.21.15.3:0/1707503358 >> 172.21.15.3:0/1707503358 conn(0x7fb904079e30 msgr2=0x7fb90407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.558+0000 7fb909cd6640 1 -- 172.21.15.3:0/1707503358 shutdown_connections 2024-09-21T06:20:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.558+0000 7fb909cd6640 1 -- 172.21.15.3:0/1707503358 wait complete. 2024-09-21T06:20:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.559+0000 7fb909cd6640 1 Processor -- start 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.559+0000 7fb909cd6640 1 -- start start 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.559+0000 7fb909cd6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb904101ff0 0x7fb904102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.559+0000 7fb909cd6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9041062d0 0x7fb9041034d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.559+0000 7fb909cd6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb904106c20 0x7fb904103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.559+0000 7fb909cd6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb904113ac0 con 0x7fb904106c20 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.559+0000 7fb909cd6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb904113940 con 0x7fb904101ff0 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.559+0000 7fb909cd6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb904113c40 con 0x7fb9041062d0 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb903fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb904106c20 0x7fb904103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb9037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb904101ff0 0x7fb904102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb903fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb904106c20 0x7fb904103a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54686/0 (socket says 172.21.15.3:54686) 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb903fff640 1 -- 172.21.15.3:0/3134210561 learned_addr learned my addr 172.21.15.3:0/3134210561 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb9037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb904101ff0 0x7fb904102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:42880/0 (socket says 172.21.15.3:42880) 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb902ffd640 1 --2- 172.21.15.3:0/3134210561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9041062d0 0x7fb9041034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb903fff640 1 -- 172.21.15.3:0/3134210561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9041062d0 msgr2=0x7fb9041034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb903fff640 1 --2- 172.21.15.3:0/3134210561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9041062d0 0x7fb9041034d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb903fff640 1 -- 172.21.15.3:0/3134210561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb904101ff0 msgr2=0x7fb904102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb903fff640 1 --2- 172.21.15.3:0/3134210561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb904101ff0 0x7fb904102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb903fff640 1 -- 172.21.15.3:0/3134210561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9041a8770 con 0x7fb904106c20 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb903fff640 1 --2- 172.21.15.3:0/3134210561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb904106c20 0x7fb904103a10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005cc0 tx=0x7fb8fc031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb900ff9640 1 -- 172.21.15.3:0/3134210561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc003b70 con 0x7fb904106c20 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb909cd6640 1 -- 172.21.15.3:0/3134210561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9041a8a00 con 0x7fb904106c20 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb900ff9640 1 -- 172.21.15.3:0/3134210561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8fc0318f0 con 0x7fb904106c20 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb900ff9640 1 -- 172.21.15.3:0/3134210561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc02f950 con 0x7fb904106c20 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.560+0000 7fb909cd6640 1 -- 172.21.15.3:0/3134210561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9041a8f90 con 0x7fb904106c20 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.561+0000 7fb8ea7fc640 1 -- 172.21.15.3:0/3134210561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e4005290 con 0x7fb904106c20 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.563+0000 7fb900ff9640 1 -- 172.21.15.3:0/3134210561 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fb8fc004b40 con 0x7fb904106c20 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.564+0000 7fb900ff9640 1 --2- 172.21.15.3:0/3134210561 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb8ec0b6680 0x7fb8ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.564+0000 7fb9037fe640 1 --2- 172.21.15.3:0/3134210561 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb8ec0b6680 0x7fb8ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.564+0000 7fb900ff9640 1 -- 172.21.15.3:0/3134210561 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fb8fc1089b0 con 0x7fb904106c20 2024-09-21T06:20:18.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.565+0000 7fb9037fe640 1 --2- 172.21.15.3:0/3134210561 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb8ec0b6680 0x7fb8ec0b8b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb9041049e0 tx=0x7fb8f4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:18.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.567+0000 7fb900ff9640 1 -- 172.21.15.3:0/3134210561 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8fc0ceee0 con 0x7fb904106c20 2024-09-21T06:20:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.808+0000 7fb8ea7fc640 1 -- 172.21.15.3:0/3134210561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8e4005580 con 0x7fb904106c20 2024-09-21T06:20:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.809+0000 7fb900ff9640 1 -- 172.21.15.3:0/3134210561 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11963 (secure 0 0 0) 0x7fb8fc0d32c0 con 0x7fb904106c20 2024-09-21T06:20:18.811 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:20:18.811 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":32,"btime":"2024-09-21T06:19:41:100740+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:19:40.732187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24670,24674,24679,24691,24704]},"id":1}]} 2024-09-21T06:20:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 32 2024-09-21T06:20:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.811+0000 7fb8ea7fc640 1 -- 172.21.15.3:0/3134210561 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb8ec0b6680 msgr2=0x7fb8ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.811+0000 7fb8ea7fc640 1 --2- 172.21.15.3:0/3134210561 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb8ec0b6680 0x7fb8ec0b8b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb9041049e0 tx=0x7fb8f4005f90 comp rx=0 tx=0).stop 2024-09-21T06:20:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.811+0000 7fb8ea7fc640 1 -- 172.21.15.3:0/3134210561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb904106c20 msgr2=0x7fb904103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.811+0000 7fb8ea7fc640 1 --2- 172.21.15.3:0/3134210561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb904106c20 0x7fb904103a10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005cc0 tx=0x7fb8fc031700 comp rx=0 tx=0).stop 2024-09-21T06:20:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.812+0000 7fb8ea7fc640 1 -- 172.21.15.3:0/3134210561 shutdown_connections 2024-09-21T06:20:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.812+0000 7fb8ea7fc640 1 --2- 172.21.15.3:0/3134210561 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb8ec0b6680 0x7fb8ec0b8b90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.812+0000 7fb8ea7fc640 1 --2- 172.21.15.3:0/3134210561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9041062d0 0x7fb9041034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.812+0000 7fb8ea7fc640 1 --2- 172.21.15.3:0/3134210561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb904106c20 0x7fb904103a10 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.812+0000 7fb8ea7fc640 1 --2- 172.21.15.3:0/3134210561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb904101ff0 0x7fb904102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.812+0000 7fb8ea7fc640 1 -- 172.21.15.3:0/3134210561 >> 172.21.15.3:0/3134210561 conn(0x7fb904079e30 msgr2=0x7fb904107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.812+0000 7fb8ea7fc640 1 -- 172.21.15.3:0/3134210561 shutdown_connections 2024-09-21T06:20:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:18.812+0000 7fb8ea7fc640 1 -- 172.21.15.3:0/3134210561 wait complete. 2024-09-21T06:20:18.956 INFO:tasks.workunit.client.0.smithi003.stdout: 1 551800 12.61 MB/sec execute 111 sec latency 68.158 ms 2024-09-21T06:20:19.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:19 smithi003 ceph-mon[31227]: pgmap v484: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 380 op/s 2024-09-21T06:20:19.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:19 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3134210561' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:20:19.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:19 smithi137 ceph-mon[34464]: pgmap v484: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 380 op/s 2024-09-21T06:20:19.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:19 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3134210561' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:20:19.649 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:20:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:19 smithi204 ceph-mon[34659]: pgmap v484: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 380 op/s 2024-09-21T06:20:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:19 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3134210561' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:20:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:19.956 INFO:tasks.workunit.client.0.smithi003.stdout: 1 554499 12.65 MB/sec execute 112 sec latency 14.990 ms 2024-09-21T06:20:20.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.654+0000 7f2e8e78d640 1 -- 172.21.15.3:0/4204401945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e88108d20 msgr2=0x7f2e8810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:20.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.654+0000 7f2e8e78d640 1 --2- 172.21.15.3:0/4204401945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e88108d20 0x7f2e8810f550 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009ca0 tx=0x7f2e8002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.654+0000 7f2e8e78d640 1 -- 172.21.15.3:0/4204401945 shutdown_connections 2024-09-21T06:20:20.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.654+0000 7f2e8e78d640 1 --2- 172.21.15.3:0/4204401945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e88108d20 0x7f2e8810f550 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.654+0000 7f2e8e78d640 1 --2- 172.21.15.3:0/4204401945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e881083d0 0x7f2e881087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.654+0000 7f2e8e78d640 1 --2- 172.21.15.3:0/4204401945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e88102ae0 0x7f2e88102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.654+0000 7f2e8e78d640 1 -- 172.21.15.3:0/4204401945 >> 172.21.15.3:0/4204401945 conn(0x7f2e88079760 msgr2=0x7f2e88079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:20.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.654+0000 7f2e8e78d640 1 -- 172.21.15.3:0/4204401945 shutdown_connections 2024-09-21T06:20:20.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.654+0000 7f2e8e78d640 1 -- 172.21.15.3:0/4204401945 wait complete. 2024-09-21T06:20:20.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.655+0000 7f2e8e78d640 1 Processor -- start 2024-09-21T06:20:20.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.655+0000 7f2e8e78d640 1 -- start start 2024-09-21T06:20:20.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.655+0000 7f2e8e78d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e88102ae0 0x7f2e8819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.655+0000 7f2e8e78d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e881083d0 0x7f2e8819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.655+0000 7f2e8e78d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e88108d20 0x7f2e8819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.655+0000 7f2e8e78d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88112990 con 0x7f2e88108d20 2024-09-21T06:20:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.655+0000 7f2e8e78d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88112810 con 0x7f2e881083d0 2024-09-21T06:20:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.655+0000 7f2e8e78d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88112b10 con 0x7f2e88102ae0 2024-09-21T06:20:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e8cd2a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e88108d20 0x7f2e8819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e877fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e881083d0 0x7f2e8819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e88102ae0 0x7f2e8819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e877fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e881083d0 0x7f2e8819b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:42908/0 (socket says 172.21.15.3:42908) 2024-09-21T06:20:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e88102ae0 0x7f2e8819b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:42550/0 (socket says 172.21.15.3:42550) 2024-09-21T06:20:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e877fe640 1 -- 172.21.15.3:0/1749757199 learned_addr learned my addr 172.21.15.3:0/1749757199 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e8cd2a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e88108d20 0x7f2e8819bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54696/0 (socket says 172.21.15.3:54696) 2024-09-21T06:20:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e877fe640 1 -- 172.21.15.3:0/1749757199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e88102ae0 msgr2=0x7f2e8819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e877fe640 1 --2- 172.21.15.3:0/1749757199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e88102ae0 0x7f2e8819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e877fe640 1 -- 172.21.15.3:0/1749757199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e88108d20 msgr2=0x7f2e8819bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e877fe640 1 --2- 172.21.15.3:0/1749757199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e88108d20 0x7f2e8819bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e877fe640 1 -- 172.21.15.3:0/1749757199 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e881a6550 con 0x7f2e881083d0 2024-09-21T06:20:20.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e8cd2a640 1 --2- 172.21.15.3:0/1749757199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e88108d20 0x7f2e8819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:20.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e877fe640 1 --2- 172.21.15.3:0/1749757199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e881083d0 0x7f2e8819b800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00e9c0 tx=0x7f2e7c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:20.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e857fa640 1 -- 172.21.15.3:0/1749757199 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c00cd80 con 0x7f2e881083d0 2024-09-21T06:20:20.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e8e78d640 1 -- 172.21.15.3:0/1749757199 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e881a6840 con 0x7f2e881083d0 2024-09-21T06:20:20.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.656+0000 7f2e8e78d640 1 -- 172.21.15.3:0/1749757199 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e881a6dd0 con 0x7f2e881083d0 2024-09-21T06:20:20.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.657+0000 7f2e857fa640 1 -- 172.21.15.3:0/1749757199 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e7c0054d0 con 0x7f2e881083d0 2024-09-21T06:20:20.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.657+0000 7f2e857fa640 1 -- 172.21.15.3:0/1749757199 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c004e60 con 0x7f2e881083d0 2024-09-21T06:20:20.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.659+0000 7f2e94d01740 1 Processor -- start 2024-09-21T06:20:20.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.659+0000 7f2e94d01740 1 -- start start 2024-09-21T06:20:20.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.659+0000 7f2e94d01740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99266070 0x556c9960d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.659+0000 7f2e94d01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c9923d1b0 0x556c9960d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.659+0000 7f2e94d01740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c99245390 0x556c9960dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.659+0000 7f2e94d01740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556c99583ea0 con 0x556c99245390 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.659+0000 7f2e94d01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556c995732d0 con 0x556c9923d1b0 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.659+0000 7f2e94d01740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556c995770e0 con 0x556c99266070 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.659+0000 7f2e857fa640 1 -- 172.21.15.3:0/1749757199 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f2e7c021440 con 0x7f2e881083d0 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.659+0000 7f2e8cd2a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99266070 0x556c9960d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.659+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c9923d1b0 0x556c9960d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.659+0000 7f2e8cd2a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99266070 0x556c9960d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:42562/0 (socket says 172.21.15.3:42562) 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.659+0000 7f2e8cd2a640 1 -- 172.21.15.3:0/2899219335 learned_addr learned my addr 172.21.15.3:0/2899219335 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.660+0000 7f2e877fe640 1 --2- 172.21.15.3:0/2899219335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c99245390 0x556c9960dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.660+0000 7f2e8cd2a640 1 -- 172.21.15.3:0/2899219335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c9923d1b0 msgr2=0x556c9960d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.660+0000 7f2e8cd2a640 1 --2- 172.21.15.3:0/2899219335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c9923d1b0 0x556c9960d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.660+0000 7f2e8cd2a640 1 -- 172.21.15.3:0/2899219335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c99245390 msgr2=0x556c9960dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.660+0000 7f2e8cd2a640 1 --2- 172.21.15.3:0/2899219335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c99245390 0x556c9960dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.660+0000 7f2e8cd2a640 1 -- 172.21.15.3:0/2899219335 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c992d1650 con 0x556c99266070 2024-09-21T06:20:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.660+0000 7f2e877fe640 1 --2- 172.21.15.3:0/2899219335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c99245390 0x556c9960dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.660+0000 7f2e8cd2a640 1 --2- 172.21.15.3:0/2899219335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99266070 0x556c9960d2c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2e80033040 tx=0x7f2e8002f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fcf158409e12336 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.660+0000 7f2e6e7fc640 1 -- 172.21.15.3:0/2899219335 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80031650 con 0x556c99266070 2024-09-21T06:20:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.660+0000 7f2e6e7fc640 1 -- 172.21.15.3:0/2899219335 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e80036040 con 0x556c99266070 2024-09-21T06:20:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.660+0000 7f2e6e7fc640 1 -- 172.21.15.3:0/2899219335 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e800044c0 con 0x556c99266070 2024-09-21T06:20:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.661+0000 7f2e857fa640 1 --2- 172.21.15.3:0/1749757199 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e700b66b0 0x7f2e700b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.661+0000 7f2e87fff640 1 --2- 172.21.15.3:0/1749757199 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e700b66b0 0x7f2e700b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.661+0000 7f2e857fa640 1 -- 172.21.15.3:0/1749757199 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2e7c0e7850 con 0x7f2e881083d0 2024-09-21T06:20:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.661+0000 7f2e94d01740 1 -- 172.21.15.3:0/2899219335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99266070 msgr2=0x556c9960d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.661+0000 7f2e94d01740 1 --2- 172.21.15.3:0/2899219335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99266070 0x556c9960d2c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2e80033040 tx=0x7f2e8002f310 comp rx=0 tx=0).stop 2024-09-21T06:20:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.661+0000 7f2e94d01740 1 -- 172.21.15.3:0/2899219335 shutdown_connections 2024-09-21T06:20:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.661+0000 7f2e94d01740 1 --2- 172.21.15.3:0/2899219335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c99245390 0x556c9960dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.661+0000 7f2e94d01740 1 --2- 172.21.15.3:0/2899219335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c9923d1b0 0x556c9960d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.661+0000 7f2e94d01740 1 --2- 172.21.15.3:0/2899219335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99266070 0x556c9960d2c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.661+0000 7f2e94d01740 1 -- 172.21.15.3:0/2899219335 >> 172.21.15.3:0/2899219335 conn(0x556c9925a460 msgr2=0x556c9960b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.661+0000 7f2e94d01740 1 -- 172.21.15.3:0/2899219335 shutdown_connections 2024-09-21T06:20:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.661+0000 7f2e94d01740 1 -- 172.21.15.3:0/2899219335 wait complete. 2024-09-21T06:20:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.661+0000 7f2e87fff640 1 --2- 172.21.15.3:0/1749757199 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e700b66b0 0x7f2e700b8bc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x556c9960bf40 tx=0x7f2e78005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.662+0000 7f2e94d01740 1 -- start start 2024-09-21T06:20:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.662+0000 7f2e94d01740 1 Processor -- start 2024-09-21T06:20:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.662+0000 7f2e94d01740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 0x556c99618920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.662+0000 7f2e94d01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99245390 0x556c99618e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.662+0000 7f2e94d01740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99266070 0x556c996193a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.662+0000 7f2e94d01740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556c993d4700 con 0x556c9923d1b0 2024-09-21T06:20:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.662+0000 7f2e94d01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556c9950e160 con 0x556c99245390 2024-09-21T06:20:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.662+0000 7f2e94d01740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556c9961b8c0 con 0x556c99266070 2024-09-21T06:20:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.662+0000 7f2e94d01740 10 client.0 _ll_register_callbacks cb 0x556c995279e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.662+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 0x556c99618920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.662+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 0x556c99618920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54712/0 (socket says 172.21.15.3:54712) 2024-09-21T06:20:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.662+0000 7f2e87fff640 1 -- 172.21.15.3:0/1437699738 learned_addr learned my addr 172.21.15.3:0/1437699738 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.662+0000 7f2e877fe640 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99245390 0x556c99618e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.663+0000 7f2e8cd2a640 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99266070 0x556c996193a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.663+0000 7f2e87fff640 1 -- 172.21.15.3:0/1437699738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99266070 msgr2=0x556c996193a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.663+0000 7f2e87fff640 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99266070 0x556c996193a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.663+0000 7f2e87fff640 1 -- 172.21.15.3:0/1437699738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99245390 msgr2=0x556c99618e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.663+0000 7f2e87fff640 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99245390 0x556c99618e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.663+0000 7f2e87fff640 1 -- 172.21.15.3:0/1437699738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c9953bc90 con 0x556c9923d1b0 2024-09-21T06:20:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.663+0000 7f2e877fe640 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99245390 0x556c99618e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.663+0000 7f2e87fff640 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 0x556c99618920 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2e780045f0 tx=0x7f2e78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2bb2c83520c5a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.663+0000 7f2e677fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.663+0000 7f2e677fe640 1 -- 172.21.15.3:0/1437699738 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7801f640 con 0x556c9923d1b0 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.663+0000 7f2e94d01740 1 -- 172.21.15.3:0/1437699738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e80036040 con 0x556c9923d1b0 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.663+0000 7f2e677fe640 1 -- 172.21.15.3:0/1437699738 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2e7801f7e0 con 0x556c9923d1b0 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.664+0000 7f2e677fe640 1 -- 172.21.15.3:0/1437699738 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2e7801f9e0 con 0x556c9923d1b0 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.664+0000 7f2e94d01740 10 client.17037 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.664+0000 7f2e94d01740 1 -- 172.21.15.3:0/1437699738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556c99627680 con 0x556c9923d1b0 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.664+0000 7f2e677fe640 1 -- 172.21.15.3:0/1437699738 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e80036040 con 0x556c9923d1b0 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.664+0000 7f2e677fe640 1 -- 172.21.15.3:0/1437699738 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f2e78027070 con 0x556c9923d1b0 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.664+0000 7f2e94d01740 10 client.17037 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.664+0000 7f2e94d01740 10 client.17037 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.664+0000 7f2e94d01740 20 client.17037 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.664+0000 7f2e94d01740 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556c992249f0 0x556c9962abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.664+0000 7f2e94d01740 4 client.17037 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.664+0000 7f2e94d01740 1 -- 172.21.15.3:0/1437699738 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c99585df0 con 0x556c992249f0 2024-09-21T06:20:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.664+0000 7f2e877fe640 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556c992249f0 0x556c9962abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.665+0000 7f2e677fe640 10 client.17037 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:20:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.665+0000 7f2e877fe640 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556c992249f0 0x556c9962abc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f1ea852b6ff38b5b server_cookie=849be44a1ae5774 in_seq=0 out_seq=0 2024-09-21T06:20:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.665+0000 7f2e677fe640 1 -- 172.21.15.3:0/1437699738 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c99585df0 con 0x556c992249f0 2024-09-21T06:20:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.665+0000 7f2e677fe640 10 client.17037 handle_command_reply: tid=0 2024-09-21T06:20:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.665+0000 7f2e94d01740 1 client.17037 shutdown 2024-09-21T06:20:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.666+0000 7f2e94d01740 1 -- 172.21.15.3:0/1437699738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 msgr2=0x556c99618920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.666+0000 7f2e94d01740 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 0x556c99618920 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2e780045f0 tx=0x7f2e78004730 comp rx=0 tx=0).stop 2024-09-21T06:20:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.666+0000 7f2e94d01740 1 -- 172.21.15.3:0/1437699738 shutdown_connections 2024-09-21T06:20:20.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.666+0000 7f2e94d01740 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556c992249f0 0x556c9962abc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.666+0000 7f2e94d01740 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99266070 0x556c996193a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.666+0000 7f2e94d01740 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99245390 0x556c99618e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.666+0000 7f2e677fe640 0 client.17037 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:20:20.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.666+0000 7f2e94d01740 1 --2- 172.21.15.3:0/1437699738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 0x556c99618920 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.666+0000 7f2e94d01740 1 -- 172.21.15.3:0/1437699738 >> 172.21.15.3:0/1437699738 conn(0x556c9925a460 msgr2=0x556c99610450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:20.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.666+0000 7f2e94d01740 1 -- 172.21.15.3:0/1437699738 shutdown_connections 2024-09-21T06:20:20.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.666+0000 7f2e94d01740 1 -- 172.21.15.3:0/1437699738 wait complete. 2024-09-21T06:20:20.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.666+0000 7f2e94d01740 20 client.17037 trim_cache size 0 max 16384 2024-09-21T06:20:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.679+0000 7f2e94d01740 1 Processor -- start 2024-09-21T06:20:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e94d01740 1 -- start start 2024-09-21T06:20:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e94d01740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c9923d1b0 0x556c9960ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e94d01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99245390 0x556c9961a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e94d01740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c99266070 0x556c9960a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e94d01740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7400e710 con 0x556c99266070 2024-09-21T06:20:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e94d01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e74010280 con 0x556c99245390 2024-09-21T06:20:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e94d01740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7400ede0 con 0x556c9923d1b0 2024-09-21T06:20:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e8cd2a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c99266070 0x556c9960a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e877fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99245390 0x556c9961a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e8cd2a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c99266070 0x556c9960a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54726/0 (socket says 172.21.15.3:54726) 2024-09-21T06:20:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e8cd2a640 1 -- 172.21.15.3:0/2041735025 learned_addr learned my addr 172.21.15.3:0/2041735025 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:20.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c9923d1b0 0x556c9960ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e877fe640 1 -- 172.21.15.3:0/2041735025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c9923d1b0 msgr2=0x556c9960ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:20.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e877fe640 1 --2- 172.21.15.3:0/2041735025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c9923d1b0 0x556c9960ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e877fe640 1 -- 172.21.15.3:0/2041735025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c99266070 msgr2=0x556c9960a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:20.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e877fe640 1 --2- 172.21.15.3:0/2041735025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c99266070 0x556c9960a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e877fe640 1 -- 172.21.15.3:0/2041735025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c99610120 con 0x556c99245390 2024-09-21T06:20:20.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.680+0000 7f2e8cd2a640 1 --2- 172.21.15.3:0/2041735025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c99266070 0x556c9960a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:20.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.681+0000 7f2e877fe640 1 --2- 172.21.15.3:0/2041735025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99245390 0x556c9961a550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0e4fe0 tx=0x7f2e7c0e9d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c209e5eb27f5d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:20.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.681+0000 7f2e6dffb640 1 -- 172.21.15.3:0/2041735025 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0a9d30 con 0x556c99245390 2024-09-21T06:20:20.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.681+0000 7f2e6dffb640 1 -- 172.21.15.3:0/2041735025 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e7c0ad1f0 con 0x556c99245390 2024-09-21T06:20:20.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.681+0000 7f2e6dffb640 1 -- 172.21.15.3:0/2041735025 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0ac940 con 0x556c99245390 2024-09-21T06:20:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.682+0000 7f2e94d01740 1 -- 172.21.15.3:0/2041735025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99245390 msgr2=0x556c9961a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.682+0000 7f2e94d01740 1 --2- 172.21.15.3:0/2041735025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99245390 0x556c9961a550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0e4fe0 tx=0x7f2e7c0e9d60 comp rx=0 tx=0).stop 2024-09-21T06:20:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.682+0000 7f2e94d01740 1 -- 172.21.15.3:0/2041735025 shutdown_connections 2024-09-21T06:20:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.682+0000 7f2e94d01740 1 --2- 172.21.15.3:0/2041735025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c99266070 0x556c9960a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.682+0000 7f2e94d01740 1 --2- 172.21.15.3:0/2041735025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99245390 0x556c9961a550 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.682+0000 7f2e94d01740 1 --2- 172.21.15.3:0/2041735025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c9923d1b0 0x556c9960ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.682+0000 7f2e94d01740 1 -- 172.21.15.3:0/2041735025 >> 172.21.15.3:0/2041735025 conn(0x556c9925a460 msgr2=0x556c99619e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.682+0000 7f2e94d01740 1 -- 172.21.15.3:0/2041735025 shutdown_connections 2024-09-21T06:20:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.682+0000 7f2e94d01740 1 -- 172.21.15.3:0/2041735025 wait complete. 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.682+0000 7f2e94d01740 1 -- start start 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.682+0000 7f2e94d01740 1 Processor -- start 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e94d01740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 0x556c99690e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e94d01740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99245390 0x556c99620ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e94d01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99266070 0x556c996211a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e94d01740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7400fbb0 con 0x556c9923d1b0 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e94d01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7400f4e0 con 0x556c99266070 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e94d01740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7400c6f0 con 0x556c99245390 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e94d01740 10 client.0 _ll_register_callbacks cb 0x7f2e78009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 0x556c99690e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99245390 0x556c99620ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e8cd2a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99266070 0x556c996211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 0x556c99690e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54732/0 (socket says 172.21.15.3:54732) 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99245390 0x556c99620ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:42588/0 (socket says 172.21.15.3:42588) 2024-09-21T06:20:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e87fff640 1 -- 172.21.15.3:0/1816676681 learned_addr learned my addr 172.21.15.3:0/1816676681 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e87fff640 1 -- 172.21.15.3:0/1816676681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99245390 msgr2=0x556c99620ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e87fff640 1 --2- 172.21.15.3:0/1816676681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99245390 0x556c99620ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e87fff640 1 -- 172.21.15.3:0/1816676681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99266070 msgr2=0x556c996211a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e87fff640 1 --2- 172.21.15.3:0/1816676681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99266070 0x556c996211a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e87fff640 1 -- 172.21.15.3:0/1816676681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c99621900 con 0x556c9923d1b0 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.683+0000 7f2e877fe640 1 --2- 172.21.15.3:0/1816676681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99245390 0x556c99620ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.684+0000 7f2e87fff640 1 --2- 172.21.15.3:0/1816676681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 0x556c99690e60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2e78000c00 tx=0x7f2e78029670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486ec2b59da879dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.684+0000 7f2e677fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.684+0000 7f2e677fe640 1 -- 172.21.15.3:0/1816676681 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78036030 con 0x556c9923d1b0 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.684+0000 7f2e94d01740 1 -- 172.21.15.3:0/1816676681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e7c0ad1f0 con 0x556c9923d1b0 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.684+0000 7f2e677fe640 1 -- 172.21.15.3:0/1816676681 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2e7801fd90 con 0x556c9923d1b0 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.684+0000 7f2e677fe640 1 -- 172.21.15.3:0/1816676681 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2e78042050 con 0x556c9923d1b0 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.684+0000 7f2e94d01740 10 client.17043 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.684+0000 7f2e94d01740 1 -- 172.21.15.3:0/1816676681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556c99699a20 con 0x556c9923d1b0 2024-09-21T06:20:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.684+0000 7f2e677fe640 1 -- 172.21.15.3:0/1816676681 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e7c0ad1f0 con 0x556c9923d1b0 2024-09-21T06:20:20.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.684+0000 7f2e677fe640 1 -- 172.21.15.3:0/1816676681 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f2e78027070 con 0x556c9923d1b0 2024-09-21T06:20:20.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.685+0000 7f2e94d01740 10 client.17043 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:20.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.685+0000 7f2e94d01740 10 client.17043 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:20:20.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.685+0000 7f2e94d01740 20 client.17043 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:20.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.685+0000 7f2e94d01740 1 --2- 172.21.15.3:0/1816676681 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556c991f9da0 0x556c9969c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:20.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.685+0000 7f2e94d01740 4 client.17043 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:20:20.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.685+0000 7f2e94d01740 1 -- 172.21.15.3:0/1816676681 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c995818a0 con 0x556c991f9da0 2024-09-21T06:20:20.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.685+0000 7f2e877fe640 1 --2- 172.21.15.3:0/1816676681 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556c991f9da0 0x556c9969c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:20.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.685+0000 7f2e677fe640 10 client.17043 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:20:20.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.685+0000 7f2e877fe640 1 --2- 172.21.15.3:0/1816676681 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556c991f9da0 0x556c9969c210 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6d796f27c8fd1074 server_cookie=bbeed7ca8b79cdff in_seq=0 out_seq=0 2024-09-21T06:20:20.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.685+0000 7f2e677fe640 1 -- 172.21.15.3:0/1816676681 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c995818a0 con 0x556c991f9da0 2024-09-21T06:20:20.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.685+0000 7f2e677fe640 10 client.17043 handle_command_reply: tid=0 2024-09-21T06:20:20.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.685+0000 7f2e94d01740 1 client.17043 shutdown 2024-09-21T06:20:20.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e94d01740 1 -- 172.21.15.3:0/1816676681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 msgr2=0x556c99690e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:20.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e94d01740 1 --2- 172.21.15.3:0/1816676681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 0x556c99690e60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2e78000c00 tx=0x7f2e78029670 comp rx=0 tx=0).stop 2024-09-21T06:20:20.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e94d01740 1 -- 172.21.15.3:0/1816676681 shutdown_connections 2024-09-21T06:20:20.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e94d01740 1 --2- 172.21.15.3:0/1816676681 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556c991f9da0 0x556c9969c210 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e94d01740 1 --2- 172.21.15.3:0/1816676681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c99245390 0x556c99620ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e94d01740 1 --2- 172.21.15.3:0/1816676681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c99266070 0x556c996211a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e94d01740 1 --2- 172.21.15.3:0/1816676681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c9923d1b0 0x556c99690e60 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e94d01740 1 -- 172.21.15.3:0/1816676681 >> 172.21.15.3:0/1816676681 conn(0x556c9925a460 msgr2=0x556c9960ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:20.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e677fe640 0 client.17043 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:20:20.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e94d01740 1 -- 172.21.15.3:0/1816676681 shutdown_connections 2024-09-21T06:20:20.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e94d01740 1 -- 172.21.15.3:0/1816676681 wait complete. 2024-09-21T06:20:20.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e94d01740 20 client.17043 trim_cache size 0 max 16384 2024-09-21T06:20:20.692 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:20:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e8e78d640 1 -- 172.21.15.3:0/1749757199 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e700b66b0 msgr2=0x7f2e700b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e8e78d640 1 --2- 172.21.15.3:0/1749757199 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e700b66b0 0x7f2e700b8bc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x556c9960bf40 tx=0x7f2e78005f90 comp rx=0 tx=0).stop 2024-09-21T06:20:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e8e78d640 1 -- 172.21.15.3:0/1749757199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e881083d0 msgr2=0x7f2e8819b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.686+0000 7f2e8e78d640 1 --2- 172.21.15.3:0/1749757199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e881083d0 0x7f2e8819b800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00e9c0 tx=0x7f2e7c00ee80 comp rx=0 tx=0).stop 2024-09-21T06:20:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.687+0000 7f2e8e78d640 1 -- 172.21.15.3:0/1749757199 shutdown_connections 2024-09-21T06:20:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.687+0000 7f2e8e78d640 1 --2- 172.21.15.3:0/1749757199 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e700b66b0 0x7f2e700b8bc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.687+0000 7f2e8e78d640 1 --2- 172.21.15.3:0/1749757199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e88108d20 0x7f2e8819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.687+0000 7f2e8e78d640 1 --2- 172.21.15.3:0/1749757199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e881083d0 0x7f2e8819b800 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.687+0000 7f2e8e78d640 1 --2- 172.21.15.3:0/1749757199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e88102ae0 0x7f2e8819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:20.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.687+0000 7f2e8e78d640 1 -- 172.21.15.3:0/1749757199 >> 172.21.15.3:0/1749757199 conn(0x7f2e88079760 msgr2=0x7f2e88104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:20.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.687+0000 7f2e8e78d640 1 -- 172.21.15.3:0/1749757199 shutdown_connections 2024-09-21T06:20:20.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:20.687+0000 7f2e8e78d640 1 -- 172.21.15.3:0/1749757199 wait complete. 2024-09-21T06:20:20.956 INFO:tasks.workunit.client.0.smithi003.stdout: 1 556507 12.60 MB/sec execute 113 sec latency 33.364 ms 2024-09-21T06:20:21.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:21 smithi003 ceph-mon[31227]: pgmap v485: 97 pgs: 97 active+clean; 112 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2024-09-21T06:20:21.643 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:20:21.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:21 smithi137 ceph-mon[34464]: pgmap v485: 97 pgs: 97 active+clean; 112 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2024-09-21T06:20:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:21 smithi204 ceph-mon[34659]: pgmap v485: 97 pgs: 97 active+clean; 112 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2024-09-21T06:20:21.915 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:21.956 INFO:tasks.workunit.client.0.smithi003.stdout: 1 558297 12.60 MB/sec execute 114 sec latency 57.595 ms 2024-09-21T06:20:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.428+0000 7f9451ffb640 1 -- 172.21.15.3:0/249515007 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c02f310 con 0x7f9454108cf0 2024-09-21T06:20:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.429+0000 7f9459631640 1 -- 172.21.15.3:0/249515007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9454108cf0 msgr2=0x7f945410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.429+0000 7f9459631640 1 --2- 172.21.15.3:0/249515007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9454108cf0 0x7f945410f520 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f944c009ca0 tx=0x7f944c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.429+0000 7f9459631640 1 -- 172.21.15.3:0/249515007 shutdown_connections 2024-09-21T06:20:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.429+0000 7f9459631640 1 --2- 172.21.15.3:0/249515007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9454108cf0 0x7f945410f520 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.429+0000 7f9459631640 1 --2- 172.21.15.3:0/249515007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94541083a0 0x7f94541087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.429+0000 7f9459631640 1 --2- 172.21.15.3:0/249515007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9454102a60 0x7f9454102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.429+0000 7f9459631640 1 -- 172.21.15.3:0/249515007 >> 172.21.15.3:0/249515007 conn(0x7f9454079730 msgr2=0x7f9454079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.429+0000 7f9459631640 1 -- 172.21.15.3:0/249515007 shutdown_connections 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.429+0000 7f9459631640 1 -- 172.21.15.3:0/249515007 wait complete. 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.429+0000 7f9459631640 1 Processor -- start 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.430+0000 7f9459631640 1 -- start start 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.430+0000 7f9459631640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9454102a60 0x7f9454199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.430+0000 7f9459631640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94541083a0 0x7f9454199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.430+0000 7f9459631640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9454108cf0 0x7f945419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.430+0000 7f9459631640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9454112960 con 0x7f9454108cf0 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.430+0000 7f9459631640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94541127e0 con 0x7f9454102a60 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.430+0000 7f9459631640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9454112ae0 con 0x7f94541083a0 2024-09-21T06:20:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.430+0000 7f94537fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9454108cf0 0x7f945419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.430+0000 7f9452ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9454102a60 0x7f9454199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.430+0000 7f94527fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94541083a0 0x7f9454199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.430+0000 7f94537fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9454108cf0 0x7f945419ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54750/0 (socket says 172.21.15.3:54750) 2024-09-21T06:20:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.430+0000 7f94537fe640 1 -- 172.21.15.3:0/1028009788 learned_addr learned my addr 172.21.15.3:0/1028009788 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.431+0000 7f9452ffd640 1 -- 172.21.15.3:0/1028009788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94541083a0 msgr2=0x7f9454199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.431+0000 7f9452ffd640 1 --2- 172.21.15.3:0/1028009788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94541083a0 0x7f9454199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.431+0000 7f9452ffd640 1 -- 172.21.15.3:0/1028009788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9454108cf0 msgr2=0x7f945419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.431+0000 7f9452ffd640 1 --2- 172.21.15.3:0/1028009788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9454108cf0 0x7f945419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.431+0000 7f9452ffd640 1 -- 172.21.15.3:0/1028009788 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945419f2e0 con 0x7f9454102a60 2024-09-21T06:20:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.431+0000 7f9452ffd640 1 --2- 172.21.15.3:0/1028009788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9454102a60 0x7f9454199370 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f944400d720 tx=0x7f944400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.431+0000 7f943bfff640 1 -- 172.21.15.3:0/1028009788 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444017070 con 0x7f9454102a60 2024-09-21T06:20:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.431+0000 7f9459631640 1 -- 172.21.15.3:0/1028009788 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945419f5d0 con 0x7f9454102a60 2024-09-21T06:20:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.431+0000 7f943bfff640 1 -- 172.21.15.3:0/1028009788 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f944400de30 con 0x7f9454102a60 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.431+0000 7f943bfff640 1 -- 172.21.15.3:0/1028009788 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94440047a0 con 0x7f9454102a60 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.431+0000 7f9459631640 1 -- 172.21.15.3:0/1028009788 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94541a5d30 con 0x7f9454102a60 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.434+0000 7f945fba5740 1 Processor -- start 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.434+0000 7f945fba5740 1 -- start start 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.434+0000 7f943bfff640 1 -- 172.21.15.3:0/1028009788 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f94440099d0 con 0x7f9454102a60 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.434+0000 7f945fba5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724f6070 0x55db7289d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.434+0000 7f945fba5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724cd1b0 0x55db7289dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.434+0000 7f945fba5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db7289e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f945fba5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55db72813ea0 con 0x55db724f6070 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f945fba5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55db728032d0 con 0x55db724cd1b0 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f945fba5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55db728070e0 con 0x55db724d5390 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f94537fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724f6070 0x55db7289d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f94527fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db7289e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f94527fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db7289e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:42628/0 (socket says 172.21.15.3:42628) 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f94527fc640 1 -- 172.21.15.3:0/1023276070 learned_addr learned my addr 172.21.15.3:0/1023276070 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f9452ffd640 1 --2- 172.21.15.3:0/1023276070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724cd1b0 0x55db7289dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f94527fc640 1 -- 172.21.15.3:0/1023276070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724cd1b0 msgr2=0x55db7289dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f94527fc640 1 --2- 172.21.15.3:0/1023276070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724cd1b0 0x55db7289dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f94527fc640 1 -- 172.21.15.3:0/1023276070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724f6070 msgr2=0x55db7289d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f94527fc640 1 --2- 172.21.15.3:0/1023276070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724f6070 0x55db7289d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f94527fc640 1 -- 172.21.15.3:0/1023276070 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db72561650 con 0x55db724d5390 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.435+0000 7f94537fe640 1 --2- 172.21.15.3:0/1023276070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724f6070 0x55db7289d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.436+0000 7f94527fc640 1 --2- 172.21.15.3:0/1023276070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db7289e120 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9448009be0 tx=0x7f94480119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e54249a030d9e58c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.436+0000 7f943bfff640 1 --2- 172.21.15.3:0/1028009788 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f943c0b6680 0x7f943c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.436+0000 7f94537fe640 1 --2- 172.21.15.3:0/1028009788 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f943c0b6680 0x7f943c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.436+0000 7f94397fa640 1 -- 172.21.15.3:0/1023276070 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448004050 con 0x55db724d5390 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.436+0000 7f943bfff640 1 -- 172.21.15.3:0/1028009788 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f94440e7290 con 0x7f9454102a60 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.436+0000 7f94397fa640 1 -- 172.21.15.3:0/1023276070 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94480041f0 con 0x55db724d5390 2024-09-21T06:20:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.436+0000 7f94397fa640 1 -- 172.21.15.3:0/1023276070 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944801a620 con 0x55db724d5390 2024-09-21T06:20:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.437+0000 7f945fba5740 1 -- 172.21.15.3:0/1023276070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 msgr2=0x55db7289e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.437+0000 7f945fba5740 1 --2- 172.21.15.3:0/1023276070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db7289e120 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9448009be0 tx=0x7f94480119a0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.437+0000 7f94537fe640 1 --2- 172.21.15.3:0/1028009788 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f943c0b6680 0x7f943c0b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f944c005e30 tx=0x7f944c005c50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.437+0000 7f945fba5740 1 -- 172.21.15.3:0/1023276070 shutdown_connections 2024-09-21T06:20:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.437+0000 7f945fba5740 1 --2- 172.21.15.3:0/1023276070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db7289e120 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.437+0000 7f945fba5740 1 --2- 172.21.15.3:0/1023276070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724cd1b0 0x55db7289dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.437+0000 7f945fba5740 1 --2- 172.21.15.3:0/1023276070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724f6070 0x55db7289d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.437+0000 7f945fba5740 1 -- 172.21.15.3:0/1023276070 >> 172.21.15.3:0/1023276070 conn(0x55db724ea460 msgr2=0x55db7289b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.437+0000 7f945fba5740 1 -- 172.21.15.3:0/1023276070 shutdown_connections 2024-09-21T06:20:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.438+0000 7f945fba5740 1 -- 172.21.15.3:0/1023276070 wait complete. 2024-09-21T06:20:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.438+0000 7f945fba5740 1 -- start start 2024-09-21T06:20:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.438+0000 7f945fba5740 1 Processor -- start 2024-09-21T06:20:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.438+0000 7f945fba5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724cd1b0 0x55db728a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.438+0000 7f945fba5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724d5390 0x55db728a9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.438+0000 7f945fba5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724f6070 0x55db728b94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.438+0000 7f945fba5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55db72664700 con 0x55db724f6070 2024-09-21T06:20:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.438+0000 7f945fba5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55db7279e160 con 0x55db724d5390 2024-09-21T06:20:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.438+0000 7f945fba5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55db728b0230 con 0x55db724cd1b0 2024-09-21T06:20:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.439+0000 7f94537fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724f6070 0x55db728b94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.439+0000 7f94527fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724d5390 0x55db728a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.439+0000 7f94537fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724f6070 0x55db728b94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54770/0 (socket says 172.21.15.3:54770) 2024-09-21T06:20:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.439+0000 7f94537fe640 1 -- 172.21.15.3:0/71265201 learned_addr learned my addr 172.21.15.3:0/71265201 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.439+0000 7f9452ffd640 1 --2- 172.21.15.3:0/71265201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724cd1b0 0x55db728a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.439+0000 7f945fba5740 10 client.0 _ll_register_callbacks cb 0x55db727b79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.439+0000 7f94527fc640 1 -- 172.21.15.3:0/71265201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724cd1b0 msgr2=0x55db728a8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:22.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.439+0000 7f94527fc640 1 --2- 172.21.15.3:0/71265201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724cd1b0 0x55db728a8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.439+0000 7f94527fc640 1 -- 172.21.15.3:0/71265201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724f6070 msgr2=0x55db728b94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:22.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.439+0000 7f94527fc640 1 --2- 172.21.15.3:0/71265201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724f6070 0x55db728b94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.440+0000 7f94527fc640 1 -- 172.21.15.3:0/71265201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db727cbc90 con 0x55db724d5390 2024-09-21T06:20:22.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.440+0000 7f94537fe640 1 --2- 172.21.15.3:0/71265201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724f6070 0x55db728b94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:22.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.440+0000 7f94527fc640 1 --2- 172.21.15.3:0/71265201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724d5390 0x55db728a9320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9448000c00 tx=0x7f9448003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54943f5c4f480398 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:22.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.440+0000 7f9436ff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:20:22.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.440+0000 7f9436ff5640 1 -- 172.21.15.3:0/71265201 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448004050 con 0x55db724d5390 2024-09-21T06:20:22.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.440+0000 7f945fba5740 1 -- 172.21.15.3:0/71265201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94480041f0 con 0x55db724d5390 2024-09-21T06:20:22.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.440+0000 7f9436ff5640 1 -- 172.21.15.3:0/71265201 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f9448020040 con 0x55db724d5390 2024-09-21T06:20:22.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.440+0000 7f9436ff5640 1 -- 172.21.15.3:0/71265201 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9448031050 con 0x55db724d5390 2024-09-21T06:20:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.440+0000 7f945fba5740 10 client.26630 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.440+0000 7f945fba5740 1 -- 172.21.15.3:0/71265201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55db728bbed0 con 0x55db724d5390 2024-09-21T06:20:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.441+0000 7f9436ff5640 1 -- 172.21.15.3:0/71265201 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94480041f0 con 0x55db724d5390 2024-09-21T06:20:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.441+0000 7f9436ff5640 1 -- 172.21.15.3:0/71265201 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f9448010920 con 0x55db724d5390 2024-09-21T06:20:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.441+0000 7f945fba5740 10 client.26630 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.441+0000 7f945fba5740 10 client.26630 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:20:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.441+0000 7f945fba5740 20 client.26630 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.441+0000 7f945fba5740 1 --2- 172.21.15.3:0/71265201 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55db724b49f0 0x55db728bf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.441+0000 7f945fba5740 4 client.26630 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.441+0000 7f945fba5740 1 -- 172.21.15.3:0/71265201 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db72815df0 con 0x55db724b49f0 2024-09-21T06:20:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.442+0000 7f9452ffd640 1 --2- 172.21.15.3:0/71265201 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55db724b49f0 0x55db728bf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.443+0000 7f9452ffd640 1 --2- 172.21.15.3:0/71265201 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55db724b49f0 0x55db728bf340 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4d8dc125f4d19d3 server_cookie=31c6dcb7266e2841 in_seq=0 out_seq=0 2024-09-21T06:20:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.443+0000 7f9436ff5640 10 client.26630 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:20:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f9436ff5640 1 -- 172.21.15.3:0/71265201 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55db72815df0 con 0x55db724b49f0 2024-09-21T06:20:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f9436ff5640 10 client.26630 handle_command_reply: tid=0 2024-09-21T06:20:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f945fba5740 1 client.26630 shutdown 2024-09-21T06:20:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f945fba5740 1 -- 172.21.15.3:0/71265201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724d5390 msgr2=0x55db728a9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f945fba5740 1 --2- 172.21.15.3:0/71265201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724d5390 0x55db728a9320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9448000c00 tx=0x7f9448003cd0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f945fba5740 1 -- 172.21.15.3:0/71265201 shutdown_connections 2024-09-21T06:20:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f945fba5740 1 --2- 172.21.15.3:0/71265201 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55db724b49f0 0x55db728bf340 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f945fba5740 1 --2- 172.21.15.3:0/71265201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724f6070 0x55db728b94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f945fba5740 1 --2- 172.21.15.3:0/71265201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724d5390 0x55db728a9320 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f945fba5740 1 --2- 172.21.15.3:0/71265201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724cd1b0 0x55db728a8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f945fba5740 1 -- 172.21.15.3:0/71265201 >> 172.21.15.3:0/71265201 conn(0x55db724ea460 msgr2=0x55db7289e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f9436ff5640 0 client.26630 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:20:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f945fba5740 1 -- 172.21.15.3:0/71265201 shutdown_connections 2024-09-21T06:20:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.444+0000 7f945fba5740 1 -- 172.21.15.3:0/71265201 wait complete. 2024-09-21T06:20:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.445+0000 7f945fba5740 20 client.26630 trim_cache size 0 max 16384 2024-09-21T06:20:22.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.458+0000 7f945fba5740 1 Processor -- start 2024-09-21T06:20:22.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.458+0000 7f945fba5740 1 -- start start 2024-09-21T06:20:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.458+0000 7f945fba5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 0x55db7289b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.458+0000 7f945fba5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db728b9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.458+0000 7f945fba5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724f6070 0x55db728a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.458+0000 7f945fba5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f94541ad460 con 0x55db724cd1b0 2024-09-21T06:20:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.458+0000 7f945fba5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94541aefd0 con 0x55db724f6070 2024-09-21T06:20:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.458+0000 7f945fba5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f94541adb30 con 0x55db724d5390 2024-09-21T06:20:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.458+0000 7f9452ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 0x55db7289b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.458+0000 7f9452ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 0x55db7289b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54780/0 (socket says 172.21.15.3:54780) 2024-09-21T06:20:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.458+0000 7f9452ffd640 1 -- 172.21.15.3:0/778357393 learned_addr learned my addr 172.21.15.3:0/778357393 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.458+0000 7f94527fc640 1 --2- 172.21.15.3:0/778357393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db728b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.459+0000 7f94537fe640 1 --2- 172.21.15.3:0/778357393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724f6070 0x55db728a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.459+0000 7f9452ffd640 1 -- 172.21.15.3:0/778357393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 msgr2=0x55db728b9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.459+0000 7f9452ffd640 1 --2- 172.21.15.3:0/778357393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db728b9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.459+0000 7f9452ffd640 1 -- 172.21.15.3:0/778357393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724f6070 msgr2=0x55db728a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.459+0000 7f9452ffd640 1 --2- 172.21.15.3:0/778357393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724f6070 0x55db728a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.459+0000 7f9452ffd640 1 -- 172.21.15.3:0/778357393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db7289f210 con 0x55db724cd1b0 2024-09-21T06:20:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.459+0000 7f9452ffd640 1 --2- 172.21.15.3:0/778357393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 0x55db7289b600 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9444009610 tx=0x7f9444094040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2452620a27fc1d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.459+0000 7f9438ff9640 1 -- 172.21.15.3:0/778357393 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94440e6040 con 0x55db724cd1b0 2024-09-21T06:20:22.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.459+0000 7f9438ff9640 1 -- 172.21.15.3:0/778357393 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94440eac30 con 0x55db724cd1b0 2024-09-21T06:20:22.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.459+0000 7f9438ff9640 1 -- 172.21.15.3:0/778357393 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94440ad820 con 0x55db724cd1b0 2024-09-21T06:20:22.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.460+0000 7f945fba5740 1 -- 172.21.15.3:0/778357393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 msgr2=0x55db7289b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.460+0000 7f945fba5740 1 --2- 172.21.15.3:0/778357393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 0x55db7289b600 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9444009610 tx=0x7f9444094040 comp rx=0 tx=0).stop 2024-09-21T06:20:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.460+0000 7f945fba5740 1 -- 172.21.15.3:0/778357393 shutdown_connections 2024-09-21T06:20:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.460+0000 7f945fba5740 1 --2- 172.21.15.3:0/778357393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db728b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.460+0000 7f945fba5740 1 --2- 172.21.15.3:0/778357393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724f6070 0x55db728a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.460+0000 7f945fba5740 1 --2- 172.21.15.3:0/778357393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 0x55db7289b600 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.460+0000 7f945fba5740 1 -- 172.21.15.3:0/778357393 >> 172.21.15.3:0/778357393 conn(0x55db724ea460 msgr2=0x55db7289aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.460+0000 7f945fba5740 1 -- 172.21.15.3:0/778357393 shutdown_connections 2024-09-21T06:20:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.460+0000 7f945fba5740 1 -- 172.21.15.3:0/778357393 wait complete. 2024-09-21T06:20:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f945fba5740 1 -- start start 2024-09-21T06:20:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f945fba5740 1 Processor -- start 2024-09-21T06:20:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f945fba5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 0x55db728b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f945fba5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db728b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f945fba5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724f6070 0x55db728b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f945fba5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f94541ae900 con 0x55db724cd1b0 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f945fba5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94541ae230 con 0x55db724f6070 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f945fba5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f94541ab440 con 0x55db724d5390 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f945fba5740 10 client.0 _ll_register_callbacks cb 0x7f944c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f9452ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 0x55db728b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f9452ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 0x55db728b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54786/0 (socket says 172.21.15.3:54786) 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f9452ffd640 1 -- 172.21.15.3:0/974603538 learned_addr learned my addr 172.21.15.3:0/974603538 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f94527fc640 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db728b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.461+0000 7f94537fe640 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724f6070 0x55db728b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.462+0000 7f9452ffd640 1 -- 172.21.15.3:0/974603538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 msgr2=0x55db728b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.462+0000 7f9452ffd640 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db728b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.462+0000 7f9452ffd640 1 -- 172.21.15.3:0/974603538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724f6070 msgr2=0x55db728b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.462+0000 7f9452ffd640 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724f6070 0x55db728b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.462+0000 7f9452ffd640 1 -- 172.21.15.3:0/974603538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db728b5990 con 0x55db724cd1b0 2024-09-21T06:20:22.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.462+0000 7f94537fe640 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724f6070 0x55db728b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.462+0000 7f9452ffd640 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 0x55db728b0e50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f94440ace20 tx=0x7f94440a9080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500fedf94b62cf58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.462+0000 7f9436ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.462+0000 7f9436ff5640 1 -- 172.21.15.3:0/974603538 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94440c3020 con 0x55db724cd1b0 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.462+0000 7f945fba5740 1 -- 172.21.15.3:0/974603538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94440eac30 con 0x55db724cd1b0 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.462+0000 7f9436ff5640 1 -- 172.21.15.3:0/974603538 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f94440e4ca0 con 0x55db724cd1b0 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.463+0000 7f9436ff5640 1 -- 172.21.15.3:0/974603538 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f94440e6050 con 0x55db724cd1b0 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.463+0000 7f945fba5740 10 client.17061 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.463+0000 7f945fba5740 1 -- 172.21.15.3:0/974603538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55db728abcc0 con 0x55db724cd1b0 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.463+0000 7f9436ff5640 1 -- 172.21.15.3:0/974603538 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94440eac30 con 0x55db724cd1b0 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.463+0000 7f9436ff5640 1 -- 172.21.15.3:0/974603538 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f94440a9390 con 0x55db724cd1b0 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.463+0000 7f945fba5740 10 client.17061 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.463+0000 7f945fba5740 10 client.17061 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.463+0000 7f945fba5740 20 client.17061 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:22.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.463+0000 7f945fba5740 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55db72489da0 0x55db728ae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.463+0000 7f945fba5740 4 client.17061 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.463+0000 7f945fba5740 1 -- 172.21.15.3:0/974603538 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db728118a0 con 0x55db72489da0 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.464+0000 7f94527fc640 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55db72489da0 0x55db728ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.464+0000 7f94527fc640 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55db72489da0 0x55db728ae450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3996ba78c6fe9059 server_cookie=fb014b01b7c5882f in_seq=0 out_seq=0 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.464+0000 7f9436ff5640 10 client.17061 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.464+0000 7f9436ff5640 1 -- 172.21.15.3:0/974603538 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db728118a0 con 0x55db72489da0 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.464+0000 7f9436ff5640 10 client.17061 handle_command_reply: tid=0 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.464+0000 7f945fba5740 1 client.17061 shutdown 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.464+0000 7f945fba5740 1 -- 172.21.15.3:0/974603538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 msgr2=0x55db728b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.464+0000 7f945fba5740 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 0x55db728b0e50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f94440ace20 tx=0x7f94440a9080 comp rx=0 tx=0).stop 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.464+0000 7f945fba5740 1 -- 172.21.15.3:0/974603538 shutdown_connections 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.464+0000 7f945fba5740 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55db72489da0 0x55db728ae450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.465+0000 7f945fba5740 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db724d5390 0x55db728b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.465+0000 7f9436ff5640 0 client.17061 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.465+0000 7f945fba5740 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db724f6070 0x55db728b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.465+0000 7f945fba5740 1 --2- 172.21.15.3:0/974603538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55db724cd1b0 0x55db728b0e50 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.465+0000 7f945fba5740 1 -- 172.21.15.3:0/974603538 >> 172.21.15.3:0/974603538 conn(0x55db724ea460 msgr2=0x55db728aa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:22.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.465+0000 7f945fba5740 1 -- 172.21.15.3:0/974603538 shutdown_connections 2024-09-21T06:20:22.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.465+0000 7f945fba5740 1 -- 172.21.15.3:0/974603538 wait complete. 2024-09-21T06:20:22.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.465+0000 7f945fba5740 20 client.17061 trim_cache size 0 max 16384 2024-09-21T06:20:22.471 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:20:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.465+0000 7f9459631640 1 -- 172.21.15.3:0/1028009788 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f943c0b6680 msgr2=0x7f943c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.465+0000 7f9459631640 1 --2- 172.21.15.3:0/1028009788 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f943c0b6680 0x7f943c0b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f944c005e30 tx=0x7f944c005c50 comp rx=0 tx=0).stop 2024-09-21T06:20:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.465+0000 7f9459631640 1 -- 172.21.15.3:0/1028009788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9454102a60 msgr2=0x7f9454199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.465+0000 7f9459631640 1 --2- 172.21.15.3:0/1028009788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9454102a60 0x7f9454199370 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f944400d720 tx=0x7f944400dbe0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.466+0000 7f9459631640 1 -- 172.21.15.3:0/1028009788 shutdown_connections 2024-09-21T06:20:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.466+0000 7f9459631640 1 --2- 172.21.15.3:0/1028009788 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f943c0b6680 0x7f943c0b8b90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.466+0000 7f9459631640 1 --2- 172.21.15.3:0/1028009788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94541083a0 0x7f9454199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.466+0000 7f9459631640 1 --2- 172.21.15.3:0/1028009788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9454108cf0 0x7f945419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.466+0000 7f9459631640 1 --2- 172.21.15.3:0/1028009788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9454102a60 0x7f9454199370 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:22.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.466+0000 7f9459631640 1 -- 172.21.15.3:0/1028009788 >> 172.21.15.3:0/1028009788 conn(0x7f9454079730 msgr2=0x7f9454104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:22.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.466+0000 7f9459631640 1 -- 172.21.15.3:0/1028009788 shutdown_connections 2024-09-21T06:20:22.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:22.466+0000 7f9459631640 1 -- 172.21.15.3:0/1028009788 wait complete. 2024-09-21T06:20:22.956 INFO:tasks.workunit.client.0.smithi003.stdout: 1 560725 12.60 MB/sec execute 115 sec latency 40.871 ms 2024-09-21T06:20:23.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:23 smithi003 ceph-mon[31227]: pgmap v486: 97 pgs: 97 active+clean; 112 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 374 op/s 2024-09-21T06:20:23.279 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:20:23.544 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:23.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:23 smithi137 ceph-mon[34464]: pgmap v486: 97 pgs: 97 active+clean; 112 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 374 op/s 2024-09-21T06:20:23.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:23 smithi204 ceph-mon[34659]: pgmap v486: 97 pgs: 97 active+clean; 112 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 374 op/s 2024-09-21T06:20:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.680+0000 7fbb15957640 1 -- 172.21.15.3:0/2482091718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb10109430 msgr2=0x7fbb10109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.680+0000 7fbb15957640 1 --2- 172.21.15.3:0/2482091718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb10109430 0x7fbb10109840 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009ca0 tx=0x7fbb0402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.680+0000 7fbb15957640 1 -- 172.21.15.3:0/2482091718 shutdown_connections 2024-09-21T06:20:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.680+0000 7fbb15957640 1 --2- 172.21.15.3:0/2482091718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb10109d80 0x7fbb101105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.680+0000 7fbb15957640 1 --2- 172.21.15.3:0/2482091718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb10109430 0x7fbb10109840 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.680+0000 7fbb15957640 1 --2- 172.21.15.3:0/2482091718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb10102060 0x7fbb10102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.680+0000 7fbb15957640 1 -- 172.21.15.3:0/2482091718 >> 172.21.15.3:0/2482091718 conn(0x7fbb10079e90 msgr2=0x7fbb1007a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.680+0000 7fbb15957640 1 -- 172.21.15.3:0/2482091718 shutdown_connections 2024-09-21T06:20:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.680+0000 7fbb15957640 1 -- 172.21.15.3:0/2482091718 wait complete. 2024-09-21T06:20:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.681+0000 7fbb15957640 1 Processor -- start 2024-09-21T06:20:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.681+0000 7fbb15957640 1 -- start start 2024-09-21T06:20:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.681+0000 7fbb15957640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb10102060 0x7fbb1019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.681+0000 7fbb15957640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb10109430 0x7fbb1019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.681+0000 7fbb15957640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb10109d80 0x7fbb1019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.681+0000 7fbb15957640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbb10113a90 con 0x7fbb10109430 2024-09-21T06:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.681+0000 7fbb15957640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb10113910 con 0x7fbb10109d80 2024-09-21T06:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.681+0000 7fbb15957640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb10113c10 con 0x7fbb10102060 2024-09-21T06:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb0e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb10109430 0x7fbb1019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb0f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb10109d80 0x7fbb1019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb0e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb10109430 0x7fbb1019aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54794/0 (socket says 172.21.15.3:54794) 2024-09-21T06:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/2212332482 learned_addr learned my addr 172.21.15.3:0/2212332482 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb0effd640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb10102060 0x7fbb1019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb0f7fe640 1 -- 172.21.15.3:0/2212332482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb10102060 msgr2=0x7fbb1019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb0f7fe640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb10102060 0x7fbb1019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb0f7fe640 1 -- 172.21.15.3:0/2212332482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb10109430 msgr2=0x7fbb1019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb0f7fe640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb10109430 0x7fbb1019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb0f7fe640 1 -- 172.21.15.3:0/2212332482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb101a0420 con 0x7fbb10109d80 2024-09-21T06:20:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb10109430 0x7fbb1019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb0f7fe640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb10109d80 0x7fbb1019fd20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800e970 tx=0x7fbb0800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb14955640 1 -- 172.21.15.3:0/2212332482 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0800ccc0 con 0x7fbb10109d80 2024-09-21T06:20:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb15957640 1 -- 172.21.15.3:0/2212332482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb101a0710 con 0x7fbb10109d80 2024-09-21T06:20:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb14955640 1 -- 172.21.15.3:0/2212332482 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb08005020 con 0x7fbb10109d80 2024-09-21T06:20:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.682+0000 7fbb14955640 1 -- 172.21.15.3:0/2212332482 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08004a00 con 0x7fbb10109d80 2024-09-21T06:20:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.683+0000 7fbb15957640 1 -- 172.21.15.3:0/2212332482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb101a6ea0 con 0x7fbb10109d80 2024-09-21T06:20:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.685+0000 7fbb1c8c5740 1 Processor -- start 2024-09-21T06:20:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.685+0000 7fbb14955640 1 -- 172.21.15.3:0/2212332482 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fbb08004ba0 con 0x7fbb10109d80 2024-09-21T06:20:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.685+0000 7fbb1c8c5740 1 -- start start 2024-09-21T06:20:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb1c8c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b8a81b0 0x56415bc4d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb1c8c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 0x56415bc4def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb1c8c5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8874d0 0x56415bc4e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb1c8c5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56415bbc26f0 con 0x56415b8874d0 2024-09-21T06:20:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb1c8c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56415bbb1b20 con 0x56415b87f2f0 2024-09-21T06:20:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb1c8c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56415bbb5930 con 0x56415b8a81b0 2024-09-21T06:20:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb0e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b8a81b0 0x56415bc4d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb0e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b8a81b0 0x56415bc4d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:42664/0 (socket says 172.21.15.3:42664) 2024-09-21T06:20:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/2005052591 learned_addr learned my addr 172.21.15.3:0/2005052591 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb0effd640 1 --2- 172.21.15.3:0/2005052591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 0x56415bc4def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/2005052591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8874d0 0x56415bc4e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb14955640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbaf80b6680 0x7fbaf80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/2005052591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 msgr2=0x56415bc4def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/2005052591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 0x56415bc4def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/2005052591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8874d0 msgr2=0x56415bc4e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/2005052591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8874d0 0x56415bc4e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/2005052591 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56415bb77530 con 0x56415b8a81b0 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb0f7fe640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbaf80b6680 0x7fbaf80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.686+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/2005052591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b8a81b0 0x56415bc4d9b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402aeb0 tx=0x7fbb04009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94dcbc227c166db6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.687+0000 7fbb14955640 1 -- 172.21.15.3:0/2212332482 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fbb080e7650 con 0x7fbb10109d80 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.687+0000 7fbaf5ffb640 1 -- 172.21.15.3:0/2005052591 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb040316e0 con 0x56415b8a81b0 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.687+0000 7fbaf5ffb640 1 -- 172.21.15.3:0/2005052591 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb04036070 con 0x56415b8a81b0 2024-09-21T06:20:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.687+0000 7fbaf5ffb640 1 -- 172.21.15.3:0/2005052591 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04004270 con 0x56415b8a81b0 2024-09-21T06:20:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.688+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/2005052591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b8a81b0 msgr2=0x56415bc4d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.688+0000 7fbb0f7fe640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbaf80b6680 0x7fbaf80b8b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbb080e53e0 tx=0x7fbb0807a190 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.688+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/2005052591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b8a81b0 0x56415bc4d9b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402aeb0 tx=0x7fbb04009920 comp rx=0 tx=0).stop 2024-09-21T06:20:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.688+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/2005052591 shutdown_connections 2024-09-21T06:20:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.688+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/2005052591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8874d0 0x56415bc4e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.688+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/2005052591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 0x56415bc4def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.688+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/2005052591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b8a81b0 0x56415bc4d9b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.688+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/2005052591 >> 172.21.15.3:0/2005052591 conn(0x56415b89c300 msgr2=0x56415bc4b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.688+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/2005052591 shutdown_connections 2024-09-21T06:20:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.688+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/2005052591 wait complete. 2024-09-21T06:20:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.688+0000 7fbb1c8c5740 1 -- start start 2024-09-21T06:20:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.688+0000 7fbb1c8c5740 1 Processor -- start 2024-09-21T06:20:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb1c8c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b87f2f0 0x56415bc58fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb1c8c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b8874d0 0x56415bc59510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb1c8c5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8a81b0 0x56415bc59a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb1c8c5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56415bae9ab0 con 0x56415b8a81b0 2024-09-21T06:20:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb1c8c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56415bb57730 con 0x56415b8874d0 2024-09-21T06:20:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb1c8c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56415bc5bf50 con 0x56415b87f2f0 2024-09-21T06:20:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb1c8c5740 10 client.0 _ll_register_callbacks cb 0x56415bbab6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb0e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8a81b0 0x56415bc59a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb0e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8a81b0 0x56415bc59a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54812/0 (socket says 172.21.15.3:54812) 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/1136728681 learned_addr learned my addr 172.21.15.3:0/1136728681 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/1136728681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b87f2f0 0x56415bc58fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb0effd640 1 --2- 172.21.15.3:0/1136728681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b8874d0 0x56415bc59510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/1136728681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b87f2f0 msgr2=0x56415bc58fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/1136728681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b87f2f0 0x56415bc58fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/1136728681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b8874d0 msgr2=0x56415bc59510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/1136728681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b8874d0 0x56415bc59510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.689+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/1136728681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56415b913790 con 0x56415b8a81b0 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.690+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/1136728681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8a81b0 0x56415bc59a90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbb04033040 tx=0x7fbb04003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd815858f79ceae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.690+0000 7fbaeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.690+0000 7fbaeeffd640 1 -- 172.21.15.3:0/1136728681 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0402fa20 con 0x56415b8a81b0 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.690+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/1136728681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb04036070 con 0x56415b8a81b0 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.690+0000 7fbaeeffd640 1 -- 172.21.15.3:0/1136728681 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fbb0402fbc0 con 0x56415b8a81b0 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.690+0000 7fbaeeffd640 1 -- 172.21.15.3:0/1136728681 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb0402fdc0 con 0x56415b8a81b0 2024-09-21T06:20:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.690+0000 7fbb1c8c5740 10 client.17073 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.690+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/1136728681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56415b98c620 con 0x56415b8a81b0 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.690+0000 7fbaeeffd640 1 -- 172.21.15.3:0/1136728681 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb04036070 con 0x56415b8a81b0 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.691+0000 7fbaeeffd640 1 -- 172.21.15.3:0/1136728681 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fbb04003ef0 con 0x56415b8a81b0 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.691+0000 7fbb1c8c5740 10 client.17073 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.691+0000 7fbb1c8c5740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.691+0000 7fbb1c8c5740 20 client.17073 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.691+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/1136728681 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56415b866d80 0x56415bc6ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.691+0000 7fbb1c8c5740 4 client.17073 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.691+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/1136728681 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56415bbc4640 con 0x56415b866d80 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.691+0000 7fbb0effd640 1 --2- 172.21.15.3:0/1136728681 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56415b866d80 0x56415bc6ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.691+0000 7fbb0effd640 1 --2- 172.21.15.3:0/1136728681 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56415b866d80 0x56415bc6ade0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a814f85d512a1b7b server_cookie=11ef95bbc8e0d065 in_seq=0 out_seq=0 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.691+0000 7fbaeeffd640 10 client.17073 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.692+0000 7fbaeeffd640 1 -- 172.21.15.3:0/1136728681 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56415bbc4640 con 0x56415b866d80 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.692+0000 7fbaeeffd640 10 client.17073 handle_command_reply: tid=0 2024-09-21T06:20:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.692+0000 7fbb1c8c5740 1 client.17073 shutdown 2024-09-21T06:20:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.692+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/1136728681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8a81b0 msgr2=0x56415bc59a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.692+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/1136728681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8a81b0 0x56415bc59a90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbb04033040 tx=0x7fbb04003e10 comp rx=0 tx=0).stop 2024-09-21T06:20:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.692+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/1136728681 shutdown_connections 2024-09-21T06:20:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.692+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/1136728681 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56415b866d80 0x56415bc6ade0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.692+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/1136728681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8a81b0 0x56415bc59a90 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.692+0000 7fbaeeffd640 0 client.17073 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.692+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/1136728681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b8874d0 0x56415bc59510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.692+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/1136728681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b87f2f0 0x56415bc58fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.692+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/1136728681 >> 172.21.15.3:0/1136728681 conn(0x56415b89c300 msgr2=0x56415bc4e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.693+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/1136728681 shutdown_connections 2024-09-21T06:20:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.693+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/1136728681 wait complete. 2024-09-21T06:20:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.693+0000 7fbb1c8c5740 20 client.17073 trim_cache size 0 max 16384 2024-09-21T06:20:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.710+0000 7fbb1c8c5740 1 Processor -- start 2024-09-21T06:20:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb1c8c5740 1 -- start start 2024-09-21T06:20:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb1c8c5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b866d80 0x56415bc4f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb1c8c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 0x56415bc561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb1c8c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b8874d0 0x56415bc566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb1c8c5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbb101af5c0 con 0x56415b866d80 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb1c8c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb101b1130 con 0x56415b87f2f0 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb1c8c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb101afc90 con 0x56415b8874d0 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb0e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b866d80 0x56415bc4f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb0e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b866d80 0x56415bc4f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54818/0 (socket says 172.21.15.3:54818) 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/928724308 learned_addr learned my addr 172.21.15.3:0/928724308 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb0effd640 1 --2- 172.21.15.3:0/928724308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 0x56415bc561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/928724308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b8874d0 msgr2=0x56415bc566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/928724308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b8874d0 0x56415bc566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/928724308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 msgr2=0x56415bc561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/928724308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 0x56415bc561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/928724308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56415bc4fcb0 con 0x56415b866d80 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.711+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/928724308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b866d80 0x56415bc4f0f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbb04004310 tx=0x7fbb04032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1204d1a1aef23a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.712+0000 7fbaf57fa640 1 -- 172.21.15.3:0/928724308 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04004500 con 0x56415b866d80 2024-09-21T06:20:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.712+0000 7fbaf57fa640 1 -- 172.21.15.3:0/928724308 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb04052070 con 0x56415b866d80 2024-09-21T06:20:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.712+0000 7fbaf57fa640 1 -- 172.21.15.3:0/928724308 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0402fa40 con 0x56415b866d80 2024-09-21T06:20:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.712+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/928724308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b866d80 msgr2=0x56415bc4f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.712+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/928724308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b866d80 0x56415bc4f0f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbb04004310 tx=0x7fbb04032eb0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.713+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/928724308 shutdown_connections 2024-09-21T06:20:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.713+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/928724308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b8874d0 0x56415bc566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.713+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/928724308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 0x56415bc561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.713+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/928724308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b866d80 0x56415bc4f0f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.713+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/928724308 >> 172.21.15.3:0/928724308 conn(0x56415b89c300 msgr2=0x56415bc4e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.713+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/928724308 shutdown_connections 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.713+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/928724308 wait complete. 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.713+0000 7fbb1c8c5740 1 -- start start 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.713+0000 7fbb1c8c5740 1 Processor -- start 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.713+0000 7fbb1c8c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b866d80 0x56415bc618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb1c8c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 0x56415bc5ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb1c8c5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8874d0 0x56415bc5f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb1c8c5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbb101b0a60 con 0x56415b8874d0 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb1c8c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb101b0390 con 0x56415b87f2f0 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb1c8c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb101ad5a0 con 0x56415b866d80 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb1c8c5740 10 client.0 _ll_register_callbacks cb 0x7fbb0402f350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb0e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8874d0 0x56415bc5f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb0e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8874d0 0x56415bc5f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54828/0 (socket says 172.21.15.3:54828) 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb0effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 0x56415bc5ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb0e7fc640 1 -- 172.21.15.3:0/474795657 learned_addr learned my addr 172.21.15.3:0/474795657 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/474795657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b866d80 0x56415bc618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb0effd640 1 -- 172.21.15.3:0/474795657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b866d80 msgr2=0x56415bc618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb0effd640 1 --2- 172.21.15.3:0/474795657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b866d80 0x56415bc618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb0effd640 1 -- 172.21.15.3:0/474795657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8874d0 msgr2=0x56415bc5f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb0effd640 1 --2- 172.21.15.3:0/474795657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8874d0 0x56415bc5f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb0effd640 1 -- 172.21.15.3:0/474795657 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56415bc5f940 con 0x56415b87f2f0 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/474795657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8874d0 0x56415bc5f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb0effd640 1 --2- 172.21.15.3:0/474795657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 0x56415bc5ed90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x56415bb57880 tx=0x7fbb000119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ef83c61baa85e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbaeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbaeeffd640 1 -- 172.21.15.3:0/474795657 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00004020 con 0x56415b87f2f0 2024-09-21T06:20:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbaeeffd640 1 -- 172.21.15.3:0/474795657 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fbb000041c0 con 0x56415b87f2f0 2024-09-21T06:20:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.714+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/474795657 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb04052070 con 0x56415b87f2f0 2024-09-21T06:20:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.715+0000 7fbaeeffd640 1 -- 172.21.15.3:0/474795657 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb000043e0 con 0x56415b87f2f0 2024-09-21T06:20:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.715+0000 7fbb1c8c5740 10 client.26654 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.715+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/474795657 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56415bcd5460 con 0x56415b87f2f0 2024-09-21T06:20:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.715+0000 7fbaeeffd640 1 -- 172.21.15.3:0/474795657 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb04052070 con 0x56415b87f2f0 2024-09-21T06:20:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.715+0000 7fbaeeffd640 1 -- 172.21.15.3:0/474795657 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fbb00003a40 con 0x56415b87f2f0 2024-09-21T06:20:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.715+0000 7fbb1c8c5740 10 client.26654 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.715+0000 7fbb1c8c5740 10 client.26654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.715+0000 7fbb1c8c5740 20 client.26654 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.715+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/474795657 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56415b8a81b0 0x56415bcd7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.715+0000 7fbb1c8c5740 4 client.26654 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:20:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.715+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/474795657 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56415bbc00f0 con 0x56415b8a81b0 2024-09-21T06:20:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.715+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/474795657 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56415b8a81b0 0x56415bcd7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.716+0000 7fbaeeffd640 10 client.26654 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.716+0000 7fbb0e7fc640 1 --2- 172.21.15.3:0/474795657 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56415b8a81b0 0x56415bcd7c20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d29bfd6cb7377e server_cookie=b3fd7fd0bfb1ab3b in_seq=0 out_seq=0 2024-09-21T06:20:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.889+0000 7fbaeeffd640 1 -- 172.21.15.3:0/474795657 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56415bbc00f0 con 0x56415b8a81b0 2024-09-21T06:20:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.889+0000 7fbaeeffd640 10 client.26654 handle_command_reply: tid=0 2024-09-21T06:20:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.890+0000 7fbb1c8c5740 1 client.26654 shutdown 2024-09-21T06:20:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.890+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/474795657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 msgr2=0x56415bc5ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.890+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/474795657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 0x56415bc5ed90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x56415bb57880 tx=0x7fbb000119b0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.890+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/474795657 shutdown_connections 2024-09-21T06:20:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.890+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/474795657 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56415b8a81b0 0x56415bcd7c20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.890+0000 7fbaeeffd640 0 client.26654 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.890+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/474795657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56415b8874d0 0x56415bc5f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.890+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/474795657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56415b87f2f0 0x56415bc5ed90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.890+0000 7fbb1c8c5740 1 --2- 172.21.15.3:0/474795657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56415b866d80 0x56415bc618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.890+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/474795657 >> 172.21.15.3:0/474795657 conn(0x56415b89c300 msgr2=0x56415bc4b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.890+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/474795657 shutdown_connections 2024-09-21T06:20:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.890+0000 7fbb1c8c5740 1 -- 172.21.15.3:0/474795657 wait complete. 2024-09-21T06:20:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.890+0000 7fbb1c8c5740 20 client.26654 trim_cache size 0 max 16384 2024-09-21T06:20:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:20:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:20:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:20:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:20:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.891+0000 7fbb15957640 1 -- 172.21.15.3:0/2212332482 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbaf80b6680 msgr2=0x7fbaf80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.891+0000 7fbb15957640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbaf80b6680 0x7fbaf80b8b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbb080e53e0 tx=0x7fbb0807a190 comp rx=0 tx=0).stop 2024-09-21T06:20:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.891+0000 7fbb15957640 1 -- 172.21.15.3:0/2212332482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb10109d80 msgr2=0x7fbb1019fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.891+0000 7fbb15957640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb10109d80 0x7fbb1019fd20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800e970 tx=0x7fbb0800ee30 comp rx=0 tx=0).stop 2024-09-21T06:20:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.891+0000 7fbb15957640 1 -- 172.21.15.3:0/2212332482 shutdown_connections 2024-09-21T06:20:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.891+0000 7fbb15957640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fbaf80b6680 0x7fbaf80b8b90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.891+0000 7fbb15957640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb10109d80 0x7fbb1019fd20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.891+0000 7fbb15957640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb10109430 0x7fbb1019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.891+0000 7fbb15957640 1 --2- 172.21.15.3:0/2212332482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb10102060 0x7fbb1019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.891+0000 7fbb15957640 1 -- 172.21.15.3:0/2212332482 >> 172.21.15.3:0/2212332482 conn(0x7fbb10079e90 msgr2=0x7fbb10107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.891+0000 7fbb15957640 1 -- 172.21.15.3:0/2212332482 shutdown_connections 2024-09-21T06:20:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:23.891+0000 7fbb15957640 1 -- 172.21.15.3:0/2212332482 wait complete. 2024-09-21T06:20:23.956 INFO:tasks.workunit.client.0.smithi003.stdout: 1 561658 12.55 MB/sec execute 116 sec latency 502.487 ms 2024-09-21T06:20:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.188+0000 7f03dd4a5640 1 -- 172.21.15.3:0/3513910379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03d8108d10 msgr2=0x7f03d810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.188+0000 7f03dd4a5640 1 --2- 172.21.15.3:0/3513910379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03d8108d10 0x7f03d810f540 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009ca0 tx=0x7f03d002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.188+0000 7f03dd4a5640 1 -- 172.21.15.3:0/3513910379 shutdown_connections 2024-09-21T06:20:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.188+0000 7f03dd4a5640 1 --2- 172.21.15.3:0/3513910379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03d8108d10 0x7f03d810f540 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.188+0000 7f03dd4a5640 1 --2- 172.21.15.3:0/3513910379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03d81083c0 0x7f03d81087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.188+0000 7f03dd4a5640 1 --2- 172.21.15.3:0/3513910379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03d8102a80 0x7f03d8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.188+0000 7f03dd4a5640 1 -- 172.21.15.3:0/3513910379 >> 172.21.15.3:0/3513910379 conn(0x7f03d8079740 msgr2=0x7f03d8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.188+0000 7f03dd4a5640 1 -- 172.21.15.3:0/3513910379 shutdown_connections 2024-09-21T06:20:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.188+0000 7f03dd4a5640 1 -- 172.21.15.3:0/3513910379 wait complete. 2024-09-21T06:20:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.189+0000 7f03dd4a5640 1 Processor -- start 2024-09-21T06:20:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.189+0000 7f03dd4a5640 1 -- start start 2024-09-21T06:20:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.189+0000 7f03dd4a5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03d8102a80 0x7f03d819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.189+0000 7f03dd4a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03d81083c0 0x7f03d819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.189+0000 7f03dd4a5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03d8108d10 0x7f03d819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.189+0000 7f03dd4a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8112980 con 0x7f03d81083c0 2024-09-21T06:20:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.189+0000 7f03dd4a5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8112800 con 0x7f03d8108d10 2024-09-21T06:20:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.189+0000 7f03dd4a5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8112b00 con 0x7f03d8102a80 2024-09-21T06:20:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.189+0000 7f03ce7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03d81083c0 0x7f03d819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.189+0000 7f03ce7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03d81083c0 0x7f03d819b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54846/0 (socket says 172.21.15.3:54846) 2024-09-21T06:20:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.189+0000 7f03ce7fc640 1 -- 172.21.15.3:0/1668826249 learned_addr learned my addr 172.21.15.3:0/1668826249 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.189+0000 7f03d6ffd640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03d8102a80 0x7f03d819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.190+0000 7f03d77fe640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03d8108d10 0x7f03d819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.190+0000 7f03ce7fc640 1 -- 172.21.15.3:0/1668826249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03d8102a80 msgr2=0x7f03d819b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.190+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03d8102a80 0x7f03d819b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.190+0000 7f03ce7fc640 1 -- 172.21.15.3:0/1668826249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03d8108d10 msgr2=0x7f03d819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.190+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03d8108d10 0x7f03d819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.190+0000 7f03ce7fc640 1 -- 172.21.15.3:0/1668826249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d81a64a0 con 0x7f03d81083c0 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.190+0000 7f03d77fe640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03d8108d10 0x7f03d819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.190+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03d81083c0 0x7f03d819b750 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f03c000e9c0 tx=0x7f03c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.190+0000 7f03d4ff9640 1 -- 172.21.15.3:0/1668826249 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c000cd80 con 0x7f03d81083c0 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.190+0000 7f03dd4a5640 1 -- 172.21.15.3:0/1668826249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d81a6790 con 0x7f03d81083c0 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.190+0000 7f03d4ff9640 1 -- 172.21.15.3:0/1668826249 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c00054d0 con 0x7f03d81083c0 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.190+0000 7f03d4ff9640 1 -- 172.21.15.3:0/1668826249 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0010430 con 0x7f03d81083c0 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.190+0000 7f03dd4a5640 1 -- 172.21.15.3:0/1668826249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d81a6d20 con 0x7f03d81083c0 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.193+0000 7f03e3a19740 1 Processor -- start 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.193+0000 7f03e3a19740 1 -- start start 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.193+0000 7f03d4ff9640 1 -- 172.21.15.3:0/1668826249 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f03c0004f90 con 0x7f03d81083c0 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.193+0000 7f03e3a19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4fe070 0x55cd7a8a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.193+0000 7f03e3a19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03e3a19740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8a6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03e3a19740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cd7a81bea0 con 0x55cd7a4dd390 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03e3a19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cd7a80b2d0 con 0x55cd7a4fe070 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03e3a19740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cd7a80f0e0 con 0x55cd7a4d51b0 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03d77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4fe070 0x55cd7a8a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03d77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4fe070 0x55cd7a8a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:43072/0 (socket says 172.21.15.3:43072) 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03d77fe640 1 -- 172.21.15.3:0/880349059 learned_addr learned my addr 172.21.15.3:0/880349059 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03d6ffd640 1 --2- 172.21.15.3:0/880349059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/880349059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03d77fe640 1 -- 172.21.15.3:0/880349059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4d51b0 msgr2=0x55cd7a8a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03d77fe640 1 --2- 172.21.15.3:0/880349059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03d77fe640 1 -- 172.21.15.3:0/880349059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 msgr2=0x55cd7a8a6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03d77fe640 1 --2- 172.21.15.3:0/880349059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8a6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03d77fe640 1 -- 172.21.15.3:0/880349059 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd7a569650 con 0x55cd7a4fe070 2024-09-21T06:20:24.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/880349059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:24.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.194+0000 7f03d77fe640 1 --2- 172.21.15.3:0/880349059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4fe070 0x55cd7a8a56a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009470 tx=0x7f03d00094a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6408e95efa00c151 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:24.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.195+0000 7f03cd7fa640 1 -- 172.21.15.3:0/880349059 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0031cf0 con 0x55cd7a4fe070 2024-09-21T06:20:24.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.195+0000 7f03cd7fa640 1 -- 172.21.15.3:0/880349059 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d0036040 con 0x55cd7a4fe070 2024-09-21T06:20:24.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.195+0000 7f03cd7fa640 1 -- 172.21.15.3:0/880349059 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d00042a0 con 0x55cd7a4fe070 2024-09-21T06:20:24.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.195+0000 7f03d4ff9640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f03c40b66b0 0x7f03c40b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.195+0000 7f03d6ffd640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f03c40b66b0 0x7f03c40b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.195+0000 7f03d4ff9640 1 -- 172.21.15.3:0/1668826249 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f03c00e7b50 con 0x7f03d81083c0 2024-09-21T06:20:24.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.196+0000 7f03e3a19740 1 -- 172.21.15.3:0/880349059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4fe070 msgr2=0x55cd7a8a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:24.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.196+0000 7f03d6ffd640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f03c40b66b0 0x7f03c40b8bc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55cd7a8a4320 tx=0x7f03c800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:24.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.196+0000 7f03e3a19740 1 --2- 172.21.15.3:0/880349059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4fe070 0x55cd7a8a56a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009470 tx=0x7f03d00094a0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.196+0000 7f03e3a19740 1 -- 172.21.15.3:0/880349059 shutdown_connections 2024-09-21T06:20:24.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.196+0000 7f03e3a19740 1 --2- 172.21.15.3:0/880349059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.196+0000 7f03e3a19740 1 --2- 172.21.15.3:0/880349059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.196+0000 7f03e3a19740 1 --2- 172.21.15.3:0/880349059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4fe070 0x55cd7a8a56a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.196+0000 7f03e3a19740 1 -- 172.21.15.3:0/880349059 >> 172.21.15.3:0/880349059 conn(0x55cd7a4f2460 msgr2=0x55cd7a8a3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.196+0000 7f03e3a19740 1 -- 172.21.15.3:0/880349059 shutdown_connections 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.196+0000 7f03e3a19740 1 -- 172.21.15.3:0/880349059 wait complete. 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.196+0000 7f03e3a19740 1 -- start start 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.196+0000 7f03e3a19740 1 Processor -- start 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03e3a19740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03e3a19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03e3a19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 0x55cd7a8b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03d6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03e3a19740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cd7a66c700 con 0x55cd7a4d51b0 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03ce7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03e3a19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cd7a7a6160 con 0x55cd7a4dd390 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03d6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54858/0 (socket says 172.21.15.3:54858) 2024-09-21T06:20:24.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03d6ffd640 1 -- 172.21.15.3:0/1699335611 learned_addr learned my addr 172.21.15.3:0/1699335611 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03e3a19740 1 -- 172.21.15.3:0/1699335611 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cd7a8b8120 con 0x55cd7a4fe070 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03d77fe640 1 --2- 172.21.15.3:0/1699335611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 0x55cd7a8b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03e3a19740 10 client.0 _ll_register_callbacks cb 0x55cd7a7bf9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03ce7fc640 1 -- 172.21.15.3:0/1699335611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 msgr2=0x55cd7a8b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/1699335611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 0x55cd7a8b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03ce7fc640 1 -- 172.21.15.3:0/1699335611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4d51b0 msgr2=0x55cd7a8b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/1699335611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03ce7fc640 1 -- 172.21.15.3:0/1699335611 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd7a7d3c90 con 0x55cd7a4dd390 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/1699335611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b5660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f03c00e52e0 tx=0x7f03c00a9eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70e290ac118791c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.197+0000 7f03be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03be7fc640 1 -- 172.21.15.3:0/1699335611 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c00ad7b0 con 0x55cd7a4dd390 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03be7fc640 1 -- 172.21.15.3:0/1699335611 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f03c00ace50 con 0x55cd7a4dd390 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03e3a19740 1 -- 172.21.15.3:0/1699335611 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03d0036040 con 0x55cd7a4dd390 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03be7fc640 1 -- 172.21.15.3:0/1699335611 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f03c00ad150 con 0x55cd7a4dd390 2024-09-21T06:20:24.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03e3a19740 10 client.26666 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03e3a19740 1 -- 172.21.15.3:0/1699335611 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cd7a8c3e10 con 0x55cd7a4dd390 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03be7fc640 1 -- 172.21.15.3:0/1699335611 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d0036040 con 0x55cd7a4dd390 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03be7fc640 1 -- 172.21.15.3:0/1699335611 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f03c00ac6b0 con 0x55cd7a4dd390 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03e3a19740 10 client.26666 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03e3a19740 10 client.26666 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03e3a19740 20 client.26666 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03e3a19740 1 --2- 172.21.15.3:0/1699335611 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55cd7a4bc9f0 0x55cd7a8c7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03e3a19740 4 client.26666 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03e3a19740 1 -- 172.21.15.3:0/1699335611 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd7a81ddf0 con 0x55cd7a4bc9f0 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.198+0000 7f03d6ffd640 1 --2- 172.21.15.3:0/1699335611 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55cd7a4bc9f0 0x55cd7a8c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.199+0000 7f03d6ffd640 1 --2- 172.21.15.3:0/1699335611 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55cd7a4bc9f0 0x55cd7a8c7350 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c059f219a0ed4a3 server_cookie=35f828cc969d2b45 in_seq=0 out_seq=0 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.199+0000 7f03be7fc640 10 client.26666 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03be7fc640 1 -- 172.21.15.3:0/1699335611 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cd7a81ddf0 con 0x55cd7a4bc9f0 2024-09-21T06:20:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03be7fc640 10 client.26666 handle_command_reply: tid=0 2024-09-21T06:20:24.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03e3a19740 1 client.26666 shutdown 2024-09-21T06:20:24.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03e3a19740 1 -- 172.21.15.3:0/1699335611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4dd390 msgr2=0x55cd7a8b5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:24.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03e3a19740 1 --2- 172.21.15.3:0/1699335611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b5660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f03c00e52e0 tx=0x7f03c00a9eb0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03e3a19740 1 -- 172.21.15.3:0/1699335611 shutdown_connections 2024-09-21T06:20:24.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03e3a19740 1 --2- 172.21.15.3:0/1699335611 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55cd7a4bc9f0 0x55cd7a8c7350 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03e3a19740 1 --2- 172.21.15.3:0/1699335611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 0x55cd7a8b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03e3a19740 1 --2- 172.21.15.3:0/1699335611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b5660 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03e3a19740 1 --2- 172.21.15.3:0/1699335611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03e3a19740 1 -- 172.21.15.3:0/1699335611 >> 172.21.15.3:0/1699335611 conn(0x55cd7a4f2460 msgr2=0x55cd7a8a1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:24.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03be7fc640 0 client.26666 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:20:24.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03e3a19740 1 -- 172.21.15.3:0/1699335611 shutdown_connections 2024-09-21T06:20:24.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03e3a19740 1 -- 172.21.15.3:0/1699335611 wait complete. 2024-09-21T06:20:24.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.200+0000 7f03e3a19740 20 client.26666 trim_cache size 0 max 16384 2024-09-21T06:20:24.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.213+0000 7f03e3a19740 1 Processor -- start 2024-09-21T06:20:24.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.213+0000 7f03e3a19740 1 -- start start 2024-09-21T06:20:24.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03e3a19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03e3a19740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03e3a19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 0x55cd7a8a6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03e3a19740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81ad460 con 0x55cd7a4dd390 2024-09-21T06:20:24.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03e3a19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81aefd0 con 0x55cd7a4d51b0 2024-09-21T06:20:24.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03e3a19740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81adb30 con 0x55cd7a4fe070 2024-09-21T06:20:24.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03ce7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03ce7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54860/0 (socket says 172.21.15.3:54860) 2024-09-21T06:20:24.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03ce7fc640 1 -- 172.21.15.3:0/1241055271 learned_addr learned my addr 172.21.15.3:0/1241055271 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:24.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03d77fe640 1 --2- 172.21.15.3:0/1241055271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 0x55cd7a8a6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03d6ffd640 1 --2- 172.21.15.3:0/1241055271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03ce7fc640 1 -- 172.21.15.3:0/1241055271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 msgr2=0x55cd7a8a6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:24.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/1241055271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 0x55cd7a8a6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03ce7fc640 1 -- 172.21.15.3:0/1241055271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4d51b0 msgr2=0x55cd7a8b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:24.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/1241055271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03ce7fc640 1 -- 172.21.15.3:0/1241055271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd7a8a6490 con 0x55cd7a4dd390 2024-09-21T06:20:24.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.214+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/1241055271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b7000 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f03c00ad6e0 tx=0x7f03c00049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73043c87ea34d589 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:24.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.215+0000 7f03ccff9640 1 -- 172.21.15.3:0/1241055271 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c00ad330 con 0x55cd7a4dd390 2024-09-21T06:20:24.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.215+0000 7f03ccff9640 1 -- 172.21.15.3:0/1241055271 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c00046c0 con 0x55cd7a4dd390 2024-09-21T06:20:24.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.215+0000 7f03ccff9640 1 -- 172.21.15.3:0/1241055271 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c00b24c0 con 0x55cd7a4dd390 2024-09-21T06:20:24.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.216+0000 7f03e3a19740 1 -- 172.21.15.3:0/1241055271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 msgr2=0x55cd7a8b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:24.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.216+0000 7f03e3a19740 1 --2- 172.21.15.3:0/1241055271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b7000 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f03c00ad6e0 tx=0x7f03c00049b0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.216+0000 7f03e3a19740 1 -- 172.21.15.3:0/1241055271 shutdown_connections 2024-09-21T06:20:24.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.216+0000 7f03e3a19740 1 --2- 172.21.15.3:0/1241055271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 0x55cd7a8a6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.216+0000 7f03e3a19740 1 --2- 172.21.15.3:0/1241055271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b7000 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.216+0000 7f03e3a19740 1 --2- 172.21.15.3:0/1241055271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.216+0000 7f03e3a19740 1 -- 172.21.15.3:0/1241055271 >> 172.21.15.3:0/1241055271 conn(0x55cd7a4f2460 msgr2=0x55cd7a8a66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:24.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.216+0000 7f03e3a19740 1 -- 172.21.15.3:0/1241055271 shutdown_connections 2024-09-21T06:20:24.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.216+0000 7f03e3a19740 1 -- 172.21.15.3:0/1241055271 wait complete. 2024-09-21T06:20:24.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.216+0000 7f03e3a19740 1 -- start start 2024-09-21T06:20:24.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.216+0000 7f03e3a19740 1 Processor -- start 2024-09-21T06:20:24.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03e3a19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03e3a19740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03e3a19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 0x55cd7a8b9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03e3a19740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81ae900 con 0x55cd7a4dd390 2024-09-21T06:20:24.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03e3a19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81ae230 con 0x55cd7a4d51b0 2024-09-21T06:20:24.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03e3a19740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81ab460 con 0x55cd7a4fe070 2024-09-21T06:20:24.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03e3a19740 10 client.0 _ll_register_callbacks cb 0x7f03c000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:43098/0 (socket says 172.21.15.3:43098) 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03d6ffd640 1 -- 172.21.15.3:0/2470022711 learned_addr learned my addr 172.21.15.3:0/2470022711 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03d77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 0x55cd7a8b9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03d6ffd640 1 -- 172.21.15.3:0/2470022711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 msgr2=0x55cd7a8b9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03d6ffd640 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 0x55cd7a8b9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03d6ffd640 1 -- 172.21.15.3:0/2470022711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 msgr2=0x55cd7a8b9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03d6ffd640 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03d6ffd640 1 -- 172.21.15.3:0/2470022711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd7a930e40 con 0x55cd7a4d51b0 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03d77fe640 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 0x55cd7a8b9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.217+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.218+0000 7f03d6ffd640 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b94e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55cd7a8c7a80 tx=0x7f03c80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=785a4a41fe1b00c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.218+0000 7f03be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.218+0000 7f03be7fc640 1 -- 172.21.15.3:0/2470022711 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8030430 con 0x55cd7a4d51b0 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.218+0000 7f03be7fc640 1 -- 172.21.15.3:0/2470022711 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f03c80305d0 con 0x55cd7a4d51b0 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.218+0000 7f03e3a19740 1 -- 172.21.15.3:0/2470022711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cd7a8198a0 con 0x55cd7a4d51b0 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.219+0000 7f03be7fc640 1 -- 172.21.15.3:0/2470022711 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f03c80307d0 con 0x55cd7a4d51b0 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.219+0000 7f03e3a19740 10 client.26672 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.219+0000 7f03e3a19740 1 -- 172.21.15.3:0/2470022711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cd7a932150 con 0x55cd7a4d51b0 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.219+0000 7f03be7fc640 1 -- 172.21.15.3:0/2470022711 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cd7a8198a0 con 0x55cd7a4d51b0 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.219+0000 7f03be7fc640 1 -- 172.21.15.3:0/2470022711 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f03c802a900 con 0x55cd7a4d51b0 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.219+0000 7f03e3a19740 10 client.26672 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.219+0000 7f03e3a19740 10 client.26672 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.219+0000 7f03e3a19740 20 client.26672 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.219+0000 7f03e3a19740 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55cd7a491da0 0x55cd7a934970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.219+0000 7f03e3a19740 4 client.26672 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:20:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.219+0000 7f03e3a19740 1 -- 172.21.15.3:0/2470022711 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd7a816cd0 con 0x55cd7a491da0 2024-09-21T06:20:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.220+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55cd7a491da0 0x55cd7a934970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.220+0000 7f03ce7fc640 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55cd7a491da0 0x55cd7a934970 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=652e182b55b2ad75 server_cookie=1e72b876818e56a3 in_seq=0 out_seq=0 2024-09-21T06:20:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.220+0000 7f03be7fc640 10 client.26672 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:20:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.220+0000 7f03be7fc640 1 -- 172.21.15.3:0/2470022711 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd7a816cd0 con 0x55cd7a491da0 2024-09-21T06:20:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.220+0000 7f03be7fc640 10 client.26672 handle_command_reply: tid=0 2024-09-21T06:20:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.221+0000 7f03e3a19740 1 client.26672 shutdown 2024-09-21T06:20:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.221+0000 7f03e3a19740 1 -- 172.21.15.3:0/2470022711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4d51b0 msgr2=0x55cd7a8b94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.221+0000 7f03e3a19740 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b94e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55cd7a8c7a80 tx=0x7f03c80046c0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.221+0000 7f03e3a19740 1 -- 172.21.15.3:0/2470022711 shutdown_connections 2024-09-21T06:20:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.221+0000 7f03e3a19740 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55cd7a491da0 0x55cd7a934970 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.221+0000 7f03e3a19740 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd7a4fe070 0x55cd7a8b9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.221+0000 7f03e3a19740 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cd7a4dd390 0x55cd7a8b9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.221+0000 7f03e3a19740 1 --2- 172.21.15.3:0/2470022711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd7a4d51b0 0x55cd7a8b94e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.221+0000 7f03be7fc640 0 client.26672 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:20:24.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.221+0000 7f03e3a19740 1 -- 172.21.15.3:0/2470022711 >> 172.21.15.3:0/2470022711 conn(0x55cd7a4f2460 msgr2=0x55cd7a8a7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:24.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.221+0000 7f03e3a19740 1 -- 172.21.15.3:0/2470022711 shutdown_connections 2024-09-21T06:20:24.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.221+0000 7f03e3a19740 1 -- 172.21.15.3:0/2470022711 wait complete. 2024-09-21T06:20:24.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.222+0000 7f03e3a19740 20 client.26672 trim_cache size 0 max 16384 2024-09-21T06:20:24.226 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:20:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.222+0000 7f03dd4a5640 1 -- 172.21.15.3:0/1668826249 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f03c40b66b0 msgr2=0x7f03c40b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.222+0000 7f03dd4a5640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f03c40b66b0 0x7f03c40b8bc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55cd7a8a4320 tx=0x7f03c800b410 comp rx=0 tx=0).stop 2024-09-21T06:20:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.222+0000 7f03dd4a5640 1 -- 172.21.15.3:0/1668826249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03d81083c0 msgr2=0x7f03d819b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.222+0000 7f03dd4a5640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03d81083c0 0x7f03d819b750 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f03c000e9c0 tx=0x7f03c000ee80 comp rx=0 tx=0).stop 2024-09-21T06:20:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.222+0000 7f03dd4a5640 1 -- 172.21.15.3:0/1668826249 shutdown_connections 2024-09-21T06:20:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.222+0000 7f03dd4a5640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f03c40b66b0 0x7f03c40b8bc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.222+0000 7f03dd4a5640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03d8108d10 0x7f03d819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.222+0000 7f03dd4a5640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03d81083c0 0x7f03d819b750 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.222+0000 7f03dd4a5640 1 --2- 172.21.15.3:0/1668826249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03d8102a80 0x7f03d819b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.222+0000 7f03dd4a5640 1 -- 172.21.15.3:0/1668826249 >> 172.21.15.3:0/1668826249 conn(0x7f03d8079740 msgr2=0x7f03d8104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.222+0000 7f03dd4a5640 1 -- 172.21.15.3:0/1668826249 shutdown_connections 2024-09-21T06:20:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:24.223+0000 7f03dd4a5640 1 -- 172.21.15.3:0/1668826249 wait complete. 2024-09-21T06:20:24.930 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:20:24.956 INFO:tasks.workunit.client.0.smithi003.stdout: 1 561658 12.44 MB/sec execute 117 sec latency 1502.543 ms 2024-09-21T06:20:25.190 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:25.453 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:25 smithi003 ceph-mon[31227]: pgmap v487: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 369 op/s 2024-09-21T06:20:25.453 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:25 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:20:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:25 smithi137 ceph-mon[34464]: pgmap v487: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 369 op/s 2024-09-21T06:20:25.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:25 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:20:25.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:25 smithi204 ceph-mon[34659]: pgmap v487: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 369 op/s 2024-09-21T06:20:25.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:25 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:20:25.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.692+0000 7fcc317df640 1 -- 172.21.15.3:0/2139127168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc2c104880 msgr2=0x7fcc2c104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.692+0000 7fcc317df640 1 --2- 172.21.15.3:0/2139127168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc2c104880 0x7fcc2c104c90 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009ca0 tx=0x7fcc1c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.692+0000 7fcc317df640 1 -- 172.21.15.3:0/2139127168 shutdown_connections 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.692+0000 7fcc317df640 1 --2- 172.21.15.3:0/2139127168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc2c1052b0 0x7fcc2c10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.692+0000 7fcc317df640 1 --2- 172.21.15.3:0/2139127168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc2c10de30 0x7fcc2c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.692+0000 7fcc317df640 1 --2- 172.21.15.3:0/2139127168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc2c104880 0x7fcc2c104c90 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.692+0000 7fcc317df640 1 -- 172.21.15.3:0/2139127168 >> 172.21.15.3:0/2139127168 conn(0x7fcc2c079730 msgr2=0x7fcc2c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.692+0000 7fcc317df640 1 -- 172.21.15.3:0/2139127168 shutdown_connections 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.692+0000 7fcc317df640 1 -- 172.21.15.3:0/2139127168 wait complete. 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.693+0000 7fcc317df640 1 Processor -- start 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.693+0000 7fcc317df640 1 -- start start 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.693+0000 7fcc317df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc2c104880 0x7fcc2c102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.693+0000 7fcc317df640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc2c1052b0 0x7fcc2c102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.693+0000 7fcc317df640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc2c10de30 0x7fcc2c103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.693+0000 7fcc317df640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c113a70 con 0x7fcc2c10de30 2024-09-21T06:20:25.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.693+0000 7fcc317df640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1138f0 con 0x7fcc2c104880 2024-09-21T06:20:25.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.693+0000 7fcc317df640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c113bf0 con 0x7fcc2c1052b0 2024-09-21T06:20:25.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.693+0000 7fcc2b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc2c10de30 0x7fcc2c103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.693+0000 7fcc2b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc2c10de30 0x7fcc2c103210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47952/0 (socket says 172.21.15.3:47952) 2024-09-21T06:20:25.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.693+0000 7fcc2b7fe640 1 -- 172.21.15.3:0/3562050759 learned_addr learned my addr 172.21.15.3:0/3562050759 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:25.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.693+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc2c1052b0 0x7fcc2c102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.693+0000 7fcc2affd640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc2c104880 0x7fcc2c102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.694+0000 7fcc2b7fe640 1 -- 172.21.15.3:0/3562050759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc2c1052b0 msgr2=0x7fcc2c102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:25.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.694+0000 7fcc2b7fe640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc2c1052b0 0x7fcc2c102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.694+0000 7fcc2b7fe640 1 -- 172.21.15.3:0/3562050759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc2c104880 msgr2=0x7fcc2c102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:25.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.694+0000 7fcc2b7fe640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc2c104880 0x7fcc2c102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.694+0000 7fcc2b7fe640 1 -- 172.21.15.3:0/3562050759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2c1a87c0 con 0x7fcc2c10de30 2024-09-21T06:20:25.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.694+0000 7fcc2b7fe640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc2c10de30 0x7fcc2c103210 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400d950 tx=0x7fcc2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.694+0000 7fcc13fff640 1 -- 172.21.15.3:0/3562050759 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24017030 con 0x7fcc2c10de30 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.694+0000 7fcc317df640 1 -- 172.21.15.3:0/3562050759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2c1a8ab0 con 0x7fcc2c10de30 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.694+0000 7fcc13fff640 1 -- 172.21.15.3:0/3562050759 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc24004020 con 0x7fcc2c10de30 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.694+0000 7fcc13fff640 1 -- 172.21.15.3:0/3562050759 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24003b50 con 0x7fcc2c10de30 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.694+0000 7fcc317df640 1 -- 172.21.15.3:0/3562050759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2c1a8fa0 con 0x7fcc2c10de30 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.697+0000 7fcc37d53740 1 Processor -- start 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.697+0000 7fcc13fff640 1 -- 172.21.15.3:0/3562050759 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fcc2401a070 con 0x7fcc2c10de30 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.697+0000 7fcc37d53740 1 -- start start 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.697+0000 7fcc37d53740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca28a070 0x55dfca6316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.697+0000 7fcc37d53740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 0x55dfca631be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.697+0000 7fcc37d53740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 0x55dfca632120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.697+0000 7fcc37d53740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dfca5a7ea0 con 0x55dfca2611b0 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.697+0000 7fcc37d53740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dfca5972d0 con 0x55dfca28a070 2024-09-21T06:20:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.697+0000 7fcc37d53740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dfca59b0e0 con 0x55dfca269390 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.697+0000 7fcc2affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 0x55dfca631be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc2affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 0x55dfca631be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47966/0 (socket says 172.21.15.3:47966) 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc2affd640 1 -- 172.21.15.3:0/4023747589 learned_addr learned my addr 172.21.15.3:0/4023747589 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/4023747589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca28a070 0x55dfca6316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc13fff640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc140b6680 0x7fcc140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/4023747589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 0x55dfca632120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc2affd640 1 -- 172.21.15.3:0/4023747589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 msgr2=0x55dfca632120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc2affd640 1 --2- 172.21.15.3:0/4023747589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 0x55dfca632120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc2affd640 1 -- 172.21.15.3:0/4023747589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca28a070 msgr2=0x55dfca6316a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc2affd640 1 --2- 172.21.15.3:0/4023747589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca28a070 0x55dfca6316a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc2affd640 1 -- 172.21.15.3:0/4023747589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfca2f5650 con 0x55dfca2611b0 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc2b7fe640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc140b6680 0x7fcc140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc13fff640 1 -- 172.21.15.3:0/3562050759 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcc240e6f30 con 0x7fcc2c10de30 2024-09-21T06:20:25.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc2affd640 1 --2- 172.21.15.3:0/4023747589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 0x55dfca631be0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009ca0 tx=0x7fcc1c02f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f646671337bb10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc117fa640 1 -- 172.21.15.3:0/4023747589 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c031cf0 con 0x55dfca2611b0 2024-09-21T06:20:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc117fa640 1 -- 172.21.15.3:0/4023747589 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1c036040 con 0x55dfca2611b0 2024-09-21T06:20:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.698+0000 7fcc117fa640 1 -- 172.21.15.3:0/4023747589 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c004280 con 0x55dfca2611b0 2024-09-21T06:20:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.699+0000 7fcc37d53740 1 -- 172.21.15.3:0/4023747589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 msgr2=0x55dfca631be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.699+0000 7fcc2b7fe640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc140b6680 0x7fcc140b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcc240e4cc0 tx=0x7fcc24094040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.699+0000 7fcc37d53740 1 --2- 172.21.15.3:0/4023747589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 0x55dfca631be0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009ca0 tx=0x7fcc1c02f420 comp rx=0 tx=0).stop 2024-09-21T06:20:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.699+0000 7fcc37d53740 1 -- 172.21.15.3:0/4023747589 shutdown_connections 2024-09-21T06:20:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.699+0000 7fcc37d53740 1 --2- 172.21.15.3:0/4023747589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 0x55dfca632120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.699+0000 7fcc37d53740 1 --2- 172.21.15.3:0/4023747589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 0x55dfca631be0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.699+0000 7fcc37d53740 1 --2- 172.21.15.3:0/4023747589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca28a070 0x55dfca6316a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.699+0000 7fcc37d53740 1 -- 172.21.15.3:0/4023747589 >> 172.21.15.3:0/4023747589 conn(0x55dfca27e460 msgr2=0x55dfca62f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.699+0000 7fcc37d53740 1 -- 172.21.15.3:0/4023747589 shutdown_connections 2024-09-21T06:20:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.699+0000 7fcc37d53740 1 -- 172.21.15.3:0/4023747589 wait complete. 2024-09-21T06:20:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc37d53740 1 -- start start 2024-09-21T06:20:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc37d53740 1 Processor -- start 2024-09-21T06:20:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc37d53740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2611b0 0x55dfca63cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc37d53740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 0x55dfca63d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc37d53740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca28a070 0x55dfca649140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc37d53740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dfca3f8700 con 0x55dfca28a070 2024-09-21T06:20:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc37d53740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dfca532160 con 0x55dfca2611b0 2024-09-21T06:20:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc37d53740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dfca63fe20 con 0x55dfca269390 2024-09-21T06:20:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc37d53740 10 client.0 _ll_register_callbacks cb 0x55dfca54b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc2a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca28a070 0x55dfca649140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc2a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca28a070 0x55dfca649140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47978/0 (socket says 172.21.15.3:47978) 2024-09-21T06:20:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc2a7fc640 1 -- 172.21.15.3:0/228008204 learned_addr learned my addr 172.21.15.3:0/228008204 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc2affd640 1 --2- 172.21.15.3:0/228008204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 0x55dfca63d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.700+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/228008204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2611b0 0x55dfca63cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.701+0000 7fcc2a7fc640 1 -- 172.21.15.3:0/228008204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 msgr2=0x55dfca63d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:25.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.701+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/228008204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 0x55dfca63d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.701+0000 7fcc2a7fc640 1 -- 172.21.15.3:0/228008204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2611b0 msgr2=0x55dfca63cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:25.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.701+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/228008204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2611b0 0x55dfca63cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.701+0000 7fcc2a7fc640 1 -- 172.21.15.3:0/228008204 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfca55fc90 con 0x55dfca28a070 2024-09-21T06:20:25.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.701+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/228008204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca28a070 0x55dfca649140 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fcc20010d80 tx=0x7fcc20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f50f94bdcfb4bed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:25.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.701+0000 7fcc0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.701+0000 7fcc0eff5640 1 -- 172.21.15.3:0/228008204 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000fcd0 con 0x55dfca28a070 2024-09-21T06:20:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.701+0000 7fcc0eff5640 1 -- 172.21.15.3:0/228008204 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcc20009040 con 0x55dfca28a070 2024-09-21T06:20:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.701+0000 7fcc37d53740 1 -- 172.21.15.3:0/228008204 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc1c036040 con 0x55dfca28a070 2024-09-21T06:20:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.701+0000 7fcc0eff5640 1 -- 172.21.15.3:0/228008204 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcc2000fe70 con 0x55dfca28a070 2024-09-21T06:20:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.701+0000 7fcc37d53740 10 client.17109 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.701+0000 7fcc37d53740 1 -- 172.21.15.3:0/228008204 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dfca64bb80 con 0x55dfca28a070 2024-09-21T06:20:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.702+0000 7fcc0eff5640 1 -- 172.21.15.3:0/228008204 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1c036040 con 0x55dfca28a070 2024-09-21T06:20:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.702+0000 7fcc0eff5640 1 -- 172.21.15.3:0/228008204 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fcc20003c90 con 0x55dfca28a070 2024-09-21T06:20:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.702+0000 7fcc37d53740 10 client.17109 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.702+0000 7fcc37d53740 10 client.17109 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:20:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.702+0000 7fcc37d53740 20 client.17109 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.702+0000 7fcc37d53740 1 --2- 172.21.15.3:0/228008204 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55dfca2489f0 0x55dfca64f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.702+0000 7fcc37d53740 4 client.17109 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.702+0000 7fcc37d53740 1 -- 172.21.15.3:0/228008204 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfca5a9df0 con 0x55dfca2489f0 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.702+0000 7fcc2affd640 1 --2- 172.21.15.3:0/228008204 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55dfca2489f0 0x55dfca64f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.703+0000 7fcc0eff5640 10 client.17109 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.703+0000 7fcc2affd640 1 --2- 172.21.15.3:0/228008204 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55dfca2489f0 0x55dfca64f080 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f8b947e4b566c8c server_cookie=9934855aa96e8600 in_seq=0 out_seq=0 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.703+0000 7fcc0eff5640 1 -- 172.21.15.3:0/228008204 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dfca5a9df0 con 0x55dfca2489f0 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.703+0000 7fcc0eff5640 10 client.17109 handle_command_reply: tid=0 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.703+0000 7fcc37d53740 1 client.17109 shutdown 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.703+0000 7fcc37d53740 1 -- 172.21.15.3:0/228008204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca28a070 msgr2=0x55dfca649140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.703+0000 7fcc37d53740 1 --2- 172.21.15.3:0/228008204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca28a070 0x55dfca649140 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fcc20010d80 tx=0x7fcc20004730 comp rx=0 tx=0).stop 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.703+0000 7fcc37d53740 1 -- 172.21.15.3:0/228008204 shutdown_connections 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.703+0000 7fcc37d53740 1 --2- 172.21.15.3:0/228008204 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55dfca2489f0 0x55dfca64f080 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.703+0000 7fcc37d53740 1 --2- 172.21.15.3:0/228008204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 0x55dfca63d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.704+0000 7fcc0eff5640 0 client.17109 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.704+0000 7fcc37d53740 1 --2- 172.21.15.3:0/228008204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca28a070 0x55dfca649140 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.704+0000 7fcc37d53740 1 --2- 172.21.15.3:0/228008204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2611b0 0x55dfca63cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.704+0000 7fcc37d53740 1 -- 172.21.15.3:0/228008204 >> 172.21.15.3:0/228008204 conn(0x55dfca27e460 msgr2=0x55dfca634830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.704+0000 7fcc37d53740 1 -- 172.21.15.3:0/228008204 shutdown_connections 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.704+0000 7fcc37d53740 1 -- 172.21.15.3:0/228008204 wait complete. 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.704+0000 7fcc37d53740 20 client.17109 trim_cache size 0 max 16384 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.717+0000 7fcc37d53740 1 Processor -- start 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.717+0000 7fcc37d53740 1 -- start start 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.717+0000 7fcc37d53740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2489f0 0x55dfca63e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc37d53740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 0x55dfca62ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc37d53740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 0x55dfca62f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc37d53740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b1870 con 0x55dfca2611b0 2024-09-21T06:20:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc37d53740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b33e0 con 0x55dfca2489f0 2024-09-21T06:20:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc37d53740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b1f40 con 0x55dfca269390 2024-09-21T06:20:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc2affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 0x55dfca62ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc2affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 0x55dfca62ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47986/0 (socket says 172.21.15.3:47986) 2024-09-21T06:20:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc2affd640 1 -- 172.21.15.3:0/463236233 learned_addr learned my addr 172.21.15.3:0/463236233 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/463236233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 0x55dfca62f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/463236233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2489f0 0x55dfca63e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc2affd640 1 -- 172.21.15.3:0/463236233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 msgr2=0x55dfca62f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc2affd640 1 --2- 172.21.15.3:0/463236233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 0x55dfca62f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc2affd640 1 -- 172.21.15.3:0/463236233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2489f0 msgr2=0x55dfca63e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc2affd640 1 --2- 172.21.15.3:0/463236233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2489f0 0x55dfca63e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc2affd640 1 -- 172.21.15.3:0/463236233 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfca6379d0 con 0x55dfca2611b0 2024-09-21T06:20:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/463236233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2489f0 0x55dfca63e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:20:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc2affd640 1 --2- 172.21.15.3:0/463236233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 0x55dfca62ebc0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c02ade0 tx=0x7fcc1c009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae6ce6e425d5669 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc10ff9640 1 -- 172.21.15.3:0/463236233 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c04a040 con 0x55dfca2611b0 2024-09-21T06:20:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc10ff9640 1 -- 172.21.15.3:0/463236233 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1c036040 con 0x55dfca2611b0 2024-09-21T06:20:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.718+0000 7fcc10ff9640 1 -- 172.21.15.3:0/463236233 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c02fa80 con 0x55dfca2611b0 2024-09-21T06:20:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.719+0000 7fcc37d53740 1 -- 172.21.15.3:0/463236233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 msgr2=0x55dfca62ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.719+0000 7fcc37d53740 1 --2- 172.21.15.3:0/463236233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 0x55dfca62ebc0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c02ade0 tx=0x7fcc1c009880 comp rx=0 tx=0).stop 2024-09-21T06:20:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.719+0000 7fcc37d53740 1 -- 172.21.15.3:0/463236233 shutdown_connections 2024-09-21T06:20:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.719+0000 7fcc37d53740 1 --2- 172.21.15.3:0/463236233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca269390 0x55dfca62f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.719+0000 7fcc37d53740 1 --2- 172.21.15.3:0/463236233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca2611b0 0x55dfca62ebc0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.719+0000 7fcc37d53740 1 --2- 172.21.15.3:0/463236233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2489f0 0x55dfca63e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.719+0000 7fcc37d53740 1 -- 172.21.15.3:0/463236233 >> 172.21.15.3:0/463236233 conn(0x55dfca27e460 msgr2=0x55dfca63e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.719+0000 7fcc37d53740 1 -- 172.21.15.3:0/463236233 shutdown_connections 2024-09-21T06:20:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.719+0000 7fcc37d53740 1 -- 172.21.15.3:0/463236233 wait complete. 2024-09-21T06:20:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc37d53740 1 -- start start 2024-09-21T06:20:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc37d53740 1 Processor -- start 2024-09-21T06:20:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc37d53740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2489f0 0x55dfca6459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc37d53740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca2611b0 0x55dfca640ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc37d53740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca269390 0x55dfca641580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc37d53740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b2d10 con 0x55dfca269390 2024-09-21T06:20:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc37d53740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b2640 con 0x55dfca2489f0 2024-09-21T06:20:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc37d53740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1af850 con 0x55dfca2611b0 2024-09-21T06:20:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc37d53740 10 client.0 _ll_register_callbacks cb 0x7fcc200109c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc2a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca269390 0x55dfca641580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc2a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca269390 0x55dfca641580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47992/0 (socket says 172.21.15.3:47992) 2024-09-21T06:20:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc2a7fc640 1 -- 172.21.15.3:0/3911775733 learned_addr learned my addr 172.21.15.3:0/3911775733 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/3911775733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2489f0 0x55dfca6459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc2affd640 1 --2- 172.21.15.3:0/3911775733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca2611b0 0x55dfca640ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc2a7fc640 1 -- 172.21.15.3:0/3911775733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca2611b0 msgr2=0x55dfca640ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.720+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/3911775733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca2611b0 0x55dfca640ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc2a7fc640 1 -- 172.21.15.3:0/3911775733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2489f0 msgr2=0x55dfca6459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/3911775733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2489f0 0x55dfca6459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc2a7fc640 1 -- 172.21.15.3:0/3911775733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfca641d10 con 0x55dfca269390 2024-09-21T06:20:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc2a7fc640 1 --2- 172.21.15.3:0/3911775733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca269390 0x55dfca641580 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcc200059f0 tx=0x7fcc20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f267cd323005b13c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc0eff5640 1 -- 172.21.15.3:0/3911775733 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20004030 con 0x55dfca269390 2024-09-21T06:20:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc0eff5640 1 -- 172.21.15.3:0/3911775733 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcc20025030 con 0x55dfca269390 2024-09-21T06:20:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc37d53740 1 -- 172.21.15.3:0/3911775733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc1c036040 con 0x55dfca269390 2024-09-21T06:20:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc0eff5640 1 -- 172.21.15.3:0/3911775733 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcc200041d0 con 0x55dfca269390 2024-09-21T06:20:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc37d53740 10 client.17115 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc37d53740 1 -- 172.21.15.3:0/3911775733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dfca6b9a60 con 0x55dfca269390 2024-09-21T06:20:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc0eff5640 1 -- 172.21.15.3:0/3911775733 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1c036040 con 0x55dfca269390 2024-09-21T06:20:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc0eff5640 1 -- 172.21.15.3:0/3911775733 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fcc2003a410 con 0x55dfca269390 2024-09-21T06:20:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc37d53740 10 client.17115 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc37d53740 10 client.17115 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:20:25.755 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:20:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.721+0000 7fcc37d53740 20 client.17115 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc37d53740 1 --2- 172.21.15.3:0/3911775733 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55dfca28a070 0x55dfca6bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc37d53740 4 client.17115 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:20:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc37d53740 1 -- 172.21.15.3:0/3911775733 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfca5a58a0 con 0x55dfca28a070 2024-09-21T06:20:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc2affd640 1 --2- 172.21.15.3:0/3911775733 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55dfca28a070 0x55dfca6bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc0eff5640 10 client.17115 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:20:25.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc2affd640 1 --2- 172.21.15.3:0/3911775733 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55dfca28a070 0x55dfca6bc1c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=273e92a05d1eedcd server_cookie=a171471622218126 in_seq=0 out_seq=0 2024-09-21T06:20:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc0eff5640 1 -- 172.21.15.3:0/3911775733 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfca5a58a0 con 0x55dfca28a070 2024-09-21T06:20:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc0eff5640 10 client.17115 handle_command_reply: tid=0 2024-09-21T06:20:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc37d53740 1 client.17115 shutdown 2024-09-21T06:20:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc37d53740 1 -- 172.21.15.3:0/3911775733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca269390 msgr2=0x55dfca641580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc37d53740 1 --2- 172.21.15.3:0/3911775733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca269390 0x55dfca641580 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcc200059f0 tx=0x7fcc20004730 comp rx=0 tx=0).stop 2024-09-21T06:20:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc37d53740 1 -- 172.21.15.3:0/3911775733 shutdown_connections 2024-09-21T06:20:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc37d53740 1 --2- 172.21.15.3:0/3911775733 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55dfca28a070 0x55dfca6bc1c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc37d53740 1 --2- 172.21.15.3:0/3911775733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfca2611b0 0x55dfca640ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc37d53740 1 --2- 172.21.15.3:0/3911775733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfca269390 0x55dfca641580 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc37d53740 1 --2- 172.21.15.3:0/3911775733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfca2489f0 0x55dfca6459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc0eff5640 0 client.17115 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:20:25.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.722+0000 7fcc37d53740 1 -- 172.21.15.3:0/3911775733 >> 172.21.15.3:0/3911775733 conn(0x55dfca27e460 msgr2=0x55dfca63e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc37d53740 1 -- 172.21.15.3:0/3911775733 shutdown_connections 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc37d53740 1 -- 172.21.15.3:0/3911775733 wait complete. 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc37d53740 20 client.17115 trim_cache size 0 max 16384 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc317df640 1 -- 172.21.15.3:0/3562050759 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc140b6680 msgr2=0x7fcc140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc317df640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc140b6680 0x7fcc140b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcc240e4cc0 tx=0x7fcc24094040 comp rx=0 tx=0).stop 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc317df640 1 -- 172.21.15.3:0/3562050759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc2c10de30 msgr2=0x7fcc2c103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc317df640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc2c10de30 0x7fcc2c103210 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400d950 tx=0x7fcc2400de10 comp rx=0 tx=0).stop 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc317df640 1 -- 172.21.15.3:0/3562050759 shutdown_connections 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc317df640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc140b6680 0x7fcc140b8b90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc317df640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc2c1052b0 0x7fcc2c102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc317df640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc2c10de30 0x7fcc2c103210 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc317df640 1 --2- 172.21.15.3:0/3562050759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc2c104880 0x7fcc2c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc317df640 1 -- 172.21.15.3:0/3562050759 >> 172.21.15.3:0/3562050759 conn(0x7fcc2c079730 msgr2=0x7fcc2c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:25.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc317df640 1 -- 172.21.15.3:0/3562050759 shutdown_connections 2024-09-21T06:20:25.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:25.723+0000 7fcc317df640 1 -- 172.21.15.3:0/3562050759 wait complete. 2024-09-21T06:20:25.956 INFO:tasks.workunit.client.0.smithi003.stdout: 1 561658 12.33 MB/sec execute 118 sec latency 2502.632 ms 2024-09-21T06:20:26.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:20:26.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:20:26.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:20:26.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:20:26.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:20:26.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:20:26.530 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:20:26.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:20:26.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:20:26.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:20:26.835 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:26.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 561658 12.23 MB/sec execute 119 sec latency 3502.685 ms 2024-09-21T06:20:27.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:27 smithi003 ceph-mon[31227]: pgmap v488: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.9 MiB/s wr, 454 op/s 2024-09-21T06:20:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.369+0000 7f74cf740640 1 -- 172.21.15.3:0/4132500970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74c8102f50 msgr2=0x7f74c810fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.369+0000 7f74cf740640 1 --2- 172.21.15.3:0/4132500970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74c8102f50 0x7f74c810fc70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009900 tx=0x7f74b802a480 comp rx=0 tx=0).stop 2024-09-21T06:20:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.369+0000 7f74cf740640 1 -- 172.21.15.3:0/4132500970 shutdown_connections 2024-09-21T06:20:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.369+0000 7f74cf740640 1 --2- 172.21.15.3:0/4132500970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74c81101b0 0x7f74c81125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.369+0000 7f74cf740640 1 --2- 172.21.15.3:0/4132500970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74c8102f50 0x7f74c810fc70 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.369+0000 7f74cf740640 1 --2- 172.21.15.3:0/4132500970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c8102600 0x7f74c8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.369+0000 7f74cf740640 1 -- 172.21.15.3:0/4132500970 >> 172.21.15.3:0/4132500970 conn(0x7f74c8079750 msgr2=0x7f74c8079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.370+0000 7f74cf740640 1 -- 172.21.15.3:0/4132500970 shutdown_connections 2024-09-21T06:20:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.370+0000 7f74cf740640 1 -- 172.21.15.3:0/4132500970 wait complete. 2024-09-21T06:20:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.370+0000 7f74cf740640 1 Processor -- start 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.370+0000 7f74cf740640 1 -- start start 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cf740640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c8102600 0x7f74c819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cf740640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74c8102f50 0x7f74c819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cf740640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74c81101b0 0x7f74c81a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cf740640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f74c8115c70 con 0x7f74c8102f50 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cf740640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f74c8115af0 con 0x7f74c81101b0 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cf740640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74c8115df0 con 0x7f74c8102600 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cccdb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74c8102f50 0x7f74c819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cccdb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74c8102f50 0x7f74c819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48022/0 (socket says 172.21.15.3:48022) 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cccdb640 1 -- 172.21.15.3:0/3428412119 learned_addr learned my addr 172.21.15.3:0/3428412119 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cd4dc640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c8102600 0x7f74c819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cdcdd640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74c81101b0 0x7f74c81a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cccdb640 1 -- 172.21.15.3:0/3428412119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c8102600 msgr2=0x7f74c819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cccdb640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c8102600 0x7f74c819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cccdb640 1 -- 172.21.15.3:0/3428412119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74c81101b0 msgr2=0x7f74c81a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cccdb640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74c81101b0 0x7f74c81a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cccdb640 1 -- 172.21.15.3:0/3428412119 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c81a25d0 con 0x7f74c8102f50 2024-09-21T06:20:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74cccdb640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74c8102f50 0x7f74c819cc00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f74b8005940 tx=0x7f74b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.371+0000 7f74be7fc640 1 -- 172.21.15.3:0/3428412119 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b803f030 con 0x7f74c8102f50 2024-09-21T06:20:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.372+0000 7f74cf740640 1 -- 172.21.15.3:0/3428412119 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c81a2860 con 0x7f74c8102f50 2024-09-21T06:20:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.372+0000 7f74be7fc640 1 -- 172.21.15.3:0/3428412119 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b8003cb0 con 0x7f74c8102f50 2024-09-21T06:20:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.372+0000 7f74be7fc640 1 -- 172.21.15.3:0/3428412119 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8035070 con 0x7f74c8102f50 2024-09-21T06:20:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.372+0000 7f74cf740640 1 -- 172.21.15.3:0/3428412119 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c81a9020 con 0x7f74c8102f50 2024-09-21T06:20:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.375+0000 7f74d5cb4740 1 Processor -- start 2024-09-21T06:20:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.375+0000 7f74d5cb4740 1 -- start start 2024-09-21T06:20:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.375+0000 7f74be7fc640 1 -- 172.21.15.3:0/3428412119 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f74b8042030 con 0x7f74c8102f50 2024-09-21T06:20:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.375+0000 7f74d5cb4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 0x561a7c5146a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.375+0000 7f74d5cb4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c1441b0 0x561a7c514be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.375+0000 7f74d5cb4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c14c390 0x561a7c515120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.375+0000 7f74d5cb4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561a7c48aea0 con 0x561a7c16d070 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.375+0000 7f74d5cb4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561a7c47a2d0 con 0x561a7c14c390 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.375+0000 7f74d5cb4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561a7c47e0e0 con 0x561a7c1441b0 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74cdcdd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 0x561a7c5146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74cdcdd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 0x561a7c5146a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48026/0 (socket says 172.21.15.3:48026) 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74cdcdd640 1 -- 172.21.15.3:0/3479970029 learned_addr learned my addr 172.21.15.3:0/3479970029 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74cccdb640 1 --2- 172.21.15.3:0/3479970029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c14c390 0x561a7c515120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74cd4dc640 1 --2- 172.21.15.3:0/3479970029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c1441b0 0x561a7c514be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74cdcdd640 1 -- 172.21.15.3:0/3479970029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c1441b0 msgr2=0x561a7c514be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74cdcdd640 1 --2- 172.21.15.3:0/3479970029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c1441b0 0x561a7c514be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74cdcdd640 1 -- 172.21.15.3:0/3479970029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c14c390 msgr2=0x561a7c515120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74cdcdd640 1 --2- 172.21.15.3:0/3479970029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c14c390 0x561a7c515120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74cdcdd640 1 -- 172.21.15.3:0/3479970029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a7c1d8650 con 0x561a7c16d070 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74cccdb640 1 --2- 172.21.15.3:0/3479970029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c14c390 0x561a7c515120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74cdcdd640 1 --2- 172.21.15.3:0/3479970029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 0x561a7c5146a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009ee0 tx=0x7f74c0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a616418565f8b6ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74b37fe640 1 -- 172.21.15.3:0/3479970029 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0016070 con 0x561a7c16d070 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.376+0000 7f74b37fe640 1 -- 172.21.15.3:0/3479970029 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c0004d20 con 0x561a7c16d070 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.377+0000 7f74b37fe640 1 -- 172.21.15.3:0/3479970029 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c00126b0 con 0x561a7c16d070 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.377+0000 7f74be7fc640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74b40b6680 0x7f74b40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.377+0000 7f74cd4dc640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74b40b6680 0x7f74b40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.377+0000 7f74be7fc640 1 -- 172.21.15.3:0/3428412119 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f74b8107f90 con 0x7f74c8102f50 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.377+0000 7f74cd4dc640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74b40b6680 0x7f74b40b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x561a7c513320 tx=0x7f74c400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.377+0000 7f74d5cb4740 1 -- 172.21.15.3:0/3479970029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 msgr2=0x561a7c5146a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.377+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/3479970029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 0x561a7c5146a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009ee0 tx=0x7f74c0010740 comp rx=0 tx=0).stop 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.377+0000 7f74d5cb4740 1 -- 172.21.15.3:0/3479970029 shutdown_connections 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.377+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/3479970029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c1441b0 0x561a7c514be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.377+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/3479970029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c14c390 0x561a7c515120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.377+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/3479970029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 0x561a7c5146a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.377+0000 7f74d5cb4740 1 -- 172.21.15.3:0/3479970029 >> 172.21.15.3:0/3479970029 conn(0x561a7c161460 msgr2=0x561a7c512590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.377+0000 7f74d5cb4740 1 -- 172.21.15.3:0/3479970029 shutdown_connections 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.378+0000 7f74d5cb4740 1 -- 172.21.15.3:0/3479970029 wait complete. 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.378+0000 7f74d5cb4740 1 -- start start 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.378+0000 7f74d5cb4740 1 Processor -- start 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.378+0000 7f74d5cb4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c1441b0 0x561a7c51fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.378+0000 7f74d5cb4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c14c390 0x561a7c520310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.378+0000 7f74d5cb4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 0x561a7c52c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.378+0000 7f74d5cb4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561a7c2db700 con 0x561a7c16d070 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.378+0000 7f74d5cb4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561a7c415160 con 0x561a7c1441b0 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.378+0000 7f74d5cb4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561a7c522e80 con 0x561a7c14c390 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74d5cb4740 10 client.0 _ll_register_callbacks cb 0x561a7c42e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74cdcdd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 0x561a7c52c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74cdcdd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 0x561a7c52c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48030/0 (socket says 172.21.15.3:48030) 2024-09-21T06:20:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74cdcdd640 1 -- 172.21.15.3:0/1928667408 learned_addr learned my addr 172.21.15.3:0/1928667408 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74cd4dc640 1 --2- 172.21.15.3:0/1928667408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c1441b0 0x561a7c51fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74cccdb640 1 --2- 172.21.15.3:0/1928667408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c14c390 0x561a7c520310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74cdcdd640 1 -- 172.21.15.3:0/1928667408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c14c390 msgr2=0x561a7c520310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74cdcdd640 1 --2- 172.21.15.3:0/1928667408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c14c390 0x561a7c520310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74cdcdd640 1 -- 172.21.15.3:0/1928667408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c1441b0 msgr2=0x561a7c51fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74cdcdd640 1 --2- 172.21.15.3:0/1928667408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c1441b0 0x561a7c51fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74cdcdd640 1 -- 172.21.15.3:0/1928667408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7c442c90 con 0x561a7c16d070 2024-09-21T06:20:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74cdcdd640 1 --2- 172.21.15.3:0/1928667408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 0x561a7c52c1d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f74c0000f80 tx=0x7f74c0009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec5a0498626949c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.379+0000 7f74b0ff9640 1 -- 172.21.15.3:0/1928667408 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0028060 con 0x561a7c16d070 2024-09-21T06:20:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.380+0000 7f74b0ff9640 1 -- 172.21.15.3:0/1928667408 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f74c00050e0 con 0x561a7c16d070 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.380+0000 7f74d5cb4740 1 -- 172.21.15.3:0/1928667408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c0004d20 con 0x561a7c16d070 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.380+0000 7f74b0ff9640 1 -- 172.21.15.3:0/1928667408 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f74c0016070 con 0x561a7c16d070 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.380+0000 7f74d5cb4740 10 client.17127 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.380+0000 7f74d5cb4740 1 -- 172.21.15.3:0/1928667408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561a7c52ec00 con 0x561a7c16d070 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.380+0000 7f74b0ff9640 1 -- 172.21.15.3:0/1928667408 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c0004d20 con 0x561a7c16d070 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.380+0000 7f74b0ff9640 1 -- 172.21.15.3:0/1928667408 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f74c000f040 con 0x561a7c16d070 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.380+0000 7f74d5cb4740 10 client.17127 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.380+0000 7f74d5cb4740 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.380+0000 7f74d5cb4740 20 client.17127 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.380+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/1928667408 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561a7c12b9f0 0x561a7c532070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.380+0000 7f74d5cb4740 4 client.17127 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.380+0000 7f74d5cb4740 1 -- 172.21.15.3:0/1928667408 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a7c48cdf0 con 0x561a7c12b9f0 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.381+0000 7f74cd4dc640 1 --2- 172.21.15.3:0/1928667408 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561a7c12b9f0 0x561a7c532070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.381+0000 7f74cd4dc640 1 --2- 172.21.15.3:0/1928667408 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561a7c12b9f0 0x561a7c532070 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24b0b70788be679a server_cookie=ad07dee30b57554b in_seq=0 out_seq=0 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.381+0000 7f74b0ff9640 10 client.17127 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74b0ff9640 1 -- 172.21.15.3:0/1928667408 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561a7c48cdf0 con 0x561a7c12b9f0 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74b0ff9640 10 client.17127 handle_command_reply: tid=0 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74d5cb4740 1 client.17127 shutdown 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74d5cb4740 1 -- 172.21.15.3:0/1928667408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 msgr2=0x561a7c52c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/1928667408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 0x561a7c52c1d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f74c0000f80 tx=0x7f74c0009c50 comp rx=0 tx=0).stop 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74d5cb4740 1 -- 172.21.15.3:0/1928667408 shutdown_connections 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/1928667408 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561a7c12b9f0 0x561a7c532070 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/1928667408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c14c390 0x561a7c520310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/1928667408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c16d070 0x561a7c52c1d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74b0ff9640 0 client.17127 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/1928667408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c1441b0 0x561a7c51fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74d5cb4740 1 -- 172.21.15.3:0/1928667408 >> 172.21.15.3:0/1928667408 conn(0x561a7c161460 msgr2=0x561a7c517880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74d5cb4740 1 -- 172.21.15.3:0/1928667408 shutdown_connections 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74d5cb4740 1 -- 172.21.15.3:0/1928667408 wait complete. 2024-09-21T06:20:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.382+0000 7f74d5cb4740 20 client.17127 trim_cache size 0 max 16384 2024-09-21T06:20:27.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74d5cb4740 1 Processor -- start 2024-09-21T06:20:27.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74d5cb4740 1 -- start start 2024-09-21T06:20:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74d5cb4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c1441b0 0x561a7c517eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74d5cb4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c14c390 0x561a7c52c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74d5cb4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c16d070 0x561a7c51fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74d5cb4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81b1810 con 0x561a7c14c390 2024-09-21T06:20:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74d5cb4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81b3380 con 0x561a7c1441b0 2024-09-21T06:20:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74d5cb4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81b1ee0 con 0x561a7c16d070 2024-09-21T06:20:27.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74cccdb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c14c390 0x561a7c52c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74cccdb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c14c390 0x561a7c52c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48034/0 (socket says 172.21.15.3:48034) 2024-09-21T06:20:27.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74cccdb640 1 -- 172.21.15.3:0/1493075957 learned_addr learned my addr 172.21.15.3:0/1493075957 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:27.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74cd4dc640 1 --2- 172.21.15.3:0/1493075957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c1441b0 0x561a7c517eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74cdcdd640 1 --2- 172.21.15.3:0/1493075957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c16d070 0x561a7c51fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74cccdb640 1 -- 172.21.15.3:0/1493075957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c16d070 msgr2=0x561a7c51fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74cccdb640 1 --2- 172.21.15.3:0/1493075957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c16d070 0x561a7c51fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74cccdb640 1 -- 172.21.15.3:0/1493075957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c1441b0 msgr2=0x561a7c517eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74cccdb640 1 --2- 172.21.15.3:0/1493075957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c1441b0 0x561a7c517eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74cccdb640 1 -- 172.21.15.3:0/1493075957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a7c5183f0 con 0x561a7c14c390 2024-09-21T06:20:27.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.396+0000 7f74cdcdd640 1 --2- 172.21.15.3:0/1493075957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c16d070 0x561a7c51fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:20:27.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.397+0000 7f74cccdb640 1 --2- 172.21.15.3:0/1493075957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c14c390 0x561a7c52c140 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f74b8105d20 tx=0x7f74b8032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4468626b308a80b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.397+0000 7f74b2ffd640 1 -- 172.21.15.3:0/1493075957 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8049040 con 0x561a7c14c390 2024-09-21T06:20:27.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.397+0000 7f74b2ffd640 1 -- 172.21.15.3:0/1493075957 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b80cd930 con 0x561a7c14c390 2024-09-21T06:20:27.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.397+0000 7f74b2ffd640 1 -- 172.21.15.3:0/1493075957 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b80ca090 con 0x561a7c14c390 2024-09-21T06:20:27.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.398+0000 7f74d5cb4740 1 -- 172.21.15.3:0/1493075957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c14c390 msgr2=0x561a7c52c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.398+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/1493075957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c14c390 0x561a7c52c140 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f74b8105d20 tx=0x7f74b8032f60 comp rx=0 tx=0).stop 2024-09-21T06:20:27.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.398+0000 7f74d5cb4740 1 -- 172.21.15.3:0/1493075957 shutdown_connections 2024-09-21T06:20:27.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.398+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/1493075957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c16d070 0x561a7c51fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.398+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/1493075957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c14c390 0x561a7c52c140 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.398+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/1493075957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c1441b0 0x561a7c517eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.398+0000 7f74d5cb4740 1 -- 172.21.15.3:0/1493075957 >> 172.21.15.3:0/1493075957 conn(0x561a7c161460 msgr2=0x561a7c5177a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.398+0000 7f74d5cb4740 1 -- 172.21.15.3:0/1493075957 shutdown_connections 2024-09-21T06:20:27.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.398+0000 7f74d5cb4740 1 -- 172.21.15.3:0/1493075957 wait complete. 2024-09-21T06:20:27.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.398+0000 7f74d5cb4740 1 -- start start 2024-09-21T06:20:27.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.398+0000 7f74d5cb4740 1 Processor -- start 2024-09-21T06:20:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74d5cb4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c1441b0 0x561a7c535840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74d5cb4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c14c390 0x561a7c532d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74d5cb4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c16d070 0x561a7c533250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74d5cb4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81b2cb0 con 0x561a7c1441b0 2024-09-21T06:20:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74d5cb4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81b25e0 con 0x561a7c16d070 2024-09-21T06:20:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74d5cb4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81af7f0 con 0x561a7c14c390 2024-09-21T06:20:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74d5cb4740 10 client.0 _ll_register_callbacks cb 0x7f74c400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74cd4dc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c1441b0 0x561a7c535840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74cd4dc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c1441b0 0x561a7c535840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48040/0 (socket says 172.21.15.3:48040) 2024-09-21T06:20:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74cd4dc640 1 -- 172.21.15.3:0/84679442 learned_addr learned my addr 172.21.15.3:0/84679442 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:27.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74cccdb640 1 --2- 172.21.15.3:0/84679442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c14c390 0x561a7c532d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74cdcdd640 1 --2- 172.21.15.3:0/84679442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c16d070 0x561a7c533250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74cd4dc640 1 -- 172.21.15.3:0/84679442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c14c390 msgr2=0x561a7c532d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74cd4dc640 1 --2- 172.21.15.3:0/84679442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c14c390 0x561a7c532d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74cd4dc640 1 -- 172.21.15.3:0/84679442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c16d070 msgr2=0x561a7c533250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74cd4dc640 1 --2- 172.21.15.3:0/84679442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c16d070 0x561a7c533250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74cd4dc640 1 -- 172.21.15.3:0/84679442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7c533990 con 0x561a7c1441b0 2024-09-21T06:20:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74cd4dc640 1 --2- 172.21.15.3:0/84679442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c1441b0 0x561a7c535840 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f74c40043f0 tx=0x7f74c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6627daa96c6ba287 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.399+0000 7f74b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74b0ff9640 1 -- 172.21.15.3:0/84679442 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c401e8f0 con 0x561a7c1441b0 2024-09-21T06:20:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74b0ff9640 1 -- 172.21.15.3:0/84679442 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f74c401ea90 con 0x561a7c1441b0 2024-09-21T06:20:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74d5cb4740 1 -- 172.21.15.3:0/84679442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b80cd930 con 0x561a7c1441b0 2024-09-21T06:20:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74b0ff9640 1 -- 172.21.15.3:0/84679442 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f74c401ec90 con 0x561a7c1441b0 2024-09-21T06:20:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74d5cb4740 10 client.17133 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:27.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74d5cb4740 1 -- 172.21.15.3:0/84679442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561a7c59c6e0 con 0x561a7c1441b0 2024-09-21T06:20:27.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74b0ff9640 1 -- 172.21.15.3:0/84679442 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b80cd930 con 0x561a7c1441b0 2024-09-21T06:20:27.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74b0ff9640 1 -- 172.21.15.3:0/84679442 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f74c401d480 con 0x561a7c1441b0 2024-09-21T06:20:27.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74d5cb4740 10 client.17133 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:27.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74d5cb4740 10 client.17133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:27.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74d5cb4740 20 client.17133 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:27.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/84679442 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561a7c100da0 0x561a7c59eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74d5cb4740 4 client.17133 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:20:27.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.400+0000 7f74d5cb4740 1 -- 172.21.15.3:0/84679442 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a7c4888a0 con 0x561a7c100da0 2024-09-21T06:20:27.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.401+0000 7f74cccdb640 1 --2- 172.21.15.3:0/84679442 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561a7c100da0 0x561a7c59eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.401+0000 7f74b0ff9640 10 client.17133 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:27.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.401+0000 7f74cccdb640 1 --2- 172.21.15.3:0/84679442 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561a7c100da0 0x561a7c59eed0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee636803c6fe3fe server_cookie=2f4e57e443a00960 in_seq=0 out_seq=0 2024-09-21T06:20:27.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.401+0000 7f74b0ff9640 1 -- 172.21.15.3:0/84679442 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a7c4888a0 con 0x561a7c100da0 2024-09-21T06:20:27.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.401+0000 7f74b0ff9640 10 client.17133 handle_command_reply: tid=0 2024-09-21T06:20:27.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.401+0000 7f74d5cb4740 1 client.17133 shutdown 2024-09-21T06:20:27.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.401+0000 7f74d5cb4740 1 -- 172.21.15.3:0/84679442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c1441b0 msgr2=0x561a7c535840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.401+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/84679442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c1441b0 0x561a7c535840 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f74c40043f0 tx=0x7f74c40046c0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.401+0000 7f74d5cb4740 1 -- 172.21.15.3:0/84679442 shutdown_connections 2024-09-21T06:20:27.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/84679442 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561a7c100da0 0x561a7c59eed0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/84679442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a7c14c390 0x561a7c532d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/84679442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a7c16d070 0x561a7c533250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74d5cb4740 1 --2- 172.21.15.3:0/84679442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a7c1441b0 0x561a7c535840 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74d5cb4740 1 -- 172.21.15.3:0/84679442 >> 172.21.15.3:0/84679442 conn(0x561a7c161460 msgr2=0x561a7c51fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74b0ff9640 0 client.17133 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74d5cb4740 1 -- 172.21.15.3:0/84679442 shutdown_connections 2024-09-21T06:20:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74d5cb4740 1 -- 172.21.15.3:0/84679442 wait complete. 2024-09-21T06:20:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74d5cb4740 20 client.17133 trim_cache size 0 max 16384 2024-09-21T06:20:27.410 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:20:27.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74cf740640 1 -- 172.21.15.3:0/3428412119 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74b40b6680 msgr2=0x7f74b40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74cf740640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74b40b6680 0x7f74b40b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x561a7c513320 tx=0x7f74c400b390 comp rx=0 tx=0).stop 2024-09-21T06:20:27.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74cf740640 1 -- 172.21.15.3:0/3428412119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74c8102f50 msgr2=0x7f74c819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74cf740640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74c8102f50 0x7f74c819cc00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f74b8005940 tx=0x7f74b80042f0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74cf740640 1 -- 172.21.15.3:0/3428412119 shutdown_connections 2024-09-21T06:20:27.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74cf740640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f74b40b6680 0x7f74b40b8b90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74cf740640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74c81101b0 0x7f74c81a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74cf740640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74c8102f50 0x7f74c819cc00 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74cf740640 1 --2- 172.21.15.3:0/3428412119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c8102600 0x7f74c819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.402+0000 7f74cf740640 1 -- 172.21.15.3:0/3428412119 >> 172.21.15.3:0/3428412119 conn(0x7f74c8079750 msgr2=0x7f74c8108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.403+0000 7f74cf740640 1 -- 172.21.15.3:0/3428412119 shutdown_connections 2024-09-21T06:20:27.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:27.403+0000 7f74cf740640 1 -- 172.21.15.3:0/3428412119 wait complete. 2024-09-21T06:20:27.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:27 smithi137 ceph-mon[34464]: pgmap v488: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.9 MiB/s wr, 454 op/s 2024-09-21T06:20:27.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:27 smithi204 ceph-mon[34659]: pgmap v488: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.9 MiB/s wr, 454 op/s 2024-09-21T06:20:27.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 561658 12.13 MB/sec execute 120 sec latency 4502.728 ms 2024-09-21T06:20:28.956 INFO:tasks.workunit.client.0.smithi003.stdout: 1 562137 12.03 MB/sec execute 121 sec latency 5039.772 ms 2024-09-21T06:20:29.255 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:20:29.256 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:20:29.552 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:29.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:29 smithi003 ceph-mon[31227]: pgmap v489: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 330 op/s 2024-09-21T06:20:29.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:29 smithi137 ceph-mon[34464]: pgmap v489: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 330 op/s 2024-09-21T06:20:29.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:29 smithi204 ceph-mon[34659]: pgmap v489: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 330 op/s 2024-09-21T06:20:29.956 INFO:tasks.workunit.client.0.smithi003.stdout: 1 564629 12.05 MB/sec execute 122 sec latency 40.076 ms 2024-09-21T06:20:30.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.043+0000 7fef854c5640 1 -- 172.21.15.3:0/869255376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef80108d10 msgr2=0x7fef8010f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:30.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.043+0000 7fef854c5640 1 --2- 172.21.15.3:0/869255376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef80108d10 0x7fef8010f540 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fef78009ca0 tx=0x7fef7802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.043+0000 7fef854c5640 1 -- 172.21.15.3:0/869255376 shutdown_connections 2024-09-21T06:20:30.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.043+0000 7fef854c5640 1 --2- 172.21.15.3:0/869255376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef80108d10 0x7fef8010f540 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.043+0000 7fef854c5640 1 --2- 172.21.15.3:0/869255376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef801083c0 0x7fef801087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.043+0000 7fef854c5640 1 --2- 172.21.15.3:0/869255376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef80102a80 0x7fef80102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.043+0000 7fef854c5640 1 -- 172.21.15.3:0/869255376 >> 172.21.15.3:0/869255376 conn(0x7fef80079740 msgr2=0x7fef80079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:30.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.043+0000 7fef854c5640 1 -- 172.21.15.3:0/869255376 shutdown_connections 2024-09-21T06:20:30.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.043+0000 7fef854c5640 1 -- 172.21.15.3:0/869255376 wait complete. 2024-09-21T06:20:30.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.044+0000 7fef854c5640 1 Processor -- start 2024-09-21T06:20:30.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.044+0000 7fef854c5640 1 -- start start 2024-09-21T06:20:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.044+0000 7fef854c5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef80102a80 0x7fef801982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.044+0000 7fef854c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef801083c0 0x7fef80198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.044+0000 7fef854c5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef80108d10 0x7fef8019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.044+0000 7fef854c5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fef80111880 con 0x7fef80102a80 2024-09-21T06:20:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.044+0000 7fef854c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef80111700 con 0x7fef801083c0 2024-09-21T06:20:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.044+0000 7fef854c5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fef80111a00 con 0x7fef80108d10 2024-09-21T06:20:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef7effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef80102a80 0x7fef801982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef7effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef80102a80 0x7fef801982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48052/0 (socket says 172.21.15.3:48052) 2024-09-21T06:20:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef7effd640 1 -- 172.21.15.3:0/4120189663 learned_addr learned my addr 172.21.15.3:0/4120189663 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef80108d10 0x7fef8019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef7e7fc640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef801083c0 0x7fef80198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef7effd640 1 -- 172.21.15.3:0/4120189663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef80108d10 msgr2=0x7fef8019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef7effd640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef80108d10 0x7fef8019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef7effd640 1 -- 172.21.15.3:0/4120189663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef801083c0 msgr2=0x7fef80198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef7effd640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef801083c0 0x7fef80198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef7effd640 1 -- 172.21.15.3:0/4120189663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8019e240 con 0x7fef80102a80 2024-09-21T06:20:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef80108d10 0x7fef8019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef7effd640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef80102a80 0x7fef801982d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fef70010ad0 tx=0x7fef70010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef67fff640 1 -- 172.21.15.3:0/4120189663 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7001a030 con 0x7fef80102a80 2024-09-21T06:20:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.045+0000 7fef854c5640 1 -- 172.21.15.3:0/4120189663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8019e530 con 0x7fef80102a80 2024-09-21T06:20:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.046+0000 7fef67fff640 1 -- 172.21.15.3:0/4120189663 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef70005020 con 0x7fef80102a80 2024-09-21T06:20:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.046+0000 7fef67fff640 1 -- 172.21.15.3:0/4120189663 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70004bc0 con 0x7fef80102a80 2024-09-21T06:20:30.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.046+0000 7fef854c5640 1 -- 172.21.15.3:0/4120189663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef801a4c90 con 0x7fef80102a80 2024-09-21T06:20:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.048+0000 7fef8ba39740 1 Processor -- start 2024-09-21T06:20:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.048+0000 7fef8ba39740 1 -- start start 2024-09-21T06:20:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.048+0000 7fef8ba39740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c45d350 0x562a9c804430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.048+0000 7fef8ba39740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c434490 0x562a9c804970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.048+0000 7fef8ba39740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c43c670 0x562a9c804eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.048+0000 7fef8ba39740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562a9c77ae60 con 0x562a9c434490 2024-09-21T06:20:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.048+0000 7fef8ba39740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562a9c76a290 con 0x562a9c43c670 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.048+0000 7fef8ba39740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562a9c76e0a0 con 0x562a9c45d350 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.049+0000 7fef67fff640 1 -- 172.21.15.3:0/4120189663 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fef7001d070 con 0x7fef80102a80 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.049+0000 7fef7f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c45d350 0x562a9c804430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.049+0000 7fef7f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c45d350 0x562a9c804430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:53080/0 (socket says 172.21.15.3:53080) 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.049+0000 7fef7f7fe640 1 -- 172.21.15.3:0/1553013155 learned_addr learned my addr 172.21.15.3:0/1553013155 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.049+0000 7fef7e7fc640 1 --2- 172.21.15.3:0/1553013155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c43c670 0x562a9c804eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.049+0000 7fef7effd640 1 --2- 172.21.15.3:0/1553013155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c434490 0x562a9c804970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.049+0000 7fef7f7fe640 1 -- 172.21.15.3:0/1553013155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c43c670 msgr2=0x562a9c804eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.049+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/1553013155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c43c670 0x562a9c804eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.049+0000 7fef7f7fe640 1 -- 172.21.15.3:0/1553013155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c434490 msgr2=0x562a9c804970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.049+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/1553013155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c434490 0x562a9c804970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.049+0000 7fef7f7fe640 1 -- 172.21.15.3:0/1553013155 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a9c4c8930 con 0x562a9c45d350 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.049+0000 7fef7e7fc640 1 --2- 172.21.15.3:0/1553013155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c43c670 0x562a9c804eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.049+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/1553013155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c45d350 0x562a9c804430 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef78033040 tx=0x7fef7802f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e17b3a4c3c777764 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.050+0000 7fef67fff640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fef680b6700 0x7fef680b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.050+0000 7fef657fa640 1 -- 172.21.15.3:0/1553013155 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78031650 con 0x562a9c45d350 2024-09-21T06:20:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.050+0000 7fef7effd640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fef680b6700 0x7fef680b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.050+0000 7fef657fa640 1 -- 172.21.15.3:0/1553013155 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef78036040 con 0x562a9c45d350 2024-09-21T06:20:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.050+0000 7fef657fa640 1 -- 172.21.15.3:0/1553013155 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78004460 con 0x562a9c45d350 2024-09-21T06:20:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.050+0000 7fef67fff640 1 -- 172.21.15.3:0/4120189663 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fef700e9790 con 0x7fef80102a80 2024-09-21T06:20:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.051+0000 7fef8ba39740 1 -- 172.21.15.3:0/1553013155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c45d350 msgr2=0x562a9c804430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.051+0000 7fef7effd640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fef680b6700 0x7fef680b8c10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fef700e6fd0 tx=0x7fef700ef550 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.051+0000 7fef8ba39740 1 --2- 172.21.15.3:0/1553013155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c45d350 0x562a9c804430 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef78033040 tx=0x7fef7802f310 comp rx=0 tx=0).stop 2024-09-21T06:20:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.051+0000 7fef8ba39740 1 -- 172.21.15.3:0/1553013155 shutdown_connections 2024-09-21T06:20:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.051+0000 7fef8ba39740 1 --2- 172.21.15.3:0/1553013155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c43c670 0x562a9c804eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.051+0000 7fef8ba39740 1 --2- 172.21.15.3:0/1553013155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c434490 0x562a9c804970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.051+0000 7fef8ba39740 1 --2- 172.21.15.3:0/1553013155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c45d350 0x562a9c804430 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.051+0000 7fef8ba39740 1 -- 172.21.15.3:0/1553013155 >> 172.21.15.3:0/1553013155 conn(0x562a9c451740 msgr2=0x562a9c802320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.051+0000 7fef8ba39740 1 -- 172.21.15.3:0/1553013155 shutdown_connections 2024-09-21T06:20:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.051+0000 7fef8ba39740 1 -- 172.21.15.3:0/1553013155 wait complete. 2024-09-21T06:20:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.051+0000 7fef8ba39740 1 -- start start 2024-09-21T06:20:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef8ba39740 1 Processor -- start 2024-09-21T06:20:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef8ba39740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c434490 0x562a9c80fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef8ba39740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c43c670 0x562a9c810530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef8ba39740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c45d350 0x562a9c81bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef8ba39740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562a9c5cb890 con 0x562a9c43c670 2024-09-21T06:20:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef8ba39740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562a9c704d40 con 0x562a9c434490 2024-09-21T06:20:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef8ba39740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562a9c812a30 con 0x562a9c45d350 2024-09-21T06:20:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef7e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c43c670 0x562a9c810530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef8ba39740 10 client.0 _ll_register_callbacks cb 0x562a9c768c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef7f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c434490 0x562a9c80fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef7e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c43c670 0x562a9c810530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48062/0 (socket says 172.21.15.3:48062) 2024-09-21T06:20:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef7e7fc640 1 -- 172.21.15.3:0/2340937816 learned_addr learned my addr 172.21.15.3:0/2340937816 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/2340937816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c45d350 0x562a9c81bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef7e7fc640 1 -- 172.21.15.3:0/2340937816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c45d350 msgr2=0x562a9c81bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef7e7fc640 1 --2- 172.21.15.3:0/2340937816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c45d350 0x562a9c81bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef7e7fc640 1 -- 172.21.15.3:0/2340937816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c434490 msgr2=0x562a9c80fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef7e7fc640 1 --2- 172.21.15.3:0/2340937816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c434490 0x562a9c80fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.052+0000 7fef7e7fc640 1 -- 172.21.15.3:0/2340937816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a9c81c310 con 0x562a9c43c670 2024-09-21T06:20:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef7e7fc640 1 --2- 172.21.15.3:0/2340937816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c43c670 0x562a9c810530 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fef74012700 tx=0x7fef74012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ea75613bd728c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef62ff5640 1 -- 172.21.15.3:0/2340937816 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74004050 con 0x562a9c43c670 2024-09-21T06:20:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef62ff5640 1 -- 172.21.15.3:0/2340937816 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fef740041f0 con 0x562a9c43c670 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef8ba39740 1 -- 172.21.15.3:0/2340937816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef78036040 con 0x562a9c43c670 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef62ff5640 1 -- 172.21.15.3:0/2340937816 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fef74004430 con 0x562a9c43c670 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef8ba39740 10 client.17145 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef8ba39740 1 -- 172.21.15.3:0/2340937816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562a9c81ecd0 con 0x562a9c43c670 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef62ff5640 1 -- 172.21.15.3:0/2340937816 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef78036040 con 0x562a9c43c670 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef62ff5640 1 -- 172.21.15.3:0/2340937816 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fef74003b90 con 0x562a9c43c670 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef8ba39740 10 client.17145 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef8ba39740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef8ba39740 20 client.17145 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.053+0000 7fef8ba39740 1 --2- 172.21.15.3:0/2340937816 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562a9c41bcd0 0x562a9c822140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.054+0000 7fef8ba39740 4 client.17145 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.054+0000 7fef8ba39740 1 -- 172.21.15.3:0/2340937816 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a9c77cdb0 con 0x562a9c41bcd0 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.054+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/2340937816 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562a9c41bcd0 0x562a9c822140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.054+0000 7fef62ff5640 10 client.17145 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.054+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/2340937816 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562a9c41bcd0 0x562a9c822140 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f87ce82ebeeefb9c server_cookie=8543d754bcced02a in_seq=0 out_seq=0 2024-09-21T06:20:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef62ff5640 1 -- 172.21.15.3:0/2340937816 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a9c77cdb0 con 0x562a9c41bcd0 2024-09-21T06:20:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef62ff5640 10 client.17145 handle_command_reply: tid=0 2024-09-21T06:20:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef8ba39740 1 client.17145 shutdown 2024-09-21T06:20:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef8ba39740 1 -- 172.21.15.3:0/2340937816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c43c670 msgr2=0x562a9c810530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef8ba39740 1 --2- 172.21.15.3:0/2340937816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c43c670 0x562a9c810530 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fef74012700 tx=0x7fef74012bc0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef8ba39740 1 -- 172.21.15.3:0/2340937816 shutdown_connections 2024-09-21T06:20:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef8ba39740 1 --2- 172.21.15.3:0/2340937816 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562a9c41bcd0 0x562a9c822140 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef8ba39740 1 --2- 172.21.15.3:0/2340937816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c45d350 0x562a9c81bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef62ff5640 0 client.17145 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef8ba39740 1 --2- 172.21.15.3:0/2340937816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c43c670 0x562a9c810530 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef8ba39740 1 --2- 172.21.15.3:0/2340937816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c434490 0x562a9c80fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef8ba39740 1 -- 172.21.15.3:0/2340937816 >> 172.21.15.3:0/2340937816 conn(0x562a9c451740 msgr2=0x562a9c8075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef8ba39740 1 -- 172.21.15.3:0/2340937816 shutdown_connections 2024-09-21T06:20:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef8ba39740 1 -- 172.21.15.3:0/2340937816 wait complete. 2024-09-21T06:20:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.055+0000 7fef8ba39740 20 client.17145 trim_cache size 0 max 16384 2024-09-21T06:20:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.074+0000 7fef8ba39740 1 Processor -- start 2024-09-21T06:20:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.074+0000 7fef8ba39740 1 -- start start 2024-09-21T06:20:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.074+0000 7fef8ba39740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c41bcd0 0x562a9c80a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.074+0000 7fef8ba39740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c434490 0x562a9c81bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.074+0000 7fef8ba39740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c43c670 0x562a9c80cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.074+0000 7fef8ba39740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fef801ad3d0 con 0x562a9c41bcd0 2024-09-21T06:20:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.074+0000 7fef8ba39740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef801aef40 con 0x562a9c43c670 2024-09-21T06:20:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.074+0000 7fef8ba39740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fef801adaa0 con 0x562a9c434490 2024-09-21T06:20:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.074+0000 7fef7f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c41bcd0 0x562a9c80a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.074+0000 7fef7e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c434490 0x562a9c81bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.074+0000 7fef7e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c434490 0x562a9c81bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:53102/0 (socket says 172.21.15.3:53102) 2024-09-21T06:20:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.074+0000 7fef7e7fc640 1 -- 172.21.15.3:0/2402482093 learned_addr learned my addr 172.21.15.3:0/2402482093 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.074+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/2402482093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c43c670 0x562a9c80cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.075+0000 7fef7e7fc640 1 -- 172.21.15.3:0/2402482093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c43c670 msgr2=0x562a9c80cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.075+0000 7fef7e7fc640 1 --2- 172.21.15.3:0/2402482093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c43c670 0x562a9c80cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.075+0000 7fef7e7fc640 1 -- 172.21.15.3:0/2402482093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c41bcd0 msgr2=0x562a9c80a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.075+0000 7fef7e7fc640 1 --2- 172.21.15.3:0/2402482093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c41bcd0 0x562a9c80a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.075+0000 7fef7e7fc640 1 -- 172.21.15.3:0/2402482093 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a9c807290 con 0x562a9c434490 2024-09-21T06:20:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.075+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/2402482093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c43c670 0x562a9c80cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.075+0000 7fef7e7fc640 1 --2- 172.21.15.3:0/2402482093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c434490 0x562a9c81bc20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fef74003f30 tx=0x7fef74025860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c98344166c7613 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.075+0000 7fef64ff9640 1 -- 172.21.15.3:0/2402482093 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74004030 con 0x562a9c434490 2024-09-21T06:20:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.075+0000 7fef64ff9640 1 -- 172.21.15.3:0/2402482093 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef740041d0 con 0x562a9c434490 2024-09-21T06:20:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.075+0000 7fef64ff9640 1 -- 172.21.15.3:0/2402482093 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7400fd60 con 0x562a9c434490 2024-09-21T06:20:30.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.076+0000 7fef8ba39740 1 -- 172.21.15.3:0/2402482093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c434490 msgr2=0x562a9c81bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:30.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.076+0000 7fef8ba39740 1 --2- 172.21.15.3:0/2402482093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c434490 0x562a9c81bc20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fef74003f30 tx=0x7fef74025860 comp rx=0 tx=0).stop 2024-09-21T06:20:30.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.076+0000 7fef8ba39740 1 -- 172.21.15.3:0/2402482093 shutdown_connections 2024-09-21T06:20:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.076+0000 7fef8ba39740 1 --2- 172.21.15.3:0/2402482093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c434490 0x562a9c81bc20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.076+0000 7fef8ba39740 1 --2- 172.21.15.3:0/2402482093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c43c670 0x562a9c80cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.076+0000 7fef8ba39740 1 --2- 172.21.15.3:0/2402482093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c41bcd0 0x562a9c80a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.076+0000 7fef8ba39740 1 -- 172.21.15.3:0/2402482093 >> 172.21.15.3:0/2402482093 conn(0x562a9c451740 msgr2=0x562a9c80e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.076+0000 7fef8ba39740 1 -- 172.21.15.3:0/2402482093 shutdown_connections 2024-09-21T06:20:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.076+0000 7fef8ba39740 1 -- 172.21.15.3:0/2402482093 wait complete. 2024-09-21T06:20:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.076+0000 7fef8ba39740 1 -- start start 2024-09-21T06:20:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef8ba39740 1 Processor -- start 2024-09-21T06:20:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef8ba39740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c41bcd0 0x562a9c814300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef8ba39740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c434490 0x562a9c814840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef8ba39740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c43c670 0x562a9c88bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef8ba39740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fef801ae870 con 0x562a9c43c670 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef8ba39740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef801ae1a0 con 0x562a9c434490 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef8ba39740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fef801ab3b0 con 0x562a9c41bcd0 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef8ba39740 10 client.0 _ll_register_callbacks cb 0x7fef74009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef7f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c41bcd0 0x562a9c814300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef7f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c41bcd0 0x562a9c814300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:53118/0 (socket says 172.21.15.3:53118) 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef7e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c434490 0x562a9c814840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef7f7fe640 1 -- 172.21.15.3:0/795832258 learned_addr learned my addr 172.21.15.3:0/795832258 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/795832258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c43c670 0x562a9c88bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef7f7fe640 1 -- 172.21.15.3:0/795832258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c434490 msgr2=0x562a9c814840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/795832258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c434490 0x562a9c814840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef7f7fe640 1 -- 172.21.15.3:0/795832258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c43c670 msgr2=0x562a9c88bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.077+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/795832258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c43c670 0x562a9c88bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.078+0000 7fef7f7fe640 1 -- 172.21.15.3:0/795832258 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a9c88c080 con 0x562a9c41bcd0 2024-09-21T06:20:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.078+0000 7fef7f7fe640 1 --2- 172.21.15.3:0/795832258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c41bcd0 0x562a9c814300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef7802a900 tx=0x7fef780040e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49a72358d167d33a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.078+0000 7fef62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:20:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.078+0000 7fef62ff5640 1 -- 172.21.15.3:0/795832258 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78003a30 con 0x562a9c41bcd0 2024-09-21T06:20:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.078+0000 7fef62ff5640 1 -- 172.21.15.3:0/795832258 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fef78003bd0 con 0x562a9c41bcd0 2024-09-21T06:20:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.078+0000 7fef8ba39740 1 -- 172.21.15.3:0/795832258 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef740041d0 con 0x562a9c41bcd0 2024-09-21T06:20:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.078+0000 7fef62ff5640 1 -- 172.21.15.3:0/795832258 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fef78003dd0 con 0x562a9c41bcd0 2024-09-21T06:20:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.078+0000 7fef8ba39740 10 client.26665 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.078+0000 7fef8ba39740 1 -- 172.21.15.3:0/795832258 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562a9c88d2a0 con 0x562a9c41bcd0 2024-09-21T06:20:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.078+0000 7fef62ff5640 1 -- 172.21.15.3:0/795832258 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef740041d0 con 0x562a9c41bcd0 2024-09-21T06:20:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.079+0000 7fef62ff5640 1 -- 172.21.15.3:0/795832258 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fef78051040 con 0x562a9c41bcd0 2024-09-21T06:20:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.079+0000 7fef8ba39740 10 client.26665 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.079+0000 7fef8ba39740 10 client.26665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.079+0000 7fef8ba39740 20 client.26665 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.079+0000 7fef8ba39740 1 --2- 172.21.15.3:0/795832258 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562a9c45d350 0x562a9c88f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.079+0000 7fef8ba39740 4 client.26665 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:20:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.079+0000 7fef8ba39740 1 -- 172.21.15.3:0/795832258 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562a9c778860 con 0x562a9c45d350 2024-09-21T06:20:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.079+0000 7fef7e7fc640 1 --2- 172.21.15.3:0/795832258 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562a9c45d350 0x562a9c88f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.079+0000 7fef7e7fc640 1 --2- 172.21.15.3:0/795832258 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562a9c45d350 0x562a9c88f670 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21840faa4465fe7 server_cookie=dfcac606f3f64911 in_seq=0 out_seq=0 2024-09-21T06:20:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.079+0000 7fef62ff5640 10 client.26665 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef62ff5640 1 -- 172.21.15.3:0/795832258 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562a9c778860 con 0x562a9c45d350 2024-09-21T06:20:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef62ff5640 10 client.26665 handle_command_reply: tid=0 2024-09-21T06:20:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef8ba39740 1 client.26665 shutdown 2024-09-21T06:20:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef8ba39740 1 -- 172.21.15.3:0/795832258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c41bcd0 msgr2=0x562a9c814300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef8ba39740 1 --2- 172.21.15.3:0/795832258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c41bcd0 0x562a9c814300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef7802a900 tx=0x7fef780040e0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef8ba39740 1 -- 172.21.15.3:0/795832258 shutdown_connections 2024-09-21T06:20:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef8ba39740 1 --2- 172.21.15.3:0/795832258 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562a9c45d350 0x562a9c88f670 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef8ba39740 1 --2- 172.21.15.3:0/795832258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a9c43c670 0x562a9c88bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef8ba39740 1 --2- 172.21.15.3:0/795832258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9c434490 0x562a9c814840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef8ba39740 1 --2- 172.21.15.3:0/795832258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a9c41bcd0 0x562a9c814300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef8ba39740 1 -- 172.21.15.3:0/795832258 >> 172.21.15.3:0/795832258 conn(0x562a9c451740 msgr2=0x562a9c80e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef62ff5640 0 client.26665 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef8ba39740 1 -- 172.21.15.3:0/795832258 shutdown_connections 2024-09-21T06:20:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.081+0000 7fef8ba39740 1 -- 172.21.15.3:0/795832258 wait complete. 2024-09-21T06:20:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.082+0000 7fef8ba39740 20 client.26665 trim_cache size 0 max 16384 2024-09-21T06:20:30.087 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:20:30.087 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:20:30.087 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "a4c8375d-619d-46de-94b8-b655e2a13724", 2024-09-21T06:20:30.087 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:20:30.087 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:20:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.082+0000 7fef854c5640 1 -- 172.21.15.3:0/4120189663 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fef680b6700 msgr2=0x7fef680b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.082+0000 7fef854c5640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fef680b6700 0x7fef680b8c10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fef700e6fd0 tx=0x7fef700ef550 comp rx=0 tx=0).stop 2024-09-21T06:20:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.082+0000 7fef854c5640 1 -- 172.21.15.3:0/4120189663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef80102a80 msgr2=0x7fef801982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.082+0000 7fef854c5640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef80102a80 0x7fef801982d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fef70010ad0 tx=0x7fef70010f90 comp rx=0 tx=0).stop 2024-09-21T06:20:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.082+0000 7fef854c5640 1 -- 172.21.15.3:0/4120189663 shutdown_connections 2024-09-21T06:20:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.082+0000 7fef854c5640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fef680b6700 0x7fef680b8c10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.082+0000 7fef854c5640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef80108d10 0x7fef8019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.082+0000 7fef854c5640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef801083c0 0x7fef80198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.082+0000 7fef854c5640 1 --2- 172.21.15.3:0/4120189663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef80102a80 0x7fef801982d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.082+0000 7fef854c5640 1 -- 172.21.15.3:0/4120189663 >> 172.21.15.3:0/4120189663 conn(0x7fef80079740 msgr2=0x7fef801043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.082+0000 7fef854c5640 1 -- 172.21.15.3:0/4120189663 shutdown_connections 2024-09-21T06:20:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:30.082+0000 7fef854c5640 1 -- 172.21.15.3:0/4120189663 wait complete. 2024-09-21T06:20:30.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 566045 12.01 MB/sec execute 123 sec latency 36.244 ms 2024-09-21T06:20:31.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:20:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:31 smithi003 ceph-mon[31227]: pgmap v490: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 356 op/s 2024-09-21T06:20:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:31 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:20:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:31 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:31 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:20:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:31 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:20:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:31 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:31 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:20:31.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:31 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:31.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:31 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:31 smithi137 ceph-mon[34464]: pgmap v490: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 356 op/s 2024-09-21T06:20:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:31 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:20:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:31 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:31 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:20:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:31 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:20:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:31 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:31 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:20:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:31 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:31.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:31 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:31 smithi204 ceph-mon[34659]: pgmap v490: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 356 op/s 2024-09-21T06:20:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:31 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:20:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:31 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:31 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:20:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:31 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:20:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:31 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:31 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:20:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:31 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:31.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:31 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.789+0000 7f86f8088640 1 -- 172.21.15.3:0/3353717032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f0102ae0 msgr2=0x7f86f0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.789+0000 7f86f8088640 1 --2- 172.21.15.3:0/3353717032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f0102ae0 0x7f86f0102ef0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009900 tx=0x7f86ec02a410 comp rx=0 tx=0).stop 2024-09-21T06:20:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.790+0000 7f86f8088640 1 -- 172.21.15.3:0/3353717032 shutdown_connections 2024-09-21T06:20:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.790+0000 7f86f8088640 1 --2- 172.21.15.3:0/3353717032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86f01083d0 0x7f86f01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.790+0000 7f86f8088640 1 --2- 172.21.15.3:0/3353717032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f0108d20 0x7f86f010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.790+0000 7f86f8088640 1 --2- 172.21.15.3:0/3353717032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f0102ae0 0x7f86f0102ef0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.790+0000 7f86f8088640 1 -- 172.21.15.3:0/3353717032 >> 172.21.15.3:0/3353717032 conn(0x7f86f00797b0 msgr2=0x7f86f0079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.790+0000 7f86f8088640 1 -- 172.21.15.3:0/3353717032 shutdown_connections 2024-09-21T06:20:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.790+0000 7f86f8088640 1 -- 172.21.15.3:0/3353717032 wait complete. 2024-09-21T06:20:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.790+0000 7f86f8088640 1 Processor -- start 2024-09-21T06:20:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.790+0000 7f86f8088640 1 -- start start 2024-09-21T06:20:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f8088640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f0102ae0 0x7f86f019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f8088640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f01083d0 0x7f86f019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f8088640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86f0108d20 0x7f86f019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f8088640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01129e0 con 0x7f86f01083d0 2024-09-21T06:20:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f8088640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0112860 con 0x7f86f0102ae0 2024-09-21T06:20:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f8088640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0112b60 con 0x7f86f0108d20 2024-09-21T06:20:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f5623640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f01083d0 0x7f86f019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f5e24640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f0102ae0 0x7f86f019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f5623640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f01083d0 0x7f86f019b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48094/0 (socket says 172.21.15.3:48094) 2024-09-21T06:20:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f5623640 1 -- 172.21.15.3:0/1058514601 learned_addr learned my addr 172.21.15.3:0/1058514601 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f6625640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86f0108d20 0x7f86f019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f5623640 1 -- 172.21.15.3:0/1058514601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86f0108d20 msgr2=0x7f86f019bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f5623640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86f0108d20 0x7f86f019bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f5623640 1 -- 172.21.15.3:0/1058514601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f0102ae0 msgr2=0x7f86f019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f5623640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f0102ae0 0x7f86f019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f5623640 1 -- 172.21.15.3:0/1058514601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f01a65c0 con 0x7f86f01083d0 2024-09-21T06:20:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.791+0000 7f86f5e24640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f0102ae0 0x7f86f019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.792+0000 7f86f5623640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f01083d0 0x7f86f019b870 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f86e000e9f0 tx=0x7f86e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.792+0000 7f86e6ffd640 1 -- 172.21.15.3:0/1058514601 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e000ccc0 con 0x7f86f01083d0 2024-09-21T06:20:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.792+0000 7f86f8088640 1 -- 172.21.15.3:0/1058514601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f0199180 con 0x7f86f01083d0 2024-09-21T06:20:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.792+0000 7f86e6ffd640 1 -- 172.21.15.3:0/1058514601 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e00108e0 con 0x7f86f01083d0 2024-09-21T06:20:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.792+0000 7f86e6ffd640 1 -- 172.21.15.3:0/1058514601 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0018860 con 0x7f86f01083d0 2024-09-21T06:20:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.792+0000 7f86f8088640 1 -- 172.21.15.3:0/1058514601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f0199710 con 0x7f86f01083d0 2024-09-21T06:20:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.794+0000 7f86fe5fc740 1 Processor -- start 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86fe5fc740 1 -- start start 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86fe5fc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f6a4070 0x55b15fa4b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86e6ffd640 1 -- 172.21.15.3:0/1058514601 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f86e0010430 con 0x7f86f01083d0 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86fe5fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f67b1b0 0x55b15fa4bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86fe5fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f683390 0x55b15fa4c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86fe5fc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b15f9c1e50 con 0x55b15f6a4070 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86fe5fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b15f9b1280 con 0x55b15f67b1b0 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86fe5fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b15f9b5090 con 0x55b15f683390 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86f6625640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f6a4070 0x55b15fa4b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86f5e24640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f67b1b0 0x55b15fa4bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86f5623640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f683390 0x55b15fa4c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86f6625640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f6a4070 0x55b15fa4b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48098/0 (socket says 172.21.15.3:48098) 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86f6625640 1 -- 172.21.15.3:0/744061502 learned_addr learned my addr 172.21.15.3:0/744061502 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86f5e24640 1 -- 172.21.15.3:0/744061502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f683390 msgr2=0x55b15fa4c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86f5e24640 1 --2- 172.21.15.3:0/744061502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f683390 0x55b15fa4c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86f5e24640 1 -- 172.21.15.3:0/744061502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f6a4070 msgr2=0x55b15fa4b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.795+0000 7f86f5e24640 1 --2- 172.21.15.3:0/744061502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f6a4070 0x55b15fa4b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.796+0000 7f86f5e24640 1 -- 172.21.15.3:0/744061502 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b15f70f650 con 0x55b15f67b1b0 2024-09-21T06:20:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.796+0000 7f86f5e24640 1 --2- 172.21.15.3:0/744061502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f67b1b0 0x55b15fa4bb90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f86f006c380 tx=0x7f86ec032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efc0c49072528b55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.796+0000 7f86dbfff640 1 -- 172.21.15.3:0/744061502 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec035050 con 0x55b15f67b1b0 2024-09-21T06:20:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.796+0000 7f86dbfff640 1 -- 172.21.15.3:0/744061502 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec0316a0 con 0x55b15f67b1b0 2024-09-21T06:20:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.796+0000 7f86dbfff640 1 -- 172.21.15.3:0/744061502 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec004770 con 0x55b15f67b1b0 2024-09-21T06:20:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.796+0000 7f86e6ffd640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86dc0b66b0 0x7f86dc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.796+0000 7f86f6625640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86dc0b66b0 0x7f86dc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.797+0000 7f86e6ffd640 1 -- 172.21.15.3:0/1058514601 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f86e00e7980 con 0x7f86f01083d0 2024-09-21T06:20:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.797+0000 7f86fe5fc740 1 -- 172.21.15.3:0/744061502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f67b1b0 msgr2=0x55b15fa4bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.797+0000 7f86f6625640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86dc0b66b0 0x7f86dc0b8bc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f86e8011630 tx=0x7f86e8009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.797+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/744061502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f67b1b0 0x55b15fa4bb90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f86f006c380 tx=0x7f86ec032f90 comp rx=0 tx=0).stop 2024-09-21T06:20:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.797+0000 7f86fe5fc740 1 -- 172.21.15.3:0/744061502 shutdown_connections 2024-09-21T06:20:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.797+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/744061502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f683390 0x55b15fa4c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.797+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/744061502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f67b1b0 0x55b15fa4bb90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.797+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/744061502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f6a4070 0x55b15fa4b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.797+0000 7f86fe5fc740 1 -- 172.21.15.3:0/744061502 >> 172.21.15.3:0/744061502 conn(0x55b15f698460 msgr2=0x55b15fa49540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.797+0000 7f86fe5fc740 1 -- 172.21.15.3:0/744061502 shutdown_connections 2024-09-21T06:20:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.797+0000 7f86fe5fc740 1 -- 172.21.15.3:0/744061502 wait complete. 2024-09-21T06:20:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.798+0000 7f86fe5fc740 1 -- start start 2024-09-21T06:20:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.798+0000 7f86fe5fc740 1 Processor -- start 2024-09-21T06:20:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.798+0000 7f86fe5fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f67b1b0 0x55b15fa56d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.798+0000 7f86fe5fc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f683390 0x55b15fa57250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.798+0000 7f86fe5fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f6a4070 0x55b15fa630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.798+0000 7f86fe5fc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b15f812700 con 0x55b15f683390 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.798+0000 7f86fe5fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b15f94c110 con 0x55b15f67b1b0 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.798+0000 7f86fe5fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b15fa59dd0 con 0x55b15f6a4070 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86f5e24640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f67b1b0 0x55b15fa56d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86f5e24640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f67b1b0 0x55b15fa56d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:32904/0 (socket says 172.21.15.3:32904) 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86f5e24640 1 -- 172.21.15.3:0/945794146 learned_addr learned my addr 172.21.15.3:0/945794146 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86fe5fc740 10 client.0 _ll_register_callbacks cb 0x55b15f965990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86f5623640 1 --2- 172.21.15.3:0/945794146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f683390 0x55b15fa57250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86f6625640 1 --2- 172.21.15.3:0/945794146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f6a4070 0x55b15fa630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86f5e24640 1 -- 172.21.15.3:0/945794146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f6a4070 msgr2=0x55b15fa630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86f5e24640 1 --2- 172.21.15.3:0/945794146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f6a4070 0x55b15fa630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86f5e24640 1 -- 172.21.15.3:0/945794146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f683390 msgr2=0x55b15fa57250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86f5e24640 1 --2- 172.21.15.3:0/945794146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f683390 0x55b15fa57250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86f5e24640 1 -- 172.21.15.3:0/945794146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b15f979c40 con 0x55b15f67b1b0 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86f5e24640 1 --2- 172.21.15.3:0/945794146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f67b1b0 0x55b15fa56d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0045b0 tx=0x7f86ec0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=891cd87c6520a1ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.799+0000 7f86d97fa640 1 -- 172.21.15.3:0/945794146 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec005520 con 0x55b15f67b1b0 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86fe5fc740 1 -- 172.21.15.3:0/945794146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ec0316a0 con 0x55b15f67b1b0 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86d97fa640 1 -- 172.21.15.3:0/945794146 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f86ec03a670 con 0x55b15f67b1b0 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86d97fa640 1 -- 172.21.15.3:0/945794146 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f86ec035020 con 0x55b15f67b1b0 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86fe5fc740 10 client.26735 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86fe5fc740 1 -- 172.21.15.3:0/945794146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b15fa65a60 con 0x55b15f67b1b0 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86d97fa640 1 -- 172.21.15.3:0/945794146 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec0316a0 con 0x55b15f67b1b0 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86d97fa640 1 -- 172.21.15.3:0/945794146 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f86ec003a50 con 0x55b15f67b1b0 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86fe5fc740 10 client.26735 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86fe5fc740 10 client.26735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86fe5fc740 20 client.26735 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/945794146 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b15f6629f0 0x55b15fa68dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86fe5fc740 4 client.26735 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86fe5fc740 1 -- 172.21.15.3:0/945794146 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b15f9c3da0 con 0x55b15f6629f0 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.800+0000 7f86f5623640 1 --2- 172.21.15.3:0/945794146 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b15f6629f0 0x55b15fa68dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.801+0000 7f86f5623640 1 --2- 172.21.15.3:0/945794146 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b15f6629f0 0x55b15fa68dc0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49944da4289bb62 server_cookie=2b9f26d00cde582b in_seq=0 out_seq=0 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.801+0000 7f86d97fa640 10 client.26735 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.802+0000 7f86d97fa640 1 -- 172.21.15.3:0/945794146 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b15f9c3da0 con 0x55b15f6629f0 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.802+0000 7f86d97fa640 10 client.26735 handle_command_reply: tid=0 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.802+0000 7f86fe5fc740 1 client.26735 shutdown 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.802+0000 7f86fe5fc740 1 -- 172.21.15.3:0/945794146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f67b1b0 msgr2=0x55b15fa56d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.802+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/945794146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f67b1b0 0x55b15fa56d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0045b0 tx=0x7f86ec0045e0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.802+0000 7f86fe5fc740 1 -- 172.21.15.3:0/945794146 shutdown_connections 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.802+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/945794146 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b15f6629f0 0x55b15fa68dc0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.802+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/945794146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f6a4070 0x55b15fa630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.802+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/945794146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f683390 0x55b15fa57250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.802+0000 7f86d97fa640 0 client.26735 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.802+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/945794146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f67b1b0 0x55b15fa56d10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.802+0000 7f86fe5fc740 1 -- 172.21.15.3:0/945794146 >> 172.21.15.3:0/945794146 conn(0x55b15f698460 msgr2=0x55b15fa4e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.803+0000 7f86fe5fc740 1 -- 172.21.15.3:0/945794146 shutdown_connections 2024-09-21T06:20:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.803+0000 7f86fe5fc740 1 -- 172.21.15.3:0/945794146 wait complete. 2024-09-21T06:20:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.803+0000 7f86fe5fc740 20 client.26735 trim_cache size 0 max 16384 2024-09-21T06:20:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.822+0000 7f86fe5fc740 1 Processor -- start 2024-09-21T06:20:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.822+0000 7f86fe5fc740 1 -- start start 2024-09-21T06:20:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.822+0000 7f86fe5fc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f67b1b0 0x55b15fa571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.822+0000 7f86fe5fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f683390 0x55b15fa63030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.822+0000 7f86fe5fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f6a4070 0x55b15fa53f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.822+0000 7f86fe5fc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01ad510 con 0x55b15f67b1b0 2024-09-21T06:20:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.822+0000 7f86fe5fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01af080 con 0x55b15f6a4070 2024-09-21T06:20:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.822+0000 7f86fe5fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01adbe0 con 0x55b15f683390 2024-09-21T06:20:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86f5e24640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f67b1b0 0x55b15fa571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86f5e24640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f67b1b0 0x55b15fa571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48114/0 (socket says 172.21.15.3:48114) 2024-09-21T06:20:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86f5623640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f683390 0x55b15fa63030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86f5e24640 1 -- 172.21.15.3:0/3474753959 learned_addr learned my addr 172.21.15.3:0/3474753959 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86f6625640 1 --2- 172.21.15.3:0/3474753959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f6a4070 0x55b15fa53f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86f5623640 1 -- 172.21.15.3:0/3474753959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f6a4070 msgr2=0x55b15fa53f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86f5623640 1 --2- 172.21.15.3:0/3474753959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f6a4070 0x55b15fa53f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86f5623640 1 -- 172.21.15.3:0/3474753959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f67b1b0 msgr2=0x55b15fa571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86f5623640 1 --2- 172.21.15.3:0/3474753959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f67b1b0 0x55b15fa571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86f5623640 1 -- 172.21.15.3:0/3474753959 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b15fa59450 con 0x55b15f683390 2024-09-21T06:20:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86f5e24640 1 --2- 172.21.15.3:0/3474753959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f67b1b0 0x55b15fa571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86f5623640 1 --2- 172.21.15.3:0/3474753959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f683390 0x55b15fa63030 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86e00e5110 tx=0x7f86e000cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a080a1ca37fbd437 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86db7fe640 1 -- 172.21.15.3:0/3474753959 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e00ad210 con 0x55b15f683390 2024-09-21T06:20:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86db7fe640 1 -- 172.21.15.3:0/3474753959 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e0010720 con 0x55b15f683390 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.823+0000 7f86db7fe640 1 -- 172.21.15.3:0/3474753959 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e00ad030 con 0x55b15f683390 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.824+0000 7f86fe5fc740 1 -- 172.21.15.3:0/3474753959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f683390 msgr2=0x55b15fa63030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.824+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/3474753959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f683390 0x55b15fa63030 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86e00e5110 tx=0x7f86e000cef0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.824+0000 7f86fe5fc740 1 -- 172.21.15.3:0/3474753959 shutdown_connections 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.824+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/3474753959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f683390 0x55b15fa63030 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.824+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/3474753959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f6a4070 0x55b15fa53f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.824+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/3474753959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f67b1b0 0x55b15fa571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.824+0000 7f86fe5fc740 1 -- 172.21.15.3:0/3474753959 >> 172.21.15.3:0/3474753959 conn(0x55b15f698460 msgr2=0x55b15fa56ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.824+0000 7f86fe5fc740 1 -- 172.21.15.3:0/3474753959 shutdown_connections 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.824+0000 7f86fe5fc740 1 -- 172.21.15.3:0/3474753959 wait complete. 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.825+0000 7f86fe5fc740 1 -- start start 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.825+0000 7f86fe5fc740 1 Processor -- start 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.825+0000 7f86fe5fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f67b1b0 0x55b15fa6a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.825+0000 7f86fe5fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f683390 0x55b15fa6a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.825+0000 7f86fe5fc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f6a4070 0x55b15fad2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.825+0000 7f86fe5fc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01ae9b0 con 0x55b15f6a4070 2024-09-21T06:20:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.825+0000 7f86fe5fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01ae2e0 con 0x55b15f683390 2024-09-21T06:20:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.825+0000 7f86fe5fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01ab4f0 con 0x55b15f67b1b0 2024-09-21T06:20:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.825+0000 7f86fe5fc740 10 client.0 _ll_register_callbacks cb 0x7f86e8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86f5623640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f683390 0x55b15fa6a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86f6625640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f6a4070 0x55b15fad2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86f5623640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f683390 0x55b15fa6a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:32928/0 (socket says 172.21.15.3:32928) 2024-09-21T06:20:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86f5623640 1 -- 172.21.15.3:0/64899249 learned_addr learned my addr 172.21.15.3:0/64899249 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86f5e24640 1 --2- 172.21.15.3:0/64899249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f67b1b0 0x55b15fa6a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86f6625640 1 -- 172.21.15.3:0/64899249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f67b1b0 msgr2=0x55b15fa6a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86f6625640 1 --2- 172.21.15.3:0/64899249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f67b1b0 0x55b15fa6a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86f6625640 1 -- 172.21.15.3:0/64899249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f683390 msgr2=0x55b15fa6a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86f6625640 1 --2- 172.21.15.3:0/64899249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f683390 0x55b15fa6a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86f6625640 1 -- 172.21.15.3:0/64899249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b15fad2a80 con 0x55b15f6a4070 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86f5623640 1 --2- 172.21.15.3:0/64899249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f683390 0x55b15fa6a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86f6625640 1 --2- 172.21.15.3:0/64899249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f6a4070 0x55b15fad2540 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x55b15fa44f00 tx=0x7f86e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3add3b2e393ca69f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86d97fa640 1 -- 172.21.15.3:0/64899249 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e802e060 con 0x55b15f6a4070 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86d97fa640 1 -- 172.21.15.3:0/64899249 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f86e800f9f0 con 0x55b15f6a4070 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.826+0000 7f86fe5fc740 1 -- 172.21.15.3:0/64899249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b15f9bf850 con 0x55b15f6a4070 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.827+0000 7f86d97fa640 1 -- 172.21.15.3:0/64899249 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f86e800fbf0 con 0x55b15f6a4070 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.827+0000 7f86fe5fc740 10 client.17169 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.827+0000 7f86fe5fc740 1 -- 172.21.15.3:0/64899249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b15fad3be0 con 0x55b15f6a4070 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.827+0000 7f86d97fa640 1 -- 172.21.15.3:0/64899249 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b15f9bf850 con 0x55b15f6a4070 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.827+0000 7f86d97fa640 1 -- 172.21.15.3:0/64899249 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f86e801d420 con 0x55b15f6a4070 2024-09-21T06:20:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.827+0000 7f86fe5fc740 10 client.17169 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.827+0000 7f86fe5fc740 10 client.17169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.827+0000 7f86fe5fc740 20 client.17169 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.827+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/64899249 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b15f637da0 0x55b15fad63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.827+0000 7f86fe5fc740 4 client.17169 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:20:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.827+0000 7f86fe5fc740 1 -- 172.21.15.3:0/64899249 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b15f9bcc80 con 0x55b15f637da0 2024-09-21T06:20:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.827+0000 7f86f5e24640 1 --2- 172.21.15.3:0/64899249 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b15f637da0 0x55b15fad63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.828+0000 7f86d97fa640 10 client.17169 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.828+0000 7f86f5e24640 1 --2- 172.21.15.3:0/64899249 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b15f637da0 0x55b15fad63d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9941cb46f80c8045 server_cookie=e85948b05a04d357 in_seq=0 out_seq=0 2024-09-21T06:20:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.828+0000 7f86d97fa640 1 -- 172.21.15.3:0/64899249 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x55b15f9bcc80 con 0x55b15f637da0 2024-09-21T06:20:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.828+0000 7f86d97fa640 10 client.17169 handle_command_reply: tid=0 2024-09-21T06:20:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.828+0000 7f86fe5fc740 1 client.17169 shutdown 2024-09-21T06:20:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.828+0000 7f86fe5fc740 1 -- 172.21.15.3:0/64899249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f6a4070 msgr2=0x55b15fad2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.829+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/64899249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f6a4070 0x55b15fad2540 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x55b15fa44f00 tx=0x7f86e8004a40 comp rx=0 tx=0).stop 2024-09-21T06:20:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.829+0000 7f86fe5fc740 1 -- 172.21.15.3:0/64899249 shutdown_connections 2024-09-21T06:20:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.829+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/64899249 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b15f637da0 0x55b15fad63d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.829+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/64899249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b15f6a4070 0x55b15fad2540 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.829+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/64899249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15f683390 0x55b15fa6a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.829+0000 7f86fe5fc740 1 --2- 172.21.15.3:0/64899249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b15f67b1b0 0x55b15fa6a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.829+0000 7f86fe5fc740 1 -- 172.21.15.3:0/64899249 >> 172.21.15.3:0/64899249 conn(0x55b15f698460 msgr2=0x55b15fa4c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.829+0000 7f86d97fa640 0 client.17169 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.829+0000 7f86fe5fc740 1 -- 172.21.15.3:0/64899249 shutdown_connections 2024-09-21T06:20:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.829+0000 7f86fe5fc740 1 -- 172.21.15.3:0/64899249 wait complete. 2024-09-21T06:20:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.829+0000 7f86fe5fc740 20 client.17169 trim_cache size 0 max 16384 2024-09-21T06:20:31.841 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:20:31.842 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:20:31.842 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-21T06:20:31.842 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:20:31.842 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:31.842 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "a4c8375d-619d-46de-94b8-b655e2a13724", 2024-09-21T06:20:31.842 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-21T06:20:31.842 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:20:31.842 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:31.842 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:20:31.842 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:20:31.842 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:20:31.842 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:31.842 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:31.843 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:31.843 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:31.843 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:31.843 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:31.843 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:20:31.843 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:20:31.843 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:31.843 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:31.843 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:31.843 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:31.843 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:31.843 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:31.844 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:20:31.844 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:20:31.844 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:31.844 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:31.844 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:31.844 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:31.844 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:31.844 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:31.844 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:20:31.844 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:20:31.844 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:31.844 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:20:31.845 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:31.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:20:31.847 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:20:31.848 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:20:31.848 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:20:31.848 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:20:31.848 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:20:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.830+0000 7f86f8088640 1 -- 172.21.15.3:0/1058514601 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86dc0b66b0 msgr2=0x7f86dc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.830+0000 7f86f8088640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86dc0b66b0 0x7f86dc0b8bc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f86e8011630 tx=0x7f86e8009500 comp rx=0 tx=0).stop 2024-09-21T06:20:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.830+0000 7f86f8088640 1 -- 172.21.15.3:0/1058514601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f01083d0 msgr2=0x7f86f019b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.830+0000 7f86f8088640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f01083d0 0x7f86f019b870 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f86e000e9f0 tx=0x7f86e000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.830+0000 7f86f8088640 1 -- 172.21.15.3:0/1058514601 shutdown_connections 2024-09-21T06:20:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.830+0000 7f86f8088640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86dc0b66b0 0x7f86dc0b8bc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.830+0000 7f86f8088640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86f0108d20 0x7f86f019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.830+0000 7f86f8088640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f01083d0 0x7f86f019b870 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.830+0000 7f86f8088640 1 --2- 172.21.15.3:0/1058514601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f0102ae0 0x7f86f019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.830+0000 7f86f8088640 1 -- 172.21.15.3:0/1058514601 >> 172.21.15.3:0/1058514601 conn(0x7f86f00797b0 msgr2=0x7f86f01064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.830+0000 7f86f8088640 1 -- 172.21.15.3:0/1058514601 shutdown_connections 2024-09-21T06:20:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:20:31.830+0000 7f86f8088640 1 -- 172.21.15.3:0/1058514601 wait complete. 2024-09-21T06:20:31.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 568487 12.04 MB/sec execute 124 sec latency 18.924 ms 2024-09-21T06:20:32.585 INFO:tasks.cephfs.filesystem:scrub status for tag:a4c8375d-619d-46de-94b8-b655e2a13724 - {'path': '/', 'tag': 'a4c8375d-619d-46de-94b8-b655e2a13724', 'options': 'recursive,force'} 2024-09-21T06:20:32.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 570036 11.99 MB/sec execute 125 sec latency 15.891 ms 2024-09-21T06:20:33.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:33 smithi003 ceph-mon[31227]: pgmap v491: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 306 op/s 2024-09-21T06:20:33.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:33 smithi137 ceph-mon[34464]: pgmap v491: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 306 op/s 2024-09-21T06:20:33.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:33 smithi204 ceph-mon[34659]: pgmap v491: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 306 op/s 2024-09-21T06:20:33.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 572247 12.01 MB/sec execute 126 sec latency 85.411 ms 2024-09-21T06:20:34.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 574088 11.96 MB/sec execute 127 sec latency 20.177 ms 2024-09-21T06:20:35.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:35 smithi003 ceph-mon[31227]: pgmap v492: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-21T06:20:35.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:35 smithi137 ceph-mon[34464]: pgmap v492: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-21T06:20:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:35 smithi204 ceph-mon[34659]: pgmap v492: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-21T06:20:35.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 576027 11.97 MB/sec execute 128 sec latency 11.294 ms 2024-09-21T06:20:36.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:36 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:20:36.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:36 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:20:36.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:36 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:20:36.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 578335 11.97 MB/sec execute 129 sec latency 20.972 ms 2024-09-21T06:20:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.994+0000 7fcc6ad8b640 1 -- 172.21.15.3:0/140445859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc64108c60 msgr2=0x7fcc6410f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.994+0000 7fcc6ad8b640 1 --2- 172.21.15.3:0/140445859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc64108c60 0x7fcc6410f490 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009ca0 tx=0x7fcc5c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:20:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.994+0000 7fcc6ad8b640 1 -- 172.21.15.3:0/140445859 shutdown_connections 2024-09-21T06:20:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.994+0000 7fcc6ad8b640 1 --2- 172.21.15.3:0/140445859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc64108c60 0x7fcc6410f490 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.994+0000 7fcc6ad8b640 1 --2- 172.21.15.3:0/140445859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc64108310 0x7fcc64108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.994+0000 7fcc6ad8b640 1 --2- 172.21.15.3:0/140445859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc641029e0 0x7fcc64102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.994+0000 7fcc6ad8b640 1 -- 172.21.15.3:0/140445859 >> 172.21.15.3:0/140445859 conn(0x7fcc640fe3c0 msgr2=0x7fcc64100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.994+0000 7fcc6ad8b640 1 -- 172.21.15.3:0/140445859 shutdown_connections 2024-09-21T06:20:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.994+0000 7fcc6ad8b640 1 -- 172.21.15.3:0/140445859 wait complete. 2024-09-21T06:20:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.995+0000 7fcc6ad8b640 1 Processor -- start 2024-09-21T06:20:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.995+0000 7fcc6ad8b640 1 -- start start 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.995+0000 7fcc6ad8b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc641029e0 0x7fcc64199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.995+0000 7fcc6ad8b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc64108310 0x7fcc641998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.995+0000 7fcc6ad8b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc64108c60 0x7fcc6419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.995+0000 7fcc6ad8b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc64112920 con 0x7fcc64108c60 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.995+0000 7fcc6ad8b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641127a0 con 0x7fcc641029e0 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.995+0000 7fcc6ad8b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc64112aa0 con 0x7fcc64108310 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.995+0000 7fcc68b27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc641029e0 0x7fcc64199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.995+0000 7fcc69328640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc64108c60 0x7fcc6419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.995+0000 7fcc68b27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc641029e0 0x7fcc64199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:38744/0 (socket says 172.21.15.3:38744) 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.995+0000 7fcc68b27640 1 -- 172.21.15.3:0/2066263030 learned_addr learned my addr 172.21.15.3:0/2066263030 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.995+0000 7fcc63fff640 1 --2- 172.21.15.3:0/2066263030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc64108310 0x7fcc641998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.996+0000 7fcc68b27640 1 -- 172.21.15.3:0/2066263030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc64108310 msgr2=0x7fcc641998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.996+0000 7fcc68b27640 1 --2- 172.21.15.3:0/2066263030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc64108310 0x7fcc641998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.996+0000 7fcc68b27640 1 -- 172.21.15.3:0/2066263030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc64108c60 msgr2=0x7fcc6419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.996+0000 7fcc68b27640 1 --2- 172.21.15.3:0/2066263030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc64108c60 0x7fcc6419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.996+0000 7fcc68b27640 1 -- 172.21.15.3:0/2066263030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6419f230 con 0x7fcc641029e0 2024-09-21T06:20:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.996+0000 7fcc68b27640 1 --2- 172.21.15.3:0/2066263030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc641029e0 0x7fcc64199300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800e9f0 tx=0x7fcc5800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.996+0000 7fcc61ffb640 1 -- 172.21.15.3:0/2066263030 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800cd80 con 0x7fcc641029e0 2024-09-21T06:20:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.996+0000 7fcc61ffb640 1 -- 172.21.15.3:0/2066263030 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc580108e0 con 0x7fcc641029e0 2024-09-21T06:20:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.996+0000 7fcc6ad8b640 1 -- 172.21.15.3:0/2066263030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6419f520 con 0x7fcc641029e0 2024-09-21T06:20:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.996+0000 7fcc61ffb640 1 -- 172.21.15.3:0/2066263030 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58018980 con 0x7fcc641029e0 2024-09-21T06:20:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.997+0000 7fcc6ad8b640 1 -- 172.21.15.3:0/2066263030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc641a5cb0 con 0x7fcc641029e0 2024-09-21T06:20:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.999+0000 7fcc71dd7740 1 Processor -- start 2024-09-21T06:20:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.999+0000 7fcc71dd7740 1 -- start start 2024-09-21T06:20:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.999+0000 7fcc61ffb640 1 -- 172.21.15.3:0/2066263030 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fcc58010430 con 0x7fcc641029e0 2024-09-21T06:20:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.999+0000 7fcc71dd7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afdeab1b0 0x561afe2509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.999+0000 7fcc71dd7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe250ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.999+0000 7fcc71dd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 0x561afe251430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.999+0000 7fcc71dd7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561afe1c56f0 con 0x561afde822f0 2024-09-21T06:20:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.999+0000 7fcc71dd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561afe1b4b20 con 0x561afde8a4d0 2024-09-21T06:20:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:36.999+0000 7fcc71dd7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561afe1b8930 con 0x561afdeab1b0 2024-09-21T06:20:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc68b27640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe250ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc69328640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afdeab1b0 0x561afe2509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 0x561afe251430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc68b27640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe250ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37722/0 (socket says 172.21.15.3:37722) 2024-09-21T06:20:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc69328640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afdeab1b0 0x561afe2509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:35994/0 (socket says 172.21.15.3:35994) 2024-09-21T06:20:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc68b27640 1 -- 172.21.15.3:0/3604620838 learned_addr learned my addr 172.21.15.3:0/3604620838 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc68b27640 1 -- 172.21.15.3:0/3604620838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afdeab1b0 msgr2=0x561afe2509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc68b27640 1 --2- 172.21.15.3:0/3604620838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afdeab1b0 0x561afe2509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc68b27640 1 -- 172.21.15.3:0/3604620838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 msgr2=0x561afe251430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc68b27640 1 --2- 172.21.15.3:0/3604620838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 0x561afe251430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc68b27640 1 -- 172.21.15.3:0/3604620838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afe17a530 con 0x561afde822f0 2024-09-21T06:20:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc69328640 1 --2- 172.21.15.3:0/3604620838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afdeab1b0 0x561afe2509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc68b27640 1 --2- 172.21.15.3:0/3604620838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe250ef0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcc580e5090 tx=0x7fcc580ebf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383a1f41fecc8c0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc61ffb640 1 --2- 172.21.15.3:0/2066263030 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc540b6700 0x7fcc540b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc4effd640 1 -- 172.21.15.3:0/3604620838 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc580fb070 con 0x561afde822f0 2024-09-21T06:20:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.000+0000 7fcc69328640 1 --2- 172.21.15.3:0/2066263030 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc540b6700 0x7fcc540b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.001+0000 7fcc61ffb640 1 -- 172.21.15.3:0/2066263030 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcc580e7990 con 0x7fcc641029e0 2024-09-21T06:20:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.001+0000 7fcc4effd640 1 -- 172.21.15.3:0/3604620838 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc58010720 con 0x561afde822f0 2024-09-21T06:20:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.001+0000 7fcc4effd640 1 -- 172.21.15.3:0/3604620838 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc580f2040 con 0x561afde822f0 2024-09-21T06:20:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.001+0000 7fcc69328640 1 --2- 172.21.15.3:0/2066263030 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc540b6700 0x7fcc540b8c10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c02f500 tx=0x7fcc5c02f490 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.001+0000 7fcc71dd7740 1 -- 172.21.15.3:0/3604620838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 msgr2=0x561afe250ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.001+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/3604620838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe250ef0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcc580e5090 tx=0x7fcc580ebf80 comp rx=0 tx=0).stop 2024-09-21T06:20:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.001+0000 7fcc71dd7740 1 -- 172.21.15.3:0/3604620838 shutdown_connections 2024-09-21T06:20:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.001+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/3604620838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 0x561afe251430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.001+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/3604620838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe250ef0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.002+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/3604620838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afdeab1b0 0x561afe2509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.002+0000 7fcc71dd7740 1 -- 172.21.15.3:0/3604620838 >> 172.21.15.3:0/3604620838 conn(0x561afde9f300 msgr2=0x561afe24e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.002+0000 7fcc71dd7740 1 -- 172.21.15.3:0/3604620838 shutdown_connections 2024-09-21T06:20:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.002+0000 7fcc71dd7740 1 -- 172.21.15.3:0/3604620838 wait complete. 2024-09-21T06:20:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.002+0000 7fcc71dd7740 1 -- start start 2024-09-21T06:20:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.002+0000 7fcc71dd7740 1 Processor -- start 2024-09-21T06:20:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc71dd7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe25c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc71dd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 0x561afe25c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc71dd7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afdeab1b0 0x561afe268420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc71dd7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561afe0ecab0 con 0x561afde822f0 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc71dd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561afe15a730 con 0x561afde8a4d0 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc71dd7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561afe25f0d0 con 0x561afdeab1b0 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc71dd7740 10 client.0 _ll_register_callbacks cb 0x561afe1ae6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc68b27640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe25c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc68b27640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe25c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37734/0 (socket says 172.21.15.3:37734) 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc68b27640 1 -- 172.21.15.3:0/1380999700 learned_addr learned my addr 172.21.15.3:0/1380999700 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc63fff640 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 0x561afe25c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc69328640 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afdeab1b0 0x561afe268420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc68b27640 1 -- 172.21.15.3:0/1380999700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afdeab1b0 msgr2=0x561afe268420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc68b27640 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afdeab1b0 0x561afe268420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc68b27640 1 -- 172.21.15.3:0/1380999700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 msgr2=0x561afe25c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc68b27640 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 0x561afe25c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc68b27640 1 -- 172.21.15.3:0/1380999700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afdf16790 con 0x561afde822f0 2024-09-21T06:20:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc63fff640 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 0x561afe25c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc68b27640 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe25c070 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcc580e7cb0 tx=0x7fcc58018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3df08d5a00093e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc43fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc43fff640 1 -- 172.21.15.3:0/1380999700 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc580f2030 con 0x561afde822f0 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc43fff640 1 -- 172.21.15.3:0/1380999700 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcc580b6db0 con 0x561afde822f0 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.003+0000 7fcc71dd7740 1 -- 172.21.15.3:0/1380999700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561afe1c7640 con 0x561afde822f0 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.004+0000 7fcc43fff640 1 -- 172.21.15.3:0/1380999700 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcc580ad6a0 con 0x561afde822f0 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.004+0000 7fcc71dd7740 10 client.17181 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.004+0000 7fcc71dd7740 1 -- 172.21.15.3:0/1380999700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561afdf8f620 con 0x561afde822f0 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.004+0000 7fcc43fff640 1 -- 172.21.15.3:0/1380999700 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561afe1c7640 con 0x561afde822f0 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.004+0000 7fcc43fff640 1 -- 172.21.15.3:0/1380999700 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fcc58018b20 con 0x561afde822f0 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.004+0000 7fcc71dd7740 10 client.17181 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.004+0000 7fcc71dd7740 10 client.17181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.004+0000 7fcc71dd7740 20 client.17181 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.004+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561afde69d80 0x561afe26dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.004+0000 7fcc71dd7740 4 client.17181 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.004+0000 7fcc71dd7740 1 -- 172.21.15.3:0/1380999700 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561afe1c30f0 con 0x561afde69d80 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.004+0000 7fcc63fff640 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561afde69d80 0x561afe26dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.005+0000 7fcc43fff640 10 client.17181 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.005+0000 7fcc63fff640 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561afde69d80 0x561afe26dfe0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59fe2c78943be73d server_cookie=4923cc51f5e85753 in_seq=0 out_seq=0 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc43fff640 1 -- 172.21.15.3:0/1380999700 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561afe1c30f0 con 0x561afde69d80 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc43fff640 10 client.17181 handle_command_reply: tid=0 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc71dd7740 1 client.17181 shutdown 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc71dd7740 1 -- 172.21.15.3:0/1380999700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 msgr2=0x561afe25c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe25c070 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcc580e7cb0 tx=0x7fcc58018420 comp rx=0 tx=0).stop 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc71dd7740 1 -- 172.21.15.3:0/1380999700 shutdown_connections 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561afde69d80 0x561afe26dfe0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afdeab1b0 0x561afe268420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc43fff640 0 client.17181 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 0x561afe25c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/1380999700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe25c070 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc71dd7740 1 -- 172.21.15.3:0/1380999700 >> 172.21.15.3:0/1380999700 conn(0x561afde9f300 msgr2=0x561afe2519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc71dd7740 1 -- 172.21.15.3:0/1380999700 shutdown_connections 2024-09-21T06:20:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc71dd7740 1 -- 172.21.15.3:0/1380999700 wait complete. 2024-09-21T06:20:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.006+0000 7fcc71dd7740 20 client.17181 trim_cache size 0 max 16384 2024-09-21T06:20:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.022+0000 7fcc71dd7740 1 Processor -- start 2024-09-21T06:20:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.022+0000 7fcc71dd7740 1 -- start start 2024-09-21T06:20:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.022+0000 7fcc71dd7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afde822f0 0x561afe2520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.022+0000 7fcc71dd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 0x561afe268390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.022+0000 7fcc71dd7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afdeab1b0 0x561afe271200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.022+0000 7fcc71dd7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641ae4a0 con 0x561afdeab1b0 2024-09-21T06:20:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.022+0000 7fcc71dd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641b0010 con 0x561afde8a4d0 2024-09-21T06:20:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.022+0000 7fcc71dd7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641aeb70 con 0x561afde822f0 2024-09-21T06:20:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.022+0000 7fcc69328640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afdeab1b0 0x561afe271200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.022+0000 7fcc69328640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afdeab1b0 0x561afe271200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37748/0 (socket says 172.21.15.3:37748) 2024-09-21T06:20:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.022+0000 7fcc68b27640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afde822f0 0x561afe2520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.022+0000 7fcc69328640 1 -- 172.21.15.3:0/2057473342 learned_addr learned my addr 172.21.15.3:0/2057473342 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.022+0000 7fcc63fff640 1 --2- 172.21.15.3:0/2057473342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 0x561afe268390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.023+0000 7fcc69328640 1 -- 172.21.15.3:0/2057473342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afde822f0 msgr2=0x561afe2520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.023+0000 7fcc69328640 1 --2- 172.21.15.3:0/2057473342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afde822f0 0x561afe2520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.023+0000 7fcc69328640 1 -- 172.21.15.3:0/2057473342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 msgr2=0x561afe268390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.023+0000 7fcc69328640 1 --2- 172.21.15.3:0/2057473342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 0x561afe268390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.023+0000 7fcc69328640 1 -- 172.21.15.3:0/2057473342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afe25d330 con 0x561afdeab1b0 2024-09-21T06:20:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.023+0000 7fcc68b27640 1 --2- 172.21.15.3:0/2057473342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afde822f0 0x561afe2520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.023+0000 7fcc69328640 1 --2- 172.21.15.3:0/2057473342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afdeab1b0 0x561afe271200 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c035000 tx=0x7fcc5c044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48b91ef1c084278 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.023+0000 7fcc4e7fc640 1 -- 172.21.15.3:0/2057473342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c003a30 con 0x561afdeab1b0 2024-09-21T06:20:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.023+0000 7fcc4e7fc640 1 -- 172.21.15.3:0/2057473342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc5c036050 con 0x561afdeab1b0 2024-09-21T06:20:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.023+0000 7fcc4e7fc640 1 -- 172.21.15.3:0/2057473342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c02f8c0 con 0x561afdeab1b0 2024-09-21T06:20:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.024+0000 7fcc71dd7740 1 -- 172.21.15.3:0/2057473342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afdeab1b0 msgr2=0x561afe271200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.024+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/2057473342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afdeab1b0 0x561afe271200 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c035000 tx=0x7fcc5c044000 comp rx=0 tx=0).stop 2024-09-21T06:20:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.024+0000 7fcc71dd7740 1 -- 172.21.15.3:0/2057473342 shutdown_connections 2024-09-21T06:20:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.024+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/2057473342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afdeab1b0 0x561afe271200 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.024+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/2057473342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afde8a4d0 0x561afe268390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.024+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/2057473342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afde822f0 0x561afe2520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.024+0000 7fcc71dd7740 1 -- 172.21.15.3:0/2057473342 >> 172.21.15.3:0/2057473342 conn(0x561afde9f300 msgr2=0x561afe2519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.024+0000 7fcc71dd7740 1 -- 172.21.15.3:0/2057473342 shutdown_connections 2024-09-21T06:20:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.024+0000 7fcc71dd7740 1 -- 172.21.15.3:0/2057473342 wait complete. 2024-09-21T06:20:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.024+0000 7fcc71dd7740 1 -- start start 2024-09-21T06:20:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.024+0000 7fcc71dd7740 1 Processor -- start 2024-09-21T06:20:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc71dd7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe2cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc71dd7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afde8a4d0 0x561afe2cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc71dd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afdeab1b0 0x561afe2d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc71dd7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641af940 con 0x561afde822f0 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc71dd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641af270 con 0x561afdeab1b0 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc71dd7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641ac480 con 0x561afde8a4d0 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc71dd7740 10 client.0 _ll_register_callbacks cb 0x7fcc580f9070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc68b27640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe2cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc68b27640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe2cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37762/0 (socket says 172.21.15.3:37762) 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc68b27640 1 -- 172.21.15.3:0/2625912566 learned_addr learned my addr 172.21.15.3:0/2625912566 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc69328640 1 --2- 172.21.15.3:0/2625912566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afdeab1b0 0x561afe2d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc63fff640 1 --2- 172.21.15.3:0/2625912566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afde8a4d0 0x561afe2cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc68b27640 1 -- 172.21.15.3:0/2625912566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afde8a4d0 msgr2=0x561afe2cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc68b27640 1 --2- 172.21.15.3:0/2625912566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afde8a4d0 0x561afe2cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc68b27640 1 -- 172.21.15.3:0/2625912566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afdeab1b0 msgr2=0x561afe2d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc68b27640 1 --2- 172.21.15.3:0/2625912566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afdeab1b0 0x561afe2d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.025+0000 7fcc68b27640 1 -- 172.21.15.3:0/2625912566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afe2d2e80 con 0x561afde822f0 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc68b27640 1 --2- 172.21.15.3:0/2625912566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe2cd1f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800e9c0 tx=0x7fcc580eaff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a35532a9d0761d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc43fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc43fff640 1 -- 172.21.15.3:0/2625912566 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc580f2070 con 0x561afde822f0 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc43fff640 1 -- 172.21.15.3:0/2625912566 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fcc580acb30 con 0x561afde822f0 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc71dd7740 1 -- 172.21.15.3:0/2625912566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc5c036050 con 0x561afde822f0 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc43fff640 1 -- 172.21.15.3:0/2625912566 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcc580a9f20 con 0x561afde822f0 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc71dd7740 10 client.17187 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc71dd7740 1 -- 172.21.15.3:0/2625912566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561afe2da270 con 0x561afde822f0 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc43fff640 1 -- 172.21.15.3:0/2625912566 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc5c036050 con 0x561afde822f0 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc43fff640 1 -- 172.21.15.3:0/2625912566 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fcc580ac5a0 con 0x561afde822f0 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc71dd7740 10 client.17187 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc71dd7740 10 client.17187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc71dd7740 20 client.17187 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/2625912566 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561afde3f1c0 0x561afe2dc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc71dd7740 4 client.17187 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:20:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.026+0000 7fcc71dd7740 1 -- 172.21.15.3:0/2625912566 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561afe1c0520 con 0x561afde3f1c0 2024-09-21T06:20:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.027+0000 7fcc63fff640 1 --2- 172.21.15.3:0/2625912566 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561afde3f1c0 0x561afe2dc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.027+0000 7fcc63fff640 1 --2- 172.21.15.3:0/2625912566 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561afde3f1c0 0x561afe2dc740 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d419612e4c144cfe server_cookie=53f07062cd048251 in_seq=0 out_seq=0 2024-09-21T06:20:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.027+0000 7fcc43fff640 10 client.17187 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.139+0000 7fcc43fff640 1 -- 172.21.15.3:0/2625912566 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561afe1c0520 con 0x561afde3f1c0 2024-09-21T06:20:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.139+0000 7fcc43fff640 10 client.17187 handle_command_reply: tid=0 2024-09-21T06:20:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.140+0000 7fcc71dd7740 1 client.17187 shutdown 2024-09-21T06:20:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.140+0000 7fcc71dd7740 1 -- 172.21.15.3:0/2625912566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 msgr2=0x561afe2cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.140+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/2625912566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe2cd1f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800e9c0 tx=0x7fcc580eaff0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.140+0000 7fcc71dd7740 1 -- 172.21.15.3:0/2625912566 shutdown_connections 2024-09-21T06:20:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.140+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/2625912566 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561afde3f1c0 0x561afe2dc740 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.140+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/2625912566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561afde8a4d0 0x561afe2cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.140+0000 7fcc43fff640 0 client.17187 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.140+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/2625912566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afdeab1b0 0x561afe2d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.140+0000 7fcc71dd7740 1 --2- 172.21.15.3:0/2625912566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afde822f0 0x561afe2cd1f0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.140+0000 7fcc71dd7740 1 -- 172.21.15.3:0/2625912566 >> 172.21.15.3:0/2625912566 conn(0x561afde9f300 msgr2=0x561afe24e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.140+0000 7fcc71dd7740 1 -- 172.21.15.3:0/2625912566 shutdown_connections 2024-09-21T06:20:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.140+0000 7fcc71dd7740 1 -- 172.21.15.3:0/2625912566 wait complete. 2024-09-21T06:20:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.140+0000 7fcc71dd7740 20 client.17187 trim_cache size 0 max 16384 2024-09-21T06:20:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:20:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:20:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:20:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:20:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.141+0000 7fcc6ad8b640 1 -- 172.21.15.3:0/2066263030 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc540b6700 msgr2=0x7fcc540b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.141+0000 7fcc6ad8b640 1 --2- 172.21.15.3:0/2066263030 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc540b6700 0x7fcc540b8c10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c02f500 tx=0x7fcc5c02f490 comp rx=0 tx=0).stop 2024-09-21T06:20:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.141+0000 7fcc6ad8b640 1 -- 172.21.15.3:0/2066263030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc641029e0 msgr2=0x7fcc64199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.141+0000 7fcc6ad8b640 1 --2- 172.21.15.3:0/2066263030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc641029e0 0x7fcc64199300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800e9f0 tx=0x7fcc5800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.141+0000 7fcc6ad8b640 1 -- 172.21.15.3:0/2066263030 shutdown_connections 2024-09-21T06:20:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.141+0000 7fcc6ad8b640 1 --2- 172.21.15.3:0/2066263030 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc540b6700 0x7fcc540b8c10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.141+0000 7fcc6ad8b640 1 --2- 172.21.15.3:0/2066263030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc64108310 0x7fcc641998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.141+0000 7fcc6ad8b640 1 --2- 172.21.15.3:0/2066263030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc64108c60 0x7fcc6419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.141+0000 7fcc6ad8b640 1 --2- 172.21.15.3:0/2066263030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc641029e0 0x7fcc64199300 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.141+0000 7fcc6ad8b640 1 -- 172.21.15.3:0/2066263030 >> 172.21.15.3:0/2066263030 conn(0x7fcc640fe3c0 msgr2=0x7fcc641016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.141+0000 7fcc6ad8b640 1 -- 172.21.15.3:0/2066263030 shutdown_connections 2024-09-21T06:20:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:37.141+0000 7fcc6ad8b640 1 -- 172.21.15.3:0/2066263030 wait complete. 2024-09-21T06:20:37.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:37 smithi003 ceph-mon[31227]: pgmap v493: 97 pgs: 97 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-21T06:20:37.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:37 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:20:37.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:37 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:20:37.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:37 smithi137 ceph-mon[34464]: pgmap v493: 97 pgs: 97 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-21T06:20:37.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:37 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:20:37.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:37 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:20:37.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:37 smithi204 ceph-mon[34659]: pgmap v493: 97 pgs: 97 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-21T06:20:37.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:37 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:20:37.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:37 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:20:37.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 579886 11.94 MB/sec execute 130 sec latency 75.063 ms 2024-09-21T06:20:38.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 582183 11.94 MB/sec execute 131 sec latency 84.931 ms 2024-09-21T06:20:39.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:39 smithi003 ceph-mon[31227]: pgmap v494: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 258 op/s 2024-09-21T06:20:39.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:39 smithi137 ceph-mon[34464]: pgmap v494: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 258 op/s 2024-09-21T06:20:39.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:39 smithi204 ceph-mon[34659]: pgmap v494: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 258 op/s 2024-09-21T06:20:39.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 583705 11.91 MB/sec execute 132 sec latency 120.617 ms 2024-09-21T06:20:40.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 586283 11.95 MB/sec execute 133 sec latency 16.891 ms 2024-09-21T06:20:41.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:41 smithi137 ceph-mon[34464]: pgmap v495: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 299 op/s 2024-09-21T06:20:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:41 smithi204 ceph-mon[34659]: pgmap v495: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 299 op/s 2024-09-21T06:20:41.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:41 smithi003 ceph-mon[31227]: pgmap v495: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 299 op/s 2024-09-21T06:20:41.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 587897 11.90 MB/sec execute 134 sec latency 16.674 ms 2024-09-21T06:20:42.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 590146 11.91 MB/sec execute 135 sec latency 14.830 ms 2024-09-21T06:20:43.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:43 smithi137 ceph-mon[34464]: pgmap v496: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 318 op/s 2024-09-21T06:20:43.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:43 smithi204 ceph-mon[34659]: pgmap v496: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 318 op/s 2024-09-21T06:20:43.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:43 smithi003 ceph-mon[31227]: pgmap v496: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 318 op/s 2024-09-21T06:20:43.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 592217 11.89 MB/sec execute 136 sec latency 37.181 ms 2024-09-21T06:20:44.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 593739 11.88 MB/sec execute 137 sec latency 44.870 ms 2024-09-21T06:20:45.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:45 smithi137 ceph-mon[34464]: pgmap v497: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 318 op/s 2024-09-21T06:20:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:45 smithi204 ceph-mon[34659]: pgmap v497: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 318 op/s 2024-09-21T06:20:45.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:45 smithi003 ceph-mon[31227]: pgmap v497: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 318 op/s 2024-09-21T06:20:45.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 596057 11.88 MB/sec execute 138 sec latency 18.284 ms 2024-09-21T06:20:46.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 597685 11.85 MB/sec execute 139 sec latency 17.534 ms 2024-09-21T06:20:47.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:47 smithi137 ceph-mon[34464]: pgmap v498: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 344 op/s 2024-09-21T06:20:47.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:47 smithi204 ceph-mon[34659]: pgmap v498: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 344 op/s 2024-09-21T06:20:47.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:47 smithi003 ceph-mon[31227]: pgmap v498: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 344 op/s 2024-09-21T06:20:47.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 600329 11.89 MB/sec execute 140 sec latency 21.034 ms 2024-09-21T06:20:48.957 INFO:tasks.workunit.client.0.smithi003.stdout: 1 601814 11.85 MB/sec execute 141 sec latency 22.809 ms 2024-09-21T06:20:49.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:49 smithi137 ceph-mon[34464]: pgmap v499: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-09-21T06:20:49.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:49 smithi204 ceph-mon[34659]: pgmap v499: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-09-21T06:20:49.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:49 smithi003 ceph-mon[31227]: pgmap v499: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-09-21T06:20:49.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 604010 11.86 MB/sec execute 142 sec latency 91.615 ms 2024-09-21T06:20:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.242+0000 7f9f8c08b640 1 -- 172.21.15.3:0/2823110719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f84108310 msgr2=0x7f9f84108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.242+0000 7f9f8c08b640 1 --2- 172.21.15.3:0/2823110719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f84108310 0x7f9f84108720 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009900 tx=0x7f9f7402a410 comp rx=0 tx=0).stop 2024-09-21T06:20:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.242+0000 7f9f8c08b640 1 -- 172.21.15.3:0/2823110719 shutdown_connections 2024-09-21T06:20:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.242+0000 7f9f8c08b640 1 --2- 172.21.15.3:0/2823110719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f84108c60 0x7f9f8410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.242+0000 7f9f8c08b640 1 --2- 172.21.15.3:0/2823110719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f84108310 0x7f9f84108720 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.242+0000 7f9f8c08b640 1 --2- 172.21.15.3:0/2823110719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f841029e0 0x7f9f84102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.242+0000 7f9f8c08b640 1 -- 172.21.15.3:0/2823110719 >> 172.21.15.3:0/2823110719 conn(0x7f9f840fe3e0 msgr2=0x7f9f84100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.242+0000 7f9f8c08b640 1 -- 172.21.15.3:0/2823110719 shutdown_connections 2024-09-21T06:20:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.242+0000 7f9f8c08b640 1 -- 172.21.15.3:0/2823110719 wait complete. 2024-09-21T06:20:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.243+0000 7f9f8c08b640 1 Processor -- start 2024-09-21T06:20:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.243+0000 7f9f8c08b640 1 -- start start 2024-09-21T06:20:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.243+0000 7f9f8c08b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f841029e0 0x7f9f8410ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.243+0000 7f9f8c08b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f84108310 0x7f9f841104b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.243+0000 7f9f8c08b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f84108c60 0x7f9f841109f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.243+0000 7f9f8c08b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9f84113a40 con 0x7f9f841029e0 2024-09-21T06:20:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.243+0000 7f9f8c08b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841138c0 con 0x7f9f84108c60 2024-09-21T06:20:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.243+0000 7f9f8c08b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9f84113bc0 con 0x7f9f84108310 2024-09-21T06:20:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.243+0000 7f9f89e27640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f841029e0 0x7f9f8410ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.243+0000 7f9f89e27640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f841029e0 0x7f9f8410ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36246/0 (socket says 172.21.15.3:36246) 2024-09-21T06:20:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.243+0000 7f9f89e27640 1 -- 172.21.15.3:0/2601829826 learned_addr learned my addr 172.21.15.3:0/2601829826 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.244+0000 7f9f89626640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f84108310 0x7f9f841104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.244+0000 7f9f8a628640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f84108c60 0x7f9f841109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.244+0000 7f9f89e27640 1 -- 172.21.15.3:0/2601829826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f84108310 msgr2=0x7f9f841104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.244+0000 7f9f89e27640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f84108310 0x7f9f841104b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.244+0000 7f9f89e27640 1 -- 172.21.15.3:0/2601829826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f84108c60 msgr2=0x7f9f841109f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.244+0000 7f9f89e27640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f84108c60 0x7f9f841109f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.244+0000 7f9f89e27640 1 -- 172.21.15.3:0/2601829826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f841a86b0 con 0x7f9f841029e0 2024-09-21T06:20:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.244+0000 7f9f89e27640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f841029e0 0x7f9f8410ff70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000e9f0 tx=0x7f9f8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.244+0000 7f9f7affd640 1 -- 172.21.15.3:0/2601829826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8000cd80 con 0x7f9f841029e0 2024-09-21T06:20:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.244+0000 7f9f7affd640 1 -- 172.21.15.3:0/2601829826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f800108e0 con 0x7f9f841029e0 2024-09-21T06:20:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.244+0000 7f9f8c08b640 1 -- 172.21.15.3:0/2601829826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f841a89a0 con 0x7f9f841029e0 2024-09-21T06:20:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.244+0000 7f9f7affd640 1 -- 172.21.15.3:0/2601829826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80018980 con 0x7f9f841029e0 2024-09-21T06:20:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.244+0000 7f9f8c08b640 1 -- 172.21.15.3:0/2601829826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f841a8ec0 con 0x7f9f841029e0 2024-09-21T06:20:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.247+0000 7f9f92ffc740 1 Processor -- start 2024-09-21T06:20:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.247+0000 7f9f92ffc740 1 -- start start 2024-09-21T06:20:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.247+0000 7f9f7affd640 1 -- 172.21.15.3:0/2601829826 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f9f80010430 con 0x7f9f841029e0 2024-09-21T06:20:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.247+0000 7f9f92ffc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764ed1b0 0x5603768929b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.247+0000 7f9f92ffc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764c42f0 0x560376892ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.247+0000 7f9f92ffc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764cc4d0 0x560376893430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.247+0000 7f9f92ffc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5603768076f0 con 0x5603764cc4d0 2024-09-21T06:20:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.247+0000 7f9f92ffc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5603767f6b20 con 0x5603764c42f0 2024-09-21T06:20:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.247+0000 7f9f92ffc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5603767fa930 con 0x5603764ed1b0 2024-09-21T06:20:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f8a628640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764ed1b0 0x5603768929b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f89626640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764cc4d0 0x560376893430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f89e27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764c42f0 0x560376892ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f8a628640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764ed1b0 0x5603768929b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:50688/0 (socket says 172.21.15.3:50688) 2024-09-21T06:20:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f89626640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764cc4d0 0x560376893430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36254/0 (socket says 172.21.15.3:36254) 2024-09-21T06:20:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f8a628640 1 -- 172.21.15.3:0/1209490487 learned_addr learned my addr 172.21.15.3:0/1209490487 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f8a628640 1 -- 172.21.15.3:0/1209490487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764c42f0 msgr2=0x560376892ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f8a628640 1 --2- 172.21.15.3:0/1209490487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764c42f0 0x560376892ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f8a628640 1 -- 172.21.15.3:0/1209490487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764cc4d0 msgr2=0x560376893430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f8a628640 1 --2- 172.21.15.3:0/1209490487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764cc4d0 0x560376893430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f8a628640 1 -- 172.21.15.3:0/1209490487 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603767bc530 con 0x5603764ed1b0 2024-09-21T06:20:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f8a628640 1 --2- 172.21.15.3:0/1209490487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764ed1b0 0x5603768929b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c007f70 tx=0x7f9f7c00f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7d9a89b26356b4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f7affd640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9f700b6680 0x7f9f700b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f6ffff640 1 -- 172.21.15.3:0/1209490487 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c010900 con 0x5603764ed1b0 2024-09-21T06:20:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.248+0000 7f9f89e27640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9f700b6680 0x7f9f700b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.249+0000 7f9f6ffff640 1 -- 172.21.15.3:0/1209490487 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f7c014070 con 0x5603764ed1b0 2024-09-21T06:20:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.249+0000 7f9f6ffff640 1 -- 172.21.15.3:0/1209490487 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c017c70 con 0x5603764ed1b0 2024-09-21T06:20:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.249+0000 7f9f7affd640 1 -- 172.21.15.3:0/2601829826 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f9f800e7a10 con 0x7f9f841029e0 2024-09-21T06:20:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.249+0000 7f9f92ffc740 1 -- 172.21.15.3:0/1209490487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764ed1b0 msgr2=0x5603768929b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.249+0000 7f9f89e27640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9f700b6680 0x7f9f700b8b90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9f800e5250 tx=0x7f9f800eb560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.249+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/1209490487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764ed1b0 0x5603768929b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c007f70 tx=0x7f9f7c00f850 comp rx=0 tx=0).stop 2024-09-21T06:20:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.250+0000 7f9f92ffc740 1 -- 172.21.15.3:0/1209490487 shutdown_connections 2024-09-21T06:20:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.250+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/1209490487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764cc4d0 0x560376893430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.250+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/1209490487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764c42f0 0x560376892ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.250+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/1209490487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764ed1b0 0x5603768929b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.250+0000 7f9f92ffc740 1 -- 172.21.15.3:0/1209490487 >> 172.21.15.3:0/1209490487 conn(0x5603764e1300 msgr2=0x5603768908a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.250+0000 7f9f92ffc740 1 -- 172.21.15.3:0/1209490487 shutdown_connections 2024-09-21T06:20:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.250+0000 7f9f92ffc740 1 -- 172.21.15.3:0/1209490487 wait complete. 2024-09-21T06:20:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.250+0000 7f9f92ffc740 1 -- start start 2024-09-21T06:20:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.250+0000 7f9f92ffc740 1 Processor -- start 2024-09-21T06:20:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f92ffc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764c42f0 0x56037689e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f92ffc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764cc4d0 0x56037689e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f92ffc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764ed1b0 0x5603768aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f92ffc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56037672eab0 con 0x5603764c42f0 2024-09-21T06:20:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f92ffc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56037679c730 con 0x5603764ed1b0 2024-09-21T06:20:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f92ffc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5603768a1130 con 0x5603764cc4d0 2024-09-21T06:20:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f92ffc740 10 client.0 _ll_register_callbacks cb 0x5603767f06b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f8a628640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764c42f0 0x56037689e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f8a628640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764c42f0 0x56037689e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36266/0 (socket says 172.21.15.3:36266) 2024-09-21T06:20:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f8a628640 1 -- 172.21.15.3:0/689373926 learned_addr learned my addr 172.21.15.3:0/689373926 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f89626640 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764cc4d0 0x56037689e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f8a628640 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764ed1b0 0x5603768aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f89626640 1 -- 172.21.15.3:0/689373926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764ed1b0 msgr2=0x5603768aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f89626640 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764ed1b0 0x5603768aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f89626640 1 -- 172.21.15.3:0/689373926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764c42f0 msgr2=0x56037689e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f89626640 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764c42f0 0x56037689e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f89626640 1 -- 172.21.15.3:0/689373926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560376558790 con 0x5603764cc4d0 2024-09-21T06:20:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f8a628640 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764c42f0 0x56037689e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f89626640 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764cc4d0 0x56037689e5a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9f7402a930 tx=0x7f9f74032f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=501f9cb65656090a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.251+0000 7f9f6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:20:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f6d7fa640 1 -- 172.21.15.3:0/689373926 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74035040 con 0x5603764cc4d0 2024-09-21T06:20:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f92ffc740 1 -- 172.21.15.3:0/689373926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f7c014070 con 0x5603764cc4d0 2024-09-21T06:20:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f6d7fa640 1 -- 172.21.15.3:0/689373926 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f9f74031860 con 0x5603764cc4d0 2024-09-21T06:20:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f6d7fa640 1 -- 172.21.15.3:0/689373926 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9f74004040 con 0x5603764cc4d0 2024-09-21T06:20:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f92ffc740 10 client.26710 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f92ffc740 1 -- 172.21.15.3:0/689373926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5603765d1620 con 0x5603764cc4d0 2024-09-21T06:20:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f6d7fa640 1 -- 172.21.15.3:0/689373926 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f7c014070 con 0x5603764cc4d0 2024-09-21T06:20:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f6d7fa640 1 -- 172.21.15.3:0/689373926 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f9f74004750 con 0x5603764cc4d0 2024-09-21T06:20:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f92ffc740 10 client.26710 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f92ffc740 10 client.26710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f92ffc740 20 client.26710 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603764abd80 0x5603768b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f92ffc740 4 client.26710 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f92ffc740 1 -- 172.21.15.3:0/689373926 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560376809640 con 0x5603764abd80 2024-09-21T06:20:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.252+0000 7f9f8a628640 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603764abd80 0x5603768b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.253+0000 7f9f6d7fa640 10 client.26710 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.253+0000 7f9f8a628640 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603764abd80 0x5603768b0140 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b8e702a5a93904c server_cookie=69f2e8ffed740eec in_seq=0 out_seq=0 2024-09-21T06:20:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f6d7fa640 1 -- 172.21.15.3:0/689373926 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560376809640 con 0x5603764abd80 2024-09-21T06:20:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f6d7fa640 10 client.26710 handle_command_reply: tid=0 2024-09-21T06:20:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f92ffc740 1 client.26710 shutdown 2024-09-21T06:20:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f92ffc740 1 -- 172.21.15.3:0/689373926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764cc4d0 msgr2=0x56037689e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764cc4d0 0x56037689e5a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9f7402a930 tx=0x7f9f74032f60 comp rx=0 tx=0).stop 2024-09-21T06:20:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f92ffc740 1 -- 172.21.15.3:0/689373926 shutdown_connections 2024-09-21T06:20:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603764abd80 0x5603768b0140 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764cc4d0 0x56037689e5a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764ed1b0 0x5603768aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/689373926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764c42f0 0x56037689e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f6d7fa640 0 client.26710 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f92ffc740 1 -- 172.21.15.3:0/689373926 >> 172.21.15.3:0/689373926 conn(0x5603764e1300 msgr2=0x5603768939e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f92ffc740 1 -- 172.21.15.3:0/689373926 shutdown_connections 2024-09-21T06:20:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f92ffc740 1 -- 172.21.15.3:0/689373926 wait complete. 2024-09-21T06:20:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.254+0000 7f9f92ffc740 20 client.26710 trim_cache size 0 max 16384 2024-09-21T06:20:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f92ffc740 1 Processor -- start 2024-09-21T06:20:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f92ffc740 1 -- start start 2024-09-21T06:20:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f92ffc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764c42f0 0x56037689e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f92ffc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764cc4d0 0x56037689fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f92ffc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764ed1b0 0x5603768aa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f92ffc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841b1830 con 0x5603764ed1b0 2024-09-21T06:20:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f92ffc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841b33a0 con 0x5603764c42f0 2024-09-21T06:20:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f92ffc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841b1f00 con 0x5603764cc4d0 2024-09-21T06:20:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f8a628640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764ed1b0 0x5603768aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f89626640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764cc4d0 0x56037689fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f89626640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764cc4d0 0x56037689fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:50718/0 (socket says 172.21.15.3:50718) 2024-09-21T06:20:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f89626640 1 -- 172.21.15.3:0/4154206320 learned_addr learned my addr 172.21.15.3:0/4154206320 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f8a628640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764ed1b0 0x5603768aa320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36274/0 (socket says 172.21.15.3:36274) 2024-09-21T06:20:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.269+0000 7f9f8a628640 1 --2- 172.21.15.3:0/4154206320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764c42f0 0x56037689e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.270+0000 7f9f89626640 1 -- 172.21.15.3:0/4154206320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764c42f0 msgr2=0x56037689e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.270+0000 7f9f89626640 1 --2- 172.21.15.3:0/4154206320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764c42f0 0x56037689e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.270+0000 7f9f89626640 1 -- 172.21.15.3:0/4154206320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764ed1b0 msgr2=0x5603768aa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.270+0000 7f9f89626640 1 --2- 172.21.15.3:0/4154206320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764ed1b0 0x5603768aa320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.270+0000 7f9f89626640 1 -- 172.21.15.3:0/4154206320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037689f320 con 0x5603764cc4d0 2024-09-21T06:20:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.270+0000 7f9f8a628640 1 --2- 172.21.15.3:0/4154206320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764ed1b0 0x5603768aa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.270+0000 7f9f89626640 1 --2- 172.21.15.3:0/4154206320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764cc4d0 0x56037689fc90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9f74004e90 tx=0x7f9f74001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e4f17a97bf956b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.270+0000 7f9f6f7fe640 1 -- 172.21.15.3:0/4154206320 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74035070 con 0x5603764cc4d0 2024-09-21T06:20:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.270+0000 7f9f6f7fe640 1 -- 172.21.15.3:0/4154206320 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f740046c0 con 0x5603764cc4d0 2024-09-21T06:20:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.270+0000 7f9f6f7fe640 1 -- 172.21.15.3:0/4154206320 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74053040 con 0x5603764cc4d0 2024-09-21T06:20:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.271+0000 7f9f92ffc740 1 -- 172.21.15.3:0/4154206320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764cc4d0 msgr2=0x56037689fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.271+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/4154206320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764cc4d0 0x56037689fc90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9f74004e90 tx=0x7f9f74001e70 comp rx=0 tx=0).stop 2024-09-21T06:20:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.271+0000 7f9f92ffc740 1 -- 172.21.15.3:0/4154206320 shutdown_connections 2024-09-21T06:20:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.271+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/4154206320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764cc4d0 0x56037689fc90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.271+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/4154206320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764ed1b0 0x5603768aa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.271+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/4154206320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764c42f0 0x56037689e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.271+0000 7f9f92ffc740 1 -- 172.21.15.3:0/4154206320 >> 172.21.15.3:0/4154206320 conn(0x5603764e1300 msgr2=0x56037689f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.271+0000 7f9f92ffc740 1 -- 172.21.15.3:0/4154206320 shutdown_connections 2024-09-21T06:20:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.271+0000 7f9f92ffc740 1 -- 172.21.15.3:0/4154206320 wait complete. 2024-09-21T06:20:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.271+0000 7f9f92ffc740 1 -- start start 2024-09-21T06:20:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.271+0000 7f9f92ffc740 1 Processor -- start 2024-09-21T06:20:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f92ffc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764c42f0 0x5603768a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f92ffc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764cc4d0 0x5603768a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f92ffc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764ed1b0 0x5603768a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f92ffc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841b2cd0 con 0x5603764cc4d0 2024-09-21T06:20:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f92ffc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841b2600 con 0x5603764ed1b0 2024-09-21T06:20:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f92ffc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841af810 con 0x5603764c42f0 2024-09-21T06:20:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f92ffc740 10 client.0 _ll_register_callbacks cb 0x7f9f74009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f89626640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764cc4d0 0x5603768a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f8a628640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764c42f0 0x5603768a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f89626640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764cc4d0 0x5603768a2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36288/0 (socket says 172.21.15.3:36288) 2024-09-21T06:20:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f89626640 1 -- 172.21.15.3:0/1286355697 learned_addr learned my addr 172.21.15.3:0/1286355697 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:20:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f8a628640 1 --2- 172.21.15.3:0/1286355697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764ed1b0 0x5603768a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f89626640 1 -- 172.21.15.3:0/1286355697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764c42f0 msgr2=0x5603768a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f89626640 1 --2- 172.21.15.3:0/1286355697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764c42f0 0x5603768a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f89626640 1 -- 172.21.15.3:0/1286355697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764ed1b0 msgr2=0x5603768a2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f89626640 1 --2- 172.21.15.3:0/1286355697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764ed1b0 0x5603768a2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.272+0000 7f9f89626640 1 -- 172.21.15.3:0/1286355697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560376919c80 con 0x5603764cc4d0 2024-09-21T06:20:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.273+0000 7f9f89626640 1 --2- 172.21.15.3:0/1286355697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764cc4d0 0x5603768a2a20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9f7402a930 tx=0x7f9f7402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899551922de9a2df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.273+0000 7f9f6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:20:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.273+0000 7f9f6d7fa640 1 -- 172.21.15.3:0/1286355697 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74035040 con 0x5603764cc4d0 2024-09-21T06:20:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.273+0000 7f9f6d7fa640 1 -- 172.21.15.3:0/1286355697 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f9f74005170 con 0x5603764cc4d0 2024-09-21T06:20:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.273+0000 7f9f92ffc740 1 -- 172.21.15.3:0/1286355697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f740046c0 con 0x5603764cc4d0 2024-09-21T06:20:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.273+0000 7f9f6d7fa640 1 -- 172.21.15.3:0/1286355697 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9f74069050 con 0x5603764cc4d0 2024-09-21T06:20:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.273+0000 7f9f92ffc740 10 client.17205 fetch_fsmap learned FSMap version 32 2024-09-21T06:20:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.273+0000 7f9f92ffc740 1 -- 172.21.15.3:0/1286355697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56037691aef0 con 0x5603764cc4d0 2024-09-21T06:20:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.273+0000 7f9f6d7fa640 1 -- 172.21.15.3:0/1286355697 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f740046c0 con 0x5603764cc4d0 2024-09-21T06:20:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.273+0000 7f9f6d7fa640 1 -- 172.21.15.3:0/1286355697 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f9f74060070 con 0x5603764cc4d0 2024-09-21T06:20:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.274+0000 7f9f92ffc740 10 client.17205 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:20:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.274+0000 7f9f92ffc740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.274+0000 7f9f92ffc740 20 client.17205 populate_metadata read hostname 'smithi003' 2024-09-21T06:20:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.274+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/1286355697 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603764811c0 0x56037691d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.274+0000 7f9f92ffc740 4 client.17205 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:20:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.274+0000 7f9f92ffc740 1 -- 172.21.15.3:0/1286355697 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603768050f0 con 0x5603764811c0 2024-09-21T06:20:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.274+0000 7f9f8a628640 1 --2- 172.21.15.3:0/1286355697 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603764811c0 0x56037691d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.274+0000 7f9f6d7fa640 10 client.17205 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.274+0000 7f9f8a628640 1 --2- 172.21.15.3:0/1286355697 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603764811c0 0x56037691d5f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0cadf7e2b3a4eb server_cookie=144ce83b3911ea2d in_seq=0 out_seq=0 2024-09-21T06:20:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f6d7fa640 1 -- 172.21.15.3:0/1286355697 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603768050f0 con 0x5603764811c0 2024-09-21T06:20:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f6d7fa640 10 client.17205 handle_command_reply: tid=0 2024-09-21T06:20:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f92ffc740 1 client.17205 shutdown 2024-09-21T06:20:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f92ffc740 1 -- 172.21.15.3:0/1286355697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764cc4d0 msgr2=0x5603768a2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/1286355697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764cc4d0 0x5603768a2a20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9f7402a930 tx=0x7f9f7402a410 comp rx=0 tx=0).stop 2024-09-21T06:20:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f92ffc740 1 -- 172.21.15.3:0/1286355697 shutdown_connections 2024-09-21T06:20:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/1286355697 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5603764811c0 0x56037691d5f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/1286355697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603764ed1b0 0x5603768a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f6d7fa640 0 client.17205 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:20:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/1286355697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5603764cc4d0 0x5603768a2a20 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f92ffc740 1 --2- 172.21.15.3:0/1286355697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603764c42f0 0x5603768a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f92ffc740 1 -- 172.21.15.3:0/1286355697 >> 172.21.15.3:0/1286355697 conn(0x5603764e1300 msgr2=0x560376890880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f92ffc740 1 -- 172.21.15.3:0/1286355697 shutdown_connections 2024-09-21T06:20:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f92ffc740 1 -- 172.21.15.3:0/1286355697 wait complete. 2024-09-21T06:20:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.431+0000 7f9f92ffc740 20 client.17205 trim_cache size 0 max 16384 2024-09-21T06:20:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:20:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:20:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:20:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:20:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.432+0000 7f9f8c08b640 1 -- 172.21.15.3:0/2601829826 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9f700b6680 msgr2=0x7f9f700b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.432+0000 7f9f8c08b640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9f700b6680 0x7f9f700b8b90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9f800e5250 tx=0x7f9f800eb560 comp rx=0 tx=0).stop 2024-09-21T06:20:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.432+0000 7f9f8c08b640 1 -- 172.21.15.3:0/2601829826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f841029e0 msgr2=0x7f9f8410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.432+0000 7f9f8c08b640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f841029e0 0x7f9f8410ff70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000e9f0 tx=0x7f9f8000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.432+0000 7f9f8c08b640 1 -- 172.21.15.3:0/2601829826 shutdown_connections 2024-09-21T06:20:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.432+0000 7f9f8c08b640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9f700b6680 0x7f9f700b8b90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.432+0000 7f9f8c08b640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f84108310 0x7f9f841104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.432+0000 7f9f8c08b640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f84108c60 0x7f9f841109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.432+0000 7f9f8c08b640 1 --2- 172.21.15.3:0/2601829826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f841029e0 0x7f9f8410ff70 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.432+0000 7f9f8c08b640 1 -- 172.21.15.3:0/2601829826 >> 172.21.15.3:0/2601829826 conn(0x7f9f840fe3e0 msgr2=0x7f9f84105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.432+0000 7f9f8c08b640 1 -- 172.21.15.3:0/2601829826 shutdown_connections 2024-09-21T06:20:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:20:50.432+0000 7f9f8c08b640 1 -- 172.21.15.3:0/2601829826 wait complete. 2024-09-21T06:20:50.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 605256 11.82 MB/sec execute 143 sec latency 82.109 ms 2024-09-21T06:20:51.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:51 smithi137 ceph-mon[34464]: pgmap v500: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2024-09-21T06:20:51.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:51 smithi204 ceph-mon[34659]: pgmap v500: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2024-09-21T06:20:51.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:51 smithi003 ceph-mon[31227]: pgmap v500: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2024-09-21T06:20:51.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 607593 11.83 MB/sec execute 144 sec latency 20.074 ms 2024-09-21T06:20:52.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 609259 11.79 MB/sec execute 145 sec latency 16.488 ms 2024-09-21T06:20:53.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:53 smithi137 ceph-mon[34464]: pgmap v501: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2024-09-21T06:20:53.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:53 smithi204 ceph-mon[34659]: pgmap v501: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2024-09-21T06:20:53.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:53 smithi003 ceph-mon[31227]: pgmap v501: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2024-09-21T06:20:53.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 611449 11.80 MB/sec execute 146 sec latency 32.093 ms 2024-09-21T06:20:54.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 613650 11.80 MB/sec execute 147 sec latency 45.912 ms 2024-09-21T06:20:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:55 smithi137 ceph-mon[34464]: pgmap v502: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 301 op/s 2024-09-21T06:20:55.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:55 smithi204 ceph-mon[34659]: pgmap v502: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 301 op/s 2024-09-21T06:20:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:55 smithi003 ceph-mon[31227]: pgmap v502: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 301 op/s 2024-09-21T06:20:55.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 615350 11.78 MB/sec execute 148 sec latency 24.290 ms 2024-09-21T06:20:56.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 618122 11.81 MB/sec execute 149 sec latency 15.543 ms 2024-09-21T06:20:57.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:57 smithi137 ceph-mon[34464]: pgmap v503: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 364 op/s 2024-09-21T06:20:57.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:57 smithi204 ceph-mon[34659]: pgmap v503: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 364 op/s 2024-09-21T06:20:57.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:57 smithi003 ceph-mon[31227]: pgmap v503: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 364 op/s 2024-09-21T06:20:57.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 619678 11.78 MB/sec execute 150 sec latency 15.952 ms 2024-09-21T06:20:58.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 621984 11.79 MB/sec execute 151 sec latency 13.580 ms 2024-09-21T06:20:59.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:20:59 smithi137 ceph-mon[34464]: pgmap v504: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 338 op/s 2024-09-21T06:20:59.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:20:59 smithi204 ceph-mon[34659]: pgmap v504: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 338 op/s 2024-09-21T06:20:59.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:20:59 smithi003 ceph-mon[31227]: pgmap v504: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 338 op/s 2024-09-21T06:20:59.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 624134 11.77 MB/sec execute 152 sec latency 31.574 ms 2024-09-21T06:21:00.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 625939 11.77 MB/sec execute 153 sec latency 24.156 ms 2024-09-21T06:21:01.642 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:01 smithi003 ceph-mon[31227]: pgmap v505: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 360 op/s 2024-09-21T06:21:01.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:01 smithi137 ceph-mon[34464]: pgmap v505: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 360 op/s 2024-09-21T06:21:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:01 smithi204 ceph-mon[34659]: pgmap v505: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 360 op/s 2024-09-21T06:21:01.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 627478 11.73 MB/sec execute 154 sec latency 31.404 ms 2024-09-21T06:21:02.587 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:21:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:02.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 629283 11.73 MB/sec execute 155 sec latency 18.108 ms 2024-09-21T06:21:03.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.447+0000 7f24b8e36640 1 -- 172.21.15.3:0/2433048490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b41094d0 msgr2=0x7f24b41098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.447+0000 7f24b8e36640 1 --2- 172.21.15.3:0/2433048490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b41094d0 0x7f24b41098e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009900 tx=0x7f24a802a410 comp rx=0 tx=0).stop 2024-09-21T06:21:03.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.447+0000 7f24b8e36640 1 -- 172.21.15.3:0/2433048490 shutdown_connections 2024-09-21T06:21:03.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.447+0000 7f24b8e36640 1 --2- 172.21.15.3:0/2433048490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b4109e20 0x7f24b4110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.447+0000 7f24b8e36640 1 --2- 172.21.15.3:0/2433048490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b41094d0 0x7f24b41098e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.447+0000 7f24b8e36640 1 --2- 172.21.15.3:0/2433048490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24b4103b90 0x7f24b4103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.447+0000 7f24b8e36640 1 -- 172.21.15.3:0/2433048490 >> 172.21.15.3:0/2433048490 conn(0x7f24b40ff5e0 msgr2=0x7f24b4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:03.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.447+0000 7f24b8e36640 1 -- 172.21.15.3:0/2433048490 shutdown_connections 2024-09-21T06:21:03.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.447+0000 7f24b8e36640 1 -- 172.21.15.3:0/2433048490 wait complete. 2024-09-21T06:21:03.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.448+0000 7f24b8e36640 1 Processor -- start 2024-09-21T06:21:03.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.448+0000 7f24b8e36640 1 -- start start 2024-09-21T06:21:03.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b8e36640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b4103b90 0x7f24b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b8e36640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24b41094d0 0x7f24b4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b8e36640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b4109e20 0x7f24b4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b8e36640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24b4113ae0 con 0x7f24b4103b90 2024-09-21T06:21:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b8e36640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24b4113960 con 0x7f24b4109e20 2024-09-21T06:21:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b8e36640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24b4113c60 con 0x7f24b41094d0 2024-09-21T06:21:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b4103b90 0x7f24b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b4103b90 0x7f24b4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53142/0 (socket says 172.21.15.3:53142) 2024-09-21T06:21:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b259c640 1 -- 172.21.15.3:0/1340275843 learned_addr learned my addr 172.21.15.3:0/1340275843 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b1d9b640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24b41094d0 0x7f24b4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b2d9d640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b4109e20 0x7f24b4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b259c640 1 -- 172.21.15.3:0/1340275843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24b41094d0 msgr2=0x7f24b4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b259c640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24b41094d0 0x7f24b4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b259c640 1 -- 172.21.15.3:0/1340275843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b4109e20 msgr2=0x7f24b4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b259c640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b4109e20 0x7f24b4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.449+0000 7f24b259c640 1 -- 172.21.15.3:0/1340275843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b41a8770 con 0x7f24b4103b90 2024-09-21T06:21:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.450+0000 7f24b259c640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b4103b90 0x7f24b4075710 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f24a400e9f0 tx=0x7f24a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.450+0000 7f24a37fe640 1 -- 172.21.15.3:0/1340275843 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a400cd80 con 0x7f24b4103b90 2024-09-21T06:21:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.450+0000 7f24a37fe640 1 -- 172.21.15.3:0/1340275843 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24a40108e0 con 0x7f24b4103b90 2024-09-21T06:21:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.450+0000 7f24b8e36640 1 -- 172.21.15.3:0/1340275843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b41a8a60 con 0x7f24b4103b90 2024-09-21T06:21:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.451+0000 7f24a37fe640 1 -- 172.21.15.3:0/1340275843 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4018980 con 0x7f24b4103b90 2024-09-21T06:21:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.451+0000 7f24b8e36640 1 -- 172.21.15.3:0/1340275843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b41a8fc0 con 0x7f24b4103b90 2024-09-21T06:21:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.453+0000 7f24a37fe640 1 -- 172.21.15.3:0/1340275843 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f24a4010430 con 0x7f24b4103b90 2024-09-21T06:21:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.454+0000 7f24a37fe640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f249c0b6780 0x7f249c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.454+0000 7f24bf3aa740 1 Processor -- start 2024-09-21T06:21:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.454+0000 7f24a37fe640 1 -- 172.21.15.3:0/1340275843 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f24a40e79f0 con 0x7f24b4103b90 2024-09-21T06:21:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.454+0000 7f24bf3aa740 1 -- start start 2024-09-21T06:21:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24b1d9b640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f249c0b6780 0x7f249c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24bf3aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f9000b070 0x557f903b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24bf3aa740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffe21b0 0x557f903b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24bf3aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24bf3aa740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557f90328e50 con 0x557f8ffe21b0 2024-09-21T06:21:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24bf3aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557f90318280 con 0x557f9000b070 2024-09-21T06:21:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24bf3aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557f9031c090 con 0x557f8ffea390 2024-09-21T06:21:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24b1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffe21b0 0x557f903b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24b259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f9000b070 0x557f903b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24b1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffe21b0 0x557f903b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53154/0 (socket says 172.21.15.3:53154) 2024-09-21T06:21:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24b259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f9000b070 0x557f903b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:35282/0 (socket says 172.21.15.3:35282) 2024-09-21T06:21:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24b1d9b640 1 -- 172.21.15.3:0/4016946208 learned_addr learned my addr 172.21.15.3:0/4016946208 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24b2d9d640 1 --2- 172.21.15.3:0/4016946208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24b259c640 1 -- 172.21.15.3:0/4016946208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 msgr2=0x557f903b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24b259c640 1 --2- 172.21.15.3:0/4016946208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24b259c640 1 -- 172.21.15.3:0/4016946208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffe21b0 msgr2=0x557f903b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.455+0000 7f24b259c640 1 --2- 172.21.15.3:0/4016946208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffe21b0 0x557f903b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.456+0000 7f24b1d9b640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f249c0b6780 0x7f249c0b8c90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f24a8005f70 tx=0x7f24a8032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.456+0000 7f24b259c640 1 -- 172.21.15.3:0/4016946208 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f90076650 con 0x557f9000b070 2024-09-21T06:21:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.456+0000 7f24b2d9d640 1 --2- 172.21.15.3:0/4016946208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.456+0000 7f24b259c640 1 --2- 172.21.15.3:0/4016946208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f9000b070 0x557f903b2270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f24a40e5120 tx=0x7f24a4010720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7497c510ddb869c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.456+0000 7f24a0ff9640 1 -- 172.21.15.3:0/4016946208 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a40b2300 con 0x557f9000b070 2024-09-21T06:21:03.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.456+0000 7f24a0ff9640 1 -- 172.21.15.3:0/4016946208 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24a40b7240 con 0x557f9000b070 2024-09-21T06:21:03.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.456+0000 7f24a0ff9640 1 -- 172.21.15.3:0/4016946208 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a40ac5a0 con 0x557f9000b070 2024-09-21T06:21:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.457+0000 7f24bf3aa740 1 -- 172.21.15.3:0/4016946208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f9000b070 msgr2=0x557f903b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.457+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/4016946208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f9000b070 0x557f903b2270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f24a40e5120 tx=0x7f24a4010720 comp rx=0 tx=0).stop 2024-09-21T06:21:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.458+0000 7f24bf3aa740 1 -- 172.21.15.3:0/4016946208 shutdown_connections 2024-09-21T06:21:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.458+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/4016946208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.458+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/4016946208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffe21b0 0x557f903b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.458+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/4016946208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f9000b070 0x557f903b2270 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.458+0000 7f24bf3aa740 1 -- 172.21.15.3:0/4016946208 >> 172.21.15.3:0/4016946208 conn(0x557f8ffff460 msgr2=0x557f903b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.458+0000 7f24bf3aa740 1 -- 172.21.15.3:0/4016946208 shutdown_connections 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.458+0000 7f24bf3aa740 1 -- 172.21.15.3:0/4016946208 wait complete. 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.458+0000 7f24bf3aa740 1 -- start start 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.458+0000 7f24bf3aa740 1 Processor -- start 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24bf3aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 0x557f903bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24bf3aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903bdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24bf3aa740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f9000b070 0x557f903c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24bf3aa740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557f90179700 con 0x557f9000b070 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24bf3aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557f902b3110 con 0x557f8ffe21b0 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24bf3aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557f903c0a50 con 0x557f8ffea390 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24bf3aa740 10 client.0 _ll_register_callbacks cb 0x557f902cc990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24b1d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903bdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24b259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 0x557f903bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24b259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 0x557f903bda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:35292/0 (socket says 172.21.15.3:35292) 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24b259c640 1 -- 172.21.15.3:0/1387858136 learned_addr learned my addr 172.21.15.3:0/1387858136 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24b2d9d640 1 --2- 172.21.15.3:0/1387858136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f9000b070 0x557f903c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24b259c640 1 -- 172.21.15.3:0/1387858136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 msgr2=0x557f903bdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24b259c640 1 --2- 172.21.15.3:0/1387858136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903bdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24b259c640 1 -- 172.21.15.3:0/1387858136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f9000b070 msgr2=0x557f903c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24b259c640 1 --2- 172.21.15.3:0/1387858136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f9000b070 0x557f903c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.459+0000 7f24b259c640 1 -- 172.21.15.3:0/1387858136 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f902e0c40 con 0x557f8ffe21b0 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.460+0000 7f24b259c640 1 --2- 172.21.15.3:0/1387858136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 0x557f903bda10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f24a40ad860 tx=0x7f24a40a9d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e6103bb0cf57e85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.460+0000 7f2499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.460+0000 7f2499ffb640 1 -- 172.21.15.3:0/1387858136 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a40a95a0 con 0x557f8ffe21b0 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.460+0000 7f2499ffb640 1 -- 172.21.15.3:0/1387858136 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f24a40aed50 con 0x557f8ffe21b0 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.460+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1387858136 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a40b7240 con 0x557f8ffe21b0 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.460+0000 7f2499ffb640 1 -- 172.21.15.3:0/1387858136 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f24a40b2300 con 0x557f8ffe21b0 2024-09-21T06:21:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.460+0000 7f24bf3aa740 10 client.26789 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.460+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1387858136 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557f903cc7e0 con 0x557f8ffe21b0 2024-09-21T06:21:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.461+0000 7f2499ffb640 1 -- 172.21.15.3:0/1387858136 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24a40b7240 con 0x557f8ffe21b0 2024-09-21T06:21:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.461+0000 7f2499ffb640 1 -- 172.21.15.3:0/1387858136 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f24a40b3490 con 0x557f8ffe21b0 2024-09-21T06:21:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.461+0000 7f24bf3aa740 10 client.26789 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.461+0000 7f24bf3aa740 10 client.26789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.461+0000 7f24bf3aa740 20 client.26789 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.461+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1387858136 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557f8ffc99f0 0x557f903cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.461+0000 7f24bf3aa740 4 client.26789 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.461+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1387858136 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f9032ada0 con 0x557f8ffc99f0 2024-09-21T06:21:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.461+0000 7f24b1d9b640 1 --2- 172.21.15.3:0/1387858136 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557f8ffc99f0 0x557f903cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.462+0000 7f2499ffb640 10 client.26789 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.462+0000 7f24b1d9b640 1 --2- 172.21.15.3:0/1387858136 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557f8ffc99f0 0x557f903cfb40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237e9ba90c41218f server_cookie=f373fa5f38a909f7 in_seq=0 out_seq=0 2024-09-21T06:21:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f2499ffb640 1 -- 172.21.15.3:0/1387858136 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f9032ada0 con 0x557f8ffc99f0 2024-09-21T06:21:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f2499ffb640 10 client.26789 handle_command_reply: tid=0 2024-09-21T06:21:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f24bf3aa740 1 client.26789 shutdown 2024-09-21T06:21:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1387858136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 msgr2=0x557f903bda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1387858136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 0x557f903bda10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f24a40ad860 tx=0x7f24a40a9d70 comp rx=0 tx=0).stop 2024-09-21T06:21:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1387858136 shutdown_connections 2024-09-21T06:21:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1387858136 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557f8ffc99f0 0x557f903cfb40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1387858136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903bdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1387858136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f9000b070 0x557f903c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1387858136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 0x557f903bda10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f2499ffb640 0 client.26789 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:03.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1387858136 >> 172.21.15.3:0/1387858136 conn(0x557f8ffff460 msgr2=0x557f903b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:03.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1387858136 shutdown_connections 2024-09-21T06:21:03.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1387858136 wait complete. 2024-09-21T06:21:03.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.463+0000 7f24bf3aa740 20 client.26789 trim_cache size 0 max 16384 2024-09-21T06:21:03.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.482+0000 7f24bf3aa740 1 Processor -- start 2024-09-21T06:21:03.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.482+0000 7f24bf3aa740 1 -- start start 2024-09-21T06:21:03.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.482+0000 7f24bf3aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 0x557f903b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.482+0000 7f24bf3aa740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffea390 0x557f903c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.482+0000 7f24bf3aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f9000b070 0x557f903b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.482+0000 7f24bf3aa740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24b41b1940 con 0x557f8ffea390 2024-09-21T06:21:03.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.482+0000 7f24bf3aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24b41b34b0 con 0x557f8ffe21b0 2024-09-21T06:21:03.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.482+0000 7f24bf3aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24b41b2010 con 0x557f9000b070 2024-09-21T06:21:03.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f24b1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffea390 0x557f903c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f24b259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 0x557f903b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f24b2d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f9000b070 0x557f903b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f24b259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 0x557f903b01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:35294/0 (socket says 172.21.15.3:35294) 2024-09-21T06:21:03.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f24b1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffea390 0x557f903c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53182/0 (socket says 172.21.15.3:53182) 2024-09-21T06:21:03.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f24b259c640 1 -- 172.21.15.3:0/1554866191 learned_addr learned my addr 172.21.15.3:0/1554866191 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:03.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f24b259c640 1 -- 172.21.15.3:0/1554866191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f9000b070 msgr2=0x557f903b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f24b259c640 1 --2- 172.21.15.3:0/1554866191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f9000b070 0x557f903b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f24b259c640 1 -- 172.21.15.3:0/1554866191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffea390 msgr2=0x557f903c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f24b259c640 1 --2- 172.21.15.3:0/1554866191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffea390 0x557f903c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f24b259c640 1 -- 172.21.15.3:0/1554866191 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f903b0a50 con 0x557f8ffe21b0 2024-09-21T06:21:03.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f24b2d9d640 1 --2- 172.21.15.3:0/1554866191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f9000b070 0x557f903b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:03.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f24b259c640 1 --2- 172.21.15.3:0/1554866191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 0x557f903b01b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f24a40b3860 tx=0x7f24a4010720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1966d0d6f4c83d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:03.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.483+0000 7f249bfff640 1 -- 172.21.15.3:0/1554866191 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a40a9f20 con 0x557f8ffe21b0 2024-09-21T06:21:03.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.484+0000 7f249bfff640 1 -- 172.21.15.3:0/1554866191 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24a40b7240 con 0x557f8ffe21b0 2024-09-21T06:21:03.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.484+0000 7f249bfff640 1 -- 172.21.15.3:0/1554866191 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a40b69b0 con 0x557f8ffe21b0 2024-09-21T06:21:03.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.485+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1554866191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 msgr2=0x557f903b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.485+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1554866191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 0x557f903b01b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f24a40b3860 tx=0x7f24a4010720 comp rx=0 tx=0).stop 2024-09-21T06:21:03.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.485+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1554866191 shutdown_connections 2024-09-21T06:21:03.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.485+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1554866191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f9000b070 0x557f903b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.485+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1554866191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffea390 0x557f903c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.485+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1554866191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f8ffe21b0 0x557f903b01b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.485+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1554866191 >> 172.21.15.3:0/1554866191 conn(0x557f8ffff460 msgr2=0x557f903afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:03.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.485+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1554866191 shutdown_connections 2024-09-21T06:21:03.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.485+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1554866191 wait complete. 2024-09-21T06:21:03.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24bf3aa740 1 -- start start 2024-09-21T06:21:03.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24bf3aa740 1 Processor -- start 2024-09-21T06:21:03.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24bf3aa740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffe21b0 0x557f903d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24bf3aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24bf3aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f9000b070 0x557f903d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24bf3aa740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24b41b2de0 con 0x557f8ffe21b0 2024-09-21T06:21:03.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24bf3aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24b41b2710 con 0x557f9000b070 2024-09-21T06:21:03.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24bf3aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24b41af920 con 0x557f8ffea390 2024-09-21T06:21:03.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24bf3aa740 10 client.0 _ll_register_callbacks cb 0x7f24a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:03.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24b259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffe21b0 0x557f903d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24b1d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24b259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffe21b0 0x557f903d31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53190/0 (socket says 172.21.15.3:53190) 2024-09-21T06:21:03.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24b1d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:40170/0 (socket says 172.21.15.3:40170) 2024-09-21T06:21:03.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24b259c640 1 -- 172.21.15.3:0/1427359056 learned_addr learned my addr 172.21.15.3:0/1427359056 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:03.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.486+0000 7f24b2d9d640 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f9000b070 0x557f903d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.487+0000 7f24b1d9b640 1 -- 172.21.15.3:0/1427359056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f9000b070 msgr2=0x557f903d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.487+0000 7f24b1d9b640 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f9000b070 0x557f903d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.487+0000 7f24b1d9b640 1 -- 172.21.15.3:0/1427359056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffe21b0 msgr2=0x557f903d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.487+0000 7f24b1d9b640 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffe21b0 0x557f903d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.487+0000 7f24b1d9b640 1 -- 172.21.15.3:0/1427359056 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f903d1220 con 0x557f8ffea390 2024-09-21T06:21:03.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.487+0000 7f24b2d9d640 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f9000b070 0x557f903d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:03.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.487+0000 7f24b259c640 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffe21b0 0x557f903d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:03.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.487+0000 7f24b1d9b640 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903d06a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x557f903b0ef0 tx=0x7f24a8044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf317385126b8666 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.487+0000 7f2499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:21:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.487+0000 7f2499ffb640 1 -- 172.21.15.3:0/1427359056 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8035040 con 0x557f8ffea390 2024-09-21T06:21:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.487+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1427359056 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a40b7240 con 0x557f8ffea390 2024-09-21T06:21:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.487+0000 7f2499ffb640 1 -- 172.21.15.3:0/1427359056 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f24a8005040 con 0x557f8ffea390 2024-09-21T06:21:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.487+0000 7f2499ffb640 1 -- 172.21.15.3:0/1427359056 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f24a8061050 con 0x557f8ffea390 2024-09-21T06:21:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.488+0000 7f24bf3aa740 10 client.26734 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.488+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1427359056 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557f9043a080 con 0x557f8ffea390 2024-09-21T06:21:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.488+0000 7f2499ffb640 1 -- 172.21.15.3:0/1427359056 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24a40b7240 con 0x557f8ffea390 2024-09-21T06:21:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.488+0000 7f2499ffb640 1 -- 172.21.15.3:0/1427359056 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f24a8004ba0 con 0x557f8ffea390 2024-09-21T06:21:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.488+0000 7f24bf3aa740 10 client.26734 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.488+0000 7f24bf3aa740 10 client.26734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.488+0000 7f24bf3aa740 20 client.26734 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:03.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.488+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557f8ff9eda0 0x557f9043c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.488+0000 7f24bf3aa740 4 client.26734 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:21:03.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.488+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1427359056 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f90326850 con 0x557f8ff9eda0 2024-09-21T06:21:03.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.488+0000 7f24b259c640 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557f8ff9eda0 0x557f9043c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.489+0000 7f2499ffb640 10 client.26734 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:03.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.489+0000 7f24b259c640 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557f8ff9eda0 0x557f9043c810 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd9b515b498a26a1 server_cookie=4c2c4d29b04bdf3a in_seq=0 out_seq=0 2024-09-21T06:21:03.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f2499ffb640 1 -- 172.21.15.3:0/1427359056 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x557f90326850 con 0x557f8ff9eda0 2024-09-21T06:21:03.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f2499ffb640 10 client.26734 handle_command_reply: tid=0 2024-09-21T06:21:03.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f24bf3aa740 1 client.26734 shutdown 2024-09-21T06:21:03.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1427359056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 msgr2=0x557f903d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903d06a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x557f903b0ef0 tx=0x7f24a8044000 comp rx=0 tx=0).stop 2024-09-21T06:21:03.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1427359056 shutdown_connections 2024-09-21T06:21:03.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x557f8ff9eda0 0x557f9043c810 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f8ffea390 0x557f903d06a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f9000b070 0x557f903d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f2499ffb640 0 client.26734 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f24bf3aa740 1 --2- 172.21.15.3:0/1427359056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557f8ffe21b0 0x557f903d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1427359056 >> 172.21.15.3:0/1427359056 conn(0x557f8ffff460 msgr2=0x557f903b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1427359056 shutdown_connections 2024-09-21T06:21:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f24bf3aa740 1 -- 172.21.15.3:0/1427359056 wait complete. 2024-09-21T06:21:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.490+0000 7f24bf3aa740 20 client.26734 trim_cache size 0 max 16384 2024-09-21T06:21:03.497 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:21:03.497 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:21:03.497 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:21:03.497 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:21:03.497 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:21:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:21:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:21:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:21:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:03.499 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:03.500 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:03.500 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:03.500 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:03.500 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:21:03.500 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:21:03.500 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:03.500 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:03.500 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:03.500 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:03.500 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:03.500 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:03.500 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:21:03.500 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:03.501 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:03.502 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:03.503 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:21:03.503 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:21:03.503 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:21:03.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.491+0000 7f24b8e36640 1 -- 172.21.15.3:0/1340275843 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f249c0b6780 msgr2=0x7f249c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.491+0000 7f24b8e36640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f249c0b6780 0x7f249c0b8c90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f24a8005f70 tx=0x7f24a8032ed0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.491+0000 7f24b8e36640 1 -- 172.21.15.3:0/1340275843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b4103b90 msgr2=0x7f24b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.491+0000 7f24b8e36640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b4103b90 0x7f24b4075710 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f24a400e9f0 tx=0x7f24a400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.491+0000 7f24b8e36640 1 -- 172.21.15.3:0/1340275843 shutdown_connections 2024-09-21T06:21:03.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.491+0000 7f24b8e36640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f249c0b6780 0x7f249c0b8c90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.491+0000 7f24b8e36640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24b41094d0 0x7f24b4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.491+0000 7f24b8e36640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b4109e20 0x7f24b4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.491+0000 7f24b8e36640 1 --2- 172.21.15.3:0/1340275843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b4103b90 0x7f24b4075710 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.491+0000 7f24b8e36640 1 -- 172.21.15.3:0/1340275843 >> 172.21.15.3:0/1340275843 conn(0x7f24b40ff5e0 msgr2=0x7f24b4101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:03.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.491+0000 7f24b8e36640 1 -- 172.21.15.3:0/1340275843 shutdown_connections 2024-09-21T06:21:03.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:03.491+0000 7f24b8e36640 1 -- 172.21.15.3:0/1340275843 wait complete. 2024-09-21T06:21:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.553+0000 7f6d450f9640 1 -- 172.21.15.3:0/2290090955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d40103ad0 msgr2=0x7f6d40103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.553+0000 7f6d450f9640 1 --2- 172.21.15.3:0/2290090955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d40103ad0 0x7f6d40103ee0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009ca0 tx=0x7f6d3002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.553+0000 7f6d450f9640 1 -- 172.21.15.3:0/2290090955 shutdown_connections 2024-09-21T06:21:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.553+0000 7f6d450f9640 1 --2- 172.21.15.3:0/2290090955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d40109d50 0x7f6d40110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.553+0000 7f6d450f9640 1 --2- 172.21.15.3:0/2290090955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d40109400 0x7f6d40109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.553+0000 7f6d450f9640 1 --2- 172.21.15.3:0/2290090955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d40103ad0 0x7f6d40103ee0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.553+0000 7f6d450f9640 1 -- 172.21.15.3:0/2290090955 >> 172.21.15.3:0/2290090955 conn(0x7f6d400ff510 msgr2=0x7f6d40101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.553+0000 7f6d450f9640 1 -- 172.21.15.3:0/2290090955 shutdown_connections 2024-09-21T06:21:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.553+0000 7f6d450f9640 1 -- 172.21.15.3:0/2290090955 wait complete. 2024-09-21T06:21:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.554+0000 7f6d450f9640 1 Processor -- start 2024-09-21T06:21:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.554+0000 7f6d450f9640 1 -- start start 2024-09-21T06:21:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d450f9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d40103ad0 0x7f6d4019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d450f9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d40109400 0x7f6d4019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d450f9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d40109d50 0x7f6d4019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d450f9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40113a10 con 0x7f6d40109400 2024-09-21T06:21:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d450f9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40113890 con 0x7f6d40109d50 2024-09-21T06:21:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d450f9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40113b90 con 0x7f6d40103ad0 2024-09-21T06:21:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d3e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d40109400 0x7f6d4019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d3e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d40109400 0x7f6d4019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53202/0 (socket says 172.21.15.3:53202) 2024-09-21T06:21:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d3e59c640 1 -- 172.21.15.3:0/1282150387 learned_addr learned my addr 172.21.15.3:0/1282150387 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d3ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d40103ad0 0x7f6d4019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d3f59e640 1 --2- 172.21.15.3:0/1282150387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d40109d50 0x7f6d4019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d3e59c640 1 -- 172.21.15.3:0/1282150387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d40103ad0 msgr2=0x7f6d4019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/1282150387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d40103ad0 0x7f6d4019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d3e59c640 1 -- 172.21.15.3:0/1282150387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d40109d50 msgr2=0x7f6d4019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/1282150387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d40109d50 0x7f6d4019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d3e59c640 1 -- 172.21.15.3:0/1282150387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d401a0380 con 0x7f6d40109400 2024-09-21T06:21:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.555+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/1282150387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d40109400 0x7f6d4019a9f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800e9f0 tx=0x7f6d2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.556+0000 7f6d37fff640 1 -- 172.21.15.3:0/1282150387 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800ccc0 con 0x7f6d40109400 2024-09-21T06:21:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.556+0000 7f6d37fff640 1 -- 172.21.15.3:0/1282150387 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d280108e0 con 0x7f6d40109400 2024-09-21T06:21:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.556+0000 7f6d450f9640 1 -- 172.21.15.3:0/1282150387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d401a0670 con 0x7f6d40109400 2024-09-21T06:21:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.556+0000 7f6d37fff640 1 -- 172.21.15.3:0/1282150387 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28018860 con 0x7f6d40109400 2024-09-21T06:21:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.556+0000 7f6d450f9640 1 -- 172.21.15.3:0/1282150387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d401a6e00 con 0x7f6d40109400 2024-09-21T06:21:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d4c06c740 1 Processor -- start 2024-09-21T06:21:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d4c06c740 1 -- start start 2024-09-21T06:21:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d37fff640 1 -- 172.21.15.3:0/1282150387 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f6d28010430 con 0x7f6d40109400 2024-09-21T06:21:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d4c06c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 0x55b6ea15d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d4c06c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea15def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d4c06c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d974d0 0x55b6ea15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d4c06c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b6ea0d26f0 con 0x55b6e9db81b0 2024-09-21T06:21:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d4c06c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b6ea0c1b20 con 0x55b6e9d974d0 2024-09-21T06:21:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d4c06c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b6ea0c5930 con 0x55b6e9d8f2f0 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d3f59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 0x55b6ea15d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d3f59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 0x55b6ea15d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53216/0 (socket says 172.21.15.3:53216) 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d3ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea15def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d3f59e640 1 -- 172.21.15.3:0/2667509157 learned_addr learned my addr 172.21.15.3:0/2667509157 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.559+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/2667509157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d974d0 0x55b6ea15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.560+0000 7f6d3f59e640 1 -- 172.21.15.3:0/2667509157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d8f2f0 msgr2=0x55b6ea15def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.560+0000 7f6d3f59e640 1 --2- 172.21.15.3:0/2667509157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea15def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.560+0000 7f6d3f59e640 1 -- 172.21.15.3:0/2667509157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d974d0 msgr2=0x55b6ea15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.560+0000 7f6d3f59e640 1 --2- 172.21.15.3:0/2667509157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d974d0 0x55b6ea15e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.560+0000 7f6d3f59e640 1 -- 172.21.15.3:0/2667509157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ea087530 con 0x55b6e9db81b0 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.560+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/2667509157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d974d0 0x55b6ea15e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.560+0000 7f6d3f59e640 1 --2- 172.21.15.3:0/2667509157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 0x55b6ea15d9b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009ca0 tx=0x7f6d38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff0bb78ab87ee0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.560+0000 7f6d357fa640 1 -- 172.21.15.3:0/2667509157 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3801e070 con 0x55b6e9db81b0 2024-09-21T06:21:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.560+0000 7f6d37fff640 1 --2- 172.21.15.3:0/1282150387 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6d2c0b6680 0x7f6d2c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.560+0000 7f6d357fa640 1 -- 172.21.15.3:0/2667509157 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d38009e50 con 0x55b6e9db81b0 2024-09-21T06:21:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.560+0000 7f6d357fa640 1 -- 172.21.15.3:0/2667509157 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3800ea10 con 0x55b6e9db81b0 2024-09-21T06:21:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.560+0000 7f6d3ed9d640 1 --2- 172.21.15.3:0/1282150387 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6d2c0b6680 0x7f6d2c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.561+0000 7f6d37fff640 1 -- 172.21.15.3:0/1282150387 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f6d280e7980 con 0x7f6d40109400 2024-09-21T06:21:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.561+0000 7f6d4c06c740 1 -- 172.21.15.3:0/2667509157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 msgr2=0x55b6ea15d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.561+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/2667509157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 0x55b6ea15d9b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009ca0 tx=0x7f6d38011810 comp rx=0 tx=0).stop 2024-09-21T06:21:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.561+0000 7f6d3ed9d640 1 --2- 172.21.15.3:0/1282150387 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6d2c0b6680 0x7f6d2c0b8b90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55b6ea15c630 tx=0x7f6d30009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.561+0000 7f6d4c06c740 1 -- 172.21.15.3:0/2667509157 shutdown_connections 2024-09-21T06:21:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.561+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/2667509157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea15def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.561+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/2667509157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d974d0 0x55b6ea15e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.561+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/2667509157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 0x55b6ea15d9b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.561+0000 7f6d4c06c740 1 -- 172.21.15.3:0/2667509157 >> 172.21.15.3:0/2667509157 conn(0x55b6e9dac300 msgr2=0x55b6ea15b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.561+0000 7f6d4c06c740 1 -- 172.21.15.3:0/2667509157 shutdown_connections 2024-09-21T06:21:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.561+0000 7f6d4c06c740 1 -- 172.21.15.3:0/2667509157 wait complete. 2024-09-21T06:21:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d4c06c740 1 -- start start 2024-09-21T06:21:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d4c06c740 1 Processor -- start 2024-09-21T06:21:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d4c06c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea16d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d4c06c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d974d0 0x55b6ea16d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d4c06c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 0x55b6ea179870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d4c06c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b6e9ff9ab0 con 0x55b6e9db81b0 2024-09-21T06:21:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d4c06c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b6ea067730 con 0x55b6e9d8f2f0 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d4c06c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b6ea170510 con 0x55b6e9d974d0 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d4c06c740 10 client.0 _ll_register_callbacks cb 0x55b6ea0bb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d3f59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 0x55b6ea179870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d3ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea16d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d3f59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 0x55b6ea179870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53224/0 (socket says 172.21.15.3:53224) 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d3f59e640 1 -- 172.21.15.3:0/1072785211 learned_addr learned my addr 172.21.15.3:0/1072785211 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.562+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d974d0 0x55b6ea16d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d3f59e640 1 -- 172.21.15.3:0/1072785211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d974d0 msgr2=0x55b6ea16d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d3f59e640 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d974d0 0x55b6ea16d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d3f59e640 1 -- 172.21.15.3:0/1072785211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d8f2f0 msgr2=0x55b6ea16d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d3f59e640 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea16d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d3f59e640 1 -- 172.21.15.3:0/1072785211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6e9e23790 con 0x55b6e9db81b0 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d974d0 0x55b6ea16d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d3ed9d640 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea16d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d3f59e640 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 0x55b6ea179870 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6d38005890 tx=0x7f6d3800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eafdc04fc6cdba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d267fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d267fc640 1 -- 172.21.15.3:0/1072785211 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3801e030 con 0x55b6e9db81b0 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d4c06c740 1 -- 172.21.15.3:0/1072785211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d38009e50 con 0x55b6e9db81b0 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d267fc640 1 -- 172.21.15.3:0/1072785211 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f6d38005050 con 0x55b6e9db81b0 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d267fc640 1 -- 172.21.15.3:0/1072785211 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6d38031050 con 0x55b6e9db81b0 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d4c06c740 10 client.17235 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.563+0000 7f6d4c06c740 1 -- 172.21.15.3:0/1072785211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6e9e9c620 con 0x55b6e9db81b0 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.564+0000 7f6d267fc640 1 -- 172.21.15.3:0/1072785211 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d38009e50 con 0x55b6e9db81b0 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.564+0000 7f6d267fc640 1 -- 172.21.15.3:0/1072785211 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f6d38004bb0 con 0x55b6e9db81b0 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.564+0000 7f6d4c06c740 10 client.17235 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.564+0000 7f6d4c06c740 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.564+0000 7f6d4c06c740 20 client.17235 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.564+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6e9d76d80 0x55b6ea17f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.564+0000 7f6d4c06c740 4 client.17235 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.564+0000 7f6d4c06c740 1 -- 172.21.15.3:0/1072785211 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ea0d4640 con 0x55b6e9d76d80 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.564+0000 7f6d3ed9d640 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6e9d76d80 0x55b6ea17f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.565+0000 7f6d267fc640 10 client.17235 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.565+0000 7f6d3ed9d640 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6e9d76d80 0x55b6ea17f480 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eceecc7cafd3841 server_cookie=90dabf31cf1582e in_seq=0 out_seq=0 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.565+0000 7f6d267fc640 1 -- 172.21.15.3:0/1072785211 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6ea0d4640 con 0x55b6e9d76d80 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.565+0000 7f6d267fc640 10 client.17235 handle_command_reply: tid=0 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.565+0000 7f6d4c06c740 1 client.17235 shutdown 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.566+0000 7f6d4c06c740 1 -- 172.21.15.3:0/1072785211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 msgr2=0x55b6ea179870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.566+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 0x55b6ea179870 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6d38005890 tx=0x7f6d3800ea40 comp rx=0 tx=0).stop 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.566+0000 7f6d4c06c740 1 -- 172.21.15.3:0/1072785211 shutdown_connections 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.566+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6e9d76d80 0x55b6ea17f480 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.566+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d974d0 0x55b6ea16d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.566+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9db81b0 0x55b6ea179870 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.566+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/1072785211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea16d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.566+0000 7f6d4c06c740 1 -- 172.21.15.3:0/1072785211 >> 172.21.15.3:0/1072785211 conn(0x55b6e9dac300 msgr2=0x55b6ea15f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.566+0000 7f6d267fc640 0 client.17235 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.566+0000 7f6d4c06c740 1 -- 172.21.15.3:0/1072785211 shutdown_connections 2024-09-21T06:21:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.566+0000 7f6d4c06c740 1 -- 172.21.15.3:0/1072785211 wait complete. 2024-09-21T06:21:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.566+0000 7f6d4c06c740 20 client.17235 trim_cache size 0 max 16384 2024-09-21T06:21:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.581+0000 7f6d4c06c740 1 Processor -- start 2024-09-21T06:21:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.582+0000 7f6d4c06c740 1 -- start start 2024-09-21T06:21:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.582+0000 7f6d4c06c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea168ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.582+0000 7f6d4c06c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d974d0 0x55b6ea16d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.582+0000 7f6d4c06c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9db81b0 0x55b6ea179770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.582+0000 7f6d4c06c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401ad460 con 0x55b6e9d8f2f0 2024-09-21T06:21:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.582+0000 7f6d4c06c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401aefd0 con 0x55b6e9d974d0 2024-09-21T06:21:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.582+0000 7f6d4c06c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401adb30 con 0x55b6e9db81b0 2024-09-21T06:21:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.582+0000 7f6d3ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea168ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.582+0000 7f6d3ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea168ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53228/0 (socket says 172.21.15.3:53228) 2024-09-21T06:21:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.582+0000 7f6d3ed9d640 1 -- 172.21.15.3:0/484676522 learned_addr learned my addr 172.21.15.3:0/484676522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.582+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/484676522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d974d0 0x55b6ea16d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.582+0000 7f6d3f59e640 1 --2- 172.21.15.3:0/484676522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9db81b0 0x55b6ea179770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.583+0000 7f6d3e59c640 1 -- 172.21.15.3:0/484676522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9db81b0 msgr2=0x55b6ea179770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.583+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/484676522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9db81b0 0x55b6ea179770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.583+0000 7f6d3e59c640 1 -- 172.21.15.3:0/484676522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 msgr2=0x55b6ea168ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.583+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/484676522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea168ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.583+0000 7f6d3e59c640 1 -- 172.21.15.3:0/484676522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ea156f50 con 0x55b6e9d974d0 2024-09-21T06:21:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.583+0000 7f6d3ed9d640 1 --2- 172.21.15.3:0/484676522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea168ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.583+0000 7f6d3f59e640 1 --2- 172.21.15.3:0/484676522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9db81b0 0x55b6ea179770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.583+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/484676522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d974d0 0x55b6ea16d8f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6d280e5110 tx=0x7f6d28009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcd4fd6913410be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.583+0000 7f6d34ff9640 1 -- 172.21.15.3:0/484676522 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d280ad030 con 0x55b6e9d974d0 2024-09-21T06:21:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.583+0000 7f6d34ff9640 1 -- 172.21.15.3:0/484676522 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d28010720 con 0x55b6e9d974d0 2024-09-21T06:21:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.583+0000 7f6d34ff9640 1 -- 172.21.15.3:0/484676522 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d280ac880 con 0x55b6e9d974d0 2024-09-21T06:21:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.584+0000 7f6d4c06c740 1 -- 172.21.15.3:0/484676522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d974d0 msgr2=0x55b6ea16d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.584+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/484676522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d974d0 0x55b6ea16d8f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6d280e5110 tx=0x7f6d28009740 comp rx=0 tx=0).stop 2024-09-21T06:21:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.584+0000 7f6d4c06c740 1 -- 172.21.15.3:0/484676522 shutdown_connections 2024-09-21T06:21:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.584+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/484676522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9db81b0 0x55b6ea179770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.584+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/484676522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9d974d0 0x55b6ea16d8f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.584+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/484676522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea168ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.584+0000 7f6d4c06c740 1 -- 172.21.15.3:0/484676522 >> 172.21.15.3:0/484676522 conn(0x55b6e9dac300 msgr2=0x55b6ea16bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.584+0000 7f6d4c06c740 1 -- 172.21.15.3:0/484676522 shutdown_connections 2024-09-21T06:21:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.584+0000 7f6d4c06c740 1 -- 172.21.15.3:0/484676522 wait complete. 2024-09-21T06:21:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d4c06c740 1 -- start start 2024-09-21T06:21:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d4c06c740 1 Processor -- start 2024-09-21T06:21:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d4c06c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea176080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d4c06c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d974d0 0x55b6ea171570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d4c06c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9db81b0 0x55b6ea171b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d4c06c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401ae900 con 0x55b6e9d8f2f0 2024-09-21T06:21:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d4c06c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401ae230 con 0x55b6e9db81b0 2024-09-21T06:21:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d4c06c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401ab440 con 0x55b6e9d974d0 2024-09-21T06:21:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d4c06c740 10 client.0 _ll_register_callbacks cb 0x7f6d3801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d3ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea176080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d3e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d974d0 0x55b6ea171570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d3ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea176080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53236/0 (socket says 172.21.15.3:53236) 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d3ed9d640 1 -- 172.21.15.3:0/3888175458 learned_addr learned my addr 172.21.15.3:0/3888175458 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d3f59e640 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9db81b0 0x55b6ea171b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d3ed9d640 1 -- 172.21.15.3:0/3888175458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d974d0 msgr2=0x55b6ea171570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d3ed9d640 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d974d0 0x55b6ea171570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d3ed9d640 1 -- 172.21.15.3:0/3888175458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9db81b0 msgr2=0x55b6ea171b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.585+0000 7f6d3ed9d640 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9db81b0 0x55b6ea171b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.586+0000 7f6d3ed9d640 1 -- 172.21.15.3:0/3888175458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ea15fee0 con 0x55b6e9d8f2f0 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.586+0000 7f6d3f59e640 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9db81b0 0x55b6ea171b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.586+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d974d0 0x55b6ea171570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.586+0000 7f6d3ed9d640 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea176080 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6d30031780 tx=0x7f6d30041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbac944dcfca3a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.586+0000 7f6d267fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.586+0000 7f6d267fc640 1 -- 172.21.15.3:0/3888175458 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30060060 con 0x55b6e9d8f2f0 2024-09-21T06:21:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.586+0000 7f6d4c06c740 1 -- 172.21.15.3:0/3888175458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6ea0d00f0 con 0x55b6e9d8f2f0 2024-09-21T06:21:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.586+0000 7f6d267fc640 1 -- 172.21.15.3:0/3888175458 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f6d30003970 con 0x55b6e9d8f2f0 2024-09-21T06:21:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.586+0000 7f6d267fc640 1 -- 172.21.15.3:0/3888175458 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6d30003b70 con 0x55b6e9d8f2f0 2024-09-21T06:21:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.586+0000 7f6d4c06c740 10 client.17241 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.586+0000 7f6d4c06c740 1 -- 172.21.15.3:0/3888175458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6ea15e7f0 con 0x55b6e9d8f2f0 2024-09-21T06:21:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.586+0000 7f6d267fc640 1 -- 172.21.15.3:0/3888175458 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b6ea0d00f0 con 0x55b6e9d8f2f0 2024-09-21T06:21:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.587+0000 7f6d267fc640 1 -- 172.21.15.3:0/3888175458 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f6d3002f6f0 con 0x55b6e9d8f2f0 2024-09-21T06:21:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.587+0000 7f6d4c06c740 10 client.17241 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.587+0000 7f6d4c06c740 10 client.17241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.587+0000 7f6d4c06c740 20 client.17241 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.587+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6e9d4c1c0 0x55b6ea1ec150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.587+0000 7f6d4c06c740 4 client.17241 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:21:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.587+0000 7f6d4c06c740 1 -- 172.21.15.3:0/3888175458 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6ea0cd520 con 0x55b6e9d4c1c0 2024-09-21T06:21:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.587+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6e9d4c1c0 0x55b6ea1ec150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.587+0000 7f6d3e59c640 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6e9d4c1c0 0x55b6ea1ec150 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba1ef0b42220bdc8 server_cookie=5e061a20cd23d795 in_seq=0 out_seq=0 2024-09-21T06:21:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.587+0000 7f6d267fc640 10 client.17241 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:03.594 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:03 smithi003 ceph-mon[31227]: pgmap v506: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 368 op/s 2024-09-21T06:21:03.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:03 smithi137 ceph-mon[34464]: pgmap v506: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 368 op/s 2024-09-21T06:21:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:03 smithi204 ceph-mon[34659]: pgmap v506: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 368 op/s 2024-09-21T06:21:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.736+0000 7f6d267fc640 1 -- 172.21.15.3:0/3888175458 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b6ea0cd520 con 0x55b6e9d4c1c0 2024-09-21T06:21:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.736+0000 7f6d267fc640 10 client.17241 handle_command_reply: tid=0 2024-09-21T06:21:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.736+0000 7f6d4c06c740 1 client.17241 shutdown 2024-09-21T06:21:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.736+0000 7f6d4c06c740 1 -- 172.21.15.3:0/3888175458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 msgr2=0x55b6ea176080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.736+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea176080 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6d30031780 tx=0x7f6d30041000 comp rx=0 tx=0).stop 2024-09-21T06:21:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.736+0000 7f6d4c06c740 1 -- 172.21.15.3:0/3888175458 shutdown_connections 2024-09-21T06:21:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.736+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6e9d4c1c0 0x55b6ea1ec150 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.736+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6e9d974d0 0x55b6ea171570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.736+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6e9db81b0 0x55b6ea171b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.736+0000 7f6d267fc640 0 client.17241 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.736+0000 7f6d4c06c740 1 --2- 172.21.15.3:0/3888175458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6e9d8f2f0 0x55b6ea176080 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.736+0000 7f6d4c06c740 1 -- 172.21.15.3:0/3888175458 >> 172.21.15.3:0/3888175458 conn(0x55b6e9dac300 msgr2=0x55b6ea1680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.737+0000 7f6d4c06c740 1 -- 172.21.15.3:0/3888175458 shutdown_connections 2024-09-21T06:21:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.737+0000 7f6d4c06c740 1 -- 172.21.15.3:0/3888175458 wait complete. 2024-09-21T06:21:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.737+0000 7f6d4c06c740 20 client.17241 trim_cache size 0 max 16384 2024-09-21T06:21:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:21:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:21:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:21:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:21:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.738+0000 7f6d450f9640 1 -- 172.21.15.3:0/1282150387 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6d2c0b6680 msgr2=0x7f6d2c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.738+0000 7f6d450f9640 1 --2- 172.21.15.3:0/1282150387 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6d2c0b6680 0x7f6d2c0b8b90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55b6ea15c630 tx=0x7f6d30009360 comp rx=0 tx=0).stop 2024-09-21T06:21:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.738+0000 7f6d450f9640 1 -- 172.21.15.3:0/1282150387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d40109400 msgr2=0x7f6d4019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.738+0000 7f6d450f9640 1 --2- 172.21.15.3:0/1282150387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d40109400 0x7f6d4019a9f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800e9f0 tx=0x7f6d2800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.738+0000 7f6d450f9640 1 -- 172.21.15.3:0/1282150387 shutdown_connections 2024-09-21T06:21:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.738+0000 7f6d450f9640 1 --2- 172.21.15.3:0/1282150387 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f6d2c0b6680 0x7f6d2c0b8b90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.738+0000 7f6d450f9640 1 --2- 172.21.15.3:0/1282150387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d40109d50 0x7f6d4019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.738+0000 7f6d450f9640 1 --2- 172.21.15.3:0/1282150387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d40109400 0x7f6d4019a9f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.738+0000 7f6d450f9640 1 --2- 172.21.15.3:0/1282150387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d40103ad0 0x7f6d4019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.738+0000 7f6d450f9640 1 -- 172.21.15.3:0/1282150387 >> 172.21.15.3:0/1282150387 conn(0x7f6d400ff510 msgr2=0x7f6d40101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.738+0000 7f6d450f9640 1 -- 172.21.15.3:0/1282150387 shutdown_connections 2024-09-21T06:21:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:03.738+0000 7f6d450f9640 1 -- 172.21.15.3:0/1282150387 wait complete. 2024-09-21T06:21:03.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 631237 11.71 MB/sec execute 156 sec latency 57.958 ms 2024-09-21T06:21:04.351 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:21:04.352 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:21:04.642 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:04.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 632947 11.71 MB/sec execute 157 sec latency 43.505 ms 2024-09-21T06:21:05.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.153+0000 7f38555e2640 1 -- 172.21.15.3:0/2796760674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385010ddf0 msgr2=0x7f3850110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:05.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.153+0000 7f38555e2640 1 --2- 172.21.15.3:0/2796760674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385010ddf0 0x7f3850110300 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3848009ca0 tx=0x7f384802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:21:05.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.153+0000 7f38555e2640 1 -- 172.21.15.3:0/2796760674 shutdown_connections 2024-09-21T06:21:05.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.153+0000 7f38555e2640 1 --2- 172.21.15.3:0/2796760674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385010ddf0 0x7f3850110300 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:05.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.153+0000 7f38555e2640 1 --2- 172.21.15.3:0/2796760674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3850105270 0x7f385010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:05.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.153+0000 7f38555e2640 1 --2- 172.21.15.3:0/2796760674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3850104840 0x7f3850104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:05.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.153+0000 7f38555e2640 1 -- 172.21.15.3:0/2796760674 >> 172.21.15.3:0/2796760674 conn(0x7f3850079730 msgr2=0x7f3850079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.153+0000 7f38555e2640 1 -- 172.21.15.3:0/2796760674 shutdown_connections 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.153+0000 7f38555e2640 1 -- 172.21.15.3:0/2796760674 wait complete. 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.154+0000 7f38555e2640 1 Processor -- start 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.154+0000 7f38555e2640 1 -- start start 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38555e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3850104840 0x7f385019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38555e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3850105270 0x7f385019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38555e2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f385010ddf0 0x7f385019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38555e2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3850113a90 con 0x7f3850105270 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38555e2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3850113910 con 0x7f3850104840 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38555e2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3850113c10 con 0x7f385010ddf0 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38467fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3850105270 0x7f385019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38467fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3850105270 0x7f385019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53264/0 (socket says 172.21.15.3:53264) 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38467fc640 1 -- 172.21.15.3:0/841115000 learned_addr learned my addr 172.21.15.3:0/841115000 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f384effd640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3850104840 0x7f385019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f384f7fe640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f385010ddf0 0x7f385019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38467fc640 1 -- 172.21.15.3:0/841115000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f385010ddf0 msgr2=0x7f385019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38467fc640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f385010ddf0 0x7f385019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38467fc640 1 -- 172.21.15.3:0/841115000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3850104840 msgr2=0x7f385019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38467fc640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3850104840 0x7f385019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38467fc640 1 -- 172.21.15.3:0/841115000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38501a03d0 con 0x7f3850105270 2024-09-21T06:21:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.155+0000 7f38467fc640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3850105270 0x7f385019aa00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f383800e9f0 tx=0x7f383800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.156+0000 7f384cff9640 1 -- 172.21.15.3:0/841115000 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383800ccc0 con 0x7f3850105270 2024-09-21T06:21:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.156+0000 7f384cff9640 1 -- 172.21.15.3:0/841115000 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38380108e0 con 0x7f3850105270 2024-09-21T06:21:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.156+0000 7f38555e2640 1 -- 172.21.15.3:0/841115000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38501a06c0 con 0x7f3850105270 2024-09-21T06:21:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.156+0000 7f384cff9640 1 -- 172.21.15.3:0/841115000 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838018860 con 0x7f3850105270 2024-09-21T06:21:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.156+0000 7f38555e2640 1 -- 172.21.15.3:0/841115000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38501a6e20 con 0x7f3850105270 2024-09-21T06:21:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.157+0000 7f38555e2640 1 -- 172.21.15.3:0/841115000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3850104c50 con 0x7f3850105270 2024-09-21T06:21:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.159+0000 7f384cff9640 1 -- 172.21.15.3:0/841115000 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f3838010430 con 0x7f3850105270 2024-09-21T06:21:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.160+0000 7f384cff9640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f383c0b6680 0x7f383c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.161+0000 7f384effd640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f383c0b6680 0x7f383c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.161+0000 7f384cff9640 1 -- 172.21.15.3:0/841115000 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f38380e7e90 con 0x7f3850105270 2024-09-21T06:21:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.161+0000 7f384effd640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f383c0b6680 0x7f383c0b8b90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f385006bc00 tx=0x7f3840005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.163+0000 7f384cff9640 1 -- 172.21.15.3:0/841115000 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f38380b27a0 con 0x7f3850105270 2024-09-21T06:21:05.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.420+0000 7f38555e2640 1 -- 172.21.15.3:0/841115000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38501a7100 con 0x7f3850105270 2024-09-21T06:21:05.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.421+0000 7f384cff9640 1 -- 172.21.15.3:0/841115000 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11963 (secure 0 0 0) 0x7f38380b76e0 con 0x7f3850105270 2024-09-21T06:21:05.424 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:21:05.424 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":32,"btime":"2024-09-21T06:19:41:100740+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:19:40.732187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24670,24674,24679,24691,24704]},"id":1}]} 2024-09-21T06:21:05.424 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 32 2024-09-21T06:21:05.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.424+0000 7f38555e2640 1 -- 172.21.15.3:0/841115000 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f383c0b6680 msgr2=0x7f383c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:05.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.424+0000 7f38555e2640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f383c0b6680 0x7f383c0b8b90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f385006bc00 tx=0x7f3840005f90 comp rx=0 tx=0).stop 2024-09-21T06:21:05.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.424+0000 7f38555e2640 1 -- 172.21.15.3:0/841115000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3850105270 msgr2=0x7f385019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:05.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.424+0000 7f38555e2640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3850105270 0x7f385019aa00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f383800e9f0 tx=0x7f383800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:21:05.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.424+0000 7f38555e2640 1 -- 172.21.15.3:0/841115000 shutdown_connections 2024-09-21T06:21:05.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.424+0000 7f38555e2640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f383c0b6680 0x7f383c0b8b90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:05.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.424+0000 7f38555e2640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f385010ddf0 0x7f385019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:05.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.424+0000 7f38555e2640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3850105270 0x7f385019aa00 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:05.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.424+0000 7f38555e2640 1 --2- 172.21.15.3:0/841115000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3850104840 0x7f385019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:05.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.424+0000 7f38555e2640 1 -- 172.21.15.3:0/841115000 >> 172.21.15.3:0/841115000 conn(0x7f3850079730 msgr2=0x7f3850107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:05.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.424+0000 7f38555e2640 1 -- 172.21.15.3:0/841115000 shutdown_connections 2024-09-21T06:21:05.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:05.424+0000 7f38555e2640 1 -- 172.21.15.3:0/841115000 wait complete. 2024-09-21T06:21:05.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:05 smithi137 ceph-mon[34464]: pgmap v507: 97 pgs: 97 active+clean; 112 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 328 op/s 2024-09-21T06:21:05.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:05 smithi204 ceph-mon[34659]: pgmap v507: 97 pgs: 97 active+clean; 112 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 328 op/s 2024-09-21T06:21:05.805 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:05 smithi003 ceph-mon[31227]: pgmap v507: 97 pgs: 97 active+clean; 112 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 328 op/s 2024-09-21T06:21:05.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 635669 11.74 MB/sec execute 158 sec latency 28.470 ms 2024-09-21T06:21:06.280 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:21:06.551 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:06.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:06 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/841115000' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:21:06.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:06 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/841115000' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:21:06.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:06 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/841115000' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:21:06.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 637205 11.71 MB/sec execute 159 sec latency 35.107 ms 2024-09-21T06:21:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.047+0000 7f93030ab640 1 -- 172.21.15.3:0/3208768097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92fc1094c0 msgr2=0x7f92fc1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.047+0000 7f93030ab640 1 --2- 172.21.15.3:0/3208768097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92fc1094c0 0x7f92fc1098d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009900 tx=0x7f92e802a410 comp rx=0 tx=0).stop 2024-09-21T06:21:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.047+0000 7f93030ab640 1 -- 172.21.15.3:0/3208768097 shutdown_connections 2024-09-21T06:21:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.047+0000 7f93030ab640 1 --2- 172.21.15.3:0/3208768097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92fc109e10 0x7f92fc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.047+0000 7f93030ab640 1 --2- 172.21.15.3:0/3208768097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92fc1094c0 0x7f92fc1098d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.047+0000 7f93030ab640 1 --2- 172.21.15.3:0/3208768097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92fc103b80 0x7f92fc103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.047+0000 7f93030ab640 1 -- 172.21.15.3:0/3208768097 >> 172.21.15.3:0/3208768097 conn(0x7f92fc0ff610 msgr2=0x7f92fc101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.048+0000 7f93030ab640 1 -- 172.21.15.3:0/3208768097 shutdown_connections 2024-09-21T06:21:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.048+0000 7f93030ab640 1 -- 172.21.15.3:0/3208768097 wait complete. 2024-09-21T06:21:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.048+0000 7f93030ab640 1 Processor -- start 2024-09-21T06:21:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.048+0000 7f93030ab640 1 -- start start 2024-09-21T06:21:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f93030ab640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92fc103b80 0x7f92fc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f93030ab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92fc1094c0 0x7f92fc19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f93030ab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92fc109e10 0x7f92fc19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f93030ab640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc113ad0 con 0x7f92fc109e10 2024-09-21T06:21:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f93030ab640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc113950 con 0x7f92fc1094c0 2024-09-21T06:21:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f93030ab640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc113c50 con 0x7f92fc103b80 2024-09-21T06:21:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f9301648640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92fc109e10 0x7f92fc19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f9300e47640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92fc103b80 0x7f92fc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f9301648640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92fc109e10 0x7f92fc19fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39704/0 (socket says 172.21.15.3:39704) 2024-09-21T06:21:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f9301648640 1 -- 172.21.15.3:0/1108128762 learned_addr learned my addr 172.21.15.3:0/1108128762 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:07.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f92fbfff640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92fc1094c0 0x7f92fc19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f9301648640 1 -- 172.21.15.3:0/1108128762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92fc103b80 msgr2=0x7f92fc19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:07.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f9301648640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92fc103b80 0x7f92fc19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f9301648640 1 -- 172.21.15.3:0/1108128762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92fc1094c0 msgr2=0x7f92fc19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:07.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f9301648640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92fc1094c0 0x7f92fc19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f9301648640 1 -- 172.21.15.3:0/1108128762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc1a03f0 con 0x7f92fc109e10 2024-09-21T06:21:07.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.049+0000 7f9300e47640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92fc103b80 0x7f92fc19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:07.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.050+0000 7f9301648640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92fc109e10 0x7f92fc19fcc0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f92f400eb10 tx=0x7f92f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:07.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.050+0000 7f92f9ffb640 1 -- 172.21.15.3:0/1108128762 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4004b20 con 0x7f92fc109e10 2024-09-21T06:21:07.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.050+0000 7f92f9ffb640 1 -- 172.21.15.3:0/1108128762 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f4004cc0 con 0x7f92fc109e10 2024-09-21T06:21:07.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.050+0000 7f93030ab640 1 -- 172.21.15.3:0/1108128762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc1a06e0 con 0x7f92fc109e10 2024-09-21T06:21:07.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.050+0000 7f92f9ffb640 1 -- 172.21.15.3:0/1108128762 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4010670 con 0x7f92fc109e10 2024-09-21T06:21:07.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.050+0000 7f93030ab640 1 -- 172.21.15.3:0/1108128762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc1a6e40 con 0x7f92fc109e10 2024-09-21T06:21:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.053+0000 7f930961f740 1 Processor -- start 2024-09-21T06:21:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.053+0000 7f92f9ffb640 1 -- 172.21.15.3:0/1108128762 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f92f4005020 con 0x7f92fc109e10 2024-09-21T06:21:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.053+0000 7f930961f740 1 -- start start 2024-09-21T06:21:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.053+0000 7f930961f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe302070 0x55abfe6a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.053+0000 7f930961f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2d91b0 0x55abfe6a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.053+0000 7f930961f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2e1390 0x55abfe6aa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.053+0000 7f930961f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55abfe61fea0 con 0x55abfe2e1390 2024-09-21T06:21:07.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.053+0000 7f930961f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55abfe60f2d0 con 0x55abfe302070 2024-09-21T06:21:07.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.053+0000 7f930961f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55abfe6130e0 con 0x55abfe2d91b0 2024-09-21T06:21:07.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe302070 0x55abfe6a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f9300e47640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2d91b0 0x55abfe6a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe302070 0x55abfe6a96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:60926/0 (socket says 172.21.15.3:60926) 2024-09-21T06:21:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f92fbfff640 1 -- 172.21.15.3:0/2643681899 learned_addr learned my addr 172.21.15.3:0/2643681899 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f92fbfff640 1 --2- 172.21.15.3:0/2643681899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2e1390 0x55abfe6aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f92f9ffb640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f92ec0b6680 0x7f92ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f9300e47640 1 -- 172.21.15.3:0/2643681899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe302070 msgr2=0x55abfe6a96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f9300e47640 1 --2- 172.21.15.3:0/2643681899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe302070 0x55abfe6a96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f9300e47640 1 -- 172.21.15.3:0/2643681899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2e1390 msgr2=0x55abfe6aa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f9300e47640 1 --2- 172.21.15.3:0/2643681899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2e1390 0x55abfe6aa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f9300e47640 1 -- 172.21.15.3:0/2643681899 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abfe36d650 con 0x55abfe2d91b0 2024-09-21T06:21:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f92fbfff640 1 --2- 172.21.15.3:0/2643681899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2e1390 0x55abfe6aa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:07.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f9301648640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f92ec0b6680 0x7f92ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f9300e47640 1 --2- 172.21.15.3:0/2643681899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2d91b0 0x55abfe6a9be0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009fd0 tx=0x7f92f00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=563bc04e00aedf99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f92f9ffb640 1 -- 172.21.15.3:0/1108128762 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f92f40e7cb0 con 0x7f92fc109e10 2024-09-21T06:21:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.054+0000 7f92e6ffd640 1 -- 172.21.15.3:0/2643681899 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0004030 con 0x55abfe2d91b0 2024-09-21T06:21:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.055+0000 7f92e6ffd640 1 -- 172.21.15.3:0/2643681899 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f00054d0 con 0x55abfe2d91b0 2024-09-21T06:21:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.055+0000 7f92e6ffd640 1 -- 172.21.15.3:0/2643681899 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0004ce0 con 0x55abfe2d91b0 2024-09-21T06:21:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.055+0000 7f930961f740 1 -- 172.21.15.3:0/2643681899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2d91b0 msgr2=0x55abfe6a9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.055+0000 7f9301648640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f92ec0b6680 0x7f92ec0b8b90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f92f40e54f0 tx=0x7f92f40d1040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.055+0000 7f930961f740 1 --2- 172.21.15.3:0/2643681899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2d91b0 0x55abfe6a9be0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009fd0 tx=0x7f92f00118d0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.056+0000 7f930961f740 1 -- 172.21.15.3:0/2643681899 shutdown_connections 2024-09-21T06:21:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.056+0000 7f930961f740 1 --2- 172.21.15.3:0/2643681899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2d91b0 0x55abfe6a9be0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.056+0000 7f930961f740 1 --2- 172.21.15.3:0/2643681899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2e1390 0x55abfe6aa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.056+0000 7f930961f740 1 --2- 172.21.15.3:0/2643681899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe302070 0x55abfe6a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.056+0000 7f930961f740 1 -- 172.21.15.3:0/2643681899 >> 172.21.15.3:0/2643681899 conn(0x55abfe2f6460 msgr2=0x55abfe6a7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.056+0000 7f930961f740 1 -- 172.21.15.3:0/2643681899 shutdown_connections 2024-09-21T06:21:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.056+0000 7f930961f740 1 -- 172.21.15.3:0/2643681899 wait complete. 2024-09-21T06:21:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.056+0000 7f930961f740 1 -- start start 2024-09-21T06:21:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.056+0000 7f930961f740 1 Processor -- start 2024-09-21T06:21:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f930961f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2d91b0 0x55abfe6b90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f930961f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2e1390 0x55abfe6b9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f930961f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe302070 0x55abfe6b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f930961f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55abfe470700 con 0x55abfe2d91b0 2024-09-21T06:21:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f930961f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55abfe5aa160 con 0x55abfe2e1390 2024-09-21T06:21:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f930961f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55abfe6bc1e0 con 0x55abfe302070 2024-09-21T06:21:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f930961f740 10 client.0 _ll_register_callbacks cb 0x55abfe5c39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2d91b0 0x55abfe6b90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f9300e47640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2e1390 0x55abfe6b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2d91b0 0x55abfe6b90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39714/0 (socket says 172.21.15.3:39714) 2024-09-21T06:21:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f9300e47640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2e1390 0x55abfe6b9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:60942/0 (socket says 172.21.15.3:60942) 2024-09-21T06:21:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f92fbfff640 1 -- 172.21.15.3:0/1162734316 learned_addr learned my addr 172.21.15.3:0/1162734316 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f92fbfff640 1 --2- 172.21.15.3:0/1162734316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe302070 0x55abfe6b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f9300e47640 1 -- 172.21.15.3:0/1162734316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe302070 msgr2=0x55abfe6b9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f9300e47640 1 --2- 172.21.15.3:0/1162734316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe302070 0x55abfe6b9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f9300e47640 1 -- 172.21.15.3:0/1162734316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2d91b0 msgr2=0x55abfe6b90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f9300e47640 1 --2- 172.21.15.3:0/1162734316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2d91b0 0x55abfe6b90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f9300e47640 1 -- 172.21.15.3:0/1162734316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abfe5d7c90 con 0x55abfe2e1390 2024-09-21T06:21:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.057+0000 7f92fbfff640 1 --2- 172.21.15.3:0/1162734316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe302070 0x55abfe6b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f9300e47640 1 --2- 172.21.15.3:0/1162734316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2e1390 0x55abfe6b9600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f92f001b040 tx=0x7f92f00050f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2612373453b5fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f92dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:21:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f92dbfff640 1 -- 172.21.15.3:0/1162734316 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0029020 con 0x55abfe2e1390 2024-09-21T06:21:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f930961f740 1 -- 172.21.15.3:0/1162734316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f00054d0 con 0x55abfe2e1390 2024-09-21T06:21:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f92dbfff640 1 -- 172.21.15.3:0/1162734316 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f92f000f750 con 0x55abfe2e1390 2024-09-21T06:21:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f92dbfff640 1 -- 172.21.15.3:0/1162734316 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f92f00052e0 con 0x55abfe2e1390 2024-09-21T06:21:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f930961f740 10 client.26831 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:07.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f930961f740 1 -- 172.21.15.3:0/1162734316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55abfe6c7eb0 con 0x55abfe2e1390 2024-09-21T06:21:07.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f92dbfff640 1 -- 172.21.15.3:0/1162734316 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f00054d0 con 0x55abfe2e1390 2024-09-21T06:21:07.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f92dbfff640 1 -- 172.21.15.3:0/1162734316 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f92f00256a0 con 0x55abfe2e1390 2024-09-21T06:21:07.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f930961f740 10 client.26831 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:07.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f930961f740 10 client.26831 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:21:07.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f930961f740 20 client.26831 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f930961f740 1 --2- 172.21.15.3:0/1162734316 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55abfe2c09f0 0x55abfe6cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f930961f740 4 client.26831 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.058+0000 7f930961f740 1 -- 172.21.15.3:0/1162734316 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abfe621df0 con 0x55abfe2c09f0 2024-09-21T06:21:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.059+0000 7f92fbfff640 1 --2- 172.21.15.3:0/1162734316 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55abfe2c09f0 0x55abfe6cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.059+0000 7f92fbfff640 1 --2- 172.21.15.3:0/1162734316 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55abfe2c09f0 0x55abfe6cb320 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8e40dc814f985f65 server_cookie=c911fb2052b755fc in_seq=0 out_seq=0 2024-09-21T06:21:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.059+0000 7f92dbfff640 10 client.26831 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:21:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f92dbfff640 1 -- 172.21.15.3:0/1162734316 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55abfe621df0 con 0x55abfe2c09f0 2024-09-21T06:21:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f92dbfff640 10 client.26831 handle_command_reply: tid=0 2024-09-21T06:21:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f930961f740 1 client.26831 shutdown 2024-09-21T06:21:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f930961f740 1 -- 172.21.15.3:0/1162734316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2e1390 msgr2=0x55abfe6b9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f930961f740 1 --2- 172.21.15.3:0/1162734316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2e1390 0x55abfe6b9600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f92f001b040 tx=0x7f92f00050f0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f930961f740 1 -- 172.21.15.3:0/1162734316 shutdown_connections 2024-09-21T06:21:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f930961f740 1 --2- 172.21.15.3:0/1162734316 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55abfe2c09f0 0x55abfe6cb320 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f930961f740 1 --2- 172.21.15.3:0/1162734316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe302070 0x55abfe6b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f930961f740 1 --2- 172.21.15.3:0/1162734316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2e1390 0x55abfe6b9600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f92dbfff640 0 client.26831 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:21:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f930961f740 1 --2- 172.21.15.3:0/1162734316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2d91b0 0x55abfe6b90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f930961f740 1 -- 172.21.15.3:0/1162734316 >> 172.21.15.3:0/1162734316 conn(0x55abfe2f6460 msgr2=0x55abfe6ab090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f930961f740 1 -- 172.21.15.3:0/1162734316 shutdown_connections 2024-09-21T06:21:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f930961f740 1 -- 172.21.15.3:0/1162734316 wait complete. 2024-09-21T06:21:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.060+0000 7f930961f740 20 client.26831 trim_cache size 0 max 16384 2024-09-21T06:21:07.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.074+0000 7f930961f740 1 Processor -- start 2024-09-21T06:21:07.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.074+0000 7f930961f740 1 -- start start 2024-09-21T06:21:07.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f930961f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2c09f0 0x55abfe6bae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f930961f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2d91b0 0x55abfe6b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f930961f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2e1390 0x55abfe6b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f930961f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1ad400 con 0x55abfe2d91b0 2024-09-21T06:21:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f930961f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1aef70 con 0x55abfe2c09f0 2024-09-21T06:21:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f930961f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1adad0 con 0x55abfe2e1390 2024-09-21T06:21:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f9300e47640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2d91b0 0x55abfe6b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f9300e47640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2d91b0 0x55abfe6b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39720/0 (socket says 172.21.15.3:39720) 2024-09-21T06:21:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f9300e47640 1 -- 172.21.15.3:0/432901805 learned_addr learned my addr 172.21.15.3:0/432901805 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f92fbfff640 1 --2- 172.21.15.3:0/432901805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2e1390 0x55abfe6b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f92fbfff640 1 --2- 172.21.15.3:0/432901805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2c09f0 0x55abfe6bae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f9300e47640 1 -- 172.21.15.3:0/432901805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2e1390 msgr2=0x55abfe6b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f9300e47640 1 --2- 172.21.15.3:0/432901805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2e1390 0x55abfe6b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f9300e47640 1 -- 172.21.15.3:0/432901805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2c09f0 msgr2=0x55abfe6bae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f9300e47640 1 --2- 172.21.15.3:0/432901805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2c09f0 0x55abfe6bae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f9300e47640 1 -- 172.21.15.3:0/432901805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abfe6cb690 con 0x55abfe2d91b0 2024-09-21T06:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f9300e47640 1 --2- 172.21.15.3:0/432901805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2d91b0 0x55abfe6b62b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f00096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ff6b2a51ae2c86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.075+0000 7f92e67fc640 1 -- 172.21.15.3:0/432901805 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0029040 con 0x55abfe2d91b0 2024-09-21T06:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.076+0000 7f92e67fc640 1 -- 172.21.15.3:0/432901805 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f0005680 con 0x55abfe2d91b0 2024-09-21T06:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.076+0000 7f92e67fc640 1 -- 172.21.15.3:0/432901805 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f000f750 con 0x55abfe2d91b0 2024-09-21T06:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.076+0000 7f930961f740 1 -- 172.21.15.3:0/432901805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2d91b0 msgr2=0x55abfe6b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.076+0000 7f930961f740 1 --2- 172.21.15.3:0/432901805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2d91b0 0x55abfe6b62b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f00096e0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.076+0000 7f930961f740 1 -- 172.21.15.3:0/432901805 shutdown_connections 2024-09-21T06:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.076+0000 7f930961f740 1 --2- 172.21.15.3:0/432901805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2e1390 0x55abfe6b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.076+0000 7f930961f740 1 --2- 172.21.15.3:0/432901805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2d91b0 0x55abfe6b62b0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.076+0000 7f930961f740 1 --2- 172.21.15.3:0/432901805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2c09f0 0x55abfe6bae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.076+0000 7f930961f740 1 -- 172.21.15.3:0/432901805 >> 172.21.15.3:0/432901805 conn(0x55abfe2f6460 msgr2=0x55abfe6ab090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.076+0000 7f930961f740 1 -- 172.21.15.3:0/432901805 shutdown_connections 2024-09-21T06:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.077+0000 7f930961f740 1 -- 172.21.15.3:0/432901805 wait complete. 2024-09-21T06:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.077+0000 7f930961f740 1 -- start start 2024-09-21T06:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.077+0000 7f930961f740 1 Processor -- start 2024-09-21T06:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.077+0000 7f930961f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2c09f0 0x55abfe6c1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.077+0000 7f930961f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2d91b0 0x55abfe6c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.077+0000 7f930961f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2e1390 0x55abfe6bcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.077+0000 7f930961f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1ae8a0 con 0x55abfe2e1390 2024-09-21T06:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.077+0000 7f930961f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1ae1d0 con 0x55abfe2d91b0 2024-09-21T06:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.077+0000 7f930961f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1ab3e0 con 0x55abfe2c09f0 2024-09-21T06:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f930961f740 10 client.0 _ll_register_callbacks cb 0x7f92f0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2c09f0 0x55abfe6c1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2c09f0 0x55abfe6c1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:57858/0 (socket says 172.21.15.3:57858) 2024-09-21T06:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f92fbfff640 1 -- 172.21.15.3:0/193769876 learned_addr learned my addr 172.21.15.3:0/193769876 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f9300e47640 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2d91b0 0x55abfe6c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f92fbfff640 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2e1390 0x55abfe6bcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f9300e47640 1 -- 172.21.15.3:0/193769876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2c09f0 msgr2=0x55abfe6c1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f9300e47640 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2c09f0 0x55abfe6c1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f9300e47640 1 -- 172.21.15.3:0/193769876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2e1390 msgr2=0x55abfe6bcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f9300e47640 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2e1390 0x55abfe6bcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f9300e47640 1 -- 172.21.15.3:0/193769876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abfe6bd6d0 con 0x55abfe2d91b0 2024-09-21T06:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f92fbfff640 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2c09f0 0x55abfe6c1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f9300e47640 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2d91b0 0x55abfe6c1ec0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f92f0025c30 tx=0x7f92f00256a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79581eeb2b7f7a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.078+0000 7f92dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f92dbfff640 1 -- 172.21.15.3:0/193769876 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0029040 con 0x55abfe2d91b0 2024-09-21T06:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f92dbfff640 1 -- 172.21.15.3:0/193769876 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f92f000f960 con 0x55abfe2d91b0 2024-09-21T06:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f930961f740 1 -- 172.21.15.3:0/193769876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f0005680 con 0x55abfe2d91b0 2024-09-21T06:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f92dbfff640 1 -- 172.21.15.3:0/193769876 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f92f000fba0 con 0x55abfe2d91b0 2024-09-21T06:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f930961f740 10 client.26837 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f930961f740 1 -- 172.21.15.3:0/193769876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55abfe735660 con 0x55abfe2d91b0 2024-09-21T06:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f92dbfff640 1 -- 172.21.15.3:0/193769876 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f003c070 con 0x55abfe2d91b0 2024-09-21T06:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f92dbfff640 1 -- 172.21.15.3:0/193769876 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f92f0019af0 con 0x55abfe2d91b0 2024-09-21T06:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f930961f740 10 client.26837 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f930961f740 10 client.26837 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f930961f740 20 client.26837 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f930961f740 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55abfe302070 0x55abfe737ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f930961f740 4 client.26837 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.079+0000 7f930961f740 1 -- 172.21.15.3:0/193769876 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f92f0005680 con 0x55abfe302070 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f92fbfff640 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55abfe302070 0x55abfe737ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f92dbfff640 10 client.26837 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f92fbfff640 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55abfe302070 0x55abfe737ca0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c41be9c878b6871b server_cookie=86dda257e6264b52 in_seq=0 out_seq=0 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f92dbfff640 1 -- 172.21.15.3:0/193769876 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f92f0005680 con 0x55abfe302070 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f92dbfff640 10 client.26837 handle_command_reply: tid=0 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f930961f740 1 client.26837 shutdown 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f930961f740 1 -- 172.21.15.3:0/193769876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2d91b0 msgr2=0x55abfe6c1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f930961f740 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2d91b0 0x55abfe6c1ec0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f92f0025c30 tx=0x7f92f00256a0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f930961f740 1 -- 172.21.15.3:0/193769876 shutdown_connections 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f930961f740 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55abfe302070 0x55abfe737ca0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f930961f740 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abfe2e1390 0x55abfe6bcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f92dbfff640 0 client.26837 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f930961f740 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abfe2d91b0 0x55abfe6c1ec0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.080+0000 7f930961f740 1 --2- 172.21.15.3:0/193769876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55abfe2c09f0 0x55abfe6c1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f930961f740 1 -- 172.21.15.3:0/193769876 >> 172.21.15.3:0/193769876 conn(0x55abfe2f6460 msgr2=0x55abfe6ab090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f930961f740 1 -- 172.21.15.3:0/193769876 shutdown_connections 2024-09-21T06:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f930961f740 1 -- 172.21.15.3:0/193769876 wait complete. 2024-09-21T06:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f930961f740 20 client.26837 trim_cache size 0 max 16384 2024-09-21T06:21:07.100 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f93030ab640 1 -- 172.21.15.3:0/1108128762 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f92ec0b6680 msgr2=0x7f92ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f93030ab640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f92ec0b6680 0x7f92ec0b8b90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f92f40e54f0 tx=0x7f92f40d1040 comp rx=0 tx=0).stop 2024-09-21T06:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f93030ab640 1 -- 172.21.15.3:0/1108128762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92fc109e10 msgr2=0x7f92fc19fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f93030ab640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92fc109e10 0x7f92fc19fcc0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f92f400eb10 tx=0x7f92f400c2d0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f93030ab640 1 -- 172.21.15.3:0/1108128762 shutdown_connections 2024-09-21T06:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f93030ab640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f92ec0b6680 0x7f92ec0b8b90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f93030ab640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92fc109e10 0x7f92fc19fcc0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f93030ab640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92fc1094c0 0x7f92fc19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f93030ab640 1 --2- 172.21.15.3:0/1108128762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92fc103b80 0x7f92fc19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.081+0000 7f93030ab640 1 -- 172.21.15.3:0/1108128762 >> 172.21.15.3:0/1108128762 conn(0x7f92fc0ff610 msgr2=0x7f92fc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.082+0000 7f93030ab640 1 -- 172.21.15.3:0/1108128762 shutdown_connections 2024-09-21T06:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:07.082+0000 7f93030ab640 1 -- 172.21.15.3:0/1108128762 wait complete. 2024-09-21T06:21:07.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:07 smithi137 ceph-mon[34464]: pgmap v508: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 354 op/s 2024-09-21T06:21:07.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:07 smithi204 ceph-mon[34659]: pgmap v508: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 354 op/s 2024-09-21T06:21:07.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:07 smithi003 ceph-mon[31227]: pgmap v508: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 354 op/s 2024-09-21T06:21:07.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 639534 11.72 MB/sec execute 160 sec latency 22.736 ms 2024-09-21T06:21:08.324 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:21:08.609 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:08.959 INFO:tasks.workunit.client.0.smithi003.stdout: 1 641127 11.69 MB/sec execute 161 sec latency 46.226 ms 2024-09-21T06:21:09.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.130+0000 7fc1ebaf0640 1 -- 172.21.15.3:0/1537682046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e4108cf0 msgr2=0x7fc1e410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:09.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.130+0000 7fc1ebaf0640 1 --2- 172.21.15.3:0/1537682046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e4108cf0 0x7fc1e410f520 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009ca0 tx=0x7fc1dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.130+0000 7fc1ebaf0640 1 -- 172.21.15.3:0/1537682046 shutdown_connections 2024-09-21T06:21:09.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.130+0000 7fc1ebaf0640 1 --2- 172.21.15.3:0/1537682046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e4108cf0 0x7fc1e410f520 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.130+0000 7fc1ebaf0640 1 --2- 172.21.15.3:0/1537682046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1e41083a0 0x7fc1e41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.130+0000 7fc1ebaf0640 1 --2- 172.21.15.3:0/1537682046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e4102a60 0x7fc1e4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.130+0000 7fc1ebaf0640 1 -- 172.21.15.3:0/1537682046 >> 172.21.15.3:0/1537682046 conn(0x7fc1e4079730 msgr2=0x7fc1e4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:09.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.130+0000 7fc1ebaf0640 1 -- 172.21.15.3:0/1537682046 shutdown_connections 2024-09-21T06:21:09.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.130+0000 7fc1ebaf0640 1 -- 172.21.15.3:0/1537682046 wait complete. 2024-09-21T06:21:09.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.131+0000 7fc1ebaf0640 1 Processor -- start 2024-09-21T06:21:09.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.131+0000 7fc1ebaf0640 1 -- start start 2024-09-21T06:21:09.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ebaf0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e4102a60 0x7fc1e419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ebaf0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e41083a0 0x7fc1e419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ebaf0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1e4108cf0 0x7fc1e419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ebaf0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4112960 con 0x7fc1e4102a60 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ebaf0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e41127e0 con 0x7fc1e4108cf0 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ebaf0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4112ae0 con 0x7fc1e41083a0 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ea08d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1e4108cf0 0x7fc1e419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ea08d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1e4108cf0 0x7fc1e419fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:60980/0 (socket says 172.21.15.3:60980) 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ea08d640 1 -- 172.21.15.3:0/623065021 learned_addr learned my addr 172.21.15.3:0/623065021 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1e988c640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e4102a60 0x7fc1e419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1e908b640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e41083a0 0x7fc1e419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ea08d640 1 -- 172.21.15.3:0/623065021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e41083a0 msgr2=0x7fc1e419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ea08d640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e41083a0 0x7fc1e419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ea08d640 1 -- 172.21.15.3:0/623065021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e4102a60 msgr2=0x7fc1e419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ea08d640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e4102a60 0x7fc1e419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.132+0000 7fc1ea08d640 1 -- 172.21.15.3:0/623065021 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e41a0400 con 0x7fc1e4108cf0 2024-09-21T06:21:09.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.133+0000 7fc1ea08d640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1e4108cf0 0x7fc1e419fd00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc005cc0 tx=0x7fc1dc031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:09.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.133+0000 7fc1daffd640 1 -- 172.21.15.3:0/623065021 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc003b70 con 0x7fc1e4108cf0 2024-09-21T06:21:09.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.133+0000 7fc1daffd640 1 -- 172.21.15.3:0/623065021 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1dc0318f0 con 0x7fc1e4108cf0 2024-09-21T06:21:09.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.133+0000 7fc1ebaf0640 1 -- 172.21.15.3:0/623065021 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e41a0690 con 0x7fc1e4108cf0 2024-09-21T06:21:09.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.133+0000 7fc1daffd640 1 -- 172.21.15.3:0/623065021 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc02f950 con 0x7fc1e4108cf0 2024-09-21T06:21:09.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.133+0000 7fc1ebaf0640 1 -- 172.21.15.3:0/623065021 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e406cde0 con 0x7fc1e4108cf0 2024-09-21T06:21:09.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.136+0000 7fc1f2064740 1 Processor -- start 2024-09-21T06:21:09.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.136+0000 7fc1daffd640 1 -- 172.21.15.3:0/623065021 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fc1dc02faf0 con 0x7fc1e4108cf0 2024-09-21T06:21:09.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1f2064740 1 -- start start 2024-09-21T06:21:09.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1f2064740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562553a1e070 0x562553dc56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1f2064740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 0x562553dc5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1f2064740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539fd390 0x562553dc6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1f2064740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562553d3bea0 con 0x5625539fd390 2024-09-21T06:21:09.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1f2064740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562553d2b2d0 con 0x5625539f51b0 2024-09-21T06:21:09.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1f2064740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562553d2f0e0 con 0x562553a1e070 2024-09-21T06:21:09.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1e988c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 0x562553dc5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1e908b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539fd390 0x562553dc6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1e988c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 0x562553dc5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:60992/0 (socket says 172.21.15.3:60992) 2024-09-21T06:21:09.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1e908b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539fd390 0x562553dc6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39744/0 (socket says 172.21.15.3:39744) 2024-09-21T06:21:09.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1e988c640 1 -- 172.21.15.3:0/3658225869 learned_addr learned my addr 172.21.15.3:0/3658225869 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:09.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1e908b640 1 --2- 172.21.15.3:0/3658225869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562553a1e070 0x562553dc56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.137+0000 7fc1daffd640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc1d00b66b0 0x7fc1d00b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.138+0000 7fc1e988c640 1 -- 172.21.15.3:0/3658225869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562553a1e070 msgr2=0x562553dc56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:09.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.138+0000 7fc1e988c640 1 --2- 172.21.15.3:0/3658225869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562553a1e070 0x562553dc56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.138+0000 7fc1e988c640 1 -- 172.21.15.3:0/3658225869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539fd390 msgr2=0x562553dc6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:09.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.138+0000 7fc1e988c640 1 --2- 172.21.15.3:0/3658225869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539fd390 0x562553dc6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.138+0000 7fc1e988c640 1 -- 172.21.15.3:0/3658225869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562553a89650 con 0x5625539f51b0 2024-09-21T06:21:09.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.138+0000 7fc1ea08d640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc1d00b66b0 0x7fc1d00b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.138+0000 7fc1e908b640 1 --2- 172.21.15.3:0/3658225869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562553a1e070 0x562553dc56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:09.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.138+0000 7fc1daffd640 1 -- 172.21.15.3:0/623065021 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fc1dc108840 con 0x7fc1e4108cf0 2024-09-21T06:21:09.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.138+0000 7fc1e988c640 1 --2- 172.21.15.3:0/3658225869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 0x562553dc5be0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009cf0 tx=0x7fc1e0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41ac680db516465d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:09.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.138+0000 7fc1cffff640 1 -- 172.21.15.3:0/3658225869 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0004230 con 0x5625539f51b0 2024-09-21T06:21:09.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.138+0000 7fc1cffff640 1 -- 172.21.15.3:0/3658225869 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e00043d0 con 0x5625539f51b0 2024-09-21T06:21:09.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.138+0000 7fc1cffff640 1 -- 172.21.15.3:0/3658225869 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0019880 con 0x5625539f51b0 2024-09-21T06:21:09.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.138+0000 7fc1ea08d640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc1d00b66b0 0x7fc1d00b8bc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc1065d0 tx=0x7fc1dc02a8c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:09.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.139+0000 7fc1f2064740 1 -- 172.21.15.3:0/3658225869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 msgr2=0x562553dc5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:09.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.139+0000 7fc1f2064740 1 --2- 172.21.15.3:0/3658225869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 0x562553dc5be0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009cf0 tx=0x7fc1e0011810 comp rx=0 tx=0).stop 2024-09-21T06:21:09.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.139+0000 7fc1f2064740 1 -- 172.21.15.3:0/3658225869 shutdown_connections 2024-09-21T06:21:09.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.139+0000 7fc1f2064740 1 --2- 172.21.15.3:0/3658225869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539fd390 0x562553dc6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.139+0000 7fc1f2064740 1 --2- 172.21.15.3:0/3658225869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 0x562553dc5be0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.139+0000 7fc1f2064740 1 --2- 172.21.15.3:0/3658225869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562553a1e070 0x562553dc56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.139+0000 7fc1f2064740 1 -- 172.21.15.3:0/3658225869 >> 172.21.15.3:0/3658225869 conn(0x562553a12460 msgr2=0x562553dc3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:09.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.139+0000 7fc1f2064740 1 -- 172.21.15.3:0/3658225869 shutdown_connections 2024-09-21T06:21:09.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.139+0000 7fc1f2064740 1 -- 172.21.15.3:0/3658225869 wait complete. 2024-09-21T06:21:09.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.140+0000 7fc1f2064740 1 -- start start 2024-09-21T06:21:09.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.140+0000 7fc1f2064740 1 Processor -- start 2024-09-21T06:21:09.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1f2064740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539f51b0 0x562553dd0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1f2064740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539fd390 0x562553dd11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1f2064740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562553a1e070 0x562553dd1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1f2064740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562553b8c700 con 0x5625539f51b0 2024-09-21T06:21:09.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1f2064740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562553cc6160 con 0x5625539fd390 2024-09-21T06:21:09.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1f2064740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562553dd3e80 con 0x562553a1e070 2024-09-21T06:21:09.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1f2064740 10 client.0 _ll_register_callbacks cb 0x562553cdf9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1e988c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539fd390 0x562553dd11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1e908b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539f51b0 0x562553dd0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1e988c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539fd390 0x562553dd11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:32770/0 (socket says 172.21.15.3:32770) 2024-09-21T06:21:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1e988c640 1 -- 172.21.15.3:0/2306119524 learned_addr learned my addr 172.21.15.3:0/2306119524 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1e908b640 1 --2- 172.21.15.3:0/2306119524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562553a1e070 0x562553dd1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1e908b640 1 -- 172.21.15.3:0/2306119524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562553a1e070 msgr2=0x562553dd1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1e908b640 1 --2- 172.21.15.3:0/2306119524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562553a1e070 0x562553dd1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1e908b640 1 -- 172.21.15.3:0/2306119524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539fd390 msgr2=0x562553dd11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1e908b640 1 --2- 172.21.15.3:0/2306119524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539fd390 0x562553dd11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.141+0000 7fc1e908b640 1 -- 172.21.15.3:0/2306119524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562553cf3c90 con 0x5625539f51b0 2024-09-21T06:21:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.142+0000 7fc1e908b640 1 --2- 172.21.15.3:0/2306119524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539f51b0 0x562553dd0c90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc1e406aaa0 tx=0x7fc1d4009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74d1a4650bf29a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.142+0000 7fc1cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.142+0000 7fc1cd7fa640 1 -- 172.21.15.3:0/2306119524 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4013170 con 0x5625539f51b0 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.142+0000 7fc1cd7fa640 1 -- 172.21.15.3:0/2306119524 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fc1d4009040 con 0x5625539f51b0 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.142+0000 7fc1f2064740 1 -- 172.21.15.3:0/2306119524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1e00043d0 con 0x5625539f51b0 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.142+0000 7fc1cd7fa640 1 -- 172.21.15.3:0/2306119524 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc1d400ed40 con 0x5625539f51b0 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.142+0000 7fc1f2064740 10 client.17277 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.142+0000 7fc1f2064740 1 -- 172.21.15.3:0/2306119524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562553ddfb30 con 0x5625539f51b0 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.142+0000 7fc1cd7fa640 1 -- 172.21.15.3:0/2306119524 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e00043d0 con 0x5625539f51b0 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.143+0000 7fc1cd7fa640 1 -- 172.21.15.3:0/2306119524 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fc1d4029440 con 0x5625539f51b0 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.143+0000 7fc1f2064740 10 client.17277 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.143+0000 7fc1f2064740 10 client.17277 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.143+0000 7fc1f2064740 20 client.17277 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.143+0000 7fc1f2064740 1 --2- 172.21.15.3:0/2306119524 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5625539dc9f0 0x562553de2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.143+0000 7fc1f2064740 4 client.17277 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.143+0000 7fc1f2064740 1 -- 172.21.15.3:0/2306119524 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562553d3ddf0 con 0x5625539dc9f0 2024-09-21T06:21:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.143+0000 7fc1e988c640 1 --2- 172.21.15.3:0/2306119524 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5625539dc9f0 0x562553de2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.143+0000 7fc1cd7fa640 10 client.17277 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:21:09.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.143+0000 7fc1e988c640 1 --2- 172.21.15.3:0/2306119524 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5625539dc9f0 0x562553de2f50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=100cba3426f6c125 server_cookie=d2874a6896326425 in_seq=0 out_seq=0 2024-09-21T06:21:09.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.144+0000 7fc1cd7fa640 1 -- 172.21.15.3:0/2306119524 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562553d3ddf0 con 0x5625539dc9f0 2024-09-21T06:21:09.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.144+0000 7fc1cd7fa640 10 client.17277 handle_command_reply: tid=0 2024-09-21T06:21:09.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.145+0000 7fc1f2064740 1 client.17277 shutdown 2024-09-21T06:21:09.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.145+0000 7fc1f2064740 1 -- 172.21.15.3:0/2306119524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539f51b0 msgr2=0x562553dd0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:09.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.145+0000 7fc1f2064740 1 --2- 172.21.15.3:0/2306119524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539f51b0 0x562553dd0c90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc1e406aaa0 tx=0x7fc1d4009a40 comp rx=0 tx=0).stop 2024-09-21T06:21:09.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.145+0000 7fc1f2064740 1 -- 172.21.15.3:0/2306119524 shutdown_connections 2024-09-21T06:21:09.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.145+0000 7fc1f2064740 1 --2- 172.21.15.3:0/2306119524 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5625539dc9f0 0x562553de2f50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.145+0000 7fc1f2064740 1 --2- 172.21.15.3:0/2306119524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562553a1e070 0x562553dd1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.145+0000 7fc1f2064740 1 --2- 172.21.15.3:0/2306119524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539fd390 0x562553dd11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.145+0000 7fc1cd7fa640 0 client.17277 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:21:09.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.145+0000 7fc1f2064740 1 --2- 172.21.15.3:0/2306119524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539f51b0 0x562553dd0c90 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.145+0000 7fc1f2064740 1 -- 172.21.15.3:0/2306119524 >> 172.21.15.3:0/2306119524 conn(0x562553a12460 msgr2=0x562553dc8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:09.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.145+0000 7fc1f2064740 1 -- 172.21.15.3:0/2306119524 shutdown_connections 2024-09-21T06:21:09.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.145+0000 7fc1f2064740 1 -- 172.21.15.3:0/2306119524 wait complete. 2024-09-21T06:21:09.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.145+0000 7fc1f2064740 20 client.17277 trim_cache size 0 max 16384 2024-09-21T06:21:09.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.158+0000 7fc1f2064740 1 Processor -- start 2024-09-21T06:21:09.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1f2064740 1 -- start start 2024-09-21T06:21:09.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1f2064740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539dc9f0 0x562553dc7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1f2064740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539f51b0 0x562553dd28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1f2064740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625539fd390 0x562553dc2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1f2064740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e41b7f10 con 0x5625539f51b0 2024-09-21T06:21:09.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1f2064740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e41b9a60 con 0x5625539dc9f0 2024-09-21T06:21:09.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1f2064740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e41b85c0 con 0x5625539fd390 2024-09-21T06:21:09.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1e988c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539f51b0 0x562553dd28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1e908b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539dc9f0 0x562553dc7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1e988c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539f51b0 0x562553dd28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39762/0 (socket says 172.21.15.3:39762) 2024-09-21T06:21:09.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1e988c640 1 -- 172.21.15.3:0/1323009493 learned_addr learned my addr 172.21.15.3:0/1323009493 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:09.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1e908b640 1 --2- 172.21.15.3:0/1323009493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625539fd390 0x562553dc2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1e988c640 1 -- 172.21.15.3:0/1323009493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625539fd390 msgr2=0x562553dc2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:09.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1e988c640 1 --2- 172.21.15.3:0/1323009493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625539fd390 0x562553dc2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1e988c640 1 -- 172.21.15.3:0/1323009493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539dc9f0 msgr2=0x562553dc7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:09.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1e988c640 1 --2- 172.21.15.3:0/1323009493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539dc9f0 0x562553dc7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1e988c640 1 -- 172.21.15.3:0/1323009493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562553dc8500 con 0x5625539f51b0 2024-09-21T06:21:09.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.159+0000 7fc1e908b640 1 --2- 172.21.15.3:0/1323009493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539dc9f0 0x562553dc7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:09.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.160+0000 7fc1e988c640 1 --2- 172.21.15.3:0/1323009493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539f51b0 0x562553dd28c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009250 tx=0x7fc1e0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf40d2c17498c1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:09.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.160+0000 7fc1cf7fe640 1 -- 172.21.15.3:0/1323009493 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0005320 con 0x5625539f51b0 2024-09-21T06:21:09.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.160+0000 7fc1cf7fe640 1 -- 172.21.15.3:0/1323009493 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e00054c0 con 0x5625539f51b0 2024-09-21T06:21:09.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.160+0000 7fc1cf7fe640 1 -- 172.21.15.3:0/1323009493 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e00197d0 con 0x5625539f51b0 2024-09-21T06:21:09.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.161+0000 7fc1f2064740 1 -- 172.21.15.3:0/1323009493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539f51b0 msgr2=0x562553dd28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:09.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.161+0000 7fc1f2064740 1 --2- 172.21.15.3:0/1323009493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539f51b0 0x562553dd28c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009250 tx=0x7fc1e0011810 comp rx=0 tx=0).stop 2024-09-21T06:21:09.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.161+0000 7fc1f2064740 1 -- 172.21.15.3:0/1323009493 shutdown_connections 2024-09-21T06:21:09.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.161+0000 7fc1f2064740 1 --2- 172.21.15.3:0/1323009493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625539fd390 0x562553dc2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.161+0000 7fc1f2064740 1 --2- 172.21.15.3:0/1323009493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539f51b0 0x562553dd28c0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.161+0000 7fc1f2064740 1 --2- 172.21.15.3:0/1323009493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539dc9f0 0x562553dc7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.161+0000 7fc1f2064740 1 -- 172.21.15.3:0/1323009493 >> 172.21.15.3:0/1323009493 conn(0x562553a12460 msgr2=0x562553dd21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.161+0000 7fc1f2064740 1 -- 172.21.15.3:0/1323009493 shutdown_connections 2024-09-21T06:21:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.161+0000 7fc1f2064740 1 -- 172.21.15.3:0/1323009493 wait complete. 2024-09-21T06:21:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.161+0000 7fc1f2064740 1 -- start start 2024-09-21T06:21:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.161+0000 7fc1f2064740 1 Processor -- start 2024-09-21T06:21:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1f2064740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625539dc9f0 0x562553dd58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1f2064740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 0x562553dd5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1f2064740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539fd390 0x562553e4c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1f2064740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e41b9390 con 0x5625539fd390 2024-09-21T06:21:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1f2064740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e41b8cc0 con 0x5625539f51b0 2024-09-21T06:21:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1f2064740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4102e70 con 0x5625539dc9f0 2024-09-21T06:21:09.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1f2064740 10 client.0 _ll_register_callbacks cb 0x7fc1d4013a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1e908b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625539dc9f0 0x562553dd58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1e988c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 0x562553dd5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1e908b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625539dc9f0 0x562553dd58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:57920/0 (socket says 172.21.15.3:57920) 2024-09-21T06:21:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1e988c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 0x562553dd5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:32790/0 (socket says 172.21.15.3:32790) 2024-09-21T06:21:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1e908b640 1 -- 172.21.15.3:0/2010292312 learned_addr learned my addr 172.21.15.3:0/2010292312 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1e908b640 1 --2- 172.21.15.3:0/2010292312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539fd390 0x562553e4c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1e988c640 1 -- 172.21.15.3:0/2010292312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625539dc9f0 msgr2=0x562553dd58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1e988c640 1 --2- 172.21.15.3:0/2010292312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625539dc9f0 0x562553dd58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1e988c640 1 -- 172.21.15.3:0/2010292312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539fd390 msgr2=0x562553e4c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1e988c640 1 --2- 172.21.15.3:0/2010292312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539fd390 0x562553e4c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.162+0000 7fc1e988c640 1 -- 172.21.15.3:0/2010292312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562553e4cee0 con 0x5625539f51b0 2024-09-21T06:21:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.163+0000 7fc1e988c640 1 --2- 172.21.15.3:0/2010292312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 0x562553dd5e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x562553cb1d30 tx=0x7fc1e0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2a2454fec6361ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.163+0000 7fc1cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.163+0000 7fc1cd7fa640 1 -- 172.21.15.3:0/2010292312 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0038020 con 0x5625539f51b0 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.163+0000 7fc1cd7fa640 1 -- 172.21.15.3:0/2010292312 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fc1e00035f0 con 0x5625539f51b0 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.163+0000 7fc1f2064740 1 -- 172.21.15.3:0/2010292312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1e00054c0 con 0x5625539f51b0 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.163+0000 7fc1cd7fa640 1 -- 172.21.15.3:0/2010292312 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc1e0043050 con 0x5625539f51b0 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.163+0000 7fc1f2064740 10 client.26855 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.163+0000 7fc1f2064740 1 -- 172.21.15.3:0/2010292312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562553e4e140 con 0x5625539f51b0 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.163+0000 7fc1cd7fa640 1 -- 172.21.15.3:0/2010292312 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e00054c0 con 0x5625539f51b0 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.164+0000 7fc1cd7fa640 1 -- 172.21.15.3:0/2010292312 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fc1e0019b60 con 0x5625539f51b0 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.164+0000 7fc1f2064740 10 client.26855 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.164+0000 7fc1f2064740 10 client.26855 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.164+0000 7fc1f2064740 20 client.26855 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.164+0000 7fc1f2064740 1 --2- 172.21.15.3:0/2010292312 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x562553a1e070 0x562553e50670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.164+0000 7fc1f2064740 4 client.26855 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:21:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.164+0000 7fc1f2064740 1 -- 172.21.15.3:0/2010292312 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562553dbe9e0 con 0x562553a1e070 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.164+0000 7fc1e908b640 1 --2- 172.21.15.3:0/2010292312 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x562553a1e070 0x562553e50670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.164+0000 7fc1cd7fa640 10 client.26855 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.164+0000 7fc1e908b640 1 --2- 172.21.15.3:0/2010292312 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x562553a1e070 0x562553e50670 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=632317f29d67b188 server_cookie=3e09ba5a692cf098 in_seq=0 out_seq=0 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1cd7fa640 1 -- 172.21.15.3:0/2010292312 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562553dbe9e0 con 0x562553a1e070 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1cd7fa640 10 client.26855 handle_command_reply: tid=0 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1f2064740 1 client.26855 shutdown 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1f2064740 1 -- 172.21.15.3:0/2010292312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 msgr2=0x562553dd5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1f2064740 1 --2- 172.21.15.3:0/2010292312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 0x562553dd5e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x562553cb1d30 tx=0x7fc1e0009ca0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1f2064740 1 -- 172.21.15.3:0/2010292312 shutdown_connections 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1f2064740 1 --2- 172.21.15.3:0/2010292312 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x562553a1e070 0x562553e50670 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1f2064740 1 --2- 172.21.15.3:0/2010292312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5625539fd390 0x562553e4c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1f2064740 1 --2- 172.21.15.3:0/2010292312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625539f51b0 0x562553dd5e00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1cd7fa640 0 client.26855 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1f2064740 1 --2- 172.21.15.3:0/2010292312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625539dc9f0 0x562553dd58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1f2064740 1 -- 172.21.15.3:0/2010292312 >> 172.21.15.3:0/2010292312 conn(0x562553a12460 msgr2=0x562553dd2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1f2064740 1 -- 172.21.15.3:0/2010292312 shutdown_connections 2024-09-21T06:21:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1f2064740 1 -- 172.21.15.3:0/2010292312 wait complete. 2024-09-21T06:21:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.165+0000 7fc1f2064740 20 client.26855 trim_cache size 0 max 16384 2024-09-21T06:21:09.172 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:21:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.166+0000 7fc1ebaf0640 1 -- 172.21.15.3:0/623065021 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc1d00b66b0 msgr2=0x7fc1d00b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.166+0000 7fc1ebaf0640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc1d00b66b0 0x7fc1d00b8bc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc1065d0 tx=0x7fc1dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.166+0000 7fc1ebaf0640 1 -- 172.21.15.3:0/623065021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1e4108cf0 msgr2=0x7fc1e419fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.166+0000 7fc1ebaf0640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1e4108cf0 0x7fc1e419fd00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc005cc0 tx=0x7fc1dc031700 comp rx=0 tx=0).stop 2024-09-21T06:21:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.166+0000 7fc1ebaf0640 1 -- 172.21.15.3:0/623065021 shutdown_connections 2024-09-21T06:21:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.166+0000 7fc1ebaf0640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc1d00b66b0 0x7fc1d00b8bc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.166+0000 7fc1ebaf0640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e41083a0 0x7fc1e419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.166+0000 7fc1ebaf0640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1e4108cf0 0x7fc1e419fd00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.166+0000 7fc1ebaf0640 1 --2- 172.21.15.3:0/623065021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e4102a60 0x7fc1e419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:09.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.166+0000 7fc1ebaf0640 1 -- 172.21.15.3:0/623065021 >> 172.21.15.3:0/623065021 conn(0x7fc1e4079730 msgr2=0x7fc1e4101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:09.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.166+0000 7fc1ebaf0640 1 -- 172.21.15.3:0/623065021 shutdown_connections 2024-09-21T06:21:09.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:09.166+0000 7fc1ebaf0640 1 -- 172.21.15.3:0/623065021 wait complete. 2024-09-21T06:21:09.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:09 smithi137 ceph-mon[34464]: pgmap v509: 97 pgs: 97 active+clean; 120 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 346 op/s 2024-09-21T06:21:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:09 smithi204 ceph-mon[34659]: pgmap v509: 97 pgs: 97 active+clean; 120 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 346 op/s 2024-09-21T06:21:09.775 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:09 smithi003 ceph-mon[31227]: pgmap v509: 97 pgs: 97 active+clean; 120 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 346 op/s 2024-09-21T06:21:09.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 643191 11.70 MB/sec execute 162 sec latency 101.265 ms 2024-09-21T06:21:10.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:21:10.346 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.817+0000 7f211f8cc640 1 -- 172.21.15.3:0/2437659149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2118108cf0 msgr2=0x7f211810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.817+0000 7f211f8cc640 1 --2- 172.21.15.3:0/2437659149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2118108cf0 0x7f211810f520 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2110009ca0 tx=0x7f211002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.817+0000 7f211f8cc640 1 -- 172.21.15.3:0/2437659149 shutdown_connections 2024-09-21T06:21:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.817+0000 7f211f8cc640 1 --2- 172.21.15.3:0/2437659149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21181083a0 0x7f21181087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.817+0000 7f211f8cc640 1 --2- 172.21.15.3:0/2437659149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2118108cf0 0x7f211810f520 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.817+0000 7f211f8cc640 1 --2- 172.21.15.3:0/2437659149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2118102a60 0x7f2118102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.817+0000 7f211f8cc640 1 -- 172.21.15.3:0/2437659149 >> 172.21.15.3:0/2437659149 conn(0x7f2118079730 msgr2=0x7f2118079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.817+0000 7f211f8cc640 1 -- 172.21.15.3:0/2437659149 shutdown_connections 2024-09-21T06:21:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.817+0000 7f211f8cc640 1 -- 172.21.15.3:0/2437659149 wait complete. 2024-09-21T06:21:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.818+0000 7f211f8cc640 1 Processor -- start 2024-09-21T06:21:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.818+0000 7f211f8cc640 1 -- start start 2024-09-21T06:21:10.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.818+0000 7f211f8cc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2118102a60 0x7f211819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.818+0000 7f211f8cc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21181083a0 0x7f211819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.818+0000 7f211f8cc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2118108cf0 0x7f211819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.818+0000 7f211f8cc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2118112980 con 0x7f2118108cf0 2024-09-21T06:21:10.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.818+0000 7f211f8cc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2118112800 con 0x7f21181083a0 2024-09-21T06:21:10.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.818+0000 7f211f8cc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2118112b00 con 0x7f2118102a60 2024-09-21T06:21:10.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.818+0000 7f211de69640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2118108cf0 0x7f211819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.818+0000 7f211d668640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2118102a60 0x7f211819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.818+0000 7f211ce67640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21181083a0 0x7f211819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.818+0000 7f211d668640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2118102a60 0x7f211819b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:57946/0 (socket says 172.21.15.3:57946) 2024-09-21T06:21:10.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.818+0000 7f211d668640 1 -- 172.21.15.3:0/2894240335 learned_addr learned my addr 172.21.15.3:0/2894240335 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.819+0000 7f211d668640 1 -- 172.21.15.3:0/2894240335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21181083a0 msgr2=0x7f211819b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.819+0000 7f211d668640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21181083a0 0x7f211819b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.819+0000 7f211d668640 1 -- 172.21.15.3:0/2894240335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2118108cf0 msgr2=0x7f211819bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.819+0000 7f211d668640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2118108cf0 0x7f211819bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.819+0000 7f211d668640 1 -- 172.21.15.3:0/2894240335 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21181990e0 con 0x7f2118102a60 2024-09-21T06:21:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.819+0000 7f211ce67640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21181083a0 0x7f211819b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.819+0000 7f211de69640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2118108cf0 0x7f211819bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.819+0000 7f211d668640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2118102a60 0x7f211819b2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f211400e9f0 tx=0x7f211400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.819+0000 7f210e7fc640 1 -- 172.21.15.3:0/2894240335 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211400cd80 con 0x7f2118102a60 2024-09-21T06:21:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.819+0000 7f210e7fc640 1 -- 172.21.15.3:0/2894240335 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21140054d0 con 0x7f2118102a60 2024-09-21T06:21:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.819+0000 7f211f8cc640 1 -- 172.21.15.3:0/2894240335 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21181993d0 con 0x7f2118102a60 2024-09-21T06:21:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.819+0000 7f210e7fc640 1 -- 172.21.15.3:0/2894240335 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114010690 con 0x7f2118102a60 2024-09-21T06:21:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.819+0000 7f211f8cc640 1 -- 172.21.15.3:0/2894240335 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21181998c0 con 0x7f2118102a60 2024-09-21T06:21:10.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f2125e40740 1 Processor -- start 2024-09-21T06:21:10.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f2125e40740 1 -- start start 2024-09-21T06:21:10.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f2125e40740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e26a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f2125e40740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 0x56192e26abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f2125e40740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e26b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f2125e40740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56192e1e0ea0 con 0x56192dea2390 2024-09-21T06:21:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f2125e40740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56192e1d02d0 con 0x56192dec3070 2024-09-21T06:21:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f2125e40740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56192e1d40e0 con 0x56192de9a1b0 2024-09-21T06:21:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f210e7fc640 1 -- 172.21.15.3:0/2894240335 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f2114004e50 con 0x7f2118102a60 2024-09-21T06:21:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f211de69640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e26a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f211ce67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e26b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f211de69640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e26a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:32816/0 (socket says 172.21.15.3:32816) 2024-09-21T06:21:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f211ce67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e26b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39812/0 (socket says 172.21.15.3:39812) 2024-09-21T06:21:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f211de69640 1 -- 172.21.15.3:0/3520079010 learned_addr learned my addr 172.21.15.3:0/3520079010 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.823+0000 7f211d668640 1 --2- 172.21.15.3:0/3520079010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 0x56192e26abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.824+0000 7f211de69640 1 -- 172.21.15.3:0/3520079010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 msgr2=0x56192e26abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.824+0000 7f211de69640 1 --2- 172.21.15.3:0/3520079010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 0x56192e26abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.824+0000 7f211de69640 1 -- 172.21.15.3:0/3520079010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 msgr2=0x56192e26b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:10.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.824+0000 7f211de69640 1 --2- 172.21.15.3:0/3520079010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e26b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.824+0000 7f211de69640 1 -- 172.21.15.3:0/3520079010 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56192df2e650 con 0x56192dec3070 2024-09-21T06:21:10.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.824+0000 7f211de69640 1 --2- 172.21.15.3:0/3520079010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e26a6a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2110033040 tx=0x7f211002f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b1770e7b38e6d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:10.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.824+0000 7f21037fe640 1 -- 172.21.15.3:0/3520079010 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110031650 con 0x56192dec3070 2024-09-21T06:21:10.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.824+0000 7f21037fe640 1 -- 172.21.15.3:0/3520079010 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2110036040 con 0x56192dec3070 2024-09-21T06:21:10.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.824+0000 7f21037fe640 1 -- 172.21.15.3:0/3520079010 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21100044c0 con 0x56192dec3070 2024-09-21T06:21:10.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.824+0000 7f210e7fc640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f21040b66b0 0x7f21040b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.824+0000 7f211d668640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f21040b66b0 0x7f21040b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.825+0000 7f210e7fc640 1 -- 172.21.15.3:0/2894240335 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f21140e7a80 con 0x7f2118102a60 2024-09-21T06:21:10.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.825+0000 7f2125e40740 1 -- 172.21.15.3:0/3520079010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 msgr2=0x56192e26a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:10.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.825+0000 7f2125e40740 1 --2- 172.21.15.3:0/3520079010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e26a6a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2110033040 tx=0x7f211002f310 comp rx=0 tx=0).stop 2024-09-21T06:21:10.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.825+0000 7f211d668640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f21040b66b0 0x7f21040b8bc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f21140e52c0 tx=0x7f2114004a90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:10.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.825+0000 7f2125e40740 1 -- 172.21.15.3:0/3520079010 shutdown_connections 2024-09-21T06:21:10.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.825+0000 7f2125e40740 1 --2- 172.21.15.3:0/3520079010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 0x56192e26abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.825+0000 7f2125e40740 1 --2- 172.21.15.3:0/3520079010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e26b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.825+0000 7f2125e40740 1 --2- 172.21.15.3:0/3520079010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e26a6a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.825+0000 7f2125e40740 1 -- 172.21.15.3:0/3520079010 >> 172.21.15.3:0/3520079010 conn(0x56192deb7460 msgr2=0x56192e268590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.825+0000 7f2125e40740 1 -- 172.21.15.3:0/3520079010 shutdown_connections 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.825+0000 7f2125e40740 1 -- 172.21.15.3:0/3520079010 wait complete. 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.826+0000 7f2125e40740 1 -- start start 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.826+0000 7f2125e40740 1 Processor -- start 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.826+0000 7f2125e40740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 0x56192e275da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.826+0000 7f2125e40740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e2762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.826+0000 7f2125e40740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e281f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.826+0000 7f2125e40740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56192e031700 con 0x56192dea2390 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.826+0000 7f2125e40740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56192e16b160 con 0x56192dec3070 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.826+0000 7f2125e40740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56192e278ca0 con 0x56192de9a1b0 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.826+0000 7f2125e40740 10 client.0 _ll_register_callbacks cb 0x56192e1849e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f211ce67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e2762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f211de69640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 0x56192e275da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f211ce67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e2762e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39816/0 (socket says 172.21.15.3:39816) 2024-09-21T06:21:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f211ce67640 1 -- 172.21.15.3:0/3083270529 learned_addr learned my addr 172.21.15.3:0/3083270529 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f211de69640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 0x56192e275da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:57956/0 (socket says 172.21.15.3:57956) 2024-09-21T06:21:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f211de69640 1 --2- 172.21.15.3:0/3083270529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e281f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f211ce67640 1 -- 172.21.15.3:0/3083270529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 msgr2=0x56192e275da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f211ce67640 1 --2- 172.21.15.3:0/3083270529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 0x56192e275da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f211ce67640 1 -- 172.21.15.3:0/3083270529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 msgr2=0x56192e281f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f211ce67640 1 --2- 172.21.15.3:0/3083270529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e281f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f211ce67640 1 -- 172.21.15.3:0/3083270529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56192e198c90 con 0x56192dea2390 2024-09-21T06:21:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f211ce67640 1 --2- 172.21.15.3:0/3083270529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e2762e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2108012d20 tx=0x7f210800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8692930c67a03cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f2100ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f2100ff9640 1 -- 172.21.15.3:0/3083270529 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21080120d0 con 0x56192dea2390 2024-09-21T06:21:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f2125e40740 1 -- 172.21.15.3:0/3083270529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2110036040 con 0x56192dea2390 2024-09-21T06:21:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.827+0000 7f2100ff9640 1 -- 172.21.15.3:0/3083270529 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2108010430 con 0x56192dea2390 2024-09-21T06:21:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.828+0000 7f2100ff9640 1 -- 172.21.15.3:0/3083270529 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2108012270 con 0x56192dea2390 2024-09-21T06:21:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.828+0000 7f2125e40740 10 client.17295 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.828+0000 7f2125e40740 1 -- 172.21.15.3:0/3083270529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56192e284ac0 con 0x56192dea2390 2024-09-21T06:21:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.828+0000 7f2100ff9640 1 -- 172.21.15.3:0/3083270529 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2110036040 con 0x56192dea2390 2024-09-21T06:21:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.828+0000 7f2100ff9640 1 -- 172.21.15.3:0/3083270529 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f210801c650 con 0x56192dea2390 2024-09-21T06:21:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.828+0000 7f2125e40740 10 client.17295 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.828+0000 7f2125e40740 10 client.17295 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:21:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.828+0000 7f2125e40740 20 client.17295 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.828+0000 7f2125e40740 1 --2- 172.21.15.3:0/3083270529 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x56192de819f0 0x56192e287fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.828+0000 7f2125e40740 4 client.17295 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.828+0000 7f2125e40740 1 -- 172.21.15.3:0/3083270529 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56192e1e2df0 con 0x56192de819f0 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.828+0000 7f211de69640 1 --2- 172.21.15.3:0/3083270529 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x56192de819f0 0x56192e287fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.829+0000 7f211de69640 1 --2- 172.21.15.3:0/3083270529 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x56192de819f0 0x56192e287fa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8352e965e368ba16 server_cookie=77410c56421f9713 in_seq=0 out_seq=0 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.829+0000 7f2100ff9640 10 client.17295 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.829+0000 7f2100ff9640 1 -- 172.21.15.3:0/3083270529 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56192e1e2df0 con 0x56192de819f0 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.829+0000 7f2100ff9640 10 client.17295 handle_command_reply: tid=0 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.829+0000 7f2125e40740 1 client.17295 shutdown 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.830+0000 7f2125e40740 1 -- 172.21.15.3:0/3083270529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 msgr2=0x56192e2762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.830+0000 7f2125e40740 1 --2- 172.21.15.3:0/3083270529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e2762e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2108012d20 tx=0x7f210800f790 comp rx=0 tx=0).stop 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.830+0000 7f2125e40740 1 -- 172.21.15.3:0/3083270529 shutdown_connections 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.830+0000 7f2125e40740 1 --2- 172.21.15.3:0/3083270529 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x56192de819f0 0x56192e287fa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.830+0000 7f2125e40740 1 --2- 172.21.15.3:0/3083270529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e281f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.830+0000 7f2125e40740 1 --2- 172.21.15.3:0/3083270529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e2762e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.830+0000 7f2100ff9640 0 client.17295 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:21:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.830+0000 7f2125e40740 1 --2- 172.21.15.3:0/3083270529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 0x56192e275da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.830+0000 7f2125e40740 1 -- 172.21.15.3:0/3083270529 >> 172.21.15.3:0/3083270529 conn(0x56192deb7460 msgr2=0x56192e26d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:10.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.830+0000 7f2125e40740 1 -- 172.21.15.3:0/3083270529 shutdown_connections 2024-09-21T06:21:10.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.830+0000 7f2125e40740 1 -- 172.21.15.3:0/3083270529 wait complete. 2024-09-21T06:21:10.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.830+0000 7f2125e40740 20 client.17295 trim_cache size 0 max 16384 2024-09-21T06:21:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.843+0000 7f2125e40740 1 Processor -- start 2024-09-21T06:21:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.843+0000 7f2125e40740 1 -- start start 2024-09-21T06:21:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.843+0000 7f2125e40740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 0x56192e268600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.843+0000 7f2125e40740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e281e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.843+0000 7f2125e40740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e26d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.843+0000 7f2125e40740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21181b1800 con 0x56192dea2390 2024-09-21T06:21:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.843+0000 7f2125e40740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f21181b3370 con 0x56192dec3070 2024-09-21T06:21:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.843+0000 7f2125e40740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f21181b1ed0 con 0x56192de9a1b0 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f211ce67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e281e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f211ce67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e281e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39818/0 (socket says 172.21.15.3:39818) 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f211ce67640 1 -- 172.21.15.3:0/2604209558 learned_addr learned my addr 172.21.15.3:0/2604209558 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f211de69640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 0x56192e268600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f211de69640 1 --2- 172.21.15.3:0/2604209558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e26d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f211ce67640 1 -- 172.21.15.3:0/2604209558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 msgr2=0x56192e268600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f211ce67640 1 --2- 172.21.15.3:0/2604209558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 0x56192e268600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f211ce67640 1 -- 172.21.15.3:0/2604209558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 msgr2=0x56192e26d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f211ce67640 1 --2- 172.21.15.3:0/2604209558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e26d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f211ce67640 1 -- 172.21.15.3:0/2604209558 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56192e26d500 con 0x56192dea2390 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f211de69640 1 --2- 172.21.15.3:0/2604209558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e26d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f211ce67640 1 --2- 172.21.15.3:0/2604209558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e281e90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2108012870 tx=0x7f2108012990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395c98e2f703260 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f2102ffd640 1 -- 172.21.15.3:0/2604209558 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2108029030 con 0x56192dea2390 2024-09-21T06:21:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f2102ffd640 1 -- 172.21.15.3:0/2604209558 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21080050b0 con 0x56192dea2390 2024-09-21T06:21:10.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.844+0000 7f2102ffd640 1 -- 172.21.15.3:0/2604209558 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2108003770 con 0x56192dea2390 2024-09-21T06:21:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.845+0000 7f2125e40740 1 -- 172.21.15.3:0/2604209558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 msgr2=0x56192e281e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.845+0000 7f2125e40740 1 --2- 172.21.15.3:0/2604209558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e281e90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2108012870 tx=0x7f2108012990 comp rx=0 tx=0).stop 2024-09-21T06:21:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.845+0000 7f2125e40740 1 -- 172.21.15.3:0/2604209558 shutdown_connections 2024-09-21T06:21:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.845+0000 7f2125e40740 1 --2- 172.21.15.3:0/2604209558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192dec3070 0x56192e26d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.845+0000 7f2125e40740 1 --2- 172.21.15.3:0/2604209558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dea2390 0x56192e281e90 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.845+0000 7f2125e40740 1 --2- 172.21.15.3:0/2604209558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192de9a1b0 0x56192e268600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.845+0000 7f2125e40740 1 -- 172.21.15.3:0/2604209558 >> 172.21.15.3:0/2604209558 conn(0x56192deb7460 msgr2=0x56192e267ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.845+0000 7f2125e40740 1 -- 172.21.15.3:0/2604209558 shutdown_connections 2024-09-21T06:21:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.845+0000 7f2125e40740 1 -- 172.21.15.3:0/2604209558 wait complete. 2024-09-21T06:21:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.845+0000 7f2125e40740 1 -- start start 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.846+0000 7f2125e40740 1 Processor -- start 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.846+0000 7f2125e40740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192de9a1b0 0x56192e2eacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.846+0000 7f2125e40740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192dea2390 0x56192e2eb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.846+0000 7f2125e40740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dec3070 0x56192e2796f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.846+0000 7f2125e40740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21181b2ca0 con 0x56192dec3070 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.846+0000 7f2125e40740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f21181b25d0 con 0x56192de9a1b0 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.846+0000 7f2125e40740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f21181af7e0 con 0x56192dea2390 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.846+0000 7f2125e40740 10 client.0 _ll_register_callbacks cb 0x7f2108009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.846+0000 7f211de69640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192de9a1b0 0x56192e2eacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.846+0000 7f211ce67640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192dea2390 0x56192e2eb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.846+0000 7f211de69640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dec3070 0x56192e2796f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.846+0000 7f211ce67640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192dea2390 0x56192e2eb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:57970/0 (socket says 172.21.15.3:57970) 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.846+0000 7f211ce67640 1 -- 172.21.15.3:0/1293522039 learned_addr learned my addr 172.21.15.3:0/1293522039 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.847+0000 7f211de69640 1 -- 172.21.15.3:0/1293522039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192dea2390 msgr2=0x56192e2eb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.847+0000 7f211de69640 1 --2- 172.21.15.3:0/1293522039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192dea2390 0x56192e2eb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.847+0000 7f211de69640 1 -- 172.21.15.3:0/1293522039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192de9a1b0 msgr2=0x56192e2eacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.847+0000 7f211de69640 1 --2- 172.21.15.3:0/1293522039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192de9a1b0 0x56192e2eacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.847+0000 7f211de69640 1 -- 172.21.15.3:0/1293522039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56192e279e60 con 0x56192dec3070 2024-09-21T06:21:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.847+0000 7f211de69640 1 --2- 172.21.15.3:0/1293522039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dec3070 0x56192e2796f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f21100094e0 tx=0x7f21100040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4e526c76d31241 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.847+0000 7f2100ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.847+0000 7f2100ff9640 1 -- 172.21.15.3:0/1293522039 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110003970 con 0x56192dec3070 2024-09-21T06:21:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.847+0000 7f2100ff9640 1 -- 172.21.15.3:0/1293522039 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f2110003b10 con 0x56192dec3070 2024-09-21T06:21:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.847+0000 7f2125e40740 1 -- 172.21.15.3:0/1293522039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21080050b0 con 0x56192dec3070 2024-09-21T06:21:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.847+0000 7f2100ff9640 1 -- 172.21.15.3:0/1293522039 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2110003d10 con 0x56192dec3070 2024-09-21T06:21:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.847+0000 7f2125e40740 10 client.17301 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.847+0000 7f2125e40740 1 -- 172.21.15.3:0/1293522039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56192e2f2470 con 0x56192dec3070 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.848+0000 7f2100ff9640 1 -- 172.21.15.3:0/1293522039 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21080050b0 con 0x56192dec3070 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.848+0000 7f2100ff9640 1 -- 172.21.15.3:0/1293522039 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f2110051040 con 0x56192dec3070 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.848+0000 7f2125e40740 10 client.17301 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.848+0000 7f2125e40740 10 client.17301 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.848+0000 7f2125e40740 20 client.17301 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.848+0000 7f2125e40740 1 --2- 172.21.15.3:0/1293522039 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x56192de56da0 0x56192e2f4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.848+0000 7f2125e40740 4 client.17301 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.848+0000 7f2125e40740 1 -- 172.21.15.3:0/1293522039 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56192e1de8a0 con 0x56192de56da0 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.848+0000 7f211ce67640 1 --2- 172.21.15.3:0/1293522039 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x56192de56da0 0x56192e2f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2100ff9640 10 client.17301 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f211ce67640 1 --2- 172.21.15.3:0/1293522039 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x56192de56da0 0x56192e2f4870 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=809b8b8cc5618b93 server_cookie=d82a2cb71116001e in_seq=0 out_seq=0 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2100ff9640 1 -- 172.21.15.3:0/1293522039 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56192e1de8a0 con 0x56192de56da0 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2100ff9640 10 client.17301 handle_command_reply: tid=0 2024-09-21T06:21:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2125e40740 1 client.17301 shutdown 2024-09-21T06:21:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2125e40740 1 -- 172.21.15.3:0/1293522039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dec3070 msgr2=0x56192e2796f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2125e40740 1 --2- 172.21.15.3:0/1293522039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dec3070 0x56192e2796f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f21100094e0 tx=0x7f21100040e0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2125e40740 1 -- 172.21.15.3:0/1293522039 shutdown_connections 2024-09-21T06:21:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2125e40740 1 --2- 172.21.15.3:0/1293522039 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x56192de56da0 0x56192e2f4870 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2125e40740 1 --2- 172.21.15.3:0/1293522039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56192dea2390 0x56192e2eb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2125e40740 1 --2- 172.21.15.3:0/1293522039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56192dec3070 0x56192e2796f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2125e40740 1 --2- 172.21.15.3:0/1293522039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56192de9a1b0 0x56192e2eacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2125e40740 1 -- 172.21.15.3:0/1293522039 >> 172.21.15.3:0/1293522039 conn(0x56192deb7460 msgr2=0x56192e26d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2100ff9640 0 client.17301 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:21:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2125e40740 1 -- 172.21.15.3:0/1293522039 shutdown_connections 2024-09-21T06:21:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2125e40740 1 -- 172.21.15.3:0/1293522039 wait complete. 2024-09-21T06:21:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.849+0000 7f2125e40740 20 client.17301 trim_cache size 0 max 16384 2024-09-21T06:21:10.857 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:21:10.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.850+0000 7f211f8cc640 1 -- 172.21.15.3:0/2894240335 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f21040b66b0 msgr2=0x7f21040b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:10.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.850+0000 7f211f8cc640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f21040b66b0 0x7f21040b8bc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f21140e52c0 tx=0x7f2114004a90 comp rx=0 tx=0).stop 2024-09-21T06:21:10.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.850+0000 7f211f8cc640 1 -- 172.21.15.3:0/2894240335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2118102a60 msgr2=0x7f211819b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:10.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.850+0000 7f211f8cc640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2118102a60 0x7f211819b2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f211400e9f0 tx=0x7f211400edf0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.850+0000 7f211f8cc640 1 -- 172.21.15.3:0/2894240335 shutdown_connections 2024-09-21T06:21:10.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.850+0000 7f211f8cc640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f21040b66b0 0x7f21040b8bc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.850+0000 7f211f8cc640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2118108cf0 0x7f211819bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.850+0000 7f211f8cc640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21181083a0 0x7f211819b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.850+0000 7f211f8cc640 1 --2- 172.21.15.3:0/2894240335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2118102a60 0x7f211819b2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:10.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.850+0000 7f211f8cc640 1 -- 172.21.15.3:0/2894240335 >> 172.21.15.3:0/2894240335 conn(0x7f2118079730 msgr2=0x7f21181052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:10.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.850+0000 7f211f8cc640 1 -- 172.21.15.3:0/2894240335 shutdown_connections 2024-09-21T06:21:10.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:10.850+0000 7f211f8cc640 1 -- 172.21.15.3:0/2894240335 wait complete. 2024-09-21T06:21:10.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 644715 11.66 MB/sec execute 163 sec latency 38.346 ms 2024-09-21T06:21:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:11 smithi003 ceph-mon[31227]: pgmap v510: 97 pgs: 97 active+clean; 112 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 375 op/s 2024-09-21T06:21:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:11 smithi137 ceph-mon[34464]: pgmap v510: 97 pgs: 97 active+clean; 112 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 375 op/s 2024-09-21T06:21:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:11 smithi204 ceph-mon[34659]: pgmap v510: 97 pgs: 97 active+clean; 112 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 375 op/s 2024-09-21T06:21:11.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:21:11.958 INFO:tasks.workunit.client.0.smithi003.stdout: 1 646748 11.67 MB/sec execute 164 sec latency 48.361 ms 2024-09-21T06:21:11.960 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:12.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.524+0000 7f5894773640 1 -- 172.21.15.3:0/3812458479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588c108d20 msgr2=0x7f588c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:12.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.524+0000 7f5894773640 1 --2- 172.21.15.3:0/3812458479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588c108d20 0x7f588c10f550 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5884009ca0 tx=0x7f588402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.525+0000 7f5894773640 1 -- 172.21.15.3:0/3812458479 shutdown_connections 2024-09-21T06:21:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.525+0000 7f5894773640 1 --2- 172.21.15.3:0/3812458479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588c108d20 0x7f588c10f550 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.525+0000 7f5894773640 1 --2- 172.21.15.3:0/3812458479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f588c1083d0 0x7f588c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.525+0000 7f5894773640 1 --2- 172.21.15.3:0/3812458479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f588c102ae0 0x7f588c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.525+0000 7f5894773640 1 -- 172.21.15.3:0/3812458479 >> 172.21.15.3:0/3812458479 conn(0x7f588c079790 msgr2=0x7f588c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.525+0000 7f5894773640 1 -- 172.21.15.3:0/3812458479 shutdown_connections 2024-09-21T06:21:12.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.525+0000 7f5894773640 1 -- 172.21.15.3:0/3812458479 wait complete. 2024-09-21T06:21:12.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.525+0000 7f5894773640 1 Processor -- start 2024-09-21T06:21:12.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.525+0000 7f5894773640 1 -- start start 2024-09-21T06:21:12.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5894773640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f588c102ae0 0x7f588c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5894773640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f588c1083d0 0x7f588c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5894773640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588c108d20 0x7f588c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5894773640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f588c112a30 con 0x7f588c108d20 2024-09-21T06:21:12.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5894773640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1128b0 con 0x7f588c1083d0 2024-09-21T06:21:12.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5894773640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f588c112bb0 con 0x7f588c102ae0 2024-09-21T06:21:12.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5892d10640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588c108d20 0x7f588c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f589250f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f588c102ae0 0x7f588c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5892d10640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588c108d20 0x7f588c19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39838/0 (socket says 172.21.15.3:39838) 2024-09-21T06:21:12.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5891d0e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f588c1083d0 0x7f588c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5892d10640 1 -- 172.21.15.3:0/1769372124 learned_addr learned my addr 172.21.15.3:0/1769372124 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:12.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5892d10640 1 -- 172.21.15.3:0/1769372124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f588c102ae0 msgr2=0x7f588c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:12.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5892d10640 1 --2- 172.21.15.3:0/1769372124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f588c102ae0 0x7f588c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5892d10640 1 -- 172.21.15.3:0/1769372124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f588c1083d0 msgr2=0x7f588c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:12.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5892d10640 1 --2- 172.21.15.3:0/1769372124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f588c1083d0 0x7f588c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5892d10640 1 -- 172.21.15.3:0/1769372124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c1a03e0 con 0x7f588c108d20 2024-09-21T06:21:12.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.526+0000 7f5892d10640 1 --2- 172.21.15.3:0/1769372124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588c108d20 0x7f588c19fcb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f588402f540 tx=0x7f58840316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:12.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.527+0000 7f58837fe640 1 -- 172.21.15.3:0/1769372124 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884003b70 con 0x7f588c108d20 2024-09-21T06:21:12.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.527+0000 7f5894773640 1 -- 172.21.15.3:0/1769372124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588c1a0670 con 0x7f588c108d20 2024-09-21T06:21:12.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.527+0000 7f58837fe640 1 -- 172.21.15.3:0/1769372124 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5884003d10 con 0x7f588c108d20 2024-09-21T06:21:12.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.527+0000 7f58837fe640 1 -- 172.21.15.3:0/1769372124 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588402f990 con 0x7f588c108d20 2024-09-21T06:21:12.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.527+0000 7f5894773640 1 -- 172.21.15.3:0/1769372124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588c1a6e30 con 0x7f588c108d20 2024-09-21T06:21:12.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.530+0000 7f589ace7740 1 Processor -- start 2024-09-21T06:21:12.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.530+0000 7f58837fe640 1 -- 172.21.15.3:0/1769372124 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f588402fb30 con 0x7f588c108d20 2024-09-21T06:21:12.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.530+0000 7f589ace7740 1 -- start start 2024-09-21T06:21:12.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f589ace7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d50e070 0x564d5d8b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f589ace7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4e51b0 0x564d5d8b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f589ace7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4ed390 0x564d5d8b6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f589ace7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564d5d82bea0 con 0x564d5d50e070 2024-09-21T06:21:12.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f589ace7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564d5d81b2d0 con 0x564d5d4e51b0 2024-09-21T06:21:12.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f589ace7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564d5d81f0e0 con 0x564d5d4ed390 2024-09-21T06:21:12.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f5891d0e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d50e070 0x564d5d8b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f5891d0e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d50e070 0x564d5d8b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39850/0 (socket says 172.21.15.3:39850) 2024-09-21T06:21:12.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f5891d0e640 1 -- 172.21.15.3:0/3647520986 learned_addr learned my addr 172.21.15.3:0/3647520986 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:12.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f589250f640 1 --2- 172.21.15.3:0/3647520986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4e51b0 0x564d5d8b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f5891d0e640 1 --2- 172.21.15.3:0/3647520986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4ed390 0x564d5d8b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f58837fe640 1 --2- 172.21.15.3:0/1769372124 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f58780b6680 0x7f58780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f5891d0e640 1 -- 172.21.15.3:0/3647520986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4ed390 msgr2=0x564d5d8b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:12.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f5891d0e640 1 --2- 172.21.15.3:0/3647520986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4ed390 0x564d5d8b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f5891d0e640 1 -- 172.21.15.3:0/3647520986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4e51b0 msgr2=0x564d5d8b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:12.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f5891d0e640 1 --2- 172.21.15.3:0/3647520986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4e51b0 0x564d5d8b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f5891d0e640 1 -- 172.21.15.3:0/3647520986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d5d579650 con 0x564d5d50e070 2024-09-21T06:21:12.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f5892d10640 1 --2- 172.21.15.3:0/1769372124 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f58780b6680 0x7f58780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.531+0000 7f58837fe640 1 -- 172.21.15.3:0/1769372124 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5884108850 con 0x7f588c108d20 2024-09-21T06:21:12.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.532+0000 7f5891d0e640 1 --2- 172.21.15.3:0/3647520986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d50e070 0x564d5d8b56a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f587c0137a0 tx=0x7f587c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2365f456e7818c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:12.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.532+0000 7f5880ff9640 1 -- 172.21.15.3:0/3647520986 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c013de0 con 0x564d5d50e070 2024-09-21T06:21:12.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.532+0000 7f5892d10640 1 --2- 172.21.15.3:0/1769372124 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f58780b6680 0x7f58780b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58841065e0 tx=0x7f588409bad0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:12.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.532+0000 7f5880ff9640 1 -- 172.21.15.3:0/3647520986 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f587c013120 con 0x564d5d50e070 2024-09-21T06:21:12.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.532+0000 7f5880ff9640 1 -- 172.21.15.3:0/3647520986 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c010690 con 0x564d5d50e070 2024-09-21T06:21:12.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.533+0000 7f589ace7740 1 -- 172.21.15.3:0/3647520986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d50e070 msgr2=0x564d5d8b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:12.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.533+0000 7f589ace7740 1 --2- 172.21.15.3:0/3647520986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d50e070 0x564d5d8b56a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f587c0137a0 tx=0x7f587c005850 comp rx=0 tx=0).stop 2024-09-21T06:21:12.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.533+0000 7f589ace7740 1 -- 172.21.15.3:0/3647520986 shutdown_connections 2024-09-21T06:21:12.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.533+0000 7f589ace7740 1 --2- 172.21.15.3:0/3647520986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4ed390 0x564d5d8b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.533+0000 7f589ace7740 1 --2- 172.21.15.3:0/3647520986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4e51b0 0x564d5d8b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.533+0000 7f589ace7740 1 --2- 172.21.15.3:0/3647520986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d50e070 0x564d5d8b56a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.533+0000 7f589ace7740 1 -- 172.21.15.3:0/3647520986 >> 172.21.15.3:0/3647520986 conn(0x564d5d502460 msgr2=0x564d5d8b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:12.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.533+0000 7f589ace7740 1 -- 172.21.15.3:0/3647520986 shutdown_connections 2024-09-21T06:21:12.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.533+0000 7f589ace7740 1 -- 172.21.15.3:0/3647520986 wait complete. 2024-09-21T06:21:12.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.533+0000 7f589ace7740 1 -- start start 2024-09-21T06:21:12.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.533+0000 7f589ace7740 1 Processor -- start 2024-09-21T06:21:12.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f589ace7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d4e51b0 0x564d5d8c0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f589ace7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4ed390 0x564d5d8c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f589ace7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d50e070 0x564d5d8cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f589ace7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564d5d67c700 con 0x564d5d4e51b0 2024-09-21T06:21:12.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f589ace7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564d5d7b6160 con 0x564d5d4ed390 2024-09-21T06:21:12.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f589ace7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564d5d8c3e20 con 0x564d5d50e070 2024-09-21T06:21:12.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f589ace7740 10 client.0 _ll_register_callbacks cb 0x564d5d7cf9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:12.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f5891d0e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d4e51b0 0x564d5d8c0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f5891d0e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d4e51b0 0x564d5d8c0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39860/0 (socket says 172.21.15.3:39860) 2024-09-21T06:21:12.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f5891d0e640 1 -- 172.21.15.3:0/3233241619 learned_addr learned my addr 172.21.15.3:0/3233241619 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:12.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f589250f640 1 --2- 172.21.15.3:0/3233241619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4ed390 0x564d5d8c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f5891d0e640 1 --2- 172.21.15.3:0/3233241619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d50e070 0x564d5d8cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f589250f640 1 -- 172.21.15.3:0/3233241619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d50e070 msgr2=0x564d5d8cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:12.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f589250f640 1 --2- 172.21.15.3:0/3233241619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d50e070 0x564d5d8cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f589250f640 1 -- 172.21.15.3:0/3233241619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d4e51b0 msgr2=0x564d5d8c0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:12.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f589250f640 1 --2- 172.21.15.3:0/3233241619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d4e51b0 0x564d5d8c0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.534+0000 7f589250f640 1 -- 172.21.15.3:0/3233241619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d5d7e3c90 con 0x564d5d4ed390 2024-09-21T06:21:12.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.535+0000 7f589250f640 1 --2- 172.21.15.3:0/3233241619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4ed390 0x564d5d8c12b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58880126b0 tx=0x7f5888012b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31b375b3b31b96aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:12.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.535+0000 7f5875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:21:12.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.535+0000 7f5875ffb640 1 -- 172.21.15.3:0/3233241619 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888004020 con 0x564d5d4ed390 2024-09-21T06:21:12.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.535+0000 7f5875ffb640 1 -- 172.21.15.3:0/3233241619 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f58880041c0 con 0x564d5d4ed390 2024-09-21T06:21:12.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.535+0000 7f589ace7740 1 -- 172.21.15.3:0/3233241619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f587c013120 con 0x564d5d4ed390 2024-09-21T06:21:12.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.535+0000 7f5875ffb640 1 -- 172.21.15.3:0/3233241619 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f58880043c0 con 0x564d5d4ed390 2024-09-21T06:21:12.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.535+0000 7f589ace7740 10 client.26885 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:12.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.535+0000 7f589ace7740 1 -- 172.21.15.3:0/3233241619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d5d8cfbb0 con 0x564d5d4ed390 2024-09-21T06:21:12.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.535+0000 7f5875ffb640 1 -- 172.21.15.3:0/3233241619 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f587c013120 con 0x564d5d4ed390 2024-09-21T06:21:12.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.535+0000 7f5875ffb640 1 -- 172.21.15.3:0/3233241619 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f5888003bc0 con 0x564d5d4ed390 2024-09-21T06:21:12.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.535+0000 7f589ace7740 10 client.26885 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:12.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.535+0000 7f589ace7740 10 client.26885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:21:12.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.535+0000 7f589ace7740 20 client.26885 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:12.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.536+0000 7f589ace7740 1 --2- 172.21.15.3:0/3233241619 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564d5d4cc9f0 0x564d5d8d3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.536+0000 7f589ace7740 4 client.26885 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.536+0000 7f589ace7740 1 -- 172.21.15.3:0/3233241619 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d5d82ddf0 con 0x564d5d4cc9f0 2024-09-21T06:21:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.536+0000 7f5891d0e640 1 --2- 172.21.15.3:0/3233241619 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564d5d4cc9f0 0x564d5d8d3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.536+0000 7f5875ffb640 10 client.26885 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:21:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.536+0000 7f5891d0e640 1 --2- 172.21.15.3:0/3233241619 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564d5d4cc9f0 0x564d5d8d3110 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27a5443007aaf1d7 server_cookie=25d5c53f53baeb6 in_seq=0 out_seq=0 2024-09-21T06:21:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f5875ffb640 1 -- 172.21.15.3:0/3233241619 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564d5d82ddf0 con 0x564d5d4cc9f0 2024-09-21T06:21:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f5875ffb640 10 client.26885 handle_command_reply: tid=0 2024-09-21T06:21:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f589ace7740 1 client.26885 shutdown 2024-09-21T06:21:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f589ace7740 1 -- 172.21.15.3:0/3233241619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4ed390 msgr2=0x564d5d8c12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f589ace7740 1 --2- 172.21.15.3:0/3233241619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4ed390 0x564d5d8c12b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58880126b0 tx=0x7f5888012b70 comp rx=0 tx=0).stop 2024-09-21T06:21:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f589ace7740 1 -- 172.21.15.3:0/3233241619 shutdown_connections 2024-09-21T06:21:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f589ace7740 1 --2- 172.21.15.3:0/3233241619 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564d5d4cc9f0 0x564d5d8d3110 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f589ace7740 1 --2- 172.21.15.3:0/3233241619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d50e070 0x564d5d8cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f589ace7740 1 --2- 172.21.15.3:0/3233241619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4ed390 0x564d5d8c12b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f5875ffb640 0 client.26885 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:21:12.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f589ace7740 1 --2- 172.21.15.3:0/3233241619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d4e51b0 0x564d5d8c0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f589ace7740 1 -- 172.21.15.3:0/3233241619 >> 172.21.15.3:0/3233241619 conn(0x564d5d502460 msgr2=0x564d5d8b8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:12.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f589ace7740 1 -- 172.21.15.3:0/3233241619 shutdown_connections 2024-09-21T06:21:12.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f589ace7740 1 -- 172.21.15.3:0/3233241619 wait complete. 2024-09-21T06:21:12.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.537+0000 7f589ace7740 20 client.26885 trim_cache size 0 max 16384 2024-09-21T06:21:12.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589ace7740 1 Processor -- start 2024-09-21T06:21:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589ace7740 1 -- start start 2024-09-21T06:21:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589ace7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4e51b0 0x564d5d8c1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589ace7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4ed390 0x564d5d8cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589ace7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d50e070 0x564d5d8b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589ace7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1af5e0 con 0x564d5d50e070 2024-09-21T06:21:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589ace7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1b1150 con 0x564d5d4e51b0 2024-09-21T06:21:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589ace7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1afcb0 con 0x564d5d4ed390 2024-09-21T06:21:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589250f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4ed390 0x564d5d8cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f5891d0e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d50e070 0x564d5d8b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589250f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4ed390 0x564d5d8cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:58004/0 (socket says 172.21.15.3:58004) 2024-09-21T06:21:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589250f640 1 -- 172.21.15.3:0/4133489494 learned_addr learned my addr 172.21.15.3:0/4133489494 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f5891d0e640 1 --2- 172.21.15.3:0/4133489494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4e51b0 0x564d5d8c1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589250f640 1 -- 172.21.15.3:0/4133489494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4e51b0 msgr2=0x564d5d8c1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589250f640 1 --2- 172.21.15.3:0/4133489494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4e51b0 0x564d5d8c1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589250f640 1 -- 172.21.15.3:0/4133489494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d50e070 msgr2=0x564d5d8b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589250f640 1 --2- 172.21.15.3:0/4133489494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d50e070 0x564d5d8b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f589250f640 1 -- 172.21.15.3:0/4133489494 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d5d8b7990 con 0x564d5d4ed390 2024-09-21T06:21:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.551+0000 7f5891d0e640 1 --2- 172.21.15.3:0/4133489494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d50e070 0x564d5d8b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.552+0000 7f589250f640 1 --2- 172.21.15.3:0/4133489494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4ed390 0x564d5d8cd070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5888001eb0 tx=0x7f58880055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b18c951ac2f2ae1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.552+0000 7f5877fff640 1 -- 172.21.15.3:0/4133489494 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888029070 con 0x564d5d4ed390 2024-09-21T06:21:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.552+0000 7f5877fff640 1 -- 172.21.15.3:0/4133489494 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5888003fe0 con 0x564d5d4ed390 2024-09-21T06:21:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.552+0000 7f5877fff640 1 -- 172.21.15.3:0/4133489494 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588801c6b0 con 0x564d5d4ed390 2024-09-21T06:21:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.553+0000 7f589ace7740 1 -- 172.21.15.3:0/4133489494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4ed390 msgr2=0x564d5d8cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.553+0000 7f589ace7740 1 --2- 172.21.15.3:0/4133489494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4ed390 0x564d5d8cd070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5888001eb0 tx=0x7f58880055f0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.553+0000 7f589ace7740 1 -- 172.21.15.3:0/4133489494 shutdown_connections 2024-09-21T06:21:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.553+0000 7f589ace7740 1 --2- 172.21.15.3:0/4133489494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4ed390 0x564d5d8cd070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.553+0000 7f589ace7740 1 --2- 172.21.15.3:0/4133489494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d50e070 0x564d5d8b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.553+0000 7f589ace7740 1 --2- 172.21.15.3:0/4133489494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d4e51b0 0x564d5d8c1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.553+0000 7f589ace7740 1 -- 172.21.15.3:0/4133489494 >> 172.21.15.3:0/4133489494 conn(0x564d5d502460 msgr2=0x564d5d8c0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.553+0000 7f589ace7740 1 -- 172.21.15.3:0/4133489494 shutdown_connections 2024-09-21T06:21:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.553+0000 7f589ace7740 1 -- 172.21.15.3:0/4133489494 wait complete. 2024-09-21T06:21:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.553+0000 7f589ace7740 1 -- start start 2024-09-21T06:21:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.553+0000 7f589ace7740 1 Processor -- start 2024-09-21T06:21:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589ace7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4e51b0 0x564d5d8c5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589ace7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d4ed390 0x564d5d8c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589ace7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d50e070 0x564d5d8c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589ace7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1b0a80 con 0x564d5d4ed390 2024-09-21T06:21:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589ace7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1b03b0 con 0x564d5d50e070 2024-09-21T06:21:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589ace7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1ad5c0 con 0x564d5d4e51b0 2024-09-21T06:21:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589ace7740 10 client.0 _ll_register_callbacks cb 0x7f5888009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589250f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d4ed390 0x564d5d8c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589250f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d4ed390 0x564d5d8c5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39878/0 (socket says 172.21.15.3:39878) 2024-09-21T06:21:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589250f640 1 -- 172.21.15.3:0/1747986648 learned_addr learned my addr 172.21.15.3:0/1747986648 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f5891d0e640 1 --2- 172.21.15.3:0/1747986648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4e51b0 0x564d5d8c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589250f640 1 -- 172.21.15.3:0/1747986648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4e51b0 msgr2=0x564d5d8c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589250f640 1 --2- 172.21.15.3:0/1747986648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4e51b0 0x564d5d8c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589250f640 1 -- 172.21.15.3:0/1747986648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d50e070 msgr2=0x564d5d8c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589250f640 1 --2- 172.21.15.3:0/1747986648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d50e070 0x564d5d8c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589250f640 1 -- 172.21.15.3:0/1747986648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d5d93cb10 con 0x564d5d4ed390 2024-09-21T06:21:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f589250f640 1 --2- 172.21.15.3:0/1747986648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d4ed390 0x564d5d8c5640 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f588801c560 tx=0x7f588800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766b28e390e7fdf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f5875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.554+0000 7f5875ffb640 1 -- 172.21.15.3:0/1747986648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888029040 con 0x564d5d4ed390 2024-09-21T06:21:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.555+0000 7f5875ffb640 1 -- 172.21.15.3:0/1747986648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f588801c870 con 0x564d5d4ed390 2024-09-21T06:21:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.555+0000 7f589ace7740 1 -- 172.21.15.3:0/1747986648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5888003fe0 con 0x564d5d4ed390 2024-09-21T06:21:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.555+0000 7f5875ffb640 1 -- 172.21.15.3:0/1747986648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f588800fe90 con 0x564d5d4ed390 2024-09-21T06:21:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.555+0000 7f589ace7740 10 client.17319 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.555+0000 7f589ace7740 1 -- 172.21.15.3:0/1747986648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d5d93dd90 con 0x564d5d4ed390 2024-09-21T06:21:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.555+0000 7f5875ffb640 1 -- 172.21.15.3:0/1747986648 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5888003fe0 con 0x564d5d4ed390 2024-09-21T06:21:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.555+0000 7f5875ffb640 1 -- 172.21.15.3:0/1747986648 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f5888026460 con 0x564d5d4ed390 2024-09-21T06:21:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.555+0000 7f589ace7740 10 client.17319 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.555+0000 7f589ace7740 10 client.17319 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:21:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.555+0000 7f589ace7740 20 client.17319 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.555+0000 7f589ace7740 1 --2- 172.21.15.3:0/1747986648 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564d5d4a1da0 0x564d5d9404b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.555+0000 7f589ace7740 4 client.17319 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:21:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.555+0000 7f589ace7740 1 -- 172.21.15.3:0/1747986648 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d5d8298a0 con 0x564d5d4a1da0 2024-09-21T06:21:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f5891d0e640 1 --2- 172.21.15.3:0/1747986648 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564d5d4a1da0 0x564d5d9404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f5891d0e640 1 --2- 172.21.15.3:0/1747986648 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564d5d4a1da0 0x564d5d9404b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b982168131637a server_cookie=7d011d1ab4390346 in_seq=0 out_seq=0 2024-09-21T06:21:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f5875ffb640 10 client.17319 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:21:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f5875ffb640 1 -- 172.21.15.3:0/1747986648 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d5d8298a0 con 0x564d5d4a1da0 2024-09-21T06:21:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f5875ffb640 10 client.17319 handle_command_reply: tid=0 2024-09-21T06:21:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f589ace7740 1 client.17319 shutdown 2024-09-21T06:21:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f589ace7740 1 -- 172.21.15.3:0/1747986648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d4ed390 msgr2=0x564d5d8c5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f589ace7740 1 --2- 172.21.15.3:0/1747986648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d4ed390 0x564d5d8c5640 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f588801c560 tx=0x7f588800f790 comp rx=0 tx=0).stop 2024-09-21T06:21:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f589ace7740 1 -- 172.21.15.3:0/1747986648 shutdown_connections 2024-09-21T06:21:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f589ace7740 1 --2- 172.21.15.3:0/1747986648 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x564d5d4a1da0 0x564d5d9404b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f589ace7740 1 --2- 172.21.15.3:0/1747986648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d5d50e070 0x564d5d8c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f589ace7740 1 --2- 172.21.15.3:0/1747986648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d5d4ed390 0x564d5d8c5640 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f589ace7740 1 --2- 172.21.15.3:0/1747986648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d5d4e51b0 0x564d5d8c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f5875ffb640 0 client.17319 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:21:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f589ace7740 1 -- 172.21.15.3:0/1747986648 >> 172.21.15.3:0/1747986648 conn(0x564d5d502460 msgr2=0x564d5d8b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.556+0000 7f589ace7740 1 -- 172.21.15.3:0/1747986648 shutdown_connections 2024-09-21T06:21:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f589ace7740 1 -- 172.21.15.3:0/1747986648 wait complete. 2024-09-21T06:21:12.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f589ace7740 20 client.17319 trim_cache size 0 max 16384 2024-09-21T06:21:12.567 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:21:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f5894773640 1 -- 172.21.15.3:0/1769372124 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f58780b6680 msgr2=0x7f58780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f5894773640 1 --2- 172.21.15.3:0/1769372124 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f58780b6680 0x7f58780b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58841065e0 tx=0x7f588409bad0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f5894773640 1 -- 172.21.15.3:0/1769372124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588c108d20 msgr2=0x7f588c19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f5894773640 1 --2- 172.21.15.3:0/1769372124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588c108d20 0x7f588c19fcb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f588402f540 tx=0x7f58840316f0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f5894773640 1 -- 172.21.15.3:0/1769372124 shutdown_connections 2024-09-21T06:21:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f5894773640 1 --2- 172.21.15.3:0/1769372124 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f58780b6680 0x7f58780b8b90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f5894773640 1 --2- 172.21.15.3:0/1769372124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588c108d20 0x7f588c19fcb0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f5894773640 1 --2- 172.21.15.3:0/1769372124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f588c1083d0 0x7f588c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f5894773640 1 --2- 172.21.15.3:0/1769372124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f588c102ae0 0x7f588c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:12.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f5894773640 1 -- 172.21.15.3:0/1769372124 >> 172.21.15.3:0/1769372124 conn(0x7f588c079790 msgr2=0x7f588c101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:12.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f5894773640 1 -- 172.21.15.3:0/1769372124 shutdown_connections 2024-09-21T06:21:12.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:12.557+0000 7f5894773640 1 -- 172.21.15.3:0/1769372124 wait complete. 2024-09-21T06:21:12.959 INFO:tasks.workunit.client.0.smithi003.stdout: 1 648746 11.64 MB/sec execute 165 sec latency 21.770 ms 2024-09-21T06:21:13.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:13 smithi137 ceph-mon[34464]: pgmap v511: 97 pgs: 97 active+clean; 117 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 350 op/s 2024-09-21T06:21:13.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:13 smithi204 ceph-mon[34659]: pgmap v511: 97 pgs: 97 active+clean; 117 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 350 op/s 2024-09-21T06:21:13.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:13 smithi003 ceph-mon[31227]: pgmap v511: 97 pgs: 97 active+clean; 117 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 350 op/s 2024-09-21T06:21:13.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:21:13.959 INFO:tasks.workunit.client.0.smithi003.stdout: 1 650510 11.65 MB/sec execute 166 sec latency 15.614 ms 2024-09-21T06:21:14.125 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.669+0000 7f4a5b5fa640 1 -- 172.21.15.3:0/2540524630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a541083a0 msgr2=0x7f4a541087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.669+0000 7f4a5b5fa640 1 --2- 172.21.15.3:0/2540524630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a541083a0 0x7f4a541087b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009900 tx=0x7f4a4402a410 comp rx=0 tx=0).stop 2024-09-21T06:21:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.669+0000 7f4a5b5fa640 1 -- 172.21.15.3:0/2540524630 shutdown_connections 2024-09-21T06:21:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.669+0000 7f4a5b5fa640 1 --2- 172.21.15.3:0/2540524630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54108cf0 0x7f4a5410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.669+0000 7f4a5b5fa640 1 --2- 172.21.15.3:0/2540524630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a541083a0 0x7f4a541087b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.669+0000 7f4a5b5fa640 1 --2- 172.21.15.3:0/2540524630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a54102a60 0x7f4a54102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.669+0000 7f4a5b5fa640 1 -- 172.21.15.3:0/2540524630 >> 172.21.15.3:0/2540524630 conn(0x7f4a54079730 msgr2=0x7f4a54079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:14.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.669+0000 7f4a5b5fa640 1 -- 172.21.15.3:0/2540524630 shutdown_connections 2024-09-21T06:21:14.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.669+0000 7f4a5b5fa640 1 -- 172.21.15.3:0/2540524630 wait complete. 2024-09-21T06:21:14.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a5b5fa640 1 Processor -- start 2024-09-21T06:21:14.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a5b5fa640 1 -- start start 2024-09-21T06:21:14.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a5b5fa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54102a60 0x7f4a5419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a5b5fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a541083a0 0x7f4a5419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a5b5fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a54108cf0 0x7f4a5419fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a5b5fa640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54113a30 con 0x7f4a54108cf0 2024-09-21T06:21:14.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a5b5fa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541138b0 con 0x7f4a541083a0 2024-09-21T06:21:14.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a5b5fa640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54113bb0 con 0x7f4a54102a60 2024-09-21T06:21:14.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a59b97640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a54108cf0 0x7f4a5419fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a59b97640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a54108cf0 0x7f4a5419fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39900/0 (socket says 172.21.15.3:39900) 2024-09-21T06:21:14.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a59b97640 1 -- 172.21.15.3:0/2280489433 learned_addr learned my addr 172.21.15.3:0/2280489433 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:14.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a58b95640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a541083a0 0x7f4a5419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a59396640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54102a60 0x7f4a5419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a59b97640 1 -- 172.21.15.3:0/2280489433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54102a60 msgr2=0x7f4a5419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:14.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a59b97640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54102a60 0x7f4a5419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a59b97640 1 -- 172.21.15.3:0/2280489433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a541083a0 msgr2=0x7f4a5419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:14.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.670+0000 7f4a59b97640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a541083a0 0x7f4a5419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.671+0000 7f4a59b97640 1 -- 172.21.15.3:0/2280489433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a541a0550 con 0x7f4a54108cf0 2024-09-21T06:21:14.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.671+0000 7f4a59b97640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a54108cf0 0x7f4a5419fe20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00eb10 tx=0x7f4a4c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:14.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.671+0000 7f4a4a7fc640 1 -- 172.21.15.3:0/2280489433 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c004b20 con 0x7f4a54108cf0 2024-09-21T06:21:14.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.671+0000 7f4a4a7fc640 1 -- 172.21.15.3:0/2280489433 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4c004cc0 con 0x7f4a54108cf0 2024-09-21T06:21:14.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.671+0000 7f4a5b5fa640 1 -- 172.21.15.3:0/2280489433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a541a0840 con 0x7f4a54108cf0 2024-09-21T06:21:14.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.671+0000 7f4a4a7fc640 1 -- 172.21.15.3:0/2280489433 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c010670 con 0x7f4a54108cf0 2024-09-21T06:21:14.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.671+0000 7f4a5b5fa640 1 -- 172.21.15.3:0/2280489433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a541a6fa0 con 0x7f4a54108cf0 2024-09-21T06:21:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.674+0000 7f4a61b6e740 1 Processor -- start 2024-09-21T06:21:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.676+0000 7f4a4a7fc640 1 -- 172.21.15.3:0/2280489433 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f4a4c005020 con 0x7f4a54108cf0 2024-09-21T06:21:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.676+0000 7f4a61b6e740 1 -- start start 2024-09-21T06:21:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.676+0000 7f4a61b6e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222ace4070 0x56222b08b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.676+0000 7f4a61b6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222acbb1b0 0x56222b08b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.676+0000 7f4a61b6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acc3390 0x56222b08bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.676+0000 7f4a61b6e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56222b001ea0 con 0x56222ace4070 2024-09-21T06:21:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.676+0000 7f4a61b6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56222aff12d0 con 0x56222acc3390 2024-09-21T06:21:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.676+0000 7f4a61b6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56222aff50e0 con 0x56222acbb1b0 2024-09-21T06:21:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a59396640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222acbb1b0 0x56222b08b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a58b95640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222ace4070 0x56222b08b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a59396640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222acbb1b0 0x56222b08b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:58044/0 (socket says 172.21.15.3:58044) 2024-09-21T06:21:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a59396640 1 -- 172.21.15.3:0/1833526647 learned_addr learned my addr 172.21.15.3:0/1833526647 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a58b95640 1 --2- 172.21.15.3:0/1833526647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acc3390 0x56222b08bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a59396640 1 -- 172.21.15.3:0/1833526647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acc3390 msgr2=0x56222b08bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a59396640 1 --2- 172.21.15.3:0/1833526647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acc3390 0x56222b08bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a59396640 1 -- 172.21.15.3:0/1833526647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222ace4070 msgr2=0x56222b08b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a59396640 1 --2- 172.21.15.3:0/1833526647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222ace4070 0x56222b08b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a59396640 1 -- 172.21.15.3:0/1833526647 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56222ad4f650 con 0x56222acbb1b0 2024-09-21T06:21:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a58b95640 1 --2- 172.21.15.3:0/1833526647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222ace4070 0x56222b08b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a4a7fc640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4a400b6680 0x7f4a400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a59396640 1 --2- 172.21.15.3:0/1833526647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222acbb1b0 0x56222b08b800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009cf0 tx=0x7f4a50011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c29e8cd963eb19d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a3f7fe640 1 -- 172.21.15.3:0/1833526647 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50004230 con 0x56222acbb1b0 2024-09-21T06:21:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.677+0000 7f4a3f7fe640 1 -- 172.21.15.3:0/1833526647 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a500043d0 con 0x56222acbb1b0 2024-09-21T06:21:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.678+0000 7f4a3f7fe640 1 -- 172.21.15.3:0/1833526647 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50019880 con 0x56222acbb1b0 2024-09-21T06:21:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.678+0000 7f4a58b95640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4a400b6680 0x7f4a400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.678+0000 7f4a4a7fc640 1 -- 172.21.15.3:0/2280489433 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f4a4c0e7c60 con 0x7f4a54108cf0 2024-09-21T06:21:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.678+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1833526647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222acbb1b0 msgr2=0x56222b08b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.678+0000 7f4a58b95640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4a400b6680 0x7f4a400b8b90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4a44000c00 tx=0x7f4a44031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.678+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1833526647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222acbb1b0 0x56222b08b800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009cf0 tx=0x7f4a50011810 comp rx=0 tx=0).stop 2024-09-21T06:21:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.679+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1833526647 shutdown_connections 2024-09-21T06:21:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.679+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1833526647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222acbb1b0 0x56222b08b800 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.679+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1833526647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acc3390 0x56222b08bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.679+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1833526647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222ace4070 0x56222b08b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.679+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1833526647 >> 172.21.15.3:0/1833526647 conn(0x56222acd8460 msgr2=0x56222b0891b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.679+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1833526647 shutdown_connections 2024-09-21T06:21:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.679+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1833526647 wait complete. 2024-09-21T06:21:14.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.679+0000 7f4a61b6e740 1 -- start start 2024-09-21T06:21:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.679+0000 7f4a61b6e740 1 Processor -- start 2024-09-21T06:21:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a61b6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acbb1b0 0x56222b0968e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a61b6e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acc3390 0x56222b096e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a61b6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222ace4070 0x56222b097360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a61b6e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56222ae52700 con 0x56222acc3390 2024-09-21T06:21:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a61b6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56222af8c160 con 0x56222acbb1b0 2024-09-21T06:21:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a61b6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56222b09deb0 con 0x56222ace4070 2024-09-21T06:21:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a61b6e740 10 client.0 _ll_register_callbacks cb 0x56222afa59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a59396640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acbb1b0 0x56222b0968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a58b95640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acc3390 0x56222b096e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a59396640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acbb1b0 0x56222b0968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:32956/0 (socket says 172.21.15.3:32956) 2024-09-21T06:21:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a59396640 1 -- 172.21.15.3:0/1176725282 learned_addr learned my addr 172.21.15.3:0/1176725282 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:14.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a59b97640 1 --2- 172.21.15.3:0/1176725282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222ace4070 0x56222b097360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a59b97640 1 -- 172.21.15.3:0/1176725282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acbb1b0 msgr2=0x56222b0968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a59b97640 1 --2- 172.21.15.3:0/1176725282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acbb1b0 0x56222b0968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a59b97640 1 -- 172.21.15.3:0/1176725282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acc3390 msgr2=0x56222b096e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a59b97640 1 --2- 172.21.15.3:0/1176725282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acc3390 0x56222b096e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a59b97640 1 -- 172.21.15.3:0/1176725282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56222afb9c90 con 0x56222ace4070 2024-09-21T06:21:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.680+0000 7f4a59396640 1 --2- 172.21.15.3:0/1176725282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acbb1b0 0x56222b0968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a59b97640 1 --2- 172.21.15.3:0/1176725282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222ace4070 0x56222b097360 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0e53f0 tx=0x7f4a4c0a9ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=970875df7cc5d792 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:21:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a3cff9640 1 -- 172.21.15.3:0/1176725282 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c0b2070 con 0x56222ace4070 2024-09-21T06:21:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a3cff9640 1 -- 172.21.15.3:0/1176725282 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f4a4c0b2210 con 0x56222ace4070 2024-09-21T06:21:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1176725282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a500043d0 con 0x56222ace4070 2024-09-21T06:21:14.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a3cff9640 1 -- 172.21.15.3:0/1176725282 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4a4c0ad600 con 0x56222ace4070 2024-09-21T06:21:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a61b6e740 10 client.26842 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1176725282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56222b0a9b20 con 0x56222ace4070 2024-09-21T06:21:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a3cff9640 1 -- 172.21.15.3:0/1176725282 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a500043d0 con 0x56222ace4070 2024-09-21T06:21:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a3cff9640 1 -- 172.21.15.3:0/1176725282 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f4a4c0acd50 con 0x56222ace4070 2024-09-21T06:21:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a61b6e740 10 client.26842 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a61b6e740 10 client.26842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a61b6e740 20 client.26842 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.681+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1176725282 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56222aca29f0 0x56222b0ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.682+0000 7f4a61b6e740 4 client.26842 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.682+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1176725282 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56222b003df0 con 0x56222aca29f0 2024-09-21T06:21:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.682+0000 7f4a59396640 1 --2- 172.21.15.3:0/1176725282 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56222aca29f0 0x56222b0ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.682+0000 7f4a59396640 1 --2- 172.21.15.3:0/1176725282 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56222aca29f0 0x56222b0ad030 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa434550c3ef144 server_cookie=1a930dca6e90f8f2 in_seq=0 out_seq=0 2024-09-21T06:21:14.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.682+0000 7f4a3cff9640 10 client.26842 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:14.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.683+0000 7f4a3cff9640 1 -- 172.21.15.3:0/1176725282 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56222b003df0 con 0x56222aca29f0 2024-09-21T06:21:14.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.683+0000 7f4a3cff9640 10 client.26842 handle_command_reply: tid=0 2024-09-21T06:21:14.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.683+0000 7f4a61b6e740 1 client.26842 shutdown 2024-09-21T06:21:14.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.683+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1176725282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222ace4070 msgr2=0x56222b097360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:14.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.683+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1176725282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222ace4070 0x56222b097360 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0e53f0 tx=0x7f4a4c0a9ac0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.683+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1176725282 shutdown_connections 2024-09-21T06:21:14.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.683+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1176725282 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56222aca29f0 0x56222b0ad030 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.683+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1176725282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222ace4070 0x56222b097360 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.683+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1176725282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acc3390 0x56222b096e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.683+0000 7f4a3cff9640 0 client.26842 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:14.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.683+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1176725282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acbb1b0 0x56222b0968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.683+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1176725282 >> 172.21.15.3:0/1176725282 conn(0x56222acd8460 msgr2=0x56222b08e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:14.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.684+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1176725282 shutdown_connections 2024-09-21T06:21:14.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.684+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1176725282 wait complete. 2024-09-21T06:21:14.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.684+0000 7f4a61b6e740 20 client.26842 trim_cache size 0 max 16384 2024-09-21T06:21:14.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a61b6e740 1 Processor -- start 2024-09-21T06:21:14.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a61b6e740 1 -- start start 2024-09-21T06:21:14.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a61b6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222acbb1b0 0x56222b0986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a61b6e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acc3390 0x56222b088760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a61b6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222ace4070 0x56222b088ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a61b6e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ad4f0 con 0x56222acc3390 2024-09-21T06:21:14.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a61b6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541af060 con 0x56222ace4070 2024-09-21T06:21:14.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a61b6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541adbc0 con 0x56222acbb1b0 2024-09-21T06:21:14.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a58b95640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acc3390 0x56222b088760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a58b95640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acc3390 0x56222b088760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39926/0 (socket says 172.21.15.3:39926) 2024-09-21T06:21:14.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a58b95640 1 -- 172.21.15.3:0/2550101886 learned_addr learned my addr 172.21.15.3:0/2550101886 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:14.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a59396640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222acbb1b0 0x56222b0986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a59b97640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222ace4070 0x56222b088ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a58b95640 1 -- 172.21.15.3:0/2550101886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222acbb1b0 msgr2=0x56222b0986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:14.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a58b95640 1 --2- 172.21.15.3:0/2550101886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222acbb1b0 0x56222b0986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a58b95640 1 -- 172.21.15.3:0/2550101886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222ace4070 msgr2=0x56222b088ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:14.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a58b95640 1 --2- 172.21.15.3:0/2550101886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222ace4070 0x56222b088ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a58b95640 1 -- 172.21.15.3:0/2550101886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56222b091540 con 0x56222acc3390 2024-09-21T06:21:14.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a59b97640 1 --2- 172.21.15.3:0/2550101886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222ace4070 0x56222b088ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:14.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.698+0000 7f4a59396640 1 --2- 172.21.15.3:0/2550101886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222acbb1b0 0x56222b0986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:14.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.699+0000 7f4a58b95640 1 --2- 172.21.15.3:0/2550101886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acc3390 0x56222b088760 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009900 tx=0x7f4a4403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d5991b4b18be60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:14.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.699+0000 7f4a3effd640 1 -- 172.21.15.3:0/2550101886 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4404f070 con 0x56222acc3390 2024-09-21T06:21:14.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.699+0000 7f4a3effd640 1 -- 172.21.15.3:0/2550101886 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a44004240 con 0x56222acc3390 2024-09-21T06:21:14.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.699+0000 7f4a3effd640 1 -- 172.21.15.3:0/2550101886 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44035040 con 0x56222acc3390 2024-09-21T06:21:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.700+0000 7f4a61b6e740 1 -- 172.21.15.3:0/2550101886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acc3390 msgr2=0x56222b088760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.700+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/2550101886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acc3390 0x56222b088760 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009900 tx=0x7f4a4403f000 comp rx=0 tx=0).stop 2024-09-21T06:21:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.700+0000 7f4a61b6e740 1 -- 172.21.15.3:0/2550101886 shutdown_connections 2024-09-21T06:21:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.700+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/2550101886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222ace4070 0x56222b088ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.700+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/2550101886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acc3390 0x56222b088760 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.700+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/2550101886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222acbb1b0 0x56222b0986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.700+0000 7f4a61b6e740 1 -- 172.21.15.3:0/2550101886 >> 172.21.15.3:0/2550101886 conn(0x56222acd8460 msgr2=0x56222b08ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.700+0000 7f4a61b6e740 1 -- 172.21.15.3:0/2550101886 shutdown_connections 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.700+0000 7f4a61b6e740 1 -- 172.21.15.3:0/2550101886 wait complete. 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.700+0000 7f4a61b6e740 1 -- start start 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.700+0000 7f4a61b6e740 1 Processor -- start 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a61b6e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acbb1b0 0x56222b0a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a61b6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acc3390 0x56222b0a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a61b6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222ace4070 0x56222b09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a61b6e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ae990 con 0x56222acbb1b0 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a61b6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ae2c0 con 0x56222acc3390 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a61b6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ab4d0 con 0x56222ace4070 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a61b6e740 10 client.0 _ll_register_callbacks cb 0x7f4a4c0d1280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a59396640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acbb1b0 0x56222b0a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a58b95640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acc3390 0x56222b0a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a59b97640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222ace4070 0x56222b09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a59396640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acbb1b0 0x56222b0a35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39932/0 (socket says 172.21.15.3:39932) 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a59396640 1 -- 172.21.15.3:0/1481723454 learned_addr learned my addr 172.21.15.3:0/1481723454 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a59396640 1 -- 172.21.15.3:0/1481723454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222ace4070 msgr2=0x56222b09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a59396640 1 --2- 172.21.15.3:0/1481723454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222ace4070 0x56222b09ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a59396640 1 -- 172.21.15.3:0/1481723454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acc3390 msgr2=0x56222b0a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a59396640 1 --2- 172.21.15.3:0/1481723454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acc3390 0x56222b0a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a59396640 1 -- 172.21.15.3:0/1481723454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56222b09f3b0 con 0x56222acbb1b0 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.701+0000 7f4a58b95640 1 --2- 172.21.15.3:0/1481723454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acc3390 0x56222b0a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a59396640 1 --2- 172.21.15.3:0/1481723454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acbb1b0 0x56222b0a35c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4a50019490 tx=0x7f4a50009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5def8678bbd06d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a3cff9640 1 -- 172.21.15.3:0/1481723454 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500037f0 con 0x56222acbb1b0 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a3cff9640 1 -- 172.21.15.3:0/1481723454 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f4a50003990 con 0x56222acbb1b0 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1481723454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a44004240 con 0x56222acbb1b0 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a3cff9640 1 -- 172.21.15.3:0/1481723454 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4a5003a050 con 0x56222acbb1b0 2024-09-21T06:21:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a61b6e740 10 client.17337 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:14.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1481723454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56222b1175b0 con 0x56222acbb1b0 2024-09-21T06:21:14.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a3cff9640 1 -- 172.21.15.3:0/1481723454 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a44004240 con 0x56222acbb1b0 2024-09-21T06:21:14.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a3cff9640 1 -- 172.21.15.3:0/1481723454 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f4a50004800 con 0x56222acbb1b0 2024-09-21T06:21:14.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a61b6e740 10 client.17337 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:14.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a61b6e740 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:14.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a61b6e740 20 client.17337 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:14.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1481723454 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56222ac77da0 0x56222b119b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:14.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a61b6e740 4 client.17337 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:21:14.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.702+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1481723454 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56222afff8a0 con 0x56222ac77da0 2024-09-21T06:21:14.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.703+0000 7f4a58b95640 1 --2- 172.21.15.3:0/1481723454 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56222ac77da0 0x56222b119b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:14.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.703+0000 7f4a3cff9640 10 client.17337 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:14.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.703+0000 7f4a58b95640 1 --2- 172.21.15.3:0/1481723454 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56222ac77da0 0x56222b119b80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8703d52bfd5fd5f server_cookie=641c81888687f9e4 in_seq=0 out_seq=0 2024-09-21T06:21:14.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a3cff9640 1 -- 172.21.15.3:0/1481723454 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56222afff8a0 con 0x56222ac77da0 2024-09-21T06:21:14.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a3cff9640 10 client.17337 handle_command_reply: tid=0 2024-09-21T06:21:14.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a61b6e740 1 client.17337 shutdown 2024-09-21T06:21:14.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1481723454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acbb1b0 msgr2=0x56222b0a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:14.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1481723454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acbb1b0 0x56222b0a35c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4a50019490 tx=0x7f4a50009910 comp rx=0 tx=0).stop 2024-09-21T06:21:14.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1481723454 shutdown_connections 2024-09-21T06:21:14.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1481723454 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56222ac77da0 0x56222b119b80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1481723454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222ace4070 0x56222b09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1481723454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56222acc3390 0x56222b0a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a61b6e740 1 --2- 172.21.15.3:0/1481723454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56222acbb1b0 0x56222b0a35c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1481723454 >> 172.21.15.3:0/1481723454 conn(0x56222acd8460 msgr2=0x56222b0a60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:14.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a3cff9640 0 client.17337 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:14.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1481723454 shutdown_connections 2024-09-21T06:21:14.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.704+0000 7f4a61b6e740 1 -- 172.21.15.3:0/1481723454 wait complete. 2024-09-21T06:21:14.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.705+0000 7f4a61b6e740 20 client.17337 trim_cache size 0 max 16384 2024-09-21T06:21:14.711 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:21:14.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.705+0000 7f4a5b5fa640 1 -- 172.21.15.3:0/2280489433 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4a400b6680 msgr2=0x7f4a400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:14.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.705+0000 7f4a5b5fa640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4a400b6680 0x7f4a400b8b90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4a44000c00 tx=0x7f4a44031400 comp rx=0 tx=0).stop 2024-09-21T06:21:14.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.705+0000 7f4a5b5fa640 1 -- 172.21.15.3:0/2280489433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a54108cf0 msgr2=0x7f4a5419fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:14.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.705+0000 7f4a5b5fa640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a54108cf0 0x7f4a5419fe20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00eb10 tx=0x7f4a4c00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.705+0000 7f4a5b5fa640 1 -- 172.21.15.3:0/2280489433 shutdown_connections 2024-09-21T06:21:14.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.705+0000 7f4a5b5fa640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4a400b6680 0x7f4a400b8b90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.705+0000 7f4a5b5fa640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a54108cf0 0x7f4a5419fe20 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.705+0000 7f4a5b5fa640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a541083a0 0x7f4a5419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.705+0000 7f4a5b5fa640 1 --2- 172.21.15.3:0/2280489433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54102a60 0x7f4a5419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:14.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.705+0000 7f4a5b5fa640 1 -- 172.21.15.3:0/2280489433 >> 172.21.15.3:0/2280489433 conn(0x7f4a54079730 msgr2=0x7f4a541044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:14.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.707+0000 7f4a5b5fa640 1 -- 172.21.15.3:0/2280489433 shutdown_connections 2024-09-21T06:21:14.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:14.707+0000 7f4a5b5fa640 1 -- 172.21.15.3:0/2280489433 wait complete. 2024-09-21T06:21:14.959 INFO:tasks.workunit.client.0.smithi003.stdout: 1 652370 11.62 MB/sec execute 167 sec latency 95.827 ms 2024-09-21T06:21:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:15 smithi003 ceph-mon[31227]: pgmap v512: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 323 op/s 2024-09-21T06:21:15.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:15 smithi137 ceph-mon[34464]: pgmap v512: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 323 op/s 2024-09-21T06:21:15.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:15 smithi204 ceph-mon[34659]: pgmap v512: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 323 op/s 2024-09-21T06:21:15.959 INFO:tasks.workunit.client.0.smithi003.stdout: 1 654107 11.62 MB/sec execute 168 sec latency 14.257 ms 2024-09-21T06:21:16.523 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:21:16.523 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:21:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.850+0000 7f26d9389640 1 -- 172.21.15.3:0/3094757305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d4105280 msgr2=0x7f26d410d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.850+0000 7f26d9389640 1 --2- 172.21.15.3:0/3094757305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d4105280 0x7f26d410d7c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009900 tx=0x7f26c802a410 comp rx=0 tx=0).stop 2024-09-21T06:21:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.850+0000 7f26d9389640 1 -- 172.21.15.3:0/3094757305 shutdown_connections 2024-09-21T06:21:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.850+0000 7f26d9389640 1 --2- 172.21.15.3:0/3094757305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d410dd80 0x7f26d4110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.850+0000 7f26d9389640 1 --2- 172.21.15.3:0/3094757305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d4105280 0x7f26d410d7c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.850+0000 7f26d9389640 1 --2- 172.21.15.3:0/3094757305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26d4104930 0x7f26d4104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.850+0000 7f26d9389640 1 -- 172.21.15.3:0/3094757305 >> 172.21.15.3:0/3094757305 conn(0x7f26d407a6e0 msgr2=0x7f26d407aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.850+0000 7f26d9389640 1 -- 172.21.15.3:0/3094757305 shutdown_connections 2024-09-21T06:21:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.850+0000 7f26d9389640 1 -- 172.21.15.3:0/3094757305 wait complete. 2024-09-21T06:21:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.851+0000 7f26d9389640 1 Processor -- start 2024-09-21T06:21:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.851+0000 7f26d9389640 1 -- start start 2024-09-21T06:21:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.851+0000 7f26d9389640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26d4104930 0x7f26d4102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.851+0000 7f26d9389640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d4105280 0x7f26d4102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d9389640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d410dd80 0x7f26d41030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d9389640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41139e0 con 0x7f26d4105280 2024-09-21T06:21:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d9389640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f26d4113860 con 0x7f26d4104930 2024-09-21T06:21:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d9389640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26d4113b60 con 0x7f26d410dd80 2024-09-21T06:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26d4104930 0x7f26d4102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d4105280 0x7f26d4102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26d4104930 0x7f26d4102660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:58448/0 (socket says 172.21.15.3:58448) 2024-09-21T06:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d410dd80 0x7f26d41030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d2ffd640 1 -- 172.21.15.3:0/104683550 learned_addr learned my addr 172.21.15.3:0/104683550 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d2ffd640 1 -- 172.21.15.3:0/104683550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d410dd80 msgr2=0x7f26d41030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/104683550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d410dd80 0x7f26d41030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d2ffd640 1 -- 172.21.15.3:0/104683550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d4105280 msgr2=0x7f26d4102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/104683550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d4105280 0x7f26d4102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d2ffd640 1 -- 172.21.15.3:0/104683550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d41a86d0 con 0x7f26d4104930 2024-09-21T06:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d37fe640 1 --2- 172.21.15.3:0/104683550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d410dd80 0x7f26d41030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.852+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/104683550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26d4104930 0x7f26d4102660 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f26c400e9f0 tx=0x7f26c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.853+0000 7f26bbfff640 1 -- 172.21.15.3:0/104683550 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c400cd80 con 0x7f26d4104930 2024-09-21T06:21:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.853+0000 7f26d9389640 1 -- 172.21.15.3:0/104683550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d41a89c0 con 0x7f26d4104930 2024-09-21T06:21:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.853+0000 7f26bbfff640 1 -- 172.21.15.3:0/104683550 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c40108e0 con 0x7f26d4104930 2024-09-21T06:21:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.853+0000 7f26bbfff640 1 -- 172.21.15.3:0/104683550 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4018980 con 0x7f26d4104930 2024-09-21T06:21:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.853+0000 7f26d9389640 1 -- 172.21.15.3:0/104683550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d41a8ee0 con 0x7f26d4104930 2024-09-21T06:21:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.855+0000 7f26e03d0740 1 Processor -- start 2024-09-21T06:21:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.855+0000 7f26bbfff640 1 -- 172.21.15.3:0/104683550 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f26c4018b20 con 0x7f26d4104930 2024-09-21T06:21:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.855+0000 7f26e03d0740 1 -- start start 2024-09-21T06:21:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26e03d0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682c031b0 0x558682fa89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26e03d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682bda2f0 0x558682fa8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26e03d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682be24d0 0x558682fa9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26e03d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558682f1d6f0 con 0x558682be24d0 2024-09-21T06:21:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26e03d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558682f0cb20 con 0x558682bda2f0 2024-09-21T06:21:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26e03d0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558682f10930 con 0x558682c031b0 2024-09-21T06:21:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682bda2f0 0x558682fa8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682be24d0 0x558682fa9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682bda2f0 0x558682fa8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:58454/0 (socket says 172.21.15.3:58454) 2024-09-21T06:21:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26d2ffd640 1 -- 172.21.15.3:0/1372560855 learned_addr learned my addr 172.21.15.3:0/1372560855 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26d37fe640 1 --2- 172.21.15.3:0/1372560855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682c031b0 0x558682fa89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26d2ffd640 1 -- 172.21.15.3:0/1372560855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682c031b0 msgr2=0x558682fa89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/1372560855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682c031b0 0x558682fa89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26d2ffd640 1 -- 172.21.15.3:0/1372560855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682be24d0 msgr2=0x558682fa9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/1372560855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682be24d0 0x558682fa9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26d2ffd640 1 -- 172.21.15.3:0/1372560855 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558682ed2530 con 0x558682bda2f0 2024-09-21T06:21:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26bbfff640 1 --2- 172.21.15.3:0/104683550 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f26bc0b66b0 0x7f26bc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.856+0000 7f26d37fe640 1 --2- 172.21.15.3:0/104683550 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f26bc0b66b0 0x7f26bc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.857+0000 7f26bbfff640 1 -- 172.21.15.3:0/104683550 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f26c40e79b0 con 0x7f26d4104930 2024-09-21T06:21:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.857+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/1372560855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682bda2f0 0x558682fa8ef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f26c40e5080 tx=0x7f26c40f04f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d9452a4b32c1bc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.857+0000 7f26b97fa640 1 -- 172.21.15.3:0/1372560855 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40e7e60 con 0x558682bda2f0 2024-09-21T06:21:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.857+0000 7f26b97fa640 1 -- 172.21.15.3:0/1372560855 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c40a9050 con 0x558682bda2f0 2024-09-21T06:21:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.857+0000 7f26b97fa640 1 -- 172.21.15.3:0/1372560855 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40ad240 con 0x558682bda2f0 2024-09-21T06:21:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.858+0000 7f26e03d0740 1 -- 172.21.15.3:0/1372560855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682bda2f0 msgr2=0x558682fa8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.858+0000 7f26d37fe640 1 --2- 172.21.15.3:0/104683550 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f26bc0b66b0 0x7f26bc0b8bc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009fd0 tx=0x7f26cc009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.858+0000 7f26e03d0740 1 --2- 172.21.15.3:0/1372560855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682bda2f0 0x558682fa8ef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f26c40e5080 tx=0x7f26c40f04f0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.858+0000 7f26e03d0740 1 -- 172.21.15.3:0/1372560855 shutdown_connections 2024-09-21T06:21:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.858+0000 7f26e03d0740 1 --2- 172.21.15.3:0/1372560855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682be24d0 0x558682fa9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.858+0000 7f26e03d0740 1 --2- 172.21.15.3:0/1372560855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682bda2f0 0x558682fa8ef0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.858+0000 7f26e03d0740 1 --2- 172.21.15.3:0/1372560855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682c031b0 0x558682fa89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.858+0000 7f26e03d0740 1 -- 172.21.15.3:0/1372560855 >> 172.21.15.3:0/1372560855 conn(0x558682bf7300 msgr2=0x558682fa68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.858+0000 7f26e03d0740 1 -- 172.21.15.3:0/1372560855 shutdown_connections 2024-09-21T06:21:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.858+0000 7f26e03d0740 1 -- 172.21.15.3:0/1372560855 wait complete. 2024-09-21T06:21:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.858+0000 7f26e03d0740 1 -- start start 2024-09-21T06:21:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.858+0000 7f26e03d0740 1 Processor -- start 2024-09-21T06:21:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26e03d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682bda2f0 0x558682fb40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26e03d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682be24d0 0x558682fb4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26e03d0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682c031b0 0x558682fc03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26e03d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558682e44ab0 con 0x558682be24d0 2024-09-21T06:21:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26e03d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558682eb2730 con 0x558682bda2f0 2024-09-21T06:21:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26e03d0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558682fb70d0 con 0x558682c031b0 2024-09-21T06:21:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26e03d0740 10 client.0 _ll_register_callbacks cb 0x558682f066b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682be24d0 0x558682fb4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682bda2f0 0x558682fb40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682be24d0 0x558682fb4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47272/0 (socket says 172.21.15.3:47272) 2024-09-21T06:21:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26d27fc640 1 -- 172.21.15.3:0/3260979826 learned_addr learned my addr 172.21.15.3:0/3260979826 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682bda2f0 0x558682fb40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:58458/0 (socket says 172.21.15.3:58458) 2024-09-21T06:21:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26d37fe640 1 --2- 172.21.15.3:0/3260979826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682c031b0 0x558682fc03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26d27fc640 1 -- 172.21.15.3:0/3260979826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682c031b0 msgr2=0x558682fc03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26d27fc640 1 --2- 172.21.15.3:0/3260979826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682c031b0 0x558682fc03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26d27fc640 1 -- 172.21.15.3:0/3260979826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682bda2f0 msgr2=0x558682fb40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26d27fc640 1 --2- 172.21.15.3:0/3260979826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682bda2f0 0x558682fb40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26d27fc640 1 -- 172.21.15.3:0/3260979826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558682c6e790 con 0x558682be24d0 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26d27fc640 1 --2- 172.21.15.3:0/3260979826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682be24d0 0x558682fb4630 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f26d406c980 tx=0x7f26c8032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8844f626aed218 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26b6ff5640 1 -- 172.21.15.3:0/3260979826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8035040 con 0x558682be24d0 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26b6ff5640 1 -- 172.21.15.3:0/3260979826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f26c80317e0 con 0x558682be24d0 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.859+0000 7f26e03d0740 1 -- 172.21.15.3:0/3260979826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c40a9050 con 0x558682be24d0 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.860+0000 7f26b6ff5640 1 -- 172.21.15.3:0/3260979826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f26c8004040 con 0x558682be24d0 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.860+0000 7f26e03d0740 10 client.17349 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.860+0000 7f26e03d0740 1 -- 172.21.15.3:0/3260979826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558682ce7620 con 0x558682be24d0 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.860+0000 7f26b6ff5640 1 -- 172.21.15.3:0/3260979826 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c40a9050 con 0x558682be24d0 2024-09-21T06:21:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.860+0000 7f26b6ff5640 1 -- 172.21.15.3:0/3260979826 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f26c8004750 con 0x558682be24d0 2024-09-21T06:21:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.860+0000 7f26e03d0740 10 client.17349 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.860+0000 7f26e03d0740 10 client.17349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.860+0000 7f26e03d0740 20 client.17349 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.860+0000 7f26e03d0740 1 --2- 172.21.15.3:0/3260979826 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558682bc1d80 0x558682fc61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.860+0000 7f26e03d0740 4 client.17349 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.860+0000 7f26e03d0740 1 -- 172.21.15.3:0/3260979826 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558682f1f640 con 0x558682bc1d80 2024-09-21T06:21:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.861+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/3260979826 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558682bc1d80 0x558682fc61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.861+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/3260979826 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558682bc1d80 0x558682fc61b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a27c262e7f73619 server_cookie=24052cd70f71a365 in_seq=0 out_seq=0 2024-09-21T06:21:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.861+0000 7f26b6ff5640 10 client.17349 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26b6ff5640 1 -- 172.21.15.3:0/3260979826 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558682f1f640 con 0x558682bc1d80 2024-09-21T06:21:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26b6ff5640 10 client.17349 handle_command_reply: tid=0 2024-09-21T06:21:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26e03d0740 1 client.17349 shutdown 2024-09-21T06:21:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26e03d0740 1 -- 172.21.15.3:0/3260979826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682be24d0 msgr2=0x558682fb4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26e03d0740 1 --2- 172.21.15.3:0/3260979826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682be24d0 0x558682fb4630 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f26d406c980 tx=0x7f26c8032eb0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26e03d0740 1 -- 172.21.15.3:0/3260979826 shutdown_connections 2024-09-21T06:21:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26e03d0740 1 --2- 172.21.15.3:0/3260979826 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558682bc1d80 0x558682fc61b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26e03d0740 1 --2- 172.21.15.3:0/3260979826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682c031b0 0x558682fc03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26e03d0740 1 --2- 172.21.15.3:0/3260979826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682be24d0 0x558682fb4630 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26b6ff5640 0 client.17349 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26e03d0740 1 --2- 172.21.15.3:0/3260979826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682bda2f0 0x558682fb40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26e03d0740 1 -- 172.21.15.3:0/3260979826 >> 172.21.15.3:0/3260979826 conn(0x558682bf7300 msgr2=0x558682fa99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26e03d0740 1 -- 172.21.15.3:0/3260979826 shutdown_connections 2024-09-21T06:21:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26e03d0740 1 -- 172.21.15.3:0/3260979826 wait complete. 2024-09-21T06:21:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.862+0000 7f26e03d0740 20 client.17349 trim_cache size 0 max 16384 2024-09-21T06:21:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26e03d0740 1 Processor -- start 2024-09-21T06:21:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26e03d0740 1 -- start start 2024-09-21T06:21:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26e03d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 0x558682fb45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26e03d0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682bda2f0 0x558682fb5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26e03d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682be24d0 0x558682fc02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26e03d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41b1840 con 0x558682bc1d80 2024-09-21T06:21:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26e03d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41b33b0 con 0x558682be24d0 2024-09-21T06:21:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26e03d0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41b1f10 con 0x558682bda2f0 2024-09-21T06:21:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 0x558682fb45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 0x558682fb45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47284/0 (socket says 172.21.15.3:47284) 2024-09-21T06:21:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26d2ffd640 1 -- 172.21.15.3:0/913680456 learned_addr learned my addr 172.21.15.3:0/913680456 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26d27fc640 1 --2- 172.21.15.3:0/913680456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682bda2f0 0x558682fb5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26d37fe640 1 --2- 172.21.15.3:0/913680456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682be24d0 0x558682fc02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26d2ffd640 1 -- 172.21.15.3:0/913680456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682bda2f0 msgr2=0x558682fb5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/913680456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682bda2f0 0x558682fb5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26d2ffd640 1 -- 172.21.15.3:0/913680456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682be24d0 msgr2=0x558682fc02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/913680456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682be24d0 0x558682fc02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.880+0000 7f26d2ffd640 1 -- 172.21.15.3:0/913680456 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558682fc6710 con 0x558682bc1d80 2024-09-21T06:21:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.881+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/913680456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 0x558682fb45d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f26c40aab60 tx=0x7f26c4018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a7b1cf0078a259 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.881+0000 7f26b8ff9640 1 -- 172.21.15.3:0/913680456 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40e7e60 con 0x558682bc1d80 2024-09-21T06:21:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.881+0000 7f26b8ff9640 1 -- 172.21.15.3:0/913680456 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c40ad400 con 0x558682bc1d80 2024-09-21T06:21:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.881+0000 7f26b8ff9640 1 -- 172.21.15.3:0/913680456 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40ad060 con 0x558682bc1d80 2024-09-21T06:21:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.882+0000 7f26e03d0740 1 -- 172.21.15.3:0/913680456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 msgr2=0x558682fb45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.882+0000 7f26e03d0740 1 --2- 172.21.15.3:0/913680456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 0x558682fb45d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f26c40aab60 tx=0x7f26c4018420 comp rx=0 tx=0).stop 2024-09-21T06:21:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.882+0000 7f26e03d0740 1 -- 172.21.15.3:0/913680456 shutdown_connections 2024-09-21T06:21:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.882+0000 7f26e03d0740 1 --2- 172.21.15.3:0/913680456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682bda2f0 0x558682fb5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.882+0000 7f26e03d0740 1 --2- 172.21.15.3:0/913680456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682be24d0 0x558682fc02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.882+0000 7f26e03d0740 1 --2- 172.21.15.3:0/913680456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 0x558682fb45d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.882+0000 7f26e03d0740 1 -- 172.21.15.3:0/913680456 >> 172.21.15.3:0/913680456 conn(0x558682bf7300 msgr2=0x558682fb5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.882+0000 7f26e03d0740 1 -- 172.21.15.3:0/913680456 shutdown_connections 2024-09-21T06:21:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.882+0000 7f26e03d0740 1 -- 172.21.15.3:0/913680456 wait complete. 2024-09-21T06:21:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.882+0000 7f26e03d0740 1 -- start start 2024-09-21T06:21:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.882+0000 7f26e03d0740 1 Processor -- start 2024-09-21T06:21:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26e03d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 0x558683028d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26e03d0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682bda2f0 0x558683029320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26e03d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682be24d0 0x558682fbc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26e03d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41b2ce0 con 0x558682bc1d80 2024-09-21T06:21:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26e03d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41b2610 con 0x558682be24d0 2024-09-21T06:21:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26e03d0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41af820 con 0x558682bda2f0 2024-09-21T06:21:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26e03d0740 10 client.0 _ll_register_callbacks cb 0x7f26c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 0x558683028d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 0x558683028d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47292/0 (socket says 172.21.15.3:47292) 2024-09-21T06:21:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26d2ffd640 1 -- 172.21.15.3:0/3304412510 learned_addr learned my addr 172.21.15.3:0/3304412510 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26d27fc640 1 --2- 172.21.15.3:0/3304412510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682bda2f0 0x558683029320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26d37fe640 1 --2- 172.21.15.3:0/3304412510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682be24d0 0x558682fbc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26d2ffd640 1 -- 172.21.15.3:0/3304412510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682bda2f0 msgr2=0x558683029320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/3304412510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682bda2f0 0x558683029320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26d2ffd640 1 -- 172.21.15.3:0/3304412510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682be24d0 msgr2=0x558682fbc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/3304412510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682be24d0 0x558682fbc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26d2ffd640 1 -- 172.21.15.3:0/3304412510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558682fbc790 con 0x558682bc1d80 2024-09-21T06:21:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26d2ffd640 1 --2- 172.21.15.3:0/3304412510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 0x558683028d40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f26c40e5740 tx=0x7f26c40aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4636e2f062b64e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.883+0000 7f26b6ff5640 1 -- 172.21.15.3:0/3304412510 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40b2070 con 0x558682bc1d80 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26b6ff5640 1 -- 172.21.15.3:0/3304412510 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f26c40a9b60 con 0x558682bc1d80 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26e03d0740 1 -- 172.21.15.3:0/3304412510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c40ad400 con 0x558682bc1d80 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26b6ff5640 1 -- 172.21.15.3:0/3304412510 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f26c40e7e60 con 0x558682bc1d80 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26e03d0740 10 client.17355 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26e03d0740 1 -- 172.21.15.3:0/3304412510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558683030510 con 0x558682bc1d80 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26b6ff5640 1 -- 172.21.15.3:0/3304412510 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c40ad400 con 0x558682bc1d80 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26b6ff5640 1 -- 172.21.15.3:0/3304412510 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f26c40ebce0 con 0x558682bc1d80 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26e03d0740 10 client.17355 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26e03d0740 10 client.17355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26e03d0740 20 client.17355 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26e03d0740 1 --2- 172.21.15.3:0/3304412510 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558682c031b0 0x558683032850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26e03d0740 4 client.17355 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:21:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26e03d0740 1 -- 172.21.15.3:0/3304412510 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558682f1b0f0 con 0x558682c031b0 2024-09-21T06:21:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.884+0000 7f26d27fc640 1 --2- 172.21.15.3:0/3304412510 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558682c031b0 0x558683032850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.885+0000 7f26b6ff5640 10 client.17355 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:16.885+0000 7f26d27fc640 1 --2- 172.21.15.3:0/3304412510 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558682c031b0 0x558683032850 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a04bfe7f3959a97 server_cookie=6c23b4c107d2b9f5 in_seq=0 out_seq=0 2024-09-21T06:21:16.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 656353 11.63 MB/sec execute 169 sec latency 67.631 ms 2024-09-21T06:21:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.028+0000 7f26b6ff5640 1 -- 172.21.15.3:0/3304412510 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558682f1b0f0 con 0x558682c031b0 2024-09-21T06:21:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.028+0000 7f26b6ff5640 10 client.17355 handle_command_reply: tid=0 2024-09-21T06:21:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.028+0000 7f26e03d0740 1 client.17355 shutdown 2024-09-21T06:21:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.028+0000 7f26e03d0740 1 -- 172.21.15.3:0/3304412510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 msgr2=0x558683028d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.028+0000 7f26e03d0740 1 --2- 172.21.15.3:0/3304412510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 0x558683028d40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f26c40e5740 tx=0x7f26c40aa000 comp rx=0 tx=0).stop 2024-09-21T06:21:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.028+0000 7f26e03d0740 1 -- 172.21.15.3:0/3304412510 shutdown_connections 2024-09-21T06:21:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.028+0000 7f26e03d0740 1 --2- 172.21.15.3:0/3304412510 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558682c031b0 0x558683032850 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.028+0000 7f26b6ff5640 0 client.17355 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.028+0000 7f26e03d0740 1 --2- 172.21.15.3:0/3304412510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558682bda2f0 0x558683029320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.028+0000 7f26e03d0740 1 --2- 172.21.15.3:0/3304412510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558682be24d0 0x558682fbc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.028+0000 7f26e03d0740 1 --2- 172.21.15.3:0/3304412510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558682bc1d80 0x558683028d40 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.028+0000 7f26e03d0740 1 -- 172.21.15.3:0/3304412510 >> 172.21.15.3:0/3304412510 conn(0x558682bf7300 msgr2=0x558682fb5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.029+0000 7f26e03d0740 1 -- 172.21.15.3:0/3304412510 shutdown_connections 2024-09-21T06:21:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.029+0000 7f26e03d0740 1 -- 172.21.15.3:0/3304412510 wait complete. 2024-09-21T06:21:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.029+0000 7f26e03d0740 20 client.17355 trim_cache size 0 max 16384 2024-09-21T06:21:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:21:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:21:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:21:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:21:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.029+0000 7f26d9389640 1 -- 172.21.15.3:0/104683550 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f26bc0b66b0 msgr2=0x7f26bc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.029+0000 7f26d9389640 1 --2- 172.21.15.3:0/104683550 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f26bc0b66b0 0x7f26bc0b8bc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009fd0 tx=0x7f26cc009500 comp rx=0 tx=0).stop 2024-09-21T06:21:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.029+0000 7f26d9389640 1 -- 172.21.15.3:0/104683550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26d4104930 msgr2=0x7f26d4102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.029+0000 7f26d9389640 1 --2- 172.21.15.3:0/104683550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26d4104930 0x7f26d4102660 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f26c400e9f0 tx=0x7f26c400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.030+0000 7f26d9389640 1 -- 172.21.15.3:0/104683550 shutdown_connections 2024-09-21T06:21:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.030+0000 7f26d9389640 1 --2- 172.21.15.3:0/104683550 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f26bc0b66b0 0x7f26bc0b8bc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.030+0000 7f26d9389640 1 --2- 172.21.15.3:0/104683550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d410dd80 0x7f26d41030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.030+0000 7f26d9389640 1 --2- 172.21.15.3:0/104683550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d4105280 0x7f26d4102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.030+0000 7f26d9389640 1 --2- 172.21.15.3:0/104683550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26d4104930 0x7f26d4102660 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.030+0000 7f26d9389640 1 -- 172.21.15.3:0/104683550 >> 172.21.15.3:0/104683550 conn(0x7f26d407a6e0 msgr2=0x7f26d4100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.030+0000 7f26d9389640 1 -- 172.21.15.3:0/104683550 shutdown_connections 2024-09-21T06:21:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:17.030+0000 7f26d9389640 1 -- 172.21.15.3:0/104683550 wait complete. 2024-09-21T06:21:17.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.353+0000 7f24e6ffd640 1 -- 172.21.15.3:0/1109500269 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc0318a0 con 0x7f24e8105300 2024-09-21T06:21:17.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.354+0000 7f24ee9c3640 1 -- 172.21.15.3:0/1109500269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e8105300 msgr2=0x7f24e810d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:17.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.354+0000 7f24ee9c3640 1 --2- 172.21.15.3:0/1109500269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e8105300 0x7f24e810d850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009900 tx=0x7f24dc02a410 comp rx=0 tx=0).stop 2024-09-21T06:21:17.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.354+0000 7f24ee9c3640 1 -- 172.21.15.3:0/1109500269 shutdown_connections 2024-09-21T06:21:17.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.354+0000 7f24ee9c3640 1 --2- 172.21.15.3:0/1109500269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e8105300 0x7f24e810d850 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.354+0000 7f24ee9c3640 1 --2- 172.21.15.3:0/1109500269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e810de10 0x7f24e8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.354+0000 7f24ee9c3640 1 --2- 172.21.15.3:0/1109500269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e81049b0 0x7f24e8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.354+0000 7f24ee9c3640 1 -- 172.21.15.3:0/1109500269 >> 172.21.15.3:0/1109500269 conn(0x7f24e8079730 msgr2=0x7f24e8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:17.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.354+0000 7f24ee9c3640 1 -- 172.21.15.3:0/1109500269 shutdown_connections 2024-09-21T06:21:17.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.354+0000 7f24ee9c3640 1 -- 172.21.15.3:0/1109500269 wait complete. 2024-09-21T06:21:17.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24ee9c3640 1 Processor -- start 2024-09-21T06:21:17.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24ee9c3640 1 -- start start 2024-09-21T06:21:17.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24ee9c3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e81049b0 0x7f24e819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24ee9c3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e8105300 0x7f24e819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24ee9c3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e810de10 0x7f24e819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24ee9c3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8113ad0 con 0x7f24e8105300 2024-09-21T06:21:17.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24ee9c3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8113950 con 0x7f24e81049b0 2024-09-21T06:21:17.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24ee9c3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8113c50 con 0x7f24e810de10 2024-09-21T06:21:17.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e8105300 0x7f24e819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e81049b0 0x7f24e819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e81049b0 0x7f24e819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:58506/0 (socket says 172.21.15.3:58506) 2024-09-21T06:21:17.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e8105300 0x7f24e819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47310/0 (socket says 172.21.15.3:47310) 2024-09-21T06:21:17.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24e7fff640 1 -- 172.21.15.3:0/3489418066 learned_addr learned my addr 172.21.15.3:0/3489418066 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:17.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.355+0000 7f24ecf60640 1 --2- 172.21.15.3:0/3489418066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e810de10 0x7f24e819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.356+0000 7f24e7fff640 1 -- 172.21.15.3:0/3489418066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e810de10 msgr2=0x7f24e819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:17.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.356+0000 7f24e7fff640 1 --2- 172.21.15.3:0/3489418066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e810de10 0x7f24e819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.356+0000 7f24e7fff640 1 -- 172.21.15.3:0/3489418066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e8105300 msgr2=0x7f24e819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:17.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.356+0000 7f24e7fff640 1 --2- 172.21.15.3:0/3489418066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e8105300 0x7f24e819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.356+0000 7f24e7fff640 1 -- 172.21.15.3:0/3489418066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e81a04d0 con 0x7f24e81049b0 2024-09-21T06:21:17.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.356+0000 7f24e7fff640 1 --2- 172.21.15.3:0/3489418066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e81049b0 0x7f24e819a560 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e9c0 tx=0x7f24d800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:17.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.356+0000 7f24e57fa640 1 -- 172.21.15.3:0/3489418066 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d800cd80 con 0x7f24e81049b0 2024-09-21T06:21:17.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.356+0000 7f24ee9c3640 1 -- 172.21.15.3:0/3489418066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e81a07c0 con 0x7f24e81049b0 2024-09-21T06:21:17.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.356+0000 7f24e57fa640 1 -- 172.21.15.3:0/3489418066 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d80035f0 con 0x7f24e81049b0 2024-09-21T06:21:17.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.356+0000 7f24e57fa640 1 -- 172.21.15.3:0/3489418066 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8010690 con 0x7f24e81049b0 2024-09-21T06:21:17.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.356+0000 7f24ee9c3640 1 -- 172.21.15.3:0/3489418066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e81a6f20 con 0x7f24e81049b0 2024-09-21T06:21:17.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.359+0000 7f24f4f37740 1 Processor -- start 2024-09-21T06:21:17.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.359+0000 7f24f4f37740 1 -- start start 2024-09-21T06:21:17.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.359+0000 7f24e57fa640 1 -- 172.21.15.3:0/3489418066 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f24d8004000 con 0x7f24e81049b0 2024-09-21T06:21:17.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.359+0000 7f24f4f37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0951aa350 0x55b095555840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.359+0000 7f24f4f37740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b095555d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.359+0000 7f24f4f37740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b095189670 0x55b0955562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.359+0000 7f24f4f37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b0954c7e60 con 0x55b0951aa350 2024-09-21T06:21:17.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.359+0000 7f24f4f37740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b0954b7290 con 0x55b095181490 2024-09-21T06:21:17.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.359+0000 7f24f4f37740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b0954bb0a0 con 0x55b095189670 2024-09-21T06:21:17.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.359+0000 7f24ecf60640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0951aa350 0x55b095555840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.359+0000 7f24ecf60640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0951aa350 0x55b095555840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47316/0 (socket says 172.21.15.3:47316) 2024-09-21T06:21:17.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.359+0000 7f24ecf60640 1 -- 172.21.15.3:0/1106786720 learned_addr learned my addr 172.21.15.3:0/1106786720 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:17.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.360+0000 7f24e77fe640 1 --2- 172.21.15.3:0/1106786720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b095189670 0x55b0955562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.360+0000 7f24e7fff640 1 --2- 172.21.15.3:0/1106786720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b095555d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.360+0000 7f24e77fe640 1 -- 172.21.15.3:0/1106786720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 msgr2=0x55b095555d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:17.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.360+0000 7f24e77fe640 1 --2- 172.21.15.3:0/1106786720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b095555d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.360+0000 7f24e77fe640 1 -- 172.21.15.3:0/1106786720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0951aa350 msgr2=0x55b095555840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:17.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.360+0000 7f24e77fe640 1 --2- 172.21.15.3:0/1106786720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0951aa350 0x55b095555840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.360+0000 7f24e77fe640 1 -- 172.21.15.3:0/1106786720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b095215930 con 0x55b095189670 2024-09-21T06:21:17.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.360+0000 7f24e7fff640 1 --2- 172.21.15.3:0/1106786720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b095555d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:21:17.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.360+0000 7f24ecf60640 1 --2- 172.21.15.3:0/1106786720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0951aa350 0x55b095555840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:17.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.360+0000 7f24e77fe640 1 --2- 172.21.15.3:0/1106786720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b095189670 0x55b0955562c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f24dc02a930 tx=0x7f24dc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33d4003af766fa84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:17.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.361+0000 7f24e57fa640 1 --2- 172.21.15.3:0/3489418066 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f24d00b6680 0x7f24d00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.361+0000 7f24ce7fc640 1 -- 172.21.15.3:0/1106786720 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc040070 con 0x55b095189670 2024-09-21T06:21:17.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.361+0000 7f24ce7fc640 1 -- 172.21.15.3:0/1106786720 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24dc004030 con 0x55b095189670 2024-09-21T06:21:17.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.361+0000 7f24ce7fc640 1 -- 172.21.15.3:0/1106786720 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc035040 con 0x55b095189670 2024-09-21T06:21:17.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.361+0000 7f24ecf60640 1 --2- 172.21.15.3:0/3489418066 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f24d00b6680 0x7f24d00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.361+0000 7f24e57fa640 1 -- 172.21.15.3:0/3489418066 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f24d80e7c10 con 0x7f24e81049b0 2024-09-21T06:21:17.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.362+0000 7f24f4f37740 1 -- 172.21.15.3:0/1106786720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b095189670 msgr2=0x55b0955562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:17.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.362+0000 7f24ecf60640 1 --2- 172.21.15.3:0/3489418066 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f24d00b6680 0x7f24d00b8b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009fd0 tx=0x7f24e000d040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:17.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.362+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1106786720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b095189670 0x55b0955562c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f24dc02a930 tx=0x7f24dc000bc0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.362+0000 7f24f4f37740 1 -- 172.21.15.3:0/1106786720 shutdown_connections 2024-09-21T06:21:17.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.362+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1106786720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b095189670 0x55b0955562c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.362+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1106786720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b095555d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.362+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1106786720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0951aa350 0x55b095555840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.362+0000 7f24f4f37740 1 -- 172.21.15.3:0/1106786720 >> 172.21.15.3:0/1106786720 conn(0x55b09519e740 msgr2=0x55b095553730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.362+0000 7f24f4f37740 1 -- 172.21.15.3:0/1106786720 shutdown_connections 2024-09-21T06:21:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.362+0000 7f24f4f37740 1 -- 172.21.15.3:0/1106786720 wait complete. 2024-09-21T06:21:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.362+0000 7f24f4f37740 1 -- start start 2024-09-21T06:21:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.363+0000 7f24f4f37740 1 Processor -- start 2024-09-21T06:21:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.363+0000 7f24f4f37740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b095181490 0x55b095561410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.363+0000 7f24f4f37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b095561950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.363+0000 7f24f4f37740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0951aa350 0x55b09556d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.363+0000 7f24f4f37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b095318890 con 0x55b095189670 2024-09-21T06:21:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.363+0000 7f24f4f37740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b095451d40 con 0x55b0951aa350 2024-09-21T06:21:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.363+0000 7f24f4f37740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b095563fc0 con 0x55b095181490 2024-09-21T06:21:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.363+0000 7f24f4f37740 10 client.0 _ll_register_callbacks cb 0x55b0954b5c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.363+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b095561950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.363+0000 7f24ecf60640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0951aa350 0x55b09556d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.363+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b095561950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47330/0 (socket says 172.21.15.3:47330) 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.363+0000 7f24e77fe640 1 -- 172.21.15.3:0/1913845352 learned_addr learned my addr 172.21.15.3:0/1913845352 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.363+0000 7f24e7fff640 1 --2- 172.21.15.3:0/1913845352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b095181490 0x55b095561410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.364+0000 7f24e77fe640 1 -- 172.21.15.3:0/1913845352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b095181490 msgr2=0x55b095561410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.364+0000 7f24e77fe640 1 --2- 172.21.15.3:0/1913845352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b095181490 0x55b095561410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.364+0000 7f24e77fe640 1 -- 172.21.15.3:0/1913845352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0951aa350 msgr2=0x55b09556d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.364+0000 7f24e77fe640 1 --2- 172.21.15.3:0/1913845352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0951aa350 0x55b09556d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.364+0000 7f24e77fe640 1 -- 172.21.15.3:0/1913845352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b09556d8a0 con 0x55b095189670 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.364+0000 7f24e77fe640 1 --2- 172.21.15.3:0/1913845352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b095561950 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f24dc005f50 tx=0x7f24dc0041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a683a16f7e155a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.364+0000 7f24c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.364+0000 7f24c77fe640 1 -- 172.21.15.3:0/1913845352 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc035070 con 0x55b095189670 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.364+0000 7f24f4f37740 1 -- 172.21.15.3:0/1913845352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24dc004030 con 0x55b095189670 2024-09-21T06:21:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.364+0000 7f24c77fe640 1 -- 172.21.15.3:0/1913845352 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f24dc0054f0 con 0x55b095189670 2024-09-21T06:21:17.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.364+0000 7f24c77fe640 1 -- 172.21.15.3:0/1913845352 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f24dc040040 con 0x55b095189670 2024-09-21T06:21:17.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.364+0000 7f24f4f37740 10 client.17364 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:17.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.364+0000 7f24f4f37740 1 -- 172.21.15.3:0/1913845352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b095570120 con 0x55b095189670 2024-09-21T06:21:17.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.365+0000 7f24c77fe640 1 -- 172.21.15.3:0/1913845352 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24dc004030 con 0x55b095189670 2024-09-21T06:21:17.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.365+0000 7f24c77fe640 1 -- 172.21.15.3:0/1913845352 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f24dc004ce0 con 0x55b095189670 2024-09-21T06:21:17.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.365+0000 7f24f4f37740 10 client.17364 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:17.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.365+0000 7f24f4f37740 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:17.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.365+0000 7f24f4f37740 20 client.17364 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:17.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.365+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1913845352 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b095168cd0 0x55b095573480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.365+0000 7f24f4f37740 4 client.17364 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:17.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.365+0000 7f24f4f37740 1 -- 172.21.15.3:0/1913845352 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0954c9db0 con 0x55b095168cd0 2024-09-21T06:21:17.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.365+0000 7f24e7fff640 1 --2- 172.21.15.3:0/1913845352 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b095168cd0 0x55b095573480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.365+0000 7f24c77fe640 10 client.17364 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:17.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.365+0000 7f24e7fff640 1 --2- 172.21.15.3:0/1913845352 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b095168cd0 0x55b095573480 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86aac6d85120581d server_cookie=12e085e6596c808 in_seq=0 out_seq=0 2024-09-21T06:21:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.368+0000 7f24c77fe640 1 -- 172.21.15.3:0/1913845352 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b0954c9db0 con 0x55b095168cd0 2024-09-21T06:21:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.368+0000 7f24c77fe640 10 client.17364 handle_command_reply: tid=0 2024-09-21T06:21:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.368+0000 7f24f4f37740 1 client.17364 shutdown 2024-09-21T06:21:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.368+0000 7f24f4f37740 1 -- 172.21.15.3:0/1913845352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 msgr2=0x55b095561950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.368+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1913845352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b095561950 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f24dc005f50 tx=0x7f24dc0041e0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.368+0000 7f24f4f37740 1 -- 172.21.15.3:0/1913845352 shutdown_connections 2024-09-21T06:21:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.368+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1913845352 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b095168cd0 0x55b095573480 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.368+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1913845352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0951aa350 0x55b09556d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.368+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1913845352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b095561950 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.368+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1913845352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b095181490 0x55b095561410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.368+0000 7f24f4f37740 1 -- 172.21.15.3:0/1913845352 >> 172.21.15.3:0/1913845352 conn(0x55b09519e740 msgr2=0x55b095558a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.368+0000 7f24c77fe640 0 client.17364 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.369+0000 7f24f4f37740 1 -- 172.21.15.3:0/1913845352 shutdown_connections 2024-09-21T06:21:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.369+0000 7f24f4f37740 1 -- 172.21.15.3:0/1913845352 wait complete. 2024-09-21T06:21:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.369+0000 7f24f4f37740 20 client.17364 trim_cache size 0 max 16384 2024-09-21T06:21:17.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.388+0000 7f24f4f37740 1 Processor -- start 2024-09-21T06:21:17.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24f4f37740 1 -- start start 2024-09-21T06:21:17.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24f4f37740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b09556d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24f4f37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b09555fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24f4f37740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0951aa350 0x55b09555ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24f4f37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ad4f0 con 0x55b095189670 2024-09-21T06:21:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24f4f37740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81af060 con 0x55b095181490 2024-09-21T06:21:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24f4f37740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81adbc0 con 0x55b0951aa350 2024-09-21T06:21:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b09555fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b09556d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b09555fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47340/0 (socket says 172.21.15.3:47340) 2024-09-21T06:21:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24e77fe640 1 -- 172.21.15.3:0/1111898831 learned_addr learned my addr 172.21.15.3:0/1111898831 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24ecf60640 1 --2- 172.21.15.3:0/1111898831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0951aa350 0x55b09555ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24e7fff640 1 -- 172.21.15.3:0/1111898831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0951aa350 msgr2=0x55b09555ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24e7fff640 1 --2- 172.21.15.3:0/1111898831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0951aa350 0x55b09555ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24e7fff640 1 -- 172.21.15.3:0/1111898831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 msgr2=0x55b09555fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24e7fff640 1 --2- 172.21.15.3:0/1111898831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b09555fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.389+0000 7f24e7fff640 1 -- 172.21.15.3:0/1111898831 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b095561dd0 con 0x55b095181490 2024-09-21T06:21:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.390+0000 7f24e7fff640 1 --2- 172.21.15.3:0/1111898831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b09556d220 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f24d80e53a0 tx=0x7f24d80aa210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9343ae98341f348 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.390+0000 7f24cdffb640 1 -- 172.21.15.3:0/1111898831 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d80a9210 con 0x55b095181490 2024-09-21T06:21:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.390+0000 7f24cdffb640 1 -- 172.21.15.3:0/1111898831 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d80ad5b0 con 0x55b095181490 2024-09-21T06:21:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.390+0000 7f24cdffb640 1 -- 172.21.15.3:0/1111898831 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d80e5620 con 0x55b095181490 2024-09-21T06:21:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.391+0000 7f24f4f37740 1 -- 172.21.15.3:0/1111898831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 msgr2=0x55b09556d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.391+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1111898831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b09556d220 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f24d80e53a0 tx=0x7f24d80aa210 comp rx=0 tx=0).stop 2024-09-21T06:21:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.391+0000 7f24f4f37740 1 -- 172.21.15.3:0/1111898831 shutdown_connections 2024-09-21T06:21:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.391+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1111898831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0951aa350 0x55b09555ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.391+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1111898831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b09555fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.391+0000 7f24f4f37740 1 --2- 172.21.15.3:0/1111898831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b09556d220 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.391+0000 7f24f4f37740 1 -- 172.21.15.3:0/1111898831 >> 172.21.15.3:0/1111898831 conn(0x55b09519e740 msgr2=0x55b09555c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.391+0000 7f24f4f37740 1 -- 172.21.15.3:0/1111898831 shutdown_connections 2024-09-21T06:21:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.391+0000 7f24f4f37740 1 -- 172.21.15.3:0/1111898831 wait complete. 2024-09-21T06:21:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.391+0000 7f24f4f37740 1 -- start start 2024-09-21T06:21:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.392+0000 7f24f4f37740 1 Processor -- start 2024-09-21T06:21:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.392+0000 7f24f4f37740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b095574710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.392+0000 7f24f4f37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b095574c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.392+0000 7f24f4f37740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0951aa350 0x55b0955dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.392+0000 7f24f4f37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ae990 con 0x55b095189670 2024-09-21T06:21:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.392+0000 7f24f4f37740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ae2c0 con 0x55b095181490 2024-09-21T06:21:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.392+0000 7f24f4f37740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ab4d0 con 0x55b0951aa350 2024-09-21T06:21:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.392+0000 7f24f4f37740 10 client.0 _ll_register_callbacks cb 0x7f24e0009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.392+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b095574c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.392+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b095574710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.392+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b095574c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47348/0 (socket says 172.21.15.3:47348) 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.392+0000 7f24e77fe640 1 -- 172.21.15.3:0/3485723756 learned_addr learned my addr 172.21.15.3:0/3485723756 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.392+0000 7f24ecf60640 1 --2- 172.21.15.3:0/3485723756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0951aa350 0x55b0955dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24e7fff640 1 -- 172.21.15.3:0/3485723756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0951aa350 msgr2=0x55b0955dcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24e7fff640 1 --2- 172.21.15.3:0/3485723756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0951aa350 0x55b0955dcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24e7fff640 1 -- 172.21.15.3:0/3485723756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 msgr2=0x55b095574c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24e7fff640 1 --2- 172.21.15.3:0/3485723756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b095574c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24e7fff640 1 -- 172.21.15.3:0/3485723756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b095575440 con 0x55b095181490 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24ecf60640 1 --2- 172.21.15.3:0/3485723756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0951aa350 0x55b0955dcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24e7fff640 1 --2- 172.21.15.3:0/3485723756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b095574710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f24d80e5450 tx=0x7f24d80a9280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b11b221e148137 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:21:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24c77fe640 1 -- 172.21.15.3:0/3485723756 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d80ae1a0 con 0x55b095181490 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24c77fe640 1 -- 172.21.15.3:0/3485723756 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f24d80e5620 con 0x55b095181490 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24f4f37740 1 -- 172.21.15.3:0/3485723756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d80ad5b0 con 0x55b095181490 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24c77fe640 1 -- 172.21.15.3:0/3485723756 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f24d80a9d10 con 0x55b095181490 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24f4f37740 10 client.26939 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.393+0000 7f24f4f37740 1 -- 172.21.15.3:0/3485723756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b0955de130 con 0x55b095181490 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.394+0000 7f24c77fe640 1 -- 172.21.15.3:0/3485723756 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d80ad5b0 con 0x55b095181490 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.394+0000 7f24c77fe640 1 -- 172.21.15.3:0/3485723756 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f24d80a93f0 con 0x55b095181490 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.394+0000 7f24f4f37740 10 client.26939 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.394+0000 7f24f4f37740 10 client.26939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.394+0000 7f24f4f37740 20 client.26939 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.394+0000 7f24f4f37740 1 --2- 172.21.15.3:0/3485723756 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b09513e080 0x55b0955e0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.394+0000 7f24f4f37740 4 client.26939 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.394+0000 7f24f4f37740 1 -- 172.21.15.3:0/3485723756 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b0954c5860 con 0x55b09513e080 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.394+0000 7f24e77fe640 1 --2- 172.21.15.3:0/3485723756 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b09513e080 0x55b0955e0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:17.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.394+0000 7f24c77fe640 10 client.26939 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.395+0000 7f24e77fe640 1 --2- 172.21.15.3:0/3485723756 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b09513e080 0x55b0955e0850 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166100611b3c17a5 server_cookie=da164dc2744071ea in_seq=0 out_seq=0 2024-09-21T06:21:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.396+0000 7f24c77fe640 1 -- 172.21.15.3:0/3485723756 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b0954c5860 con 0x55b09513e080 2024-09-21T06:21:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.396+0000 7f24c77fe640 10 client.26939 handle_command_reply: tid=0 2024-09-21T06:21:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.396+0000 7f24f4f37740 1 client.26939 shutdown 2024-09-21T06:21:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.397+0000 7f24f4f37740 1 -- 172.21.15.3:0/3485723756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 msgr2=0x55b095574710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.397+0000 7f24f4f37740 1 --2- 172.21.15.3:0/3485723756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b095574710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f24d80e5450 tx=0x7f24d80a9280 comp rx=0 tx=0).stop 2024-09-21T06:21:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.397+0000 7f24f4f37740 1 -- 172.21.15.3:0/3485723756 shutdown_connections 2024-09-21T06:21:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.397+0000 7f24f4f37740 1 --2- 172.21.15.3:0/3485723756 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b09513e080 0x55b0955e0850 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.397+0000 7f24c77fe640 0 client.26939 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.397+0000 7f24f4f37740 1 --2- 172.21.15.3:0/3485723756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0951aa350 0x55b0955dcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.397+0000 7f24f4f37740 1 --2- 172.21.15.3:0/3485723756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b095189670 0x55b095574c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.397+0000 7f24f4f37740 1 --2- 172.21.15.3:0/3485723756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b095181490 0x55b095574710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.397+0000 7f24f4f37740 1 -- 172.21.15.3:0/3485723756 >> 172.21.15.3:0/3485723756 conn(0x55b09519e740 msgr2=0x55b09555c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:17.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.397+0000 7f24f4f37740 1 -- 172.21.15.3:0/3485723756 shutdown_connections 2024-09-21T06:21:17.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.397+0000 7f24f4f37740 1 -- 172.21.15.3:0/3485723756 wait complete. 2024-09-21T06:21:17.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.397+0000 7f24f4f37740 20 client.26939 trim_cache size 0 max 16384 2024-09-21T06:21:17.404 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:21:17.404 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:21:17.404 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58", 2024-09-21T06:21:17.404 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:21:17.404 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:21:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.398+0000 7f24ee9c3640 1 -- 172.21.15.3:0/3489418066 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f24d00b6680 msgr2=0x7f24d00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.398+0000 7f24ee9c3640 1 --2- 172.21.15.3:0/3489418066 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f24d00b6680 0x7f24d00b8b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009fd0 tx=0x7f24e000d040 comp rx=0 tx=0).stop 2024-09-21T06:21:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.398+0000 7f24ee9c3640 1 -- 172.21.15.3:0/3489418066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e81049b0 msgr2=0x7f24e819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.398+0000 7f24ee9c3640 1 --2- 172.21.15.3:0/3489418066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e81049b0 0x7f24e819a560 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e9c0 tx=0x7f24d800ee80 comp rx=0 tx=0).stop 2024-09-21T06:21:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.398+0000 7f24ee9c3640 1 -- 172.21.15.3:0/3489418066 shutdown_connections 2024-09-21T06:21:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.398+0000 7f24ee9c3640 1 --2- 172.21.15.3:0/3489418066 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f24d00b6680 0x7f24d00b8b90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.398+0000 7f24ee9c3640 1 --2- 172.21.15.3:0/3489418066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e810de10 0x7f24e819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.398+0000 7f24ee9c3640 1 --2- 172.21.15.3:0/3489418066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e8105300 0x7f24e819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.398+0000 7f24ee9c3640 1 --2- 172.21.15.3:0/3489418066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e81049b0 0x7f24e819a560 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.398+0000 7f24ee9c3640 1 -- 172.21.15.3:0/3489418066 >> 172.21.15.3:0/3489418066 conn(0x7f24e8079730 msgr2=0x7f24e81008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.398+0000 7f24ee9c3640 1 -- 172.21.15.3:0/3489418066 shutdown_connections 2024-09-21T06:21:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:17.398+0000 7f24ee9c3640 1 -- 172.21.15.3:0/3489418066 wait complete. 2024-09-21T06:21:17.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:17 smithi003 ceph-mon[31227]: pgmap v513: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 334 op/s 2024-09-21T06:21:17.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:17 smithi137 ceph-mon[34464]: pgmap v513: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 334 op/s 2024-09-21T06:21:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:17 smithi204 ceph-mon[34659]: pgmap v513: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 334 op/s 2024-09-21T06:21:17.961 INFO:tasks.workunit.client.0.smithi003.stdout: 1 657204 11.60 MB/sec execute 170 sec latency 540.067 ms 2024-09-21T06:21:18.223 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:21:18.490 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:18.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:18 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:21:18.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:18 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:18.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:18 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:21:18.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:18 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:21:18.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:18 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:21:18.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:18 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:18.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:18 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:18.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:18 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:21:18.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:18 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:18.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:18 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:21:18.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:18 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:21:18.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:18 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:21:18.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:18 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:18.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:18 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:18 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:21:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:18 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:18 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:21:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:18 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:21:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:18 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:21:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:18 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:18.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:18 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:18.961 INFO:tasks.workunit.client.0.smithi003.stdout: 1 657649 11.53 MB/sec execute 171 sec latency 1047.030 ms 2024-09-21T06:21:19.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.073+0000 7f897cf2c640 1 -- 172.21.15.3:0/2842658158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8978103b80 msgr2=0x7f8978103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:19.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.073+0000 7f897cf2c640 1 --2- 172.21.15.3:0/2842658158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8978103b80 0x7f8978103f90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8968009900 tx=0x7f896802a410 comp rx=0 tx=0).stop 2024-09-21T06:21:19.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.073+0000 7f897cf2c640 1 -- 172.21.15.3:0/2842658158 shutdown_connections 2024-09-21T06:21:19.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.073+0000 7f897cf2c640 1 --2- 172.21.15.3:0/2842658158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89781094c0 0x7f89781098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.073+0000 7f897cf2c640 1 --2- 172.21.15.3:0/2842658158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8978109e10 0x7f8978110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.073+0000 7f897cf2c640 1 --2- 172.21.15.3:0/2842658158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8978103b80 0x7f8978103f90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.073+0000 7f897cf2c640 1 -- 172.21.15.3:0/2842658158 >> 172.21.15.3:0/2842658158 conn(0x7f89780ff610 msgr2=0x7f8978101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:19.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.073+0000 7f897cf2c640 1 -- 172.21.15.3:0/2842658158 shutdown_connections 2024-09-21T06:21:19.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.073+0000 7f897cf2c640 1 -- 172.21.15.3:0/2842658158 wait complete. 2024-09-21T06:21:19.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.074+0000 7f897cf2c640 1 Processor -- start 2024-09-21T06:21:19.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.075+0000 7f897cf2c640 1 -- start start 2024-09-21T06:21:19.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.075+0000 7f897cf2c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8978103b80 0x7f897819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.075+0000 7f897cf2c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89781094c0 0x7f897819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.075+0000 7f897cf2c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8978109e10 0x7f897819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.075+0000 7f897cf2c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8978113ad0 con 0x7f89781094c0 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.075+0000 7f897cf2c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8978113950 con 0x7f8978109e10 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.075+0000 7f897cf2c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8978113c50 con 0x7f8978103b80 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.075+0000 7f896ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89781094c0 0x7f897819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.075+0000 7f897659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8978103b80 0x7f897819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.075+0000 7f897659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8978103b80 0x7f897819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:39850/0 (socket says 172.21.15.3:39850) 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.075+0000 7f896ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89781094c0 0x7f897819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47360/0 (socket says 172.21.15.3:47360) 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.075+0000 7f897659c640 1 -- 172.21.15.3:0/3338472262 learned_addr learned my addr 172.21.15.3:0/3338472262 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.075+0000 7f8976d9d640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8978109e10 0x7f897819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.076+0000 7f8976d9d640 1 -- 172.21.15.3:0/3338472262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8978103b80 msgr2=0x7f897819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.076+0000 7f8976d9d640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8978103b80 0x7f897819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.076+0000 7f8976d9d640 1 -- 172.21.15.3:0/3338472262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89781094c0 msgr2=0x7f897819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.076+0000 7f8976d9d640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89781094c0 0x7f897819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.076+0000 7f8976d9d640 1 -- 172.21.15.3:0/3338472262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89781a0450 con 0x7f8978109e10 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.076+0000 7f897659c640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8978103b80 0x7f897819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:19.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.076+0000 7f8976d9d640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8978109e10 0x7f897819fd20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8970009fd0 tx=0x7f897000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:19.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.076+0000 7f896f7fe640 1 -- 172.21.15.3:0/3338472262 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897000ce20 con 0x7f8978109e10 2024-09-21T06:21:19.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.076+0000 7f896f7fe640 1 -- 172.21.15.3:0/3338472262 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8970005020 con 0x7f8978109e10 2024-09-21T06:21:19.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.076+0000 7f897cf2c640 1 -- 172.21.15.3:0/3338472262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89781a0740 con 0x7f8978109e10 2024-09-21T06:21:19.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.076+0000 7f896f7fe640 1 -- 172.21.15.3:0/3338472262 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970010690 con 0x7f8978109e10 2024-09-21T06:21:19.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.076+0000 7f897cf2c640 1 -- 172.21.15.3:0/3338472262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89781a6ea0 con 0x7f8978109e10 2024-09-21T06:21:19.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.079+0000 7f89834a0740 1 Processor -- start 2024-09-21T06:21:19.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.080+0000 7f896f7fe640 1 -- 172.21.15.3:0/3338472262 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f8970004b20 con 0x7f8978109e10 2024-09-21T06:21:19.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.080+0000 7f89834a0740 1 -- start start 2024-09-21T06:21:19.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.080+0000 7f89834a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce026aa070 0x55ce02a51650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.080+0000 7f89834a0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a51b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.080+0000 7f89834a0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce02689390 0x55ce02a520d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.080+0000 7f89834a0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ce029c7e50 con 0x55ce026811b0 2024-09-21T06:21:19.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.080+0000 7f89834a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ce029b7280 con 0x55ce026aa070 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.080+0000 7f89834a0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ce029bb090 con 0x55ce02689390 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.080+0000 7f897659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a51b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.080+0000 7f896ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce026aa070 0x55ce02a51650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.080+0000 7f897659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a51b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47362/0 (socket says 172.21.15.3:47362) 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.080+0000 7f897659c640 1 -- 172.21.15.3:0/3329532454 learned_addr learned my addr 172.21.15.3:0/3329532454 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.080+0000 7f896ffff640 1 --2- 172.21.15.3:0/3329532454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce02689390 0x55ce02a520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.081+0000 7f896ffff640 1 -- 172.21.15.3:0/3329532454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce02689390 msgr2=0x55ce02a520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.081+0000 7f896ffff640 1 --2- 172.21.15.3:0/3329532454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce02689390 0x55ce02a520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.081+0000 7f896ffff640 1 -- 172.21.15.3:0/3329532454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 msgr2=0x55ce02a51b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.081+0000 7f896ffff640 1 --2- 172.21.15.3:0/3329532454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a51b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.081+0000 7f896ffff640 1 -- 172.21.15.3:0/3329532454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce02715650 con 0x55ce026aa070 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.081+0000 7f896ffff640 1 --2- 172.21.15.3:0/3329532454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce026aa070 0x55ce02a51650 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8960009730 tx=0x7f89600096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98554a7b2dc72200 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.081+0000 7f896f7fe640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89640b6680 0x7f89640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.081+0000 7f896cff9640 1 -- 172.21.15.3:0/3329532454 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960004050 con 0x55ce026aa070 2024-09-21T06:21:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.081+0000 7f896cff9640 1 -- 172.21.15.3:0/3329532454 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89600054d0 con 0x55ce026aa070 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.081+0000 7f896cff9640 1 -- 172.21.15.3:0/3329532454 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960004bc0 con 0x55ce026aa070 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.081+0000 7f897659c640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89640b6680 0x7f89640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.081+0000 7f896f7fe640 1 -- 172.21.15.3:0/3338472262 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f89700e7db0 con 0x7f8978109e10 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.082+0000 7f89834a0740 1 -- 172.21.15.3:0/3329532454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce026aa070 msgr2=0x55ce02a51650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.082+0000 7f897659c640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89640b6680 0x7f89640b8b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f89680096c0 tx=0x7f89680058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.082+0000 7f89834a0740 1 --2- 172.21.15.3:0/3329532454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce026aa070 0x55ce02a51650 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8960009730 tx=0x7f89600096c0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.082+0000 7f89834a0740 1 -- 172.21.15.3:0/3329532454 shutdown_connections 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.082+0000 7f89834a0740 1 --2- 172.21.15.3:0/3329532454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce02689390 0x55ce02a520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.082+0000 7f89834a0740 1 --2- 172.21.15.3:0/3329532454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a51b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.082+0000 7f89834a0740 1 --2- 172.21.15.3:0/3329532454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce026aa070 0x55ce02a51650 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.082+0000 7f89834a0740 1 -- 172.21.15.3:0/3329532454 >> 172.21.15.3:0/3329532454 conn(0x55ce0269e460 msgr2=0x55ce02a4f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.082+0000 7f89834a0740 1 -- 172.21.15.3:0/3329532454 shutdown_connections 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.082+0000 7f89834a0740 1 -- 172.21.15.3:0/3329532454 wait complete. 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.083+0000 7f89834a0740 1 -- start start 2024-09-21T06:21:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.083+0000 7f89834a0740 1 Processor -- start 2024-09-21T06:21:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f89834a0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a5cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f89834a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce02689390 0x55ce02a5d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f89834a0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce026aa070 0x55ce02a69150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f89834a0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ce02818700 con 0x55ce026811b0 2024-09-21T06:21:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f89834a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ce02952110 con 0x55ce02689390 2024-09-21T06:21:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f89834a0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ce02a5fe30 con 0x55ce026aa070 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f89834a0740 10 client.0 _ll_register_callbacks cb 0x55ce0296b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f897659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a5cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f896ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce02689390 0x55ce02a5d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f897659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a5cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47370/0 (socket says 172.21.15.3:47370) 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f897659c640 1 -- 172.21.15.3:0/4197504621 learned_addr learned my addr 172.21.15.3:0/4197504621 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f8976d9d640 1 --2- 172.21.15.3:0/4197504621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce026aa070 0x55ce02a69150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f8976d9d640 1 -- 172.21.15.3:0/4197504621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce02689390 msgr2=0x55ce02a5d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f8976d9d640 1 --2- 172.21.15.3:0/4197504621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce02689390 0x55ce02a5d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f8976d9d640 1 -- 172.21.15.3:0/4197504621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 msgr2=0x55ce02a5cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f8976d9d640 1 --2- 172.21.15.3:0/4197504621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a5cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.084+0000 7f8976d9d640 1 -- 172.21.15.3:0/4197504621 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce0297fc40 con 0x55ce026aa070 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.085+0000 7f8976d9d640 1 --2- 172.21.15.3:0/4197504621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce026aa070 0x55ce02a69150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f89700e5540 tx=0x7f89700ad3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0039af3b958269f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.085+0000 7f895dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:21:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.085+0000 7f895dffb640 1 -- 172.21.15.3:0/4197504621 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89700d12e0 con 0x55ce026aa070 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.085+0000 7f89834a0740 1 -- 172.21.15.3:0/4197504621 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89600054d0 con 0x55ce026aa070 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.085+0000 7f895dffb640 1 -- 172.21.15.3:0/4197504621 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f89700b2220 con 0x55ce026aa070 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.085+0000 7f895dffb640 1 -- 172.21.15.3:0/4197504621 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f89700ad750 con 0x55ce026aa070 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.085+0000 7f89834a0740 10 client.26896 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.085+0000 7f89834a0740 1 -- 172.21.15.3:0/4197504621 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ce02a70030 con 0x55ce026aa070 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.085+0000 7f895dffb640 1 -- 172.21.15.3:0/4197504621 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89600054d0 con 0x55ce026aa070 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.086+0000 7f895dffb640 1 -- 172.21.15.3:0/4197504621 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f89700a9590 con 0x55ce026aa070 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.086+0000 7f89834a0740 10 client.26896 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.086+0000 7f89834a0740 10 client.26896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.086+0000 7f89834a0740 20 client.26896 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.086+0000 7f89834a0740 1 --2- 172.21.15.3:0/4197504621 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55ce026689f0 0x55ce02a734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.086+0000 7f89834a0740 4 client.26896 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.086+0000 7f89834a0740 1 -- 172.21.15.3:0/4197504621 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce029c9da0 con 0x55ce026689f0 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.086+0000 7f897659c640 1 --2- 172.21.15.3:0/4197504621 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55ce026689f0 0x55ce02a734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.086+0000 7f895dffb640 10 client.26896 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.086+0000 7f897659c640 1 --2- 172.21.15.3:0/4197504621 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55ce026689f0 0x55ce02a734d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c0ed255122866e server_cookie=c64d2a4ecef7c216 in_seq=0 out_seq=0 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.087+0000 7f895dffb640 1 -- 172.21.15.3:0/4197504621 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ce029c9da0 con 0x55ce026689f0 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.087+0000 7f895dffb640 10 client.26896 handle_command_reply: tid=0 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.087+0000 7f89834a0740 1 client.26896 shutdown 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.087+0000 7f89834a0740 1 -- 172.21.15.3:0/4197504621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce026aa070 msgr2=0x55ce02a69150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.087+0000 7f89834a0740 1 --2- 172.21.15.3:0/4197504621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce026aa070 0x55ce02a69150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f89700e5540 tx=0x7f89700ad3b0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.087+0000 7f89834a0740 1 -- 172.21.15.3:0/4197504621 shutdown_connections 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.087+0000 7f89834a0740 1 --2- 172.21.15.3:0/4197504621 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55ce026689f0 0x55ce02a734d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.087+0000 7f89834a0740 1 --2- 172.21.15.3:0/4197504621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce026aa070 0x55ce02a69150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.087+0000 7f89834a0740 1 --2- 172.21.15.3:0/4197504621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce02689390 0x55ce02a5d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.087+0000 7f895dffb640 0 client.26896 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.087+0000 7f89834a0740 1 --2- 172.21.15.3:0/4197504621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a5cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.087+0000 7f89834a0740 1 -- 172.21.15.3:0/4197504621 >> 172.21.15.3:0/4197504621 conn(0x55ce0269e460 msgr2=0x55ce02a547e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:19.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.088+0000 7f89834a0740 1 -- 172.21.15.3:0/4197504621 shutdown_connections 2024-09-21T06:21:19.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.088+0000 7f89834a0740 1 -- 172.21.15.3:0/4197504621 wait complete. 2024-09-21T06:21:19.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.088+0000 7f89834a0740 20 client.26896 trim_cache size 0 max 16384 2024-09-21T06:21:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.108+0000 7f89834a0740 1 Processor -- start 2024-09-21T06:21:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.108+0000 7f89834a0740 1 -- start start 2024-09-21T06:21:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f89834a0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a5d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f89834a0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce02689390 0x55ce02a69050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f89834a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce026aa070 0x55ce02a59f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f89834a0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f89781ad450 con 0x55ce026811b0 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f89834a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89781aefc0 con 0x55ce026aa070 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f89834a0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f89781adb20 con 0x55ce02689390 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f897659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a5d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f897659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a5d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47372/0 (socket says 172.21.15.3:47372) 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f896ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce02689390 0x55ce02a69050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f897659c640 1 -- 172.21.15.3:0/594012405 learned_addr learned my addr 172.21.15.3:0/594012405 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f8976d9d640 1 --2- 172.21.15.3:0/594012405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce026aa070 0x55ce02a59f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f897659c640 1 -- 172.21.15.3:0/594012405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce02689390 msgr2=0x55ce02a69050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f897659c640 1 --2- 172.21.15.3:0/594012405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce02689390 0x55ce02a69050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f897659c640 1 -- 172.21.15.3:0/594012405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce026aa070 msgr2=0x55ce02a59f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f897659c640 1 --2- 172.21.15.3:0/594012405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce026aa070 0x55ce02a59f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f897659c640 1 -- 172.21.15.3:0/594012405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce02a53da0 con 0x55ce026811b0 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f8976d9d640 1 --2- 172.21.15.3:0/594012405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce026aa070 0x55ce02a59f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.109+0000 7f897659c640 1 --2- 172.21.15.3:0/594012405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a5d230 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8968006010 tx=0x7f896803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a93ef4b424ffac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:19.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.111+0000 7f895ffff640 1 -- 172.21.15.3:0/594012405 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968043070 con 0x55ce026811b0 2024-09-21T06:21:19.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.111+0000 7f895ffff640 1 -- 172.21.15.3:0/594012405 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8968005500 con 0x55ce026811b0 2024-09-21T06:21:19.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.111+0000 7f895ffff640 1 -- 172.21.15.3:0/594012405 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968035040 con 0x55ce026811b0 2024-09-21T06:21:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.112+0000 7f89834a0740 1 -- 172.21.15.3:0/594012405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 msgr2=0x55ce02a5d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.112+0000 7f89834a0740 1 --2- 172.21.15.3:0/594012405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a5d230 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8968006010 tx=0x7f896803f000 comp rx=0 tx=0).stop 2024-09-21T06:21:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.112+0000 7f89834a0740 1 -- 172.21.15.3:0/594012405 shutdown_connections 2024-09-21T06:21:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.112+0000 7f89834a0740 1 --2- 172.21.15.3:0/594012405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce02689390 0x55ce02a69050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.112+0000 7f89834a0740 1 --2- 172.21.15.3:0/594012405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce026aa070 0x55ce02a59f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.112+0000 7f89834a0740 1 --2- 172.21.15.3:0/594012405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026811b0 0x55ce02a5d230 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.112+0000 7f89834a0740 1 -- 172.21.15.3:0/594012405 >> 172.21.15.3:0/594012405 conn(0x55ce0269e460 msgr2=0x55ce02a5cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.112+0000 7f89834a0740 1 -- 172.21.15.3:0/594012405 shutdown_connections 2024-09-21T06:21:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.112+0000 7f89834a0740 1 -- 172.21.15.3:0/594012405 wait complete. 2024-09-21T06:21:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.112+0000 7f89834a0740 1 -- start start 2024-09-21T06:21:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.112+0000 7f89834a0740 1 Processor -- start 2024-09-21T06:21:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f89834a0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce026811b0 0x55ce02a61280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f89834a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce02689390 0x55ce02a617c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f89834a0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026aa070 0x55ce02a61d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f89834a0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f89781ae8f0 con 0x55ce026aa070 2024-09-21T06:21:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f89834a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89781ae220 con 0x55ce02689390 2024-09-21T06:21:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f89834a0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f89781ab430 con 0x55ce026811b0 2024-09-21T06:21:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f897659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce026811b0 0x55ce02a61280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f897659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce026811b0 0x55ce02a61280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:39888/0 (socket says 172.21.15.3:39888) 2024-09-21T06:21:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f897659c640 1 -- 172.21.15.3:0/1910927500 learned_addr learned my addr 172.21.15.3:0/1910927500 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f8976d9d640 1 --2- 172.21.15.3:0/1910927500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026aa070 0x55ce02a61d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f89834a0740 10 client.0 _ll_register_callbacks cb 0x7f8970004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f896ffff640 1 --2- 172.21.15.3:0/1910927500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce02689390 0x55ce02a617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f897659c640 1 -- 172.21.15.3:0/1910927500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce02689390 msgr2=0x55ce02a617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f897659c640 1 --2- 172.21.15.3:0/1910927500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce02689390 0x55ce02a617c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f897659c640 1 -- 172.21.15.3:0/1910927500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026aa070 msgr2=0x55ce02a61d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:19.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f897659c640 1 --2- 172.21.15.3:0/1910927500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026aa070 0x55ce02a61d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.113+0000 7f897659c640 1 -- 172.21.15.3:0/1910927500 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce02add040 con 0x55ce026811b0 2024-09-21T06:21:19.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f897659c640 1 --2- 172.21.15.3:0/1910927500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce026811b0 0x55ce02a61280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8968005940 tx=0x7f89680317a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=646bfd03c9b19cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:19.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f895dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:21:19.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f895dffb640 1 -- 172.21.15.3:0/1910927500 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968060060 con 0x55ce026811b0 2024-09-21T06:21:19.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f895dffb640 1 -- 172.21.15.3:0/1910927500 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f8968004770 con 0x55ce026811b0 2024-09-21T06:21:19.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f89834a0740 1 -- 172.21.15.3:0/1910927500 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8968005500 con 0x55ce026811b0 2024-09-21T06:21:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f895dffb640 1 -- 172.21.15.3:0/1910927500 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8968004970 con 0x55ce026811b0 2024-09-21T06:21:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f89834a0740 10 client.26902 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f89834a0740 1 -- 172.21.15.3:0/1910927500 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ce02a698f0 con 0x55ce026811b0 2024-09-21T06:21:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f895dffb640 1 -- 172.21.15.3:0/1910927500 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8968005500 con 0x55ce026811b0 2024-09-21T06:21:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f895dffb640 1 -- 172.21.15.3:0/1910927500 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f896803d450 con 0x55ce026811b0 2024-09-21T06:21:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f89834a0740 10 client.26902 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f89834a0740 10 client.26902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f89834a0740 20 client.26902 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f89834a0740 1 --2- 172.21.15.3:0/1910927500 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55ce0263dda0 0x55ce02a6c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f89834a0740 4 client.26902 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f89834a0740 1 -- 172.21.15.3:0/1910927500 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce029c5850 con 0x55ce0263dda0 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.114+0000 7f896ffff640 1 --2- 172.21.15.3:0/1910927500 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55ce0263dda0 0x55ce02a6c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f896ffff640 1 --2- 172.21.15.3:0/1910927500 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55ce0263dda0 0x55ce02a6c0e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f0788733301a1e7 server_cookie=975fe89cd5c3fb33 in_seq=0 out_seq=0 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f895dffb640 10 client.26902 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f895dffb640 1 -- 172.21.15.3:0/1910927500 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x55ce029c5850 con 0x55ce0263dda0 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f895dffb640 10 client.26902 handle_command_reply: tid=0 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f89834a0740 1 client.26902 shutdown 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f89834a0740 1 -- 172.21.15.3:0/1910927500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce026811b0 msgr2=0x55ce02a61280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f89834a0740 1 --2- 172.21.15.3:0/1910927500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce026811b0 0x55ce02a61280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8968005940 tx=0x7f89680317a0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f89834a0740 1 -- 172.21.15.3:0/1910927500 shutdown_connections 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f89834a0740 1 --2- 172.21.15.3:0/1910927500 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55ce0263dda0 0x55ce02a6c0e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f89834a0740 1 --2- 172.21.15.3:0/1910927500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ce026aa070 0x55ce02a61d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f89834a0740 1 --2- 172.21.15.3:0/1910927500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce02689390 0x55ce02a617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f895dffb640 0 client.26902 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:19.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f89834a0740 1 --2- 172.21.15.3:0/1910927500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce026811b0 0x55ce02a61280 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.116+0000 7f89834a0740 1 -- 172.21.15.3:0/1910927500 >> 172.21.15.3:0/1910927500 conn(0x55ce0269e460 msgr2=0x55ce02a5e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:19.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.117+0000 7f89834a0740 1 -- 172.21.15.3:0/1910927500 shutdown_connections 2024-09-21T06:21:19.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.117+0000 7f89834a0740 1 -- 172.21.15.3:0/1910927500 wait complete. 2024-09-21T06:21:19.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.117+0000 7f89834a0740 20 client.26902 trim_cache size 0 max 16384 2024-09-21T06:21:19.122 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:21:19.122 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:21:19.122 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-21T06:21:19.122 INFO:teuthology.orchestra.run.smithi003.stdout: "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58": { 2024-09-21T06:21:19.122 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58", 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:21:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58": { 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:21:17.396219+0000", 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:21:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:21:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:21:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:21:19.128 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:21:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.117+0000 7f897cf2c640 1 -- 172.21.15.3:0/3338472262 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89640b6680 msgr2=0x7f89640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.117+0000 7f897cf2c640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89640b6680 0x7f89640b8b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f89680096c0 tx=0x7f89680058d0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.117+0000 7f897cf2c640 1 -- 172.21.15.3:0/3338472262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8978109e10 msgr2=0x7f897819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.117+0000 7f897cf2c640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8978109e10 0x7f897819fd20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8970009fd0 tx=0x7f897000ef10 comp rx=0 tx=0).stop 2024-09-21T06:21:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.117+0000 7f897cf2c640 1 -- 172.21.15.3:0/3338472262 shutdown_connections 2024-09-21T06:21:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.117+0000 7f897cf2c640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f89640b6680 0x7f89640b8b90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.117+0000 7f897cf2c640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8978109e10 0x7f897819fd20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.117+0000 7f897cf2c640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89781094c0 0x7f897819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.117+0000 7f897cf2c640 1 --2- 172.21.15.3:0/3338472262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8978103b80 0x7f897819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:19.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.117+0000 7f897cf2c640 1 -- 172.21.15.3:0/3338472262 >> 172.21.15.3:0/3338472262 conn(0x7f89780ff610 msgr2=0x7f8978101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:19.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.118+0000 7f897cf2c640 1 -- 172.21.15.3:0/3338472262 shutdown_connections 2024-09-21T06:21:19.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:19.118+0000 7f897cf2c640 1 -- 172.21.15.3:0/3338472262 wait complete. 2024-09-21T06:21:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:19 smithi204 ceph-mon[34659]: pgmap v514: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 320 op/s 2024-09-21T06:21:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:19 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:19.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:19 smithi003 ceph-mon[31227]: pgmap v514: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 320 op/s 2024-09-21T06:21:19.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:19 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:19.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:19 smithi137 ceph-mon[34464]: pgmap v514: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 320 op/s 2024-09-21T06:21:19.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:19 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:19.932 INFO:tasks.cephfs.filesystem:scrub status for tag:ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58 - {'path': '/', 'tag': 'ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58', 'options': 'recursive,force'} 2024-09-21T06:21:19.961 INFO:tasks.workunit.client.0.smithi003.stdout: 1 659914 11.54 MB/sec execute 172 sec latency 75.537 ms 2024-09-21T06:21:20.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 661498 11.51 MB/sec execute 173 sec latency 34.189 ms 2024-09-21T06:21:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:21 smithi204 ceph-mon[34659]: pgmap v515: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 303 op/s 2024-09-21T06:21:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:21 smithi003 ceph-mon[31227]: pgmap v515: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 303 op/s 2024-09-21T06:21:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:21 smithi137 ceph-mon[34464]: pgmap v515: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 303 op/s 2024-09-21T06:21:21.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 664220 11.54 MB/sec execute 174 sec latency 18.359 ms 2024-09-21T06:21:22.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 665752 11.52 MB/sec execute 175 sec latency 19.022 ms 2024-09-21T06:21:23.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:23 smithi204 ceph-mon[34659]: pgmap v516: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2024-09-21T06:21:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:23 smithi003 ceph-mon[31227]: pgmap v516: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2024-09-21T06:21:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:23 smithi137 ceph-mon[34464]: pgmap v516: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2024-09-21T06:21:23.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 668006 11.53 MB/sec execute 176 sec latency 58.885 ms 2024-09-21T06:21:24.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 670205 11.51 MB/sec execute 177 sec latency 15.475 ms 2024-09-21T06:21:25.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:25 smithi204 ceph-mon[34659]: pgmap v517: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-09-21T06:21:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:25 smithi003 ceph-mon[31227]: pgmap v517: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-09-21T06:21:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:25 smithi137 ceph-mon[34464]: pgmap v517: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-09-21T06:21:25.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 671737 11.51 MB/sec execute 178 sec latency 53.283 ms 2024-09-21T06:21:26.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:21:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:26 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:21:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:21:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:21:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:21:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:21:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:21:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:21:26.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:21:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:21:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:26 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:21:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:21:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:21:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:21:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:21:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:21:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:21:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:21:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:21:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:26 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:21:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:21:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:21:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:21:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:21:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:21:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:21:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:21:26.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 674387 11.52 MB/sec execute 179 sec latency 16.639 ms 2024-09-21T06:21:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:27 smithi003 ceph-mon[31227]: pgmap v518: 97 pgs: 97 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 367 op/s 2024-09-21T06:21:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:27 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:21:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:27 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:21:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:27 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi003 to 177.0M 2024-09-21T06:21:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:27 smithi003 ceph-mon[31227]: Unable to set osd_memory_target on smithi003 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-21T06:21:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:27 smithi137 ceph-mon[34464]: pgmap v518: 97 pgs: 97 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 367 op/s 2024-09-21T06:21:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:27 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:21:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:27 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:21:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:27 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi003 to 177.0M 2024-09-21T06:21:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:27 smithi137 ceph-mon[34464]: Unable to set osd_memory_target on smithi003 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-21T06:21:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:27 smithi204 ceph-mon[34659]: pgmap v518: 97 pgs: 97 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 367 op/s 2024-09-21T06:21:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:27 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:21:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:27 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:21:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:27 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi003 to 177.0M 2024-09-21T06:21:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:27 smithi204 ceph-mon[34659]: Unable to set osd_memory_target on smithi003 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-21T06:21:27.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 675946 11.50 MB/sec execute 180 sec latency 39.160 ms 2024-09-21T06:21:28.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 678512 11.53 MB/sec execute 181 sec latency 18.322 ms 2024-09-21T06:21:29.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:29 smithi003 ceph-mon[31227]: pgmap v519: 97 pgs: 97 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 358 op/s 2024-09-21T06:21:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:29 smithi137 ceph-mon[34464]: pgmap v519: 97 pgs: 97 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 358 op/s 2024-09-21T06:21:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:29 smithi204 ceph-mon[34659]: pgmap v519: 97 pgs: 97 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 358 op/s 2024-09-21T06:21:29.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 680015 11.49 MB/sec execute 182 sec latency 17.334 ms 2024-09-21T06:21:30.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.153+0000 7f3419f19640 1 -- 172.21.15.3:0/1678944776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3414108300 msgr2=0x7f3414108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:30.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.153+0000 7f3419f19640 1 --2- 172.21.15.3:0/1678944776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3414108300 0x7f3414108710 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3408009900 tx=0x7f340802a410 comp rx=0 tx=0).stop 2024-09-21T06:21:30.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.153+0000 7f3419f19640 1 -- 172.21.15.3:0/1678944776 shutdown_connections 2024-09-21T06:21:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.153+0000 7f3419f19640 1 --2- 172.21.15.3:0/1678944776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3414108c50 0x7f341410f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.153+0000 7f3419f19640 1 --2- 172.21.15.3:0/1678944776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3414108300 0x7f3414108710 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.153+0000 7f3419f19640 1 --2- 172.21.15.3:0/1678944776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34141029d0 0x7f3414102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.153+0000 7f3419f19640 1 -- 172.21.15.3:0/1678944776 >> 172.21.15.3:0/1678944776 conn(0x7f341407a6f0 msgr2=0x7f341407ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.153+0000 7f3419f19640 1 -- 172.21.15.3:0/1678944776 shutdown_connections 2024-09-21T06:21:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.153+0000 7f3419f19640 1 -- 172.21.15.3:0/1678944776 wait complete. 2024-09-21T06:21:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.153+0000 7f3419f19640 1 Processor -- start 2024-09-21T06:21:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f3419f19640 1 -- start start 2024-09-21T06:21:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f3419f19640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34141029d0 0x7f341419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f3419f19640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3414108300 0x7f341419b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f3419f19640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3414108c50 0x7f341419bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f3419f19640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3414112910 con 0x7f34141029d0 2024-09-21T06:21:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f3419f19640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3414112790 con 0x7f3414108300 2024-09-21T06:21:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f3419f19640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3414112a90 con 0x7f3414108c50 2024-09-21T06:21:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f34137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34141029d0 0x7f341419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f34137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34141029d0 0x7f341419b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58860/0 (socket says 172.21.15.3:58860) 2024-09-21T06:21:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f34137fe640 1 -- 172.21.15.3:0/322835348 learned_addr learned my addr 172.21.15.3:0/322835348 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f3412ffd640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3414108300 0x7f341419b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f3413fff640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3414108c50 0x7f341419bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f34137fe640 1 -- 172.21.15.3:0/322835348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3414108c50 msgr2=0x7f341419bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f34137fe640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3414108c50 0x7f341419bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f34137fe640 1 -- 172.21.15.3:0/322835348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3414108300 msgr2=0x7f341419b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f34137fe640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3414108300 0x7f341419b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.154+0000 7f34137fe640 1 -- 172.21.15.3:0/322835348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34141a6550 con 0x7f34141029d0 2024-09-21T06:21:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.155+0000 7f34137fe640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34141029d0 0x7f341419b2c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f340400e9f0 tx=0x7f340400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.155+0000 7f3410ff9640 1 -- 172.21.15.3:0/322835348 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340400cd80 con 0x7f34141029d0 2024-09-21T06:21:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.155+0000 7f3410ff9640 1 -- 172.21.15.3:0/322835348 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34040054d0 con 0x7f34141029d0 2024-09-21T06:21:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.155+0000 7f3419f19640 1 -- 172.21.15.3:0/322835348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34141a6840 con 0x7f34141029d0 2024-09-21T06:21:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.155+0000 7f3410ff9640 1 -- 172.21.15.3:0/322835348 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404010690 con 0x7f34141029d0 2024-09-21T06:21:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.155+0000 7f3419f19640 1 -- 172.21.15.3:0/322835348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34141a6d60 con 0x7f34141029d0 2024-09-21T06:21:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.157+0000 7f3420e83740 1 Processor -- start 2024-09-21T06:21:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3420e83740 1 -- start start 2024-09-21T06:21:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3410ff9640 1 -- 172.21.15.3:0/322835348 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f3404004e90 con 0x7f34141029d0 2024-09-21T06:21:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3420e83740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297abb71b0 0x56297af5c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3420e83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 0x56297af5cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3420e83740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab964d0 0x56297af5d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3420e83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56297aed16f0 con 0x56297ab8e2f0 2024-09-21T06:21:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3420e83740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56297aec0b20 con 0x56297ab964d0 2024-09-21T06:21:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3420e83740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56297aec4930 con 0x56297abb71b0 2024-09-21T06:21:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f34137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 0x56297af5cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f34137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 0x56297af5cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58876/0 (socket says 172.21.15.3:58876) 2024-09-21T06:21:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f34137fe640 1 -- 172.21.15.3:0/1094691099 learned_addr learned my addr 172.21.15.3:0/1094691099 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3413fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297abb71b0 0x56297af5c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3412ffd640 1 --2- 172.21.15.3:0/1094691099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab964d0 0x56297af5d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3412ffd640 1 -- 172.21.15.3:0/1094691099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297abb71b0 msgr2=0x56297af5c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3412ffd640 1 --2- 172.21.15.3:0/1094691099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297abb71b0 0x56297af5c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3412ffd640 1 -- 172.21.15.3:0/1094691099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 msgr2=0x56297af5cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3412ffd640 1 --2- 172.21.15.3:0/1094691099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 0x56297af5cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f3412ffd640 1 -- 172.21.15.3:0/1094691099 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56297ae86530 con 0x56297ab964d0 2024-09-21T06:21:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.158+0000 7f34137fe640 1 --2- 172.21.15.3:0/1094691099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 0x56297af5cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.159+0000 7f3410ff9640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f33fc0b6700 0x7f33fc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.159+0000 7f3412ffd640 1 --2- 172.21.15.3:0/1094691099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab964d0 0x56297af5d430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f341406c6f0 tx=0x7f3408032ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40d948db6f18696b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.159+0000 7f3413fff640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f33fc0b6700 0x7f33fc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.159+0000 7f3410ff9640 1 -- 172.21.15.3:0/322835348 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f34040e7bc0 con 0x7f34141029d0 2024-09-21T06:21:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.159+0000 7f33f9ffb640 1 -- 172.21.15.3:0/1094691099 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408035030 con 0x56297ab964d0 2024-09-21T06:21:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.159+0000 7f33f9ffb640 1 -- 172.21.15.3:0/1094691099 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3408004030 con 0x56297ab964d0 2024-09-21T06:21:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.159+0000 7f33f9ffb640 1 -- 172.21.15.3:0/1094691099 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408005470 con 0x56297ab964d0 2024-09-21T06:21:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.160+0000 7f3420e83740 1 -- 172.21.15.3:0/1094691099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab964d0 msgr2=0x56297af5d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.160+0000 7f3413fff640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f33fc0b6700 0x7f33fc0b8c10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f340c005fd0 tx=0x7f340c009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.160+0000 7f3420e83740 1 --2- 172.21.15.3:0/1094691099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab964d0 0x56297af5d430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f341406c6f0 tx=0x7f3408032ba0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.160+0000 7f3420e83740 1 -- 172.21.15.3:0/1094691099 shutdown_connections 2024-09-21T06:21:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.160+0000 7f3420e83740 1 --2- 172.21.15.3:0/1094691099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab964d0 0x56297af5d430 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.160+0000 7f3420e83740 1 --2- 172.21.15.3:0/1094691099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 0x56297af5cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.160+0000 7f3420e83740 1 --2- 172.21.15.3:0/1094691099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297abb71b0 0x56297af5c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.160+0000 7f3420e83740 1 -- 172.21.15.3:0/1094691099 >> 172.21.15.3:0/1094691099 conn(0x56297abab300 msgr2=0x56297af5a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:30.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.160+0000 7f3420e83740 1 -- 172.21.15.3:0/1094691099 shutdown_connections 2024-09-21T06:21:30.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.160+0000 7f3420e83740 1 -- 172.21.15.3:0/1094691099 wait complete. 2024-09-21T06:21:30.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f3420e83740 1 -- start start 2024-09-21T06:21:30.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f3420e83740 1 Processor -- start 2024-09-21T06:21:30.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f3420e83740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297ab8e2f0 0x56297af68060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f3420e83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab964d0 0x56297af685a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f3420e83740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297abb71b0 0x56297af787d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f3420e83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56297adf8ab0 con 0x56297ab964d0 2024-09-21T06:21:30.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f3420e83740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56297ae66730 con 0x56297abb71b0 2024-09-21T06:21:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f3420e83740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56297af6f4e0 con 0x56297ab8e2f0 2024-09-21T06:21:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f3420e83740 10 client.0 _ll_register_callbacks cb 0x56297aeba6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f3412ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab964d0 0x56297af685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f3412ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab964d0 0x56297af685a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58886/0 (socket says 172.21.15.3:58886) 2024-09-21T06:21:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f3412ffd640 1 -- 172.21.15.3:0/2116249593 learned_addr learned my addr 172.21.15.3:0/2116249593 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f3413fff640 1 --2- 172.21.15.3:0/2116249593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297abb71b0 0x56297af787d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.161+0000 7f34137fe640 1 --2- 172.21.15.3:0/2116249593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297ab8e2f0 0x56297af68060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.162+0000 7f3412ffd640 1 -- 172.21.15.3:0/2116249593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297ab8e2f0 msgr2=0x56297af68060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.162+0000 7f3412ffd640 1 --2- 172.21.15.3:0/2116249593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297ab8e2f0 0x56297af68060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.162+0000 7f3412ffd640 1 -- 172.21.15.3:0/2116249593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297abb71b0 msgr2=0x56297af787d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.162+0000 7f3412ffd640 1 --2- 172.21.15.3:0/2116249593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297abb71b0 0x56297af787d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.162+0000 7f3412ffd640 1 -- 172.21.15.3:0/2116249593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56297ac22790 con 0x56297ab964d0 2024-09-21T06:21:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.162+0000 7f3412ffd640 1 --2- 172.21.15.3:0/2116249593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab964d0 0x56297af685a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f3408000c00 tx=0x7f3408005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212e1f9e3183a301 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.162+0000 7f33f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.162+0000 7f33f2ffd640 1 -- 172.21.15.3:0/2116249593 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408035070 con 0x56297ab964d0 2024-09-21T06:21:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.162+0000 7f33f2ffd640 1 -- 172.21.15.3:0/2116249593 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f3408005480 con 0x56297ab964d0 2024-09-21T06:21:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.162+0000 7f3420e83740 1 -- 172.21.15.3:0/2116249593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3408004030 con 0x56297ab964d0 2024-09-21T06:21:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.162+0000 7f33f2ffd640 1 -- 172.21.15.3:0/2116249593 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f34080314a0 con 0x56297ab964d0 2024-09-21T06:21:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.162+0000 7f3420e83740 10 client.17400 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.162+0000 7f3420e83740 1 -- 172.21.15.3:0/2116249593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56297ac9b620 con 0x56297ab964d0 2024-09-21T06:21:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.163+0000 7f33f2ffd640 1 -- 172.21.15.3:0/2116249593 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3408004030 con 0x56297ab964d0 2024-09-21T06:21:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.163+0000 7f33f2ffd640 1 -- 172.21.15.3:0/2116249593 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f34080035f0 con 0x56297ab964d0 2024-09-21T06:21:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.163+0000 7f3420e83740 10 client.17400 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.163+0000 7f3420e83740 10 client.17400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.163+0000 7f3420e83740 20 client.17400 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.163+0000 7f3420e83740 1 --2- 172.21.15.3:0/2116249593 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56297ab75d80 0x56297af7e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.163+0000 7f3420e83740 4 client.17400 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.163+0000 7f3420e83740 1 -- 172.21.15.3:0/2116249593 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56297aed3640 con 0x56297ab75d80 2024-09-21T06:21:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.163+0000 7f34137fe640 1 --2- 172.21.15.3:0/2116249593 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56297ab75d80 0x56297af7e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.163+0000 7f34137fe640 1 --2- 172.21.15.3:0/2116249593 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56297ab75d80 0x56297af7e380 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6905680c1fd0b07 server_cookie=6db4ac0aaa44d34f in_seq=0 out_seq=0 2024-09-21T06:21:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.163+0000 7f33f2ffd640 10 client.17400 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f33f2ffd640 1 -- 172.21.15.3:0/2116249593 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56297aed3640 con 0x56297ab75d80 2024-09-21T06:21:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f33f2ffd640 10 client.17400 handle_command_reply: tid=0 2024-09-21T06:21:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f3420e83740 1 client.17400 shutdown 2024-09-21T06:21:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f3420e83740 1 -- 172.21.15.3:0/2116249593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab964d0 msgr2=0x56297af685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f3420e83740 1 --2- 172.21.15.3:0/2116249593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab964d0 0x56297af685a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f3408000c00 tx=0x7f3408005d70 comp rx=0 tx=0).stop 2024-09-21T06:21:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f3420e83740 1 -- 172.21.15.3:0/2116249593 shutdown_connections 2024-09-21T06:21:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f3420e83740 1 --2- 172.21.15.3:0/2116249593 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56297ab75d80 0x56297af7e380 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f3420e83740 1 --2- 172.21.15.3:0/2116249593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297abb71b0 0x56297af787d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f3420e83740 1 --2- 172.21.15.3:0/2116249593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab964d0 0x56297af685a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f33f2ffd640 0 client.17400 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f3420e83740 1 --2- 172.21.15.3:0/2116249593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297ab8e2f0 0x56297af68060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f3420e83740 1 -- 172.21.15.3:0/2116249593 >> 172.21.15.3:0/2116249593 conn(0x56297abab300 msgr2=0x56297af5d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f3420e83740 1 -- 172.21.15.3:0/2116249593 shutdown_connections 2024-09-21T06:21:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f3420e83740 1 -- 172.21.15.3:0/2116249593 wait complete. 2024-09-21T06:21:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.165+0000 7f3420e83740 20 client.17400 trim_cache size 0 max 16384 2024-09-21T06:21:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.181+0000 7f3420e83740 1 Processor -- start 2024-09-21T06:21:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.181+0000 7f3420e83740 1 -- start start 2024-09-21T06:21:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.181+0000 7f3420e83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 0x56297af5e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.181+0000 7f3420e83740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab964d0 0x56297af69c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.181+0000 7f3420e83740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297abb71b0 0x56297af64220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.181+0000 7f3420e83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34141ad3a0 con 0x56297ab8e2f0 2024-09-21T06:21:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.181+0000 7f3420e83740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34141aef10 con 0x56297ab964d0 2024-09-21T06:21:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.181+0000 7f3420e83740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34141ada70 con 0x56297abb71b0 2024-09-21T06:21:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f34137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 0x56297af5e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f34137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 0x56297af5e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58896/0 (socket says 172.21.15.3:58896) 2024-09-21T06:21:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f34137fe640 1 -- 172.21.15.3:0/2689109934 learned_addr learned my addr 172.21.15.3:0/2689109934 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f3412ffd640 1 --2- 172.21.15.3:0/2689109934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab964d0 0x56297af69c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f3413fff640 1 --2- 172.21.15.3:0/2689109934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297abb71b0 0x56297af64220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f34137fe640 1 -- 172.21.15.3:0/2689109934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297abb71b0 msgr2=0x56297af64220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f34137fe640 1 --2- 172.21.15.3:0/2689109934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297abb71b0 0x56297af64220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f34137fe640 1 -- 172.21.15.3:0/2689109934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab964d0 msgr2=0x56297af69c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f34137fe640 1 --2- 172.21.15.3:0/2689109934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab964d0 0x56297af69c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f34137fe640 1 -- 172.21.15.3:0/2689109934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56297af68ef0 con 0x56297ab8e2f0 2024-09-21T06:21:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f3412ffd640 1 --2- 172.21.15.3:0/2689109934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab964d0 0x56297af69c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f34137fe640 1 --2- 172.21.15.3:0/2689109934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 0x56297af5e3a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f34040ad9e0 tx=0x7f34040eb710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee35bace5a3c0ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f33f97fa640 1 -- 172.21.15.3:0/2689109934 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34040a98f0 con 0x56297ab8e2f0 2024-09-21T06:21:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f33f97fa640 1 -- 172.21.15.3:0/2689109934 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34040ad560 con 0x56297ab8e2f0 2024-09-21T06:21:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.182+0000 7f33f97fa640 1 -- 172.21.15.3:0/2689109934 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34040a91c0 con 0x56297ab8e2f0 2024-09-21T06:21:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.183+0000 7f3420e83740 1 -- 172.21.15.3:0/2689109934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 msgr2=0x56297af5e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.183+0000 7f3420e83740 1 --2- 172.21.15.3:0/2689109934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 0x56297af5e3a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f34040ad9e0 tx=0x7f34040eb710 comp rx=0 tx=0).stop 2024-09-21T06:21:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.183+0000 7f3420e83740 1 -- 172.21.15.3:0/2689109934 shutdown_connections 2024-09-21T06:21:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.183+0000 7f3420e83740 1 --2- 172.21.15.3:0/2689109934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297abb71b0 0x56297af64220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.183+0000 7f3420e83740 1 --2- 172.21.15.3:0/2689109934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab964d0 0x56297af69c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.183+0000 7f3420e83740 1 --2- 172.21.15.3:0/2689109934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297ab8e2f0 0x56297af5e3a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.183+0000 7f3420e83740 1 -- 172.21.15.3:0/2689109934 >> 172.21.15.3:0/2689109934 conn(0x56297abab300 msgr2=0x56297af69580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.183+0000 7f3420e83740 1 -- 172.21.15.3:0/2689109934 shutdown_connections 2024-09-21T06:21:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.183+0000 7f3420e83740 1 -- 172.21.15.3:0/2689109934 wait complete. 2024-09-21T06:21:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f3420e83740 1 -- start start 2024-09-21T06:21:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f3420e83740 1 Processor -- start 2024-09-21T06:21:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f3420e83740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab8e2f0 0x56297af80a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f3420e83740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297ab964d0 0x56297af80fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f3420e83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297abb71b0 0x56297afe6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f3420e83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34141ae840 con 0x56297abb71b0 2024-09-21T06:21:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f3420e83740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34141ae170 con 0x56297ab8e2f0 2024-09-21T06:21:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f3420e83740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34141ab380 con 0x56297ab964d0 2024-09-21T06:21:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f3420e83740 10 client.0 _ll_register_callbacks cb 0x7f34040eb5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f3413fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297abb71b0 0x56297afe6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f34137fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab8e2f0 0x56297af80a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f3413fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297abb71b0 0x56297afe6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58906/0 (socket says 172.21.15.3:58906) 2024-09-21T06:21:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f34137fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab8e2f0 0x56297af80a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:38362/0 (socket says 172.21.15.3:38362) 2024-09-21T06:21:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f3413fff640 1 -- 172.21.15.3:0/3311164915 learned_addr learned my addr 172.21.15.3:0/3311164915 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.184+0000 7f3412ffd640 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297ab964d0 0x56297af80fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f3413fff640 1 -- 172.21.15.3:0/3311164915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297ab964d0 msgr2=0x56297af80fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f3413fff640 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297ab964d0 0x56297af80fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f3413fff640 1 -- 172.21.15.3:0/3311164915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab8e2f0 msgr2=0x56297af80a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f3413fff640 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab8e2f0 0x56297af80a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f3413fff640 1 -- 172.21.15.3:0/3311164915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56297afe7040 con 0x56297abb71b0 2024-09-21T06:21:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f3412ffd640 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297ab964d0 0x56297af80fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f34137fe640 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab8e2f0 0x56297af80a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f3413fff640 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297abb71b0 0x56297afe6b00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f340c01ea00 tx=0x7f340c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f7d36f42d7f079 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f33f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f33f2ffd640 1 -- 172.21.15.3:0/3311164915 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c01d620 con 0x56297abb71b0 2024-09-21T06:21:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f33f2ffd640 1 -- 172.21.15.3:0/3311164915 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f340c01d7c0 con 0x56297abb71b0 2024-09-21T06:21:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f3420e83740 1 -- 172.21.15.3:0/3311164915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34040ad560 con 0x56297abb71b0 2024-09-21T06:21:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f33f2ffd640 1 -- 172.21.15.3:0/3311164915 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f340c037050 con 0x56297abb71b0 2024-09-21T06:21:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f3420e83740 10 client.17406 fetch_fsmap learned FSMap version 32 2024-09-21T06:21:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.185+0000 7f3420e83740 1 -- 172.21.15.3:0/3311164915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56297afe82c0 con 0x56297abb71b0 2024-09-21T06:21:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.186+0000 7f33f2ffd640 1 -- 172.21.15.3:0/3311164915 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34040ad560 con 0x56297abb71b0 2024-09-21T06:21:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.186+0000 7f33f2ffd640 1 -- 172.21.15.3:0/3311164915 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f340c029b80 con 0x56297abb71b0 2024-09-21T06:21:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.186+0000 7f3420e83740 10 client.17406 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:21:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.186+0000 7f3420e83740 10 client.17406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.186+0000 7f3420e83740 20 client.17406 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.186+0000 7f3420e83740 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56297ab4b1c0 0x56297afeab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.186+0000 7f3420e83740 4 client.17406 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:21:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.186+0000 7f3420e83740 1 -- 172.21.15.3:0/3311164915 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56297aecf0f0 con 0x56297ab4b1c0 2024-09-21T06:21:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.186+0000 7f34137fe640 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56297ab4b1c0 0x56297afeab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.186+0000 7f33f2ffd640 10 client.17406 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.186+0000 7f34137fe640 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56297ab4b1c0 0x56297afeab30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4812d0a2f63508a5 server_cookie=d1ee807b3ab3be07 in_seq=0 out_seq=0 2024-09-21T06:21:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.348+0000 7f33f2ffd640 1 -- 172.21.15.3:0/3311164915 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56297aecf0f0 con 0x56297ab4b1c0 2024-09-21T06:21:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.348+0000 7f33f2ffd640 10 client.17406 handle_command_reply: tid=0 2024-09-21T06:21:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.348+0000 7f3420e83740 1 client.17406 shutdown 2024-09-21T06:21:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.348+0000 7f3420e83740 1 -- 172.21.15.3:0/3311164915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297abb71b0 msgr2=0x56297afe6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.348+0000 7f3420e83740 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297abb71b0 0x56297afe6b00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f340c01ea00 tx=0x7f340c004a40 comp rx=0 tx=0).stop 2024-09-21T06:21:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.348+0000 7f3420e83740 1 -- 172.21.15.3:0/3311164915 shutdown_connections 2024-09-21T06:21:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.348+0000 7f3420e83740 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56297ab4b1c0 0x56297afeab30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.349+0000 7f33f2ffd640 0 client.17406 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.349+0000 7f3420e83740 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56297ab964d0 0x56297af80fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.349+0000 7f3420e83740 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56297abb71b0 0x56297afe6b00 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.349+0000 7f3420e83740 1 --2- 172.21.15.3:0/3311164915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297ab8e2f0 0x56297af80a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.349+0000 7f3420e83740 1 -- 172.21.15.3:0/3311164915 >> 172.21.15.3:0/3311164915 conn(0x56297abab300 msgr2=0x56297af59630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.349+0000 7f3420e83740 1 -- 172.21.15.3:0/3311164915 shutdown_connections 2024-09-21T06:21:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.349+0000 7f3420e83740 1 -- 172.21.15.3:0/3311164915 wait complete. 2024-09-21T06:21:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.349+0000 7f3420e83740 20 client.17406 trim_cache size 0 max 16384 2024-09-21T06:21:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:21:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:21:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:21:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:21:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.350+0000 7f3419f19640 1 -- 172.21.15.3:0/322835348 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f33fc0b6700 msgr2=0x7f33fc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.350+0000 7f3419f19640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f33fc0b6700 0x7f33fc0b8c10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f340c005fd0 tx=0x7f340c009500 comp rx=0 tx=0).stop 2024-09-21T06:21:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.350+0000 7f3419f19640 1 -- 172.21.15.3:0/322835348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34141029d0 msgr2=0x7f341419b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.350+0000 7f3419f19640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34141029d0 0x7f341419b2c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f340400e9f0 tx=0x7f340400edf0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.350+0000 7f3419f19640 1 -- 172.21.15.3:0/322835348 shutdown_connections 2024-09-21T06:21:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.350+0000 7f3419f19640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f33fc0b6700 0x7f33fc0b8c10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.350+0000 7f3419f19640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3414108c50 0x7f341419bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.350+0000 7f3419f19640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3414108300 0x7f341419b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.350+0000 7f3419f19640 1 --2- 172.21.15.3:0/322835348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34141029d0 0x7f341419b2c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.350+0000 7f3419f19640 1 -- 172.21.15.3:0/322835348 >> 172.21.15.3:0/322835348 conn(0x7f341407a6f0 msgr2=0x7f34141063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.351+0000 7f3419f19640 1 -- 172.21.15.3:0/322835348 shutdown_connections 2024-09-21T06:21:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:30.351+0000 7f3419f19640 1 -- 172.21.15.3:0/322835348 wait complete. 2024-09-21T06:21:30.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 682090 11.50 MB/sec execute 183 sec latency 69.426 ms 2024-09-21T06:21:31.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:31 smithi003 ceph-mon[31227]: pgmap v520: 97 pgs: 97 active+clean; 124 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 350 op/s 2024-09-21T06:21:31.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:31 smithi137 ceph-mon[34464]: pgmap v520: 97 pgs: 97 active+clean; 124 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 350 op/s 2024-09-21T06:21:31.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:31 smithi204 ceph-mon[34659]: pgmap v520: 97 pgs: 97 active+clean; 124 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 350 op/s 2024-09-21T06:21:31.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 683549 11.47 MB/sec execute 184 sec latency 47.857 ms 2024-09-21T06:21:32.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:32 smithi003 ceph-mon[31227]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:21:32.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:32 smithi137 ceph-mon[34464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:21:32.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:32 smithi204 ceph-mon[34659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:21:32.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 685826 11.48 MB/sec execute 185 sec latency 15.969 ms 2024-09-21T06:21:33.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:33 smithi003 ceph-mon[31227]: pgmap v521: 97 pgs: 97 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 325 op/s 2024-09-21T06:21:33.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:33 smithi137 ceph-mon[34464]: pgmap v521: 97 pgs: 97 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 325 op/s 2024-09-21T06:21:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:33 smithi204 ceph-mon[34659]: pgmap v521: 97 pgs: 97 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 325 op/s 2024-09-21T06:21:33.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 687990 11.47 MB/sec execute 186 sec latency 33.896 ms 2024-09-21T06:21:34.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 689875 11.47 MB/sec execute 187 sec latency 14.414 ms 2024-09-21T06:21:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:35 smithi003 ceph-mon[31227]: pgmap v522: 97 pgs: 97 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 266 op/s 2024-09-21T06:21:35.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:35 smithi137 ceph-mon[34464]: pgmap v522: 97 pgs: 97 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 266 op/s 2024-09-21T06:21:35.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:35 smithi204 ceph-mon[34659]: pgmap v522: 97 pgs: 97 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 266 op/s 2024-09-21T06:21:35.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 692457 11.50 MB/sec execute 188 sec latency 78.966 ms 2024-09-21T06:21:36.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 693984 11.47 MB/sec execute 189 sec latency 17.961 ms 2024-09-21T06:21:37.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:37 smithi003 ceph-mon[31227]: pgmap v523: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 371 op/s 2024-09-21T06:21:37.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:37 smithi137 ceph-mon[34464]: pgmap v523: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 371 op/s 2024-09-21T06:21:37.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:37 smithi204 ceph-mon[34659]: pgmap v523: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 371 op/s 2024-09-21T06:21:37.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 696305 11.48 MB/sec execute 190 sec latency 16.713 ms 2024-09-21T06:21:38.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 698019 11.45 MB/sec execute 191 sec latency 17.122 ms 2024-09-21T06:21:39.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:39 smithi003 ceph-mon[31227]: pgmap v524: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-09-21T06:21:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:39 smithi137 ceph-mon[34464]: pgmap v524: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-09-21T06:21:39.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:39 smithi204 ceph-mon[34659]: pgmap v524: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-09-21T06:21:39.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 700113 11.46 MB/sec execute 192 sec latency 15.313 ms 2024-09-21T06:21:40.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 702074 11.45 MB/sec execute 193 sec latency 73.540 ms 2024-09-21T06:21:41.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:41 smithi003 ceph-mon[31227]: pgmap v525: 97 pgs: 97 active+clean; 121 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 366 op/s 2024-09-21T06:21:41.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:41 smithi137 ceph-mon[34464]: pgmap v525: 97 pgs: 97 active+clean; 121 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 366 op/s 2024-09-21T06:21:41.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:41 smithi204 ceph-mon[34659]: pgmap v525: 97 pgs: 97 active+clean; 121 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 366 op/s 2024-09-21T06:21:41.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 703863 11.45 MB/sec execute 194 sec latency 27.115 ms 2024-09-21T06:21:42.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 706341 11.45 MB/sec execute 195 sec latency 19.460 ms 2024-09-21T06:21:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.465+0000 7f191248b640 1 -- 172.21.15.3:0/1605917082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f190c108c30 msgr2=0x7f190c10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.465+0000 7f191248b640 1 --2- 172.21.15.3:0/1605917082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f190c108c30 0x7f190c10f460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1904009ca0 tx=0x7f190402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.465+0000 7f191248b640 1 -- 172.21.15.3:0/1605917082 shutdown_connections 2024-09-21T06:21:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.465+0000 7f191248b640 1 --2- 172.21.15.3:0/1605917082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f190c108c30 0x7f190c10f460 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.465+0000 7f191248b640 1 --2- 172.21.15.3:0/1605917082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f190c1082e0 0x7f190c1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.465+0000 7f191248b640 1 --2- 172.21.15.3:0/1605917082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f190c1029b0 0x7f190c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.465+0000 7f191248b640 1 -- 172.21.15.3:0/1605917082 >> 172.21.15.3:0/1605917082 conn(0x7f190c07a740 msgr2=0x7f190c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.465+0000 7f191248b640 1 -- 172.21.15.3:0/1605917082 shutdown_connections 2024-09-21T06:21:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.465+0000 7f191248b640 1 -- 172.21.15.3:0/1605917082 wait complete. 2024-09-21T06:21:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.466+0000 7f191248b640 1 Processor -- start 2024-09-21T06:21:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.466+0000 7f191248b640 1 -- start start 2024-09-21T06:21:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.466+0000 7f191248b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f190c1029b0 0x7f190c19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f191248b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f190c1082e0 0x7f190c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f191248b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f190c108c30 0x7f190c1a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f191248b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1128f0 con 0x7f190c1082e0 2024-09-21T06:21:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f191248b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f190c112770 con 0x7f190c108c30 2024-09-21T06:21:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f191248b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f190c112a70 con 0x7f190c1029b0 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f190b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f190c1082e0 0x7f190c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f190bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f190c1029b0 0x7f190c19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f190b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f190c1082e0 0x7f190c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59638/0 (socket says 172.21.15.3:59638) 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f190b7fe640 1 -- 172.21.15.3:0/1785440894 learned_addr learned my addr 172.21.15.3:0/1785440894 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f1910a28640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f190c108c30 0x7f190c1a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f190b7fe640 1 -- 172.21.15.3:0/1785440894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f190c1029b0 msgr2=0x7f190c19c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f190b7fe640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f190c1029b0 0x7f190c19c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f190b7fe640 1 -- 172.21.15.3:0/1785440894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f190c108c30 msgr2=0x7f190c1a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f190b7fe640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f190c108c30 0x7f190c1a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f190b7fe640 1 -- 172.21.15.3:0/1785440894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c1a24e0 con 0x7f190c1082e0 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f190bfff640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f190c1029b0 0x7f190c19c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f1910a28640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f190c108c30 0x7f190c1a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f190b7fe640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f190c1082e0 0x7f190c19cb50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f190000e970 tx=0x7f190000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.467+0000 7f19097fa640 1 -- 172.21.15.3:0/1785440894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190000cd80 con 0x7f190c1082e0 2024-09-21T06:21:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.468+0000 7f191248b640 1 -- 172.21.15.3:0/1785440894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c1a27d0 con 0x7f190c1082e0 2024-09-21T06:21:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.468+0000 7f19097fa640 1 -- 172.21.15.3:0/1785440894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19000035f0 con 0x7f190c1082e0 2024-09-21T06:21:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.468+0000 7f19097fa640 1 -- 172.21.15.3:0/1785440894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900010690 con 0x7f190c1082e0 2024-09-21T06:21:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.468+0000 7f191248b640 1 -- 172.21.15.3:0/1785440894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c1a8f60 con 0x7f190c1082e0 2024-09-21T06:21:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.470+0000 7f19193fc740 1 Processor -- start 2024-09-21T06:21:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f19193fc740 1 -- start start 2024-09-21T06:21:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f19193fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 0x5629021f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f19193fc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e282f0 0x5629021f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f19193fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 0x5629021f7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f19193fc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56290216b6f0 con 0x562901e282f0 2024-09-21T06:21:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f19193fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56290215ab20 con 0x562901e304d0 2024-09-21T06:21:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f19193fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56290215e930 con 0x562901e511b0 2024-09-21T06:21:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f190bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e282f0 0x5629021f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f190bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e282f0 0x5629021f6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59646/0 (socket says 172.21.15.3:59646) 2024-09-21T06:21:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f190bfff640 1 -- 172.21.15.3:0/517654852 learned_addr learned my addr 172.21.15.3:0/517654852 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f1910a28640 1 --2- 172.21.15.3:0/517654852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 0x5629021f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f19097fa640 1 -- 172.21.15.3:0/1785440894 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f1900003aa0 con 0x7f190c1082e0 2024-09-21T06:21:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f190bfff640 1 -- 172.21.15.3:0/517654852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 msgr2=0x5629021f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f190bfff640 1 --2- 172.21.15.3:0/517654852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 0x5629021f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f190bfff640 1 -- 172.21.15.3:0/517654852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 msgr2=0x5629021f7430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:21:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f190bfff640 1 --2- 172.21.15.3:0/517654852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 0x5629021f7430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.471+0000 7f190bfff640 1 -- 172.21.15.3:0/517654852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562902120530 con 0x562901e282f0 2024-09-21T06:21:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.472+0000 7f190bfff640 1 --2- 172.21.15.3:0/517654852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e282f0 0x5629021f6ef0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0115f0 tx=0x7f18fc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625992d444089a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.472+0000 7f18f27fc640 1 -- 172.21.15.3:0/517654852 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc01e070 con 0x562901e282f0 2024-09-21T06:21:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.472+0000 7f18f27fc640 1 -- 172.21.15.3:0/517654852 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18fc00faf0 con 0x562901e282f0 2024-09-21T06:21:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.472+0000 7f18f27fc640 1 -- 172.21.15.3:0/517654852 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc019ac0 con 0x562901e282f0 2024-09-21T06:21:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.472+0000 7f19097fa640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18f40b6680 0x7f18f40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.472+0000 7f19097fa640 1 -- 172.21.15.3:0/1785440894 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f19000e7960 con 0x7f190c1082e0 2024-09-21T06:21:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.472+0000 7f1910a28640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18f40b6680 0x7f18f40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.473+0000 7f19193fc740 1 -- 172.21.15.3:0/517654852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e282f0 msgr2=0x5629021f6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.473+0000 7f19193fc740 1 --2- 172.21.15.3:0/517654852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e282f0 0x5629021f6ef0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0115f0 tx=0x7f18fc011ab0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.473+0000 7f19193fc740 1 -- 172.21.15.3:0/517654852 shutdown_connections 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.473+0000 7f19193fc740 1 --2- 172.21.15.3:0/517654852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 0x5629021f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.473+0000 7f19193fc740 1 --2- 172.21.15.3:0/517654852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e282f0 0x5629021f6ef0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.473+0000 7f19193fc740 1 --2- 172.21.15.3:0/517654852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 0x5629021f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.473+0000 7f19193fc740 1 -- 172.21.15.3:0/517654852 >> 172.21.15.3:0/517654852 conn(0x562901e45300 msgr2=0x5629021f48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.473+0000 7f1910a28640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18f40b6680 0x7f18f40b8b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1904006010 tx=0x7f1904032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.473+0000 7f19193fc740 1 -- 172.21.15.3:0/517654852 shutdown_connections 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.473+0000 7f19193fc740 1 -- 172.21.15.3:0/517654852 wait complete. 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.474+0000 7f19193fc740 1 -- start start 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.474+0000 7f19193fc740 1 Processor -- start 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.474+0000 7f19193fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e304d0 0x562902202430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.474+0000 7f19193fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e511b0 0x562902202970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.474+0000 7f19193fc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x56290220e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.474+0000 7f19193fc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562902092ab0 con 0x562901e0fd80 2024-09-21T06:21:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.474+0000 7f19193fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562902100730 con 0x562901e511b0 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.474+0000 7f19193fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562902205130 con 0x562901e304d0 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.474+0000 7f19193fc740 10 client.0 _ll_register_callbacks cb 0x5629021546b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.474+0000 7f190b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e511b0 0x562902202970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.474+0000 7f190bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e304d0 0x562902202430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.474+0000 7f1910a28640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x56290220e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f190b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e511b0 0x562902202970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:50866/0 (socket says 172.21.15.3:50866) 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f190b7fe640 1 -- 172.21.15.3:0/3693825636 learned_addr learned my addr 172.21.15.3:0/3693825636 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f190bfff640 1 --2- 172.21.15.3:0/3693825636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e304d0 0x562902202430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:42008/0 (socket says 172.21.15.3:42008) 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f190b7fe640 1 -- 172.21.15.3:0/3693825636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e304d0 msgr2=0x562902202430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f190b7fe640 1 --2- 172.21.15.3:0/3693825636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e304d0 0x562902202430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f190b7fe640 1 -- 172.21.15.3:0/3693825636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 msgr2=0x56290220e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f190b7fe640 1 --2- 172.21.15.3:0/3693825636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x56290220e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f190b7fe640 1 -- 172.21.15.3:0/3693825636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562901ebc790 con 0x562901e511b0 2024-09-21T06:21:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f190bfff640 1 --2- 172.21.15.3:0/3693825636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e304d0 0x562902202430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f190b7fe640 1 --2- 172.21.15.3:0/3693825636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e511b0 0x562902202970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f19000e5150 tx=0x7f19000ad6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0e570008ab7996 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f18eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f18eb7fe640 1 -- 172.21.15.3:0/3693825636 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19000d2020 con 0x562901e511b0 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f18eb7fe640 1 -- 172.21.15.3:0/3693825636 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f19000aeb20 con 0x562901e511b0 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.475+0000 7f19193fc740 1 -- 172.21.15.3:0/3693825636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18fc00faf0 con 0x562901e511b0 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.476+0000 7f18eb7fe640 1 -- 172.21.15.3:0/3693825636 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f19000b2070 con 0x562901e511b0 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.476+0000 7f19193fc740 10 client.26984 fetch_fsmap learned FSMap version 33 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.476+0000 7f19193fc740 1 -- 172.21.15.3:0/3693825636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562901f35620 con 0x562901e511b0 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.476+0000 7f18eb7fe640 1 -- 172.21.15.3:0/3693825636 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18fc00faf0 con 0x562901e511b0 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.476+0000 7f18eb7fe640 1 -- 172.21.15.3:0/3693825636 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f19000ac4c0 con 0x562901e511b0 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.476+0000 7f19193fc740 10 client.26984 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.476+0000 7f19193fc740 10 client.26984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.476+0000 7f19193fc740 20 client.26984 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.476+0000 7f19193fc740 1 --2- 172.21.15.3:0/3693825636 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562901de51c0 0x562902214350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.476+0000 7f19193fc740 4 client.26984 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.476+0000 7f19193fc740 1 -- 172.21.15.3:0/3693825636 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56290216d640 con 0x562901de51c0 2024-09-21T06:21:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.476+0000 7f190bfff640 1 --2- 172.21.15.3:0/3693825636 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562901de51c0 0x562902214350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.477+0000 7f18eb7fe640 10 client.26984 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.477+0000 7f190bfff640 1 --2- 172.21.15.3:0/3693825636 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562901de51c0 0x562902214350 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19953cff435abc40 server_cookie=bbe67f0443a2d2bb in_seq=0 out_seq=0 2024-09-21T06:21:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f18eb7fe640 1 -- 172.21.15.3:0/3693825636 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56290216d640 con 0x562901de51c0 2024-09-21T06:21:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f18eb7fe640 10 client.26984 handle_command_reply: tid=0 2024-09-21T06:21:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f19193fc740 1 client.26984 shutdown 2024-09-21T06:21:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f19193fc740 1 -- 172.21.15.3:0/3693825636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e511b0 msgr2=0x562902202970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f19193fc740 1 --2- 172.21.15.3:0/3693825636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e511b0 0x562902202970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f19000e5150 tx=0x7f19000ad6f0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f19193fc740 1 -- 172.21.15.3:0/3693825636 shutdown_connections 2024-09-21T06:21:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f19193fc740 1 --2- 172.21.15.3:0/3693825636 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562901de51c0 0x562902214350 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f19193fc740 1 --2- 172.21.15.3:0/3693825636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x56290220e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f19193fc740 1 --2- 172.21.15.3:0/3693825636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e511b0 0x562902202970 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f19193fc740 1 --2- 172.21.15.3:0/3693825636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e304d0 0x562902202430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f19193fc740 1 -- 172.21.15.3:0/3693825636 >> 172.21.15.3:0/3693825636 conn(0x562901e45300 msgr2=0x5629021f79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f18eb7fe640 0 client.26984 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f19193fc740 1 -- 172.21.15.3:0/3693825636 shutdown_connections 2024-09-21T06:21:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f19193fc740 1 -- 172.21.15.3:0/3693825636 wait complete. 2024-09-21T06:21:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.478+0000 7f19193fc740 20 client.26984 trim_cache size 0 max 16384 2024-09-21T06:21:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.496+0000 7f19193fc740 1 Processor -- start 2024-09-21T06:21:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.496+0000 7f19193fc740 1 -- start start 2024-09-21T06:21:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.496+0000 7f19193fc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x562902203e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.496+0000 7f19193fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 0x56290220e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.496+0000 7f19193fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 0x56290220e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.496+0000 7f19193fc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1b17f0 con 0x562901e0fd80 2024-09-21T06:21:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.496+0000 7f19193fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1b3330 con 0x562901e304d0 2024-09-21T06:21:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.496+0000 7f19193fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1b1ec0 con 0x562901e511b0 2024-09-21T06:21:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.496+0000 7f190bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x562902203e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.496+0000 7f190b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 0x56290220e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.496+0000 7f190bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x562902203e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59662/0 (socket says 172.21.15.3:59662) 2024-09-21T06:21:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.496+0000 7f190bfff640 1 -- 172.21.15.3:0/3768382110 learned_addr learned my addr 172.21.15.3:0/3768382110 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.496+0000 7f1910a28640 1 --2- 172.21.15.3:0/3768382110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 0x56290220e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.497+0000 7f190b7fe640 1 -- 172.21.15.3:0/3768382110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 msgr2=0x56290220e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.497+0000 7f190b7fe640 1 --2- 172.21.15.3:0/3768382110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 0x56290220e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.497+0000 7f190b7fe640 1 -- 172.21.15.3:0/3768382110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 msgr2=0x562902203e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.497+0000 7f190b7fe640 1 --2- 172.21.15.3:0/3768382110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x562902203e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.497+0000 7f190b7fe640 1 -- 172.21.15.3:0/3768382110 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629021f8ee0 con 0x562901e304d0 2024-09-21T06:21:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.497+0000 7f190bfff640 1 --2- 172.21.15.3:0/3768382110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x562902203e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.497+0000 7f190b7fe640 1 --2- 172.21.15.3:0/3768382110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 0x56290220e350 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f19000ad780 tx=0x7f1900000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d0755f4451b6604 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.497+0000 7f18f1ffb640 1 -- 172.21.15.3:0/3768382110 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19000d2070 con 0x562901e304d0 2024-09-21T06:21:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.497+0000 7f18f1ffb640 1 -- 172.21.15.3:0/3768382110 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19000adef0 con 0x562901e304d0 2024-09-21T06:21:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.497+0000 7f18f1ffb640 1 -- 172.21.15.3:0/3768382110 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19000a9140 con 0x562901e304d0 2024-09-21T06:21:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.498+0000 7f19193fc740 1 -- 172.21.15.3:0/3768382110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 msgr2=0x56290220e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.498+0000 7f19193fc740 1 --2- 172.21.15.3:0/3768382110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 0x56290220e350 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f19000ad780 tx=0x7f1900000bc0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.498+0000 7f19193fc740 1 -- 172.21.15.3:0/3768382110 shutdown_connections 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.498+0000 7f19193fc740 1 --2- 172.21.15.3:0/3768382110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 0x56290220e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.498+0000 7f19193fc740 1 --2- 172.21.15.3:0/3768382110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 0x56290220e350 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.498+0000 7f19193fc740 1 --2- 172.21.15.3:0/3768382110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x562902203e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.498+0000 7f19193fc740 1 -- 172.21.15.3:0/3768382110 >> 172.21.15.3:0/3768382110 conn(0x562901e45300 msgr2=0x562902203740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.498+0000 7f19193fc740 1 -- 172.21.15.3:0/3768382110 shutdown_connections 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.498+0000 7f19193fc740 1 -- 172.21.15.3:0/3768382110 wait complete. 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.498+0000 7f19193fc740 1 -- start start 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f19193fc740 1 Processor -- start 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f19193fc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x56290220a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f19193fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 0x56290220ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f19193fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 0x562902205e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f19193fc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1b2c60 con 0x562901e0fd80 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f19193fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1b2590 con 0x562901e304d0 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f19193fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1af7d0 con 0x562901e511b0 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f19193fc740 10 client.0 _ll_register_callbacks cb 0x7f19000a9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f190bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x56290220a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f190b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 0x56290220ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f1910a28640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 0x562902205e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f190bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x56290220a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59678/0 (socket says 172.21.15.3:59678) 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f190bfff640 1 -- 172.21.15.3:0/2295425094 learned_addr learned my addr 172.21.15.3:0/2295425094 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f190bfff640 1 -- 172.21.15.3:0/2295425094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 msgr2=0x562902205e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f190bfff640 1 --2- 172.21.15.3:0/2295425094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 0x562902205e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f190bfff640 1 -- 172.21.15.3:0/2295425094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 msgr2=0x56290220ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f190bfff640 1 --2- 172.21.15.3:0/2295425094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 0x56290220ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.499+0000 7f190bfff640 1 -- 172.21.15.3:0/2295425094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562902206620 con 0x562901e0fd80 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f190bfff640 1 --2- 172.21.15.3:0/2295425094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x56290220a820 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x5629021f5630 tx=0x7f18fc0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a2d6bfa2f66492 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f18eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f18eb7fe640 1 -- 172.21.15.3:0/2295425094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc031060 con 0x562901e0fd80 2024-09-21T06:21:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f18eb7fe640 1 -- 172.21.15.3:0/2295425094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f18fc00f740 con 0x562901e0fd80 2024-09-21T06:21:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f19193fc740 1 -- 172.21.15.3:0/2295425094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19000adef0 con 0x562901e0fd80 2024-09-21T06:21:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f18eb7fe640 1 -- 172.21.15.3:0/2295425094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f18fc01e040 con 0x562901e0fd80 2024-09-21T06:21:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f19193fc740 10 client.17424 fetch_fsmap learned FSMap version 33 2024-09-21T06:21:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f19193fc740 1 -- 172.21.15.3:0/2295425094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56290227e8d0 con 0x562901e0fd80 2024-09-21T06:21:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f18eb7fe640 1 -- 172.21.15.3:0/2295425094 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19000adef0 con 0x562901e0fd80 2024-09-21T06:21:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f18eb7fe640 1 -- 172.21.15.3:0/2295425094 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f18fc019780 con 0x562901e0fd80 2024-09-21T06:21:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f19193fc740 10 client.17424 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:21:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f19193fc740 10 client.17424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f19193fc740 20 client.17424 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f19193fc740 1 --2- 172.21.15.3:0/2295425094 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56290227ee90 0x5629022812b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f19193fc740 4 client.17424 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:21:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.500+0000 7f19193fc740 1 -- 172.21.15.3:0/2295425094 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629021690f0 con 0x56290227ee90 2024-09-21T06:21:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.501+0000 7f190b7fe640 1 --2- 172.21.15.3:0/2295425094 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56290227ee90 0x5629022812b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.501+0000 7f190b7fe640 1 --2- 172.21.15.3:0/2295425094 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56290227ee90 0x5629022812b0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=869733804711b899 server_cookie=f0c964baa8c65688 in_seq=0 out_seq=0 2024-09-21T06:21:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.501+0000 7f18eb7fe640 10 client.17424 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.632+0000 7f18eb7fe640 1 -- 172.21.15.3:0/2295425094 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5629021690f0 con 0x56290227ee90 2024-09-21T06:21:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.632+0000 7f18eb7fe640 10 client.17424 handle_command_reply: tid=0 2024-09-21T06:21:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.632+0000 7f19193fc740 1 client.17424 shutdown 2024-09-21T06:21:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.633+0000 7f19193fc740 1 -- 172.21.15.3:0/2295425094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 msgr2=0x56290220a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.633+0000 7f19193fc740 1 --2- 172.21.15.3:0/2295425094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x56290220a820 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x5629021f5630 tx=0x7f18fc0196a0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.633+0000 7f19193fc740 1 -- 172.21.15.3:0/2295425094 shutdown_connections 2024-09-21T06:21:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.633+0000 7f19193fc740 1 --2- 172.21.15.3:0/2295425094 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x56290227ee90 0x5629022812b0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.633+0000 7f19193fc740 1 --2- 172.21.15.3:0/2295425094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562901e511b0 0x562902205e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.633+0000 7f19193fc740 1 --2- 172.21.15.3:0/2295425094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562901e304d0 0x56290220ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.633+0000 7f18eb7fe640 0 client.17424 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.633+0000 7f19193fc740 1 --2- 172.21.15.3:0/2295425094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562901e0fd80 0x56290220a820 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.633+0000 7f19193fc740 1 -- 172.21.15.3:0/2295425094 >> 172.21.15.3:0/2295425094 conn(0x562901e45300 msgr2=0x562902203e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.633+0000 7f19193fc740 1 -- 172.21.15.3:0/2295425094 shutdown_connections 2024-09-21T06:21:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.633+0000 7f19193fc740 1 -- 172.21.15.3:0/2295425094 wait complete. 2024-09-21T06:21:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.633+0000 7f19193fc740 20 client.17424 trim_cache size 0 max 16384 2024-09-21T06:21:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:21:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:21:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:21:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:21:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.634+0000 7f191248b640 1 -- 172.21.15.3:0/1785440894 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18f40b6680 msgr2=0x7f18f40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.634+0000 7f191248b640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18f40b6680 0x7f18f40b8b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1904006010 tx=0x7f1904032ed0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.634+0000 7f191248b640 1 -- 172.21.15.3:0/1785440894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f190c1082e0 msgr2=0x7f190c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.634+0000 7f191248b640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f190c1082e0 0x7f190c19cb50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f190000e970 tx=0x7f190000ee30 comp rx=0 tx=0).stop 2024-09-21T06:21:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.634+0000 7f191248b640 1 -- 172.21.15.3:0/1785440894 shutdown_connections 2024-09-21T06:21:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.634+0000 7f191248b640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18f40b6680 0x7f18f40b8b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.634+0000 7f191248b640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f190c108c30 0x7f190c1a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.634+0000 7f191248b640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f190c1082e0 0x7f190c19cb50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.634+0000 7f191248b640 1 --2- 172.21.15.3:0/1785440894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f190c1029b0 0x7f190c19c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.634+0000 7f191248b640 1 -- 172.21.15.3:0/1785440894 >> 172.21.15.3:0/1785440894 conn(0x7f190c07a740 msgr2=0x7f190c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.634+0000 7f191248b640 1 -- 172.21.15.3:0/1785440894 shutdown_connections 2024-09-21T06:21:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:43.634+0000 7f191248b640 1 -- 172.21.15.3:0/1785440894 wait complete. 2024-09-21T06:21:43.696 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:43 smithi137 ceph-mon[34464]: pgmap v526: 97 pgs: 97 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-09-21T06:21:43.705 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:43 smithi204 ceph-mon[34659]: pgmap v526: 97 pgs: 97 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-09-21T06:21:43.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:43 smithi003 ceph-mon[31227]: pgmap v526: 97 pgs: 97 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-09-21T06:21:43.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 707560 11.43 MB/sec execute 196 sec latency 97.712 ms 2024-09-21T06:21:44.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 710223 11.46 MB/sec execute 197 sec latency 15.900 ms 2024-09-21T06:21:45.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:45 smithi003 ceph-mon[31227]: pgmap v527: 97 pgs: 97 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 295 op/s 2024-09-21T06:21:45.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:45 smithi137 ceph-mon[34464]: pgmap v527: 97 pgs: 97 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 295 op/s 2024-09-21T06:21:45.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:45 smithi204 ceph-mon[34659]: pgmap v527: 97 pgs: 97 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 295 op/s 2024-09-21T06:21:45.962 INFO:tasks.workunit.client.0.smithi003.stdout: 1 711648 11.43 MB/sec execute 198 sec latency 49.319 ms 2024-09-21T06:21:46.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 714005 11.44 MB/sec execute 199 sec latency 17.203 ms 2024-09-21T06:21:47.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:47 smithi003 ceph-mon[31227]: pgmap v528: 97 pgs: 97 active+clean; 121 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-09-21T06:21:47.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:47 smithi137 ceph-mon[34464]: pgmap v528: 97 pgs: 97 active+clean; 121 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-09-21T06:21:47.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:47 smithi204 ceph-mon[34659]: pgmap v528: 97 pgs: 97 active+clean; 121 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-09-21T06:21:47.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 715626 11.42 MB/sec execute 200 sec latency 19.493 ms 2024-09-21T06:21:48.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 717777 11.43 MB/sec execute 201 sec latency 12.117 ms 2024-09-21T06:21:49.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:49 smithi003 ceph-mon[31227]: pgmap v529: 97 pgs: 97 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 316 op/s 2024-09-21T06:21:49.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:49 smithi137 ceph-mon[34464]: pgmap v529: 97 pgs: 97 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 316 op/s 2024-09-21T06:21:49.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:49 smithi204 ceph-mon[34659]: pgmap v529: 97 pgs: 97 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 316 op/s 2024-09-21T06:21:49.934 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:21:49.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 720097 11.42 MB/sec execute 202 sec latency 18.539 ms 2024-09-21T06:21:50.205 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:50.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.743+0000 7f55ad3c3640 1 -- 172.21.15.3:0/1426340424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55a8103bc0 msgr2=0x7f55a8103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:50.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.743+0000 7f55ad3c3640 1 --2- 172.21.15.3:0/1426340424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55a8103bc0 0x7f55a8103fd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5598009900 tx=0x7f559802a410 comp rx=0 tx=0).stop 2024-09-21T06:21:50.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.743+0000 7f55ad3c3640 1 -- 172.21.15.3:0/1426340424 shutdown_connections 2024-09-21T06:21:50.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.743+0000 7f55ad3c3640 1 --2- 172.21.15.3:0/1426340424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55a81094b0 0x7f55a81098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.743+0000 7f55ad3c3640 1 --2- 172.21.15.3:0/1426340424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a8109e00 0x7f55a8110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.743+0000 7f55ad3c3640 1 --2- 172.21.15.3:0/1426340424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55a8103bc0 0x7f55a8103fd0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.743+0000 7f55ad3c3640 1 -- 172.21.15.3:0/1426340424 >> 172.21.15.3:0/1426340424 conn(0x7f55a807b910 msgr2=0x7f55a807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:50.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.743+0000 7f55ad3c3640 1 -- 172.21.15.3:0/1426340424 shutdown_connections 2024-09-21T06:21:50.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.743+0000 7f55ad3c3640 1 -- 172.21.15.3:0/1426340424 wait complete. 2024-09-21T06:21:50.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.744+0000 7f55ad3c3640 1 Processor -- start 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.744+0000 7f55ad3c3640 1 -- start start 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.744+0000 7f55ad3c3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55a8103bc0 0x7f55a81a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.744+0000 7f55ad3c3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55a81094b0 0x7f55a81a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.744+0000 7f55ad3c3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a8109e00 0x7f55a81ac820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.744+0000 7f55ad3c3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8113ac0 con 0x7f55a81094b0 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.744+0000 7f55ad3c3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8113940 con 0x7f55a8109e00 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.744+0000 7f55ad3c3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8113c40 con 0x7f55a8103bc0 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.744+0000 7f55a6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55a8103bc0 0x7f55a81a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.744+0000 7f55a6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55a8103bc0 0x7f55a81a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:51960/0 (socket says 172.21.15.3:51960) 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.744+0000 7f55a6ffd640 1 -- 172.21.15.3:0/1295070810 learned_addr learned my addr 172.21.15.3:0/1295070810 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.744+0000 7f55a77fe640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a8109e00 0x7f55a81ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.745+0000 7f559e7fc640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55a81094b0 0x7f55a81a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.745+0000 7f55a6ffd640 1 -- 172.21.15.3:0/1295070810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a8109e00 msgr2=0x7f55a81ac820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:50.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.745+0000 7f55a6ffd640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a8109e00 0x7f55a81ac820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.745+0000 7f55a6ffd640 1 -- 172.21.15.3:0/1295070810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55a81094b0 msgr2=0x7f55a81a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:50.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.745+0000 7f55a6ffd640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55a81094b0 0x7f55a81a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.745+0000 7f55a6ffd640 1 -- 172.21.15.3:0/1295070810 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a81a2580 con 0x7f55a8103bc0 2024-09-21T06:21:50.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.745+0000 7f55a77fe640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a8109e00 0x7f55a81ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:50.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.745+0000 7f55a6ffd640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55a8103bc0 0x7f55a81a17a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5598005940 tx=0x7f55980040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:50.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.746+0000 7f55a4ff9640 1 -- 172.21.15.3:0/1295070810 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559803f030 con 0x7f55a8103bc0 2024-09-21T06:21:50.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.746+0000 7f55a4ff9640 1 -- 172.21.15.3:0/1295070810 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55980044c0 con 0x7f55a8103bc0 2024-09-21T06:21:50.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.746+0000 7f55a4ff9640 1 -- 172.21.15.3:0/1295070810 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598035070 con 0x7f55a8103bc0 2024-09-21T06:21:50.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.746+0000 7f55ad3c3640 1 -- 172.21.15.3:0/1295070810 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a81acdc0 con 0x7f55a8103bc0 2024-09-21T06:21:50.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.746+0000 7f55ad3c3640 1 -- 172.21.15.3:0/1295070810 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a81ad1f0 con 0x7f55a8103bc0 2024-09-21T06:21:50.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.748+0000 7f55b3937740 1 Processor -- start 2024-09-21T06:21:50.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.749+0000 7f55b3937740 1 -- start start 2024-09-21T06:21:50.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.749+0000 7f55b3937740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86325070 0x55dc866cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.749+0000 7f55b3937740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.749+0000 7f55b3937740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86304390 0x55dc866cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.749+0000 7f55b3937740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dc86642e50 con 0x55dc86325070 2024-09-21T06:21:50.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.749+0000 7f55b3937740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dc86632280 con 0x55dc86304390 2024-09-21T06:21:50.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.749+0000 7f55b3937740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dc86636090 con 0x55dc862fc1b0 2024-09-21T06:21:50.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.749+0000 7f559e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86304390 0x55dc866cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.749+0000 7f559e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86304390 0x55dc866cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:50998/0 (socket says 172.21.15.3:50998) 2024-09-21T06:21:50.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.750+0000 7f559e7fc640 1 -- 172.21.15.3:0/3980239020 learned_addr learned my addr 172.21.15.3:0/3980239020 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.750+0000 7f55a4ff9640 1 -- 172.21.15.3:0/1295070810 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f5598042030 con 0x7f55a8103bc0 2024-09-21T06:21:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.750+0000 7f55a77fe640 1 --2- 172.21.15.3:0/3980239020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86325070 0x55dc866cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.750+0000 7f55a6ffd640 1 --2- 172.21.15.3:0/3980239020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.750+0000 7f559e7fc640 1 -- 172.21.15.3:0/3980239020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 msgr2=0x55dc866ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.750+0000 7f559e7fc640 1 --2- 172.21.15.3:0/3980239020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.750+0000 7f559e7fc640 1 -- 172.21.15.3:0/3980239020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86325070 msgr2=0x55dc866cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.750+0000 7f559e7fc640 1 --2- 172.21.15.3:0/3980239020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86325070 0x55dc866cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.750+0000 7f559e7fc640 1 -- 172.21.15.3:0/3980239020 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc86390650 con 0x55dc86304390 2024-09-21T06:21:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.750+0000 7f55a77fe640 1 --2- 172.21.15.3:0/3980239020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86325070 0x55dc866cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:50.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.750+0000 7f559e7fc640 1 --2- 172.21.15.3:0/3980239020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86304390 0x55dc866cd0d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5590009bb0 tx=0x7f55900117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80290dca0deff1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:50.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.750+0000 7f559d7fa640 1 -- 172.21.15.3:0/3980239020 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590005020 con 0x55dc86304390 2024-09-21T06:21:50.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.751+0000 7f559d7fa640 1 -- 172.21.15.3:0/3980239020 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55900047a0 con 0x55dc86304390 2024-09-21T06:21:50.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.751+0000 7f559d7fa640 1 -- 172.21.15.3:0/3980239020 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590012670 con 0x55dc86304390 2024-09-21T06:21:50.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.751+0000 7f55a4ff9640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55940b6680 0x7f55940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.751+0000 7f55a77fe640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55940b6680 0x7f55940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.751+0000 7f55a4ff9640 1 -- 172.21.15.3:0/1295070810 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f5598107b40 con 0x7f55a8103bc0 2024-09-21T06:21:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.752+0000 7f55b3937740 1 -- 172.21.15.3:0/3980239020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86304390 msgr2=0x55dc866cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.752+0000 7f55b3937740 1 --2- 172.21.15.3:0/3980239020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86304390 0x55dc866cd0d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5590009bb0 tx=0x7f55900117b0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.752+0000 7f55b3937740 1 -- 172.21.15.3:0/3980239020 shutdown_connections 2024-09-21T06:21:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.752+0000 7f55b3937740 1 --2- 172.21.15.3:0/3980239020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.752+0000 7f55b3937740 1 --2- 172.21.15.3:0/3980239020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86304390 0x55dc866cd0d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.752+0000 7f55b3937740 1 --2- 172.21.15.3:0/3980239020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86325070 0x55dc866cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.752+0000 7f55b3937740 1 -- 172.21.15.3:0/3980239020 >> 172.21.15.3:0/3980239020 conn(0x55dc86319460 msgr2=0x55dc866ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.752+0000 7f55a77fe640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55940b6680 0x7f55940b8b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f55a0005fd0 tx=0x7f55a0009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.752+0000 7f55b3937740 1 -- 172.21.15.3:0/3980239020 shutdown_connections 2024-09-21T06:21:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.752+0000 7f55b3937740 1 -- 172.21.15.3:0/3980239020 wait complete. 2024-09-21T06:21:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.752+0000 7f55b3937740 1 -- start start 2024-09-21T06:21:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.752+0000 7f55b3937740 1 Processor -- start 2024-09-21T06:21:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55b3937740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866d7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55b3937740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86304390 0x55dc866d81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55b3937740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86325070 0x55dc866e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55b3937740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dc86493700 con 0x55dc86304390 2024-09-21T06:21:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55b3937740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dc865cd110 con 0x55dc86325070 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55b3937740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dc866dae30 con 0x55dc862fc1b0 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f559e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86304390 0x55dc866d81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55b3937740 10 client.0 _ll_register_callbacks cb 0x55dc865e6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55a77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86325070 0x55dc866e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55a77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86325070 0x55dc866e4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:51002/0 (socket says 172.21.15.3:51002) 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55a77fe640 1 -- 172.21.15.3:0/210285922 learned_addr learned my addr 172.21.15.3:0/210285922 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55a6ffd640 1 --2- 172.21.15.3:0/210285922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866d7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55a6ffd640 1 -- 172.21.15.3:0/210285922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86325070 msgr2=0x55dc866e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55a6ffd640 1 --2- 172.21.15.3:0/210285922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86325070 0x55dc866e4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55a6ffd640 1 -- 172.21.15.3:0/210285922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86304390 msgr2=0x55dc866d81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55a6ffd640 1 --2- 172.21.15.3:0/210285922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86304390 0x55dc866d81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.753+0000 7f55a6ffd640 1 -- 172.21.15.3:0/210285922 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc865fac40 con 0x55dc862fc1b0 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f55a6ffd640 1 --2- 172.21.15.3:0/210285922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866d7cb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5598105270 tx=0x7f5598009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cb207cb2576322 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f558e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:21:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f558e7fc640 1 -- 172.21.15.3:0/210285922 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980cd2c0 con 0x55dc862fc1b0 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f558e7fc640 1 -- 172.21.15.3:0/210285922 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f55980ccbd0 con 0x55dc862fc1b0 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f55b3937740 1 -- 172.21.15.3:0/210285922 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55900047a0 con 0x55dc862fc1b0 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f558e7fc640 1 -- 172.21.15.3:0/210285922 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f55980c90e0 con 0x55dc862fc1b0 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f55b3937740 10 client.26950 fetch_fsmap learned FSMap version 33 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f55b3937740 1 -- 172.21.15.3:0/210285922 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dc866e6ae0 con 0x55dc862fc1b0 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f558e7fc640 1 -- 172.21.15.3:0/210285922 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55980cd480 con 0x55dc862fc1b0 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f558e7fc640 1 -- 172.21.15.3:0/210285922 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f55980cc640 con 0x55dc862fc1b0 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f55b3937740 10 client.26950 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f55b3937740 10 client.26950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f55b3937740 20 client.26950 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f55b3937740 1 --2- 172.21.15.3:0/210285922 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dc862e39f0 0x55dc866ea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f55b3937740 4 client.26950 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.754+0000 7f55b3937740 1 -- 172.21.15.3:0/210285922 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f55900047a0 con 0x55dc862e39f0 2024-09-21T06:21:50.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.755+0000 7f559e7fc640 1 --2- 172.21.15.3:0/210285922 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dc862e39f0 0x55dc866ea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.755+0000 7f558e7fc640 10 client.26950 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:50.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.755+0000 7f559e7fc640 1 --2- 172.21.15.3:0/210285922 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dc862e39f0 0x55dc866ea0a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=444c5c25d13b1ffe server_cookie=e597f75f2186dc1a in_seq=0 out_seq=0 2024-09-21T06:21:50.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f558e7fc640 1 -- 172.21.15.3:0/210285922 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f55900047a0 con 0x55dc862e39f0 2024-09-21T06:21:50.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f558e7fc640 10 client.26950 handle_command_reply: tid=0 2024-09-21T06:21:50.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f55b3937740 1 client.26950 shutdown 2024-09-21T06:21:50.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f55b3937740 1 -- 172.21.15.3:0/210285922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 msgr2=0x55dc866d7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:50.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f55b3937740 1 --2- 172.21.15.3:0/210285922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866d7cb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5598105270 tx=0x7f5598009640 comp rx=0 tx=0).stop 2024-09-21T06:21:50.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f55b3937740 1 -- 172.21.15.3:0/210285922 shutdown_connections 2024-09-21T06:21:50.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f55b3937740 1 --2- 172.21.15.3:0/210285922 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dc862e39f0 0x55dc866ea0a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f55b3937740 1 --2- 172.21.15.3:0/210285922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86325070 0x55dc866e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f55b3937740 1 --2- 172.21.15.3:0/210285922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86304390 0x55dc866d81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f558e7fc640 0 client.26950 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:50.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f55b3937740 1 --2- 172.21.15.3:0/210285922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866d7cb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f55b3937740 1 -- 172.21.15.3:0/210285922 >> 172.21.15.3:0/210285922 conn(0x55dc86319460 msgr2=0x55dc866cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:50.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f55b3937740 1 -- 172.21.15.3:0/210285922 shutdown_connections 2024-09-21T06:21:50.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f55b3937740 1 -- 172.21.15.3:0/210285922 wait complete. 2024-09-21T06:21:50.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.756+0000 7f55b3937740 20 client.26950 trim_cache size 0 max 16384 2024-09-21T06:21:50.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.775+0000 7f55b3937740 1 Processor -- start 2024-09-21T06:21:50.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.775+0000 7f55b3937740 1 -- start start 2024-09-21T06:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.776+0000 7f55b3937740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866d8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.776+0000 7f55b3937740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86304390 0x55dc866cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.776+0000 7f55b3937740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86325070 0x55dc866cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.776+0000 7f55b3937740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81b5bc0 con 0x55dc86304390 2024-09-21T06:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.776+0000 7f55b3937740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81b7700 con 0x55dc86325070 2024-09-21T06:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.776+0000 7f55b3937740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81b6290 con 0x55dc862fc1b0 2024-09-21T06:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.776+0000 7f559e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86304390 0x55dc866cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.776+0000 7f559e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86304390 0x55dc866cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53736/0 (socket says 172.21.15.3:53736) 2024-09-21T06:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.776+0000 7f559e7fc640 1 -- 172.21.15.3:0/2866201294 learned_addr learned my addr 172.21.15.3:0/2866201294 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.776+0000 7f55a77fe640 1 --2- 172.21.15.3:0/2866201294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86325070 0x55dc866cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.776+0000 7f55a6ffd640 1 --2- 172.21.15.3:0/2866201294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866d8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.777+0000 7f559e7fc640 1 -- 172.21.15.3:0/2866201294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 msgr2=0x55dc866d8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.777+0000 7f559e7fc640 1 --2- 172.21.15.3:0/2866201294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866d8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.777+0000 7f559e7fc640 1 -- 172.21.15.3:0/2866201294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86325070 msgr2=0x55dc866cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:50.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.777+0000 7f559e7fc640 1 --2- 172.21.15.3:0/2866201294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86325070 0x55dc866cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.777+0000 7f559e7fc640 1 -- 172.21.15.3:0/2866201294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc866cae30 con 0x55dc86304390 2024-09-21T06:21:50.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.777+0000 7f55a6ffd640 1 --2- 172.21.15.3:0/2866201294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866d8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:50.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.777+0000 7f55a77fe640 1 --2- 172.21.15.3:0/2866201294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86325070 0x55dc866cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:50.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.777+0000 7f559e7fc640 1 --2- 172.21.15.3:0/2866201294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86304390 0x55dc866cf750 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5590009780 tx=0x7f55900117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c02fe54c103725e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.777+0000 7f559cff9640 1 -- 172.21.15.3:0/2866201294 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559001f040 con 0x55dc86304390 2024-09-21T06:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.777+0000 7f559cff9640 1 -- 172.21.15.3:0/2866201294 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55900035f0 con 0x55dc86304390 2024-09-21T06:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.777+0000 7f559cff9640 1 -- 172.21.15.3:0/2866201294 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590026670 con 0x55dc86304390 2024-09-21T06:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.778+0000 7f55b3937740 1 -- 172.21.15.3:0/2866201294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86304390 msgr2=0x55dc866cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.778+0000 7f55b3937740 1 --2- 172.21.15.3:0/2866201294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86304390 0x55dc866cf750 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5590009780 tx=0x7f55900117b0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.778+0000 7f55b3937740 1 -- 172.21.15.3:0/2866201294 shutdown_connections 2024-09-21T06:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.778+0000 7f55b3937740 1 --2- 172.21.15.3:0/2866201294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86325070 0x55dc866cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.778+0000 7f55b3937740 1 --2- 172.21.15.3:0/2866201294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86304390 0x55dc866cf750 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.778+0000 7f55b3937740 1 --2- 172.21.15.3:0/2866201294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866d8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.778+0000 7f55b3937740 1 -- 172.21.15.3:0/2866201294 >> 172.21.15.3:0/2866201294 conn(0x55dc86319460 msgr2=0x55dc866d7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:50.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.779+0000 7f55b3937740 1 -- 172.21.15.3:0/2866201294 shutdown_connections 2024-09-21T06:21:50.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.779+0000 7f55b3937740 1 -- 172.21.15.3:0/2866201294 wait complete. 2024-09-21T06:21:50.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.779+0000 7f55b3937740 1 -- start start 2024-09-21T06:21:50.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.779+0000 7f55b3937740 1 Processor -- start 2024-09-21T06:21:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55b3937740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866dc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55b3937740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86304390 0x55dc866dc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55b3937740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86325070 0x55dc86753a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55b3937740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81b7030 con 0x55dc86325070 2024-09-21T06:21:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55b3937740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81b6960 con 0x55dc86304390 2024-09-21T06:21:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55b3937740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81b3ba0 con 0x55dc862fc1b0 2024-09-21T06:21:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55b3937740 10 client.0 _ll_register_callbacks cb 0x7f559803b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55a77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86325070 0x55dc86753a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55a6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866dc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55a77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86325070 0x55dc86753a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53740/0 (socket says 172.21.15.3:53740) 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55a77fe640 1 -- 172.21.15.3:0/4164945540 learned_addr learned my addr 172.21.15.3:0/4164945540 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f559e7fc640 1 --2- 172.21.15.3:0/4164945540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86304390 0x55dc866dc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55a6ffd640 1 -- 172.21.15.3:0/4164945540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86304390 msgr2=0x55dc866dc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55a6ffd640 1 --2- 172.21.15.3:0/4164945540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86304390 0x55dc866dc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55a6ffd640 1 -- 172.21.15.3:0/4164945540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86325070 msgr2=0x55dc86753a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55a6ffd640 1 --2- 172.21.15.3:0/4164945540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86325070 0x55dc86753a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f55a6ffd640 1 -- 172.21.15.3:0/4164945540 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc866dd1e0 con 0x55dc862fc1b0 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.780+0000 7f559e7fc640 1 --2- 172.21.15.3:0/4164945540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86304390 0x55dc866dc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f55a6ffd640 1 --2- 172.21.15.3:0/4164945540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866dc3b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f55980cd670 tx=0x7f55980c9e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef12bc0431a42c85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f558e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f558e7fc640 1 -- 172.21.15.3:0/4164945540 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980cd0e0 con 0x55dc862fc1b0 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f558e7fc640 1 -- 172.21.15.3:0/4164945540 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f55980ccbd0 con 0x55dc862fc1b0 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f55b3937740 1 -- 172.21.15.3:0/4164945540 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55900035f0 con 0x55dc862fc1b0 2024-09-21T06:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f558e7fc640 1 -- 172.21.15.3:0/4164945540 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f55980cd480 con 0x55dc862fc1b0 2024-09-21T06:21:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f55b3937740 10 client.26956 fetch_fsmap learned FSMap version 33 2024-09-21T06:21:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f55b3937740 1 -- 172.21.15.3:0/4164945540 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dc86755060 con 0x55dc862fc1b0 2024-09-21T06:21:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f558e7fc640 1 -- 172.21.15.3:0/4164945540 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55900035f0 con 0x55dc862fc1b0 2024-09-21T06:21:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f558e7fc640 1 -- 172.21.15.3:0/4164945540 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f55980f2070 con 0x55dc862fc1b0 2024-09-21T06:21:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f55b3937740 10 client.26956 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:21:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f55b3937740 10 client.26956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f55b3937740 20 client.26956 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f55b3937740 1 --2- 172.21.15.3:0/4164945540 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dc862b8da0 0x55dc86757730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.781+0000 7f55b3937740 4 client.26956 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:21:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.782+0000 7f55b3937740 1 -- 172.21.15.3:0/4164945540 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc86644da0 con 0x55dc862b8da0 2024-09-21T06:21:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.782+0000 7f559e7fc640 1 --2- 172.21.15.3:0/4164945540 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dc862b8da0 0x55dc86757730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.782+0000 7f558e7fc640 10 client.26956 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.782+0000 7f559e7fc640 1 --2- 172.21.15.3:0/4164945540 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dc862b8da0 0x55dc86757730 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f459e0029fa4f60 server_cookie=2e5b29090c5a2a73 in_seq=0 out_seq=0 2024-09-21T06:21:50.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f558e7fc640 1 -- 172.21.15.3:0/4164945540 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x55dc86644da0 con 0x55dc862b8da0 2024-09-21T06:21:50.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f558e7fc640 10 client.26956 handle_command_reply: tid=0 2024-09-21T06:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f55b3937740 1 client.26956 shutdown 2024-09-21T06:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f55b3937740 1 -- 172.21.15.3:0/4164945540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 msgr2=0x55dc866dc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f55b3937740 1 --2- 172.21.15.3:0/4164945540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866dc3b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f55980cd670 tx=0x7f55980c9e80 comp rx=0 tx=0).stop 2024-09-21T06:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f55b3937740 1 -- 172.21.15.3:0/4164945540 shutdown_connections 2024-09-21T06:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f55b3937740 1 --2- 172.21.15.3:0/4164945540 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dc862b8da0 0x55dc86757730 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f55b3937740 1 --2- 172.21.15.3:0/4164945540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dc86325070 0x55dc86753a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f55b3937740 1 --2- 172.21.15.3:0/4164945540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc86304390 0x55dc866dc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f55b3937740 1 --2- 172.21.15.3:0/4164945540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc862fc1b0 0x55dc866dc3b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f55b3937740 1 -- 172.21.15.3:0/4164945540 >> 172.21.15.3:0/4164945540 conn(0x55dc86319460 msgr2=0x55dc866d6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f558e7fc640 0 client.26956 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f55b3937740 1 -- 172.21.15.3:0/4164945540 shutdown_connections 2024-09-21T06:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f55b3937740 1 -- 172.21.15.3:0/4164945540 wait complete. 2024-09-21T06:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.783+0000 7f55b3937740 20 client.26956 trim_cache size 0 max 16384 2024-09-21T06:21:50.789 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:21:50.789 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:21:50.789 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:21:50.789 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:21:50.789 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:21:50.789 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:21:50.789 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:50.789 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:50.789 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:50.789 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:50.789 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:50.789 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:50.790 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:50.791 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58": { 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:21:17.396219+0000", 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:50.792 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:21:50.793 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:21:50.794 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:21:50.795 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:21:50.795 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:21:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.784+0000 7f55ad3c3640 1 -- 172.21.15.3:0/1295070810 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55940b6680 msgr2=0x7f55940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.784+0000 7f55ad3c3640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55940b6680 0x7f55940b8b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f55a0005fd0 tx=0x7f55a0009500 comp rx=0 tx=0).stop 2024-09-21T06:21:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.784+0000 7f55ad3c3640 1 -- 172.21.15.3:0/1295070810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55a8103bc0 msgr2=0x7f55a81a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.784+0000 7f55ad3c3640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55a8103bc0 0x7f55a81a17a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5598005940 tx=0x7f55980040c0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.784+0000 7f55ad3c3640 1 -- 172.21.15.3:0/1295070810 shutdown_connections 2024-09-21T06:21:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.784+0000 7f55ad3c3640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f55940b6680 0x7f55940b8b90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.784+0000 7f55ad3c3640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a8109e00 0x7f55a81ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.784+0000 7f55ad3c3640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55a81094b0 0x7f55a81a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.784+0000 7f55ad3c3640 1 --2- 172.21.15.3:0/1295070810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55a8103bc0 0x7f55a81a17a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.784+0000 7f55ad3c3640 1 -- 172.21.15.3:0/1295070810 >> 172.21.15.3:0/1295070810 conn(0x7f55a807b910 msgr2=0x7f55a8107320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.784+0000 7f55ad3c3640 1 -- 172.21.15.3:0/1295070810 shutdown_connections 2024-09-21T06:21:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:50.784+0000 7f55ad3c3640 1 -- 172.21.15.3:0/1295070810 wait complete. 2024-09-21T06:21:50.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 721567 11.41 MB/sec execute 203 sec latency 67.180 ms 2024-09-21T06:21:51.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:51 smithi003 ceph-mon[31227]: pgmap v530: 97 pgs: 97 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 366 op/s 2024-09-21T06:21:51.714 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:21:51.715 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:21:51.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:51 smithi137 ceph-mon[34464]: pgmap v530: 97 pgs: 97 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 366 op/s 2024-09-21T06:21:51.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:51 smithi204 ceph-mon[34659]: pgmap v530: 97 pgs: 97 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 366 op/s 2024-09-21T06:21:51.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 724212 11.42 MB/sec execute 204 sec latency 29.108 ms 2024-09-21T06:21:52.040 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:52.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.568+0000 7fe335caf640 1 -- 172.21.15.3:0/8539921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33010ddf0 msgr2=0x7fe330110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.568+0000 7fe335caf640 1 --2- 172.21.15.3:0/8539921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33010ddf0 0x7fe330110300 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe328009ca0 tx=0x7fe32802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:21:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.568+0000 7fe335caf640 1 -- 172.21.15.3:0/8539921 shutdown_connections 2024-09-21T06:21:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.568+0000 7fe335caf640 1 --2- 172.21.15.3:0/8539921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33010ddf0 0x7fe330110300 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.568+0000 7fe335caf640 1 --2- 172.21.15.3:0/8539921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe330105270 0x7fe33010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.568+0000 7fe335caf640 1 --2- 172.21.15.3:0/8539921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe330104840 0x7fe330104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.568+0000 7fe335caf640 1 -- 172.21.15.3:0/8539921 >> 172.21.15.3:0/8539921 conn(0x7fe330079730 msgr2=0x7fe330079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.568+0000 7fe335caf640 1 -- 172.21.15.3:0/8539921 shutdown_connections 2024-09-21T06:21:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.568+0000 7fe335caf640 1 -- 172.21.15.3:0/8539921 wait complete. 2024-09-21T06:21:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.569+0000 7fe335caf640 1 Processor -- start 2024-09-21T06:21:52.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.569+0000 7fe335caf640 1 -- start start 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.569+0000 7fe335caf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe330104840 0x7fe33019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.569+0000 7fe335caf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe330105270 0x7fe33019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.569+0000 7fe335caf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33010ddf0 0x7fe33019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.569+0000 7fe335caf640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe330113a70 con 0x7fe33010ddf0 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.569+0000 7fe335caf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe3301138f0 con 0x7fe330105270 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.569+0000 7fe335caf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe330113bf0 con 0x7fe330104840 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.569+0000 7fe32f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe330104840 0x7fe33019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe32effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe330105270 0x7fe33019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe32f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe330104840 0x7fe33019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:52012/0 (socket says 172.21.15.3:52012) 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe32effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe330105270 0x7fe33019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:51030/0 (socket says 172.21.15.3:51030) 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33010ddf0 0x7fe33019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe32f7fe640 1 -- 172.21.15.3:0/4253517692 learned_addr learned my addr 172.21.15.3:0/4253517692 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe32f7fe640 1 -- 172.21.15.3:0/4253517692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe330105270 msgr2=0x7fe33019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe32f7fe640 1 --2- 172.21.15.3:0/4253517692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe330105270 0x7fe33019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe32f7fe640 1 -- 172.21.15.3:0/4253517692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33010ddf0 msgr2=0x7fe33019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe32f7fe640 1 --2- 172.21.15.3:0/4253517692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33010ddf0 0x7fe33019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe32f7fe640 1 -- 172.21.15.3:0/4253517692 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3301a03b0 con 0x7fe330104840 2024-09-21T06:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe32f7fe640 1 --2- 172.21.15.3:0/4253517692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe330104840 0x7fe33019a440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe32000e9c0 tx=0x7fe32000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe32cff9640 1 -- 172.21.15.3:0/4253517692 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32000cd80 con 0x7fe330104840 2024-09-21T06:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe32cff9640 1 -- 172.21.15.3:0/4253517692 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3200035f0 con 0x7fe330104840 2024-09-21T06:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.570+0000 7fe335caf640 1 -- 172.21.15.3:0/4253517692 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3301a06a0 con 0x7fe330104840 2024-09-21T06:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.571+0000 7fe32cff9640 1 -- 172.21.15.3:0/4253517692 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320010690 con 0x7fe330104840 2024-09-21T06:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.571+0000 7fe335caf640 1 -- 172.21.15.3:0/4253517692 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe330102630 con 0x7fe330104840 2024-09-21T06:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.572+0000 7fe335caf640 1 -- 172.21.15.3:0/4253517692 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe330104c50 con 0x7fe330104840 2024-09-21T06:21:52.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.573+0000 7fe32cff9640 1 -- 172.21.15.3:0/4253517692 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fe3200037a0 con 0x7fe330104840 2024-09-21T06:21:52.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.574+0000 7fe32cff9640 1 --2- 172.21.15.3:0/4253517692 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe3180b6700 0x7fe3180b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:52.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.574+0000 7fe32effd640 1 --2- 172.21.15.3:0/4253517692 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe3180b6700 0x7fe3180b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:52.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.574+0000 7fe32cff9640 1 -- 172.21.15.3:0/4253517692 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fe3200e7b20 con 0x7fe330104840 2024-09-21T06:21:52.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.575+0000 7fe32effd640 1 --2- 172.21.15.3:0/4253517692 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe3180b6700 0x7fe3180b8c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe33010f8c0 tx=0x7fe324012040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:52.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.577+0000 7fe32cff9640 1 -- 172.21.15.3:0/4253517692 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe3200ae050 con 0x7fe330104840 2024-09-21T06:21:52.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.822+0000 7fe335caf640 1 -- 172.21.15.3:0/4253517692 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe330102800 con 0x7fe330104840 2024-09-21T06:21:52.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.822+0000 7fe32cff9640 1 -- 172.21.15.3:0/4253517692 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11962 (secure 0 0 0) 0x7fe3200b2430 con 0x7fe330104840 2024-09-21T06:21:52.824 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:21:52.825 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":33,"btime":"2024-09-21T06:21:31:413510+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:21:30.733858+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/399369618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":399369618},{"type":"v1","addr":"172.21.15.137:6835","nonce":399369618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:21:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 33 2024-09-21T06:21:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.825+0000 7fe335caf640 1 -- 172.21.15.3:0/4253517692 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe3180b6700 msgr2=0x7fe3180b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.825+0000 7fe335caf640 1 --2- 172.21.15.3:0/4253517692 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe3180b6700 0x7fe3180b8c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe33010f8c0 tx=0x7fe324012040 comp rx=0 tx=0).stop 2024-09-21T06:21:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.825+0000 7fe335caf640 1 -- 172.21.15.3:0/4253517692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe330104840 msgr2=0x7fe33019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.825+0000 7fe335caf640 1 --2- 172.21.15.3:0/4253517692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe330104840 0x7fe33019a440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe32000e9c0 tx=0x7fe32000ee80 comp rx=0 tx=0).stop 2024-09-21T06:21:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.825+0000 7fe335caf640 1 -- 172.21.15.3:0/4253517692 shutdown_connections 2024-09-21T06:21:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.825+0000 7fe335caf640 1 --2- 172.21.15.3:0/4253517692 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe3180b6700 0x7fe3180b8c10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.825+0000 7fe335caf640 1 --2- 172.21.15.3:0/4253517692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33010ddf0 0x7fe33019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.825+0000 7fe335caf640 1 --2- 172.21.15.3:0/4253517692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe330105270 0x7fe33019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.825+0000 7fe335caf640 1 --2- 172.21.15.3:0/4253517692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe330104840 0x7fe33019a440 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.825+0000 7fe335caf640 1 -- 172.21.15.3:0/4253517692 >> 172.21.15.3:0/4253517692 conn(0x7fe330079730 msgr2=0x7fe330107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.825+0000 7fe335caf640 1 -- 172.21.15.3:0/4253517692 shutdown_connections 2024-09-21T06:21:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:52.825+0000 7fe335caf640 1 -- 172.21.15.3:0/4253517692 wait complete. 2024-09-21T06:21:52.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 725829 11.41 MB/sec execute 205 sec latency 20.519 ms 2024-09-21T06:21:53.528 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:21:53.798 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:53.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:53 smithi003 ceph-mon[31227]: pgmap v531: 97 pgs: 97 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 322 op/s 2024-09-21T06:21:53.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:53 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/4253517692' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:21:53.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:53 smithi137 ceph-mon[34464]: pgmap v531: 97 pgs: 97 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 322 op/s 2024-09-21T06:21:53.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:53 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/4253517692' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:21:53.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:53 smithi204 ceph-mon[34659]: pgmap v531: 97 pgs: 97 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 322 op/s 2024-09-21T06:21:53.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:53 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/4253517692' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:21:53.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 728218 11.43 MB/sec execute 206 sec latency 41.560 ms 2024-09-21T06:21:54.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.625+0000 7f9c9359e640 1 -- 172.21.15.3:0/627176972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c9410dc20 msgr2=0x7f9c94110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:54.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.625+0000 7f9c9359e640 1 --2- 172.21.15.3:0/627176972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c9410dc20 0x7f9c94110110 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009ca0 tx=0x7f9c8c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.625+0000 7f9c9359e640 1 -- 172.21.15.3:0/627176972 shutdown_connections 2024-09-21T06:21:54.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.625+0000 7f9c9359e640 1 --2- 172.21.15.3:0/627176972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c9410dc20 0x7f9c94110110 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.625+0000 7f9c9359e640 1 --2- 172.21.15.3:0/627176972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c94105110 0x7f9c9410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.625+0000 7f9c9359e640 1 --2- 172.21.15.3:0/627176972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c941047c0 0x7f9c94104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.625+0000 7f9c9359e640 1 -- 172.21.15.3:0/627176972 >> 172.21.15.3:0/627176972 conn(0x7f9c940fe2c0 msgr2=0x7f9c94100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:54.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.625+0000 7f9c9359e640 1 -- 172.21.15.3:0/627176972 shutdown_connections 2024-09-21T06:21:54.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.625+0000 7f9c9359e640 1 -- 172.21.15.3:0/627176972 wait complete. 2024-09-21T06:21:54.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.626+0000 7f9c9359e640 1 Processor -- start 2024-09-21T06:21:54.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.626+0000 7f9c9359e640 1 -- start start 2024-09-21T06:21:54.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c9359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c941047c0 0x7f9c9419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c9359e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c94105110 0x7f9c9419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c9359e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c9410dc20 0x7f9c9419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c9359e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941138a0 con 0x7f9c94105110 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c9359e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94113720 con 0x7f9c9410dc20 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c9359e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94113a20 con 0x7f9c941047c0 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c91d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c94105110 0x7f9c9419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c91d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c94105110 0x7f9c9419a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53768/0 (socket says 172.21.15.3:53768) 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c91d9b640 1 -- 172.21.15.3:0/3151933481 learned_addr learned my addr 172.21.15.3:0/3151933481 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c92d9d640 1 --2- 172.21.15.3:0/3151933481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c9410dc20 0x7f9c9419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c941047c0 0x7f9c9419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c91d9b640 1 -- 172.21.15.3:0/3151933481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c941047c0 msgr2=0x7f9c9419a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c91d9b640 1 --2- 172.21.15.3:0/3151933481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c941047c0 0x7f9c9419a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c91d9b640 1 -- 172.21.15.3:0/3151933481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c9410dc20 msgr2=0x7f9c9419fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:54.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c91d9b640 1 --2- 172.21.15.3:0/3151933481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c9410dc20 0x7f9c9419fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c91d9b640 1 -- 172.21.15.3:0/3151933481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c941a0240 con 0x7f9c94105110 2024-09-21T06:21:54.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.627+0000 7f9c91d9b640 1 --2- 172.21.15.3:0/3151933481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c94105110 0x7f9c9419a870 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800e9f0 tx=0x7f9c8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:54.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.628+0000 7f9c837fe640 1 -- 172.21.15.3:0/3151933481 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800ccc0 con 0x7f9c94105110 2024-09-21T06:21:54.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.628+0000 7f9c9359e640 1 -- 172.21.15.3:0/3151933481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c941a0530 con 0x7f9c94105110 2024-09-21T06:21:54.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.628+0000 7f9c837fe640 1 -- 172.21.15.3:0/3151933481 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c880108e0 con 0x7f9c94105110 2024-09-21T06:21:54.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.628+0000 7f9c837fe640 1 -- 172.21.15.3:0/3151933481 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88018860 con 0x7f9c94105110 2024-09-21T06:21:54.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.628+0000 7f9c9359e640 1 -- 172.21.15.3:0/3151933481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c941a6c90 con 0x7f9c94105110 2024-09-21T06:21:54.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.630+0000 7f9c9f376740 1 Processor -- start 2024-09-21T06:21:54.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c9f376740 1 -- start start 2024-09-21T06:21:54.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c837fe640 1 -- 172.21.15.3:0/3151933481 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f9c88018a00 con 0x7f9c94105110 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c9f376740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 0x5652f0df8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c9f376740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a281b0 0x5652f0df89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c9f376740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a30390 0x5652f0df8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c9f376740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5652f0d6eea0 con 0x5652f0a281b0 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c9f376740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5652f0d5e2d0 con 0x5652f0a30390 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c9f376740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5652f0d620e0 con 0x5652f0a51070 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a281b0 0x5652f0df89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c92d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 0x5652f0df8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a281b0 0x5652f0df89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53774/0 (socket says 172.21.15.3:53774) 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c9259c640 1 -- 172.21.15.3:0/1668052156 learned_addr learned my addr 172.21.15.3:0/1668052156 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c91d9b640 1 --2- 172.21.15.3:0/1668052156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a30390 0x5652f0df8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c92d9d640 1 -- 172.21.15.3:0/1668052156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a30390 msgr2=0x5652f0df8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c92d9d640 1 --2- 172.21.15.3:0/1668052156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a30390 0x5652f0df8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c92d9d640 1 -- 172.21.15.3:0/1668052156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a281b0 msgr2=0x5652f0df89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:54.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c92d9d640 1 --2- 172.21.15.3:0/1668052156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a281b0 0x5652f0df89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.631+0000 7f9c92d9d640 1 -- 172.21.15.3:0/1668052156 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f0abc650 con 0x5652f0a51070 2024-09-21T06:21:54.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.632+0000 7f9c92d9d640 1 --2- 172.21.15.3:0/1668052156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 0x5652f0df8460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c005e30 tx=0x7f9c8c02f410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb63db46443f100 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:54.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.632+0000 7f9c837fe640 1 --2- 172.21.15.3:0/3151933481 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9c7c0b6680 0x7f9c7c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.632+0000 7f9c80ff9640 1 -- 172.21.15.3:0/1668052156 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c031cf0 con 0x5652f0a51070 2024-09-21T06:21:54.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.632+0000 7f9c80ff9640 1 -- 172.21.15.3:0/1668052156 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c8c036030 con 0x5652f0a51070 2024-09-21T06:21:54.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.632+0000 7f9c80ff9640 1 -- 172.21.15.3:0/1668052156 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c004040 con 0x5652f0a51070 2024-09-21T06:21:54.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.632+0000 7f9c9259c640 1 --2- 172.21.15.3:0/3151933481 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9c7c0b6680 0x7f9c7c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.632+0000 7f9c837fe640 1 -- 172.21.15.3:0/3151933481 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f9c880e7950 con 0x7f9c94105110 2024-09-21T06:21:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.633+0000 7f9c9f376740 1 -- 172.21.15.3:0/1668052156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 msgr2=0x5652f0df8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.633+0000 7f9c9259c640 1 --2- 172.21.15.3:0/3151933481 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9c7c0b6680 0x7f9c7c0b8b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5652f0df70e0 tx=0x7f9c84005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.633+0000 7f9c9f376740 1 --2- 172.21.15.3:0/1668052156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 0x5652f0df8460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c005e30 tx=0x7f9c8c02f410 comp rx=0 tx=0).stop 2024-09-21T06:21:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.633+0000 7f9c9f376740 1 -- 172.21.15.3:0/1668052156 shutdown_connections 2024-09-21T06:21:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.633+0000 7f9c9f376740 1 --2- 172.21.15.3:0/1668052156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a30390 0x5652f0df8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.633+0000 7f9c9f376740 1 --2- 172.21.15.3:0/1668052156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a281b0 0x5652f0df89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.633+0000 7f9c9f376740 1 --2- 172.21.15.3:0/1668052156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 0x5652f0df8460 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.633+0000 7f9c9f376740 1 -- 172.21.15.3:0/1668052156 >> 172.21.15.3:0/1668052156 conn(0x5652f0a45460 msgr2=0x5652f0df6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.633+0000 7f9c9f376740 1 -- 172.21.15.3:0/1668052156 shutdown_connections 2024-09-21T06:21:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.633+0000 7f9c9f376740 1 -- 172.21.15.3:0/1668052156 wait complete. 2024-09-21T06:21:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c9f376740 1 -- start start 2024-09-21T06:21:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c9f376740 1 Processor -- start 2024-09-21T06:21:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c9f376740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a281b0 0x5652f0e03b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c9f376740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a30390 0x5652f0e040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c9f376740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a51070 0x5652f0e0fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c9f376740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5652f0bbf700 con 0x5652f0a30390 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c9f376740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5652f0cf9160 con 0x5652f0a51070 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c9f376740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5652f0e06a60 con 0x5652f0a281b0 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c9f376740 10 client.0 _ll_register_callbacks cb 0x5652f0d129e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c91d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a30390 0x5652f0e040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c92d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a51070 0x5652f0e0fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c91d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a30390 0x5652f0e040c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53778/0 (socket says 172.21.15.3:53778) 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c91d9b640 1 -- 172.21.15.3:0/3185282124 learned_addr learned my addr 172.21.15.3:0/3185282124 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.634+0000 7f9c9259c640 1 --2- 172.21.15.3:0/3185282124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a281b0 0x5652f0e03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.635+0000 7f9c92d9d640 1 -- 172.21.15.3:0/3185282124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a281b0 msgr2=0x5652f0e03b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.635+0000 7f9c92d9d640 1 --2- 172.21.15.3:0/3185282124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a281b0 0x5652f0e03b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.635+0000 7f9c92d9d640 1 -- 172.21.15.3:0/3185282124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a30390 msgr2=0x5652f0e040c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.635+0000 7f9c92d9d640 1 --2- 172.21.15.3:0/3185282124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a30390 0x5652f0e040c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.635+0000 7f9c92d9d640 1 -- 172.21.15.3:0/3185282124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f0d26c90 con 0x5652f0a51070 2024-09-21T06:21:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.635+0000 7f9c92d9d640 1 --2- 172.21.15.3:0/3185282124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a51070 0x5652f0e0fd80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009bf0 tx=0x7f9c8c0099f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcdb64a71bf041f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.635+0000 7f9c79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:21:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.635+0000 7f9c79ffb640 1 -- 172.21.15.3:0/3185282124 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c04a040 con 0x5652f0a51070 2024-09-21T06:21:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.635+0000 7f9c9f376740 1 -- 172.21.15.3:0/3185282124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c8c036030 con 0x5652f0a51070 2024-09-21T06:21:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.635+0000 7f9c79ffb640 1 -- 172.21.15.3:0/3185282124 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f9c8c031650 con 0x5652f0a51070 2024-09-21T06:21:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.635+0000 7f9c79ffb640 1 -- 172.21.15.3:0/3185282124 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9c8c009360 con 0x5652f0a51070 2024-09-21T06:21:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.635+0000 7f9c9f376740 10 client.27026 fetch_fsmap learned FSMap version 33 2024-09-21T06:21:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.635+0000 7f9c9f376740 1 -- 172.21.15.3:0/3185282124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5652f0e127a0 con 0x5652f0a51070 2024-09-21T06:21:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.636+0000 7f9c79ffb640 1 -- 172.21.15.3:0/3185282124 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c8c036030 con 0x5652f0a51070 2024-09-21T06:21:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.636+0000 7f9c79ffb640 1 -- 172.21.15.3:0/3185282124 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f9c8c04d030 con 0x5652f0a51070 2024-09-21T06:21:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.636+0000 7f9c9f376740 10 client.27026 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:21:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.636+0000 7f9c9f376740 10 client.27026 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:21:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.636+0000 7f9c9f376740 20 client.27026 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.636+0000 7f9c9f376740 1 --2- 172.21.15.3:0/3185282124 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5652f0a0f9f0 0x5652f0e15b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.636+0000 7f9c9f376740 4 client.27026 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.636+0000 7f9c9f376740 1 -- 172.21.15.3:0/3185282124 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652f0d70df0 con 0x5652f0a0f9f0 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.636+0000 7f9c9259c640 1 --2- 172.21.15.3:0/3185282124 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5652f0a0f9f0 0x5652f0e15b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c79ffb640 10 client.27026 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c9259c640 1 --2- 172.21.15.3:0/3185282124 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5652f0a0f9f0 0x5652f0e15b00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7acdeb575ef59401 server_cookie=c0445a4b0df9d063 in_seq=0 out_seq=0 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c79ffb640 1 -- 172.21.15.3:0/3185282124 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5652f0d70df0 con 0x5652f0a0f9f0 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c79ffb640 10 client.27026 handle_command_reply: tid=0 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c9f376740 1 client.27026 shutdown 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c9f376740 1 -- 172.21.15.3:0/3185282124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a51070 msgr2=0x5652f0e0fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c9f376740 1 --2- 172.21.15.3:0/3185282124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a51070 0x5652f0e0fd80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009bf0 tx=0x7f9c8c0099f0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c9f376740 1 -- 172.21.15.3:0/3185282124 shutdown_connections 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c9f376740 1 --2- 172.21.15.3:0/3185282124 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5652f0a0f9f0 0x5652f0e15b00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c9f376740 1 --2- 172.21.15.3:0/3185282124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a51070 0x5652f0e0fd80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c9f376740 1 --2- 172.21.15.3:0/3185282124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a30390 0x5652f0e040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c79ffb640 0 client.27026 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:21:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c9f376740 1 --2- 172.21.15.3:0/3185282124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a281b0 0x5652f0e03b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.637+0000 7f9c9f376740 1 -- 172.21.15.3:0/3185282124 >> 172.21.15.3:0/3185282124 conn(0x5652f0a45460 msgr2=0x5652f0dfb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.638+0000 7f9c9f376740 1 -- 172.21.15.3:0/3185282124 shutdown_connections 2024-09-21T06:21:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.638+0000 7f9c9f376740 1 -- 172.21.15.3:0/3185282124 wait complete. 2024-09-21T06:21:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.638+0000 7f9c9f376740 20 client.27026 trim_cache size 0 max 16384 2024-09-21T06:21:54.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9f376740 1 Processor -- start 2024-09-21T06:21:54.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9f376740 1 -- start start 2024-09-21T06:21:54.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9f376740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a281b0 0x5652f0df63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9f376740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a30390 0x5652f0e0fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9f376740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 0x5652f0e03930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9f376740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941af480 con 0x5652f0a281b0 2024-09-21T06:21:54.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9f376740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941b0fc0 con 0x5652f0a30390 2024-09-21T06:21:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9f376740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941afb50 con 0x5652f0a51070 2024-09-21T06:21:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a281b0 0x5652f0df63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a281b0 0x5652f0df63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53794/0 (socket says 172.21.15.3:53794) 2024-09-21T06:21:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9259c640 1 -- 172.21.15.3:0/1830678229 learned_addr learned my addr 172.21.15.3:0/1830678229 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c91d9b640 1 --2- 172.21.15.3:0/1830678229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a30390 0x5652f0e0fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c92d9d640 1 --2- 172.21.15.3:0/1830678229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 0x5652f0e03930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9259c640 1 -- 172.21.15.3:0/1830678229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 msgr2=0x5652f0e03930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9259c640 1 --2- 172.21.15.3:0/1830678229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 0x5652f0e03930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9259c640 1 -- 172.21.15.3:0/1830678229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a30390 msgr2=0x5652f0e0fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9259c640 1 --2- 172.21.15.3:0/1830678229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a30390 0x5652f0e0fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.652+0000 7f9c9259c640 1 -- 172.21.15.3:0/1830678229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f0e02080 con 0x5652f0a281b0 2024-09-21T06:21:54.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.653+0000 7f9c9259c640 1 --2- 172.21.15.3:0/1830678229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a281b0 0x5652f0df63c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9c84012cb0 tx=0x7f9c8401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a2159187e72ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:54.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.653+0000 7f9c7bfff640 1 -- 172.21.15.3:0/1830678229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8401dd90 con 0x5652f0a281b0 2024-09-21T06:21:54.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.653+0000 7f9c7bfff640 1 -- 172.21.15.3:0/1830678229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c840054d0 con 0x5652f0a281b0 2024-09-21T06:21:54.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.653+0000 7f9c7bfff640 1 -- 172.21.15.3:0/1830678229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84030680 con 0x5652f0a281b0 2024-09-21T06:21:54.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.653+0000 7f9c9f376740 1 -- 172.21.15.3:0/1830678229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a281b0 msgr2=0x5652f0df63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:54.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.653+0000 7f9c9f376740 1 --2- 172.21.15.3:0/1830678229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a281b0 0x5652f0df63c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9c84012cb0 tx=0x7f9c8401b000 comp rx=0 tx=0).stop 2024-09-21T06:21:54.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 -- 172.21.15.3:0/1830678229 shutdown_connections 2024-09-21T06:21:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 --2- 172.21.15.3:0/1830678229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 0x5652f0e03930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 --2- 172.21.15.3:0/1830678229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a30390 0x5652f0e0fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 --2- 172.21.15.3:0/1830678229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a281b0 0x5652f0df63c0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 -- 172.21.15.3:0/1830678229 >> 172.21.15.3:0/1830678229 conn(0x5652f0a45460 msgr2=0x5652f0df5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 -- 172.21.15.3:0/1830678229 shutdown_connections 2024-09-21T06:21:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 -- 172.21.15.3:0/1830678229 wait complete. 2024-09-21T06:21:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 -- start start 2024-09-21T06:21:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 Processor -- start 2024-09-21T06:21:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a281b0 0x5652f0e07d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a30390 0x5652f0e08270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 0x5652f0e087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941b08f0 con 0x5652f0a30390 2024-09-21T06:21:54.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941b0220 con 0x5652f0a281b0 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.654+0000 7f9c9f376740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941ad460 con 0x5652f0a51070 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c91d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a30390 0x5652f0e08270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a281b0 0x5652f0e07d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c91d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a30390 0x5652f0e08270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53796/0 (socket says 172.21.15.3:53796) 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c91d9b640 1 -- 172.21.15.3:0/3227506006 learned_addr learned my addr 172.21.15.3:0/3227506006 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a281b0 0x5652f0e07d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:51074/0 (socket says 172.21.15.3:51074) 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c9f376740 10 client.0 _ll_register_callbacks cb 0x7f9c84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c92d9d640 1 --2- 172.21.15.3:0/3227506006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 0x5652f0e087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c9259c640 1 -- 172.21.15.3:0/3227506006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 msgr2=0x5652f0e087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c9259c640 1 --2- 172.21.15.3:0/3227506006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 0x5652f0e087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c9259c640 1 -- 172.21.15.3:0/3227506006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a30390 msgr2=0x5652f0e08270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c9259c640 1 --2- 172.21.15.3:0/3227506006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a30390 0x5652f0e08270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c9259c640 1 -- 172.21.15.3:0/3227506006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f0e7f370 con 0x5652f0a281b0 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c91d9b640 1 --2- 172.21.15.3:0/3227506006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a30390 0x5652f0e08270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:54.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c9259c640 1 --2- 172.21.15.3:0/3227506006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a281b0 0x5652f0e07d30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9c84032040 tx=0x7f9c84009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ce99b97262748ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c79ffb640 1 -- 172.21.15.3:0/3227506006 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84004840 con 0x5652f0a281b0 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c79ffb640 1 -- 172.21.15.3:0/3227506006 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f9c840049e0 con 0x5652f0a281b0 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.655+0000 7f9c9f376740 1 -- 172.21.15.3:0/3227506006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c840054d0 con 0x5652f0a281b0 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.656+0000 7f9c79ffb640 1 -- 172.21.15.3:0/3227506006 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9c84004df0 con 0x5652f0a281b0 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.656+0000 7f9c9f376740 10 client.27032 fetch_fsmap learned FSMap version 33 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.656+0000 7f9c9f376740 1 -- 172.21.15.3:0/3227506006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5652f0e805f0 con 0x5652f0a281b0 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.656+0000 7f9c79ffb640 1 -- 172.21.15.3:0/3227506006 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c840054d0 con 0x5652f0a281b0 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.656+0000 7f9c79ffb640 1 -- 172.21.15.3:0/3227506006 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f9c8402a420 con 0x5652f0a281b0 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.656+0000 7f9c9f376740 10 client.27032 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.656+0000 7f9c9f376740 10 client.27032 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.656+0000 7f9c9f376740 20 client.27032 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.656+0000 7f9c9f376740 1 --2- 172.21.15.3:0/3227506006 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5652f09e4da0 0x5652f0e82d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.656+0000 7f9c9f376740 4 client.27032 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:21:54.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.656+0000 7f9c9f376740 1 -- 172.21.15.3:0/3227506006 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652f0df17a0 con 0x5652f09e4da0 2024-09-21T06:21:54.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.656+0000 7f9c91d9b640 1 --2- 172.21.15.3:0/3227506006 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5652f09e4da0 0x5652f0e82d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:54.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c79ffb640 10 client.27032 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:21:54.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c91d9b640 1 --2- 172.21.15.3:0/3227506006 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5652f09e4da0 0x5652f0e82d70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da0d70536860a2cb server_cookie=471576e5880d1e5a in_seq=0 out_seq=0 2024-09-21T06:21:54.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c79ffb640 1 -- 172.21.15.3:0/3227506006 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652f0df17a0 con 0x5652f09e4da0 2024-09-21T06:21:54.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c79ffb640 10 client.27032 handle_command_reply: tid=0 2024-09-21T06:21:54.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c9f376740 1 client.27032 shutdown 2024-09-21T06:21:54.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c9f376740 1 -- 172.21.15.3:0/3227506006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a281b0 msgr2=0x5652f0e07d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:54.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c9f376740 1 --2- 172.21.15.3:0/3227506006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a281b0 0x5652f0e07d30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9c84032040 tx=0x7f9c84009c40 comp rx=0 tx=0).stop 2024-09-21T06:21:54.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c9f376740 1 -- 172.21.15.3:0/3227506006 shutdown_connections 2024-09-21T06:21:54.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c9f376740 1 --2- 172.21.15.3:0/3227506006 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x5652f09e4da0 0x5652f0e82d70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c9f376740 1 --2- 172.21.15.3:0/3227506006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f0a51070 0x5652f0e087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c9f376740 1 --2- 172.21.15.3:0/3227506006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652f0a30390 0x5652f0e08270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c9f376740 1 --2- 172.21.15.3:0/3227506006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652f0a281b0 0x5652f0e07d30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c79ffb640 0 client.27032 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:21:54.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c9f376740 1 -- 172.21.15.3:0/3227506006 >> 172.21.15.3:0/3227506006 conn(0x5652f0a45460 msgr2=0x5652f0e03930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:54.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c9f376740 1 -- 172.21.15.3:0/3227506006 shutdown_connections 2024-09-21T06:21:54.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c9f376740 1 -- 172.21.15.3:0/3227506006 wait complete. 2024-09-21T06:21:54.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.657+0000 7f9c9f376740 20 client.27032 trim_cache size 0 max 16384 2024-09-21T06:21:54.664 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:21:54.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.658+0000 7f9c9359e640 1 -- 172.21.15.3:0/3151933481 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9c7c0b6680 msgr2=0x7f9c7c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:54.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.658+0000 7f9c9359e640 1 --2- 172.21.15.3:0/3151933481 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9c7c0b6680 0x7f9c7c0b8b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5652f0df70e0 tx=0x7f9c84005f90 comp rx=0 tx=0).stop 2024-09-21T06:21:54.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.658+0000 7f9c9359e640 1 -- 172.21.15.3:0/3151933481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c94105110 msgr2=0x7f9c9419a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:54.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.658+0000 7f9c9359e640 1 --2- 172.21.15.3:0/3151933481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c94105110 0x7f9c9419a870 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800e9f0 tx=0x7f9c8800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.658+0000 7f9c9359e640 1 -- 172.21.15.3:0/3151933481 shutdown_connections 2024-09-21T06:21:54.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.658+0000 7f9c9359e640 1 --2- 172.21.15.3:0/3151933481 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f9c7c0b6680 0x7f9c7c0b8b90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.658+0000 7f9c9359e640 1 --2- 172.21.15.3:0/3151933481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c9410dc20 0x7f9c9419fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.658+0000 7f9c9359e640 1 --2- 172.21.15.3:0/3151933481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c94105110 0x7f9c9419a870 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.658+0000 7f9c9359e640 1 --2- 172.21.15.3:0/3151933481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c941047c0 0x7f9c9419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.658+0000 7f9c9359e640 1 -- 172.21.15.3:0/3151933481 >> 172.21.15.3:0/3151933481 conn(0x7f9c940fe2c0 msgr2=0x7f9c941076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.658+0000 7f9c9359e640 1 -- 172.21.15.3:0/3151933481 shutdown_connections 2024-09-21T06:21:54.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:54.658+0000 7f9c9359e640 1 -- 172.21.15.3:0/3151933481 wait complete. 2024-09-21T06:21:54.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 729839 11.40 MB/sec execute 207 sec latency 16.763 ms 2024-09-21T06:21:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:55 smithi003 ceph-mon[31227]: pgmap v532: 97 pgs: 97 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 307 op/s 2024-09-21T06:21:55.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:55 smithi137 ceph-mon[34464]: pgmap v532: 97 pgs: 97 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 307 op/s 2024-09-21T06:21:55.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:55 smithi204 ceph-mon[34659]: pgmap v532: 97 pgs: 97 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 307 op/s 2024-09-21T06:21:55.916 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:21:55.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 731855 11.41 MB/sec execute 208 sec latency 49.328 ms 2024-09-21T06:21:56.191 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.687+0000 7f74259ce640 1 -- 172.21.15.3:0/1674723494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74201083a0 msgr2=0x7f74201087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.687+0000 7f74259ce640 1 --2- 172.21.15.3:0/1674723494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74201083a0 0x7f74201087b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7408009900 tx=0x7f740802a410 comp rx=0 tx=0).stop 2024-09-21T06:21:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.687+0000 7f74259ce640 1 -- 172.21.15.3:0/1674723494 shutdown_connections 2024-09-21T06:21:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.687+0000 7f74259ce640 1 --2- 172.21.15.3:0/1674723494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7420108cf0 0x7f742010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.687+0000 7f74259ce640 1 --2- 172.21.15.3:0/1674723494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74201083a0 0x7f74201087b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.687+0000 7f74259ce640 1 --2- 172.21.15.3:0/1674723494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7420102a60 0x7f7420102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.687+0000 7f74259ce640 1 -- 172.21.15.3:0/1674723494 >> 172.21.15.3:0/1674723494 conn(0x7f7420079730 msgr2=0x7f7420079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.687+0000 7f74259ce640 1 -- 172.21.15.3:0/1674723494 shutdown_connections 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.687+0000 7f74259ce640 1 -- 172.21.15.3:0/1674723494 wait complete. 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.688+0000 7f74259ce640 1 Processor -- start 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.688+0000 7f74259ce640 1 -- start start 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f74259ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7420102a60 0x7f742019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f74259ce640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74201083a0 0x7f742019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f74259ce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7420108cf0 0x7f742019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f74259ce640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7420113a80 con 0x7f7420108cf0 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f74259ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7420113900 con 0x7f7420102a60 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f74259ce640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7420113c00 con 0x7f74201083a0 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f741effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7420102a60 0x7f742019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f741f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7420108cf0 0x7f742019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f741e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74201083a0 0x7f742019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f741effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7420102a60 0x7f742019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:52460/0 (socket says 172.21.15.3:52460) 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f741effd640 1 -- 172.21.15.3:0/2013012443 learned_addr learned my addr 172.21.15.3:0/2013012443 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f741f7fe640 1 -- 172.21.15.3:0/2013012443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74201083a0 msgr2=0x7f742019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f741f7fe640 1 --2- 172.21.15.3:0/2013012443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74201083a0 0x7f742019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f741f7fe640 1 -- 172.21.15.3:0/2013012443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7420102a60 msgr2=0x7f742019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f741f7fe640 1 --2- 172.21.15.3:0/2013012443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7420102a60 0x7f742019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.689+0000 7f741f7fe640 1 -- 172.21.15.3:0/2013012443 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74201a04e0 con 0x7f7420108cf0 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.690+0000 7f741f7fe640 1 --2- 172.21.15.3:0/2013012443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7420108cf0 0x7f742019fdb0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7410009fd0 tx=0x7f741000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.690+0000 7f74249cc640 1 -- 172.21.15.3:0/2013012443 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741000ce20 con 0x7f7420108cf0 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.690+0000 7f74259ce640 1 -- 172.21.15.3:0/2013012443 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74201a07d0 con 0x7f7420108cf0 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.690+0000 7f74249cc640 1 -- 172.21.15.3:0/2013012443 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7410005020 con 0x7f7420108cf0 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.690+0000 7f74249cc640 1 -- 172.21.15.3:0/2013012443 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410010690 con 0x7f7420108cf0 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.690+0000 7f74259ce640 1 -- 172.21.15.3:0/2013012443 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74201a6f30 con 0x7f7420108cf0 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f742bf42740 1 Processor -- start 2024-09-21T06:21:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f74249cc640 1 -- 172.21.15.3:0/2013012443 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f7410010830 con 0x7f7420108cf0 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f742bf42740 1 -- start start 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f742bf42740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f422c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f742bf42740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f42800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f742bf42740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b7a390 0x556237f42d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f742bf42740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556237eb8ea0 con 0x556237b9b070 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f742bf42740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556237ea82d0 con 0x556237b7a390 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f742bf42740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556237eac0e0 con 0x556237b721b0 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f741e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f741effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f42800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f741effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f42800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:36458/0 (socket says 172.21.15.3:36458) 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f741e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60876/0 (socket says 172.21.15.3:60876) 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.693+0000 7f741effd640 1 -- 172.21.15.3:0/3733196996 learned_addr learned my addr 172.21.15.3:0/3733196996 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f741e7fc640 1 --2- 172.21.15.3:0/3733196996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b7a390 0x556237f42d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f741effd640 1 -- 172.21.15.3:0/3733196996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b7a390 msgr2=0x556237f42d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f741effd640 1 --2- 172.21.15.3:0/3733196996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b7a390 0x556237f42d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f741effd640 1 -- 172.21.15.3:0/3733196996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 msgr2=0x556237f422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f741effd640 1 --2- 172.21.15.3:0/3733196996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f741effd640 1 -- 172.21.15.3:0/3733196996 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556237c06650 con 0x556237b721b0 2024-09-21T06:21:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f741e7fc640 1 --2- 172.21.15.3:0/3733196996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b7a390 0x556237f42d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f74249cc640 1 --2- 172.21.15.3:0/2013012443 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f740c0b6680 0x7f740c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f741effd640 1 --2- 172.21.15.3:0/3733196996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f42800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7418009dc0 tx=0x7f7418011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cef4ee31a8b39395 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f741e7fc640 1 --2- 172.21.15.3:0/2013012443 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f740c0b6680 0x7f740c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f74249cc640 1 -- 172.21.15.3:0/2013012443 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f74100e7d40 con 0x7f7420108cf0 2024-09-21T06:21:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f7415ffb640 1 -- 172.21.15.3:0/3733196996 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418005220 con 0x556237b721b0 2024-09-21T06:21:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f7415ffb640 1 -- 172.21.15.3:0/3733196996 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7418004c50 con 0x556237b721b0 2024-09-21T06:21:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.694+0000 7f7415ffb640 1 -- 172.21.15.3:0/3733196996 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74180197b0 con 0x556237b721b0 2024-09-21T06:21:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.695+0000 7f742bf42740 1 -- 172.21.15.3:0/3733196996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 msgr2=0x556237f42800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.695+0000 7f741e7fc640 1 --2- 172.21.15.3:0/2013012443 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f740c0b6680 0x7f740c0b8b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7408000c00 tx=0x7f7408042040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.695+0000 7f742bf42740 1 --2- 172.21.15.3:0/3733196996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f42800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7418009dc0 tx=0x7f7418011810 comp rx=0 tx=0).stop 2024-09-21T06:21:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.695+0000 7f742bf42740 1 -- 172.21.15.3:0/3733196996 shutdown_connections 2024-09-21T06:21:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.695+0000 7f742bf42740 1 --2- 172.21.15.3:0/3733196996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f42800 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.695+0000 7f742bf42740 1 --2- 172.21.15.3:0/3733196996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b7a390 0x556237f42d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.695+0000 7f742bf42740 1 --2- 172.21.15.3:0/3733196996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.695+0000 7f742bf42740 1 -- 172.21.15.3:0/3733196996 >> 172.21.15.3:0/3733196996 conn(0x556237b8f460 msgr2=0x556237f401b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.695+0000 7f742bf42740 1 -- 172.21.15.3:0/3733196996 shutdown_connections 2024-09-21T06:21:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.696+0000 7f742bf42740 1 -- 172.21.15.3:0/3733196996 wait complete. 2024-09-21T06:21:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.696+0000 7f742bf42740 1 -- start start 2024-09-21T06:21:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.696+0000 7f742bf42740 1 Processor -- start 2024-09-21T06:21:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.697+0000 7f742bf42740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f4d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.697+0000 7f742bf42740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b7a390 0x556237f4dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.697+0000 7f742bf42740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b9b070 0x556237f4e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.697+0000 7f741effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f4d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.697+0000 7f741e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b7a390 0x556237f4dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.697+0000 7f741effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f4d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:36470/0 (socket says 172.21.15.3:36470) 2024-09-21T06:21:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.697+0000 7f741effd640 1 -- 172.21.15.3:0/1483638289 learned_addr learned my addr 172.21.15.3:0/1483638289 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.697+0000 7f742bf42740 1 -- 172.21.15.3:0/1483638289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556237d09700 con 0x556237b7a390 2024-09-21T06:21:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.697+0000 7f742bf42740 1 -- 172.21.15.3:0/1483638289 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556237e43160 con 0x556237b9b070 2024-09-21T06:21:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.697+0000 7f742bf42740 1 -- 172.21.15.3:0/1483638289 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556237f50aa0 con 0x556237b721b0 2024-09-21T06:21:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.697+0000 7f742bf42740 10 client.0 _ll_register_callbacks cb 0x556237e5c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.698+0000 7f741f7fe640 1 --2- 172.21.15.3:0/1483638289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b9b070 0x556237f4e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.698+0000 7f741effd640 1 -- 172.21.15.3:0/1483638289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b9b070 msgr2=0x556237f4e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.698+0000 7f741effd640 1 --2- 172.21.15.3:0/1483638289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b9b070 0x556237f4e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.698+0000 7f741effd640 1 -- 172.21.15.3:0/1483638289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b7a390 msgr2=0x556237f4dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.698+0000 7f741effd640 1 --2- 172.21.15.3:0/1483638289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b7a390 0x556237f4dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.698+0000 7f741effd640 1 -- 172.21.15.3:0/1483638289 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556237e70c90 con 0x556237b721b0 2024-09-21T06:21:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.698+0000 7f741effd640 1 --2- 172.21.15.3:0/1483638289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f4d840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7418005700 tx=0x7f74180048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5a3a6f39fc09a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.698+0000 7f73feffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:21:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.699+0000 7f73feffd640 1 -- 172.21.15.3:0/1483638289 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418027070 con 0x556237b721b0 2024-09-21T06:21:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.699+0000 7f73feffd640 1 -- 172.21.15.3:0/1483638289 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f7418004450 con 0x556237b721b0 2024-09-21T06:21:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.699+0000 7f742bf42740 1 -- 172.21.15.3:0/1483638289 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7418004c50 con 0x556237b721b0 2024-09-21T06:21:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.699+0000 7f73feffd640 1 -- 172.21.15.3:0/1483638289 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7418031050 con 0x556237b721b0 2024-09-21T06:21:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.699+0000 7f742bf42740 10 client.26992 fetch_fsmap learned FSMap version 33 2024-09-21T06:21:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.699+0000 7f742bf42740 1 -- 172.21.15.3:0/1483638289 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556237f64e60 con 0x556237b721b0 2024-09-21T06:21:56.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.700+0000 7f73feffd640 1 -- 172.21.15.3:0/1483638289 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7418004c50 con 0x556237b721b0 2024-09-21T06:21:56.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.700+0000 7f73feffd640 1 -- 172.21.15.3:0/1483638289 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7418019510 con 0x556237b721b0 2024-09-21T06:21:56.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.700+0000 7f742bf42740 10 client.26992 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:21:56.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.700+0000 7f742bf42740 10 client.26992 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:21:56.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.700+0000 7f742bf42740 20 client.26992 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:56.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.700+0000 7f742bf42740 1 --2- 172.21.15.3:0/1483638289 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556237b599f0 0x556237f682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.700+0000 7f742bf42740 4 client.26992 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:56.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.700+0000 7f742bf42740 1 -- 172.21.15.3:0/1483638289 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556237ebadf0 con 0x556237b599f0 2024-09-21T06:21:56.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.700+0000 7f741e7fc640 1 --2- 172.21.15.3:0/1483638289 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556237b599f0 0x556237f682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.701+0000 7f741e7fc640 1 --2- 172.21.15.3:0/1483638289 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556237b599f0 0x556237f682d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=59034fffece681e7 server_cookie=95bd42fea1fffe34 in_seq=0 out_seq=0 2024-09-21T06:21:56.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.701+0000 7f73feffd640 10 client.26992 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:21:56.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.701+0000 7f73feffd640 1 -- 172.21.15.3:0/1483638289 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556237ebadf0 con 0x556237b599f0 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f73feffd640 10 client.26992 handle_command_reply: tid=0 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f742bf42740 1 client.26992 shutdown 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f742bf42740 1 -- 172.21.15.3:0/1483638289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 msgr2=0x556237f4d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f742bf42740 1 --2- 172.21.15.3:0/1483638289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f4d840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7418005700 tx=0x7f74180048f0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f742bf42740 1 -- 172.21.15.3:0/1483638289 shutdown_connections 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f742bf42740 1 --2- 172.21.15.3:0/1483638289 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556237b599f0 0x556237f682d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f73feffd640 0 client.26992 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f742bf42740 1 --2- 172.21.15.3:0/1483638289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b9b070 0x556237f4e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f742bf42740 1 --2- 172.21.15.3:0/1483638289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b7a390 0x556237f4dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f742bf42740 1 --2- 172.21.15.3:0/1483638289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f4d840 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f742bf42740 1 -- 172.21.15.3:0/1483638289 >> 172.21.15.3:0/1483638289 conn(0x556237b8f460 msgr2=0x556237f454a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f742bf42740 1 -- 172.21.15.3:0/1483638289 shutdown_connections 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f742bf42740 1 -- 172.21.15.3:0/1483638289 wait complete. 2024-09-21T06:21:56.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.702+0000 7f742bf42740 20 client.26992 trim_cache size 0 max 16384 2024-09-21T06:21:56.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.716+0000 7f742bf42740 1 Processor -- start 2024-09-21T06:21:56.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.716+0000 7f742bf42740 1 -- start start 2024-09-21T06:21:56.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.716+0000 7f742bf42740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b721b0 0x556237f4f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.716+0000 7f742bf42740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b7a390 0x556237f4aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.716+0000 7f742bf42740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f4af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.716+0000 7f742bf42740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f74201ad490 con 0x556237b9b070 2024-09-21T06:21:56.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.716+0000 7f742bf42740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f74201aefd0 con 0x556237b721b0 2024-09-21T06:21:56.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.716+0000 7f742bf42740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74201adb60 con 0x556237b7a390 2024-09-21T06:21:56.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.717+0000 7f741e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b7a390 0x556237f4aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.717+0000 7f741effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b721b0 0x556237f4f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.717+0000 7f741f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f4af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.717+0000 7f741e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b7a390 0x556237f4aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:36474/0 (socket says 172.21.15.3:36474) 2024-09-21T06:21:56.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.717+0000 7f741e7fc640 1 -- 172.21.15.3:0/2392118245 learned_addr learned my addr 172.21.15.3:0/2392118245 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:56.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.717+0000 7f741effd640 1 --2- 172.21.15.3:0/2392118245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b721b0 0x556237f4f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:52480/0 (socket says 172.21.15.3:52480) 2024-09-21T06:21:56.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.717+0000 7f741e7fc640 1 -- 172.21.15.3:0/2392118245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b721b0 msgr2=0x556237f4f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.717+0000 7f741e7fc640 1 --2- 172.21.15.3:0/2392118245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b721b0 0x556237f4f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.717+0000 7f741e7fc640 1 -- 172.21.15.3:0/2392118245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 msgr2=0x556237f4af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.717+0000 7f741e7fc640 1 --2- 172.21.15.3:0/2392118245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f4af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.717+0000 7f741e7fc640 1 -- 172.21.15.3:0/2392118245 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556237f3b5d0 con 0x556237b7a390 2024-09-21T06:21:56.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.717+0000 7f741e7fc640 1 --2- 172.21.15.3:0/2392118245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b7a390 0x556237f4aa30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7408031630 tx=0x7f7408009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8841b35881c67864 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:56.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.718+0000 7f74157fa640 1 -- 172.21.15.3:0/2392118245 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408035040 con 0x556237b7a390 2024-09-21T06:21:56.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.718+0000 7f74157fa640 1 -- 172.21.15.3:0/2392118245 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7408059070 con 0x556237b7a390 2024-09-21T06:21:56.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.718+0000 7f74157fa640 1 -- 172.21.15.3:0/2392118245 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740804e450 con 0x556237b7a390 2024-09-21T06:21:56.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.718+0000 7f742bf42740 1 -- 172.21.15.3:0/2392118245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b7a390 msgr2=0x556237f4aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.718+0000 7f742bf42740 1 --2- 172.21.15.3:0/2392118245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b7a390 0x556237f4aa30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7408031630 tx=0x7f7408009260 comp rx=0 tx=0).stop 2024-09-21T06:21:56.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.719+0000 7f742bf42740 1 -- 172.21.15.3:0/2392118245 shutdown_connections 2024-09-21T06:21:56.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.719+0000 7f742bf42740 1 --2- 172.21.15.3:0/2392118245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b7a390 0x556237f4aa30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.719+0000 7f742bf42740 1 --2- 172.21.15.3:0/2392118245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f4af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.719+0000 7f742bf42740 1 --2- 172.21.15.3:0/2392118245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b721b0 0x556237f4f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.719+0000 7f742bf42740 1 -- 172.21.15.3:0/2392118245 >> 172.21.15.3:0/2392118245 conn(0x556237b8f460 msgr2=0x556237f43cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:56.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.719+0000 7f742bf42740 1 -- 172.21.15.3:0/2392118245 shutdown_connections 2024-09-21T06:21:56.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.719+0000 7f742bf42740 1 -- 172.21.15.3:0/2392118245 wait complete. 2024-09-21T06:21:56.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.719+0000 7f742bf42740 1 -- start start 2024-09-21T06:21:56.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.719+0000 7f742bf42740 1 Processor -- start 2024-09-21T06:21:56.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f742bf42740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f5ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f742bf42740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b7a390 0x556237f5c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f742bf42740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f5c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f742bf42740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f74201ae900 con 0x556237b9b070 2024-09-21T06:21:56.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f742bf42740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f74201ae230 con 0x556237b7a390 2024-09-21T06:21:56.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f742bf42740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74201ab470 con 0x556237b721b0 2024-09-21T06:21:56.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f742bf42740 10 client.0 _ll_register_callbacks cb 0x7f74100093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:56.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f741f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f5c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f741e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b7a390 0x556237f5c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f741f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f5c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60894/0 (socket says 172.21.15.3:60894) 2024-09-21T06:21:56.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f741f7fe640 1 -- 172.21.15.3:0/4096888992 learned_addr learned my addr 172.21.15.3:0/4096888992 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:56.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f741e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b7a390 0x556237f5c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:52494/0 (socket says 172.21.15.3:52494) 2024-09-21T06:21:56.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f741effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f5ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f741f7fe640 1 -- 172.21.15.3:0/4096888992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 msgr2=0x556237f5ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f741f7fe640 1 --2- 172.21.15.3:0/4096888992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f5ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f741f7fe640 1 -- 172.21.15.3:0/4096888992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b7a390 msgr2=0x556237f5c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f741f7fe640 1 --2- 172.21.15.3:0/4096888992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b7a390 0x556237f5c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f741f7fe640 1 -- 172.21.15.3:0/4096888992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556237f5d0f0 con 0x556237b9b070 2024-09-21T06:21:56.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f741f7fe640 1 --2- 172.21.15.3:0/4096888992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f5c960 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f74100e5ad0 tx=0x7f74100ad400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b7876ddf5ce032 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:56.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.720+0000 7f73feffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:56.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f73feffd640 1 -- 172.21.15.3:0/4096888992 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74100b26b0 con 0x556237b9b070 2024-09-21T06:21:56.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f73feffd640 1 -- 172.21.15.3:0/4096888992 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f74100cfbe0 con 0x556237b9b070 2024-09-21T06:21:56.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f742bf42740 1 -- 172.21.15.3:0/4096888992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7408059070 con 0x556237b9b070 2024-09-21T06:21:56.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f73feffd640 1 -- 172.21.15.3:0/4096888992 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f74100b75f0 con 0x556237b9b070 2024-09-21T06:21:56.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f742bf42740 10 client.17484 fetch_fsmap learned FSMap version 33 2024-09-21T06:21:56.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f742bf42740 1 -- 172.21.15.3:0/4096888992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556237fd2c80 con 0x556237b9b070 2024-09-21T06:21:56.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f73feffd640 1 -- 172.21.15.3:0/4096888992 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74100cfde0 con 0x556237b9b070 2024-09-21T06:21:56.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f73feffd640 1 -- 172.21.15.3:0/4096888992 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f74100b21e0 con 0x556237b9b070 2024-09-21T06:21:56.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f742bf42740 10 client.17484 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:21:56.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f742bf42740 10 client.17484 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:21:56.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f742bf42740 20 client.17484 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:56.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f742bf42740 1 --2- 172.21.15.3:0/4096888992 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556237b2eda0 0x556237fd5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f742bf42740 4 client.17484 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.721+0000 7f742bf42740 1 -- 172.21.15.3:0/4096888992 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7408059070 con 0x556237b2eda0 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.722+0000 7f741effd640 1 --2- 172.21.15.3:0/4096888992 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556237b2eda0 0x556237fd5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.722+0000 7f73feffd640 10 client.17484 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.722+0000 7f741effd640 1 --2- 172.21.15.3:0/4096888992 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556237b2eda0 0x556237fd5460 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dec62ba0cd9f748a server_cookie=6e05f5bada797228 in_seq=0 out_seq=0 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.722+0000 7f73feffd640 1 -- 172.21.15.3:0/4096888992 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7408059070 con 0x556237b2eda0 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.722+0000 7f73feffd640 10 client.17484 handle_command_reply: tid=0 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.722+0000 7f742bf42740 1 client.17484 shutdown 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f742bf42740 1 -- 172.21.15.3:0/4096888992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 msgr2=0x556237f5c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f742bf42740 1 --2- 172.21.15.3:0/4096888992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f5c960 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f74100e5ad0 tx=0x7f74100ad400 comp rx=0 tx=0).stop 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f742bf42740 1 -- 172.21.15.3:0/4096888992 shutdown_connections 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f742bf42740 1 --2- 172.21.15.3:0/4096888992 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x556237b2eda0 0x556237fd5460 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f742bf42740 1 --2- 172.21.15.3:0/4096888992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556237b9b070 0x556237f5c960 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f742bf42740 1 --2- 172.21.15.3:0/4096888992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556237b7a390 0x556237f5c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f742bf42740 1 --2- 172.21.15.3:0/4096888992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556237b721b0 0x556237f5ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f742bf42740 1 -- 172.21.15.3:0/4096888992 >> 172.21.15.3:0/4096888992 conn(0x556237b8f460 msgr2=0x556237f4dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:56.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f73feffd640 0 client.17484 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:21:56.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f742bf42740 1 -- 172.21.15.3:0/4096888992 shutdown_connections 2024-09-21T06:21:56.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f742bf42740 1 -- 172.21.15.3:0/4096888992 wait complete. 2024-09-21T06:21:56.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f742bf42740 20 client.17484 trim_cache size 0 max 16384 2024-09-21T06:21:56.729 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:21:56.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f74259ce640 1 -- 172.21.15.3:0/2013012443 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f740c0b6680 msgr2=0x7f740c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f74259ce640 1 --2- 172.21.15.3:0/2013012443 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f740c0b6680 0x7f740c0b8b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7408000c00 tx=0x7f7408042040 comp rx=0 tx=0).stop 2024-09-21T06:21:56.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f74259ce640 1 -- 172.21.15.3:0/2013012443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7420108cf0 msgr2=0x7f742019fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.723+0000 7f74259ce640 1 --2- 172.21.15.3:0/2013012443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7420108cf0 0x7f742019fdb0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7410009fd0 tx=0x7f741000ef10 comp rx=0 tx=0).stop 2024-09-21T06:21:56.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.724+0000 7f74259ce640 1 -- 172.21.15.3:0/2013012443 shutdown_connections 2024-09-21T06:21:56.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.724+0000 7f74259ce640 1 --2- 172.21.15.3:0/2013012443 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f740c0b6680 0x7f740c0b8b90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.724+0000 7f74259ce640 1 --2- 172.21.15.3:0/2013012443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74201083a0 0x7f742019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.724+0000 7f74259ce640 1 --2- 172.21.15.3:0/2013012443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7420108cf0 0x7f742019fdb0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.724+0000 7f74259ce640 1 --2- 172.21.15.3:0/2013012443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7420102a60 0x7f742019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.724+0000 7f74259ce640 1 -- 172.21.15.3:0/2013012443 >> 172.21.15.3:0/2013012443 conn(0x7f7420079730 msgr2=0x7f7420104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:56.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.724+0000 7f74259ce640 1 -- 172.21.15.3:0/2013012443 shutdown_connections 2024-09-21T06:21:56.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:56.724+0000 7f74259ce640 1 -- 172.21.15.3:0/2013012443 wait complete. 2024-09-21T06:21:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.746+0000 7fc70e0a4640 1 -- 172.21.15.3:0/603505927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7081029e0 msgr2=0x7fc708102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.746+0000 7fc70e0a4640 1 --2- 172.21.15.3:0/603505927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7081029e0 0x7fc708102df0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009900 tx=0x7fc6f802a410 comp rx=0 tx=0).stop 2024-09-21T06:21:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.746+0000 7fc70e0a4640 1 -- 172.21.15.3:0/603505927 shutdown_connections 2024-09-21T06:21:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.746+0000 7fc70e0a4640 1 --2- 172.21.15.3:0/603505927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc708108c60 0x7fc70810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.746+0000 7fc70e0a4640 1 --2- 172.21.15.3:0/603505927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc708108310 0x7fc708108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.746+0000 7fc70e0a4640 1 --2- 172.21.15.3:0/603505927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7081029e0 0x7fc708102df0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.746+0000 7fc70e0a4640 1 -- 172.21.15.3:0/603505927 >> 172.21.15.3:0/603505927 conn(0x7fc7080fe3c0 msgr2=0x7fc708100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.746+0000 7fc70e0a4640 1 -- 172.21.15.3:0/603505927 shutdown_connections 2024-09-21T06:21:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.746+0000 7fc70e0a4640 1 -- 172.21.15.3:0/603505927 wait complete. 2024-09-21T06:21:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.747+0000 7fc70e0a4640 1 Processor -- start 2024-09-21T06:21:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.747+0000 7fc70e0a4640 1 -- start start 2024-09-21T06:21:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.747+0000 7fc70e0a4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7081029e0 0x7fc708199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.747+0000 7fc70e0a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc708108310 0x7fc7081998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.747+0000 7fc70e0a4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc708108c60 0x7fc70819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.747+0000 7fc70e0a4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc708112920 con 0x7fc708108c60 2024-09-21T06:21:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.747+0000 7fc70e0a4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081127a0 con 0x7fc708108310 2024-09-21T06:21:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.747+0000 7fc70e0a4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc708112aa0 con 0x7fc7081029e0 2024-09-21T06:21:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.748+0000 7fc7077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7081029e0 0x7fc708199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.748+0000 7fc706ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc708108310 0x7fc7081998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.748+0000 7fc7077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7081029e0 0x7fc708199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:36494/0 (socket says 172.21.15.3:36494) 2024-09-21T06:21:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.748+0000 7fc707fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc708108c60 0x7fc70819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.748+0000 7fc7077fe640 1 -- 172.21.15.3:0/1373145266 learned_addr learned my addr 172.21.15.3:0/1373145266 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.748+0000 7fc7077fe640 1 -- 172.21.15.3:0/1373145266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc708108310 msgr2=0x7fc7081998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.748+0000 7fc7077fe640 1 --2- 172.21.15.3:0/1373145266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc708108310 0x7fc7081998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.748+0000 7fc7077fe640 1 -- 172.21.15.3:0/1373145266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc708108c60 msgr2=0x7fc70819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.748+0000 7fc7077fe640 1 --2- 172.21.15.3:0/1373145266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc708108c60 0x7fc70819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.748+0000 7fc7077fe640 1 -- 172.21.15.3:0/1373145266 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70819f270 con 0x7fc7081029e0 2024-09-21T06:21:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.748+0000 7fc7077fe640 1 --2- 172.21.15.3:0/1373145266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7081029e0 0x7fc708199340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8005940 tx=0x7fc6f80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.749+0000 7fc704ff9640 1 -- 172.21.15.3:0/1373145266 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f803f030 con 0x7fc7081029e0 2024-09-21T06:21:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.749+0000 7fc704ff9640 1 -- 172.21.15.3:0/1373145266 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6f80044c0 con 0x7fc7081029e0 2024-09-21T06:21:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.749+0000 7fc70e0a4640 1 -- 172.21.15.3:0/1373145266 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70819f500 con 0x7fc7081029e0 2024-09-21T06:21:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.749+0000 7fc704ff9640 1 -- 172.21.15.3:0/1373145266 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8035070 con 0x7fc7081029e0 2024-09-21T06:21:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.749+0000 7fc70e0a4640 1 -- 172.21.15.3:0/1373145266 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7081a5cf0 con 0x7fc7081029e0 2024-09-21T06:21:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.751+0000 7fc71500e740 1 Processor -- start 2024-09-21T06:21:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.751+0000 7fc71500e740 1 -- start start 2024-09-21T06:21:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc71500e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6d51b0 0x560a5fa7a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc71500e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6ac2f0 0x560a5fa7aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc71500e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6b44d0 0x560a5fa7b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc71500e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560a5f9ef6f0 con 0x560a5f6b44d0 2024-09-21T06:21:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc71500e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560a5f9deb20 con 0x560a5f6d51b0 2024-09-21T06:21:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc71500e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560a5f9e2930 con 0x560a5f6ac2f0 2024-09-21T06:21:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc707fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6d51b0 0x560a5fa7a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc706ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6b44d0 0x560a5fa7b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc707fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6d51b0 0x560a5fa7a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:52516/0 (socket says 172.21.15.3:52516) 2024-09-21T06:21:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc706ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6b44d0 0x560a5fa7b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60922/0 (socket says 172.21.15.3:60922) 2024-09-21T06:21:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc707fff640 1 -- 172.21.15.3:0/4293528765 learned_addr learned my addr 172.21.15.3:0/4293528765 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc707fff640 1 -- 172.21.15.3:0/4293528765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6ac2f0 msgr2=0x560a5fa7aef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:21:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc707fff640 1 --2- 172.21.15.3:0/4293528765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6ac2f0 0x560a5fa7aef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc707fff640 1 -- 172.21.15.3:0/4293528765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6b44d0 msgr2=0x560a5fa7b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc707fff640 1 --2- 172.21.15.3:0/4293528765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6b44d0 0x560a5fa7b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc707fff640 1 -- 172.21.15.3:0/4293528765 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a5f9a4530 con 0x560a5f6d51b0 2024-09-21T06:21:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.752+0000 7fc706ffd640 1 --2- 172.21.15.3:0/4293528765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6b44d0 0x560a5fa7b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.753+0000 7fc707fff640 1 --2- 172.21.15.3:0/4293528765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6d51b0 0x560a5fa7a9b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc700011440 tx=0x7fc700011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea7d4cc866868e1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.753+0000 7fc6edffb640 1 -- 172.21.15.3:0/4293528765 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700004d20 con 0x560a5f6d51b0 2024-09-21T06:21:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.753+0000 7fc6edffb640 1 -- 172.21.15.3:0/4293528765 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc700004050 con 0x560a5f6d51b0 2024-09-21T06:21:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.753+0000 7fc6edffb640 1 -- 172.21.15.3:0/4293528765 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700010750 con 0x560a5f6d51b0 2024-09-21T06:21:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.753+0000 7fc704ff9640 1 -- 172.21.15.3:0/1373145266 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fc6f8042030 con 0x7fc7081029e0 2024-09-21T06:21:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.754+0000 7fc71500e740 1 -- 172.21.15.3:0/4293528765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6d51b0 msgr2=0x560a5fa7a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.754+0000 7fc71500e740 1 --2- 172.21.15.3:0/4293528765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6d51b0 0x560a5fa7a9b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc700011440 tx=0x7fc700011900 comp rx=0 tx=0).stop 2024-09-21T06:21:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.754+0000 7fc71500e740 1 -- 172.21.15.3:0/4293528765 shutdown_connections 2024-09-21T06:21:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.754+0000 7fc71500e740 1 --2- 172.21.15.3:0/4293528765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6ac2f0 0x560a5fa7aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.754+0000 7fc71500e740 1 --2- 172.21.15.3:0/4293528765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6b44d0 0x560a5fa7b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.754+0000 7fc71500e740 1 --2- 172.21.15.3:0/4293528765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6d51b0 0x560a5fa7a9b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.754+0000 7fc71500e740 1 -- 172.21.15.3:0/4293528765 >> 172.21.15.3:0/4293528765 conn(0x560a5f6c9300 msgr2=0x560a5fa788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.754+0000 7fc71500e740 1 -- 172.21.15.3:0/4293528765 shutdown_connections 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.754+0000 7fc71500e740 1 -- 172.21.15.3:0/4293528765 wait complete. 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.754+0000 7fc71500e740 1 -- start start 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc704ff9640 1 --2- 172.21.15.3:0/1373145266 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6f00b6590 0x7fc6f00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc71500e740 1 Processor -- start 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc707fff640 1 --2- 172.21.15.3:0/1373145266 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6f00b6590 0x7fc6f00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc704ff9640 1 -- 172.21.15.3:0/1373145266 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fc6f8107720 con 0x7fc7081029e0 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc71500e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6b44d0 0x560a5fa86170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc71500e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6d51b0 0x560a5fa866b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc71500e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f693d80 0x560a5faa3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc71500e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560a5f916ab0 con 0x560a5f693d80 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc71500e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560a5f984730 con 0x560a5f6d51b0 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc71500e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560a5fa9a140 con 0x560a5f6b44d0 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc71500e740 10 client.0 _ll_register_callbacks cb 0x560a5f9d86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc7077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6b44d0 0x560a5fa86170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc707fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f693d80 0x560a5faa3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc7077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6b44d0 0x560a5fa86170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:36504/0 (socket says 172.21.15.3:36504) 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.755+0000 7fc7077fe640 1 -- 172.21.15.3:0/1164431876 learned_addr learned my addr 172.21.15.3:0/1164431876 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc706ffd640 1 --2- 172.21.15.3:0/1164431876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6d51b0 0x560a5fa866b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc707fff640 1 --2- 172.21.15.3:0/1373145266 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6f00b6590 0x7fc6f00b8aa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc700009de0 tx=0x7fc700005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc7077fe640 1 -- 172.21.15.3:0/1164431876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6d51b0 msgr2=0x560a5fa866b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc7077fe640 1 --2- 172.21.15.3:0/1164431876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6d51b0 0x560a5fa866b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc7077fe640 1 -- 172.21.15.3:0/1164431876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f693d80 msgr2=0x560a5faa3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc7077fe640 1 --2- 172.21.15.3:0/1164431876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f693d80 0x560a5faa3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc7077fe640 1 -- 172.21.15.3:0/1164431876 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a5f740790 con 0x560a5f6b44d0 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc7077fe640 1 --2- 172.21.15.3:0/1164431876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6b44d0 0x560a5fa86170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc6f81054b0 tx=0x7fc6f80c9630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c21e4d7b9e76f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc6e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc6e6ffd640 1 -- 172.21.15.3:0/1164431876 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8049040 con 0x560a5f6b44d0 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc6e6ffd640 1 -- 172.21.15.3:0/1164431876 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fc6f80d6d40 con 0x560a5f6b44d0 2024-09-21T06:21:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc71500e740 1 -- 172.21.15.3:0/1164431876 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc700004050 con 0x560a5f6b44d0 2024-09-21T06:21:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc6e6ffd640 1 -- 172.21.15.3:0/1164431876 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc6f80cd170 con 0x560a5f6b44d0 2024-09-21T06:21:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc71500e740 10 client.27007 fetch_fsmap learned FSMap version 33 2024-09-21T06:21:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.756+0000 7fc71500e740 1 -- 172.21.15.3:0/1164431876 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560a5f7b9620 con 0x560a5f6b44d0 2024-09-21T06:21:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.757+0000 7fc6e6ffd640 1 -- 172.21.15.3:0/1164431876 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc700004050 con 0x560a5f6b44d0 2024-09-21T06:21:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.757+0000 7fc6e6ffd640 1 -- 172.21.15.3:0/1164431876 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc6f804c060 con 0x560a5f6b44d0 2024-09-21T06:21:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.757+0000 7fc71500e740 10 client.27007 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:21:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.757+0000 7fc71500e740 10 client.27007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.757+0000 7fc71500e740 20 client.27007 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.757+0000 7fc71500e740 1 --2- 172.21.15.3:0/1164431876 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a5f6691c0 0x560a5faa92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.757+0000 7fc71500e740 4 client.27007 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.757+0000 7fc71500e740 1 -- 172.21.15.3:0/1164431876 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a5f9f1640 con 0x560a5f6691c0 2024-09-21T06:21:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.757+0000 7fc706ffd640 1 --2- 172.21.15.3:0/1164431876 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a5f6691c0 0x560a5faa92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.757+0000 7fc6e6ffd640 10 client.27007 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.757+0000 7fc706ffd640 1 --2- 172.21.15.3:0/1164431876 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a5f6691c0 0x560a5faa92b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8020b1e2447c604c server_cookie=f90e759fe462b1db in_seq=0 out_seq=0 2024-09-21T06:21:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc6e6ffd640 1 -- 172.21.15.3:0/1164431876 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560a5f9f1640 con 0x560a5f6691c0 2024-09-21T06:21:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc6e6ffd640 10 client.27007 handle_command_reply: tid=0 2024-09-21T06:21:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc71500e740 1 client.27007 shutdown 2024-09-21T06:21:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc71500e740 1 -- 172.21.15.3:0/1164431876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6b44d0 msgr2=0x560a5fa86170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc71500e740 1 --2- 172.21.15.3:0/1164431876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6b44d0 0x560a5fa86170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc6f81054b0 tx=0x7fc6f80c9630 comp rx=0 tx=0).stop 2024-09-21T06:21:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc71500e740 1 -- 172.21.15.3:0/1164431876 shutdown_connections 2024-09-21T06:21:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc71500e740 1 --2- 172.21.15.3:0/1164431876 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a5f6691c0 0x560a5faa92b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc71500e740 1 --2- 172.21.15.3:0/1164431876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f693d80 0x560a5faa3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc71500e740 1 --2- 172.21.15.3:0/1164431876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6d51b0 0x560a5fa866b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc71500e740 1 --2- 172.21.15.3:0/1164431876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6b44d0 0x560a5fa86170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc71500e740 1 -- 172.21.15.3:0/1164431876 >> 172.21.15.3:0/1164431876 conn(0x560a5f6c9300 msgr2=0x560a5fa7b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc6e6ffd640 0 client.27007 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc71500e740 1 -- 172.21.15.3:0/1164431876 shutdown_connections 2024-09-21T06:21:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc71500e740 1 -- 172.21.15.3:0/1164431876 wait complete. 2024-09-21T06:21:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.759+0000 7fc71500e740 20 client.27007 trim_cache size 0 max 16384 2024-09-21T06:21:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.774+0000 7fc71500e740 1 Processor -- start 2024-09-21T06:21:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.774+0000 7fc71500e740 1 -- start start 2024-09-21T06:21:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.774+0000 7fc71500e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f693d80 0x560a5fa801f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.774+0000 7fc71500e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6b44d0 0x560a5faa3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.774+0000 7fc71500e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6d51b0 0x560a5faa38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.774+0000 7fc71500e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc708074360 con 0x560a5f693d80 2024-09-21T06:21:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.774+0000 7fc71500e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081a6770 con 0x560a5f6b44d0 2024-09-21T06:21:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.774+0000 7fc71500e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc708074a30 con 0x560a5f6d51b0 2024-09-21T06:21:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.774+0000 7fc7077fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f693d80 0x560a5fa801f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.774+0000 7fc7077fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f693d80 0x560a5fa801f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60938/0 (socket says 172.21.15.3:60938) 2024-09-21T06:21:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.774+0000 7fc7077fe640 1 -- 172.21.15.3:0/3576967127 learned_addr learned my addr 172.21.15.3:0/3576967127 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.774+0000 7fc707fff640 1 --2- 172.21.15.3:0/3576967127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6d51b0 0x560a5faa38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.774+0000 7fc706ffd640 1 --2- 172.21.15.3:0/3576967127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6b44d0 0x560a5faa3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.775+0000 7fc7077fe640 1 -- 172.21.15.3:0/3576967127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6d51b0 msgr2=0x560a5faa38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.775+0000 7fc7077fe640 1 --2- 172.21.15.3:0/3576967127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6d51b0 0x560a5faa38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.775+0000 7fc7077fe640 1 -- 172.21.15.3:0/3576967127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6b44d0 msgr2=0x560a5faa3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.775+0000 7fc7077fe640 1 --2- 172.21.15.3:0/3576967127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6b44d0 0x560a5faa3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.775+0000 7fc7077fe640 1 -- 172.21.15.3:0/3576967127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a5fa85f70 con 0x560a5f693d80 2024-09-21T06:21:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.775+0000 7fc707fff640 1 --2- 172.21.15.3:0/3576967127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6d51b0 0x560a5faa38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.775+0000 7fc7077fe640 1 --2- 172.21.15.3:0/3576967127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f693d80 0x560a5fa801f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8006010 tx=0x7fc6f8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9c35f23ec478b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.775+0000 7fc6ed7fa640 1 -- 172.21.15.3:0/3576967127 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8049070 con 0x560a5f693d80 2024-09-21T06:21:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.775+0000 7fc6ed7fa640 1 -- 172.21.15.3:0/3576967127 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6f80d6fd0 con 0x560a5f693d80 2024-09-21T06:21:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.775+0000 7fc6ed7fa640 1 -- 172.21.15.3:0/3576967127 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f80cc590 con 0x560a5f693d80 2024-09-21T06:21:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.776+0000 7fc71500e740 1 -- 172.21.15.3:0/3576967127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f693d80 msgr2=0x560a5fa801f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.776+0000 7fc71500e740 1 --2- 172.21.15.3:0/3576967127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f693d80 0x560a5fa801f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8006010 tx=0x7fc6f8005c90 comp rx=0 tx=0).stop 2024-09-21T06:21:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.776+0000 7fc71500e740 1 -- 172.21.15.3:0/3576967127 shutdown_connections 2024-09-21T06:21:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.776+0000 7fc71500e740 1 --2- 172.21.15.3:0/3576967127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f6d51b0 0x560a5faa38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.776+0000 7fc71500e740 1 --2- 172.21.15.3:0/3576967127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6b44d0 0x560a5faa3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.776+0000 7fc71500e740 1 --2- 172.21.15.3:0/3576967127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f693d80 0x560a5fa801f0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.776+0000 7fc71500e740 1 -- 172.21.15.3:0/3576967127 >> 172.21.15.3:0/3576967127 conn(0x560a5f6c9300 msgr2=0x560a5fa7fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.776+0000 7fc71500e740 1 -- 172.21.15.3:0/3576967127 shutdown_connections 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.776+0000 7fc71500e740 1 -- 172.21.15.3:0/3576967127 wait complete. 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.776+0000 7fc71500e740 1 -- start start 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc71500e740 1 Processor -- start 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc71500e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f693d80 0x560a5fa9ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc71500e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6b44d0 0x560a5fa9b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc71500e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6d51b0 0x560a5fb0bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc71500e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081a60a0 con 0x560a5f6d51b0 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc71500e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc708075100 con 0x560a5f6b44d0 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc71500e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc708072340 con 0x560a5f693d80 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc71500e740 10 client.0 _ll_register_callbacks cb 0x7fc6f8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc707fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6d51b0 0x560a5fb0bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc707fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6d51b0 0x560a5fb0bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60950/0 (socket says 172.21.15.3:60950) 2024-09-21T06:21:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc707fff640 1 -- 172.21.15.3:0/3125568075 learned_addr learned my addr 172.21.15.3:0/3125568075 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc706ffd640 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6b44d0 0x560a5fa9b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc7077fe640 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f693d80 0x560a5fa9ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc707fff640 1 -- 172.21.15.3:0/3125568075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f693d80 msgr2=0x560a5fa9ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc707fff640 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f693d80 0x560a5fa9ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc707fff640 1 -- 172.21.15.3:0/3125568075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6b44d0 msgr2=0x560a5fa9b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc707fff640 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6b44d0 0x560a5fa9b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc707fff640 1 -- 172.21.15.3:0/3125568075 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a5fb0c330 con 0x560a5f6d51b0 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.777+0000 7fc706ffd640 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6b44d0 0x560a5fa9b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc707fff640 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6d51b0 0x560a5fb0bc90 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc7000048f0 tx=0x7fc700009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32efbe9767d92ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc6e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc6e6ffd640 1 -- 172.21.15.3:0/3125568075 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700035020 con 0x560a5f6d51b0 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc6e6ffd640 1 -- 172.21.15.3:0/3125568075 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7fc700010820 con 0x560a5f6d51b0 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc71500e740 1 -- 172.21.15.3:0/3125568075 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6f80d6fd0 con 0x560a5f6d51b0 2024-09-21T06:21:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc6e6ffd640 1 -- 172.21.15.3:0/3125568075 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc70001d040 con 0x560a5f6d51b0 2024-09-21T06:21:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc71500e740 10 client.17502 fetch_fsmap learned FSMap version 33 2024-09-21T06:21:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc71500e740 1 -- 172.21.15.3:0/3125568075 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560a5fb13750 con 0x560a5f6d51b0 2024-09-21T06:21:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc6e6ffd640 1 -- 172.21.15.3:0/3125568075 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6f80d6fd0 con 0x560a5f6d51b0 2024-09-21T06:21:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc6e6ffd640 1 -- 172.21.15.3:0/3125568075 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc700031a90 con 0x560a5f6d51b0 2024-09-21T06:21:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc71500e740 10 client.17502 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:21:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc71500e740 10 client.17502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc71500e740 20 client.17502 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.778+0000 7fc71500e740 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a5fb13d60 0x560a5fb161b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.779+0000 7fc71500e740 4 client.17502 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:21:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.779+0000 7fc71500e740 1 -- 172.21.15.3:0/3125568075 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a5f9ed0f0 con 0x560a5fb13d60 2024-09-21T06:21:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.779+0000 7fc7077fe640 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a5fb13d60 0x560a5fb161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.779+0000 7fc6e6ffd640 10 client.17502 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.779+0000 7fc7077fe640 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a5fb13d60 0x560a5fb161b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb6c013a5209299 server_cookie=79cba6816b142b1c in_seq=0 out_seq=0 2024-09-21T06:21:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.940+0000 7fc6e6ffd640 1 -- 172.21.15.3:0/3125568075 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a5f9ed0f0 con 0x560a5fb13d60 2024-09-21T06:21:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.940+0000 7fc6e6ffd640 10 client.17502 handle_command_reply: tid=0 2024-09-21T06:21:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.940+0000 7fc71500e740 1 client.17502 shutdown 2024-09-21T06:21:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.941+0000 7fc71500e740 1 -- 172.21.15.3:0/3125568075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6d51b0 msgr2=0x560a5fb0bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.941+0000 7fc71500e740 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6d51b0 0x560a5fb0bc90 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc7000048f0 tx=0x7fc700009be0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.941+0000 7fc71500e740 1 -- 172.21.15.3:0/3125568075 shutdown_connections 2024-09-21T06:21:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.941+0000 7fc71500e740 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x560a5fb13d60 0x560a5fb161b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.941+0000 7fc71500e740 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a5f6d51b0 0x560a5fb0bc90 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.941+0000 7fc6e6ffd640 0 client.17502 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:21:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.941+0000 7fc71500e740 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5f6b44d0 0x560a5fa9b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.941+0000 7fc71500e740 1 --2- 172.21.15.3:0/3125568075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a5f693d80 0x560a5fa9ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.941+0000 7fc71500e740 1 -- 172.21.15.3:0/3125568075 >> 172.21.15.3:0/3125568075 conn(0x560a5f6c9300 msgr2=0x560a5fa780f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.941+0000 7fc71500e740 1 -- 172.21.15.3:0/3125568075 shutdown_connections 2024-09-21T06:21:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.941+0000 7fc71500e740 1 -- 172.21.15.3:0/3125568075 wait complete. 2024-09-21T06:21:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.941+0000 7fc71500e740 20 client.17502 trim_cache size 0 max 16384 2024-09-21T06:21:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:21:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:21:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:21:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:21:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.942+0000 7fc70e0a4640 1 -- 172.21.15.3:0/1373145266 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6f00b6590 msgr2=0x7fc6f00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.942+0000 7fc70e0a4640 1 --2- 172.21.15.3:0/1373145266 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6f00b6590 0x7fc6f00b8aa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc700009de0 tx=0x7fc700005fd0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.942+0000 7fc70e0a4640 1 -- 172.21.15.3:0/1373145266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7081029e0 msgr2=0x7fc708199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.942+0000 7fc70e0a4640 1 --2- 172.21.15.3:0/1373145266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7081029e0 0x7fc708199340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8005940 tx=0x7fc6f80040c0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.942+0000 7fc70e0a4640 1 -- 172.21.15.3:0/1373145266 shutdown_connections 2024-09-21T06:21:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.942+0000 7fc70e0a4640 1 --2- 172.21.15.3:0/1373145266 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc6f00b6590 0x7fc6f00b8aa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.942+0000 7fc70e0a4640 1 --2- 172.21.15.3:0/1373145266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc708108c60 0x7fc70819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.942+0000 7fc70e0a4640 1 --2- 172.21.15.3:0/1373145266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc708108310 0x7fc7081998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.942+0000 7fc70e0a4640 1 --2- 172.21.15.3:0/1373145266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7081029e0 0x7fc708199340 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.942+0000 7fc70e0a4640 1 -- 172.21.15.3:0/1373145266 >> 172.21.15.3:0/1373145266 conn(0x7fc7080fe3c0 msgr2=0x7fc708101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.942+0000 7fc70e0a4640 1 -- 172.21.15.3:0/1373145266 shutdown_connections 2024-09-21T06:21:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:21:56.942+0000 7fc70e0a4640 1 -- 172.21.15.3:0/1373145266 wait complete. 2024-09-21T06:21:56.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 733135 11.38 MB/sec execute 209 sec latency 64.845 ms 2024-09-21T06:21:57.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:21:57.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:57 smithi003 ceph-mon[31227]: pgmap v533: 97 pgs: 97 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 348 op/s 2024-09-21T06:21:57.800 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:57.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:57 smithi137 ceph-mon[34464]: pgmap v533: 97 pgs: 97 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 348 op/s 2024-09-21T06:21:57.895 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:21:57 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e[62522]: ignoring --setuser ceph since I am not root 2024-09-21T06:21:57.895 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:21:57 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e[62522]: ignoring --setgroup ceph since I am not root 2024-09-21T06:21:57.895 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:21:57 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e[62522]: starting mds.e at 2024-09-21T06:21:57.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:57 smithi204 ceph-mon[34659]: pgmap v533: 97 pgs: 97 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 348 op/s 2024-09-21T06:21:57.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 735361 11.39 MB/sec execute 210 sec latency 17.988 ms 2024-09-21T06:21:58.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.291+0000 7f20dd532640 1 -- 172.21.15.3:0/3454009272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d8109df0 msgr2=0x7f20d8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:58.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.291+0000 7f20dd532640 1 --2- 172.21.15.3:0/3454009272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d8109df0 0x7f20d8110620 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009ca0 tx=0x7f20d002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.291+0000 7f20dd532640 1 -- 172.21.15.3:0/3454009272 shutdown_connections 2024-09-21T06:21:58.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.291+0000 7f20dd532640 1 --2- 172.21.15.3:0/3454009272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d8109df0 0x7f20d8110620 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.291+0000 7f20dd532640 1 --2- 172.21.15.3:0/3454009272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20d81094a0 0x7f20d81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.291+0000 7f20dd532640 1 --2- 172.21.15.3:0/3454009272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20d8103b60 0x7f20d8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.291+0000 7f20dd532640 1 -- 172.21.15.3:0/3454009272 >> 172.21.15.3:0/3454009272 conn(0x7f20d807b850 msgr2=0x7f20d807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:58.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.291+0000 7f20dd532640 1 -- 172.21.15.3:0/3454009272 shutdown_connections 2024-09-21T06:21:58.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.291+0000 7f20dd532640 1 -- 172.21.15.3:0/3454009272 wait complete. 2024-09-21T06:21:58.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.292+0000 7f20dd532640 1 Processor -- start 2024-09-21T06:21:58.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.292+0000 7f20dd532640 1 -- start start 2024-09-21T06:21:58.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.292+0000 7f20dd532640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20d8103b60 0x7f20d819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20dd532640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20d81094a0 0x7f20d819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20dd532640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d8109df0 0x7f20d819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20dd532640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f20d8113ab0 con 0x7f20d8109df0 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20dd532640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20d8113930 con 0x7f20d81094a0 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20dd532640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f20d8113c30 con 0x7f20d8103b60 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20d67fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20d81094a0 0x7f20d819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20d77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d8109df0 0x7f20d819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20d67fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20d81094a0 0x7f20d819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:52562/0 (socket says 172.21.15.3:52562) 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20d67fc640 1 -- 172.21.15.3:0/3365914753 learned_addr learned my addr 172.21.15.3:0/3365914753 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20d6ffd640 1 --2- 172.21.15.3:0/3365914753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20d8103b60 0x7f20d819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20d67fc640 1 -- 172.21.15.3:0/3365914753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20d8103b60 msgr2=0x7f20d819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20d67fc640 1 --2- 172.21.15.3:0/3365914753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20d8103b60 0x7f20d819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20d67fc640 1 -- 172.21.15.3:0/3365914753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d8109df0 msgr2=0x7f20d819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20d67fc640 1 --2- 172.21.15.3:0/3365914753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d8109df0 0x7f20d819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20d67fc640 1 -- 172.21.15.3:0/3365914753 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d81a0370 con 0x7f20d81094a0 2024-09-21T06:21:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20d67fc640 1 --2- 172.21.15.3:0/3365914753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20d81094a0 0x7f20d819a9a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00e970 tx=0x7f20cc00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:58.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20bffff640 1 -- 172.21.15.3:0/3365914753 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc00cd80 con 0x7f20d81094a0 2024-09-21T06:21:58.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20dd532640 1 -- 172.21.15.3:0/3365914753 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d81a0660 con 0x7f20d81094a0 2024-09-21T06:21:58.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.293+0000 7f20bffff640 1 -- 172.21.15.3:0/3365914753 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20cc0035f0 con 0x7f20d81094a0 2024-09-21T06:21:58.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.294+0000 7f20bffff640 1 -- 172.21.15.3:0/3365914753 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc010690 con 0x7f20d81094a0 2024-09-21T06:21:58.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.294+0000 7f20dd532640 1 -- 172.21.15.3:0/3365914753 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d81a6dc0 con 0x7f20d81094a0 2024-09-21T06:21:58.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.296+0000 7f20e3aa6740 1 Processor -- start 2024-09-21T06:21:58.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.296+0000 7f20e3aa6740 1 -- start start 2024-09-21T06:21:58.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.296+0000 7f20e3aa6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8465070 0x5641e880c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.296+0000 7f20e3aa6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e843c1b0 0x5641e880cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.296+0000 7f20e3aa6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e8444390 0x5641e880d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.296+0000 7f20e3aa6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5641e8782ea0 con 0x5641e8465070 2024-09-21T06:21:58.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.296+0000 7f20e3aa6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5641e87722d0 con 0x5641e8444390 2024-09-21T06:21:58.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.296+0000 7f20e3aa6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5641e87760e0 con 0x5641e843c1b0 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20d77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8465070 0x5641e880c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20d6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e843c1b0 0x5641e880cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20d77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8465070 0x5641e880c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60990/0 (socket says 172.21.15.3:60990) 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20d77fe640 1 -- 172.21.15.3:0/4049700392 learned_addr learned my addr 172.21.15.3:0/4049700392 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20bffff640 1 -- 172.21.15.3:0/3365914753 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f20cc003aa0 con 0x7f20d81094a0 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20d77fe640 1 -- 172.21.15.3:0/4049700392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e843c1b0 msgr2=0x5641e880cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20d77fe640 1 --2- 172.21.15.3:0/4049700392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e843c1b0 0x5641e880cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20d77fe640 1 -- 172.21.15.3:0/4049700392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e8444390 msgr2=0x5641e880d120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20d77fe640 1 --2- 172.21.15.3:0/4049700392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e8444390 0x5641e880d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20d77fe640 1 -- 172.21.15.3:0/4049700392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641e84d0650 con 0x5641e8465070 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20d77fe640 1 --2- 172.21.15.3:0/4049700392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8465070 0x5641e880c6a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f20d002f4c0 tx=0x7f20d0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba999cea0bb5e63c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20bd7fa640 1 -- 172.21.15.3:0/4049700392 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0031cf0 con 0x5641e8465070 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20bd7fa640 1 -- 172.21.15.3:0/4049700392 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20d0036030 con 0x5641e8465070 2024-09-21T06:21:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.297+0000 7f20bd7fa640 1 -- 172.21.15.3:0/4049700392 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d00042a0 con 0x5641e8465070 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.298+0000 7f20bffff640 1 --2- 172.21.15.3:0/3365914753 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f20c00b6680 0x7f20c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.298+0000 7f20d6ffd640 1 --2- 172.21.15.3:0/3365914753 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f20c00b6680 0x7f20c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.298+0000 7f20bffff640 1 -- 172.21.15.3:0/3365914753 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f20cc0e79a0 con 0x7f20d81094a0 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.298+0000 7f20e3aa6740 1 -- 172.21.15.3:0/4049700392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8465070 msgr2=0x5641e880c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.298+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/4049700392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8465070 0x5641e880c6a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f20d002f4c0 tx=0x7f20d0009800 comp rx=0 tx=0).stop 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.298+0000 7f20e3aa6740 1 -- 172.21.15.3:0/4049700392 shutdown_connections 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.298+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/4049700392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e843c1b0 0x5641e880cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.298+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/4049700392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e8444390 0x5641e880d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.298+0000 7f20d6ffd640 1 --2- 172.21.15.3:0/3365914753 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f20c00b6680 0x7f20c00b8b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x5641e880b320 tx=0x7f20c800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.298+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/4049700392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8465070 0x5641e880c6a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.298+0000 7f20e3aa6740 1 -- 172.21.15.3:0/4049700392 >> 172.21.15.3:0/4049700392 conn(0x5641e8459460 msgr2=0x5641e880a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.298+0000 7f20e3aa6740 1 -- 172.21.15.3:0/4049700392 shutdown_connections 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.298+0000 7f20e3aa6740 1 -- 172.21.15.3:0/4049700392 wait complete. 2024-09-21T06:21:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.299+0000 7f20e3aa6740 1 -- start start 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.299+0000 7f20e3aa6740 1 Processor -- start 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.299+0000 7f20e3aa6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e843c1b0 0x5641e881c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.299+0000 7f20e3aa6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8444390 0x5641e881c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.299+0000 7f20e3aa6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e8465070 0x5641e8828330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.299+0000 7f20e3aa6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5641e85d3700 con 0x5641e8444390 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.299+0000 7f20e3aa6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5641e870d160 con 0x5641e843c1b0 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.299+0000 7f20e3aa6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5641e881f000 con 0x5641e8465070 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.299+0000 7f20e3aa6740 10 client.0 _ll_register_callbacks cb 0x5641e87269e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.299+0000 7f20d67fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8444390 0x5641e881c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.299+0000 7f20d67fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8444390 0x5641e881c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32768/0 (socket says 172.21.15.3:32768) 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.299+0000 7f20d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e843c1b0 0x5641e881c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.299+0000 7f20d67fc640 1 -- 172.21.15.3:0/335936296 learned_addr learned my addr 172.21.15.3:0/335936296 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20d77fe640 1 --2- 172.21.15.3:0/335936296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e8465070 0x5641e8828330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20d67fc640 1 -- 172.21.15.3:0/335936296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e8465070 msgr2=0x5641e8828330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20d67fc640 1 --2- 172.21.15.3:0/335936296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e8465070 0x5641e8828330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20d67fc640 1 -- 172.21.15.3:0/335936296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e843c1b0 msgr2=0x5641e881c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20d67fc640 1 --2- 172.21.15.3:0/335936296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e843c1b0 0x5641e881c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20d67fc640 1 -- 172.21.15.3:0/335936296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e873ac90 con 0x5641e8444390 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20d6ffd640 1 --2- 172.21.15.3:0/335936296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e843c1b0 0x5641e881c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20d67fc640 1 --2- 172.21.15.3:0/335936296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8444390 0x5641e881c680 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00c6a0 tx=0x7f20cc0ad220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b474eed4017792 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20baff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20baff5640 1 -- 172.21.15.3:0/335936296 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc0adf10 con 0x5641e8444390 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20e3aa6740 1 -- 172.21.15.3:0/335936296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20d0036030 con 0x5641e8444390 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20baff5640 1 -- 172.21.15.3:0/335936296 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f20cc0a9160 con 0x5641e8444390 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20baff5640 1 -- 172.21.15.3:0/335936296 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f20cc0b22f0 con 0x5641e8444390 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20e3aa6740 10 client.17520 fetch_fsmap learned FSMap version 33 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.300+0000 7f20e3aa6740 1 -- 172.21.15.3:0/335936296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5641e882acf0 con 0x5641e8444390 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.301+0000 7f20baff5640 1 -- 172.21.15.3:0/335936296 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20d0036030 con 0x5641e8444390 2024-09-21T06:21:58.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.301+0000 7f20baff5640 1 -- 172.21.15.3:0/335936296 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f20cc0b35a0 con 0x5641e8444390 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.301+0000 7f20e3aa6740 10 client.17520 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.301+0000 7f20e3aa6740 10 client.17520 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.301+0000 7f20e3aa6740 20 client.17520 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.301+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/335936296 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x5641e84239f0 0x5641e882e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.301+0000 7f20e3aa6740 4 client.17520 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.301+0000 7f20e3aa6740 1 -- 172.21.15.3:0/335936296 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641e8784df0 con 0x5641e84239f0 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.301+0000 7f20d6ffd640 1 --2- 172.21.15.3:0/335936296 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x5641e84239f0 0x5641e882e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.302+0000 7f20baff5640 10 client.17520 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.302+0000 7f20d6ffd640 1 --2- 172.21.15.3:0/335936296 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x5641e84239f0 0x5641e882e1f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d0e437c72a8f96a server_cookie=6f880d19dc09d22a in_seq=0 out_seq=0 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.302+0000 7f20baff5640 1 -- 172.21.15.3:0/335936296 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5641e8784df0 con 0x5641e84239f0 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.302+0000 7f20baff5640 10 client.17520 handle_command_reply: tid=0 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.302+0000 7f20e3aa6740 1 client.17520 shutdown 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.303+0000 7f20e3aa6740 1 -- 172.21.15.3:0/335936296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8444390 msgr2=0x5641e881c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.303+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/335936296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8444390 0x5641e881c680 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00c6a0 tx=0x7f20cc0ad220 comp rx=0 tx=0).stop 2024-09-21T06:21:58.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.303+0000 7f20e3aa6740 1 -- 172.21.15.3:0/335936296 shutdown_connections 2024-09-21T06:21:58.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.303+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/335936296 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x5641e84239f0 0x5641e882e1f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.303+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/335936296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e8465070 0x5641e8828330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.303+0000 7f20baff5640 0 client.17520 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:21:58.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.303+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/335936296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8444390 0x5641e881c680 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.303+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/335936296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e843c1b0 0x5641e881c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.303+0000 7f20e3aa6740 1 -- 172.21.15.3:0/335936296 >> 172.21.15.3:0/335936296 conn(0x5641e8459460 msgr2=0x5641e8808bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:58.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.303+0000 7f20e3aa6740 1 -- 172.21.15.3:0/335936296 shutdown_connections 2024-09-21T06:21:58.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.303+0000 7f20e3aa6740 1 -- 172.21.15.3:0/335936296 wait complete. 2024-09-21T06:21:58.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.303+0000 7f20e3aa6740 20 client.17520 trim_cache size 0 max 16384 2024-09-21T06:21:58.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.316+0000 7f20e3aa6740 1 Processor -- start 2024-09-21T06:21:58.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.316+0000 7f20e3aa6740 1 -- start start 2024-09-21T06:21:58.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20e3aa6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e84239f0 0x5641e881b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20e3aa6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e843c1b0 0x5641e8811f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20e3aa6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e8444390 0x5641e8819330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20e3aa6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81ad330 con 0x5641e84239f0 2024-09-21T06:21:58.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20e3aa6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81aee70 con 0x5641e843c1b0 2024-09-21T06:21:58.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20e3aa6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81ada00 con 0x5641e8444390 2024-09-21T06:21:58.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20d6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e84239f0 0x5641e881b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.429 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:21:58.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20d6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e84239f0 0x5641e881b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32772/0 (socket says 172.21.15.3:32772) 2024-09-21T06:21:58.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20d6ffd640 1 -- 172.21.15.3:0/3050459140 learned_addr learned my addr 172.21.15.3:0/3050459140 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:58.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20d67fc640 1 --2- 172.21.15.3:0/3050459140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e843c1b0 0x5641e8811f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20d77fe640 1 --2- 172.21.15.3:0/3050459140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e8444390 0x5641e8819330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20d6ffd640 1 -- 172.21.15.3:0/3050459140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e8444390 msgr2=0x5641e8819330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:58.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20d6ffd640 1 --2- 172.21.15.3:0/3050459140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e8444390 0x5641e8819330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20d6ffd640 1 -- 172.21.15.3:0/3050459140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e843c1b0 msgr2=0x5641e8811f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:58.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20d6ffd640 1 --2- 172.21.15.3:0/3050459140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e843c1b0 0x5641e8811f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20d6ffd640 1 -- 172.21.15.3:0/3050459140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641e881a640 con 0x5641e84239f0 2024-09-21T06:21:58.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.317+0000 7f20d6ffd640 1 --2- 172.21.15.3:0/3050459140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e84239f0 0x5641e881b030 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f20c80043f0 tx=0x7f20c8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ecfcd311e2ce2c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:58.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.318+0000 7f20bcff9640 1 -- 172.21.15.3:0/3050459140 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c801ea90 con 0x5641e84239f0 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.318+0000 7f20bcff9640 1 -- 172.21.15.3:0/3050459140 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20c801ec30 con 0x5641e84239f0 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.318+0000 7f20bcff9640 1 -- 172.21.15.3:0/3050459140 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c801d700 con 0x5641e84239f0 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.319+0000 7f20e3aa6740 1 -- 172.21.15.3:0/3050459140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e84239f0 msgr2=0x5641e881b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.319+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/3050459140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e84239f0 0x5641e881b030 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f20c80043f0 tx=0x7f20c8004610 comp rx=0 tx=0).stop 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.319+0000 7f20e3aa6740 1 -- 172.21.15.3:0/3050459140 shutdown_connections 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.319+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/3050459140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e8444390 0x5641e8819330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.319+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/3050459140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e843c1b0 0x5641e8811f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.319+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/3050459140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e84239f0 0x5641e881b030 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.319+0000 7f20e3aa6740 1 -- 172.21.15.3:0/3050459140 >> 172.21.15.3:0/3050459140 conn(0x5641e8459460 msgr2=0x5641e8811840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.319+0000 7f20e3aa6740 1 -- 172.21.15.3:0/3050459140 shutdown_connections 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.319+0000 7f20e3aa6740 1 -- 172.21.15.3:0/3050459140 wait complete. 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.319+0000 7f20e3aa6740 1 -- start start 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.319+0000 7f20e3aa6740 1 Processor -- start 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20e3aa6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e84239f0 0x5641e8820480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20e3aa6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e843c1b0 0x5641e88209c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20e3aa6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8444390 0x5641e8820f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20e3aa6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81ae7a0 con 0x5641e8444390 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20e3aa6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81ae0d0 con 0x5641e84239f0 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20e3aa6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81ab310 con 0x5641e843c1b0 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20e3aa6740 10 client.0 _ll_register_callbacks cb 0x7f20cc0ad5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e84239f0 0x5641e8820480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20d67fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e843c1b0 0x5641e88209c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20d77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8444390 0x5641e8820f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e84239f0 0x5641e8820480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:52596/0 (socket says 172.21.15.3:52596) 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20d67fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e843c1b0 0x5641e88209c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:36596/0 (socket says 172.21.15.3:36596) 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20d6ffd640 1 -- 172.21.15.3:0/1698588853 learned_addr learned my addr 172.21.15.3:0/1698588853 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20d77fe640 1 -- 172.21.15.3:0/1698588853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e843c1b0 msgr2=0x5641e88209c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20d77fe640 1 --2- 172.21.15.3:0/1698588853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e843c1b0 0x5641e88209c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20d77fe640 1 -- 172.21.15.3:0/1698588853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e84239f0 msgr2=0x5641e8820480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20d77fe640 1 --2- 172.21.15.3:0/1698588853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e84239f0 0x5641e8820480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20d77fe640 1 -- 172.21.15.3:0/1698588853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e8897cf0 con 0x5641e8444390 2024-09-21T06:21:58.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20d67fc640 1 --2- 172.21.15.3:0/1698588853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e843c1b0 0x5641e88209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:58.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.320+0000 7f20d6ffd640 1 --2- 172.21.15.3:0/1698588853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e84239f0 0x5641e8820480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:58.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20d77fe640 1 --2- 172.21.15.3:0/1698588853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8444390 0x5641e8820f00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f20d0005e30 tx=0x7f20d0031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92663c3a2261787a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:58.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20baff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:21:58.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20baff5640 1 -- 172.21.15.3:0/1698588853 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0055020 con 0x5641e8444390 2024-09-21T06:21:58.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20baff5640 1 -- 172.21.15.3:0/1698588853 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8463+0+0 (secure 0 0 0) 0x7f20d0031650 con 0x5641e8444390 2024-09-21T06:21:58.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20e3aa6740 1 -- 172.21.15.3:0/1698588853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c801ec30 con 0x5641e8444390 2024-09-21T06:21:58.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20baff5640 1 -- 172.21.15.3:0/1698588853 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f20d004a040 con 0x5641e8444390 2024-09-21T06:21:58.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20e3aa6740 10 client.17526 fetch_fsmap learned FSMap version 33 2024-09-21T06:21:58.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20e3aa6740 1 -- 172.21.15.3:0/1698588853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5641e8898f30 con 0x5641e8444390 2024-09-21T06:21:58.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20baff5640 1 -- 172.21.15.3:0/1698588853 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20c801ec30 con 0x5641e8444390 2024-09-21T06:21:58.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20baff5640 1 -- 172.21.15.3:0/1698588853 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f20d0043070 con 0x5641e8444390 2024-09-21T06:21:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20e3aa6740 10 client.17526 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:21:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20e3aa6740 10 client.17526 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:21:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20e3aa6740 20 client.17526 populate_metadata read hostname 'smithi003' 2024-09-21T06:21:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/1698588853 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x5641e8465070 0x5641e889b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20e3aa6740 4 client.17526 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:21:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.321+0000 7f20e3aa6740 1 -- 172.21.15.3:0/1698588853 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641e87808a0 con 0x5641e8465070 2024-09-21T06:21:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.322+0000 7f20d6ffd640 1 --2- 172.21.15.3:0/1698588853 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x5641e8465070 0x5641e889b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.322+0000 7f20d6ffd640 1 --2- 172.21.15.3:0/1698588853 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x5641e8465070 0x5641e889b4b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=890e700c8185213e server_cookie=96c7d818595b66ca in_seq=0 out_seq=0 2024-09-21T06:21:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.322+0000 7f20baff5640 10 client.17526 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:21:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.322+0000 7f20baff5640 1 -- 172.21.15.3:0/1698588853 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641e87808a0 con 0x5641e8465070 2024-09-21T06:21:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.322+0000 7f20baff5640 10 client.17526 handle_command_reply: tid=0 2024-09-21T06:21:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.322+0000 7f20e3aa6740 1 client.17526 shutdown 2024-09-21T06:21:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.322+0000 7f20e3aa6740 1 -- 172.21.15.3:0/1698588853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8444390 msgr2=0x5641e8820f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.322+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/1698588853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8444390 0x5641e8820f00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f20d0005e30 tx=0x7f20d0031d20 comp rx=0 tx=0).stop 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20e3aa6740 1 -- 172.21.15.3:0/1698588853 shutdown_connections 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/1698588853 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x5641e8465070 0x5641e889b4b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/1698588853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641e843c1b0 0x5641e88209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/1698588853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5641e8444390 0x5641e8820f00 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20e3aa6740 1 --2- 172.21.15.3:0/1698588853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641e84239f0 0x5641e8820480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20e3aa6740 1 -- 172.21.15.3:0/1698588853 >> 172.21.15.3:0/1698588853 conn(0x5641e8459460 msgr2=0x5641e880d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20baff5640 0 client.17526 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20e3aa6740 1 -- 172.21.15.3:0/1698588853 shutdown_connections 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20e3aa6740 1 -- 172.21.15.3:0/1698588853 wait complete. 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20e3aa6740 20 client.17526 trim_cache size 0 max 16384 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20dd532640 1 -- 172.21.15.3:0/3365914753 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f20c00b6680 msgr2=0x7f20c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20dd532640 1 --2- 172.21.15.3:0/3365914753 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f20c00b6680 0x7f20c00b8b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x5641e880b320 tx=0x7f20c800b390 comp rx=0 tx=0).stop 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20dd532640 1 -- 172.21.15.3:0/3365914753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20d81094a0 msgr2=0x7f20d819a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:58.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20dd532640 1 --2- 172.21.15.3:0/3365914753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20d81094a0 0x7f20d819a9a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00e970 tx=0x7f20cc00ee30 comp rx=0 tx=0).stop 2024-09-21T06:21:58.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20dd532640 1 -- 172.21.15.3:0/3365914753 shutdown_connections 2024-09-21T06:21:58.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20dd532640 1 --2- 172.21.15.3:0/3365914753 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f20c00b6680 0x7f20c00b8b90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20dd532640 1 --2- 172.21.15.3:0/3365914753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d8109df0 0x7f20d819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20dd532640 1 --2- 172.21.15.3:0/3365914753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20d81094a0 0x7f20d819a9a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20dd532640 1 --2- 172.21.15.3:0/3365914753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20d8103b60 0x7f20d819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:58.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.323+0000 7f20dd532640 1 -- 172.21.15.3:0/3365914753 >> 172.21.15.3:0/3365914753 conn(0x7f20d807b850 msgr2=0x7f20d8105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:58.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.324+0000 7f20dd532640 1 -- 172.21.15.3:0/3365914753 shutdown_connections 2024-09-21T06:21:58.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:21:58.324+0000 7f20dd532640 1 -- 172.21.15.3:0/3365914753 wait complete. 2024-09-21T06:21:58.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:58 smithi003 ceph-mon[31227]: daemon mds.e restarted 2024-09-21T06:21:58.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:58 smithi137 ceph-mon[34464]: daemon mds.e restarted 2024-09-21T06:21:58.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:58 smithi204 ceph-mon[34659]: daemon mds.e restarted 2024-09-21T06:21:58.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 736677 11.36 MB/sec execute 211 sec latency 71.515 ms 2024-09-21T06:21:59.246 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:21:59.507 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:59 smithi003 ceph-mon[31227]: pgmap v534: 97 pgs: 97 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 275 op/s 2024-09-21T06:21:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:59 smithi003 ceph-mon[31227]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:21:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:59 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.137:6834/2984446797,v1:172.21.15.137:6835/2984446797] up:boot 2024-09-21T06:21:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:59 smithi003 ceph-mon[31227]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T06:21:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:59 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:21:59.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:21:59 smithi003 ceph-mon[31227]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:21:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:59 smithi137 ceph-mon[34464]: pgmap v534: 97 pgs: 97 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 275 op/s 2024-09-21T06:21:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:59 smithi137 ceph-mon[34464]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:21:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:59 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.137:6834/2984446797,v1:172.21.15.137:6835/2984446797] up:boot 2024-09-21T06:21:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:59 smithi137 ceph-mon[34464]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T06:21:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:59 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:21:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:21:59 smithi137 ceph-mon[34464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:21:59.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:59 smithi204 ceph-mon[34659]: pgmap v534: 97 pgs: 97 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 275 op/s 2024-09-21T06:21:59.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:59 smithi204 ceph-mon[34659]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:21:59.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:59 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.137:6834/2984446797,v1:172.21.15.137:6835/2984446797] up:boot 2024-09-21T06:21:59.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:59 smithi204 ceph-mon[34659]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T06:21:59.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:59 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:21:59.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:21:59 smithi204 ceph-mon[34659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:21:59.963 INFO:tasks.workunit.client.0.smithi003.stdout: 1 738894 11.37 MB/sec execute 212 sec latency 26.294 ms 2024-09-21T06:22:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.085+0000 7f94846a7640 1 -- 172.21.15.3:0/1010319281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f947c109e10 msgr2=0x7f947c110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.085+0000 7f94846a7640 1 --2- 172.21.15.3:0/1010319281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f947c109e10 0x7f947c110640 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f946c009ca0 tx=0x7f946c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.085+0000 7f94846a7640 1 -- 172.21.15.3:0/1010319281 shutdown_connections 2024-09-21T06:22:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.085+0000 7f94846a7640 1 --2- 172.21.15.3:0/1010319281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f947c109e10 0x7f947c110640 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.085+0000 7f94846a7640 1 --2- 172.21.15.3:0/1010319281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f947c1094c0 0x7f947c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.085+0000 7f94846a7640 1 --2- 172.21.15.3:0/1010319281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f947c103b80 0x7f947c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.085+0000 7f94846a7640 1 -- 172.21.15.3:0/1010319281 >> 172.21.15.3:0/1010319281 conn(0x7f947c0ff610 msgr2=0x7f947c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.085+0000 7f94846a7640 1 -- 172.21.15.3:0/1010319281 shutdown_connections 2024-09-21T06:22:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.085+0000 7f94846a7640 1 -- 172.21.15.3:0/1010319281 wait complete. 2024-09-21T06:22:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.086+0000 7f94846a7640 1 Processor -- start 2024-09-21T06:22:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.086+0000 7f94846a7640 1 -- start start 2024-09-21T06:22:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.086+0000 7f94846a7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f947c103b80 0x7f947c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.086+0000 7f94846a7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f947c1094c0 0x7f947c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.086+0000 7f94846a7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f947c109e10 0x7f947c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.086+0000 7f94846a7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f947c113ad0 con 0x7f947c1094c0 2024-09-21T06:22:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.086+0000 7f94846a7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f947c113950 con 0x7f947c109e10 2024-09-21T06:22:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.086+0000 7f94846a7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f947c113c50 con 0x7f947c103b80 2024-09-21T06:22:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.086+0000 7f9481c42640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f947c1094c0 0x7f947c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.086+0000 7f9481c42640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f947c1094c0 0x7f947c19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32808/0 (socket says 172.21.15.3:32808) 2024-09-21T06:22:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.086+0000 7f9481c42640 1 -- 172.21.15.3:0/657618294 learned_addr learned my addr 172.21.15.3:0/657618294 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.086+0000 7f9482c44640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f947c109e10 0x7f947c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.086+0000 7f9482443640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f947c103b80 0x7f947c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.087+0000 7f9481c42640 1 -- 172.21.15.3:0/657618294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f947c103b80 msgr2=0x7f947c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.087+0000 7f9481c42640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f947c103b80 0x7f947c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.087+0000 7f9481c42640 1 -- 172.21.15.3:0/657618294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f947c109e10 msgr2=0x7f947c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.087+0000 7f9481c42640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f947c109e10 0x7f947c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.087+0000 7f9481c42640 1 -- 172.21.15.3:0/657618294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947c1a0470 con 0x7f947c1094c0 2024-09-21T06:22:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.087+0000 7f9481c42640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f947c1094c0 0x7f947c19aaa0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f947000e9f0 tx=0x7f947000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.087+0000 7f947b7fe640 1 -- 172.21.15.3:0/657618294 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947000ccc0 con 0x7f947c1094c0 2024-09-21T06:22:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.087+0000 7f94846a7640 1 -- 172.21.15.3:0/657618294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947c1a0760 con 0x7f947c1094c0 2024-09-21T06:22:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.087+0000 7f947b7fe640 1 -- 172.21.15.3:0/657618294 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94700108e0 con 0x7f947c1094c0 2024-09-21T06:22:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.087+0000 7f947b7fe640 1 -- 172.21.15.3:0/657618294 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470018860 con 0x7f947c1094c0 2024-09-21T06:22:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.087+0000 7f94846a7640 1 -- 172.21.15.3:0/657618294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947c1a6ec0 con 0x7f947c1094c0 2024-09-21T06:22:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.091+0000 7f948ac1b740 1 Processor -- start 2024-09-21T06:22:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.091+0000 7f948ac1b740 1 -- start start 2024-09-21T06:22:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.091+0000 7f947b7fe640 1 -- 172.21.15.3:0/657618294 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f9470010430 con 0x7f947c1094c0 2024-09-21T06:22:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.091+0000 7f948ac1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7e0070 0x55771ab872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.091+0000 7f948ac1b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7b71b0 0x55771ab87800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.091+0000 7f948ac1b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab87d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.091+0000 7f948ac1b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55771aafdea0 con 0x55771a7bf390 2024-09-21T06:22:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.091+0000 7f948ac1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55771aaed2d0 con 0x55771a7e0070 2024-09-21T06:22:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.091+0000 7f948ac1b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55771aaf10e0 con 0x55771a7b71b0 2024-09-21T06:22:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.091+0000 7f9481c42640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab87d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.091+0000 7f9482c44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7e0070 0x55771ab872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.091+0000 7f9482c44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7e0070 0x55771ab872c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:52636/0 (socket says 172.21.15.3:52636) 2024-09-21T06:22:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.091+0000 7f9482c44640 1 -- 172.21.15.3:0/2044564194 learned_addr learned my addr 172.21.15.3:0/2044564194 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.092+0000 7f9482443640 1 --2- 172.21.15.3:0/2044564194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7b71b0 0x55771ab87800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.092+0000 7f9482c44640 1 -- 172.21.15.3:0/2044564194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7b71b0 msgr2=0x55771ab87800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.092+0000 7f9482c44640 1 --2- 172.21.15.3:0/2044564194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7b71b0 0x55771ab87800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.092+0000 7f9482c44640 1 -- 172.21.15.3:0/2044564194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 msgr2=0x55771ab87d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.092+0000 7f9482c44640 1 --2- 172.21.15.3:0/2044564194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab87d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.092+0000 7f9482c44640 1 -- 172.21.15.3:0/2044564194 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55771a84b650 con 0x55771a7e0070 2024-09-21T06:22:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.092+0000 7f9482c44640 1 --2- 172.21.15.3:0/2044564194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7e0070 0x55771ab872c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f946c02f4c0 tx=0x7f946c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7ccae5d5e680fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.092+0000 7f947b7fe640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f94640b6680 0x7f94640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.092+0000 7f9478ff9640 1 -- 172.21.15.3:0/2044564194 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c031cf0 con 0x55771a7e0070 2024-09-21T06:22:00.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.092+0000 7f9482443640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f94640b6680 0x7f94640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.092+0000 7f947b7fe640 1 -- 172.21.15.3:0/657618294 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8463+0+0 (secure 0 0 0) 0x7f94700e7910 con 0x7f947c1094c0 2024-09-21T06:22:00.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.092+0000 7f9478ff9640 1 -- 172.21.15.3:0/2044564194 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f946c036030 con 0x55771a7e0070 2024-09-21T06:22:00.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.092+0000 7f9478ff9640 1 -- 172.21.15.3:0/2044564194 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c0042a0 con 0x55771a7e0070 2024-09-21T06:22:00.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.093+0000 7f948ac1b740 1 -- 172.21.15.3:0/2044564194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7e0070 msgr2=0x55771ab872c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.093+0000 7f9482443640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f94640b6680 0x7f94640b8b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55771ab85f40 tx=0x7f947400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:00.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.093+0000 7f948ac1b740 1 --2- 172.21.15.3:0/2044564194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7e0070 0x55771ab872c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f946c02f4c0 tx=0x7f946c009800 comp rx=0 tx=0).stop 2024-09-21T06:22:00.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.093+0000 7f948ac1b740 1 -- 172.21.15.3:0/2044564194 shutdown_connections 2024-09-21T06:22:00.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.093+0000 7f948ac1b740 1 --2- 172.21.15.3:0/2044564194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7b71b0 0x55771ab87800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.093+0000 7f948ac1b740 1 --2- 172.21.15.3:0/2044564194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab87d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.093+0000 7f948ac1b740 1 --2- 172.21.15.3:0/2044564194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7e0070 0x55771ab872c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.093+0000 7f948ac1b740 1 -- 172.21.15.3:0/2044564194 >> 172.21.15.3:0/2044564194 conn(0x55771a7d4460 msgr2=0x55771ab851b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:00.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f948ac1b740 1 -- 172.21.15.3:0/2044564194 shutdown_connections 2024-09-21T06:22:00.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f948ac1b740 1 -- 172.21.15.3:0/2044564194 wait complete. 2024-09-21T06:22:00.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f948ac1b740 1 -- start start 2024-09-21T06:22:00.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f948ac1b740 1 Processor -- start 2024-09-21T06:22:00.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f948ac1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7b71b0 0x55771ab92920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f948ac1b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab92e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f948ac1b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7e0070 0x55771ab9ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f948ac1b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55771a94e700 con 0x55771a7bf390 2024-09-21T06:22:00.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f948ac1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55771aa88160 con 0x55771a7b71b0 2024-09-21T06:22:00.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f948ac1b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55771ab958c0 con 0x55771a7e0070 2024-09-21T06:22:00.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f948ac1b740 10 client.0 _ll_register_callbacks cb 0x55771aaa19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:00.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f9481c42640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab92e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f9481c42640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab92e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32816/0 (socket says 172.21.15.3:32816) 2024-09-21T06:22:00.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f9481c42640 1 -- 172.21.15.3:0/4294420120 learned_addr learned my addr 172.21.15.3:0/4294420120 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:00.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.094+0000 7f9482443640 1 --2- 172.21.15.3:0/4294420120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7b71b0 0x55771ab92920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.095+0000 7f9482c44640 1 --2- 172.21.15.3:0/4294420120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7e0070 0x55771ab9ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.095+0000 7f9481c42640 1 -- 172.21.15.3:0/4294420120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7e0070 msgr2=0x55771ab9ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:00.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.095+0000 7f9481c42640 1 --2- 172.21.15.3:0/4294420120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7e0070 0x55771ab9ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.095+0000 7f9481c42640 1 -- 172.21.15.3:0/4294420120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7b71b0 msgr2=0x55771ab92920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:00.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.095+0000 7f9481c42640 1 --2- 172.21.15.3:0/4294420120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7b71b0 0x55771ab92920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.095+0000 7f9481c42640 1 -- 172.21.15.3:0/4294420120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55771aab5c90 con 0x55771a7bf390 2024-09-21T06:22:00.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.095+0000 7f9481c42640 1 --2- 172.21.15.3:0/4294420120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab92e60 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f94700e5100 tx=0x7f947000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f060b288f759654 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:00.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.095+0000 7f9461ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.095+0000 7f9461ffb640 1 -- 172.21.15.3:0/4294420120 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94700a9ea0 con 0x55771a7bf390 2024-09-21T06:22:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.095+0000 7f9461ffb640 1 -- 172.21.15.3:0/4294420120 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8463+0+0 (secure 0 0 0) 0x7f94700ac520 con 0x55771a7bf390 2024-09-21T06:22:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.095+0000 7f948ac1b740 1 -- 172.21.15.3:0/4294420120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f946c036030 con 0x55771a7bf390 2024-09-21T06:22:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.096+0000 7f9461ffb640 1 -- 172.21.15.3:0/4294420120 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f94700acfc0 con 0x55771a7bf390 2024-09-21T06:22:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.096+0000 7f948ac1b740 10 client.17538 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.096+0000 7f948ac1b740 1 -- 172.21.15.3:0/4294420120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55771aba5a10 con 0x55771a7bf390 2024-09-21T06:22:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.096+0000 7f9461ffb640 1 -- 172.21.15.3:0/4294420120 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f946c036030 con 0x55771a7bf390 2024-09-21T06:22:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.096+0000 7f9461ffb640 1 -- 172.21.15.3:0/4294420120 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f94700ac720 con 0x55771a7bf390 2024-09-21T06:22:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.096+0000 7f948ac1b740 10 client.17538 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:00.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.096+0000 7f948ac1b740 10 client.17538 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:22:00.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.096+0000 7f948ac1b740 20 client.17538 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:00.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.096+0000 7f948ac1b740 1 --2- 172.21.15.3:0/4294420120 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55771a79e9f0 0x55771aba8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.096+0000 7f948ac1b740 4 client.17538 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:00.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.096+0000 7f948ac1b740 1 -- 172.21.15.3:0/4294420120 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55771aaffdf0 con 0x55771a79e9f0 2024-09-21T06:22:00.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.096+0000 7f9482443640 1 --2- 172.21.15.3:0/4294420120 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55771a79e9f0 0x55771aba8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.097+0000 7f9461ffb640 10 client.17538 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:22:00.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.097+0000 7f9482443640 1 --2- 172.21.15.3:0/4294420120 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55771a79e9f0 0x55771aba8f20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae4d47e46e237588 server_cookie=f40c3635cb491a14 in_seq=0 out_seq=0 2024-09-21T06:22:00.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f9461ffb640 1 -- 172.21.15.3:0/4294420120 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55771aaffdf0 con 0x55771a79e9f0 2024-09-21T06:22:00.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f9461ffb640 10 client.17538 handle_command_reply: tid=0 2024-09-21T06:22:00.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f948ac1b740 1 client.17538 shutdown 2024-09-21T06:22:00.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f948ac1b740 1 -- 172.21.15.3:0/4294420120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 msgr2=0x55771ab92e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f948ac1b740 1 --2- 172.21.15.3:0/4294420120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab92e60 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f94700e5100 tx=0x7f947000ce60 comp rx=0 tx=0).stop 2024-09-21T06:22:00.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f948ac1b740 1 -- 172.21.15.3:0/4294420120 shutdown_connections 2024-09-21T06:22:00.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f948ac1b740 1 --2- 172.21.15.3:0/4294420120 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55771a79e9f0 0x55771aba8f20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f948ac1b740 1 --2- 172.21.15.3:0/4294420120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7e0070 0x55771ab9ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f948ac1b740 1 --2- 172.21.15.3:0/4294420120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab92e60 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f9461ffb640 0 client.17538 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:22:00.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f948ac1b740 1 --2- 172.21.15.3:0/4294420120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7b71b0 0x55771ab92920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f948ac1b740 1 -- 172.21.15.3:0/4294420120 >> 172.21.15.3:0/4294420120 conn(0x55771a7d4460 msgr2=0x55771ab8a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:00.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f948ac1b740 1 -- 172.21.15.3:0/4294420120 shutdown_connections 2024-09-21T06:22:00.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f948ac1b740 1 -- 172.21.15.3:0/4294420120 wait complete. 2024-09-21T06:22:00.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.098+0000 7f948ac1b740 20 client.17538 trim_cache size 0 max 16384 2024-09-21T06:22:00.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.113+0000 7f948ac1b740 1 Processor -- start 2024-09-21T06:22:00.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.113+0000 7f948ac1b740 1 -- start start 2024-09-21T06:22:00.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.113+0000 7f948ac1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a79e9f0 0x55771ab94550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.113+0000 7f948ac1b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7b71b0 0x55771ab84760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.113+0000 7f948ac1b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab84ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.113+0000 7f948ac1b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1ad450 con 0x55771a7bf390 2024-09-21T06:22:00.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.113+0000 7f948ac1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1aef90 con 0x55771a79e9f0 2024-09-21T06:22:00.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.113+0000 7f948ac1b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1adb20 con 0x55771a7b71b0 2024-09-21T06:22:00.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9482c44640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab84ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9482443640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a79e9f0 0x55771ab94550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9482c44640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab84ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32822/0 (socket says 172.21.15.3:32822) 2024-09-21T06:22:00.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9482c44640 1 -- 172.21.15.3:0/266708884 learned_addr learned my addr 172.21.15.3:0/266708884 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:00.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9481c42640 1 --2- 172.21.15.3:0/266708884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7b71b0 0x55771ab84760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9482c44640 1 -- 172.21.15.3:0/266708884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7b71b0 msgr2=0x55771ab84760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:00.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9482c44640 1 --2- 172.21.15.3:0/266708884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7b71b0 0x55771ab84760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9482c44640 1 -- 172.21.15.3:0/266708884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a79e9f0 msgr2=0x55771ab94550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:00.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9482c44640 1 --2- 172.21.15.3:0/266708884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a79e9f0 0x55771ab94550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9482c44640 1 -- 172.21.15.3:0/266708884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55771ab939c0 con 0x55771a7bf390 2024-09-21T06:22:00.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9481c42640 1 --2- 172.21.15.3:0/266708884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7b71b0 0x55771ab84760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:00.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9482443640 1 --2- 172.21.15.3:0/266708884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a79e9f0 0x55771ab94550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:00.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9482c44640 1 --2- 172.21.15.3:0/266708884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab84ca0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f946c02f4c0 tx=0x7f946c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fe6de861cc88a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:00.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.114+0000 7f9463fff640 1 -- 172.21.15.3:0/266708884 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c04a040 con 0x55771a7bf390 2024-09-21T06:22:00.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.115+0000 7f9463fff640 1 -- 172.21.15.3:0/266708884 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f946c036040 con 0x55771a7bf390 2024-09-21T06:22:00.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.115+0000 7f9463fff640 1 -- 172.21.15.3:0/266708884 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c004040 con 0x55771a7bf390 2024-09-21T06:22:00.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.115+0000 7f948ac1b740 1 -- 172.21.15.3:0/266708884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 msgr2=0x55771ab84ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.115+0000 7f948ac1b740 1 --2- 172.21.15.3:0/266708884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab84ca0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f946c02f4c0 tx=0x7f946c000f40 comp rx=0 tx=0).stop 2024-09-21T06:22:00.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.115+0000 7f948ac1b740 1 -- 172.21.15.3:0/266708884 shutdown_connections 2024-09-21T06:22:00.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.115+0000 7f948ac1b740 1 --2- 172.21.15.3:0/266708884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a7b71b0 0x55771ab84760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.115+0000 7f948ac1b740 1 --2- 172.21.15.3:0/266708884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7bf390 0x55771ab84ca0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.115+0000 7f948ac1b740 1 --2- 172.21.15.3:0/266708884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a79e9f0 0x55771ab94550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.116+0000 7f948ac1b740 1 -- 172.21.15.3:0/266708884 >> 172.21.15.3:0/266708884 conn(0x55771a7d4460 msgr2=0x55771ab93e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:00.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.116+0000 7f948ac1b740 1 -- 172.21.15.3:0/266708884 shutdown_connections 2024-09-21T06:22:00.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.116+0000 7f948ac1b740 1 -- 172.21.15.3:0/266708884 wait complete. 2024-09-21T06:22:00.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.116+0000 7f948ac1b740 1 -- start start 2024-09-21T06:22:00.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.116+0000 7f948ac1b740 1 Processor -- start 2024-09-21T06:22:00.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.116+0000 7f948ac1b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a79e9f0 0x55771ab96d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.116+0000 7f948ac1b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7b71b0 0x55771ab97280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.116+0000 7f948ac1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7bf390 0x55771ab977e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.116+0000 7f948ac1b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1ae8c0 con 0x55771a7b71b0 2024-09-21T06:22:00.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f948ac1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1ae1f0 con 0x55771a7bf390 2024-09-21T06:22:00.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f948ac1b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1ab430 con 0x55771a79e9f0 2024-09-21T06:22:00.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f948ac1b740 10 client.0 _ll_register_callbacks cb 0x7f94700a7f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:00.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f9482443640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a79e9f0 0x55771ab96d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f9482c44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7bf390 0x55771ab977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f9482443640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a79e9f0 0x55771ab96d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:36658/0 (socket says 172.21.15.3:36658) 2024-09-21T06:22:00.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f9482443640 1 -- 172.21.15.3:0/3636690144 learned_addr learned my addr 172.21.15.3:0/3636690144 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:00.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f9482443640 1 -- 172.21.15.3:0/3636690144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7bf390 msgr2=0x55771ab977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:00.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f9482443640 1 --2- 172.21.15.3:0/3636690144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7bf390 0x55771ab977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f9482443640 1 -- 172.21.15.3:0/3636690144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7b71b0 msgr2=0x55771ab97280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:00.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f9482443640 1 --2- 172.21.15.3:0/3636690144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7b71b0 0x55771ab97280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f9482443640 1 -- 172.21.15.3:0/3636690144 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55771ac12b10 con 0x55771a79e9f0 2024-09-21T06:22:00.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f9482443640 1 --2- 172.21.15.3:0/3636690144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a79e9f0 0x55771ab96d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9474005fd0 tx=0x7f94740046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=218614bbcd5e9662 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:00.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.117+0000 7f9461ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:22:00.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.118+0000 7f9461ffb640 1 -- 172.21.15.3:0/3636690144 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474035060 con 0x55771a79e9f0 2024-09-21T06:22:00.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.118+0000 7f9461ffb640 1 -- 172.21.15.3:0/3636690144 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8463+0+0 (secure 0 0 0) 0x7f9474017030 con 0x55771a79e9f0 2024-09-21T06:22:00.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.118+0000 7f948ac1b740 1 -- 172.21.15.3:0/3636690144 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f946c036040 con 0x55771a79e9f0 2024-09-21T06:22:00.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.118+0000 7f9461ffb640 1 -- 172.21.15.3:0/3636690144 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f947401eaa0 con 0x55771a79e9f0 2024-09-21T06:22:00.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.118+0000 7f948ac1b740 10 client.27055 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:00.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.118+0000 7f948ac1b740 1 -- 172.21.15.3:0/3636690144 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55771ac13e20 con 0x55771a79e9f0 2024-09-21T06:22:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.118+0000 7f9461ffb640 1 -- 172.21.15.3:0/3636690144 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f946c036040 con 0x55771a79e9f0 2024-09-21T06:22:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.118+0000 7f9461ffb640 1 -- 172.21.15.3:0/3636690144 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f9474011440 con 0x55771a79e9f0 2024-09-21T06:22:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.118+0000 7f948ac1b740 10 client.27055 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.118+0000 7f948ac1b740 10 client.27055 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:22:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.118+0000 7f948ac1b740 20 client.27055 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f948ac1b740 1 --2- 172.21.15.3:0/3636690144 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55771a7e0070 0x55771ac165f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f948ac1b740 4 client.27055 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:22:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f948ac1b740 1 -- 172.21.15.3:0/3636690144 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55771aafb8a0 con 0x55771a7e0070 2024-09-21T06:22:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f9481c42640 1 --2- 172.21.15.3:0/3636690144 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55771a7e0070 0x55771ac165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f9461ffb640 10 client.27055 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:22:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f9481c42640 1 --2- 172.21.15.3:0/3636690144 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55771a7e0070 0x55771ac165f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35fe0b2b1b26f61c server_cookie=dfb9ce1a40309923 in_seq=0 out_seq=0 2024-09-21T06:22:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f9461ffb640 1 -- 172.21.15.3:0/3636690144 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55771aafb8a0 con 0x55771a7e0070 2024-09-21T06:22:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f9461ffb640 10 client.27055 handle_command_reply: tid=0 2024-09-21T06:22:00.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f948ac1b740 1 client.27055 shutdown 2024-09-21T06:22:00.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f948ac1b740 1 -- 172.21.15.3:0/3636690144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a79e9f0 msgr2=0x55771ab96d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f948ac1b740 1 --2- 172.21.15.3:0/3636690144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a79e9f0 0x55771ab96d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9474005fd0 tx=0x7f94740046c0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f948ac1b740 1 -- 172.21.15.3:0/3636690144 shutdown_connections 2024-09-21T06:22:00.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f948ac1b740 1 --2- 172.21.15.3:0/3636690144 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55771a7e0070 0x55771ac165f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f948ac1b740 1 --2- 172.21.15.3:0/3636690144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55771a7bf390 0x55771ab977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f9461ffb640 0 client.27055 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:22:00.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.119+0000 7f948ac1b740 1 --2- 172.21.15.3:0/3636690144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55771a7b71b0 0x55771ab97280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.120+0000 7f948ac1b740 1 --2- 172.21.15.3:0/3636690144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55771a79e9f0 0x55771ab96d40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.120+0000 7f948ac1b740 1 -- 172.21.15.3:0/3636690144 >> 172.21.15.3:0/3636690144 conn(0x55771a7d4460 msgr2=0x55771ab93be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:00.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.120+0000 7f948ac1b740 1 -- 172.21.15.3:0/3636690144 shutdown_connections 2024-09-21T06:22:00.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.120+0000 7f948ac1b740 1 -- 172.21.15.3:0/3636690144 wait complete. 2024-09-21T06:22:00.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.120+0000 7f948ac1b740 20 client.27055 trim_cache size 0 max 16384 2024-09-21T06:22:00.128 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:22:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.120+0000 7f94846a7640 1 -- 172.21.15.3:0/657618294 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f94640b6680 msgr2=0x7f94640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.120+0000 7f94846a7640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f94640b6680 0x7f94640b8b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55771ab85f40 tx=0x7f947400b390 comp rx=0 tx=0).stop 2024-09-21T06:22:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.120+0000 7f94846a7640 1 -- 172.21.15.3:0/657618294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f947c1094c0 msgr2=0x7f947c19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.120+0000 7f94846a7640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f947c1094c0 0x7f947c19aaa0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f947000e9f0 tx=0x7f947000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.120+0000 7f94846a7640 1 -- 172.21.15.3:0/657618294 shutdown_connections 2024-09-21T06:22:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.121+0000 7f94846a7640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f94640b6680 0x7f94640b8b90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.121+0000 7f94846a7640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f947c109e10 0x7f947c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.121+0000 7f94846a7640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f947c1094c0 0x7f947c19aaa0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.121+0000 7f94846a7640 1 --2- 172.21.15.3:0/657618294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f947c103b80 0x7f947c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.121+0000 7f94846a7640 1 -- 172.21.15.3:0/657618294 >> 172.21.15.3:0/657618294 conn(0x7f947c0ff610 msgr2=0x7f947c106a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:00.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.121+0000 7f94846a7640 1 -- 172.21.15.3:0/657618294 shutdown_connections 2024-09-21T06:22:00.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:00.121+0000 7f94846a7640 1 -- 172.21.15.3:0/657618294 wait complete. 2024-09-21T06:22:00.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 740278 11.35 MB/sec execute 213 sec latency 93.918 ms 2024-09-21T06:22:01.116 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:22:01.394 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:01.656 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:01 smithi003 ceph-mon[31227]: pgmap v536: 97 pgs: 97 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 302 op/s 2024-09-21T06:22:01.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:01 smithi137 ceph-mon[34464]: pgmap v536: 97 pgs: 97 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 302 op/s 2024-09-21T06:22:01.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:01 smithi204 ceph-mon[34659]: pgmap v536: 97 pgs: 97 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 302 op/s 2024-09-21T06:22:01.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 740909 11.30 MB/sec execute 214 sec latency 245.293 ms 2024-09-21T06:22:01.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.962+0000 7f3ecf7fe640 1 -- 172.21.15.3:0/4040445786 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4035020 con 0x7f3ed0109df0 2024-09-21T06:22:01.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.964+0000 7f3ed6ab7640 1 -- 172.21.15.3:0/4040445786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed0109df0 msgr2=0x7f3ed0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:01.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.964+0000 7f3ed6ab7640 1 --2- 172.21.15.3:0/4040445786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed0109df0 0x7f3ed0110620 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009900 tx=0x7f3ec402a410 comp rx=0 tx=0).stop 2024-09-21T06:22:01.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.964+0000 7f3ed6ab7640 1 -- 172.21.15.3:0/4040445786 shutdown_connections 2024-09-21T06:22:01.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.964+0000 7f3ed6ab7640 1 --2- 172.21.15.3:0/4040445786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ed01094a0 0x7f3ed01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.964+0000 7f3ed6ab7640 1 --2- 172.21.15.3:0/4040445786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed0109df0 0x7f3ed0110620 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.964+0000 7f3ed6ab7640 1 --2- 172.21.15.3:0/4040445786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed0103b60 0x7f3ed0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.964+0000 7f3ed6ab7640 1 -- 172.21.15.3:0/4040445786 >> 172.21.15.3:0/4040445786 conn(0x7f3ed00ff5d0 msgr2=0x7f3ed0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:01.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.965+0000 7f3ecffff640 1 --2- 172.21.15.3:0/4040445786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ed01094a0 0x7f3ed01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:22:01.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.965+0000 7f3ed6ab7640 1 -- 172.21.15.3:0/4040445786 shutdown_connections 2024-09-21T06:22:01.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.965+0000 7f3ed6ab7640 1 -- 172.21.15.3:0/4040445786 wait complete. 2024-09-21T06:22:01.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.966+0000 7f3ed6ab7640 1 Processor -- start 2024-09-21T06:22:01.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.966+0000 7f3ed6ab7640 1 -- start start 2024-09-21T06:22:01.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.967+0000 7f3ed6ab7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed0103b60 0x7f3ed01103d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:01.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.967+0000 7f3ed6ab7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ed0109df0 0x7f3ed010d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:01.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.967+0000 7f3ed6ab7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed010dc00 0x7f3ed010e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:01.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.967+0000 7f3ed6ab7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed0076c70 con 0x7f3ed010dc00 2024-09-21T06:22:01.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.967+0000 7f3ed6ab7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed0076af0 con 0x7f3ed0103b60 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.967+0000 7f3ed6ab7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed0076df0 con 0x7f3ed0109df0 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.967+0000 7f3ed4853640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed0103b60 0x7f3ed01103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.967+0000 7f3ed4853640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed0103b60 0x7f3ed01103d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:52688/0 (socket says 172.21.15.3:52688) 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.967+0000 7f3ed4853640 1 -- 172.21.15.3:0/3597632778 learned_addr learned my addr 172.21.15.3:0/3597632778 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.967+0000 7f3ecffff640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ed0109df0 0x7f3ed010d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.968+0000 7f3ed5054640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed010dc00 0x7f3ed010e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.968+0000 7f3ecffff640 1 -- 172.21.15.3:0/3597632778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed0103b60 msgr2=0x7f3ed01103d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.968+0000 7f3ecffff640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed0103b60 0x7f3ed01103d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.968+0000 7f3ecffff640 1 -- 172.21.15.3:0/3597632778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed010dc00 msgr2=0x7f3ed010e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.968+0000 7f3ecffff640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed010dc00 0x7f3ed010e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.968+0000 7f3ecffff640 1 -- 172.21.15.3:0/3597632778 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed011dac0 con 0x7f3ed0109df0 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.968+0000 7f3ecffff640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ed0109df0 0x7f3ed010d6c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ed01045b0 tx=0x7f3ec000cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.969+0000 7f3ecdffb640 1 -- 172.21.15.3:0/3597632778 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0007b80 con 0x7f3ed0109df0 2024-09-21T06:22:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.969+0000 7f3ecdffb640 1 -- 172.21.15.3:0/3597632778 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec000ae10 con 0x7f3ed0109df0 2024-09-21T06:22:01.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.969+0000 7f3ed6ab7640 1 -- 172.21.15.3:0/3597632778 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed011ddb0 con 0x7f3ed0109df0 2024-09-21T06:22:01.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.969+0000 7f3ed6ab7640 1 -- 172.21.15.3:0/3597632778 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed011e240 con 0x7f3ed0109df0 2024-09-21T06:22:01.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.970+0000 7f3ecdffb640 1 -- 172.21.15.3:0/3597632778 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0018660 con 0x7f3ed0109df0 2024-09-21T06:22:01.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.976+0000 7f3ecdffb640 1 -- 172.21.15.3:0/3597632778 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f3ec000f450 con 0x7f3ed0109df0 2024-09-21T06:22:01.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.976+0000 7f3edd02b740 1 Processor -- start 2024-09-21T06:22:01.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.976+0000 7f3edd02b740 1 -- start start 2024-09-21T06:22:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.977+0000 7f3edd02b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1ac02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.977+0000 7f3edd02b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f01b0 0x5653e1ac0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.977+0000 7f3edd02b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f8390 0x5653e1ac0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.977+0000 7f3edd02b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5653e1a36ea0 con 0x5653e16f8390 2024-09-21T06:22:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.977+0000 7f3edd02b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5653e1a262d0 con 0x5653e16f01b0 2024-09-21T06:22:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.977+0000 7f3edd02b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5653e1a2a0e0 con 0x5653e1719070 2024-09-21T06:22:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.978+0000 7f3ecdffb640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3eb80b6700 0x7f3eb80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.978+0000 7f3ecdffb640 1 -- 172.21.15.3:0/3597632778 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8551+0+0 (secure 0 0 0) 0x7f3ec00e7730 con 0x7f3ed0109df0 2024-09-21T06:22:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.979+0000 7f3ed5054640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3eb80b6700 0x7f3eb80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.979+0000 7f3ed5054640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1ac02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.979+0000 7f3ed4853640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f01b0 0x5653e1ac0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.979+0000 7f3ed5054640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1ac02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:36690/0 (socket says 172.21.15.3:36690) 2024-09-21T06:22:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.979+0000 7f3ed4853640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f01b0 0x5653e1ac0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:52700/0 (socket says 172.21.15.3:52700) 2024-09-21T06:22:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.979+0000 7f3ed5054640 1 -- 172.21.15.3:0/4046644202 learned_addr learned my addr 172.21.15.3:0/4046644202 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.979+0000 7f3ed5054640 1 -- 172.21.15.3:0/4046644202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f01b0 msgr2=0x5653e1ac0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.979+0000 7f3ed5054640 1 --2- 172.21.15.3:0/4046644202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f01b0 0x5653e1ac0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.979+0000 7f3ed5054640 1 -- 172.21.15.3:0/4046644202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f8390 msgr2=0x5653e1ac0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.979+0000 7f3ed5054640 1 --2- 172.21.15.3:0/4046644202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f8390 0x5653e1ac0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.979+0000 7f3ed5054640 1 -- 172.21.15.3:0/4046644202 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653e1784650 con 0x5653e1719070 2024-09-21T06:22:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.980+0000 7f3ed5054640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3eb80b6700 0x7f3eb80b8c10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4005f70 tx=0x7f3ec4005f00 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:01.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.980+0000 7f3ed4853640 1 --2- 172.21.15.3:0/4046644202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f01b0 0x5653e1ac0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:22:01.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.981+0000 7f3ed5054640 1 --2- 172.21.15.3:0/4046644202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1ac02c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4031910 tx=0x7f3ec4044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3845b89ff1dcf67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:01.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.981+0000 7f3eb6ffd640 1 -- 172.21.15.3:0/4046644202 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec404d070 con 0x5653e1719070 2024-09-21T06:22:01.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.982+0000 7f3eb6ffd640 1 -- 172.21.15.3:0/4046644202 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec40035f0 con 0x5653e1719070 2024-09-21T06:22:01.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.982+0000 7f3eb6ffd640 1 -- 172.21.15.3:0/4046644202 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4035040 con 0x5653e1719070 2024-09-21T06:22:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.983+0000 7f3edd02b740 1 -- 172.21.15.3:0/4046644202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 msgr2=0x5653e1ac02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.983+0000 7f3edd02b740 1 --2- 172.21.15.3:0/4046644202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1ac02c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4031910 tx=0x7f3ec4044000 comp rx=0 tx=0).stop 2024-09-21T06:22:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.984+0000 7f3edd02b740 1 -- 172.21.15.3:0/4046644202 shutdown_connections 2024-09-21T06:22:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.984+0000 7f3edd02b740 1 --2- 172.21.15.3:0/4046644202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f8390 0x5653e1ac0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.984+0000 7f3edd02b740 1 --2- 172.21.15.3:0/4046644202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f01b0 0x5653e1ac0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.984+0000 7f3edd02b740 1 --2- 172.21.15.3:0/4046644202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1ac02c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.984+0000 7f3edd02b740 1 -- 172.21.15.3:0/4046644202 >> 172.21.15.3:0/4046644202 conn(0x5653e170d460 msgr2=0x5653e1abe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.984+0000 7f3edd02b740 1 -- 172.21.15.3:0/4046644202 shutdown_connections 2024-09-21T06:22:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.984+0000 7f3edd02b740 1 -- 172.21.15.3:0/4046644202 wait complete. 2024-09-21T06:22:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.985+0000 7f3edd02b740 1 -- start start 2024-09-21T06:22:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.985+0000 7f3edd02b740 1 Processor -- start 2024-09-21T06:22:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.986+0000 7f3edd02b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e16f01b0 0x5653e1acba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.986+0000 7f3edd02b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f8390 0x5653e1acbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.986+0000 7f3edd02b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e1719070 0x5653e1ad7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.986+0000 7f3edd02b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5653e1887700 con 0x5653e1719070 2024-09-21T06:22:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.986+0000 7f3edd02b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5653e19c1160 con 0x5653e16f8390 2024-09-21T06:22:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.986+0000 7f3edd02b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5653e1acea40 con 0x5653e16f01b0 2024-09-21T06:22:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.986+0000 7f3edd02b740 10 client.0 _ll_register_callbacks cb 0x5653e19da9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.987+0000 7f3ed5054640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e1719070 0x5653e1ad7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.987+0000 7f3ed5054640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e1719070 0x5653e1ad7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32864/0 (socket says 172.21.15.3:32864) 2024-09-21T06:22:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.987+0000 7f3ed5054640 1 -- 172.21.15.3:0/2181908994 learned_addr learned my addr 172.21.15.3:0/2181908994 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.987+0000 7f3ed5054640 1 -- 172.21.15.3:0/2181908994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e16f01b0 msgr2=0x5653e1acba90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:22:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.987+0000 7f3ed5054640 1 --2- 172.21.15.3:0/2181908994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e16f01b0 0x5653e1acba90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.987+0000 7f3ed5054640 1 -- 172.21.15.3:0/2181908994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f8390 msgr2=0x5653e1acbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:01.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.987+0000 7f3ed5054640 1 --2- 172.21.15.3:0/2181908994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f8390 0x5653e1acbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.987+0000 7f3ed5054640 1 -- 172.21.15.3:0/2181908994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653e19eec90 con 0x5653e1719070 2024-09-21T06:22:01.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.988+0000 7f3ed5054640 1 --2- 172.21.15.3:0/2181908994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e1719070 0x5653e1ad7d30 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f3ec402a930 tx=0x7f3ec4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8dde372db96602 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:01.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.988+0000 7f3eaffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:01.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.989+0000 7f3eaffff640 1 -- 172.21.15.3:0/2181908994 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec404d040 con 0x5653e1719070 2024-09-21T06:22:01.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.989+0000 7f3eaffff640 1 -- 172.21.15.3:0/2181908994 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8551+0+0 (secure 0 0 0) 0x7f3ec40037a0 con 0x5653e1719070 2024-09-21T06:22:01.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.990+0000 7f3edd02b740 1 -- 172.21.15.3:0/2181908994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec40035f0 con 0x5653e1719070 2024-09-21T06:22:01.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.990+0000 7f3eaffff640 1 -- 172.21.15.3:0/2181908994 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3ec4035040 con 0x5653e1719070 2024-09-21T06:22:01.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.990+0000 7f3edd02b740 10 client.17547 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:01.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.990+0000 7f3edd02b740 1 -- 172.21.15.3:0/2181908994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5653e1ada870 con 0x5653e1719070 2024-09-21T06:22:01.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.990+0000 7f3eaffff640 1 -- 172.21.15.3:0/2181908994 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec40035f0 con 0x5653e1719070 2024-09-21T06:22:01.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.992+0000 7f3eaffff640 1 -- 172.21.15.3:0/2181908994 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f3ec403a050 con 0x5653e1719070 2024-09-21T06:22:01.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.992+0000 7f3edd02b740 10 client.17547 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:01.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.992+0000 7f3edd02b740 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:01.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.992+0000 7f3edd02b740 20 client.17547 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:01.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.992+0000 7f3edd02b740 1 --2- 172.21.15.3:0/2181908994 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5653e16d79f0 0x5653e1addbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:01.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.992+0000 7f3edd02b740 4 client.17547 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:01.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.992+0000 7f3edd02b740 1 -- 172.21.15.3:0/2181908994 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653e1a38df0 con 0x5653e16d79f0 2024-09-21T06:22:01.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.993+0000 7f3ed4853640 1 --2- 172.21.15.3:0/2181908994 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5653e16d79f0 0x5653e1addbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:01.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.995+0000 7f3ed4853640 1 --2- 172.21.15.3:0/2181908994 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5653e16d79f0 0x5653e1addbd0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48cf4a391ee7f0c2 server_cookie=75e458646d3ebbb9 in_seq=0 out_seq=0 2024-09-21T06:22:01.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.995+0000 7f3eaffff640 10 client.17547 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:01.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.996+0000 7f3eaffff640 1 -- 172.21.15.3:0/2181908994 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5653e1a38df0 con 0x5653e16d79f0 2024-09-21T06:22:01.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.996+0000 7f3eaffff640 10 client.17547 handle_command_reply: tid=0 2024-09-21T06:22:01.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.996+0000 7f3edd02b740 1 client.17547 shutdown 2024-09-21T06:22:01.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.996+0000 7f3edd02b740 1 -- 172.21.15.3:0/2181908994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e1719070 msgr2=0x5653e1ad7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:01.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.996+0000 7f3edd02b740 1 --2- 172.21.15.3:0/2181908994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e1719070 0x5653e1ad7d30 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f3ec402a930 tx=0x7f3ec4005fd0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.996+0000 7f3edd02b740 1 -- 172.21.15.3:0/2181908994 shutdown_connections 2024-09-21T06:22:01.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.996+0000 7f3edd02b740 1 --2- 172.21.15.3:0/2181908994 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5653e16d79f0 0x5653e1addbd0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.996+0000 7f3edd02b740 1 --2- 172.21.15.3:0/2181908994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e1719070 0x5653e1ad7d30 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.996+0000 7f3edd02b740 1 --2- 172.21.15.3:0/2181908994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f8390 0x5653e1acbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.996+0000 7f3edd02b740 1 --2- 172.21.15.3:0/2181908994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e16f01b0 0x5653e1acba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:01.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.996+0000 7f3edd02b740 1 -- 172.21.15.3:0/2181908994 >> 172.21.15.3:0/2181908994 conn(0x5653e170d460 msgr2=0x5653e1ac3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:01.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.997+0000 7f3eaffff640 0 client.17547 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:01.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.997+0000 7f3edd02b740 1 -- 172.21.15.3:0/2181908994 shutdown_connections 2024-09-21T06:22:02.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.997+0000 7f3edd02b740 1 -- 172.21.15.3:0/2181908994 wait complete. 2024-09-21T06:22:02.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:01.997+0000 7f3edd02b740 20 client.17547 trim_cache size 0 max 16384 2024-09-21T06:22:02.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.025+0000 7f3edd02b740 1 Processor -- start 2024-09-21T06:22:02.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.026+0000 7f3edd02b740 1 -- start start 2024-09-21T06:22:02.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.026+0000 7f3edd02b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f01b0 0x5653e1acd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:02.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.026+0000 7f3edd02b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f8390 0x5653e1ac8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:02.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.026+0000 7f3edd02b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1ac91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:02.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.026+0000 7f3edd02b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01bea80 con 0x5653e16f8390 2024-09-21T06:22:02.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.026+0000 7f3edd02b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01c05c0 con 0x5653e16f01b0 2024-09-21T06:22:02.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.026+0000 7f3edd02b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01bf150 con 0x5653e1719070 2024-09-21T06:22:02.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.027+0000 7f3ed5054640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1ac91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:02.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.027+0000 7f3ed5054640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1ac91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:36712/0 (socket says 172.21.15.3:36712) 2024-09-21T06:22:02.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.027+0000 7f3ed5054640 1 -- 172.21.15.3:0/2313980177 learned_addr learned my addr 172.21.15.3:0/2313980177 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:02.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.027+0000 7f3ecffff640 1 --2- 172.21.15.3:0/2313980177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f8390 0x5653e1ac8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:02.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.028+0000 7f3ecffff640 1 -- 172.21.15.3:0/2313980177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 msgr2=0x5653e1ac91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:02.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.028+0000 7f3ecffff640 1 --2- 172.21.15.3:0/2313980177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1ac91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.028+0000 7f3ecffff640 1 -- 172.21.15.3:0/2313980177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f01b0 msgr2=0x5653e1acd6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:22:02.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.028+0000 7f3ecffff640 1 --2- 172.21.15.3:0/2313980177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f01b0 0x5653e1acd6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.028+0000 7f3ecffff640 1 -- 172.21.15.3:0/2313980177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653e1ab83b0 con 0x5653e16f8390 2024-09-21T06:22:02.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.028+0000 7f3ecffff640 1 --2- 172.21.15.3:0/2313980177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f8390 0x5653e1ac8c80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00e4e00 tx=0x7f3ec00ad390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eadf24c41474bdd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:02.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.029+0000 7f3eb67fc640 1 -- 172.21.15.3:0/2313980177 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00e5280 con 0x5653e16f8390 2024-09-21T06:22:02.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.029+0000 7f3eb67fc640 1 -- 172.21.15.3:0/2313980177 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec00ad010 con 0x5653e16f8390 2024-09-21T06:22:02.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.029+0000 7f3eb67fc640 1 -- 172.21.15.3:0/2313980177 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00a8e50 con 0x5653e16f8390 2024-09-21T06:22:02.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.030+0000 7f3edd02b740 1 -- 172.21.15.3:0/2313980177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f8390 msgr2=0x5653e1ac8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:02.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.030+0000 7f3edd02b740 1 --2- 172.21.15.3:0/2313980177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f8390 0x5653e1ac8c80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00e4e00 tx=0x7f3ec00ad390 comp rx=0 tx=0).stop 2024-09-21T06:22:02.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.031+0000 7f3edd02b740 1 -- 172.21.15.3:0/2313980177 shutdown_connections 2024-09-21T06:22:02.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.031+0000 7f3edd02b740 1 --2- 172.21.15.3:0/2313980177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1ac91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.031+0000 7f3edd02b740 1 --2- 172.21.15.3:0/2313980177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f8390 0x5653e1ac8c80 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.031+0000 7f3edd02b740 1 --2- 172.21.15.3:0/2313980177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f01b0 0x5653e1acd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.031+0000 7f3edd02b740 1 -- 172.21.15.3:0/2313980177 >> 172.21.15.3:0/2313980177 conn(0x5653e170d460 msgr2=0x5653e1accfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:02.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.031+0000 7f3edd02b740 1 -- 172.21.15.3:0/2313980177 shutdown_connections 2024-09-21T06:22:02.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.031+0000 7f3edd02b740 1 -- 172.21.15.3:0/2313980177 wait complete. 2024-09-21T06:22:02.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.032+0000 7f3edd02b740 1 -- start start 2024-09-21T06:22:02.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.032+0000 7f3edd02b740 1 Processor -- start 2024-09-21T06:22:02.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.033+0000 7f3edd02b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f01b0 0x5653e1ae08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:02.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.033+0000 7f3edd02b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f8390 0x5653e1ae0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:02.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.033+0000 7f3edd02b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1b3e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:02.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.033+0000 7f3edd02b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01bfef0 con 0x5653e16f01b0 2024-09-21T06:22:02.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.033+0000 7f3edd02b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01bf820 con 0x5653e16f8390 2024-09-21T06:22:02.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.033+0000 7f3edd02b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01bca80 con 0x5653e1719070 2024-09-21T06:22:02.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.033+0000 7f3edd02b740 10 client.0 _ll_register_callbacks cb 0x7f3ec404b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:02.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.033+0000 7f3ed5054640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1b3e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:02.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.033+0000 7f3ed5054640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1b3e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:36716/0 (socket says 172.21.15.3:36716) 2024-09-21T06:22:02.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.033+0000 7f3ed5054640 1 -- 172.21.15.3:0/1468437244 learned_addr learned my addr 172.21.15.3:0/1468437244 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:02.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.034+0000 7f3ed5054640 1 -- 172.21.15.3:0/1468437244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f8390 msgr2=0x5653e1ae0ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:22:02.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.034+0000 7f3ed5054640 1 --2- 172.21.15.3:0/1468437244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f8390 0x5653e1ae0ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.034+0000 7f3ed5054640 1 -- 172.21.15.3:0/1468437244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f01b0 msgr2=0x5653e1ae08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:02.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.034+0000 7f3ed5054640 1 --2- 172.21.15.3:0/1468437244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f01b0 0x5653e1ae08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.034+0000 7f3ed5054640 1 -- 172.21.15.3:0/1468437244 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653e1b3ea50 con 0x5653e1719070 2024-09-21T06:22:02.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.035+0000 7f3ed5054640 1 --2- 172.21.15.3:0/1468437244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1b3e3e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009600 tx=0x7f3ec4031f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f171b4d4fa8fba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:02.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.035+0000 7f3eaffff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:22:02.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.035+0000 7f3eaffff640 1 -- 172.21.15.3:0/1468437244 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec404d070 con 0x5653e1719070 2024-09-21T06:22:02.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.036+0000 7f3eaffff640 1 -- 172.21.15.3:0/1468437244 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8551+0+0 (secure 0 0 0) 0x7f3ec4045070 con 0x5653e1719070 2024-09-21T06:22:02.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.036+0000 7f3edd02b740 1 -- 172.21.15.3:0/1468437244 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec00ad010 con 0x5653e1719070 2024-09-21T06:22:02.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.036+0000 7f3eaffff640 1 -- 172.21.15.3:0/1468437244 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3ec40516f0 con 0x5653e1719070 2024-09-21T06:22:02.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.037+0000 7f3edd02b740 10 client.27067 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:02.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.037+0000 7f3edd02b740 1 -- 172.21.15.3:0/1468437244 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5653e1b48080 con 0x5653e1719070 2024-09-21T06:22:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.039+0000 7f3eaffff640 1 -- 172.21.15.3:0/1468437244 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec00ad010 con 0x5653e1719070 2024-09-21T06:22:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.039+0000 7f3eaffff640 1 -- 172.21.15.3:0/1468437244 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f3ec403a040 con 0x5653e1719070 2024-09-21T06:22:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.039+0000 7f3edd02b740 10 client.27067 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.039+0000 7f3edd02b740 10 client.27067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.039+0000 7f3edd02b740 20 client.27067 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.039+0000 7f3edd02b740 1 --2- 172.21.15.3:0/1468437244 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5653e16acda0 0x5653e1b4a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:02.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.039+0000 7f3edd02b740 4 client.27067 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:22:02.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.039+0000 7f3edd02b740 1 -- 172.21.15.3:0/1468437244 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653e1a348a0 con 0x5653e16acda0 2024-09-21T06:22:02.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.040+0000 7f3ed4853640 1 --2- 172.21.15.3:0/1468437244 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5653e16acda0 0x5653e1b4a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:02.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.041+0000 7f3eaffff640 10 client.27067 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:02.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.044+0000 7f3ed4853640 1 --2- 172.21.15.3:0/1468437244 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5653e16acda0 0x5653e1b4a7e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c6825cb617ad42 server_cookie=84a991419e5ad50e in_seq=0 out_seq=0 2024-09-21T06:22:02.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.045+0000 7f3eaffff640 1 -- 172.21.15.3:0/1468437244 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653e1a348a0 con 0x5653e16acda0 2024-09-21T06:22:02.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.045+0000 7f3eaffff640 10 client.27067 handle_command_reply: tid=0 2024-09-21T06:22:02.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.045+0000 7f3edd02b740 1 client.27067 shutdown 2024-09-21T06:22:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.047+0000 7f3edd02b740 1 -- 172.21.15.3:0/1468437244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 msgr2=0x5653e1b3e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.047+0000 7f3edd02b740 1 --2- 172.21.15.3:0/1468437244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1b3e3e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009600 tx=0x7f3ec4031f30 comp rx=0 tx=0).stop 2024-09-21T06:22:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.047+0000 7f3edd02b740 1 -- 172.21.15.3:0/1468437244 shutdown_connections 2024-09-21T06:22:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.047+0000 7f3edd02b740 1 --2- 172.21.15.3:0/1468437244 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5653e16acda0 0x5653e1b4a7e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.047+0000 7f3edd02b740 1 --2- 172.21.15.3:0/1468437244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653e1719070 0x5653e1b3e3e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.047+0000 7f3edd02b740 1 --2- 172.21.15.3:0/1468437244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653e16f8390 0x5653e1ae0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.047+0000 7f3edd02b740 1 --2- 172.21.15.3:0/1468437244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653e16f01b0 0x5653e1ae08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.047+0000 7f3edd02b740 1 -- 172.21.15.3:0/1468437244 >> 172.21.15.3:0/1468437244 conn(0x5653e170d460 msgr2=0x5653e1ac1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.048+0000 7f3eaffff640 0 client.27067 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:02.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.049+0000 7f3edd02b740 1 -- 172.21.15.3:0/1468437244 shutdown_connections 2024-09-21T06:22:02.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.049+0000 7f3edd02b740 1 -- 172.21.15.3:0/1468437244 wait complete. 2024-09-21T06:22:02.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.049+0000 7f3edd02b740 20 client.27067 trim_cache size 0 max 16384 2024-09-21T06:22:02.051 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:22:02.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.050+0000 7f3ed6ab7640 1 -- 172.21.15.3:0/3597632778 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3eb80b6700 msgr2=0x7f3eb80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:02.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.050+0000 7f3ed6ab7640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3eb80b6700 0x7f3eb80b8c10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4005f70 tx=0x7f3ec4005f00 comp rx=0 tx=0).stop 2024-09-21T06:22:02.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.050+0000 7f3ed6ab7640 1 -- 172.21.15.3:0/3597632778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ed0109df0 msgr2=0x7f3ed010d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:02.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.050+0000 7f3ed6ab7640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ed0109df0 0x7f3ed010d6c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ed01045b0 tx=0x7f3ec000cd80 comp rx=0 tx=0).stop 2024-09-21T06:22:02.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.050+0000 7f3ed6ab7640 1 -- 172.21.15.3:0/3597632778 shutdown_connections 2024-09-21T06:22:02.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.050+0000 7f3ed6ab7640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3eb80b6700 0x7f3eb80b8c10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.050+0000 7f3ed6ab7640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ed0109df0 0x7f3ed010d6c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.050+0000 7f3ed6ab7640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed010dc00 0x7f3ed010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.050+0000 7f3ed6ab7640 1 --2- 172.21.15.3:0/3597632778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed0103b60 0x7f3ed01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.050+0000 7f3ed6ab7640 1 -- 172.21.15.3:0/3597632778 >> 172.21.15.3:0/3597632778 conn(0x7f3ed00ff5d0 msgr2=0x7f3ed01072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:02.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.050+0000 7f3ed6ab7640 1 -- 172.21.15.3:0/3597632778 shutdown_connections 2024-09-21T06:22:02.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:02.050+0000 7f3ed6ab7640 1 -- 172.21.15.3:0/3597632778 wait complete. 2024-09-21T06:22:02.815 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:02 smithi003 ceph-mon[31227]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:22:02.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:02 smithi137 ceph-mon[34464]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:22:02.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:02 smithi204 ceph-mon[34659]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:22:02.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 741960 11.28 MB/sec execute 215 sec latency 34.799 ms 2024-09-21T06:22:03.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:03 smithi003 ceph-mon[31227]: pgmap v538: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 439 op/s 2024-09-21T06:22:03.864 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:22:03.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:22:03.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:03 smithi137 ceph-mon[34464]: pgmap v538: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 439 op/s 2024-09-21T06:22:03.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:03 smithi204 ceph-mon[34659]: pgmap v538: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 439 op/s 2024-09-21T06:22:03.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 743194 11.26 MB/sec execute 216 sec latency 20.752 ms 2024-09-21T06:22:04.132 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:04.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.647+0000 7fd607e2b640 1 -- 172.21.15.3:0/216246801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600103b90 msgr2=0x7fd600103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.647+0000 7fd607e2b640 1 --2- 172.21.15.3:0/216246801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600103b90 0x7fd600103fa0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009900 tx=0x7fd5fc02a410 comp rx=0 tx=0).stop 2024-09-21T06:22:04.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.647+0000 7fd607e2b640 1 -- 172.21.15.3:0/216246801 shutdown_connections 2024-09-21T06:22:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.647+0000 7fd607e2b640 1 --2- 172.21.15.3:0/216246801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6001094d0 0x7fd6001098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.647+0000 7fd607e2b640 1 --2- 172.21.15.3:0/216246801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd600109e20 0x7fd600110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.647+0000 7fd607e2b640 1 --2- 172.21.15.3:0/216246801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600103b90 0x7fd600103fa0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.647+0000 7fd607e2b640 1 -- 172.21.15.3:0/216246801 >> 172.21.15.3:0/216246801 conn(0x7fd6000ff5e0 msgr2=0x7fd600101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.647+0000 7fd607e2b640 1 -- 172.21.15.3:0/216246801 shutdown_connections 2024-09-21T06:22:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.647+0000 7fd607e2b640 1 -- 172.21.15.3:0/216246801 wait complete. 2024-09-21T06:22:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.647+0000 7fd607e2b640 1 Processor -- start 2024-09-21T06:22:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd607e2b640 1 -- start start 2024-09-21T06:22:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd607e2b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd600103b90 0x7fd60019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd607e2b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6001094d0 0x7fd60019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd607e2b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600109e20 0x7fd60019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd607e2b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd600113ae0 con 0x7fd600109e20 2024-09-21T06:22:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd607e2b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd600113960 con 0x7fd600103b90 2024-09-21T06:22:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd607e2b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd600113c60 con 0x7fd6001094d0 2024-09-21T06:22:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd6063c8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600109e20 0x7fd60019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd6063c8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600109e20 0x7fd60019fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32910/0 (socket says 172.21.15.3:32910) 2024-09-21T06:22:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd6063c8640 1 -- 172.21.15.3:0/2597635521 learned_addr learned my addr 172.21.15.3:0/2597635521 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd605bc7640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd600103b90 0x7fd60019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd6053c6640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6001094d0 0x7fd60019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd6063c8640 1 -- 172.21.15.3:0/2597635521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6001094d0 msgr2=0x7fd60019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd6063c8640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6001094d0 0x7fd60019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd6063c8640 1 -- 172.21.15.3:0/2597635521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd600103b90 msgr2=0x7fd60019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd6063c8640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd600103b90 0x7fd60019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd6063c8640 1 -- 172.21.15.3:0/2597635521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6001a0420 con 0x7fd600109e20 2024-09-21T06:22:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.648+0000 7fd6063c8640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600109e20 0x7fd60019fcf0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800eb10 tx=0x7fd5f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.649+0000 7fd5f6ffd640 1 -- 172.21.15.3:0/2597635521 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8004b20 con 0x7fd600109e20 2024-09-21T06:22:04.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.649+0000 7fd5f6ffd640 1 -- 172.21.15.3:0/2597635521 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5f8004cc0 con 0x7fd600109e20 2024-09-21T06:22:04.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.649+0000 7fd607e2b640 1 -- 172.21.15.3:0/2597635521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6001a0710 con 0x7fd600109e20 2024-09-21T06:22:04.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.649+0000 7fd5f6ffd640 1 -- 172.21.15.3:0/2597635521 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8010670 con 0x7fd600109e20 2024-09-21T06:22:04.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.649+0000 7fd607e2b640 1 -- 172.21.15.3:0/2597635521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6001a6e70 con 0x7fd600109e20 2024-09-21T06:22:04.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.652+0000 7fd5f6ffd640 1 -- 172.21.15.3:0/2597635521 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fd5f8005020 con 0x7fd600109e20 2024-09-21T06:22:04.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.652+0000 7fd60e39f740 1 Processor -- start 2024-09-21T06:22:04.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.652+0000 7fd60e39f740 1 -- start start 2024-09-21T06:22:04.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.652+0000 7fd60e39f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3cce9350 0x556b3d090430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.652+0000 7fd60e39f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3ccc0490 0x556b3d090970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.652+0000 7fd60e39f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d090eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.652+0000 7fd60e39f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556b3d006e60 con 0x556b3ccc8670 2024-09-21T06:22:04.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.652+0000 7fd60e39f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556b3cff6290 con 0x556b3ccc0490 2024-09-21T06:22:04.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.652+0000 7fd60e39f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556b3cffa0a0 con 0x556b3cce9350 2024-09-21T06:22:04.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.652+0000 7fd605bc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3ccc0490 0x556b3d090970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.652+0000 7fd6053c6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d090eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.652+0000 7fd605bc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3ccc0490 0x556b3d090970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:52754/0 (socket says 172.21.15.3:52754) 2024-09-21T06:22:04.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.652+0000 7fd605bc7640 1 -- 172.21.15.3:0/731692482 learned_addr learned my addr 172.21.15.3:0/731692482 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:04.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.653+0000 7fd6053c6640 1 --2- 172.21.15.3:0/731692482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3cce9350 0x556b3d090430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.653+0000 7fd5f6ffd640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd5ec0b6680 0x7fd5ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.653+0000 7fd605bc7640 1 -- 172.21.15.3:0/731692482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3cce9350 msgr2=0x556b3d090430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.653+0000 7fd605bc7640 1 --2- 172.21.15.3:0/731692482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3cce9350 0x556b3d090430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.653+0000 7fd605bc7640 1 -- 172.21.15.3:0/731692482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 msgr2=0x556b3d090eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.653+0000 7fd6063c8640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd5ec0b6680 0x7fd5ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.653+0000 7fd605bc7640 1 --2- 172.21.15.3:0/731692482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d090eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.653+0000 7fd605bc7640 1 -- 172.21.15.3:0/731692482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b3cd54930 con 0x556b3ccc0490 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.653+0000 7fd6053c6640 1 --2- 172.21.15.3:0/731692482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d090eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.653+0000 7fd5f6ffd640 1 -- 172.21.15.3:0/2597635521 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8551+0+0 (secure 0 0 0) 0x7fd5f80e7d10 con 0x7fd600109e20 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.653+0000 7fd605bc7640 1 --2- 172.21.15.3:0/731692482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3ccc0490 0x556b3d090970 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009900 tx=0x7fd5fc032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2f988c94d24200e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.654+0000 7fd6063c8640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd5ec0b6680 0x7fd5ec0b8b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80e54f0 tx=0x7fd5f807a7f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.654+0000 7fd5ebfff640 1 -- 172.21.15.3:0/731692482 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc040070 con 0x556b3ccc0490 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.654+0000 7fd5ebfff640 1 -- 172.21.15.3:0/731692482 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5fc004030 con 0x556b3ccc0490 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.654+0000 7fd5ebfff640 1 -- 172.21.15.3:0/731692482 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc035040 con 0x556b3ccc0490 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.654+0000 7fd60e39f740 1 -- 172.21.15.3:0/731692482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3ccc0490 msgr2=0x556b3d090970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.654+0000 7fd60e39f740 1 --2- 172.21.15.3:0/731692482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3ccc0490 0x556b3d090970 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009900 tx=0x7fd5fc032b90 comp rx=0 tx=0).stop 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.655+0000 7fd60e39f740 1 -- 172.21.15.3:0/731692482 shutdown_connections 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.655+0000 7fd60e39f740 1 --2- 172.21.15.3:0/731692482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d090eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.655+0000 7fd60e39f740 1 --2- 172.21.15.3:0/731692482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3ccc0490 0x556b3d090970 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.655+0000 7fd60e39f740 1 --2- 172.21.15.3:0/731692482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3cce9350 0x556b3d090430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.655+0000 7fd60e39f740 1 -- 172.21.15.3:0/731692482 >> 172.21.15.3:0/731692482 conn(0x556b3ccdd740 msgr2=0x556b3d08e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:04.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.655+0000 7fd60e39f740 1 -- 172.21.15.3:0/731692482 shutdown_connections 2024-09-21T06:22:04.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.655+0000 7fd60e39f740 1 -- 172.21.15.3:0/731692482 wait complete. 2024-09-21T06:22:04.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.655+0000 7fd60e39f740 1 -- start start 2024-09-21T06:22:04.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.655+0000 7fd60e39f740 1 Processor -- start 2024-09-21T06:22:04.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd60e39f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d0a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd60e39f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d0a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd60e39f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 0x556b3d0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd60e39f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556b3ce57890 con 0x556b3ccc8670 2024-09-21T06:22:04.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd60e39f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556b3cf90d40 con 0x556b3cce9350 2024-09-21T06:22:04.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd60e39f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556b3d09ebb0 con 0x556b3ccc0490 2024-09-21T06:22:04.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd60e39f740 10 client.0 _ll_register_callbacks cb 0x556b3cff4c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:04.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd605bc7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d0a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd6053c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d0a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd6053c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d0a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:36748/0 (socket says 172.21.15.3:36748) 2024-09-21T06:22:04.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd6053c6640 1 -- 172.21.15.3:0/66183384 learned_addr learned my addr 172.21.15.3:0/66183384 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:04.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd6053c6640 1 --2- 172.21.15.3:0/66183384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 0x556b3d0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd6053c6640 1 -- 172.21.15.3:0/66183384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 msgr2=0x556b3d0a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd6053c6640 1 --2- 172.21.15.3:0/66183384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 0x556b3d0a6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd6053c6640 1 -- 172.21.15.3:0/66183384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 msgr2=0x556b3d0a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd6053c6640 1 --2- 172.21.15.3:0/66183384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d0a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.656+0000 7fd6053c6640 1 -- 172.21.15.3:0/66183384 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3d0a66d0 con 0x556b3ccc0490 2024-09-21T06:22:04.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd6053c6640 1 --2- 172.21.15.3:0/66183384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d0a0a60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0012d40 tx=0x7fd5f0009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe9d66a9a50c8273 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:04.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd5e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:22:04.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd5e97fa640 1 -- 172.21.15.3:0/66183384 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f001a040 con 0x556b3ccc0490 2024-09-21T06:22:04.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd60e39f740 1 -- 172.21.15.3:0/66183384 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5fc004030 con 0x556b3ccc0490 2024-09-21T06:22:04.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd5e97fa640 1 -- 172.21.15.3:0/66183384 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8551+0+0 (secure 0 0 0) 0x7fd5f0011250 con 0x556b3ccc0490 2024-09-21T06:22:04.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd5e97fa640 1 -- 172.21.15.3:0/66183384 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd5f000fe60 con 0x556b3ccc0490 2024-09-21T06:22:04.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd60e39f740 10 client.27079 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:04.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd60e39f740 1 -- 172.21.15.3:0/66183384 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556b3d0af220 con 0x556b3ccc0490 2024-09-21T06:22:04.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd5e97fa640 1 -- 172.21.15.3:0/66183384 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5fc004030 con 0x556b3ccc0490 2024-09-21T06:22:04.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd5e97fa640 1 -- 172.21.15.3:0/66183384 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fd5f0003bc0 con 0x556b3ccc0490 2024-09-21T06:22:04.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd60e39f740 10 client.27079 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:04.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd60e39f740 10 client.27079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:04.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd60e39f740 20 client.27079 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:04.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd60e39f740 1 --2- 172.21.15.3:0/66183384 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556b3cca7cd0 0x556b3d0b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd60e39f740 4 client.27079 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:04.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.657+0000 7fd60e39f740 1 -- 172.21.15.3:0/66183384 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b3d008db0 con 0x556b3cca7cd0 2024-09-21T06:22:04.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.658+0000 7fd605bc7640 1 --2- 172.21.15.3:0/66183384 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556b3cca7cd0 0x556b3d0b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.658+0000 7fd5e97fa640 10 client.27079 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.658+0000 7fd605bc7640 1 --2- 172.21.15.3:0/66183384 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556b3cca7cd0 0x556b3d0b2660 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dfc77e33c562dcd server_cookie=bde709fe1ee8ef0d in_seq=0 out_seq=0 2024-09-21T06:22:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.659+0000 7fd5e97fa640 1 -- 172.21.15.3:0/66183384 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b3d008db0 con 0x556b3cca7cd0 2024-09-21T06:22:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.659+0000 7fd5e97fa640 10 client.27079 handle_command_reply: tid=0 2024-09-21T06:22:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.659+0000 7fd60e39f740 1 client.27079 shutdown 2024-09-21T06:22:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.659+0000 7fd60e39f740 1 -- 172.21.15.3:0/66183384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 msgr2=0x556b3d0a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.659+0000 7fd60e39f740 1 --2- 172.21.15.3:0/66183384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d0a0a60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0012d40 tx=0x7fd5f0009ac0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.660+0000 7fd60e39f740 1 -- 172.21.15.3:0/66183384 shutdown_connections 2024-09-21T06:22:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.660+0000 7fd60e39f740 1 --2- 172.21.15.3:0/66183384 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556b3cca7cd0 0x556b3d0b2660 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.660+0000 7fd60e39f740 1 --2- 172.21.15.3:0/66183384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 0x556b3d0a6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.660+0000 7fd60e39f740 1 --2- 172.21.15.3:0/66183384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d0a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.660+0000 7fd60e39f740 1 --2- 172.21.15.3:0/66183384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d0a0a60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.660+0000 7fd5e97fa640 0 client.27079 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:04.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.660+0000 7fd60e39f740 1 -- 172.21.15.3:0/66183384 >> 172.21.15.3:0/66183384 conn(0x556b3ccdd740 msgr2=0x556b3d0935c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:04.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.660+0000 7fd60e39f740 1 -- 172.21.15.3:0/66183384 shutdown_connections 2024-09-21T06:22:04.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.660+0000 7fd60e39f740 1 -- 172.21.15.3:0/66183384 wait complete. 2024-09-21T06:22:04.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.660+0000 7fd60e39f740 20 client.27079 trim_cache size 0 max 16384 2024-09-21T06:22:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.678+0000 7fd60e39f740 1 Processor -- start 2024-09-21T06:22:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.678+0000 7fd60e39f740 1 -- start start 2024-09-21T06:22:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.678+0000 7fd60e39f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d09d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.678+0000 7fd60e39f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d09d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd60e39f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 0x556b3d0b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd60e39f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001ad5a0 con 0x556b3ccc8670 2024-09-21T06:22:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd60e39f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001af0e0 con 0x556b3cce9350 2024-09-21T06:22:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd60e39f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001adc70 con 0x556b3ccc0490 2024-09-21T06:22:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd605bc7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d09d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd605bc7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d09d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32926/0 (socket says 172.21.15.3:32926) 2024-09-21T06:22:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd605bc7640 1 -- 172.21.15.3:0/935599121 learned_addr learned my addr 172.21.15.3:0/935599121 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd6053c6640 1 --2- 172.21.15.3:0/935599121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 0x556b3d0b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd6053c6640 1 --2- 172.21.15.3:0/935599121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d09d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd605bc7640 1 -- 172.21.15.3:0/935599121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 msgr2=0x556b3d09d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd605bc7640 1 --2- 172.21.15.3:0/935599121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d09d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd605bc7640 1 -- 172.21.15.3:0/935599121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 msgr2=0x556b3d0b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd605bc7640 1 --2- 172.21.15.3:0/935599121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 0x556b3d0b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd605bc7640 1 -- 172.21.15.3:0/935599121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b3d09ba30 con 0x556b3ccc8670 2024-09-21T06:22:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd605bc7640 1 --2- 172.21.15.3:0/935599121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d09d8d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009900 tx=0x7fd5fc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5c9d37a13e388 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.679+0000 7fd5eb7fe640 1 -- 172.21.15.3:0/935599121 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc040070 con 0x556b3ccc8670 2024-09-21T06:22:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.680+0000 7fd5eb7fe640 1 -- 172.21.15.3:0/935599121 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5fc031d10 con 0x556b3ccc8670 2024-09-21T06:22:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.680+0000 7fd5eb7fe640 1 -- 172.21.15.3:0/935599121 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc035040 con 0x556b3ccc8670 2024-09-21T06:22:04.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.680+0000 7fd60e39f740 1 -- 172.21.15.3:0/935599121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 msgr2=0x556b3d09d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.680+0000 7fd60e39f740 1 --2- 172.21.15.3:0/935599121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d09d8d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009900 tx=0x7fd5fc000bc0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.681+0000 7fd60e39f740 1 -- 172.21.15.3:0/935599121 shutdown_connections 2024-09-21T06:22:04.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.681+0000 7fd60e39f740 1 --2- 172.21.15.3:0/935599121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 0x556b3d0b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.681+0000 7fd60e39f740 1 --2- 172.21.15.3:0/935599121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d09d8d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.681+0000 7fd60e39f740 1 --2- 172.21.15.3:0/935599121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d09d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.681+0000 7fd60e39f740 1 -- 172.21.15.3:0/935599121 >> 172.21.15.3:0/935599121 conn(0x556b3ccdd740 msgr2=0x556b3d09f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.681+0000 7fd60e39f740 1 -- 172.21.15.3:0/935599121 shutdown_connections 2024-09-21T06:22:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.681+0000 7fd60e39f740 1 -- 172.21.15.3:0/935599121 wait complete. 2024-09-21T06:22:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.681+0000 7fd60e39f740 1 -- start start 2024-09-21T06:22:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.681+0000 7fd60e39f740 1 Processor -- start 2024-09-21T06:22:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd60e39f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d0a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd60e39f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d0a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd60e39f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 0x556b3d1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd60e39f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001aea10 con 0x556b3ccc8670 2024-09-21T06:22:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd60e39f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001ae340 con 0x556b3cce9350 2024-09-21T06:22:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd60e39f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001ab5a0 con 0x556b3ccc0490 2024-09-21T06:22:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd60e39f740 10 client.0 _ll_register_callbacks cb 0x7fd5f0012540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd605bc7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d0a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd605bc7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d0a90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32936/0 (socket says 172.21.15.3:32936) 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd605bc7640 1 -- 172.21.15.3:0/3629505861 learned_addr learned my addr 172.21.15.3:0/3629505861 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd6053c6640 1 --2- 172.21.15.3:0/3629505861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 0x556b3d1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd6053c6640 1 --2- 172.21.15.3:0/3629505861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d0a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd6053c6640 1 -- 172.21.15.3:0/3629505861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 msgr2=0x556b3d0a8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd6053c6640 1 --2- 172.21.15.3:0/3629505861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d0a8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd6053c6640 1 -- 172.21.15.3:0/3629505861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 msgr2=0x556b3d0a90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd6053c6640 1 --2- 172.21.15.3:0/3629505861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d0a90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.682+0000 7fd6053c6640 1 -- 172.21.15.3:0/3629505861 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3d116740 con 0x556b3cce9350 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd6053c6640 1 --2- 172.21.15.3:0/3629505861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 0x556b3d1160d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00114a0 tx=0x7fd5f0005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919570198d935eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd5e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd5e97fa640 1 -- 172.21.15.3:0/3629505861 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f002d040 con 0x556b3cce9350 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd5e97fa640 1 -- 172.21.15.3:0/3629505861 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8551+0+0 (secure 0 0 0) 0x7fd5f0011230 con 0x556b3cce9350 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd60e39f740 1 -- 172.21.15.3:0/3629505861 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5fc031d10 con 0x556b3cce9350 2024-09-21T06:22:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd5e97fa640 1 -- 172.21.15.3:0/3629505861 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd5f0004040 con 0x556b3cce9350 2024-09-21T06:22:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd60e39f740 10 client.27134 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd60e39f740 1 -- 172.21.15.3:0/3629505861 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556b3d11db50 con 0x556b3cce9350 2024-09-21T06:22:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd5e97fa640 1 -- 172.21.15.3:0/3629505861 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5fc031d10 con 0x556b3cce9350 2024-09-21T06:22:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd5e97fa640 1 -- 172.21.15.3:0/3629505861 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fd5f0033070 con 0x556b3cce9350 2024-09-21T06:22:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd60e39f740 10 client.27134 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd60e39f740 10 client.27134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd60e39f740 20 client.27134 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd60e39f740 1 --2- 172.21.15.3:0/3629505861 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556b3cc7d080 0x556b3d120190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd60e39f740 4 client.27134 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:22:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.683+0000 7fd60e39f740 1 -- 172.21.15.3:0/3629505861 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556b3d004860 con 0x556b3cc7d080 2024-09-21T06:22:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.684+0000 7fd605bc7640 1 --2- 172.21.15.3:0/3629505861 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556b3cc7d080 0x556b3d120190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.684+0000 7fd5e97fa640 10 client.27134 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:04.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.684+0000 7fd605bc7640 1 --2- 172.21.15.3:0/3629505861 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556b3cc7d080 0x556b3d120190 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b53d2d9cb21e54b server_cookie=2c796885fb9579a7 in_seq=0 out_seq=0 2024-09-21T06:22:04.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.686+0000 7fd5e97fa640 1 -- 172.21.15.3:0/3629505861 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556b3d004860 con 0x556b3cc7d080 2024-09-21T06:22:04.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.686+0000 7fd5e97fa640 10 client.27134 handle_command_reply: tid=0 2024-09-21T06:22:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.686+0000 7fd60e39f740 1 client.27134 shutdown 2024-09-21T06:22:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd60e39f740 1 -- 172.21.15.3:0/3629505861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 msgr2=0x556b3d1160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd60e39f740 1 --2- 172.21.15.3:0/3629505861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 0x556b3d1160d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00114a0 tx=0x7fd5f0005850 comp rx=0 tx=0).stop 2024-09-21T06:22:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd60e39f740 1 -- 172.21.15.3:0/3629505861 shutdown_connections 2024-09-21T06:22:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd60e39f740 1 --2- 172.21.15.3:0/3629505861 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556b3cc7d080 0x556b3d120190 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd60e39f740 1 --2- 172.21.15.3:0/3629505861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b3cce9350 0x556b3d1160d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd60e39f740 1 --2- 172.21.15.3:0/3629505861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b3ccc8670 0x556b3d0a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd60e39f740 1 --2- 172.21.15.3:0/3629505861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b3ccc0490 0x556b3d0a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd5e97fa640 0 client.27134 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd60e39f740 1 -- 172.21.15.3:0/3629505861 >> 172.21.15.3:0/3629505861 conn(0x556b3ccdd740 msgr2=0x556b3d0913f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd60e39f740 1 -- 172.21.15.3:0/3629505861 shutdown_connections 2024-09-21T06:22:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd60e39f740 1 -- 172.21.15.3:0/3629505861 wait complete. 2024-09-21T06:22:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd60e39f740 20 client.27134 trim_cache size 0 max 16384 2024-09-21T06:22:04.692 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:22:04.692 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:22:04.692 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "714b9934-a2d3-4a78-b185-01254776061b", 2024-09-21T06:22:04.692 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:22:04.692 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:22:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd607e2b640 1 -- 172.21.15.3:0/2597635521 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd5ec0b6680 msgr2=0x7fd5ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd607e2b640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd5ec0b6680 0x7fd5ec0b8b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80e54f0 tx=0x7fd5f807a7f0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd607e2b640 1 -- 172.21.15.3:0/2597635521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600109e20 msgr2=0x7fd60019fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.687+0000 7fd607e2b640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600109e20 0x7fd60019fcf0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800eb10 tx=0x7fd5f800c2d0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.688+0000 7fd607e2b640 1 -- 172.21.15.3:0/2597635521 shutdown_connections 2024-09-21T06:22:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.688+0000 7fd607e2b640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd5ec0b6680 0x7fd5ec0b8b90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.688+0000 7fd607e2b640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6001094d0 0x7fd60019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.688+0000 7fd607e2b640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600109e20 0x7fd60019fcf0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.688+0000 7fd607e2b640 1 --2- 172.21.15.3:0/2597635521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd600103b90 0x7fd60019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.688+0000 7fd607e2b640 1 -- 172.21.15.3:0/2597635521 >> 172.21.15.3:0/2597635521 conn(0x7fd6000ff5e0 msgr2=0x7fd600101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:04.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.688+0000 7fd607e2b640 1 -- 172.21.15.3:0/2597635521 shutdown_connections 2024-09-21T06:22:04.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:04.688+0000 7fd607e2b640 1 -- 172.21.15.3:0/2597635521 wait complete. 2024-09-21T06:22:04.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 745338 11.26 MB/sec execute 217 sec latency 44.128 ms 2024-09-21T06:22:05.820 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:05 smithi003 ceph-mon[31227]: pgmap v539: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-09-21T06:22:05.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:05 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:22:05.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:05 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:05.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:05 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:22:05.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:05 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:22:05.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:05 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:05.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:05 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:05.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:22:05.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:05 smithi137 ceph-mon[34464]: pgmap v539: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-09-21T06:22:05.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:05 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:22:05.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:05 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:05.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:05 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:22:05.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:05 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:22:05.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:05 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:05.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:05 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:05.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:05 smithi204 ceph-mon[34659]: pgmap v539: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-09-21T06:22:05.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:05 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:22:05.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:05 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:05.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:05 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:22:05.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:05 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:22:05.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:05 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:05.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:05 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:05.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 746831 11.25 MB/sec execute 218 sec latency 43.181 ms 2024-09-21T06:22:06.151 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:06.554 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:06 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:22:06.554 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:06 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:06.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.810+0000 7f82b42d0640 1 -- 172.21.15.3:0/3058387343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac105300 msgr2=0x7f82ac10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.810+0000 7f82b42d0640 1 --2- 172.21.15.3:0/3058387343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac105300 0x7f82ac10d850 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f829c009900 tx=0x7f829c02a410 comp rx=0 tx=0).stop 2024-09-21T06:22:06.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.810+0000 7f82b42d0640 1 -- 172.21.15.3:0/3058387343 shutdown_connections 2024-09-21T06:22:06.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.810+0000 7f82b42d0640 1 --2- 172.21.15.3:0/3058387343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82ac10de10 0x7f82ac110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.810+0000 7f82b42d0640 1 --2- 172.21.15.3:0/3058387343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac105300 0x7f82ac10d850 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.810+0000 7f82b42d0640 1 --2- 172.21.15.3:0/3058387343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ac1049b0 0x7f82ac104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.810+0000 7f82b42d0640 1 -- 172.21.15.3:0/3058387343 >> 172.21.15.3:0/3058387343 conn(0x7f82ac079730 msgr2=0x7f82ac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:06.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.810+0000 7f82b42d0640 1 -- 172.21.15.3:0/3058387343 shutdown_connections 2024-09-21T06:22:06.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.810+0000 7f82b42d0640 1 -- 172.21.15.3:0/3058387343 wait complete. 2024-09-21T06:22:06.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.811+0000 7f82b42d0640 1 Processor -- start 2024-09-21T06:22:06.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.811+0000 7f82b42d0640 1 -- start start 2024-09-21T06:22:06.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.811+0000 7f82b42d0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ac1049b0 0x7f82ac1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.811+0000 7f82b42d0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac105300 0x7f82ac1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.811+0000 7f82b42d0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82ac10de10 0x7f82ac1ac790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.811+0000 7f82b42d0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac113a70 con 0x7f82ac105300 2024-09-21T06:22:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.811+0000 7f82b42d0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1138f0 con 0x7f82ac10de10 2024-09-21T06:22:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.811+0000 7f82b42d0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac113bf0 con 0x7f82ac1049b0 2024-09-21T06:22:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.811+0000 7f82b206c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ac1049b0 0x7f82ac1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.811+0000 7f82b206c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ac1049b0 0x7f82ac1a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:33708/0 (socket says 172.21.15.3:33708) 2024-09-21T06:22:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.811+0000 7f82b206c640 1 -- 172.21.15.3:0/3468275097 learned_addr learned my addr 172.21.15.3:0/3468275097 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.811+0000 7f82b186b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac105300 0x7f82ac1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.811+0000 7f82b286d640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82ac10de10 0x7f82ac1ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.812+0000 7f82b186b640 1 -- 172.21.15.3:0/3468275097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ac1049b0 msgr2=0x7f82ac1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.812+0000 7f82b186b640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ac1049b0 0x7f82ac1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.812+0000 7f82b186b640 1 -- 172.21.15.3:0/3468275097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82ac10de10 msgr2=0x7f82ac1ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.812+0000 7f82b186b640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82ac10de10 0x7f82ac1ac790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.812+0000 7f82b186b640 1 -- 172.21.15.3:0/3468275097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac1accd0 con 0x7f82ac105300 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.812+0000 7f82b206c640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ac1049b0 0x7f82ac1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.812+0000 7f82b186b640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac105300 0x7f82ac1a1c50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f829c02a930 tx=0x7f829c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.812+0000 7f82a37fe640 1 -- 172.21.15.3:0/3468275097 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c03f070 con 0x7f82ac105300 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.812+0000 7f82b42d0640 1 -- 172.21.15.3:0/3468275097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac1acea0 con 0x7f82ac105300 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.812+0000 7f82a37fe640 1 -- 172.21.15.3:0/3468275097 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f829c004490 con 0x7f82ac105300 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.812+0000 7f82b42d0640 1 -- 172.21.15.3:0/3468275097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac1ad240 con 0x7f82ac105300 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.812+0000 7f82a37fe640 1 -- 172.21.15.3:0/3468275097 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c035070 con 0x7f82ac105300 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.815+0000 7f82ba844740 1 Processor -- start 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.815+0000 7f82ba844740 1 -- start start 2024-09-21T06:22:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.815+0000 7f82a37fe640 1 -- 172.21.15.3:0/3468275097 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f829c043400 con 0x7f82ac105300 2024-09-21T06:22:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.815+0000 7f82ba844740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558af070 0x55e755c56270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.815+0000 7f82ba844740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7558861b0 0x55e755c567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.815+0000 7f82ba844740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e75588e390 0x55e755c56cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.815+0000 7f82ba844740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e755bcce50 con 0x55e7558af070 2024-09-21T06:22:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.815+0000 7f82ba844740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e755bbc280 con 0x55e7558861b0 2024-09-21T06:22:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.815+0000 7f82ba844740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e755bc0090 con 0x55e75588e390 2024-09-21T06:22:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.816+0000 7f82b206c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7558861b0 0x55e755c567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.816+0000 7f82b286d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558af070 0x55e755c56270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.816+0000 7f82b206c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7558861b0 0x55e755c567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:46466/0 (socket says 172.21.15.3:46466) 2024-09-21T06:22:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.816+0000 7f82b286d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558af070 0x55e755c56270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52998/0 (socket says 172.21.15.3:52998) 2024-09-21T06:22:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.816+0000 7f82b206c640 1 -- 172.21.15.3:0/2356886229 learned_addr learned my addr 172.21.15.3:0/2356886229 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.816+0000 7f82b186b640 1 --2- 172.21.15.3:0/2356886229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e75588e390 0x55e755c56cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.816+0000 7f82b206c640 1 -- 172.21.15.3:0/2356886229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e75588e390 msgr2=0x55e755c56cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.816+0000 7f82b206c640 1 --2- 172.21.15.3:0/2356886229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e75588e390 0x55e755c56cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.816+0000 7f82b206c640 1 -- 172.21.15.3:0/2356886229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558af070 msgr2=0x55e755c56270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.816+0000 7f82b206c640 1 --2- 172.21.15.3:0/2356886229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558af070 0x55e755c56270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.816+0000 7f82b206c640 1 -- 172.21.15.3:0/2356886229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e75591a650 con 0x55e7558861b0 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.816+0000 7f82b286d640 1 --2- 172.21.15.3:0/2356886229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558af070 0x55e755c56270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.817+0000 7f82b206c640 1 --2- 172.21.15.3:0/2356886229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7558861b0 0x55e755c567b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f82a80115f0 tx=0x7f82a8011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b23511a12b5f3f86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.817+0000 7f82a37fe640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82980b6730 0x7f82980b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.817+0000 7f82a0d9a640 1 -- 172.21.15.3:0/2356886229 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a801e070 con 0x55e7558861b0 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.817+0000 7f82b286d640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82980b6730 0x7f82980b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.817+0000 7f82a0d9a640 1 -- 172.21.15.3:0/2356886229 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a800fbb0 con 0x55e7558861b0 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.817+0000 7f82a37fe640 1 -- 172.21.15.3:0/3468275097 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8551+0+0 (secure 0 0 0) 0x7f829c107b20 con 0x7f82ac105300 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.817+0000 7f82a0d9a640 1 -- 172.21.15.3:0/2356886229 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8019be0 con 0x55e7558861b0 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.818+0000 7f82ba844740 1 -- 172.21.15.3:0/2356886229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7558861b0 msgr2=0x55e755c567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.818+0000 7f82ba844740 1 --2- 172.21.15.3:0/2356886229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7558861b0 0x55e755c567b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f82a80115f0 tx=0x7f82a8011ab0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.818+0000 7f82b286d640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82980b6730 0x7f82980b8c40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009fd0 tx=0x7f82a4009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.819+0000 7f82ba844740 1 -- 172.21.15.3:0/2356886229 shutdown_connections 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.819+0000 7f82ba844740 1 --2- 172.21.15.3:0/2356886229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e75588e390 0x55e755c56cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.819+0000 7f82ba844740 1 --2- 172.21.15.3:0/2356886229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7558861b0 0x55e755c567b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.819+0000 7f82ba844740 1 --2- 172.21.15.3:0/2356886229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558af070 0x55e755c56270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.819+0000 7f82ba844740 1 -- 172.21.15.3:0/2356886229 >> 172.21.15.3:0/2356886229 conn(0x55e7558a3460 msgr2=0x55e755c54160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.819+0000 7f82ba844740 1 -- 172.21.15.3:0/2356886229 shutdown_connections 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.819+0000 7f82ba844740 1 -- 172.21.15.3:0/2356886229 wait complete. 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.819+0000 7f82ba844740 1 -- start start 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.819+0000 7f82ba844740 1 Processor -- start 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82ba844740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558861b0 0x55e755c61990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82ba844740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e75588e390 0x55e755c61ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82ba844740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558af070 0x55e755c6dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82ba844740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e755a1d700 con 0x55e7558861b0 2024-09-21T06:22:06.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82ba844740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e755b57110 con 0x55e75588e390 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82ba844740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e755c64a50 con 0x55e7558af070 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82ba844740 10 client.0 _ll_register_callbacks cb 0x55e755b70990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82b206c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558861b0 0x55e755c61990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82b186b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e75588e390 0x55e755c61ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82b206c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558861b0 0x55e755c61990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53008/0 (socket says 172.21.15.3:53008) 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82b206c640 1 -- 172.21.15.3:0/1899510811 learned_addr learned my addr 172.21.15.3:0/1899510811 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82b286d640 1 --2- 172.21.15.3:0/1899510811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558af070 0x55e755c6dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82b206c640 1 -- 172.21.15.3:0/1899510811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558af070 msgr2=0x55e755c6dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82b206c640 1 --2- 172.21.15.3:0/1899510811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558af070 0x55e755c6dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82b206c640 1 -- 172.21.15.3:0/1899510811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e75588e390 msgr2=0x55e755c61ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82b206c640 1 --2- 172.21.15.3:0/1899510811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e75588e390 0x55e755c61ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82b206c640 1 -- 172.21.15.3:0/1899510811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e755b84c40 con 0x55e7558861b0 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.820+0000 7f82b286d640 1 --2- 172.21.15.3:0/1899510811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558af070 0x55e755c6dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:06.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.821+0000 7f82b206c640 1 --2- 172.21.15.3:0/1899510811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558861b0 0x55e755c61990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f82a8019720 tx=0x7f82a8019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f71d978d97e96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:06.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.821+0000 7f8295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:06.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.821+0000 7f8295ffb640 1 -- 172.21.15.3:0/1899510811 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a801e040 con 0x55e7558861b0 2024-09-21T06:22:06.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.821+0000 7f82ba844740 1 -- 172.21.15.3:0/1899510811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a800fbb0 con 0x55e7558861b0 2024-09-21T06:22:06.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.821+0000 7f8295ffb640 1 -- 172.21.15.3:0/1899510811 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8551+0+0 (secure 0 0 0) 0x7f82a800fd60 con 0x55e7558861b0 2024-09-21T06:22:06.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.821+0000 7f8295ffb640 1 -- 172.21.15.3:0/1899510811 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f82a8004020 con 0x55e7558861b0 2024-09-21T06:22:06.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.821+0000 7f82ba844740 10 client.17580 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:06.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.821+0000 7f82ba844740 1 -- 172.21.15.3:0/1899510811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e755c70760 con 0x55e7558861b0 2024-09-21T06:22:06.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.821+0000 7f8295ffb640 1 -- 172.21.15.3:0/1899510811 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a800fbb0 con 0x55e7558861b0 2024-09-21T06:22:06.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.822+0000 7f8295ffb640 1 -- 172.21.15.3:0/1899510811 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f82a8021070 con 0x55e7558861b0 2024-09-21T06:22:06.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.822+0000 7f82ba844740 10 client.17580 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:06.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.822+0000 7f82ba844740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:06.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.822+0000 7f82ba844740 20 client.17580 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:06.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.822+0000 7f82ba844740 1 --2- 172.21.15.3:0/1899510811 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e75586d9f0 0x55e755c73bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.822+0000 7f82ba844740 4 client.17580 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.822+0000 7f82ba844740 1 -- 172.21.15.3:0/1899510811 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e755bceda0 con 0x55e75586d9f0 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.822+0000 7f82b186b640 1 --2- 172.21.15.3:0/1899510811 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e75586d9f0 0x55e755c73bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.822+0000 7f8295ffb640 10 client.17580 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.822+0000 7f82b186b640 1 --2- 172.21.15.3:0/1899510811 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e75586d9f0 0x55e755c73bd0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7df15adaec21ca server_cookie=54e85b4ff1e5b2d1 in_seq=0 out_seq=0 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.823+0000 7f8295ffb640 1 -- 172.21.15.3:0/1899510811 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e755bceda0 con 0x55e75586d9f0 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.823+0000 7f8295ffb640 10 client.17580 handle_command_reply: tid=0 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.823+0000 7f82ba844740 1 client.17580 shutdown 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.823+0000 7f82ba844740 1 -- 172.21.15.3:0/1899510811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558861b0 msgr2=0x55e755c61990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.823+0000 7f82ba844740 1 --2- 172.21.15.3:0/1899510811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558861b0 0x55e755c61990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f82a8019720 tx=0x7f82a8019640 comp rx=0 tx=0).stop 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.823+0000 7f82ba844740 1 -- 172.21.15.3:0/1899510811 shutdown_connections 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.823+0000 7f82ba844740 1 --2- 172.21.15.3:0/1899510811 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e75586d9f0 0x55e755c73bd0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.823+0000 7f82ba844740 1 --2- 172.21.15.3:0/1899510811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558af070 0x55e755c6dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.823+0000 7f82ba844740 1 --2- 172.21.15.3:0/1899510811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e75588e390 0x55e755c61ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.823+0000 7f82ba844740 1 --2- 172.21.15.3:0/1899510811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558861b0 0x55e755c61990 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.823+0000 7f82ba844740 1 -- 172.21.15.3:0/1899510811 >> 172.21.15.3:0/1899510811 conn(0x55e7558a3460 msgr2=0x55e755c59450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:06.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.823+0000 7f8295ffb640 0 client.17580 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:06.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.823+0000 7f82ba844740 1 -- 172.21.15.3:0/1899510811 shutdown_connections 2024-09-21T06:22:06.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.824+0000 7f82ba844740 1 -- 172.21.15.3:0/1899510811 wait complete. 2024-09-21T06:22:06.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.824+0000 7f82ba844740 20 client.17580 trim_cache size 0 max 16384 2024-09-21T06:22:06.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82ba844740 1 Processor -- start 2024-09-21T06:22:06.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82ba844740 1 -- start start 2024-09-21T06:22:06.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82ba844740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7558861b0 0x55e755c57c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82ba844740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e75588e390 0x55e755c6dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82ba844740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558af070 0x55e755c635c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82ba844740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1b5d30 con 0x55e75588e390 2024-09-21T06:22:06.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82ba844740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1b7870 con 0x55e7558861b0 2024-09-21T06:22:06.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82ba844740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1b6400 con 0x55e7558af070 2024-09-21T06:22:06.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82b186b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e75588e390 0x55e755c6dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82b186b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e75588e390 0x55e755c6dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53012/0 (socket says 172.21.15.3:53012) 2024-09-21T06:22:06.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82b186b640 1 -- 172.21.15.3:0/1615504220 learned_addr learned my addr 172.21.15.3:0/1615504220 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:06.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82b206c640 1 --2- 172.21.15.3:0/1615504220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7558861b0 0x55e755c57c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82b286d640 1 --2- 172.21.15.3:0/1615504220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558af070 0x55e755c635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.842+0000 7f82b186b640 1 -- 172.21.15.3:0/1615504220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558af070 msgr2=0x55e755c635c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.843+0000 7f82b186b640 1 --2- 172.21.15.3:0/1615504220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558af070 0x55e755c635c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.843+0000 7f82b186b640 1 -- 172.21.15.3:0/1615504220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7558861b0 msgr2=0x55e755c57c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.843+0000 7f82b186b640 1 --2- 172.21.15.3:0/1615504220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7558861b0 0x55e755c57c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.843+0000 7f82b186b640 1 -- 172.21.15.3:0/1615504220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e755c62c50 con 0x55e75588e390 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.843+0000 7f82b206c640 1 --2- 172.21.15.3:0/1615504220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7558861b0 0x55e755c57c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.843+0000 7f82b186b640 1 --2- 172.21.15.3:0/1615504220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e75588e390 0x55e755c6dc40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f829c1052b0 tx=0x7f829c10b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ff01ab4392ae52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.843+0000 7f8297fff640 1 -- 172.21.15.3:0/1615504220 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c048040 con 0x55e75588e390 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.843+0000 7f8297fff640 1 -- 172.21.15.3:0/1615504220 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f829c10c040 con 0x55e75588e390 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.843+0000 7f8297fff640 1 -- 172.21.15.3:0/1615504220 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0c9170 con 0x55e75588e390 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.844+0000 7f82ba844740 1 -- 172.21.15.3:0/1615504220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e75588e390 msgr2=0x55e755c6dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.844+0000 7f82ba844740 1 --2- 172.21.15.3:0/1615504220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e75588e390 0x55e755c6dc40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f829c1052b0 tx=0x7f829c10b670 comp rx=0 tx=0).stop 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.844+0000 7f82ba844740 1 -- 172.21.15.3:0/1615504220 shutdown_connections 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.844+0000 7f82ba844740 1 --2- 172.21.15.3:0/1615504220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558af070 0x55e755c635c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.844+0000 7f82ba844740 1 --2- 172.21.15.3:0/1615504220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e75588e390 0x55e755c6dc40 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.844+0000 7f82ba844740 1 --2- 172.21.15.3:0/1615504220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7558861b0 0x55e755c57c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.844+0000 7f82ba844740 1 -- 172.21.15.3:0/1615504220 >> 172.21.15.3:0/1615504220 conn(0x55e7558a3460 msgr2=0x55e755c62eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.844+0000 7f82ba844740 1 -- 172.21.15.3:0/1615504220 shutdown_connections 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.844+0000 7f82ba844740 1 -- 172.21.15.3:0/1615504220 wait complete. 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.844+0000 7f82ba844740 1 -- start start 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.844+0000 7f82ba844740 1 Processor -- start 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82ba844740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558861b0 0x55e755c680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82ba844740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e75588e390 0x55e755c68600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82ba844740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558af070 0x55e755cdd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82ba844740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1b71a0 con 0x55e7558af070 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82ba844740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1b6ad0 con 0x55e75588e390 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82ba844740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1b3d30 con 0x55e7558861b0 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82ba844740 10 client.0 _ll_register_callbacks cb 0x7f82a801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82b206c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558861b0 0x55e755c680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82b186b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e75588e390 0x55e755c68600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82b286d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558af070 0x55e755cdd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82b206c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558861b0 0x55e755c680c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:33752/0 (socket says 172.21.15.3:33752) 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82b206c640 1 -- 172.21.15.3:0/845507008 learned_addr learned my addr 172.21.15.3:0/845507008 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82b186b640 1 -- 172.21.15.3:0/845507008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558861b0 msgr2=0x55e755c680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82b186b640 1 --2- 172.21.15.3:0/845507008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558861b0 0x55e755c680c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82b186b640 1 -- 172.21.15.3:0/845507008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558af070 msgr2=0x55e755cdd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82b186b640 1 --2- 172.21.15.3:0/845507008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558af070 0x55e755cdd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82b186b640 1 -- 172.21.15.3:0/845507008 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e755cdda00 con 0x55e75588e390 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82b206c640 1 --2- 172.21.15.3:0/845507008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558861b0 0x55e755c680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.845+0000 7f82b286d640 1 --2- 172.21.15.3:0/845507008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558af070 0x55e755cdd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.846+0000 7f82b186b640 1 --2- 172.21.15.3:0/845507008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e75588e390 0x55e755c68600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55e755bccf20 tx=0x7f829c0ca0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0a544e8e20c8cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.846+0000 7f8295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.846+0000 7f8295ffb640 1 -- 172.21.15.3:0/845507008 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c048070 con 0x55e75588e390 2024-09-21T06:22:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.846+0000 7f82ba844740 1 -- 172.21.15.3:0/845507008 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f829c10c040 con 0x55e75588e390 2024-09-21T06:22:06.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.846+0000 7f8295ffb640 1 -- 172.21.15.3:0/845507008 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8551+0+0 (secure 0 0 0) 0x7f829c0d2070 con 0x55e75588e390 2024-09-21T06:22:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.846+0000 7f8295ffb640 1 -- 172.21.15.3:0/845507008 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f829c0c9170 con 0x55e75588e390 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.846+0000 7f82ba844740 10 client.27152 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.846+0000 7f82ba844740 1 -- 172.21.15.3:0/845507008 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e755cdeae0 con 0x55e75588e390 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.846+0000 7f8295ffb640 1 -- 172.21.15.3:0/845507008 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f829c10c040 con 0x55e75588e390 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.846+0000 7f8295ffb640 1 -- 172.21.15.3:0/845507008 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f829c04b060 con 0x55e75588e390 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.847+0000 7f82ba844740 10 client.27152 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.847+0000 7f82ba844740 10 client.27152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.847+0000 7f82ba844740 20 client.27152 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.847+0000 7f82ba844740 1 --2- 172.21.15.3:0/845507008 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e755842da0 0x55e755ce1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.847+0000 7f82ba844740 4 client.27152 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.847+0000 7f82ba844740 1 -- 172.21.15.3:0/845507008 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e755bca850 con 0x55e755842da0 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.847+0000 7f82b206c640 1 --2- 172.21.15.3:0/845507008 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e755842da0 0x55e755ce1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.847+0000 7f8295ffb640 10 client.27152 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.847+0000 7f82b206c640 1 --2- 172.21.15.3:0/845507008 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e755842da0 0x55e755ce1200 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1f476e8c9d80c server_cookie=5e169bd7c449fb8c in_seq=0 out_seq=0 2024-09-21T06:22:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f8295ffb640 1 -- 172.21.15.3:0/845507008 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x55e755bca850 con 0x55e755842da0 2024-09-21T06:22:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f8295ffb640 10 client.27152 handle_command_reply: tid=0 2024-09-21T06:22:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f82ba844740 1 client.27152 shutdown 2024-09-21T06:22:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f82ba844740 1 -- 172.21.15.3:0/845507008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e75588e390 msgr2=0x55e755c68600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f82ba844740 1 --2- 172.21.15.3:0/845507008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e75588e390 0x55e755c68600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55e755bccf20 tx=0x7f829c0ca0c0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f82ba844740 1 -- 172.21.15.3:0/845507008 shutdown_connections 2024-09-21T06:22:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f82ba844740 1 --2- 172.21.15.3:0/845507008 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e755842da0 0x55e755ce1200 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f82ba844740 1 --2- 172.21.15.3:0/845507008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7558af070 0x55e755cdd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f82ba844740 1 --2- 172.21.15.3:0/845507008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e75588e390 0x55e755c68600 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f8295ffb640 0 client.27152 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f82ba844740 1 --2- 172.21.15.3:0/845507008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7558861b0 0x55e755c680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f82ba844740 1 -- 172.21.15.3:0/845507008 >> 172.21.15.3:0/845507008 conn(0x55e7558a3460 msgr2=0x55e755c57a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f82ba844740 1 -- 172.21.15.3:0/845507008 shutdown_connections 2024-09-21T06:22:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f82ba844740 1 -- 172.21.15.3:0/845507008 wait complete. 2024-09-21T06:22:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.848+0000 7f82ba844740 20 client.27152 trim_cache size 0 max 16384 2024-09-21T06:22:06.855 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:22:06.855 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:22:06.855 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-21T06:22:06.855 INFO:teuthology.orchestra.run.smithi003.stdout: "714b9934-a2d3-4a78-b185-01254776061b": { 2024-09-21T06:22:06.855 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:06.855 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "714b9934-a2d3-4a78-b185-01254776061b", 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:22:06.856 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: "714b9934-a2d3-4a78-b185-01254776061b": { 2024-09-21T06:22:06.857 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:04.685380+0000", 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:06.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:06.859 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58": { 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:21:17.396219+0000", 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:06.860 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:22:06.861 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:06.862 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:06.862 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:06.862 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:06.862 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:06.862 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:22:06.862 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:22:06.862 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:22:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.849+0000 7f82b42d0640 1 -- 172.21.15.3:0/3468275097 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82980b6730 msgr2=0x7f82980b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.849+0000 7f82b42d0640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82980b6730 0x7f82980b8c40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009fd0 tx=0x7f82a4009500 comp rx=0 tx=0).stop 2024-09-21T06:22:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.849+0000 7f82b42d0640 1 -- 172.21.15.3:0/3468275097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac105300 msgr2=0x7f82ac1a1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.849+0000 7f82b42d0640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac105300 0x7f82ac1a1c50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f829c02a930 tx=0x7f829c031a20 comp rx=0 tx=0).stop 2024-09-21T06:22:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.849+0000 7f82b42d0640 1 -- 172.21.15.3:0/3468275097 shutdown_connections 2024-09-21T06:22:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.849+0000 7f82b42d0640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82980b6730 0x7f82980b8c40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.849+0000 7f82b42d0640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82ac10de10 0x7f82ac1ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.849+0000 7f82b42d0640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac105300 0x7f82ac1a1c50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.849+0000 7f82b42d0640 1 --2- 172.21.15.3:0/3468275097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ac1049b0 0x7f82ac1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.849+0000 7f82b42d0640 1 -- 172.21.15.3:0/3468275097 >> 172.21.15.3:0/3468275097 conn(0x7f82ac079730 msgr2=0x7f82ac0ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:06.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.849+0000 7f82b42d0640 1 -- 172.21.15.3:0/3468275097 shutdown_connections 2024-09-21T06:22:06.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:06.849+0000 7f82b42d0640 1 -- 172.21.15.3:0/3468275097 wait complete. 2024-09-21T06:22:06.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:06 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:22:06.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:06 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:06.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:06 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:22:06.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:06 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:06.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 749200 11.27 MB/sec execute 219 sec latency 65.696 ms 2024-09-21T06:22:07.609 INFO:tasks.cephfs.filesystem:scrub status for tag:714b9934-a2d3-4a78-b185-01254776061b - {'path': '/', 'tag': '714b9934-a2d3-4a78-b185-01254776061b', 'options': 'recursive,force'} 2024-09-21T06:22:07.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:07 smithi003 ceph-mon[31227]: pgmap v540: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 121 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 437 op/s 2024-09-21T06:22:07.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:07 smithi137 ceph-mon[34464]: pgmap v540: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 121 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 437 op/s 2024-09-21T06:22:07.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:07 smithi204 ceph-mon[34659]: pgmap v540: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 121 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 437 op/s 2024-09-21T06:22:07.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 750688 11.24 MB/sec execute 220 sec latency 17.458 ms 2024-09-21T06:22:08.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 753035 11.26 MB/sec execute 221 sec latency 16.077 ms 2024-09-21T06:22:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:09 smithi003 ceph-mon[31227]: pgmap v541: 97 pgs: 97 active+clean; 122 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 379 op/s 2024-09-21T06:22:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:09 smithi003 ceph-mon[31227]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:22:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:09 smithi137 ceph-mon[34464]: pgmap v541: 97 pgs: 97 active+clean; 122 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 379 op/s 2024-09-21T06:22:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:09 smithi137 ceph-mon[34464]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:22:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:09 smithi204 ceph-mon[34659]: pgmap v541: 97 pgs: 97 active+clean; 122 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 379 op/s 2024-09-21T06:22:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:09 smithi204 ceph-mon[34659]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:22:09.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 754545 11.23 MB/sec execute 222 sec latency 22.773 ms 2024-09-21T06:22:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.050+0000 7f5212d2b640 1 -- 172.21.15.3:0/3411428683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c109d80 msgr2=0x7f520c1105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.050+0000 7f5212d2b640 1 --2- 172.21.15.3:0/3411428683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c109d80 0x7f520c1105b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5204009900 tx=0x7f520402a410 comp rx=0 tx=0).stop 2024-09-21T06:22:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.051+0000 7f5212d2b640 1 -- 172.21.15.3:0/3411428683 shutdown_connections 2024-09-21T06:22:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.051+0000 7f5212d2b640 1 --2- 172.21.15.3:0/3411428683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f520c109430 0x7f520c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.051+0000 7f5212d2b640 1 --2- 172.21.15.3:0/3411428683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c109d80 0x7f520c1105b0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.051+0000 7f5212d2b640 1 --2- 172.21.15.3:0/3411428683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f520c103b00 0x7f520c103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.051+0000 7f5212d2b640 1 -- 172.21.15.3:0/3411428683 >> 172.21.15.3:0/3411428683 conn(0x7f520c0ff500 msgr2=0x7f520c101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.051+0000 7f5212d2b640 1 -- 172.21.15.3:0/3411428683 shutdown_connections 2024-09-21T06:22:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.051+0000 7f5212d2b640 1 -- 172.21.15.3:0/3411428683 wait complete. 2024-09-21T06:22:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.051+0000 7f5212d2b640 1 Processor -- start 2024-09-21T06:22:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.051+0000 7f5212d2b640 1 -- start start 2024-09-21T06:22:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f5212d2b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f520c103b00 0x7f520c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f5212d2b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f520c109430 0x7f520c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f5212d2b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c109d80 0x7f520c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f5212d2b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f520c113a40 con 0x7f520c109d80 2024-09-21T06:22:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f5212d2b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1138c0 con 0x7f520c103b00 2024-09-21T06:22:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f5212d2b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f520c113bc0 con 0x7f520c109430 2024-09-21T06:22:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f52112c8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c109d80 0x7f520c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f5210ac7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f520c103b00 0x7f520c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f52112c8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c109d80 0x7f520c076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53034/0 (socket says 172.21.15.3:53034) 2024-09-21T06:22:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f52112c8640 1 -- 172.21.15.3:0/206864965 learned_addr learned my addr 172.21.15.3:0/206864965 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f520bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f520c109430 0x7f520c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f52112c8640 1 -- 172.21.15.3:0/206864965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f520c109430 msgr2=0x7f520c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f52112c8640 1 --2- 172.21.15.3:0/206864965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f520c109430 0x7f520c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f52112c8640 1 -- 172.21.15.3:0/206864965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f520c103b00 msgr2=0x7f520c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f52112c8640 1 --2- 172.21.15.3:0/206864965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f520c103b00 0x7f520c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f52112c8640 1 -- 172.21.15.3:0/206864965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c1a8790 con 0x7f520c109d80 2024-09-21T06:22:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f52112c8640 1 --2- 172.21.15.3:0/206864965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c109d80 0x7f520c076190 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5204005940 tx=0x7f5204005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f5209ffb640 1 -- 172.21.15.3:0/206864965 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520403f030 con 0x7f520c109d80 2024-09-21T06:22:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f5209ffb640 1 -- 172.21.15.3:0/206864965 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52040043d0 con 0x7f520c109d80 2024-09-21T06:22:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f5209ffb640 1 -- 172.21.15.3:0/206864965 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204035070 con 0x7f520c109d80 2024-09-21T06:22:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.052+0000 7f5212d2b640 1 -- 172.21.15.3:0/206864965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520c1a8a20 con 0x7f520c109d80 2024-09-21T06:22:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.053+0000 7f5212d2b640 1 -- 172.21.15.3:0/206864965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520c1a8ec0 con 0x7f520c109d80 2024-09-21T06:22:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.055+0000 7f5219c98740 1 Processor -- start 2024-09-21T06:22:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.055+0000 7f5209ffb640 1 -- 172.21.15.3:0/206864965 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f5204042030 con 0x7f520c109d80 2024-09-21T06:22:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.055+0000 7f5219c98740 1 -- start start 2024-09-21T06:22:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5219c98740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0159181b0 0x55a015cbd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5219c98740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158ef2f0 0x55a015cbdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5219c98740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158f74d0 0x55a015cbe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5219c98740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a015c326f0 con 0x55a0159181b0 2024-09-21T06:22:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5219c98740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a015c21b20 con 0x55a0158ef2f0 2024-09-21T06:22:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5219c98740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a015c25930 con 0x55a0158f74d0 2024-09-21T06:22:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f520bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0159181b0 0x55a015cbd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5210ac7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158ef2f0 0x55a015cbdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f520bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0159181b0 0x55a015cbd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53042/0 (socket says 172.21.15.3:53042) 2024-09-21T06:22:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f520bfff640 1 -- 172.21.15.3:0/3939089361 learned_addr learned my addr 172.21.15.3:0/3939089361 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f520bfff640 1 --2- 172.21.15.3:0/3939089361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158f74d0 0x55a015cbe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5209ffb640 1 --2- 172.21.15.3:0/206864965 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51f40b66b0 0x7f51f40b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5210ac7640 1 -- 172.21.15.3:0/3939089361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158f74d0 msgr2=0x55a015cbe430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5210ac7640 1 --2- 172.21.15.3:0/3939089361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158f74d0 0x55a015cbe430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5210ac7640 1 -- 172.21.15.3:0/3939089361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0159181b0 msgr2=0x55a015cbd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5210ac7640 1 --2- 172.21.15.3:0/3939089361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0159181b0 0x55a015cbd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5210ac7640 1 -- 172.21.15.3:0/3939089361 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a015be7530 con 0x55a0158ef2f0 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f520bfff640 1 --2- 172.21.15.3:0/3939089361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0159181b0 0x55a015cbd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f52112c8640 1 --2- 172.21.15.3:0/206864965 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51f40b66b0 0x7f51f40b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.056+0000 7f5209ffb640 1 -- 172.21.15.3:0/206864965 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f5204107a10 con 0x7f520c109d80 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.057+0000 7f5210ac7640 1 --2- 172.21.15.3:0/3939089361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158ef2f0 0x55a015cbdef0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009fa0 tx=0x7f51fc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19fe8cb7c5973181 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.057+0000 7f51f2ffd640 1 -- 172.21.15.3:0/3939089361 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc003c30 con 0x55a0158ef2f0 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.057+0000 7f52112c8640 1 --2- 172.21.15.3:0/206864965 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51f40b66b0 0x7f51f40b8bc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5204031440 tx=0x7f520409a520 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.057+0000 7f51f2ffd640 1 -- 172.21.15.3:0/3939089361 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51fc003dd0 con 0x55a0158ef2f0 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.057+0000 7f51f2ffd640 1 -- 172.21.15.3:0/3939089361 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00f950 con 0x55a0158ef2f0 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.058+0000 7f5219c98740 1 -- 172.21.15.3:0/3939089361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158ef2f0 msgr2=0x55a015cbdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.058+0000 7f5219c98740 1 --2- 172.21.15.3:0/3939089361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158ef2f0 0x55a015cbdef0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009fa0 tx=0x7f51fc0118d0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.058+0000 7f5219c98740 1 -- 172.21.15.3:0/3939089361 shutdown_connections 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.058+0000 7f5219c98740 1 --2- 172.21.15.3:0/3939089361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158f74d0 0x55a015cbe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.058+0000 7f5219c98740 1 --2- 172.21.15.3:0/3939089361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158ef2f0 0x55a015cbdef0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.058+0000 7f5219c98740 1 --2- 172.21.15.3:0/3939089361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0159181b0 0x55a015cbd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.058+0000 7f5219c98740 1 -- 172.21.15.3:0/3939089361 >> 172.21.15.3:0/3939089361 conn(0x55a01590c300 msgr2=0x55a015cbb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.058+0000 7f5219c98740 1 -- 172.21.15.3:0/3939089361 shutdown_connections 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.058+0000 7f5219c98740 1 -- 172.21.15.3:0/3939089361 wait complete. 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.058+0000 7f5219c98740 1 -- start start 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.058+0000 7f5219c98740 1 Processor -- start 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f5219c98740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158ef2f0 0x55a015cc9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f5219c98740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158f74d0 0x55a015cc9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f5219c98740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0159181b0 0x55a015cd5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f5219c98740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a015b59ab0 con 0x55a0158f74d0 2024-09-21T06:22:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f5219c98740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a015bc7730 con 0x55a0158ef2f0 2024-09-21T06:22:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f5219c98740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a015ccc130 con 0x55a0159181b0 2024-09-21T06:22:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f5219c98740 10 client.0 _ll_register_callbacks cb 0x55a015c1b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f5210ac7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158f74d0 0x55a015cc9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f5210ac7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158f74d0 0x55a015cc9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53046/0 (socket says 172.21.15.3:53046) 2024-09-21T06:22:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f520bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158ef2f0 0x55a015cc9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f5210ac7640 1 -- 172.21.15.3:0/466818567 learned_addr learned my addr 172.21.15.3:0/466818567 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f520bfff640 1 --2- 172.21.15.3:0/466818567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0159181b0 0x55a015cd5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f520bfff640 1 -- 172.21.15.3:0/466818567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0159181b0 msgr2=0x55a015cd5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f520bfff640 1 --2- 172.21.15.3:0/466818567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0159181b0 0x55a015cd5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f520bfff640 1 -- 172.21.15.3:0/466818567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158f74d0 msgr2=0x55a015cc9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.059+0000 7f520bfff640 1 --2- 172.21.15.3:0/466818567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158f74d0 0x55a015cc9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.060+0000 7f520bfff640 1 -- 172.21.15.3:0/466818567 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a015983790 con 0x55a0158ef2f0 2024-09-21T06:22:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.060+0000 7f520bfff640 1 --2- 172.21.15.3:0/466818567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158ef2f0 0x55a015cc9130 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f52000058c0 tx=0x7f5200005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=181ea853a8fffead server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.060+0000 7f51ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:22:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.060+0000 7f51ebfff640 1 -- 172.21.15.3:0/466818567 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52000104f0 con 0x55a0158ef2f0 2024-09-21T06:22:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.060+0000 7f51ebfff640 1 -- 172.21.15.3:0/466818567 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f5200010690 con 0x55a0158ef2f0 2024-09-21T06:22:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.060+0000 7f5219c98740 1 -- 172.21.15.3:0/466818567 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51fc003dd0 con 0x55a0158ef2f0 2024-09-21T06:22:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.060+0000 7f51ebfff640 1 -- 172.21.15.3:0/466818567 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f520002d050 con 0x55a0158ef2f0 2024-09-21T06:22:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.060+0000 7f5219c98740 10 client.27164 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.060+0000 7f5219c98740 1 -- 172.21.15.3:0/466818567 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a0159fc620 con 0x55a0158ef2f0 2024-09-21T06:22:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.060+0000 7f51ebfff640 1 -- 172.21.15.3:0/466818567 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51fc003dd0 con 0x55a0158ef2f0 2024-09-21T06:22:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.061+0000 7f51ebfff640 1 -- 172.21.15.3:0/466818567 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f520000e870 con 0x55a0158ef2f0 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.061+0000 7f5219c98740 10 client.27164 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.061+0000 7f5219c98740 10 client.27164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.061+0000 7f5219c98740 20 client.27164 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.061+0000 7f5219c98740 1 --2- 172.21.15.3:0/466818567 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a0158d6d80 0x55a015cdb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.061+0000 7f5219c98740 4 client.27164 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.061+0000 7f5219c98740 1 -- 172.21.15.3:0/466818567 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a015c34640 con 0x55a0158d6d80 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.061+0000 7f5210ac7640 1 --2- 172.21.15.3:0/466818567 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a0158d6d80 0x55a015cdb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.061+0000 7f5210ac7640 1 --2- 172.21.15.3:0/466818567 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a0158d6d80 0x55a015cdb1c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b720429e74c35a server_cookie=e1ce8c07f7600a6c in_seq=0 out_seq=0 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.061+0000 7f51ebfff640 10 client.27164 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.062+0000 7f51ebfff640 1 -- 172.21.15.3:0/466818567 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a015c34640 con 0x55a0158d6d80 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.062+0000 7f51ebfff640 10 client.27164 handle_command_reply: tid=0 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.062+0000 7f5219c98740 1 client.27164 shutdown 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.062+0000 7f5219c98740 1 -- 172.21.15.3:0/466818567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158ef2f0 msgr2=0x55a015cc9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.062+0000 7f5219c98740 1 --2- 172.21.15.3:0/466818567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158ef2f0 0x55a015cc9130 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f52000058c0 tx=0x7f5200005f90 comp rx=0 tx=0).stop 2024-09-21T06:22:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.062+0000 7f5219c98740 1 -- 172.21.15.3:0/466818567 shutdown_connections 2024-09-21T06:22:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.062+0000 7f5219c98740 1 --2- 172.21.15.3:0/466818567 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a0158d6d80 0x55a015cdb1c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.063+0000 7f5219c98740 1 --2- 172.21.15.3:0/466818567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0159181b0 0x55a015cd5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.063+0000 7f5219c98740 1 --2- 172.21.15.3:0/466818567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158f74d0 0x55a015cc9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.063+0000 7f51ebfff640 0 client.27164 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.063+0000 7f5219c98740 1 --2- 172.21.15.3:0/466818567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158ef2f0 0x55a015cc9130 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.063+0000 7f5219c98740 1 -- 172.21.15.3:0/466818567 >> 172.21.15.3:0/466818567 conn(0x55a01590c300 msgr2=0x55a015cbe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.063+0000 7f5219c98740 1 -- 172.21.15.3:0/466818567 shutdown_connections 2024-09-21T06:22:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.063+0000 7f5219c98740 1 -- 172.21.15.3:0/466818567 wait complete. 2024-09-21T06:22:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.063+0000 7f5219c98740 20 client.27164 trim_cache size 0 max 16384 2024-09-21T06:22:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.078+0000 7f5219c98740 1 Processor -- start 2024-09-21T06:22:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f5219c98740 1 -- start start 2024-09-21T06:22:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f5219c98740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158d6d80 0x55a015cc9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f5219c98740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158ef2f0 0x55a015cbf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f5219c98740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158f74d0 0x55a015ccad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f5219c98740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1b1960 con 0x55a0158f74d0 2024-09-21T06:22:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f5219c98740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1b34a0 con 0x55a0158d6d80 2024-09-21T06:22:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f5219c98740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1b2030 con 0x55a0158ef2f0 2024-09-21T06:22:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f520bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158d6d80 0x55a015cc9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f520bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158f74d0 0x55a015ccad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f520bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158d6d80 0x55a015cc9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:46552/0 (socket says 172.21.15.3:46552) 2024-09-21T06:22:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f520bfff640 1 -- 172.21.15.3:0/2509561182 learned_addr learned my addr 172.21.15.3:0/2509561182 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f5210ac7640 1 --2- 172.21.15.3:0/2509561182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158ef2f0 0x55a015cbf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f520bfff640 1 -- 172.21.15.3:0/2509561182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158ef2f0 msgr2=0x55a015cbf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f520bfff640 1 --2- 172.21.15.3:0/2509561182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158ef2f0 0x55a015cbf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f520bfff640 1 -- 172.21.15.3:0/2509561182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158f74d0 msgr2=0x55a015ccad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f520bfff640 1 --2- 172.21.15.3:0/2509561182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158f74d0 0x55a015ccad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.079+0000 7f520bfff640 1 -- 172.21.15.3:0/2509561182 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a015ccb6f0 con 0x55a0158d6d80 2024-09-21T06:22:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.080+0000 7f520bfff640 1 --2- 172.21.15.3:0/2509561182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158d6d80 0x55a015cc9610 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5200025400 tx=0x7f5200005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c857dd0adf8c0bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.080+0000 7f51f27fc640 1 -- 172.21.15.3:0/2509561182 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200013070 con 0x55a0158d6d80 2024-09-21T06:22:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.080+0000 7f51f27fc640 1 -- 172.21.15.3:0/2509561182 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5200013210 con 0x55a0158d6d80 2024-09-21T06:22:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.080+0000 7f51f27fc640 1 -- 172.21.15.3:0/2509561182 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520000e7a0 con 0x55a0158d6d80 2024-09-21T06:22:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.081+0000 7f5219c98740 1 -- 172.21.15.3:0/2509561182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158d6d80 msgr2=0x55a015cc9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.081+0000 7f5219c98740 1 --2- 172.21.15.3:0/2509561182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158d6d80 0x55a015cc9610 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5200025400 tx=0x7f5200005850 comp rx=0 tx=0).stop 2024-09-21T06:22:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.081+0000 7f5219c98740 1 -- 172.21.15.3:0/2509561182 shutdown_connections 2024-09-21T06:22:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.081+0000 7f5219c98740 1 --2- 172.21.15.3:0/2509561182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158ef2f0 0x55a015cbf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.081+0000 7f5219c98740 1 --2- 172.21.15.3:0/2509561182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158f74d0 0x55a015ccad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.081+0000 7f5219c98740 1 --2- 172.21.15.3:0/2509561182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158d6d80 0x55a015cc9610 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.081+0000 7f5219c98740 1 -- 172.21.15.3:0/2509561182 >> 172.21.15.3:0/2509561182 conn(0x55a01590c300 msgr2=0x55a015cca650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.081+0000 7f5219c98740 1 -- 172.21.15.3:0/2509561182 shutdown_connections 2024-09-21T06:22:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.081+0000 7f5219c98740 1 -- 172.21.15.3:0/2509561182 wait complete. 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.081+0000 7f5219c98740 1 -- start start 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.082+0000 7f5219c98740 1 Processor -- start 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.082+0000 7f5219c98740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158d6d80 0x55a015cd18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.082+0000 7f5219c98740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158ef2f0 0x55a015cccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.082+0000 7f5219c98740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158f74d0 0x55a015ccd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.082+0000 7f5219c98740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1b2dd0 con 0x55a0158d6d80 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.082+0000 7f5219c98740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1b2700 con 0x55a0158f74d0 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.082+0000 7f5219c98740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1af960 con 0x55a0158ef2f0 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.082+0000 7f5219c98740 10 client.0 _ll_register_callbacks cb 0x7f5200009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.082+0000 7f520bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158d6d80 0x55a015cd18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.082+0000 7f5210ac7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158ef2f0 0x55a015cccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.082+0000 7f520bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158d6d80 0x55a015cd18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53072/0 (socket says 172.21.15.3:53072) 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.082+0000 7f520bfff640 1 -- 172.21.15.3:0/3278525099 learned_addr learned my addr 172.21.15.3:0/3278525099 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.082+0000 7f520bfff640 1 --2- 172.21.15.3:0/3278525099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158f74d0 0x55a015ccd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f520bfff640 1 -- 172.21.15.3:0/3278525099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158ef2f0 msgr2=0x55a015cccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f520bfff640 1 --2- 172.21.15.3:0/3278525099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158ef2f0 0x55a015cccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f520bfff640 1 -- 172.21.15.3:0/3278525099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158f74d0 msgr2=0x55a015ccd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f520bfff640 1 --2- 172.21.15.3:0/3278525099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158f74d0 0x55a015ccd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f520bfff640 1 -- 172.21.15.3:0/3278525099 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a015ccd7c0 con 0x55a0158d6d80 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f5210ac7640 1 --2- 172.21.15.3:0/3278525099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158ef2f0 0x55a015cccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f520bfff640 1 --2- 172.21.15.3:0/3278525099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158d6d80 0x55a015cd18a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5200009350 tx=0x7f5200025430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c17488808dafaf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f51ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f51ebfff640 1 -- 172.21.15.3:0/3278525099 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200005070 con 0x55a0158d6d80 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f51ebfff640 1 -- 172.21.15.3:0/3278525099 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f5200005210 con 0x55a0158d6d80 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f5219c98740 1 -- 172.21.15.3:0/3278525099 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5200013210 con 0x55a0158d6d80 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f51ebfff640 1 -- 172.21.15.3:0/3278525099 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5200042050 con 0x55a0158d6d80 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f5219c98740 10 client.17604 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f5219c98740 1 -- 172.21.15.3:0/3278525099 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a015d45810 con 0x55a0158d6d80 2024-09-21T06:22:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.083+0000 7f51ebfff640 1 -- 172.21.15.3:0/3278525099 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5200013210 con 0x55a0158d6d80 2024-09-21T06:22:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.084+0000 7f51ebfff640 1 -- 172.21.15.3:0/3278525099 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f52000039d0 con 0x55a0158d6d80 2024-09-21T06:22:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.084+0000 7f5219c98740 10 client.17604 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.084+0000 7f5219c98740 10 client.17604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.084+0000 7f5219c98740 20 client.17604 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.084+0000 7f5219c98740 1 --2- 172.21.15.3:0/3278525099 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a0159181b0 0x55a015d48020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.084+0000 7f5219c98740 4 client.17604 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:22:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.084+0000 7f5219c98740 1 -- 172.21.15.3:0/3278525099 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a015c300f0 con 0x55a0159181b0 2024-09-21T06:22:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.084+0000 7f5210ac7640 1 --2- 172.21.15.3:0/3278525099 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a0159181b0 0x55a015d48020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.084+0000 7f51ebfff640 10 client.17604 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.084+0000 7f5210ac7640 1 --2- 172.21.15.3:0/3278525099 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a0159181b0 0x55a015d48020 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de59eb76497f2852 server_cookie=96176243da0d9928 in_seq=0 out_seq=0 2024-09-21T06:22:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.204+0000 7f51ebfff640 1 -- 172.21.15.3:0/3278525099 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a015c300f0 con 0x55a0159181b0 2024-09-21T06:22:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.204+0000 7f51ebfff640 10 client.17604 handle_command_reply: tid=0 2024-09-21T06:22:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.205+0000 7f5219c98740 1 client.17604 shutdown 2024-09-21T06:22:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.205+0000 7f5219c98740 1 -- 172.21.15.3:0/3278525099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158d6d80 msgr2=0x55a015cd18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.205+0000 7f5219c98740 1 --2- 172.21.15.3:0/3278525099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158d6d80 0x55a015cd18a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5200009350 tx=0x7f5200025430 comp rx=0 tx=0).stop 2024-09-21T06:22:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.205+0000 7f5219c98740 1 -- 172.21.15.3:0/3278525099 shutdown_connections 2024-09-21T06:22:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.205+0000 7f5219c98740 1 --2- 172.21.15.3:0/3278525099 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55a0159181b0 0x55a015d48020 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.205+0000 7f5219c98740 1 --2- 172.21.15.3:0/3278525099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0158ef2f0 0x55a015cccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.205+0000 7f5219c98740 1 --2- 172.21.15.3:0/3278525099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0158f74d0 0x55a015ccd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.205+0000 7f51ebfff640 0 client.17604 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.205+0000 7f5219c98740 1 --2- 172.21.15.3:0/3278525099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a0158d6d80 0x55a015cd18a0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.205+0000 7f5219c98740 1 -- 172.21.15.3:0/3278525099 >> 172.21.15.3:0/3278525099 conn(0x55a01590c300 msgr2=0x55a015cbf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.205+0000 7f5219c98740 1 -- 172.21.15.3:0/3278525099 shutdown_connections 2024-09-21T06:22:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.205+0000 7f5219c98740 1 -- 172.21.15.3:0/3278525099 wait complete. 2024-09-21T06:22:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.205+0000 7f5219c98740 20 client.17604 trim_cache size 0 max 16384 2024-09-21T06:22:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:22:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:22:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:22:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:22:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.206+0000 7f5212d2b640 1 -- 172.21.15.3:0/206864965 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51f40b66b0 msgr2=0x7f51f40b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.206+0000 7f5212d2b640 1 --2- 172.21.15.3:0/206864965 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51f40b66b0 0x7f51f40b8bc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5204031440 tx=0x7f520409a520 comp rx=0 tx=0).stop 2024-09-21T06:22:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.206+0000 7f5212d2b640 1 -- 172.21.15.3:0/206864965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c109d80 msgr2=0x7f520c076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.206+0000 7f5212d2b640 1 --2- 172.21.15.3:0/206864965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c109d80 0x7f520c076190 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5204005940 tx=0x7f5204005e20 comp rx=0 tx=0).stop 2024-09-21T06:22:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.206+0000 7f5212d2b640 1 -- 172.21.15.3:0/206864965 shutdown_connections 2024-09-21T06:22:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.206+0000 7f5212d2b640 1 --2- 172.21.15.3:0/206864965 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f51f40b66b0 0x7f51f40b8bc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.206+0000 7f5212d2b640 1 --2- 172.21.15.3:0/206864965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f520c109430 0x7f520c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.206+0000 7f5212d2b640 1 --2- 172.21.15.3:0/206864965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c109d80 0x7f520c076190 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.206+0000 7f5212d2b640 1 --2- 172.21.15.3:0/206864965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f520c103b00 0x7f520c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.206+0000 7f5212d2b640 1 -- 172.21.15.3:0/206864965 >> 172.21.15.3:0/206864965 conn(0x7f520c0ff500 msgr2=0x7f520c108a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.206+0000 7f5212d2b640 1 -- 172.21.15.3:0/206864965 shutdown_connections 2024-09-21T06:22:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:10.206+0000 7f5212d2b640 1 -- 172.21.15.3:0/206864965 wait complete. 2024-09-21T06:22:10.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 756658 11.24 MB/sec execute 223 sec latency 66.198 ms 2024-09-21T06:22:11.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:11 smithi003 ceph-mon[31227]: pgmap v543: 97 pgs: 97 active+clean; 127 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 265 op/s 2024-09-21T06:22:11.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:11 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:22:11.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:11 smithi137 ceph-mon[34464]: pgmap v543: 97 pgs: 97 active+clean; 127 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 265 op/s 2024-09-21T06:22:11.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:11 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:22:11.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:11 smithi204 ceph-mon[34659]: pgmap v543: 97 pgs: 97 active+clean; 127 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 265 op/s 2024-09-21T06:22:11.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:11 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:22:11.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 758093 11.22 MB/sec execute 224 sec latency 50.928 ms 2024-09-21T06:22:12.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:12 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:22:12.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:12 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:22:12.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:12 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:22:12.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:12 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:22:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:12 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:22:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:12 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:22:12.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 760266 11.23 MB/sec execute 225 sec latency 16.262 ms 2024-09-21T06:22:13.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:13 smithi137 ceph-mon[34464]: pgmap v544: 97 pgs: 97 active+clean; 127 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-09-21T06:22:13.804 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:13 smithi204 ceph-mon[34659]: pgmap v544: 97 pgs: 97 active+clean; 127 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-09-21T06:22:13.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:13 smithi003 ceph-mon[31227]: pgmap v544: 97 pgs: 97 active+clean; 127 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-09-21T06:22:13.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 762363 11.21 MB/sec execute 226 sec latency 17.235 ms 2024-09-21T06:22:14.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 764168 11.21 MB/sec execute 227 sec latency 26.075 ms 2024-09-21T06:22:15.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:15 smithi003 ceph-mon[31227]: pgmap v545: 97 pgs: 97 active+clean; 123 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2024-09-21T06:22:15.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:15 smithi137 ceph-mon[34464]: pgmap v545: 97 pgs: 97 active+clean; 123 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2024-09-21T06:22:15.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:15 smithi204 ceph-mon[34659]: pgmap v545: 97 pgs: 97 active+clean; 123 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2024-09-21T06:22:15.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 766582 11.22 MB/sec execute 228 sec latency 27.571 ms 2024-09-21T06:22:16.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 767831 11.20 MB/sec execute 229 sec latency 43.822 ms 2024-09-21T06:22:17.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:17 smithi003 ceph-mon[31227]: pgmap v546: 97 pgs: 97 active+clean; 125 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 353 op/s 2024-09-21T06:22:17.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:17 smithi137 ceph-mon[34464]: pgmap v546: 97 pgs: 97 active+clean; 125 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 353 op/s 2024-09-21T06:22:17.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:17 smithi204 ceph-mon[34659]: pgmap v546: 97 pgs: 97 active+clean; 125 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 353 op/s 2024-09-21T06:22:17.964 INFO:tasks.workunit.client.0.smithi003.stdout: 1 770465 11.23 MB/sec execute 230 sec latency 19.730 ms 2024-09-21T06:22:18.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 771992 11.21 MB/sec execute 231 sec latency 17.046 ms 2024-09-21T06:22:19.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:19 smithi003 ceph-mon[31227]: pgmap v547: 97 pgs: 97 active+clean; 125 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-09-21T06:22:19.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:19 smithi137 ceph-mon[34464]: pgmap v547: 97 pgs: 97 active+clean; 125 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-09-21T06:22:19.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:19 smithi204 ceph-mon[34659]: pgmap v547: 97 pgs: 97 active+clean; 125 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-09-21T06:22:19.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 774328 11.22 MB/sec execute 232 sec latency 15.701 ms 2024-09-21T06:22:20.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 776140 11.19 MB/sec execute 233 sec latency 16.586 ms 2024-09-21T06:22:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:21 smithi003 ceph-mon[31227]: pgmap v548: 97 pgs: 97 active+clean; 129 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 353 op/s 2024-09-21T06:22:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:21 smithi137 ceph-mon[34464]: pgmap v548: 97 pgs: 97 active+clean; 129 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 353 op/s 2024-09-21T06:22:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:21 smithi204 ceph-mon[34659]: pgmap v548: 97 pgs: 97 active+clean; 129 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 353 op/s 2024-09-21T06:22:21.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 777960 11.20 MB/sec execute 234 sec latency 43.036 ms 2024-09-21T06:22:22.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 779839 11.18 MB/sec execute 235 sec latency 18.154 ms 2024-09-21T06:22:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.306+0000 7ff72716a640 1 -- 172.21.15.3:0/2139931786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff720103ad0 msgr2=0x7ff720103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.306+0000 7ff72716a640 1 --2- 172.21.15.3:0/2139931786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff720103ad0 0x7ff720103ee0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff710009900 tx=0x7ff71002a410 comp rx=0 tx=0).stop 2024-09-21T06:22:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.306+0000 7ff72716a640 1 -- 172.21.15.3:0/2139931786 shutdown_connections 2024-09-21T06:22:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.306+0000 7ff72716a640 1 --2- 172.21.15.3:0/2139931786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff720109400 0x7ff720109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.306+0000 7ff72716a640 1 --2- 172.21.15.3:0/2139931786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720109d50 0x7ff720110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.306+0000 7ff72716a640 1 --2- 172.21.15.3:0/2139931786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff720103ad0 0x7ff720103ee0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.306+0000 7ff72716a640 1 -- 172.21.15.3:0/2139931786 >> 172.21.15.3:0/2139931786 conn(0x7ff7200ff510 msgr2=0x7ff720101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.306+0000 7ff72716a640 1 -- 172.21.15.3:0/2139931786 shutdown_connections 2024-09-21T06:22:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.306+0000 7ff72716a640 1 -- 172.21.15.3:0/2139931786 wait complete. 2024-09-21T06:22:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.307+0000 7ff72716a640 1 Processor -- start 2024-09-21T06:22:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.307+0000 7ff72716a640 1 -- start start 2024-09-21T06:22:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.307+0000 7ff72716a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720103ad0 0x7ff72019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.307+0000 7ff72716a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff720109400 0x7ff72019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.307+0000 7ff72716a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff720109d50 0x7ff72019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.307+0000 7ff72716a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff720113a60 con 0x7ff720103ad0 2024-09-21T06:22:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.307+0000 7ff72716a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201138e0 con 0x7ff720109400 2024-09-21T06:22:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.307+0000 7ff72716a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff720113be0 con 0x7ff720109d50 2024-09-21T06:22:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.307+0000 7ff724f06640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720103ad0 0x7ff72019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.307+0000 7ff724f06640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720103ad0 0x7ff72019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34092/0 (socket says 172.21.15.3:34092) 2024-09-21T06:22:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.307+0000 7ff724f06640 1 -- 172.21.15.3:0/696352403 learned_addr learned my addr 172.21.15.3:0/696352403 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.307+0000 7ff71ffff640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff720109400 0x7ff72019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.307+0000 7ff725707640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff720109d50 0x7ff72019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.308+0000 7ff724f06640 1 -- 172.21.15.3:0/696352403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff720109d50 msgr2=0x7ff72019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.308+0000 7ff724f06640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff720109d50 0x7ff72019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.308+0000 7ff724f06640 1 -- 172.21.15.3:0/696352403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff720109400 msgr2=0x7ff72019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.308+0000 7ff724f06640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff720109400 0x7ff72019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.308+0000 7ff724f06640 1 -- 172.21.15.3:0/696352403 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7201a03f0 con 0x7ff720103ad0 2024-09-21T06:22:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.308+0000 7ff724f06640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720103ad0 0x7ff72019a4c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff710005ab0 tx=0x7ff710003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.308+0000 7ff71dffb640 1 -- 172.21.15.3:0/696352403 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71003f070 con 0x7ff720103ad0 2024-09-21T06:22:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.308+0000 7ff71dffb640 1 -- 172.21.15.3:0/696352403 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff710031b60 con 0x7ff720103ad0 2024-09-21T06:22:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.308+0000 7ff71dffb640 1 -- 172.21.15.3:0/696352403 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710035030 con 0x7ff720103ad0 2024-09-21T06:22:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.308+0000 7ff72716a640 1 -- 172.21.15.3:0/696352403 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7201a0680 con 0x7ff720103ad0 2024-09-21T06:22:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.308+0000 7ff72716a640 1 -- 172.21.15.3:0/696352403 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7201a0b20 con 0x7ff720103ad0 2024-09-21T06:22:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.311+0000 7ff72e0d4740 1 Processor -- start 2024-09-21T06:22:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.311+0000 7ff72e0d4740 1 -- start start 2024-09-21T06:22:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff72e0d4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3aff1b0 0x562ec3ea49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff72e0d4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 0x562ec3ea4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff72e0d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3ade4d0 0x562ec3ea5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff72e0d4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562ec3e196f0 con 0x562ec3aff1b0 2024-09-21T06:22:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff72e0d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562ec3e08b20 con 0x562ec3ade4d0 2024-09-21T06:22:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff72e0d4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562ec3e0c930 con 0x562ec3ad62f0 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff71dffb640 1 -- 172.21.15.3:0/696352403 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7ff710042070 con 0x7ff720103ad0 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff725707640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3aff1b0 0x562ec3ea49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff725707640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3aff1b0 0x562ec3ea49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34098/0 (socket says 172.21.15.3:34098) 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff725707640 1 -- 172.21.15.3:0/1468426900 learned_addr learned my addr 172.21.15.3:0/1468426900 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff71ffff640 1 --2- 172.21.15.3:0/1468426900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3ade4d0 0x562ec3ea5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff724f06640 1 --2- 172.21.15.3:0/1468426900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 0x562ec3ea4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff725707640 1 -- 172.21.15.3:0/1468426900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 msgr2=0x562ec3ea4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff725707640 1 --2- 172.21.15.3:0/1468426900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 0x562ec3ea4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff725707640 1 -- 172.21.15.3:0/1468426900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3ade4d0 msgr2=0x562ec3ea5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff725707640 1 --2- 172.21.15.3:0/1468426900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3ade4d0 0x562ec3ea5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.312+0000 7ff725707640 1 -- 172.21.15.3:0/1468426900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ec3dce530 con 0x562ec3aff1b0 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.313+0000 7ff725707640 1 --2- 172.21.15.3:0/1468426900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3aff1b0 0x562ec3ea49b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff718009f60 tx=0x7ff718009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c1331b25025118 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.313+0000 7ff706ffd640 1 -- 172.21.15.3:0/1468426900 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71801e070 con 0x562ec3aff1b0 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.313+0000 7ff706ffd640 1 -- 172.21.15.3:0/1468426900 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff718003c30 con 0x562ec3aff1b0 2024-09-21T06:22:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.313+0000 7ff706ffd640 1 -- 172.21.15.3:0/1468426900 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71800f940 con 0x562ec3aff1b0 2024-09-21T06:22:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.313+0000 7ff71dffb640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7080b6680 0x7ff7080b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.313+0000 7ff724f06640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7080b6680 0x7ff7080b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.313+0000 7ff71dffb640 1 -- 172.21.15.3:0/696352403 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7ff710107e50 con 0x7ff720103ad0 2024-09-21T06:22:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.314+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1468426900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3aff1b0 msgr2=0x562ec3ea49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.314+0000 7ff724f06640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7080b6680 0x7ff7080b8b90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff710105660 tx=0x7ff710009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.314+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1468426900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3aff1b0 0x562ec3ea49b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff718009f60 tx=0x7ff718009f90 comp rx=0 tx=0).stop 2024-09-21T06:22:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.314+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1468426900 shutdown_connections 2024-09-21T06:22:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.314+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1468426900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 0x562ec3ea4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.314+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1468426900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3ade4d0 0x562ec3ea5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.314+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1468426900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3aff1b0 0x562ec3ea49b0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.314+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1468426900 >> 172.21.15.3:0/1468426900 conn(0x562ec3af3300 msgr2=0x562ec3ea28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.314+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1468426900 shutdown_connections 2024-09-21T06:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.314+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1468426900 wait complete. 2024-09-21T06:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.315+0000 7ff72e0d4740 1 -- start start 2024-09-21T06:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.315+0000 7ff72e0d4740 1 Processor -- start 2024-09-21T06:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.315+0000 7ff72e0d4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 0x562ec3eb0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.315+0000 7ff72e0d4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 0x562ec3eb0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.315+0000 7ff72e0d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3aff1b0 0x562ec3ebc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.315+0000 7ff72e0d4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562ec3d40ab0 con 0x562ec3ade4d0 2024-09-21T06:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.315+0000 7ff72e0d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562ec3dae730 con 0x562ec3aff1b0 2024-09-21T06:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.315+0000 7ff72e0d4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562ec3eb3130 con 0x562ec3ad62f0 2024-09-21T06:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.315+0000 7ff72e0d4740 10 client.0 _ll_register_callbacks cb 0x562ec3e026b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.315+0000 7ff71ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 0x562ec3eb0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff725707640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3aff1b0 0x562ec3ebc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff71ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 0x562ec3eb0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34110/0 (socket says 172.21.15.3:34110) 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff71ffff640 1 -- 172.21.15.3:0/1299477969 learned_addr learned my addr 172.21.15.3:0/1299477969 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff725707640 1 --2- 172.21.15.3:0/1299477969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 0x562ec3eb0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff71ffff640 1 -- 172.21.15.3:0/1299477969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 msgr2=0x562ec3eb0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff71ffff640 1 --2- 172.21.15.3:0/1299477969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 0x562ec3eb0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff71ffff640 1 -- 172.21.15.3:0/1299477969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3aff1b0 msgr2=0x562ec3ebc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff71ffff640 1 --2- 172.21.15.3:0/1299477969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3aff1b0 0x562ec3ebc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff71ffff640 1 -- 172.21.15.3:0/1299477969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ec3b6a790 con 0x562ec3ade4d0 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff725707640 1 --2- 172.21.15.3:0/1299477969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3aff1b0 0x562ec3ebc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff71ffff640 1 --2- 172.21.15.3:0/1299477969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 0x562ec3eb0570 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff714012700 tx=0x7ff714012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436041713b51095f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff6fffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff6fffff640 1 -- 172.21.15.3:0/1299477969 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71400f920 con 0x562ec3ade4d0 2024-09-21T06:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff6fffff640 1 -- 172.21.15.3:0/1299477969 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7ff71400fac0 con 0x562ec3ade4d0 2024-09-21T06:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.316+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1299477969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff718003c30 con 0x562ec3ade4d0 2024-09-21T06:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.317+0000 7ff6fffff640 1 -- 172.21.15.3:0/1299477969 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff71400fcc0 con 0x562ec3ade4d0 2024-09-21T06:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.317+0000 7ff72e0d4740 10 client.17616 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.317+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1299477969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562ec3be3620 con 0x562ec3ade4d0 2024-09-21T06:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.317+0000 7ff6fffff640 1 -- 172.21.15.3:0/1299477969 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff718003c30 con 0x562ec3ade4d0 2024-09-21T06:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.317+0000 7ff6fffff640 1 -- 172.21.15.3:0/1299477969 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7ff71401c420 con 0x562ec3ade4d0 2024-09-21T06:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.317+0000 7ff72e0d4740 10 client.17616 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.317+0000 7ff72e0d4740 10 client.17616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.317+0000 7ff72e0d4740 20 client.17616 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.317+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1299477969 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ec3abdd80 0x562ec3ec2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.317+0000 7ff72e0d4740 4 client.17616 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.317+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1299477969 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ec3e1b640 con 0x562ec3abdd80 2024-09-21T06:22:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.317+0000 7ff725707640 1 --2- 172.21.15.3:0/1299477969 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ec3abdd80 0x562ec3ec2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.318+0000 7ff6fffff640 10 client.17616 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.318+0000 7ff725707640 1 --2- 172.21.15.3:0/1299477969 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ec3abdd80 0x562ec3ec2230 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46027d8a82f777f2 server_cookie=325fc26d102cb17a in_seq=0 out_seq=0 2024-09-21T06:22:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.319+0000 7ff6fffff640 1 -- 172.21.15.3:0/1299477969 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562ec3e1b640 con 0x562ec3abdd80 2024-09-21T06:22:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.319+0000 7ff6fffff640 10 client.17616 handle_command_reply: tid=0 2024-09-21T06:22:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.319+0000 7ff72e0d4740 1 client.17616 shutdown 2024-09-21T06:22:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.319+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1299477969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 msgr2=0x562ec3eb0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.319+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1299477969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 0x562ec3eb0570 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff714012700 tx=0x7ff714012bc0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.319+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1299477969 shutdown_connections 2024-09-21T06:22:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.319+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1299477969 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ec3abdd80 0x562ec3ec2230 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.319+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1299477969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3aff1b0 0x562ec3ebc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.319+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1299477969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 0x562ec3eb0570 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.319+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1299477969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 0x562ec3eb0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.319+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1299477969 >> 172.21.15.3:0/1299477969 conn(0x562ec3af3300 msgr2=0x562ec3ea59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.320+0000 7ff6fffff640 0 client.17616 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.320+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1299477969 shutdown_connections 2024-09-21T06:22:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.320+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1299477969 wait complete. 2024-09-21T06:22:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.320+0000 7ff72e0d4740 20 client.17616 trim_cache size 0 max 16384 2024-09-21T06:22:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.335+0000 7ff72e0d4740 1 Processor -- start 2024-09-21T06:22:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.335+0000 7ff72e0d4740 1 -- start start 2024-09-21T06:22:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.335+0000 7ff72e0d4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 0x562ec3ea60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.335+0000 7ff72e0d4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 0x562ec3ebc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.335+0000 7ff72e0d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3aff1b0 0x562ec3ead200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.335+0000 7ff72e0d4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201af7f0 con 0x562ec3ade4d0 2024-09-21T06:22:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.335+0000 7ff72e0d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201b1330 con 0x562ec3aff1b0 2024-09-21T06:22:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.335+0000 7ff72e0d4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201afec0 con 0x562ec3ad62f0 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.335+0000 7ff71ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 0x562ec3ebc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.335+0000 7ff71ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 0x562ec3ebc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34114/0 (socket says 172.21.15.3:34114) 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.335+0000 7ff71ffff640 1 -- 172.21.15.3:0/1446471531 learned_addr learned my addr 172.21.15.3:0/1446471531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.336+0000 7ff725707640 1 --2- 172.21.15.3:0/1446471531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 0x562ec3ea60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.336+0000 7ff725707640 1 --2- 172.21.15.3:0/1446471531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3aff1b0 0x562ec3ead200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.336+0000 7ff71ffff640 1 -- 172.21.15.3:0/1446471531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 msgr2=0x562ec3ea60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.336+0000 7ff71ffff640 1 --2- 172.21.15.3:0/1446471531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 0x562ec3ea60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.336+0000 7ff71ffff640 1 -- 172.21.15.3:0/1446471531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3aff1b0 msgr2=0x562ec3ead200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.336+0000 7ff71ffff640 1 --2- 172.21.15.3:0/1446471531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3aff1b0 0x562ec3ead200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.336+0000 7ff71ffff640 1 -- 172.21.15.3:0/1446471531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ec3e9dd50 con 0x562ec3ade4d0 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.336+0000 7ff71ffff640 1 --2- 172.21.15.3:0/1446471531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 0x562ec3ebc320 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff714012420 tx=0x7ff71400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaffb5eac80451e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.336+0000 7ff7067fc640 1 -- 172.21.15.3:0/1446471531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71400fe60 con 0x562ec3ade4d0 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.336+0000 7ff7067fc640 1 -- 172.21.15.3:0/1446471531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff714005560 con 0x562ec3ade4d0 2024-09-21T06:22:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.336+0000 7ff7067fc640 1 -- 172.21.15.3:0/1446471531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7140035f0 con 0x562ec3ade4d0 2024-09-21T06:22:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.337+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1446471531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 msgr2=0x562ec3ebc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.337+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1446471531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 0x562ec3ebc320 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff714012420 tx=0x7ff71400f810 comp rx=0 tx=0).stop 2024-09-21T06:22:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.337+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1446471531 shutdown_connections 2024-09-21T06:22:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.337+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1446471531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3aff1b0 0x562ec3ead200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.337+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1446471531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ade4d0 0x562ec3ebc320 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.337+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/1446471531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3ad62f0 0x562ec3ea60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.337+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1446471531 >> 172.21.15.3:0/1446471531 conn(0x562ec3af3300 msgr2=0x562ec3ea59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.337+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1446471531 shutdown_connections 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.337+0000 7ff72e0d4740 1 -- 172.21.15.3:0/1446471531 wait complete. 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.337+0000 7ff72e0d4740 1 -- start start 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.337+0000 7ff72e0d4740 1 Processor -- start 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff72e0d4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ad62f0 0x562ec3eb56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff72e0d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3ade4d0 0x562ec3eb5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff72e0d4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3aff1b0 0x562ec3eb6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff72e0d4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201b0c60 con 0x562ec3ad62f0 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff72e0d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201b0590 con 0x562ec3ade4d0 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff72e0d4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201ad7f0 con 0x562ec3aff1b0 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff72e0d4740 10 client.0 _ll_register_callbacks cb 0x7ff714009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff725707640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ad62f0 0x562ec3eb56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff725707640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ad62f0 0x562ec3eb56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34128/0 (socket says 172.21.15.3:34128) 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff71ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3ade4d0 0x562ec3eb5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff725707640 1 -- 172.21.15.3:0/3540606723 learned_addr learned my addr 172.21.15.3:0/3540606723 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff725707640 1 --2- 172.21.15.3:0/3540606723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3aff1b0 0x562ec3eb6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff725707640 1 -- 172.21.15.3:0/3540606723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3aff1b0 msgr2=0x562ec3eb6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff725707640 1 --2- 172.21.15.3:0/3540606723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3aff1b0 0x562ec3eb6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.338+0000 7ff725707640 1 -- 172.21.15.3:0/3540606723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3ade4d0 msgr2=0x562ec3eb5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.339+0000 7ff725707640 1 --2- 172.21.15.3:0/3540606723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3ade4d0 0x562ec3eb5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.339+0000 7ff725707640 1 -- 172.21.15.3:0/3540606723 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ec3f33790 con 0x562ec3ad62f0 2024-09-21T06:22:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.339+0000 7ff71ffff640 1 --2- 172.21.15.3:0/3540606723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3ade4d0 0x562ec3eb5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.339+0000 7ff725707640 1 --2- 172.21.15.3:0/3540606723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ad62f0 0x562ec3eb56a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff71800f4f0 tx=0x7ff714009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c20d674af440da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.339+0000 7ff6fffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.339+0000 7ff6fffff640 1 -- 172.21.15.3:0/3540606723 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71801e070 con 0x562ec3ad62f0 2024-09-21T06:22:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.339+0000 7ff6fffff640 1 -- 172.21.15.3:0/3540606723 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7ff718004050 con 0x562ec3ad62f0 2024-09-21T06:22:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.339+0000 7ff72e0d4740 1 -- 172.21.15.3:0/3540606723 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff714005560 con 0x562ec3ad62f0 2024-09-21T06:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.339+0000 7ff6fffff640 1 -- 172.21.15.3:0/3540606723 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff718004250 con 0x562ec3ad62f0 2024-09-21T06:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.339+0000 7ff72e0d4740 10 client.17622 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.339+0000 7ff72e0d4740 1 -- 172.21.15.3:0/3540606723 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562ec3f34980 con 0x562ec3ad62f0 2024-09-21T06:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.339+0000 7ff6fffff640 1 -- 172.21.15.3:0/3540606723 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff714005560 con 0x562ec3ad62f0 2024-09-21T06:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.339+0000 7ff6fffff640 1 -- 172.21.15.3:0/3540606723 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7ff71800fc50 con 0x562ec3ad62f0 2024-09-21T06:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.340+0000 7ff72e0d4740 10 client.17622 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.340+0000 7ff72e0d4740 10 client.17622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.340+0000 7ff72e0d4740 20 client.17622 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.340+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/3540606723 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ec3a931c0 0x562ec3f36fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.340+0000 7ff72e0d4740 4 client.17622 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.340+0000 7ff72e0d4740 1 -- 172.21.15.3:0/3540606723 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ec3e170f0 con 0x562ec3a931c0 2024-09-21T06:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.340+0000 7ff71ffff640 1 --2- 172.21.15.3:0/3540606723 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ec3a931c0 0x562ec3f36fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.340+0000 7ff6fffff640 10 client.17622 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.340+0000 7ff71ffff640 1 --2- 172.21.15.3:0/3540606723 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ec3a931c0 0x562ec3f36fd0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=388a2e338d9672ae server_cookie=2c7fd1fdc5681cb8 in_seq=0 out_seq=0 2024-09-21T06:22:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff6fffff640 1 -- 172.21.15.3:0/3540606723 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562ec3e170f0 con 0x562ec3a931c0 2024-09-21T06:22:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff6fffff640 10 client.17622 handle_command_reply: tid=0 2024-09-21T06:22:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff72e0d4740 1 client.17622 shutdown 2024-09-21T06:22:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff72e0d4740 1 -- 172.21.15.3:0/3540606723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ad62f0 msgr2=0x562ec3eb56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/3540606723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ad62f0 0x562ec3eb56a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff71800f4f0 tx=0x7ff714009e30 comp rx=0 tx=0).stop 2024-09-21T06:22:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff72e0d4740 1 -- 172.21.15.3:0/3540606723 shutdown_connections 2024-09-21T06:22:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/3540606723 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ec3a931c0 0x562ec3f36fd0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/3540606723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ec3aff1b0 0x562ec3eb6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff6fffff640 0 client.17622 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/3540606723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec3ade4d0 0x562ec3eb5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff72e0d4740 1 --2- 172.21.15.3:0/3540606723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ec3ad62f0 0x562ec3eb56a0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff72e0d4740 1 -- 172.21.15.3:0/3540606723 >> 172.21.15.3:0/3540606723 conn(0x562ec3af3300 msgr2=0x562ec3ea2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff72e0d4740 1 -- 172.21.15.3:0/3540606723 shutdown_connections 2024-09-21T06:22:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff72e0d4740 1 -- 172.21.15.3:0/3540606723 wait complete. 2024-09-21T06:22:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.484+0000 7ff72e0d4740 20 client.17622 trim_cache size 0 max 16384 2024-09-21T06:22:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:22:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:22:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:22:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:22:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.485+0000 7ff72716a640 1 -- 172.21.15.3:0/696352403 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7080b6680 msgr2=0x7ff7080b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.485+0000 7ff72716a640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7080b6680 0x7ff7080b8b90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff710105660 tx=0x7ff710009260 comp rx=0 tx=0).stop 2024-09-21T06:22:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.485+0000 7ff72716a640 1 -- 172.21.15.3:0/696352403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720103ad0 msgr2=0x7ff72019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.485+0000 7ff72716a640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720103ad0 0x7ff72019a4c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff710005ab0 tx=0x7ff710003c10 comp rx=0 tx=0).stop 2024-09-21T06:22:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.485+0000 7ff72716a640 1 -- 172.21.15.3:0/696352403 shutdown_connections 2024-09-21T06:22:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.485+0000 7ff72716a640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff7080b6680 0x7ff7080b8b90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.485+0000 7ff72716a640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff720109d50 0x7ff72019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.485+0000 7ff72716a640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff720109400 0x7ff72019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.485+0000 7ff72716a640 1 --2- 172.21.15.3:0/696352403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720103ad0 0x7ff72019a4c0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.485+0000 7ff72716a640 1 -- 172.21.15.3:0/696352403 >> 172.21.15.3:0/696352403 conn(0x7ff7200ff510 msgr2=0x7ff720101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.485+0000 7ff72716a640 1 -- 172.21.15.3:0/696352403 shutdown_connections 2024-09-21T06:22:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:23.485+0000 7ff72716a640 1 -- 172.21.15.3:0/696352403 wait complete. 2024-09-21T06:22:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:23 smithi003 ceph-mon[31227]: pgmap v549: 97 pgs: 97 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 320 op/s 2024-09-21T06:22:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:23 smithi137 ceph-mon[34464]: pgmap v549: 97 pgs: 97 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 320 op/s 2024-09-21T06:22:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:23 smithi204 ceph-mon[34659]: pgmap v549: 97 pgs: 97 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 320 op/s 2024-09-21T06:22:23.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 781553 11.19 MB/sec execute 236 sec latency 70.999 ms 2024-09-21T06:22:24.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 783594 11.17 MB/sec execute 237 sec latency 15.697 ms 2024-09-21T06:22:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:25 smithi003 ceph-mon[31227]: pgmap v550: 97 pgs: 97 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 259 op/s 2024-09-21T06:22:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:25 smithi137 ceph-mon[34464]: pgmap v550: 97 pgs: 97 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 259 op/s 2024-09-21T06:22:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:25 smithi204 ceph-mon[34659]: pgmap v550: 97 pgs: 97 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 259 op/s 2024-09-21T06:22:25.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 785348 11.18 MB/sec execute 238 sec latency 30.837 ms 2024-09-21T06:22:26.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:26 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:22:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:26 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:22:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:26 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:22:26.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 787448 11.18 MB/sec execute 239 sec latency 113.253 ms 2024-09-21T06:22:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:27 smithi003 ceph-mon[31227]: pgmap v551: 97 pgs: 97 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 383 op/s 2024-09-21T06:22:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:22:27.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:22:27.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:22:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:27 smithi137 ceph-mon[34464]: pgmap v551: 97 pgs: 97 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 383 op/s 2024-09-21T06:22:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:22:27.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:22:27.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:22:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:27 smithi204 ceph-mon[34659]: pgmap v551: 97 pgs: 97 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 383 op/s 2024-09-21T06:22:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:22:27.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:22:27.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:22:27.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:22:27.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 789097 11.17 MB/sec execute 240 sec latency 21.373 ms 2024-09-21T06:22:28.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:28 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-21T06:22:28.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:28 smithi003 ceph-mon[31227]: Adjusting osd_memory_target on smithi137 to 176.0M 2024-09-21T06:22:28.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:28 smithi003 ceph-mon[31227]: Unable to set osd_memory_target on smithi204 to 184561664: error parsing value: Value '184561664' is below minimum 939524096 2024-09-21T06:22:28.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:28 smithi003 ceph-mon[31227]: Unable to set osd_memory_target on smithi137 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T06:22:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:28 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-21T06:22:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:28 smithi137 ceph-mon[34464]: Adjusting osd_memory_target on smithi137 to 176.0M 2024-09-21T06:22:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:28 smithi137 ceph-mon[34464]: Unable to set osd_memory_target on smithi204 to 184561664: error parsing value: Value '184561664' is below minimum 939524096 2024-09-21T06:22:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:28 smithi137 ceph-mon[34464]: Unable to set osd_memory_target on smithi137 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T06:22:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:28 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-21T06:22:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:28 smithi204 ceph-mon[34659]: Adjusting osd_memory_target on smithi137 to 176.0M 2024-09-21T06:22:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:28 smithi204 ceph-mon[34659]: Unable to set osd_memory_target on smithi204 to 184561664: error parsing value: Value '184561664' is below minimum 939524096 2024-09-21T06:22:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:28 smithi204 ceph-mon[34659]: Unable to set osd_memory_target on smithi137 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T06:22:28.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 791852 11.19 MB/sec execute 241 sec latency 15.467 ms 2024-09-21T06:22:29.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:29 smithi003 ceph-mon[31227]: pgmap v552: 97 pgs: 97 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 297 op/s 2024-09-21T06:22:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:29 smithi137 ceph-mon[34464]: pgmap v552: 97 pgs: 97 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 297 op/s 2024-09-21T06:22:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:29 smithi204 ceph-mon[34659]: pgmap v552: 97 pgs: 97 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 297 op/s 2024-09-21T06:22:29.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 793334 11.17 MB/sec execute 242 sec latency 18.206 ms 2024-09-21T06:22:30.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:30 smithi003 ceph-mon[31227]: pgmap v553: 97 pgs: 97 active+clean; 127 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 372 op/s 2024-09-21T06:22:30.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:30 smithi137 ceph-mon[34464]: pgmap v553: 97 pgs: 97 active+clean; 127 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 372 op/s 2024-09-21T06:22:30.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:30 smithi204 ceph-mon[34659]: pgmap v553: 97 pgs: 97 active+clean; 127 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 372 op/s 2024-09-21T06:22:30.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 795638 11.18 MB/sec execute 243 sec latency 18.585 ms 2024-09-21T06:22:31.966 INFO:tasks.workunit.client.0.smithi003.stdout: 1 797086 11.16 MB/sec execute 244 sec latency 95.229 ms 2024-09-21T06:22:32.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 799245 11.17 MB/sec execute 245 sec latency 19.296 ms 2024-09-21T06:22:33.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:32 smithi003 ceph-mon[31227]: pgmap v554: 97 pgs: 97 active+clean; 127 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-09-21T06:22:33.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:32 smithi137 ceph-mon[34464]: pgmap v554: 97 pgs: 97 active+clean; 127 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-09-21T06:22:33.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:32 smithi204 ceph-mon[34659]: pgmap v554: 97 pgs: 97 active+clean; 127 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-09-21T06:22:33.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 801252 11.15 MB/sec execute 246 sec latency 22.086 ms 2024-09-21T06:22:34.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 803134 11.16 MB/sec execute 247 sec latency 16.776 ms 2024-09-21T06:22:35.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:34 smithi003 ceph-mon[31227]: pgmap v555: 97 pgs: 97 active+clean; 133 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-09-21T06:22:35.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:34 smithi137 ceph-mon[34464]: pgmap v555: 97 pgs: 97 active+clean; 133 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-09-21T06:22:35.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:34 smithi204 ceph-mon[34659]: pgmap v555: 97 pgs: 97 active+clean; 133 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-09-21T06:22:35.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 805921 11.18 MB/sec execute 248 sec latency 15.767 ms 2024-09-21T06:22:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.595+0000 7f77b148b640 1 -- 172.21.15.3:0/2408892073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77ac10dd50 msgr2=0x7f77ac110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.595+0000 7f77b148b640 1 --2- 172.21.15.3:0/2408892073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77ac10dd50 0x7f77ac110260 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009ca0 tx=0x7f77a402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.596+0000 7f77b148b640 1 -- 172.21.15.3:0/2408892073 shutdown_connections 2024-09-21T06:22:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.596+0000 7f77b148b640 1 --2- 172.21.15.3:0/2408892073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77ac10dd50 0x7f77ac110260 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.596+0000 7f77b148b640 1 --2- 172.21.15.3:0/2408892073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77ac1052d0 0x7f77ac10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.596+0000 7f77b148b640 1 --2- 172.21.15.3:0/2408892073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77ac104980 0x7f77ac104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.596+0000 7f77b148b640 1 -- 172.21.15.3:0/2408892073 >> 172.21.15.3:0/2408892073 conn(0x7f77ac07a6f0 msgr2=0x7f77ac07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.596+0000 7f77b148b640 1 -- 172.21.15.3:0/2408892073 shutdown_connections 2024-09-21T06:22:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.596+0000 7f77b148b640 1 -- 172.21.15.3:0/2408892073 wait complete. 2024-09-21T06:22:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.596+0000 7f77b148b640 1 Processor -- start 2024-09-21T06:22:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.596+0000 7f77b148b640 1 -- start start 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77b148b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77ac104980 0x7f77ac1026b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77b148b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77ac1052d0 0x7f77ac102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77b148b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77ac10dd50 0x7f77ac103130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77b148b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1139d0 con 0x7f77ac10dd50 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77b148b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac113850 con 0x7f77ac104980 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77b148b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac113b50 con 0x7f77ac1052d0 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77ab7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77ac10dd50 0x7f77ac103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77aa7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77ac1052d0 0x7f77ac102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77ac104980 0x7f77ac1026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77aa7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77ac1052d0 0x7f77ac102bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:38402/0 (socket says 172.21.15.3:38402) 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77ab7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77ac10dd50 0x7f77ac103130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47772/0 (socket says 172.21.15.3:47772) 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77aa7fc640 1 -- 172.21.15.3:0/1375609218 learned_addr learned my addr 172.21.15.3:0/1375609218 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77aa7fc640 1 -- 172.21.15.3:0/1375609218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77ac104980 msgr2=0x7f77ac1026b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77aa7fc640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77ac104980 0x7f77ac1026b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77aa7fc640 1 -- 172.21.15.3:0/1375609218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77ac10dd50 msgr2=0x7f77ac103130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77aa7fc640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77ac10dd50 0x7f77ac103130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77aa7fc640 1 -- 172.21.15.3:0/1375609218 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac1a8620 con 0x7f77ac1052d0 2024-09-21T06:22:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77aaffd640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77ac104980 0x7f77ac1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.597+0000 7f77ab7fe640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77ac10dd50 0x7f77ac103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.598+0000 7f77aa7fc640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77ac1052d0 0x7f77ac102bf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f77a000e9a0 tx=0x7f77a000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.598+0000 7f7793fff640 1 -- 172.21.15.3:0/1375609218 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a000cd80 con 0x7f77ac1052d0 2024-09-21T06:22:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.598+0000 7f77b148b640 1 -- 172.21.15.3:0/1375609218 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ac1a8910 con 0x7f77ac1052d0 2024-09-21T06:22:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.598+0000 7f7793fff640 1 -- 172.21.15.3:0/1375609218 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77a00035f0 con 0x7f77ac1052d0 2024-09-21T06:22:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.598+0000 7f7793fff640 1 -- 172.21.15.3:0/1375609218 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0010690 con 0x7f77ac1052d0 2024-09-21T06:22:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.598+0000 7f77b148b640 1 -- 172.21.15.3:0/1375609218 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ac1a8eb0 con 0x7f77ac1052d0 2024-09-21T06:22:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77b83fb740 1 Processor -- start 2024-09-21T06:22:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77b83fb740 1 -- start start 2024-09-21T06:22:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77b83fb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac43a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77b83fb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac06c2f0 0x5593ac43aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77b83fb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 0x5593ac43b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77b83fb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5593ac3af6f0 con 0x5593ac0951b0 2024-09-21T06:22:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77b83fb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5593ac39eb20 con 0x5593ac06c2f0 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77b83fb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5593ac3a2930 con 0x5593ac0744d0 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f7793fff640 1 -- 172.21.15.3:0/1375609218 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f77a0003fb0 con 0x7f77ac1052d0 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77ab7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac43a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77ab7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac43a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47776/0 (socket says 172.21.15.3:47776) 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77ab7fe640 1 -- 172.21.15.3:0/510708450 learned_addr learned my addr 172.21.15.3:0/510708450 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77aaffd640 1 --2- 172.21.15.3:0/510708450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac06c2f0 0x5593ac43aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77aa7fc640 1 --2- 172.21.15.3:0/510708450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 0x5593ac43b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77ab7fe640 1 -- 172.21.15.3:0/510708450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 msgr2=0x5593ac43b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77ab7fe640 1 --2- 172.21.15.3:0/510708450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 0x5593ac43b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77ab7fe640 1 -- 172.21.15.3:0/510708450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac06c2f0 msgr2=0x5593ac43aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77ab7fe640 1 --2- 172.21.15.3:0/510708450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac06c2f0 0x5593ac43aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77ab7fe640 1 -- 172.21.15.3:0/510708450 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593ac364530 con 0x5593ac0951b0 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.601+0000 7f77aaffd640 1 --2- 172.21.15.3:0/510708450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac06c2f0 0x5593ac43aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.602+0000 7f77ab7fe640 1 --2- 172.21.15.3:0/510708450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac43a9b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f77a4033040 tx=0x7f77a402f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dde89a0e2caf87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.602+0000 7f77917fa640 1 -- 172.21.15.3:0/510708450 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4031650 con 0x5593ac0951b0 2024-09-21T06:22:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.602+0000 7f77917fa640 1 -- 172.21.15.3:0/510708450 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77a4036040 con 0x5593ac0951b0 2024-09-21T06:22:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.602+0000 7f77917fa640 1 -- 172.21.15.3:0/510708450 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4004520 con 0x5593ac0951b0 2024-09-21T06:22:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.602+0000 7f7793fff640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f77940b6730 0x7f77940b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.602+0000 7f77aaffd640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f77940b6730 0x7f77940b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.602+0000 7f7793fff640 1 -- 172.21.15.3:0/1375609218 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f77a00e79d0 con 0x7f77ac1052d0 2024-09-21T06:22:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.603+0000 7f77b83fb740 1 -- 172.21.15.3:0/510708450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 msgr2=0x5593ac43a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.603+0000 7f77aaffd640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f77940b6730 0x7f77940b8c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5593ac439630 tx=0x7f779c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.603+0000 7f77b83fb740 1 --2- 172.21.15.3:0/510708450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac43a9b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f77a4033040 tx=0x7f77a402f3c0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.603+0000 7f77b83fb740 1 -- 172.21.15.3:0/510708450 shutdown_connections 2024-09-21T06:22:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.603+0000 7f77b83fb740 1 --2- 172.21.15.3:0/510708450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 0x5593ac43b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.603+0000 7f77b83fb740 1 --2- 172.21.15.3:0/510708450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac06c2f0 0x5593ac43aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.603+0000 7f77b83fb740 1 --2- 172.21.15.3:0/510708450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac43a9b0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.603+0000 7f77b83fb740 1 -- 172.21.15.3:0/510708450 >> 172.21.15.3:0/510708450 conn(0x5593ac089300 msgr2=0x5593ac4388a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.603+0000 7f77b83fb740 1 -- 172.21.15.3:0/510708450 shutdown_connections 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.603+0000 7f77b83fb740 1 -- 172.21.15.3:0/510708450 wait complete. 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.603+0000 7f77b83fb740 1 -- start start 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77b83fb740 1 Processor -- start 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77b83fb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac06c2f0 0x5593ac446070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77b83fb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac0744d0 0x5593ac4465b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77b83fb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac452240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77b83fb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5593ac2d6ab0 con 0x5593ac0951b0 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77b83fb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5593ac344730 con 0x5593ac0744d0 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77b83fb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5593ac448f50 con 0x5593ac06c2f0 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77b83fb740 10 client.0 _ll_register_callbacks cb 0x5593ac3986b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77ab7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac452240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac06c2f0 0x5593ac446070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77ab7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac452240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47782/0 (socket says 172.21.15.3:47782) 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77ab7fe640 1 -- 172.21.15.3:0/2907527069 learned_addr learned my addr 172.21.15.3:0/2907527069 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77aa7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac0744d0 0x5593ac4465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77ab7fe640 1 -- 172.21.15.3:0/2907527069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac06c2f0 msgr2=0x5593ac446070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77ab7fe640 1 --2- 172.21.15.3:0/2907527069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac06c2f0 0x5593ac446070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77ab7fe640 1 -- 172.21.15.3:0/2907527069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac0744d0 msgr2=0x5593ac4465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77ab7fe640 1 --2- 172.21.15.3:0/2907527069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac0744d0 0x5593ac4465b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77ab7fe640 1 -- 172.21.15.3:0/2907527069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593ac100790 con 0x5593ac0951b0 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.604+0000 7f77aa7fc640 1 --2- 172.21.15.3:0/2907527069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac0744d0 0x5593ac4465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f77ab7fe640 1 --2- 172.21.15.3:0/2907527069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac452240 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f77a40094e0 tx=0x7f77a40040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5446048b30920fde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f778eff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f778eff5640 1 -- 172.21.15.3:0/2907527069 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4003970 con 0x5593ac0951b0 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f778eff5640 1 -- 172.21.15.3:0/2907527069 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f77a4003b10 con 0x5593ac0951b0 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f77b83fb740 1 -- 172.21.15.3:0/2907527069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77a4036040 con 0x5593ac0951b0 2024-09-21T06:22:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f778eff5640 1 -- 172.21.15.3:0/2907527069 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f77a405e050 con 0x5593ac0951b0 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f77b83fb740 10 client.17634 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f77b83fb740 1 -- 172.21.15.3:0/2907527069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5593ac179620 con 0x5593ac0951b0 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f778eff5640 1 -- 172.21.15.3:0/2907527069 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77a4036040 con 0x5593ac0951b0 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f778eff5640 1 -- 172.21.15.3:0/2907527069 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f77a404f070 con 0x5593ac0951b0 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f77b83fb740 10 client.17634 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f77b83fb740 10 client.17634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f77b83fb740 20 client.17634 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f77b83fb740 1 --2- 172.21.15.3:0/2907527069 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5593ac053d80 0x5593ac457e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f77b83fb740 4 client.17634 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.605+0000 7f77b83fb740 1 -- 172.21.15.3:0/2907527069 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593ac3b1640 con 0x5593ac053d80 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.606+0000 7f77aaffd640 1 --2- 172.21.15.3:0/2907527069 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5593ac053d80 0x5593ac457e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.606+0000 7f77aaffd640 1 --2- 172.21.15.3:0/2907527069 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5593ac053d80 0x5593ac457e00 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6e538a58a42cf51 server_cookie=47fd370c625e9f8e in_seq=0 out_seq=0 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.606+0000 7f778eff5640 10 client.17634 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f778eff5640 1 -- 172.21.15.3:0/2907527069 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5593ac3b1640 con 0x5593ac053d80 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f778eff5640 10 client.17634 handle_command_reply: tid=0 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f77b83fb740 1 client.17634 shutdown 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f77b83fb740 1 -- 172.21.15.3:0/2907527069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 msgr2=0x5593ac452240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f77b83fb740 1 --2- 172.21.15.3:0/2907527069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac452240 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f77a40094e0 tx=0x7f77a40040e0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f77b83fb740 1 -- 172.21.15.3:0/2907527069 shutdown_connections 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f77b83fb740 1 --2- 172.21.15.3:0/2907527069 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5593ac053d80 0x5593ac457e00 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f77b83fb740 1 --2- 172.21.15.3:0/2907527069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac452240 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f77b83fb740 1 --2- 172.21.15.3:0/2907527069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac0744d0 0x5593ac4465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f77b83fb740 1 --2- 172.21.15.3:0/2907527069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac06c2f0 0x5593ac446070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f77b83fb740 1 -- 172.21.15.3:0/2907527069 >> 172.21.15.3:0/2907527069 conn(0x5593ac089300 msgr2=0x5593ac43b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f778eff5640 0 client.17634 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f77b83fb740 1 -- 172.21.15.3:0/2907527069 shutdown_connections 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f77b83fb740 1 -- 172.21.15.3:0/2907527069 wait complete. 2024-09-21T06:22:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.607+0000 7f77b83fb740 20 client.17634 trim_cache size 0 max 16384 2024-09-21T06:22:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77b83fb740 1 Processor -- start 2024-09-21T06:22:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77b83fb740 1 -- start start 2024-09-21T06:22:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77b83fb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac06c2f0 0x5593ac43c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77b83fb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 0x5593ac447ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77b83fb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac452140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77b83fb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1b1760 con 0x5593ac0951b0 2024-09-21T06:22:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77b83fb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1b32a0 con 0x5593ac06c2f0 2024-09-21T06:22:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77b83fb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1b1e30 con 0x5593ac0744d0 2024-09-21T06:22:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77ab7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac452140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac06c2f0 0x5593ac43c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77ab7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac452140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47786/0 (socket says 172.21.15.3:47786) 2024-09-21T06:22:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac06c2f0 0x5593ac43c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:35774/0 (socket says 172.21.15.3:35774) 2024-09-21T06:22:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77ab7fe640 1 -- 172.21.15.3:0/3163862650 learned_addr learned my addr 172.21.15.3:0/3163862650 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.623+0000 7f77aa7fc640 1 --2- 172.21.15.3:0/3163862650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 0x5593ac447ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.624+0000 7f77ab7fe640 1 -- 172.21.15.3:0/3163862650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 msgr2=0x5593ac447ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.624+0000 7f77ab7fe640 1 --2- 172.21.15.3:0/3163862650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 0x5593ac447ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.624+0000 7f77ab7fe640 1 -- 172.21.15.3:0/3163862650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac06c2f0 msgr2=0x5593ac43c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.624+0000 7f77ab7fe640 1 --2- 172.21.15.3:0/3163862650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac06c2f0 0x5593ac43c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.624+0000 7f77ab7fe640 1 -- 172.21.15.3:0/3163862650 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593ac43ccb0 con 0x5593ac0951b0 2024-09-21T06:22:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.624+0000 7f77aa7fc640 1 --2- 172.21.15.3:0/3163862650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 0x5593ac447ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.624+0000 7f77aaffd640 1 --2- 172.21.15.3:0/3163862650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac06c2f0 0x5593ac43c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.624+0000 7f77ab7fe640 1 --2- 172.21.15.3:0/3163862650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac452140 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f77a4006010 tx=0x7f77a402f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86ad4c7d9c6521c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.624+0000 7f7790ff9640 1 -- 172.21.15.3:0/3163862650 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4004150 con 0x5593ac0951b0 2024-09-21T06:22:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.624+0000 7f7790ff9640 1 -- 172.21.15.3:0/3163862650 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77a4036040 con 0x5593ac0951b0 2024-09-21T06:22:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.624+0000 7f7790ff9640 1 -- 172.21.15.3:0/3163862650 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4003970 con 0x5593ac0951b0 2024-09-21T06:22:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.625+0000 7f77b83fb740 1 -- 172.21.15.3:0/3163862650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 msgr2=0x5593ac452140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.625+0000 7f77b83fb740 1 --2- 172.21.15.3:0/3163862650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac452140 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f77a4006010 tx=0x7f77a402f4e0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.625+0000 7f77b83fb740 1 -- 172.21.15.3:0/3163862650 shutdown_connections 2024-09-21T06:22:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.625+0000 7f77b83fb740 1 --2- 172.21.15.3:0/3163862650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 0x5593ac447ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.625+0000 7f77b83fb740 1 --2- 172.21.15.3:0/3163862650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac0951b0 0x5593ac452140 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.625+0000 7f77b83fb740 1 --2- 172.21.15.3:0/3163862650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac06c2f0 0x5593ac43c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.625+0000 7f77b83fb740 1 -- 172.21.15.3:0/3163862650 >> 172.21.15.3:0/3163862650 conn(0x5593ac089300 msgr2=0x5593ac447590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.625+0000 7f77b83fb740 1 -- 172.21.15.3:0/3163862650 shutdown_connections 2024-09-21T06:22:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.625+0000 7f77b83fb740 1 -- 172.21.15.3:0/3163862650 wait complete. 2024-09-21T06:22:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77b83fb740 1 -- start start 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77b83fb740 1 Processor -- start 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77b83fb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac06c2f0 0x5593ac44d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77b83fb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 0x5593ac44da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77b83fb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac0951b0 0x5593ac4c0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77b83fb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1b2bd0 con 0x5593ac06c2f0 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77b83fb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1b2500 con 0x5593ac0951b0 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77b83fb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1af760 con 0x5593ac0744d0 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac06c2f0 0x5593ac44d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77b83fb740 10 client.0 _ll_register_callbacks cb 0x7f779c00bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac06c2f0 0x5593ac44d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47798/0 (socket says 172.21.15.3:47798) 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77aaffd640 1 -- 172.21.15.3:0/3698875442 learned_addr learned my addr 172.21.15.3:0/3698875442 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77aa7fc640 1 --2- 172.21.15.3:0/3698875442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 0x5593ac44da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.626+0000 7f77ab7fe640 1 --2- 172.21.15.3:0/3698875442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac0951b0 0x5593ac4c0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.627+0000 7f77aaffd640 1 -- 172.21.15.3:0/3698875442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 msgr2=0x5593ac44da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.627+0000 7f77aaffd640 1 --2- 172.21.15.3:0/3698875442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 0x5593ac44da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.627+0000 7f77aaffd640 1 -- 172.21.15.3:0/3698875442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac0951b0 msgr2=0x5593ac4c0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.627+0000 7f77aaffd640 1 --2- 172.21.15.3:0/3698875442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac0951b0 0x5593ac4c0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.627+0000 7f77aaffd640 1 -- 172.21.15.3:0/3698875442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593ac4c0bb0 con 0x5593ac06c2f0 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.627+0000 7f77aaffd640 1 --2- 172.21.15.3:0/3698875442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac06c2f0 0x5593ac44d520 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x5593ac4395c0 tx=0x7f779c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fdafdb3855a24d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.627+0000 7f778eff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.627+0000 7f778eff5640 1 -- 172.21.15.3:0/3698875442 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c035060 con 0x5593ac06c2f0 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.627+0000 7f778eff5640 1 -- 172.21.15.3:0/3698875442 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f779c01e4e0 con 0x5593ac06c2f0 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.627+0000 7f77b83fb740 1 -- 172.21.15.3:0/3698875442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77a4036040 con 0x5593ac06c2f0 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.627+0000 7f778eff5640 1 -- 172.21.15.3:0/3698875442 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f779c01e6e0 con 0x5593ac06c2f0 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.627+0000 7f77b83fb740 10 client.17640 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.627+0000 7f77b83fb740 1 -- 172.21.15.3:0/3698875442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5593ac4c1cf0 con 0x5593ac06c2f0 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.628+0000 7f778eff5640 1 -- 172.21.15.3:0/3698875442 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77a4036040 con 0x5593ac06c2f0 2024-09-21T06:22:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.628+0000 7f778eff5640 1 -- 172.21.15.3:0/3698875442 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f779c030a80 con 0x5593ac06c2f0 2024-09-21T06:22:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.628+0000 7f77b83fb740 10 client.17640 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.628+0000 7f77b83fb740 10 client.17640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.628+0000 7f77b83fb740 20 client.17640 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.628+0000 7f77b83fb740 1 --2- 172.21.15.3:0/3698875442 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5593ac0291c0 0x5593ac4c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.628+0000 7f77b83fb740 4 client.17640 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:22:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.628+0000 7f77b83fb740 1 -- 172.21.15.3:0/3698875442 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593ac3ad0f0 con 0x5593ac0291c0 2024-09-21T06:22:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.629+0000 7f77aa7fc640 1 --2- 172.21.15.3:0/3698875442 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5593ac0291c0 0x5593ac4c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.629+0000 7f77aa7fc640 1 --2- 172.21.15.3:0/3698875442 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5593ac0291c0 0x5593ac4c44c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce38f73d42ff12a7 server_cookie=a8641fcb69390d3e in_seq=0 out_seq=0 2024-09-21T06:22:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.629+0000 7f778eff5640 10 client.17640 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.796+0000 7f778eff5640 1 -- 172.21.15.3:0/3698875442 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5593ac3ad0f0 con 0x5593ac0291c0 2024-09-21T06:22:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.796+0000 7f778eff5640 10 client.17640 handle_command_reply: tid=0 2024-09-21T06:22:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.797+0000 7f77b83fb740 1 client.17640 shutdown 2024-09-21T06:22:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.797+0000 7f77b83fb740 1 -- 172.21.15.3:0/3698875442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac06c2f0 msgr2=0x5593ac44d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.797+0000 7f77b83fb740 1 --2- 172.21.15.3:0/3698875442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac06c2f0 0x5593ac44d520 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x5593ac4395c0 tx=0x7f779c0046c0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.797+0000 7f77b83fb740 1 -- 172.21.15.3:0/3698875442 shutdown_connections 2024-09-21T06:22:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.797+0000 7f77b83fb740 1 --2- 172.21.15.3:0/3698875442 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5593ac0291c0 0x5593ac4c44c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.797+0000 7f77b83fb740 1 --2- 172.21.15.3:0/3698875442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593ac0744d0 0x5593ac44da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.797+0000 7f778eff5640 0 client.17640 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.797+0000 7f77b83fb740 1 --2- 172.21.15.3:0/3698875442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ac0951b0 0x5593ac4c0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.797+0000 7f77b83fb740 1 --2- 172.21.15.3:0/3698875442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593ac06c2f0 0x5593ac44d520 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.797+0000 7f77b83fb740 1 -- 172.21.15.3:0/3698875442 >> 172.21.15.3:0/3698875442 conn(0x5593ac089300 msgr2=0x5593ac43c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.797+0000 7f77b83fb740 1 -- 172.21.15.3:0/3698875442 shutdown_connections 2024-09-21T06:22:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.797+0000 7f77b83fb740 1 -- 172.21.15.3:0/3698875442 wait complete. 2024-09-21T06:22:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.797+0000 7f77b83fb740 20 client.17640 trim_cache size 0 max 16384 2024-09-21T06:22:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:22:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:22:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:22:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:22:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.798+0000 7f77b148b640 1 -- 172.21.15.3:0/1375609218 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f77940b6730 msgr2=0x7f77940b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.798+0000 7f77b148b640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f77940b6730 0x7f77940b8c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5593ac439630 tx=0x7f779c00b390 comp rx=0 tx=0).stop 2024-09-21T06:22:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.798+0000 7f77b148b640 1 -- 172.21.15.3:0/1375609218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77ac1052d0 msgr2=0x7f77ac102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.798+0000 7f77b148b640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77ac1052d0 0x7f77ac102bf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f77a000e9a0 tx=0x7f77a000ee60 comp rx=0 tx=0).stop 2024-09-21T06:22:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.798+0000 7f77b148b640 1 -- 172.21.15.3:0/1375609218 shutdown_connections 2024-09-21T06:22:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.798+0000 7f77b148b640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f77940b6730 0x7f77940b8c40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.798+0000 7f77b148b640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77ac1052d0 0x7f77ac102bf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.798+0000 7f77b148b640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77ac10dd50 0x7f77ac103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.798+0000 7f77b148b640 1 --2- 172.21.15.3:0/1375609218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77ac104980 0x7f77ac1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.798+0000 7f77b148b640 1 -- 172.21.15.3:0/1375609218 >> 172.21.15.3:0/1375609218 conn(0x7f77ac07a6f0 msgr2=0x7f77ac100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.798+0000 7f77b148b640 1 -- 172.21.15.3:0/1375609218 shutdown_connections 2024-09-21T06:22:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:36.798+0000 7f77b148b640 1 -- 172.21.15.3:0/1375609218 wait complete. 2024-09-21T06:22:36.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 807022 11.15 MB/sec execute 249 sec latency 130.865 ms 2024-09-21T06:22:37.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:36 smithi003 ceph-mon[31227]: pgmap v556: 97 pgs: 97 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 386 op/s 2024-09-21T06:22:37.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:36 smithi137 ceph-mon[34464]: pgmap v556: 97 pgs: 97 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 386 op/s 2024-09-21T06:22:37.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:36 smithi204 ceph-mon[34659]: pgmap v556: 97 pgs: 97 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 386 op/s 2024-09-21T06:22:37.661 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:22:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:37.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 809650 11.17 MB/sec execute 250 sec latency 19.319 ms 2024-09-21T06:22:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.528+0000 7f3bf2605640 1 -- 172.21.15.3:0/1939593093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec105270 msgr2=0x7f3bec10d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.528+0000 7f3bf2605640 1 --2- 172.21.15.3:0/1939593093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec105270 0x7f3bec10d8b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009900 tx=0x7f3be002a410 comp rx=0 tx=0).stop 2024-09-21T06:22:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.528+0000 7f3bf2605640 1 -- 172.21.15.3:0/1939593093 shutdown_connections 2024-09-21T06:22:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.528+0000 7f3bf2605640 1 --2- 172.21.15.3:0/1939593093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bec10ddf0 0x7f3bec110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.528+0000 7f3bf2605640 1 --2- 172.21.15.3:0/1939593093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec105270 0x7f3bec10d8b0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.528+0000 7f3bf2605640 1 --2- 172.21.15.3:0/1939593093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bec104840 0x7f3bec104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.528+0000 7f3bf2605640 1 -- 172.21.15.3:0/1939593093 >> 172.21.15.3:0/1939593093 conn(0x7f3bec079730 msgr2=0x7f3bec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.528+0000 7f3bf2605640 1 -- 172.21.15.3:0/1939593093 shutdown_connections 2024-09-21T06:22:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.528+0000 7f3bf2605640 1 -- 172.21.15.3:0/1939593093 wait complete. 2024-09-21T06:22:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.529+0000 7f3bf2605640 1 Processor -- start 2024-09-21T06:22:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.529+0000 7f3bf2605640 1 -- start start 2024-09-21T06:22:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.529+0000 7f3bf2605640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec104840 0x7f3bec19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.529+0000 7f3bf2605640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bec105270 0x7f3bec19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.529+0000 7f3bf2605640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bec10ddf0 0x7f3bec19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.529+0000 7f3bf2605640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec113ac0 con 0x7f3bec104840 2024-09-21T06:22:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.529+0000 7f3bf2605640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec113940 con 0x7f3bec105270 2024-09-21T06:22:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.529+0000 7f3bf2605640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec113c40 con 0x7f3bec10ddf0 2024-09-21T06:22:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.529+0000 7f3bebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec104840 0x7f3bec19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.529+0000 7f3bebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec104840 0x7f3bec19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47806/0 (socket says 172.21.15.3:47806) 2024-09-21T06:22:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.529+0000 7f3bebfff640 1 -- 172.21.15.3:0/3595507803 learned_addr learned my addr 172.21.15.3:0/3595507803 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.529+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bec105270 0x7f3bec19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.529+0000 7f3bf0ba2640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bec10ddf0 0x7f3bec19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.530+0000 7f3bebfff640 1 -- 172.21.15.3:0/3595507803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bec10ddf0 msgr2=0x7f3bec19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.530+0000 7f3bebfff640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bec10ddf0 0x7f3bec19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.530+0000 7f3bebfff640 1 -- 172.21.15.3:0/3595507803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bec105270 msgr2=0x7f3bec19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.530+0000 7f3bebfff640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bec105270 0x7f3bec19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.530+0000 7f3bebfff640 1 -- 172.21.15.3:0/3595507803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bec1a03e0 con 0x7f3bec104840 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.530+0000 7f3bf0ba2640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bec10ddf0 0x7f3bec19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.530+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bec105270 0x7f3bec19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.530+0000 7f3bebfff640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec104840 0x7f3bec19a470 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00e9c0 tx=0x7f3bdc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.530+0000 7f3be97fa640 1 -- 172.21.15.3:0/3595507803 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc00cd80 con 0x7f3bec104840 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.530+0000 7f3be97fa640 1 -- 172.21.15.3:0/3595507803 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bdc0035f0 con 0x7f3bec104840 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.530+0000 7f3bf2605640 1 -- 172.21.15.3:0/3595507803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bec1a06d0 con 0x7f3bec104840 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.530+0000 7f3be97fa640 1 -- 172.21.15.3:0/3595507803 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc010690 con 0x7f3bec104840 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.530+0000 7f3bf2605640 1 -- 172.21.15.3:0/3595507803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bec1a6e30 con 0x7f3bec104840 2024-09-21T06:22:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.533+0000 7f3bf8b79740 1 Processor -- start 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.533+0000 7f3bf8b79740 1 -- start start 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.533+0000 7f3be97fa640 1 -- 172.21.15.3:0/3595507803 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f3bdc004000 con 0x7f3bec104840 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.533+0000 7f3bf8b79740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b831070 0x55de2bbd8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.533+0000 7f3bf8b79740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b8081b0 0x55de2bbd8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.533+0000 7f3bf8b79740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b810390 0x55de2bbd90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.533+0000 7f3bf8b79740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55de2bb4ee50 con 0x55de2b831070 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.533+0000 7f3bf8b79740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55de2bb3e280 con 0x55de2b8081b0 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.533+0000 7f3bf8b79740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55de2bb42090 con 0x55de2b810390 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.534+0000 7f3bebfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b8081b0 0x55de2bbd8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.534+0000 7f3bf0ba2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b831070 0x55de2bbd8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.534+0000 7f3bebfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b8081b0 0x55de2bbd8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:35824/0 (socket says 172.21.15.3:35824) 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.534+0000 7f3bebfff640 1 -- 172.21.15.3:0/3344575459 learned_addr learned my addr 172.21.15.3:0/3344575459 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.534+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/3344575459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b810390 0x55de2bbd90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.534+0000 7f3beb7fe640 1 -- 172.21.15.3:0/3344575459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b8081b0 msgr2=0x55de2bbd8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:38.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.534+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/3344575459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b8081b0 0x55de2bbd8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.534+0000 7f3beb7fe640 1 -- 172.21.15.3:0/3344575459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b831070 msgr2=0x55de2bbd8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:38.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.534+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/3344575459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b831070 0x55de2bbd8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.534+0000 7f3beb7fe640 1 -- 172.21.15.3:0/3344575459 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de2b89c650 con 0x55de2b810390 2024-09-21T06:22:38.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.534+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/3344575459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b810390 0x55de2bbd90d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009230 tx=0x7f3be0004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2539acdcc62176b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:38.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.534+0000 7f3be97fa640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3bd40b6680 0x7f3bd40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.534+0000 7f3bd27fc640 1 -- 172.21.15.3:0/3344575459 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0035040 con 0x55de2b810390 2024-09-21T06:22:38.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.535+0000 7f3bf0ba2640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3bd40b6680 0x7f3bd40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.535+0000 7f3bd27fc640 1 -- 172.21.15.3:0/3344575459 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3be0004770 con 0x55de2b810390 2024-09-21T06:22:38.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.535+0000 7f3bd27fc640 1 -- 172.21.15.3:0/3344575459 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0003970 con 0x55de2b810390 2024-09-21T06:22:38.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.535+0000 7f3be97fa640 1 -- 172.21.15.3:0/3595507803 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f3bdc0e7c40 con 0x7f3bec104840 2024-09-21T06:22:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.535+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3344575459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b810390 msgr2=0x55de2bbd90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.535+0000 7f3bf0ba2640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3bd40b6680 0x7f3bd40b8b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009fd0 tx=0x7f3be4009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.535+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3344575459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b810390 0x55de2bbd90d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009230 tx=0x7f3be0004470 comp rx=0 tx=0).stop 2024-09-21T06:22:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.536+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3344575459 shutdown_connections 2024-09-21T06:22:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.536+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3344575459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b810390 0x55de2bbd90d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.536+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3344575459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b8081b0 0x55de2bbd8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.536+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3344575459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b831070 0x55de2bbd8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.536+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3344575459 >> 172.21.15.3:0/3344575459 conn(0x55de2b825460 msgr2=0x55de2bbd6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.536+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3344575459 shutdown_connections 2024-09-21T06:22:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.536+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3344575459 wait complete. 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.536+0000 7f3bf8b79740 1 -- start start 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.536+0000 7f3bf8b79740 1 Processor -- start 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bf8b79740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b8081b0 0x55de2bbe3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bf8b79740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b810390 0x55de2bbe4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bf8b79740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b831070 0x55de2bbf00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bf8b79740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55de2b99f700 con 0x55de2b8081b0 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bf8b79740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55de2bad9110 con 0x55de2b810390 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bf8b79740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55de2bbe6dd0 con 0x55de2b831070 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bf8b79740 10 client.0 _ll_register_callbacks cb 0x55de2baf2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b8081b0 0x55de2bbe3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b8081b0 0x55de2bbe3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47832/0 (socket says 172.21.15.3:47832) 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bebfff640 1 -- 172.21.15.3:0/2606209758 learned_addr learned my addr 172.21.15.3:0/2606209758 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b810390 0x55de2bbe4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bf0ba2640 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b831070 0x55de2bbf00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bebfff640 1 -- 172.21.15.3:0/2606209758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b831070 msgr2=0x55de2bbf00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:38.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bebfff640 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b831070 0x55de2bbf00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bebfff640 1 -- 172.21.15.3:0/2606209758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b810390 msgr2=0x55de2bbe4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:38.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bebfff640 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b810390 0x55de2bbe4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3bebfff640 1 -- 172.21.15.3:0/2606209758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de2bb06c40 con 0x55de2b8081b0 2024-09-21T06:22:38.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.537+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b810390 0x55de2bbe4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:38.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.538+0000 7f3bebfff640 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b8081b0 0x55de2bbe3d40 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0058c0 tx=0x7f3bdc0eb380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd66cd26dab511be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:38.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.538+0000 7f3bcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:38.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.538+0000 7f3bcb7fe640 1 -- 172.21.15.3:0/2606209758 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc0a9940 con 0x55de2b8081b0 2024-09-21T06:22:38.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.538+0000 7f3bcb7fe640 1 -- 172.21.15.3:0/2606209758 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f3bdc0042f0 con 0x55de2b8081b0 2024-09-21T06:22:38.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.538+0000 7f3bf8b79740 1 -- 172.21.15.3:0/2606209758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be0004770 con 0x55de2b8081b0 2024-09-21T06:22:38.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.538+0000 7f3bcb7fe640 1 -- 172.21.15.3:0/2606209758 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3bdc0ad210 con 0x55de2b8081b0 2024-09-21T06:22:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.538+0000 7f3bf8b79740 10 client.17652 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.538+0000 7f3bf8b79740 1 -- 172.21.15.3:0/2606209758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55de2bbf2ae0 con 0x55de2b8081b0 2024-09-21T06:22:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.538+0000 7f3bcb7fe640 1 -- 172.21.15.3:0/2606209758 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3be0004770 con 0x55de2b8081b0 2024-09-21T06:22:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.538+0000 7f3bcb7fe640 1 -- 172.21.15.3:0/2606209758 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f3bdc0a93f0 con 0x55de2b8081b0 2024-09-21T06:22:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.538+0000 7f3bf8b79740 10 client.17652 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.539+0000 7f3bf8b79740 10 client.17652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.539+0000 7f3bf8b79740 20 client.17652 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.539+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de2b7ef9f0 0x55de2bbf5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.539+0000 7f3bf8b79740 4 client.17652 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.539+0000 7f3bf8b79740 1 -- 172.21.15.3:0/2606209758 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de2bb50da0 con 0x55de2b7ef9f0 2024-09-21T06:22:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.539+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de2b7ef9f0 0x55de2bbf5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.539+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de2b7ef9f0 0x55de2bbf5ff0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f03dc6740e6c8218 server_cookie=8ecf9b4504798efa in_seq=0 out_seq=0 2024-09-21T06:22:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.539+0000 7f3bcb7fe640 10 client.17652 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.540+0000 7f3bcb7fe640 1 -- 172.21.15.3:0/2606209758 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de2bb50da0 con 0x55de2b7ef9f0 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.540+0000 7f3bcb7fe640 10 client.17652 handle_command_reply: tid=0 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.541+0000 7f3bf8b79740 1 client.17652 shutdown 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.541+0000 7f3bf8b79740 1 -- 172.21.15.3:0/2606209758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b8081b0 msgr2=0x55de2bbe3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.541+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b8081b0 0x55de2bbe3d40 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0058c0 tx=0x7f3bdc0eb380 comp rx=0 tx=0).stop 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.541+0000 7f3bf8b79740 1 -- 172.21.15.3:0/2606209758 shutdown_connections 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.541+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de2b7ef9f0 0x55de2bbf5ff0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.541+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b831070 0x55de2bbf00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.541+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b810390 0x55de2bbe4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.541+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/2606209758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b8081b0 0x55de2bbe3d40 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.541+0000 7f3bf8b79740 1 -- 172.21.15.3:0/2606209758 >> 172.21.15.3:0/2606209758 conn(0x55de2b825460 msgr2=0x55de2bbdb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.541+0000 7f3bcb7fe640 0 client.17652 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.541+0000 7f3bf8b79740 1 -- 172.21.15.3:0/2606209758 shutdown_connections 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.541+0000 7f3bf8b79740 1 -- 172.21.15.3:0/2606209758 wait complete. 2024-09-21T06:22:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.541+0000 7f3bf8b79740 20 client.17652 trim_cache size 0 max 16384 2024-09-21T06:22:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.560+0000 7f3bf8b79740 1 Processor -- start 2024-09-21T06:22:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.560+0000 7f3bf8b79740 1 -- start start 2024-09-21T06:22:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.560+0000 7f3bf8b79740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b8081b0 0x55de2bbe4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.560+0000 7f3bf8b79740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b810390 0x55de2bbec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.560+0000 7f3bf8b79740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b831070 0x55de2bbec970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.560+0000 7f3bf8b79740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1af690 con 0x55de2b810390 2024-09-21T06:22:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3bf8b79740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1b11d0 con 0x55de2b8081b0 2024-09-21T06:22:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3bf8b79740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1afd60 con 0x55de2b831070 2024-09-21T06:22:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3beb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b810390 0x55de2bbec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3bebfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b8081b0 0x55de2bbe4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3beb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b810390 0x55de2bbec430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47842/0 (socket says 172.21.15.3:47842) 2024-09-21T06:22:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3beb7fe640 1 -- 172.21.15.3:0/3462156651 learned_addr learned my addr 172.21.15.3:0/3462156651 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3bf0ba2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b831070 0x55de2bbec970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3beb7fe640 1 -- 172.21.15.3:0/3462156651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b831070 msgr2=0x55de2bbec970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/3462156651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b831070 0x55de2bbec970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3beb7fe640 1 -- 172.21.15.3:0/3462156651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b8081b0 msgr2=0x55de2bbe4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/3462156651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b8081b0 0x55de2bbe4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3beb7fe640 1 -- 172.21.15.3:0/3462156651 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de2bbda940 con 0x55de2b810390 2024-09-21T06:22:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/3462156651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b810390 0x55de2bbec430 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009900 tx=0x7f3be00053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b9e6d5b6dfc346 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:38.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.561+0000 7f3bd1ffb640 1 -- 172.21.15.3:0/3462156651 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0035040 con 0x55de2b810390 2024-09-21T06:22:38.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.562+0000 7f3bd1ffb640 1 -- 172.21.15.3:0/3462156651 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3be0004c20 con 0x55de2b810390 2024-09-21T06:22:38.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.562+0000 7f3bd1ffb640 1 -- 172.21.15.3:0/3462156651 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be00039c0 con 0x55de2b810390 2024-09-21T06:22:38.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.563+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3462156651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b810390 msgr2=0x55de2bbec430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:38.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.563+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3462156651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b810390 0x55de2bbec430 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009900 tx=0x7f3be00053d0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.563+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3462156651 shutdown_connections 2024-09-21T06:22:38.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.563+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3462156651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b831070 0x55de2bbec970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.563+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3462156651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b810390 0x55de2bbec430 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.564+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3462156651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b8081b0 0x55de2bbe4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.564+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3462156651 >> 172.21.15.3:0/3462156651 conn(0x55de2b825460 msgr2=0x55de2bbe3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:38.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.564+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3462156651 shutdown_connections 2024-09-21T06:22:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.564+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3462156651 wait complete. 2024-09-21T06:22:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.564+0000 7f3bf8b79740 1 -- start start 2024-09-21T06:22:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.565+0000 7f3bf8b79740 1 Processor -- start 2024-09-21T06:22:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.565+0000 7f3bf8b79740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b8081b0 0x55de2bc58970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.565+0000 7f3bf8b79740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b810390 0x55de2bc58ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.565+0000 7f3bebfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b8081b0 0x55de2bc58970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.565+0000 7f3bebfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b8081b0 0x55de2bc58970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:38484/0 (socket says 172.21.15.3:38484) 2024-09-21T06:22:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.565+0000 7f3bf8b79740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b831070 0x55de2bc639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.565+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3176761727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1b0b00 con 0x55de2b831070 2024-09-21T06:22:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.565+0000 7f3bebfff640 1 -- 172.21.15.3:0/3176761727 learned_addr learned my addr 172.21.15.3:0/3176761727 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.565+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3176761727 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1b0430 con 0x55de2b810390 2024-09-21T06:22:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.565+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3176761727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1ad690 con 0x55de2b8081b0 2024-09-21T06:22:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.565+0000 7f3bf8b79740 10 client.0 _ll_register_callbacks cb 0x7f3bdc003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.566+0000 7f3bf0ba2640 1 --2- 172.21.15.3:0/3176761727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b831070 0x55de2bc639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.566+0000 7f3beb7fe640 1 --2- 172.21.15.3:0/3176761727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b810390 0x55de2bc58ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.566+0000 7f3bf0ba2640 1 -- 172.21.15.3:0/3176761727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b8081b0 msgr2=0x55de2bc58970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.566+0000 7f3bf0ba2640 1 --2- 172.21.15.3:0/3176761727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b8081b0 0x55de2bc58970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.566+0000 7f3bf0ba2640 1 -- 172.21.15.3:0/3176761727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b810390 msgr2=0x55de2bc58ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.566+0000 7f3bf0ba2640 1 --2- 172.21.15.3:0/3176761727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b810390 0x55de2bc58ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.566+0000 7f3bf0ba2640 1 -- 172.21.15.3:0/3176761727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de2bc63f30 con 0x55de2b831070 2024-09-21T06:22:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.567+0000 7f3bf0ba2640 1 --2- 172.21.15.3:0/3176761727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b831070 0x55de2bc639f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3be40127c0 tx=0x7f3be4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d1a9d3ac82ed5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.567+0000 7f3bcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.567+0000 7f3bcb7fe640 1 -- 172.21.15.3:0/3176761727 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4004830 con 0x55de2b831070 2024-09-21T06:22:38.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.567+0000 7f3bcb7fe640 1 -- 172.21.15.3:0/3176761727 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f3be4018030 con 0x55de2b831070 2024-09-21T06:22:38.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.567+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3176761727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be0004c20 con 0x55de2b831070 2024-09-21T06:22:38.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.567+0000 7f3bcb7fe640 1 -- 172.21.15.3:0/3176761727 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3be4037050 con 0x55de2b831070 2024-09-21T06:22:38.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.567+0000 7f3bf8b79740 10 client.17658 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:38.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.567+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3176761727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55de2bc65070 con 0x55de2b831070 2024-09-21T06:22:38.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.568+0000 7f3bcb7fe640 1 -- 172.21.15.3:0/3176761727 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3be4005020 con 0x55de2b831070 2024-09-21T06:22:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.569+0000 7f3bcb7fe640 1 -- 172.21.15.3:0/3176761727 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f3be402a450 con 0x55de2b831070 2024-09-21T06:22:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.569+0000 7f3bf8b79740 10 client.17658 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.569+0000 7f3bf8b79740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.569+0000 7f3bf8b79740 20 client.17658 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.569+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3176761727 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de2b7c4da0 0x55de2bc678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.569+0000 7f3bf8b79740 4 client.17658 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:22:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.569+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3176761727 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3be0004c20 con 0x55de2b7c4da0 2024-09-21T06:22:38.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.569+0000 7f3bebfff640 1 --2- 172.21.15.3:0/3176761727 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de2b7c4da0 0x55de2bc678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:38.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.570+0000 7f3bcb7fe640 10 client.17658 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:38.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.570+0000 7f3bebfff640 1 --2- 172.21.15.3:0/3176761727 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de2b7c4da0 0x55de2bc678e0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=240c803251374cba server_cookie=a70b788fa8050084 in_seq=0 out_seq=0 2024-09-21T06:22:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.571+0000 7f3bcb7fe640 1 -- 172.21.15.3:0/3176761727 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x7f3be0004c20 con 0x55de2b7c4da0 2024-09-21T06:22:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.571+0000 7f3bcb7fe640 10 client.17658 handle_command_reply: tid=0 2024-09-21T06:22:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.571+0000 7f3bf8b79740 1 client.17658 shutdown 2024-09-21T06:22:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.571+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3176761727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b831070 msgr2=0x55de2bc639f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.571+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3176761727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b831070 0x55de2bc639f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3be40127c0 tx=0x7f3be4004a40 comp rx=0 tx=0).stop 2024-09-21T06:22:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.572+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3176761727 shutdown_connections 2024-09-21T06:22:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.572+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3176761727 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55de2b7c4da0 0x55de2bc678e0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.572+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3176761727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de2b831070 0x55de2bc639f0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.572+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3176761727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de2b810390 0x55de2bc58ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.572+0000 7f3bcb7fe640 0 client.17658 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.572+0000 7f3bf8b79740 1 --2- 172.21.15.3:0/3176761727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de2b8081b0 0x55de2bc58970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.572+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3176761727 >> 172.21.15.3:0/3176761727 conn(0x55de2b825460 msgr2=0x55de2bbe21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.572+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3176761727 shutdown_connections 2024-09-21T06:22:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.572+0000 7f3bf8b79740 1 -- 172.21.15.3:0/3176761727 wait complete. 2024-09-21T06:22:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.572+0000 7f3bf8b79740 20 client.17658 trim_cache size 0 max 16384 2024-09-21T06:22:38.575 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:22:38.575 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:22:38.575 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:22:38.575 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:22:38.575 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:22:38.575 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:22:38.575 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:38.575 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:38.576 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:38.576 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:38.576 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:38.576 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:38.576 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:22:38.576 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:22:38.576 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:38.576 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:38.576 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:38.576 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:38.576 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:38.576 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:38.576 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: "714b9934-a2d3-4a78-b185-01254776061b": { 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:04.685380+0000", 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:38.577 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:38.578 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58": { 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:21:17.396219+0000", 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:38.579 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:22:38.580 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:38.581 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:38.581 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:38.581 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:38.581 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:38.581 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:38.581 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:22:38.581 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:22:38.581 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:38.581 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:38.581 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:38.581 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:38.581 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:38.581 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:22:38.582 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:22:38.582 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:22:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.574+0000 7f3bf2605640 1 -- 172.21.15.3:0/3595507803 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3bd40b6680 msgr2=0x7f3bd40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.574+0000 7f3bf2605640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3bd40b6680 0x7f3bd40b8b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009fd0 tx=0x7f3be4009500 comp rx=0 tx=0).stop 2024-09-21T06:22:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.574+0000 7f3bf2605640 1 -- 172.21.15.3:0/3595507803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec104840 msgr2=0x7f3bec19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.574+0000 7f3bf2605640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec104840 0x7f3bec19a470 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00e9c0 tx=0x7f3bdc00ee80 comp rx=0 tx=0).stop 2024-09-21T06:22:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.574+0000 7f3bf2605640 1 -- 172.21.15.3:0/3595507803 shutdown_connections 2024-09-21T06:22:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.574+0000 7f3bf2605640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3bd40b6680 0x7f3bd40b8b90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.575+0000 7f3bf2605640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bec10ddf0 0x7f3bec19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.575+0000 7f3bf2605640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bec105270 0x7f3bec19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.575+0000 7f3bf2605640 1 --2- 172.21.15.3:0/3595507803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec104840 0x7f3bec19a470 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.575+0000 7f3bf2605640 1 -- 172.21.15.3:0/3595507803 >> 172.21.15.3:0/3595507803 conn(0x7f3bec079730 msgr2=0x7f3bec1007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.575+0000 7f3bf2605640 1 -- 172.21.15.3:0/3595507803 shutdown_connections 2024-09-21T06:22:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:38.575+0000 7f3bf2605640 1 -- 172.21.15.3:0/3595507803 wait complete. 2024-09-21T06:22:38.965 INFO:tasks.workunit.client.0.smithi003.stdout: 1 811161 11.15 MB/sec execute 251 sec latency 67.934 ms 2024-09-21T06:22:39.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:38 smithi003 ceph-mon[31227]: pgmap v557: 97 pgs: 97 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 261 op/s 2024-09-21T06:22:39.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:38 smithi137 ceph-mon[34464]: pgmap v557: 97 pgs: 97 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 261 op/s 2024-09-21T06:22:39.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:38 smithi204 ceph-mon[34659]: pgmap v557: 97 pgs: 97 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 261 op/s 2024-09-21T06:22:39.439 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:22:39.439 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:22:39.721 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:39.966 INFO:tasks.workunit.client.0.smithi003.stdout: 1 813463 11.16 MB/sec execute 252 sec latency 14.943 ms 2024-09-21T06:22:40.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.234+0000 7fd0086e4640 1 -- 172.21.15.3:0/4134510390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd000103b80 msgr2=0x7fd000103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:40.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.234+0000 7fd0086e4640 1 --2- 172.21.15.3:0/4134510390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd000103b80 0x7fd000103f90 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009900 tx=0x7fcffc02a410 comp rx=0 tx=0).stop 2024-09-21T06:22:40.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.234+0000 7fd0086e4640 1 -- 172.21.15.3:0/4134510390 shutdown_connections 2024-09-21T06:22:40.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.234+0000 7fd0086e4640 1 --2- 172.21.15.3:0/4134510390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0001094c0 0x7fd0001098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:40.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.235+0000 7fd0086e4640 1 --2- 172.21.15.3:0/4134510390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd000109e10 0x7fd000110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:40.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.235+0000 7fd0086e4640 1 --2- 172.21.15.3:0/4134510390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd000103b80 0x7fd000103f90 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:40.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.235+0000 7fd0086e4640 1 -- 172.21.15.3:0/4134510390 >> 172.21.15.3:0/4134510390 conn(0x7fd0000ff610 msgr2=0x7fd000101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:40.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.235+0000 7fd0086e4640 1 -- 172.21.15.3:0/4134510390 shutdown_connections 2024-09-21T06:22:40.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.235+0000 7fd0086e4640 1 -- 172.21.15.3:0/4134510390 wait complete. 2024-09-21T06:22:40.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.235+0000 7fd0086e4640 1 Processor -- start 2024-09-21T06:22:40.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.235+0000 7fd0086e4640 1 -- start start 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd0086e4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd000103b80 0x7fd00019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd0086e4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0001094c0 0x7fd00019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd0086e4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd000109e10 0x7fd00019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd0086e4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd000113ad0 con 0x7fd000103b80 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd0086e4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd000113950 con 0x7fd000109e10 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd0086e4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd000113c50 con 0x7fd0001094c0 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd006480640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd000103b80 0x7fd00019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd006480640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd000103b80 0x7fd00019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47866/0 (socket says 172.21.15.3:47866) 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd006480640 1 -- 172.21.15.3:0/617183834 learned_addr learned my addr 172.21.15.3:0/617183834 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd005c7f640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0001094c0 0x7fd00019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd006c81640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd000109e10 0x7fd00019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd006480640 1 -- 172.21.15.3:0/617183834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0001094c0 msgr2=0x7fd00019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd006480640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0001094c0 0x7fd00019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd006480640 1 -- 172.21.15.3:0/617183834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd000109e10 msgr2=0x7fd00019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:40.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd006480640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd000109e10 0x7fd00019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:40.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd006480640 1 -- 172.21.15.3:0/617183834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0001a0430 con 0x7fd000103b80 2024-09-21T06:22:40.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd005c7f640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0001094c0 0x7fd00019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:40.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.236+0000 7fd006480640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd000103b80 0x7fd00019a4c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fcffc005940 tx=0x7fcffc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:40.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.237+0000 7fcff77fe640 1 -- 172.21.15.3:0/617183834 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc03f030 con 0x7fd000103b80 2024-09-21T06:22:40.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.237+0000 7fcff77fe640 1 -- 172.21.15.3:0/617183834 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcffc0044c0 con 0x7fd000103b80 2024-09-21T06:22:40.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.237+0000 7fcff77fe640 1 -- 172.21.15.3:0/617183834 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc035070 con 0x7fd000103b80 2024-09-21T06:22:40.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.237+0000 7fd0086e4640 1 -- 172.21.15.3:0/617183834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0001a06c0 con 0x7fd000103b80 2024-09-21T06:22:40.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.237+0000 7fd0086e4640 1 -- 172.21.15.3:0/617183834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0001a6e80 con 0x7fd000103b80 2024-09-21T06:22:40.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.238+0000 7fd0086e4640 1 -- 172.21.15.3:0/617183834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd000104e00 con 0x7fd000103b80 2024-09-21T06:22:40.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.240+0000 7fcff77fe640 1 -- 172.21.15.3:0/617183834 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fcffc042030 con 0x7fd000103b80 2024-09-21T06:22:40.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.241+0000 7fcff77fe640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcfec0b6700 0x7fcfec0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:40.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.241+0000 7fcff77fe640 1 -- 172.21.15.3:0/617183834 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7fcffc107cb0 con 0x7fd000103b80 2024-09-21T06:22:40.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.241+0000 7fd005c7f640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcfec0b6700 0x7fcfec0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:40.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.242+0000 7fd005c7f640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcfec0b6700 0x7fcfec0b8c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd000108fa0 tx=0x7fcff0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:40.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.245+0000 7fcff77fe640 1 -- 172.21.15.3:0/617183834 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcffc0d2590 con 0x7fd000103b80 2024-09-21T06:22:40.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.487+0000 7fd0086e4640 1 -- 172.21.15.3:0/617183834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd000103f90 con 0x7fd000103b80 2024-09-21T06:22:40.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.488+0000 7fcff77fe640 1 -- 172.21.15.3:0/617183834 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11965 (secure 0 0 0) 0x7fd000103f90 con 0x7fd000103b80 2024-09-21T06:22:40.490 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:22:40.491 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":35,"btime":"2024-09-21T06:21:58:471023+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:21:58.470998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27019":{"gid":27019,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/2984446797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2984446797},{"type":"v1","addr":"172.21.15.137:6835","nonce":2984446797}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:22:40.491 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 35 2024-09-21T06:22:40.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.491+0000 7fd0086e4640 1 -- 172.21.15.3:0/617183834 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcfec0b6700 msgr2=0x7fcfec0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:40.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.491+0000 7fd0086e4640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcfec0b6700 0x7fcfec0b8c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd000108fa0 tx=0x7fcff0005f90 comp rx=0 tx=0).stop 2024-09-21T06:22:40.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.491+0000 7fd0086e4640 1 -- 172.21.15.3:0/617183834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd000103b80 msgr2=0x7fd00019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:40.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.491+0000 7fd0086e4640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd000103b80 0x7fd00019a4c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fcffc005940 tx=0x7fcffc0040c0 comp rx=0 tx=0).stop 2024-09-21T06:22:40.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.491+0000 7fd0086e4640 1 -- 172.21.15.3:0/617183834 shutdown_connections 2024-09-21T06:22:40.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.491+0000 7fd0086e4640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcfec0b6700 0x7fcfec0b8c10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:40.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.491+0000 7fd0086e4640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0001094c0 0x7fd00019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:40.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.491+0000 7fd0086e4640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd000109e10 0x7fd00019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:40.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.491+0000 7fd0086e4640 1 --2- 172.21.15.3:0/617183834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd000103b80 0x7fd00019a4c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:40.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.491+0000 7fd0086e4640 1 -- 172.21.15.3:0/617183834 >> 172.21.15.3:0/617183834 conn(0x7fd0000ff610 msgr2=0x7fd000101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:40.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.491+0000 7fd0086e4640 1 -- 172.21.15.3:0/617183834 shutdown_connections 2024-09-21T06:22:40.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:40.491+0000 7fd0086e4640 1 -- 172.21.15.3:0/617183834 wait complete. 2024-09-21T06:22:40.966 INFO:tasks.workunit.client.0.smithi003.stdout: 1 815542 11.15 MB/sec execute 253 sec latency 23.720 ms 2024-09-21T06:22:41.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:40 smithi003 ceph-mon[31227]: pgmap v558: 97 pgs: 97 active+clean; 124 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 334 op/s 2024-09-21T06:22:41.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:40 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/617183834' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:22:41.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:40 smithi137 ceph-mon[34464]: pgmap v558: 97 pgs: 97 active+clean; 124 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 334 op/s 2024-09-21T06:22:41.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:40 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/617183834' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:22:41.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:40 smithi204 ceph-mon[34659]: pgmap v558: 97 pgs: 97 active+clean; 124 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 334 op/s 2024-09-21T06:22:41.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:40 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/617183834' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:22:41.223 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:22:41.491 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:41.966 INFO:tasks.workunit.client.0.smithi003.stdout: 1 817213 11.15 MB/sec execute 254 sec latency 44.387 ms 2024-09-21T06:22:41.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.984+0000 7fd161098640 1 -- 172.21.15.3:0/3588756055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd15c105380 msgr2=0x7fd15c10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.984+0000 7fd161098640 1 --2- 172.21.15.3:0/3588756055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd15c105380 0x7fd15c10d8d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd150009900 tx=0x7fd15002a410 comp rx=0 tx=0).stop 2024-09-21T06:22:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.984+0000 7fd161098640 1 -- 172.21.15.3:0/3588756055 shutdown_connections 2024-09-21T06:22:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.984+0000 7fd161098640 1 --2- 172.21.15.3:0/3588756055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd15c10de60 0x7fd15c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.984+0000 7fd161098640 1 --2- 172.21.15.3:0/3588756055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd15c105380 0x7fd15c10d8d0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.984+0000 7fd161098640 1 --2- 172.21.15.3:0/3588756055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd15c104a30 0x7fd15c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.984+0000 7fd161098640 1 -- 172.21.15.3:0/3588756055 >> 172.21.15.3:0/3588756055 conn(0x7fd15c079760 msgr2=0x7fd15c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.984+0000 7fd161098640 1 -- 172.21.15.3:0/3588756055 shutdown_connections 2024-09-21T06:22:42.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.984+0000 7fd161098640 1 -- 172.21.15.3:0/3588756055 wait complete. 2024-09-21T06:22:42.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.985+0000 7fd161098640 1 Processor -- start 2024-09-21T06:22:42.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.985+0000 7fd161098640 1 -- start start 2024-09-21T06:22:42.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.985+0000 7fd161098640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd15c104a30 0x7fd15c102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.985+0000 7fd161098640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd15c105380 0x7fd15c102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.985+0000 7fd161098640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd15c10de60 0x7fd15c1031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.985+0000 7fd161098640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c113aa0 con 0x7fd15c105380 2024-09-21T06:22:42.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd161098640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c113920 con 0x7fd15c104a30 2024-09-21T06:22:42.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd161098640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c113c20 con 0x7fd15c10de60 2024-09-21T06:22:42.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd15a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd15c105380 0x7fd15c102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd15a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd15c105380 0x7fd15c102cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47884/0 (socket says 172.21.15.3:47884) 2024-09-21T06:22:42.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd15ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd15c104a30 0x7fd15c102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd15a59c640 1 -- 172.21.15.3:0/371104101 learned_addr learned my addr 172.21.15.3:0/371104101 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:42.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd15b59e640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd15c10de60 0x7fd15c1031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd15a59c640 1 -- 172.21.15.3:0/371104101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd15c10de60 msgr2=0x7fd15c1031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:42.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd15a59c640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd15c10de60 0x7fd15c1031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd15a59c640 1 -- 172.21.15.3:0/371104101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd15c104a30 msgr2=0x7fd15c102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:42.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd15a59c640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd15c104a30 0x7fd15c102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd15a59c640 1 -- 172.21.15.3:0/371104101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c1a8770 con 0x7fd15c105380 2024-09-21T06:22:42.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd15b59e640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd15c10de60 0x7fd15c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:22:42.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd15ad9d640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd15c104a30 0x7fd15c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:42.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd15a59c640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd15c105380 0x7fd15c102cb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd150006010 tx=0x7fd150004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:42.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.986+0000 7fd143fff640 1 -- 172.21.15.3:0/371104101 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15003f030 con 0x7fd15c105380 2024-09-21T06:22:42.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.987+0000 7fd161098640 1 -- 172.21.15.3:0/371104101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15c1a8a00 con 0x7fd15c105380 2024-09-21T06:22:42.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.987+0000 7fd143fff640 1 -- 172.21.15.3:0/371104101 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd150003aa0 con 0x7fd15c105380 2024-09-21T06:22:42.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.987+0000 7fd143fff640 1 -- 172.21.15.3:0/371104101 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150035070 con 0x7fd15c105380 2024-09-21T06:22:42.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.987+0000 7fd161098640 1 -- 172.21.15.3:0/371104101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd15c1a8f30 con 0x7fd15c105380 2024-09-21T06:22:42.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.989+0000 7fd16760c740 1 Processor -- start 2024-09-21T06:22:42.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.989+0000 7fd16760c740 1 -- start start 2024-09-21T06:22:42.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd16760c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f756a5070 0x556f75a4c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd16760c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f7567c1b0 0x556f75a4c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd16760c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f75684390 0x556f75a4cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd16760c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556f759c2ea0 con 0x556f756a5070 2024-09-21T06:22:42.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd16760c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556f759b22d0 con 0x556f75684390 2024-09-21T06:22:42.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd16760c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556f759b60e0 con 0x556f7567c1b0 2024-09-21T06:22:42.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd143fff640 1 -- 172.21.15.3:0/371104101 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fd150042030 con 0x7fd15c105380 2024-09-21T06:22:42.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd15b59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f756a5070 0x556f75a4c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd15ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f7567c1b0 0x556f75a4c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd15b59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f756a5070 0x556f75a4c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47900/0 (socket says 172.21.15.3:47900) 2024-09-21T06:22:42.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd15ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f7567c1b0 0x556f75a4c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:38520/0 (socket says 172.21.15.3:38520) 2024-09-21T06:22:42.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd15b59e640 1 -- 172.21.15.3:0/3647356942 learned_addr learned my addr 172.21.15.3:0/3647356942 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:42.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd15a59c640 1 --2- 172.21.15.3:0/3647356942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f75684390 0x556f75a4cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd15ad9d640 1 -- 172.21.15.3:0/3647356942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f75684390 msgr2=0x556f75a4cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:42.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd15ad9d640 1 --2- 172.21.15.3:0/3647356942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f75684390 0x556f75a4cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd15ad9d640 1 -- 172.21.15.3:0/3647356942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f756a5070 msgr2=0x556f75a4c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:42.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd15ad9d640 1 --2- 172.21.15.3:0/3647356942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f756a5070 0x556f75a4c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd15ad9d640 1 -- 172.21.15.3:0/3647356942 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f75710650 con 0x556f7567c1b0 2024-09-21T06:22:42.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.990+0000 7fd15a59c640 1 --2- 172.21.15.3:0/3647356942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f75684390 0x556f75a4cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:22:42.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.991+0000 7fd15ad9d640 1 --2- 172.21.15.3:0/3647356942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f7567c1b0 0x556f75a4c800 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd15c06c8f0 tx=0x7fd14c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8502edc7f28867cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:42.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.991+0000 7fd1417fa640 1 -- 172.21.15.3:0/3647356942 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c0040f0 con 0x556f7567c1b0 2024-09-21T06:22:42.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.991+0000 7fd1417fa640 1 -- 172.21.15.3:0/3647356942 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd14c0054d0 con 0x556f7567c1b0 2024-09-21T06:22:42.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.991+0000 7fd1417fa640 1 -- 172.21.15.3:0/3647356942 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c004d00 con 0x556f7567c1b0 2024-09-21T06:22:42.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.991+0000 7fd143fff640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd1440b6680 0x7fd1440b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.991+0000 7fd15b59e640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd1440b6680 0x7fd1440b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.991+0000 7fd143fff640 1 -- 172.21.15.3:0/371104101 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7fd150107c70 con 0x7fd15c105380 2024-09-21T06:22:42.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.992+0000 7fd15b59e640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd1440b6680 0x7fd1440b8b90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd154013570 tx=0x7fd1540098a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:42.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.992+0000 7fd16760c740 1 -- 172.21.15.3:0/3647356942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f7567c1b0 msgr2=0x556f75a4c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:42.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.992+0000 7fd16760c740 1 --2- 172.21.15.3:0/3647356942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f7567c1b0 0x556f75a4c800 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd15c06c8f0 tx=0x7fd14c0118d0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.992+0000 7fd16760c740 1 -- 172.21.15.3:0/3647356942 shutdown_connections 2024-09-21T06:22:42.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.992+0000 7fd16760c740 1 --2- 172.21.15.3:0/3647356942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f7567c1b0 0x556f75a4c800 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.992+0000 7fd16760c740 1 --2- 172.21.15.3:0/3647356942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f75684390 0x556f75a4cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.992+0000 7fd16760c740 1 --2- 172.21.15.3:0/3647356942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f756a5070 0x556f75a4c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.992+0000 7fd16760c740 1 -- 172.21.15.3:0/3647356942 >> 172.21.15.3:0/3647356942 conn(0x556f75699460 msgr2=0x556f75a4a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:42.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.992+0000 7fd16760c740 1 -- 172.21.15.3:0/3647356942 shutdown_connections 2024-09-21T06:22:42.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.992+0000 7fd16760c740 1 -- 172.21.15.3:0/3647356942 wait complete. 2024-09-21T06:22:42.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.992+0000 7fd16760c740 1 -- start start 2024-09-21T06:22:42.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd16760c740 1 Processor -- start 2024-09-21T06:22:42.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd16760c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 0x556f75a57930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd16760c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f75684390 0x556f75a57e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd16760c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f756a5070 0x556f75a583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd16760c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556f75813700 con 0x556f7567c1b0 2024-09-21T06:22:42.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd16760c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556f7594d160 con 0x556f75684390 2024-09-21T06:22:42.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd16760c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556f75a5aaa0 con 0x556f756a5070 2024-09-21T06:22:42.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd16760c740 10 client.0 _ll_register_callbacks cb 0x556f759669e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:42.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd15ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 0x556f75a57930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd15ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 0x556f75a57930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47906/0 (socket says 172.21.15.3:47906) 2024-09-21T06:22:42.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd15ad9d640 1 -- 172.21.15.3:0/66494396 learned_addr learned my addr 172.21.15.3:0/66494396 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:42.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd15b59e640 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f756a5070 0x556f75a583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd15a59c640 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f75684390 0x556f75a57e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd15ad9d640 1 -- 172.21.15.3:0/66494396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f756a5070 msgr2=0x556f75a583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:42.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.993+0000 7fd15ad9d640 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f756a5070 0x556f75a583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.994+0000 7fd15ad9d640 1 -- 172.21.15.3:0/66494396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f75684390 msgr2=0x556f75a57e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:42.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.994+0000 7fd15ad9d640 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f75684390 0x556f75a57e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.994+0000 7fd15ad9d640 1 -- 172.21.15.3:0/66494396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f7597ac90 con 0x556f7567c1b0 2024-09-21T06:22:42.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.994+0000 7fd15b59e640 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f756a5070 0x556f75a583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:42.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.994+0000 7fd15a59c640 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f75684390 0x556f75a57e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:42.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.994+0000 7fd15ad9d640 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 0x556f75a57930 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0049c0 tx=0x7fd14c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19990cd8b7d4732 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:42.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.994+0000 7fd13eff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:42.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.994+0000 7fd13eff5640 1 -- 172.21.15.3:0/66494396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c029020 con 0x556f7567c1b0 2024-09-21T06:22:42.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.994+0000 7fd13eff5640 1 -- 172.21.15.3:0/66494396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7fd14c003ba0 con 0x556f7567c1b0 2024-09-21T06:22:42.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.994+0000 7fd16760c740 1 -- 172.21.15.3:0/66494396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd14c0054d0 con 0x556f7567c1b0 2024-09-21T06:22:42.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.994+0000 7fd13eff5640 1 -- 172.21.15.3:0/66494396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd14c003da0 con 0x556f7567c1b0 2024-09-21T06:22:42.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.994+0000 7fd16760c740 10 client.17676 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:42.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.994+0000 7fd16760c740 1 -- 172.21.15.3:0/66494396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556f75a66750 con 0x556f7567c1b0 2024-09-21T06:22:42.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.995+0000 7fd13eff5640 1 -- 172.21.15.3:0/66494396 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd14c0054d0 con 0x556f7567c1b0 2024-09-21T06:22:42.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.995+0000 7fd13eff5640 1 -- 172.21.15.3:0/66494396 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fd14c025470 con 0x556f7567c1b0 2024-09-21T06:22:42.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.995+0000 7fd16760c740 10 client.17676 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:42.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.995+0000 7fd16760c740 10 client.17676 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:22:42.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.995+0000 7fd16760c740 20 client.17676 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:42.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.995+0000 7fd16760c740 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556f756639f0 0x556f75a69ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.995+0000 7fd16760c740 4 client.17676 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:42.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.995+0000 7fd16760c740 1 -- 172.21.15.3:0/66494396 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f759c4df0 con 0x556f756639f0 2024-09-21T06:22:42.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.995+0000 7fd15a59c640 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556f756639f0 0x556f75a69ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.996+0000 7fd15a59c640 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556f756639f0 0x556f75a69ab0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5f671d73d7e247fd server_cookie=f247ee0f57a089a3 in_seq=0 out_seq=0 2024-09-21T06:22:42.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.996+0000 7fd13eff5640 10 client.17676 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:22:42.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.996+0000 7fd13eff5640 1 -- 172.21.15.3:0/66494396 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f759c4df0 con 0x556f756639f0 2024-09-21T06:22:42.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.996+0000 7fd13eff5640 10 client.17676 handle_command_reply: tid=0 2024-09-21T06:22:42.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.996+0000 7fd16760c740 1 client.17676 shutdown 2024-09-21T06:22:42.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.997+0000 7fd16760c740 1 -- 172.21.15.3:0/66494396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 msgr2=0x556f75a57930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:42.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.997+0000 7fd16760c740 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 0x556f75a57930 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0049c0 tx=0x7fd14c003820 comp rx=0 tx=0).stop 2024-09-21T06:22:42.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.997+0000 7fd16760c740 1 -- 172.21.15.3:0/66494396 shutdown_connections 2024-09-21T06:22:42.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.997+0000 7fd16760c740 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556f756639f0 0x556f75a69ab0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.997+0000 7fd13eff5640 0 client.17676 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:22:42.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.997+0000 7fd16760c740 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f756a5070 0x556f75a583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.997+0000 7fd16760c740 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f75684390 0x556f75a57e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.997+0000 7fd16760c740 1 --2- 172.21.15.3:0/66494396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 0x556f75a57930 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.997+0000 7fd16760c740 1 -- 172.21.15.3:0/66494396 >> 172.21.15.3:0/66494396 conn(0x556f75699460 msgr2=0x556f75a4f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:42.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.997+0000 7fd16760c740 1 -- 172.21.15.3:0/66494396 shutdown_connections 2024-09-21T06:22:42.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.997+0000 7fd16760c740 1 -- 172.21.15.3:0/66494396 wait complete. 2024-09-21T06:22:42.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:41.997+0000 7fd16760c740 20 client.17676 trim_cache size 0 max 16384 2024-09-21T06:22:42.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.010+0000 7fd16760c740 1 Processor -- start 2024-09-21T06:22:42.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.010+0000 7fd16760c740 1 -- start start 2024-09-21T06:22:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.010+0000 7fd16760c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f756639f0 0x556f75a57df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.010+0000 7fd16760c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 0x556f75a49760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.010+0000 7fd16760c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f75684390 0x556f75a49ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.010+0000 7fd16760c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1b1950 con 0x556f7567c1b0 2024-09-21T06:22:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.010+0000 7fd16760c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1b3490 con 0x556f756639f0 2024-09-21T06:22:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.010+0000 7fd16760c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1b2020 con 0x556f75684390 2024-09-21T06:22:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd15a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 0x556f75a49760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd15ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f756639f0 0x556f75a57df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd15a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 0x556f75a49760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47910/0 (socket says 172.21.15.3:47910) 2024-09-21T06:22:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd15ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f756639f0 0x556f75a57df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:35924/0 (socket says 172.21.15.3:35924) 2024-09-21T06:22:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd15a59c640 1 -- 172.21.15.3:0/3727424993 learned_addr learned my addr 172.21.15.3:0/3727424993 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd15b59e640 1 --2- 172.21.15.3:0/3727424993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f75684390 0x556f75a49ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd15ad9d640 1 -- 172.21.15.3:0/3727424993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f75684390 msgr2=0x556f75a49ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd15ad9d640 1 --2- 172.21.15.3:0/3727424993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f75684390 0x556f75a49ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd15ad9d640 1 -- 172.21.15.3:0/3727424993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 msgr2=0x556f75a49760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd15ad9d640 1 --2- 172.21.15.3:0/3727424993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 0x556f75a49760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd15ad9d640 1 -- 172.21.15.3:0/3727424993 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f75a63cc0 con 0x556f756639f0 2024-09-21T06:22:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd15a59c640 1 --2- 172.21.15.3:0/3727424993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 0x556f75a49760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd15ad9d640 1 --2- 172.21.15.3:0/3727424993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f756639f0 0x556f75a57df0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd14c000c00 tx=0x7fd14c005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9d0f483a38a0c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd140ff9640 1 -- 172.21.15.3:0/3727424993 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c029040 con 0x556f756639f0 2024-09-21T06:22:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd140ff9640 1 -- 172.21.15.3:0/3727424993 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd14c004800 con 0x556f756639f0 2024-09-21T06:22:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.011+0000 7fd140ff9640 1 -- 172.21.15.3:0/3727424993 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c025cd0 con 0x556f756639f0 2024-09-21T06:22:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.012+0000 7fd16760c740 1 -- 172.21.15.3:0/3727424993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f756639f0 msgr2=0x556f75a57df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.012+0000 7fd16760c740 1 --2- 172.21.15.3:0/3727424993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f756639f0 0x556f75a57df0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd14c000c00 tx=0x7fd14c005970 comp rx=0 tx=0).stop 2024-09-21T06:22:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.012+0000 7fd16760c740 1 -- 172.21.15.3:0/3727424993 shutdown_connections 2024-09-21T06:22:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.012+0000 7fd16760c740 1 --2- 172.21.15.3:0/3727424993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f75684390 0x556f75a49ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.012+0000 7fd16760c740 1 --2- 172.21.15.3:0/3727424993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f7567c1b0 0x556f75a49760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.012+0000 7fd16760c740 1 --2- 172.21.15.3:0/3727424993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f756639f0 0x556f75a57df0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.012+0000 7fd16760c740 1 -- 172.21.15.3:0/3727424993 >> 172.21.15.3:0/3727424993 conn(0x556f75699460 msgr2=0x556f75a576e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.013+0000 7fd16760c740 1 -- 172.21.15.3:0/3727424993 shutdown_connections 2024-09-21T06:22:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.013+0000 7fd16760c740 1 -- 172.21.15.3:0/3727424993 wait complete. 2024-09-21T06:22:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.013+0000 7fd16760c740 1 -- start start 2024-09-21T06:22:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.013+0000 7fd16760c740 1 Processor -- start 2024-09-21T06:22:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.013+0000 7fd16760c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f756639f0 0x556f75a60120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.013+0000 7fd16760c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f7567c1b0 0x556f75a606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.013+0000 7fd16760c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f75684390 0x556f75a5b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.013+0000 7fd16760c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1b2dc0 con 0x556f756639f0 2024-09-21T06:22:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.013+0000 7fd16760c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1b26f0 con 0x556f7567c1b0 2024-09-21T06:22:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.013+0000 7fd16760c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1af950 con 0x556f75684390 2024-09-21T06:22:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.013+0000 7fd16760c740 10 client.0 _ll_register_callbacks cb 0x7fd154009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.014+0000 7fd15ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f756639f0 0x556f75a60120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.014+0000 7fd15a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f7567c1b0 0x556f75a606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.014+0000 7fd15ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f756639f0 0x556f75a60120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47924/0 (socket says 172.21.15.3:47924) 2024-09-21T06:22:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.014+0000 7fd15ad9d640 1 -- 172.21.15.3:0/3755394623 learned_addr learned my addr 172.21.15.3:0/3755394623 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.014+0000 7fd15b59e640 1 --2- 172.21.15.3:0/3755394623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f75684390 0x556f75a5b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.014+0000 7fd15ad9d640 1 -- 172.21.15.3:0/3755394623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f75684390 msgr2=0x556f75a5b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.014+0000 7fd15ad9d640 1 --2- 172.21.15.3:0/3755394623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f75684390 0x556f75a5b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.014+0000 7fd15ad9d640 1 -- 172.21.15.3:0/3755394623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f7567c1b0 msgr2=0x556f75a606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.014+0000 7fd15ad9d640 1 --2- 172.21.15.3:0/3755394623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f7567c1b0 0x556f75a606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.014+0000 7fd15ad9d640 1 -- 172.21.15.3:0/3755394623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f75a5bec0 con 0x556f756639f0 2024-09-21T06:22:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.014+0000 7fd15ad9d640 1 --2- 172.21.15.3:0/3755394623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f756639f0 0x556f75a60120 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd14c025a80 tx=0x7fd14c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576838b199249810 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.014+0000 7fd13eff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.014+0000 7fd13eff5640 1 -- 172.21.15.3:0/3755394623 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c029040 con 0x556f756639f0 2024-09-21T06:22:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.015+0000 7fd13eff5640 1 -- 172.21.15.3:0/3755394623 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7fd14c003ba0 con 0x556f756639f0 2024-09-21T06:22:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.015+0000 7fd16760c740 1 -- 172.21.15.3:0/3755394623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556f759c08a0 con 0x556f756639f0 2024-09-21T06:22:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.015+0000 7fd13eff5640 1 -- 172.21.15.3:0/3755394623 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd14c04e050 con 0x556f756639f0 2024-09-21T06:22:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.015+0000 7fd16760c740 10 client.17682 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.015+0000 7fd16760c740 1 -- 172.21.15.3:0/3755394623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556f75ad3e90 con 0x556f756639f0 2024-09-21T06:22:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.015+0000 7fd13eff5640 1 -- 172.21.15.3:0/3755394623 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556f759c08a0 con 0x556f756639f0 2024-09-21T06:22:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.015+0000 7fd13eff5640 1 -- 172.21.15.3:0/3755394623 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fd14c038ce0 con 0x556f756639f0 2024-09-21T06:22:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.015+0000 7fd16760c740 10 client.17682 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.015+0000 7fd16760c740 10 client.17682 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:22:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.015+0000 7fd16760c740 20 client.17682 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.015+0000 7fd16760c740 1 --2- 172.21.15.3:0/3755394623 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556f756a5070 0x556f75ad6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.015+0000 7fd16760c740 4 client.17682 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:22:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.015+0000 7fd16760c740 1 -- 172.21.15.3:0/3755394623 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f759bdcd0 con 0x556f756a5070 2024-09-21T06:22:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.016+0000 7fd15a59c640 1 --2- 172.21.15.3:0/3755394623 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556f756a5070 0x556f75ad6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.016+0000 7fd15a59c640 1 --2- 172.21.15.3:0/3755394623 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556f756a5070 0x556f75ad6500 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=28ca7a9be745f23c server_cookie=7b03885c5dce86aa in_seq=0 out_seq=0 2024-09-21T06:22:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.016+0000 7fd13eff5640 10 client.17682 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:22:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.016+0000 7fd13eff5640 1 -- 172.21.15.3:0/3755394623 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f759bdcd0 con 0x556f756a5070 2024-09-21T06:22:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.016+0000 7fd13eff5640 10 client.17682 handle_command_reply: tid=0 2024-09-21T06:22:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.016+0000 7fd16760c740 1 client.17682 shutdown 2024-09-21T06:22:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.017+0000 7fd16760c740 1 -- 172.21.15.3:0/3755394623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f756639f0 msgr2=0x556f75a60120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.017+0000 7fd16760c740 1 --2- 172.21.15.3:0/3755394623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f756639f0 0x556f75a60120 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd14c025a80 tx=0x7fd14c004470 comp rx=0 tx=0).stop 2024-09-21T06:22:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.017+0000 7fd16760c740 1 -- 172.21.15.3:0/3755394623 shutdown_connections 2024-09-21T06:22:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.017+0000 7fd16760c740 1 --2- 172.21.15.3:0/3755394623 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x556f756a5070 0x556f75ad6500 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.017+0000 7fd16760c740 1 --2- 172.21.15.3:0/3755394623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f75684390 0x556f75a5b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.017+0000 7fd16760c740 1 --2- 172.21.15.3:0/3755394623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f7567c1b0 0x556f75a606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.017+0000 7fd16760c740 1 --2- 172.21.15.3:0/3755394623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556f756639f0 0x556f75a60120 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.017+0000 7fd13eff5640 0 client.17682 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:22:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.017+0000 7fd16760c740 1 -- 172.21.15.3:0/3755394623 >> 172.21.15.3:0/3755394623 conn(0x556f75699460 msgr2=0x556f75a4d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.017+0000 7fd16760c740 1 -- 172.21.15.3:0/3755394623 shutdown_connections 2024-09-21T06:22:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.017+0000 7fd16760c740 1 -- 172.21.15.3:0/3755394623 wait complete. 2024-09-21T06:22:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.017+0000 7fd16760c740 20 client.17682 trim_cache size 0 max 16384 2024-09-21T06:22:42.031 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:22:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.018+0000 7fd161098640 1 -- 172.21.15.3:0/371104101 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd1440b6680 msgr2=0x7fd1440b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.018+0000 7fd161098640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd1440b6680 0x7fd1440b8b90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd154013570 tx=0x7fd1540098a0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.018+0000 7fd161098640 1 -- 172.21.15.3:0/371104101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd15c105380 msgr2=0x7fd15c102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.018+0000 7fd161098640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd15c105380 0x7fd15c102cb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd150006010 tx=0x7fd150004060 comp rx=0 tx=0).stop 2024-09-21T06:22:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.018+0000 7fd161098640 1 -- 172.21.15.3:0/371104101 shutdown_connections 2024-09-21T06:22:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.018+0000 7fd161098640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd1440b6680 0x7fd1440b8b90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.018+0000 7fd161098640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd15c10de60 0x7fd15c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.018+0000 7fd161098640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd15c105380 0x7fd15c102cb0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.018+0000 7fd161098640 1 --2- 172.21.15.3:0/371104101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd15c104a30 0x7fd15c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.018+0000 7fd161098640 1 -- 172.21.15.3:0/371104101 >> 172.21.15.3:0/371104101 conn(0x7fd15c079760 msgr2=0x7fd15c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.018+0000 7fd161098640 1 -- 172.21.15.3:0/371104101 shutdown_connections 2024-09-21T06:22:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:42.018+0000 7fd161098640 1 -- 172.21.15.3:0/371104101 wait complete. 2024-09-21T06:22:42.784 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:22:42.966 INFO:tasks.workunit.client.0.smithi003.stdout: 1 819759 11.16 MB/sec execute 255 sec latency 26.892 ms 2024-09-21T06:22:43.056 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:43.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:42 smithi003 ceph-mon[31227]: pgmap v559: 97 pgs: 97 active+clean; 128 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-09-21T06:22:43.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:42 smithi137 ceph-mon[34464]: pgmap v559: 97 pgs: 97 active+clean; 128 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-09-21T06:22:43.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:42 smithi204 ceph-mon[34659]: pgmap v559: 97 pgs: 97 active+clean; 128 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-09-21T06:22:43.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.596+0000 7f54ba899640 1 -- 172.21.15.3:0/3600325888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54b4103b80 msgr2=0x7f54b4103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:43.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.596+0000 7f54ba899640 1 --2- 172.21.15.3:0/3600325888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54b4103b80 0x7f54b4103f90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009900 tx=0x7f54a402a410 comp rx=0 tx=0).stop 2024-09-21T06:22:43.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.596+0000 7f54ba899640 1 -- 172.21.15.3:0/3600325888 shutdown_connections 2024-09-21T06:22:43.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.596+0000 7f54ba899640 1 --2- 172.21.15.3:0/3600325888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54b4109e10 0x7f54b4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.596+0000 7f54ba899640 1 --2- 172.21.15.3:0/3600325888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b41094c0 0x7f54b41098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.596+0000 7f54ba899640 1 --2- 172.21.15.3:0/3600325888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54b4103b80 0x7f54b4103f90 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.596+0000 7f54ba899640 1 -- 172.21.15.3:0/3600325888 >> 172.21.15.3:0/3600325888 conn(0x7f54b40ff610 msgr2=0x7f54b4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:43.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.596+0000 7f54ba899640 1 -- 172.21.15.3:0/3600325888 shutdown_connections 2024-09-21T06:22:43.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.596+0000 7f54ba899640 1 -- 172.21.15.3:0/3600325888 wait complete. 2024-09-21T06:22:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.597+0000 7f54ba899640 1 Processor -- start 2024-09-21T06:22:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.597+0000 7f54ba899640 1 -- start start 2024-09-21T06:22:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.597+0000 7f54ba899640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54b4103b80 0x7f54b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.597+0000 7f54ba899640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54b41094c0 0x7f54b4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54ba899640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b4109e10 0x7f54b4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54ba899640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4113ad0 con 0x7f54b41094c0 2024-09-21T06:22:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54ba899640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4113950 con 0x7f54b4109e10 2024-09-21T06:22:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54ba899640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4113c50 con 0x7f54b4103b80 2024-09-21T06:22:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54b3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54b4103b80 0x7f54b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54b8e36640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b4109e10 0x7f54b4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54b3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54b4103b80 0x7f54b4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:38564/0 (socket says 172.21.15.3:38564) 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54b3fff640 1 -- 172.21.15.3:0/735605820 learned_addr learned my addr 172.21.15.3:0/735605820 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54b37fe640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54b41094c0 0x7f54b4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54b3fff640 1 -- 172.21.15.3:0/735605820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b4109e10 msgr2=0x7f54b4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54b3fff640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b4109e10 0x7f54b4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54b3fff640 1 -- 172.21.15.3:0/735605820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54b41094c0 msgr2=0x7f54b4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54b3fff640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54b41094c0 0x7f54b4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54b3fff640 1 -- 172.21.15.3:0/735605820 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b41a87e0 con 0x7f54b4103b80 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54b8e36640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b4109e10 0x7f54b4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.598+0000 7f54b3fff640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54b4103b80 0x7f54b4075710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005940 tx=0x7f54a40040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.599+0000 7f54b17fa640 1 -- 172.21.15.3:0/735605820 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a403f030 con 0x7f54b4103b80 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.599+0000 7f54ba899640 1 -- 172.21.15.3:0/735605820 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b41a8a70 con 0x7f54b4103b80 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.599+0000 7f54b17fa640 1 -- 172.21.15.3:0/735605820 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a40044c0 con 0x7f54b4103b80 2024-09-21T06:22:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.599+0000 7f54b17fa640 1 -- 172.21.15.3:0/735605820 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4035070 con 0x7f54b4103b80 2024-09-21T06:22:43.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.599+0000 7f54ba899640 1 -- 172.21.15.3:0/735605820 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b41a8fa0 con 0x7f54b4103b80 2024-09-21T06:22:43.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.601+0000 7f54c0e0d740 1 Processor -- start 2024-09-21T06:22:43.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.601+0000 7f54c0e0d740 1 -- start start 2024-09-21T06:22:43.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54b17fa640 1 -- 172.21.15.3:0/735605820 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f54a4042030 con 0x7f54b4103b80 2024-09-21T06:22:43.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54c0e0d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbae0070 0x55b0bbe872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54c0e0d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbab71b0 0x55b0bbe87800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54c0e0d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbabf390 0x55b0bbe87d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54c0e0d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b0bbdfdea0 con 0x55b0bbabf390 2024-09-21T06:22:43.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54c0e0d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b0bbded2d0 con 0x55b0bbae0070 2024-09-21T06:22:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54c0e0d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b0bbdf10e0 con 0x55b0bbab71b0 2024-09-21T06:22:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54b3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbab71b0 0x55b0bbe87800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54b8e36640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbae0070 0x55b0bbe872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54b37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbabf390 0x55b0bbe87d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54b8e36640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbae0070 0x55b0bbe872c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:35974/0 (socket says 172.21.15.3:35974) 2024-09-21T06:22:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54b8e36640 1 -- 172.21.15.3:0/2183615355 learned_addr learned my addr 172.21.15.3:0/2183615355 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54b8e36640 1 -- 172.21.15.3:0/2183615355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbab71b0 msgr2=0x55b0bbe87800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54b8e36640 1 --2- 172.21.15.3:0/2183615355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbab71b0 0x55b0bbe87800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54b8e36640 1 -- 172.21.15.3:0/2183615355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbabf390 msgr2=0x55b0bbe87d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54b8e36640 1 --2- 172.21.15.3:0/2183615355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbabf390 0x55b0bbe87d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54b8e36640 1 -- 172.21.15.3:0/2183615355 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0bbb4b650 con 0x55b0bbae0070 2024-09-21T06:22:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.602+0000 7f54b37fe640 1 --2- 172.21.15.3:0/2183615355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbabf390 0x55b0bbe87d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:43.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.603+0000 7f54b8e36640 1 --2- 172.21.15.3:0/2183615355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbae0070 0x55b0bbe872c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f54ac011790 tx=0x7f54ac011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c719425e7a46e788 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:43.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.603+0000 7f549a7fc640 1 -- 172.21.15.3:0/2183615355 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac003c30 con 0x55b0bbae0070 2024-09-21T06:22:43.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.603+0000 7f54b17fa640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f549c0b6700 0x7f549c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.603+0000 7f549a7fc640 1 -- 172.21.15.3:0/2183615355 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54ac00fe50 con 0x55b0bbae0070 2024-09-21T06:22:43.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.603+0000 7f549a7fc640 1 -- 172.21.15.3:0/2183615355 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0193e0 con 0x55b0bbae0070 2024-09-21T06:22:43.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.603+0000 7f54b3fff640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f549c0b6700 0x7f549c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.603+0000 7f54b17fa640 1 -- 172.21.15.3:0/735605820 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f54a4107a10 con 0x7f54b4103b80 2024-09-21T06:22:43.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.604+0000 7f54c0e0d740 1 -- 172.21.15.3:0/2183615355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbae0070 msgr2=0x55b0bbe872c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.604+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/2183615355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbae0070 0x55b0bbe872c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f54ac011790 tx=0x7f54ac011c50 comp rx=0 tx=0).stop 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.604+0000 7f54b3fff640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f549c0b6700 0x7f549c0b8c10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f54a4105220 tx=0x7f54a410b560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.604+0000 7f54c0e0d740 1 -- 172.21.15.3:0/2183615355 shutdown_connections 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.604+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/2183615355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbab71b0 0x55b0bbe87800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.604+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/2183615355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbabf390 0x55b0bbe87d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.604+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/2183615355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbae0070 0x55b0bbe872c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.604+0000 7f54c0e0d740 1 -- 172.21.15.3:0/2183615355 >> 172.21.15.3:0/2183615355 conn(0x55b0bbad4460 msgr2=0x55b0bbe851b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.604+0000 7f54c0e0d740 1 -- 172.21.15.3:0/2183615355 shutdown_connections 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.604+0000 7f54c0e0d740 1 -- 172.21.15.3:0/2183615355 wait complete. 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.604+0000 7f54c0e0d740 1 -- start start 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.605+0000 7f54c0e0d740 1 Processor -- start 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.605+0000 7f54c0e0d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 0x55b0bbe92940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.605+0000 7f54c0e0d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbabf390 0x55b0bbe92e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.605+0000 7f54c0e0d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbae0070 0x55b0bbe9ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.605+0000 7f54c0e0d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b0bbc4e700 con 0x55b0bbab71b0 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.605+0000 7f54c0e0d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b0bbd88160 con 0x55b0bbabf390 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.605+0000 7f54c0e0d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b0bbe95aa0 con 0x55b0bbae0070 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.605+0000 7f54c0e0d740 10 client.0 _ll_register_callbacks cb 0x55b0bbda19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.605+0000 7f54b8e36640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 0x55b0bbe92940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.605+0000 7f54b8e36640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 0x55b0bbe92940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47956/0 (socket says 172.21.15.3:47956) 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.605+0000 7f54b8e36640 1 -- 172.21.15.3:0/3431637852 learned_addr learned my addr 172.21.15.3:0/3431637852 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.605+0000 7f54b37fe640 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbabf390 0x55b0bbe92e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.605+0000 7f54b8e36640 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbae0070 0x55b0bbe9ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54b37fe640 1 -- 172.21.15.3:0/3431637852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbae0070 msgr2=0x55b0bbe9ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54b37fe640 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbae0070 0x55b0bbe9ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54b37fe640 1 -- 172.21.15.3:0/3431637852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 msgr2=0x55b0bbe92940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54b37fe640 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 0x55b0bbe92940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54b37fe640 1 -- 172.21.15.3:0/3431637852 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0bbdb5c90 con 0x55b0bbabf390 2024-09-21T06:22:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54b8e36640 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 0x55b0bbe92940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54b37fe640 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbabf390 0x55b0bbe92e80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f54b406c970 tx=0x7f54a8012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d96ce787943b1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54937fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:22:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54937fe640 1 -- 172.21.15.3:0/3431637852 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8004030 con 0x55b0bbabf390 2024-09-21T06:22:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54937fe640 1 -- 172.21.15.3:0/3431637852 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f54a80041d0 con 0x55b0bbabf390 2024-09-21T06:22:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54c0e0d740 1 -- 172.21.15.3:0/3431637852 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54ac00fe50 con 0x55b0bbabf390 2024-09-21T06:22:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54937fe640 1 -- 172.21.15.3:0/3431637852 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f54a80043d0 con 0x55b0bbabf390 2024-09-21T06:22:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54c0e0d740 10 client.27260 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54c0e0d740 1 -- 172.21.15.3:0/3431637852 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b0bbea1820 con 0x55b0bbabf390 2024-09-21T06:22:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.606+0000 7f54937fe640 1 -- 172.21.15.3:0/3431637852 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54ac00fe50 con 0x55b0bbabf390 2024-09-21T06:22:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.607+0000 7f54937fe640 1 -- 172.21.15.3:0/3431637852 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f54a8003ae0 con 0x55b0bbabf390 2024-09-21T06:22:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.607+0000 7f54c0e0d740 10 client.27260 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.607+0000 7f54c0e0d740 10 client.27260 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:22:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.607+0000 7f54c0e0d740 20 client.27260 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:43.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.607+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55b0bba9e9f0 0x55b0bbea4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.607+0000 7f54c0e0d740 4 client.27260 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:43.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.607+0000 7f54c0e0d740 1 -- 172.21.15.3:0/3431637852 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0bbdffdf0 con 0x55b0bba9e9f0 2024-09-21T06:22:43.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.607+0000 7f54b8e36640 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55b0bba9e9f0 0x55b0bbea4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.607+0000 7f54937fe640 10 client.27260 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:22:43.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.607+0000 7f54b8e36640 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55b0bba9e9f0 0x55b0bbea4d30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2d250306fd4026b server_cookie=11c17907a2dc2688 in_seq=0 out_seq=0 2024-09-21T06:22:43.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.609+0000 7f54937fe640 1 -- 172.21.15.3:0/3431637852 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b0bbdffdf0 con 0x55b0bba9e9f0 2024-09-21T06:22:43.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.609+0000 7f54937fe640 10 client.27260 handle_command_reply: tid=0 2024-09-21T06:22:43.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.609+0000 7f54c0e0d740 1 client.27260 shutdown 2024-09-21T06:22:43.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.610+0000 7f54c0e0d740 1 -- 172.21.15.3:0/3431637852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbabf390 msgr2=0x55b0bbe92e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:43.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.610+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbabf390 0x55b0bbe92e80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f54b406c970 tx=0x7f54a8012b60 comp rx=0 tx=0).stop 2024-09-21T06:22:43.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.610+0000 7f54c0e0d740 1 -- 172.21.15.3:0/3431637852 shutdown_connections 2024-09-21T06:22:43.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.610+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55b0bba9e9f0 0x55b0bbea4d30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.610+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbae0070 0x55b0bbe9ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.610+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbabf390 0x55b0bbe92e80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.610+0000 7f54937fe640 0 client.27260 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:22:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.610+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/3431637852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 0x55b0bbe92940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.610+0000 7f54c0e0d740 1 -- 172.21.15.3:0/3431637852 >> 172.21.15.3:0/3431637852 conn(0x55b0bbad4460 msgr2=0x55b0bbe8a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.610+0000 7f54c0e0d740 1 -- 172.21.15.3:0/3431637852 shutdown_connections 2024-09-21T06:22:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.610+0000 7f54c0e0d740 1 -- 172.21.15.3:0/3431637852 wait complete. 2024-09-21T06:22:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.610+0000 7f54c0e0d740 20 client.27260 trim_cache size 0 max 16384 2024-09-21T06:22:43.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.624+0000 7f54c0e0d740 1 Processor -- start 2024-09-21T06:22:43.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.624+0000 7f54c0e0d740 1 -- start start 2024-09-21T06:22:43.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.624+0000 7f54c0e0d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 0x55b0bbe9ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.624+0000 7f54c0e0d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbabf390 0x55b0bbe926f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.624+0000 7f54c0e0d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbae0070 0x55b0bbe92c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.624+0000 7f54c0e0d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f54b41b18f0 con 0x55b0bbab71b0 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.624+0000 7f54c0e0d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f54b41b3430 con 0x55b0bbae0070 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.624+0000 7f54c0e0d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f54b41b1fc0 con 0x55b0bbabf390 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.624+0000 7f54b8e36640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 0x55b0bbe9ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.624+0000 7f54b8e36640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 0x55b0bbe9ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47972/0 (socket says 172.21.15.3:47972) 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.624+0000 7f54b8e36640 1 -- 172.21.15.3:0/3709663318 learned_addr learned my addr 172.21.15.3:0/3709663318 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.624+0000 7f54b8e36640 1 --2- 172.21.15.3:0/3709663318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbae0070 0x55b0bbe92c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.624+0000 7f54b37fe640 1 --2- 172.21.15.3:0/3709663318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbabf390 0x55b0bbe926f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.625+0000 7f54b8e36640 1 -- 172.21.15.3:0/3709663318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbabf390 msgr2=0x55b0bbe926f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.625+0000 7f54b8e36640 1 --2- 172.21.15.3:0/3709663318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbabf390 0x55b0bbe926f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.625+0000 7f54b8e36640 1 -- 172.21.15.3:0/3709663318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbae0070 msgr2=0x55b0bbe92c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.625+0000 7f54b8e36640 1 --2- 172.21.15.3:0/3709663318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbae0070 0x55b0bbe92c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.625+0000 7f54b8e36640 1 -- 172.21.15.3:0/3709663318 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0bbe85aa0 con 0x55b0bbab71b0 2024-09-21T06:22:43.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.625+0000 7f54b8e36640 1 --2- 172.21.15.3:0/3709663318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 0x55b0bbe9ec90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f54ac005c40 tx=0x7f54ac004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3c6afdc038ad0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.625+0000 7f5499ffb640 1 -- 172.21.15.3:0/3709663318 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac01f040 con 0x55b0bbab71b0 2024-09-21T06:22:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.625+0000 7f5499ffb640 1 -- 172.21.15.3:0/3709663318 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54ac0040d0 con 0x55b0bbab71b0 2024-09-21T06:22:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.625+0000 7f5499ffb640 1 -- 172.21.15.3:0/3709663318 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac01a840 con 0x55b0bbab71b0 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.626+0000 7f54c0e0d740 1 -- 172.21.15.3:0/3709663318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 msgr2=0x55b0bbe9ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.626+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/3709663318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 0x55b0bbe9ec90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f54ac005c40 tx=0x7f54ac004610 comp rx=0 tx=0).stop 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.626+0000 7f54c0e0d740 1 -- 172.21.15.3:0/3709663318 shutdown_connections 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.626+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/3709663318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbabf390 0x55b0bbe926f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.626+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/3709663318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbae0070 0x55b0bbe92c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.626+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/3709663318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbab71b0 0x55b0bbe9ec90 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.626+0000 7f54c0e0d740 1 -- 172.21.15.3:0/3709663318 >> 172.21.15.3:0/3709663318 conn(0x55b0bbad4460 msgr2=0x55b0bbe911c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.626+0000 7f54c0e0d740 1 -- 172.21.15.3:0/3709663318 shutdown_connections 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.626+0000 7f54c0e0d740 1 -- 172.21.15.3:0/3709663318 wait complete. 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.626+0000 7f54c0e0d740 1 -- start start 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54c0e0d740 1 Processor -- start 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54c0e0d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbab71b0 0x55b0bbe96730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54c0e0d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbabf390 0x55b0bbe96d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54c0e0d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbae0070 0x55b0bbe9ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54c0e0d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f54b41b2d60 con 0x55b0bbabf390 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54c0e0d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f54b41b2690 con 0x55b0bbab71b0 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54c0e0d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f54b41af8f0 con 0x55b0bbae0070 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54c0e0d740 10 client.0 _ll_register_callbacks cb 0x7f54a8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54b37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbabf390 0x55b0bbe96d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54b8e36640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbab71b0 0x55b0bbe96730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54b37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbabf390 0x55b0bbe96d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47978/0 (socket says 172.21.15.3:47978) 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54b37fe640 1 -- 172.21.15.3:0/737121082 learned_addr learned my addr 172.21.15.3:0/737121082 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54b8e36640 1 --2- 172.21.15.3:0/737121082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbae0070 0x55b0bbe9ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54b8e36640 1 -- 172.21.15.3:0/737121082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbae0070 msgr2=0x55b0bbe9ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54b8e36640 1 --2- 172.21.15.3:0/737121082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbae0070 0x55b0bbe9ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54b8e36640 1 -- 172.21.15.3:0/737121082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbabf390 msgr2=0x55b0bbe96d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54b8e36640 1 --2- 172.21.15.3:0/737121082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbabf390 0x55b0bbe96d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.627+0000 7f54b8e36640 1 -- 172.21.15.3:0/737121082 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0bbe9b270 con 0x55b0bbab71b0 2024-09-21T06:22:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.628+0000 7f54b37fe640 1 --2- 172.21.15.3:0/737121082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbabf390 0x55b0bbe96d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.628+0000 7f54b8e36640 1 --2- 172.21.15.3:0/737121082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbab71b0 0x55b0bbe96730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f54ac013040 tx=0x7f54ac02f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d18c88a17cb4439 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.628+0000 7f54937fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.628+0000 7f54937fe640 1 -- 172.21.15.3:0/737121082 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac004340 con 0x55b0bbab71b0 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.628+0000 7f54937fe640 1 -- 172.21.15.3:0/737121082 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f54ac01f070 con 0x55b0bbab71b0 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.628+0000 7f54c0e0d740 1 -- 172.21.15.3:0/737121082 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54ac0040d0 con 0x55b0bbab71b0 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.628+0000 7f54937fe640 1 -- 172.21.15.3:0/737121082 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f54ac003a30 con 0x55b0bbab71b0 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.628+0000 7f54c0e0d740 10 client.27266 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.628+0000 7f54c0e0d740 1 -- 172.21.15.3:0/737121082 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b0bbf0f1d0 con 0x55b0bbab71b0 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.628+0000 7f54937fe640 1 -- 172.21.15.3:0/737121082 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54ac0040d0 con 0x55b0bbab71b0 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.629+0000 7f54937fe640 1 -- 172.21.15.3:0/737121082 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f54ac022050 con 0x55b0bbab71b0 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.629+0000 7f54c0e0d740 10 client.27266 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.629+0000 7f54c0e0d740 10 client.27266 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.629+0000 7f54c0e0d740 20 client.27266 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.629+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/737121082 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55b0bba73da0 0x55b0bbf11690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.629+0000 7f54c0e0d740 4 client.27266 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:22:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.629+0000 7f54c0e0d740 1 -- 172.21.15.3:0/737121082 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0bbdfb8a0 con 0x55b0bba73da0 2024-09-21T06:22:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.629+0000 7f54b37fe640 1 --2- 172.21.15.3:0/737121082 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55b0bba73da0 0x55b0bbf11690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.629+0000 7f54937fe640 10 client.27266 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:22:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.629+0000 7f54b37fe640 1 --2- 172.21.15.3:0/737121082 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55b0bba73da0 0x55b0bbf11690 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4e73178c1252c989 server_cookie=a5c1dbdc1d8dd8cf in_seq=0 out_seq=0 2024-09-21T06:22:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.629+0000 7f54937fe640 1 -- 172.21.15.3:0/737121082 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0bbdfb8a0 con 0x55b0bba73da0 2024-09-21T06:22:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.629+0000 7f54937fe640 10 client.27266 handle_command_reply: tid=0 2024-09-21T06:22:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.629+0000 7f54c0e0d740 1 client.27266 shutdown 2024-09-21T06:22:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54c0e0d740 1 -- 172.21.15.3:0/737121082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbab71b0 msgr2=0x55b0bbe96730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:43.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/737121082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbab71b0 0x55b0bbe96730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f54ac013040 tx=0x7f54ac02f040 comp rx=0 tx=0).stop 2024-09-21T06:22:43.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54c0e0d740 1 -- 172.21.15.3:0/737121082 shutdown_connections 2024-09-21T06:22:43.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/737121082 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55b0bba73da0 0x55b0bbf11690 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/737121082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0bbae0070 0x55b0bbe9ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/737121082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b0bbabf390 0x55b0bbe96d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54c0e0d740 1 --2- 172.21.15.3:0/737121082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0bbab71b0 0x55b0bbe96730 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54937fe640 0 client.27266 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:22:43.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54c0e0d740 1 -- 172.21.15.3:0/737121082 >> 172.21.15.3:0/737121082 conn(0x55b0bbad4460 msgr2=0x55b0bbe882f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:43.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54c0e0d740 1 -- 172.21.15.3:0/737121082 shutdown_connections 2024-09-21T06:22:43.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54c0e0d740 1 -- 172.21.15.3:0/737121082 wait complete. 2024-09-21T06:22:43.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54c0e0d740 20 client.27266 trim_cache size 0 max 16384 2024-09-21T06:22:43.636 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:22:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54ba899640 1 -- 172.21.15.3:0/735605820 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f549c0b6700 msgr2=0x7f549c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54ba899640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f549c0b6700 0x7f549c0b8c10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f54a4105220 tx=0x7f54a410b560 comp rx=0 tx=0).stop 2024-09-21T06:22:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54ba899640 1 -- 172.21.15.3:0/735605820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54b4103b80 msgr2=0x7f54b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.630+0000 7f54ba899640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54b4103b80 0x7f54b4075710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005940 tx=0x7f54a40040c0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.631+0000 7f54ba899640 1 -- 172.21.15.3:0/735605820 shutdown_connections 2024-09-21T06:22:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.631+0000 7f54ba899640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f549c0b6700 0x7f549c0b8c10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.631+0000 7f54ba899640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b4109e10 0x7f54b4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.631+0000 7f54ba899640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54b41094c0 0x7f54b4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.631+0000 7f54ba899640 1 --2- 172.21.15.3:0/735605820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54b4103b80 0x7f54b4075710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.631+0000 7f54ba899640 1 -- 172.21.15.3:0/735605820 >> 172.21.15.3:0/735605820 conn(0x7f54b40ff610 msgr2=0x7f54b4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.631+0000 7f54ba899640 1 -- 172.21.15.3:0/735605820 shutdown_connections 2024-09-21T06:22:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:43.631+0000 7f54ba899640 1 -- 172.21.15.3:0/735605820 wait complete. 2024-09-21T06:22:43.966 INFO:tasks.workunit.client.0.smithi003.stdout: 1 821048 11.14 MB/sec execute 256 sec latency 40.613 ms 2024-09-21T06:22:44.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:22:44.779 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:44.966 INFO:tasks.workunit.client.0.smithi003.stdout: 1 823647 11.16 MB/sec execute 257 sec latency 17.758 ms 2024-09-21T06:22:45.099 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:44 smithi003 ceph-mon[31227]: pgmap v560: 97 pgs: 97 active+clean; 128 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 308 op/s 2024-09-21T06:22:45.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:44 smithi137 ceph-mon[34464]: pgmap v560: 97 pgs: 97 active+clean; 128 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 308 op/s 2024-09-21T06:22:45.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:44 smithi204 ceph-mon[34659]: pgmap v560: 97 pgs: 97 active+clean; 128 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 308 op/s 2024-09-21T06:22:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.313+0000 7fe8ea83b640 1 -- 172.21.15.3:0/814771488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8e4102a60 msgr2=0x7fe8e4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.313+0000 7fe8ea83b640 1 --2- 172.21.15.3:0/814771488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8e4102a60 0x7fe8e4102e70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009900 tx=0x7fe8d402a410 comp rx=0 tx=0).stop 2024-09-21T06:22:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.313+0000 7fe8ea83b640 1 -- 172.21.15.3:0/814771488 shutdown_connections 2024-09-21T06:22:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.313+0000 7fe8ea83b640 1 --2- 172.21.15.3:0/814771488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e4108cf0 0x7fe8e410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.313+0000 7fe8ea83b640 1 --2- 172.21.15.3:0/814771488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8e41083a0 0x7fe8e41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.313+0000 7fe8ea83b640 1 --2- 172.21.15.3:0/814771488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8e4102a60 0x7fe8e4102e70 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.313+0000 7fe8ea83b640 1 -- 172.21.15.3:0/814771488 >> 172.21.15.3:0/814771488 conn(0x7fe8e4079730 msgr2=0x7fe8e4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.313+0000 7fe8ea83b640 1 -- 172.21.15.3:0/814771488 shutdown_connections 2024-09-21T06:22:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.313+0000 7fe8ea83b640 1 -- 172.21.15.3:0/814771488 wait complete. 2024-09-21T06:22:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.314+0000 7fe8ea83b640 1 Processor -- start 2024-09-21T06:22:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.314+0000 7fe8ea83b640 1 -- start start 2024-09-21T06:22:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8ea83b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8e4102a60 0x7fe8e41993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8ea83b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e41083a0 0x7fe8e4199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8ea83b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8e4108cf0 0x7fe8e419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8ea83b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e41129b0 con 0x7fe8e4102a60 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8ea83b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e4112830 con 0x7fe8e4108cf0 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8ea83b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e4112b30 con 0x7fe8e41083a0 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8e3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8e4102a60 0x7fe8e41993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8e3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8e4102a60 0x7fe8e41993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51792/0 (socket says 172.21.15.3:51792) 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8e3fff640 1 -- 172.21.15.3:0/2468561288 learned_addr learned my addr 172.21.15.3:0/2468561288 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8e4108cf0 0x7fe8e419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8e37fe640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e41083a0 0x7fe8e4199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8e3fff640 1 -- 172.21.15.3:0/2468561288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e41083a0 msgr2=0x7fe8e4199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8e3fff640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e41083a0 0x7fe8e4199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8e3fff640 1 -- 172.21.15.3:0/2468561288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8e4108cf0 msgr2=0x7fe8e419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8e3fff640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8e4108cf0 0x7fe8e419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.315+0000 7fe8e3fff640 1 -- 172.21.15.3:0/2468561288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e419f330 con 0x7fe8e4102a60 2024-09-21T06:22:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.316+0000 7fe8e3fff640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8e4102a60 0x7fe8e41993c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4005ab0 tx=0x7fe8d4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.316+0000 7fe8e17fa640 1 -- 172.21.15.3:0/2468561288 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d403f070 con 0x7fe8e4102a60 2024-09-21T06:22:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.316+0000 7fe8ea83b640 1 -- 172.21.15.3:0/2468561288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e419f5c0 con 0x7fe8e4102a60 2024-09-21T06:22:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.316+0000 7fe8e17fa640 1 -- 172.21.15.3:0/2468561288 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8d4031d60 con 0x7fe8e4102a60 2024-09-21T06:22:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.316+0000 7fe8e17fa640 1 -- 172.21.15.3:0/2468561288 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4035030 con 0x7fe8e4102a60 2024-09-21T06:22:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.316+0000 7fe8ea83b640 1 -- 172.21.15.3:0/2468561288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e41a5d80 con 0x7fe8e4102a60 2024-09-21T06:22:45.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.318+0000 7fe8f0daf740 1 Processor -- start 2024-09-21T06:22:45.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.319+0000 7fe8f0daf740 1 -- start start 2024-09-21T06:22:45.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.319+0000 7fe8e17fa640 1 -- 172.21.15.3:0/2468561288 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fe8d4042070 con 0x7fe8e4102a60 2024-09-21T06:22:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.319+0000 7fe8f0daf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142d05070 0x55b1430ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.319+0000 7fe8f0daf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142cdc1b0 0x55b1430acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.319+0000 7fe8f0daf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.319+0000 7fe8f0daf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b143022ea0 con 0x55b142cdc1b0 2024-09-21T06:22:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.319+0000 7fe8f0daf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b1430122d0 con 0x55b142d05070 2024-09-21T06:22:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.319+0000 7fe8f0daf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b1430160e0 con 0x55b142ce4390 2024-09-21T06:22:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.319+0000 7fe8e8dd8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142d05070 0x55b1430ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.319+0000 7fe8e3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142cdc1b0 0x55b1430acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.319+0000 7fe8e8dd8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142d05070 0x55b1430ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:39610/0 (socket says 172.21.15.3:39610) 2024-09-21T06:22:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.319+0000 7fe8e8dd8640 1 -- 172.21.15.3:0/2934417919 learned_addr learned my addr 172.21.15.3:0/2934417919 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.319+0000 7fe8e37fe640 1 --2- 172.21.15.3:0/2934417919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.320+0000 7fe8e8dd8640 1 -- 172.21.15.3:0/2934417919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 msgr2=0x55b1430ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.320+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/2934417919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430ad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.320+0000 7fe8e8dd8640 1 -- 172.21.15.3:0/2934417919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142cdc1b0 msgr2=0x55b1430acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.320+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/2934417919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142cdc1b0 0x55b1430acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.320+0000 7fe8e8dd8640 1 -- 172.21.15.3:0/2934417919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b142d70650 con 0x55b142d05070 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.320+0000 7fe8e17fa640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe8cc0b6680 0x7fe8cc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.320+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/2934417919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142d05070 0x55b1430ac6a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00ae10 tx=0x7fe8dc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=918f5dbff06cba5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.320+0000 7fe8e3fff640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe8cc0b6680 0x7fe8cc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.320+0000 7fe8ca7fc640 1 -- 172.21.15.3:0/2934417919 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc003b70 con 0x55b142d05070 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.320+0000 7fe8ca7fc640 1 -- 172.21.15.3:0/2934417919 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc003d10 con 0x55b142d05070 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.320+0000 7fe8ca7fc640 1 -- 172.21.15.3:0/2934417919 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0109a0 con 0x55b142d05070 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.320+0000 7fe8e17fa640 1 -- 172.21.15.3:0/2468561288 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7fe8d4107fd0 con 0x7fe8e4102a60 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.321+0000 7fe8f0daf740 1 -- 172.21.15.3:0/2934417919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142d05070 msgr2=0x55b1430ac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.321+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/2934417919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142d05070 0x55b1430ac6a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00ae10 tx=0x7fe8dc00f790 comp rx=0 tx=0).stop 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.321+0000 7fe8e3fff640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe8cc0b6680 0x7fe8cc0b8b90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe8d41057e0 tx=0x7fe8d410bb20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.321+0000 7fe8f0daf740 1 -- 172.21.15.3:0/2934417919 shutdown_connections 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.321+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/2934417919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.321+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/2934417919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142cdc1b0 0x55b1430acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.321+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/2934417919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142d05070 0x55b1430ac6a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.321+0000 7fe8f0daf740 1 -- 172.21.15.3:0/2934417919 >> 172.21.15.3:0/2934417919 conn(0x55b142cf9460 msgr2=0x55b1430aa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.321+0000 7fe8f0daf740 1 -- 172.21.15.3:0/2934417919 shutdown_connections 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.321+0000 7fe8f0daf740 1 -- 172.21.15.3:0/2934417919 wait complete. 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.322+0000 7fe8f0daf740 1 -- start start 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.322+0000 7fe8f0daf740 1 Processor -- start 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8f0daf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142cdc1b0 0x55b1430b9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8f0daf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430ba400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8f0daf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142d05070 0x55b1430ba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8f0daf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b142e73700 con 0x55b142cdc1b0 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8f0daf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b142fad160 con 0x55b142d05070 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8f0daf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b1430bcf20 con 0x55b142ce4390 2024-09-21T06:22:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8f0daf740 10 client.0 _ll_register_callbacks cb 0x55b142fc69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8e8dd8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142cdc1b0 0x55b1430b9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8e37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430ba400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8e37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430ba400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:52660/0 (socket says 172.21.15.3:52660) 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8e37fe640 1 -- 172.21.15.3:0/3355227176 learned_addr learned my addr 172.21.15.3:0/3355227176 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/3355227176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142cdc1b0 0x55b1430b9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51812/0 (socket says 172.21.15.3:51812) 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/3355227176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142d05070 0x55b1430ba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8e37fe640 1 -- 172.21.15.3:0/3355227176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142d05070 msgr2=0x55b1430ba940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8e37fe640 1 --2- 172.21.15.3:0/3355227176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142d05070 0x55b1430ba940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8e37fe640 1 -- 172.21.15.3:0/3355227176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142cdc1b0 msgr2=0x55b1430b9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8e37fe640 1 --2- 172.21.15.3:0/3355227176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142cdc1b0 0x55b1430b9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8e37fe640 1 -- 172.21.15.3:0/3355227176 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b142fdac90 con 0x55b142ce4390 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8e37fe640 1 --2- 172.21.15.3:0/3355227176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430ba400 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800bf80 tx=0x7fe8d80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3c79dea9d86717a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.323+0000 7fe8c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:22:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8c37fe640 1 -- 172.21.15.3:0/3355227176 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8004020 con 0x55b142ce4390 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8f0daf740 1 -- 172.21.15.3:0/3355227176 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8dc003d10 con 0x55b142ce4390 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8c37fe640 1 -- 172.21.15.3:0/3355227176 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7fe8d80041c0 con 0x55b142ce4390 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8c37fe640 1 -- 172.21.15.3:0/3355227176 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe8d80043c0 con 0x55b142ce4390 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8f0daf740 10 client.27226 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8f0daf740 1 -- 172.21.15.3:0/3355227176 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b1430c8be0 con 0x55b142ce4390 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8c37fe640 1 -- 172.21.15.3:0/3355227176 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc003d10 con 0x55b142ce4390 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8c37fe640 1 -- 172.21.15.3:0/3355227176 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fe8d8004c60 con 0x55b142ce4390 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8f0daf740 10 client.27226 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8f0daf740 10 client.27226 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8f0daf740 20 client.27226 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/3355227176 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b142cc39f0 0x55b1430cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8f0daf740 4 client.27226 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.324+0000 7fe8f0daf740 1 -- 172.21.15.3:0/3355227176 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b143024df0 con 0x55b142cc39f0 2024-09-21T06:22:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.325+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/3355227176 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b142cc39f0 0x55b1430cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.325+0000 7fe8c37fe640 10 client.27226 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:22:45.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.325+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/3355227176 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b142cc39f0 0x55b1430cc170 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb2fcd901204e8d2 server_cookie=aa87db2cd06f9ae in_seq=0 out_seq=0 2024-09-21T06:22:45.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8c37fe640 1 -- 172.21.15.3:0/3355227176 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b143024df0 con 0x55b142cc39f0 2024-09-21T06:22:45.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8c37fe640 10 client.27226 handle_command_reply: tid=0 2024-09-21T06:22:45.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8f0daf740 1 client.27226 shutdown 2024-09-21T06:22:45.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8f0daf740 1 -- 172.21.15.3:0/3355227176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 msgr2=0x55b1430ba400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:45.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/3355227176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430ba400 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800bf80 tx=0x7fe8d80118d0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8f0daf740 1 -- 172.21.15.3:0/3355227176 shutdown_connections 2024-09-21T06:22:45.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/3355227176 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b142cc39f0 0x55b1430cc170 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/3355227176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430ba400 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/3355227176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142d05070 0x55b1430ba940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/3355227176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142cdc1b0 0x55b1430b9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8f0daf740 1 -- 172.21.15.3:0/3355227176 >> 172.21.15.3:0/3355227176 conn(0x55b142cf9460 msgr2=0x55b1430ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:45.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8c37fe640 0 client.27226 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:22:45.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8f0daf740 1 -- 172.21.15.3:0/3355227176 shutdown_connections 2024-09-21T06:22:45.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8f0daf740 1 -- 172.21.15.3:0/3355227176 wait complete. 2024-09-21T06:22:45.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.326+0000 7fe8f0daf740 20 client.27226 trim_cache size 0 max 16384 2024-09-21T06:22:45.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.340+0000 7fe8f0daf740 1 Processor -- start 2024-09-21T06:22:45.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.340+0000 7fe8f0daf740 1 -- start start 2024-09-21T06:22:45.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.340+0000 7fe8f0daf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142cdc1b0 0x55b1430bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.340+0000 7fe8f0daf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142ce4390 0x55b1430adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.340+0000 7fe8f0daf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142d05070 0x55b1430b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.340+0000 7fe8f0daf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e41ad4e0 con 0x55b142ce4390 2024-09-21T06:22:45.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.340+0000 7fe8f0daf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e41af020 con 0x55b142cdc1b0 2024-09-21T06:22:45.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.340+0000 7fe8f0daf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e41adbb0 con 0x55b142d05070 2024-09-21T06:22:45.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8e37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142ce4390 0x55b1430adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8e37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142ce4390 0x55b1430adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51814/0 (socket says 172.21.15.3:51814) 2024-09-21T06:22:45.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8e37fe640 1 -- 172.21.15.3:0/3919972679 learned_addr learned my addr 172.21.15.3:0/3919972679 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:45.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/3919972679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142d05070 0x55b1430b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/3919972679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142cdc1b0 0x55b1430bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8e8dd8640 1 -- 172.21.15.3:0/3919972679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142cdc1b0 msgr2=0x55b1430bbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:45.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/3919972679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142cdc1b0 0x55b1430bbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8e8dd8640 1 -- 172.21.15.3:0/3919972679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142ce4390 msgr2=0x55b1430adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:45.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/3919972679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142ce4390 0x55b1430adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8e8dd8640 1 -- 172.21.15.3:0/3919972679 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1430ad4e0 con 0x55b142d05070 2024-09-21T06:22:45.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8e37fe640 1 --2- 172.21.15.3:0/3919972679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142ce4390 0x55b1430adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:45.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/3919972679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142d05070 0x55b1430b70b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00ade0 tx=0x7fe8dc00af60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5d2c4ebc9b52c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:45.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8c9ffb640 1 -- 172.21.15.3:0/3919972679 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc004c00 con 0x55b142d05070 2024-09-21T06:22:45.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8c9ffb640 1 -- 172.21.15.3:0/3919972679 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc004da0 con 0x55b142d05070 2024-09-21T06:22:45.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.341+0000 7fe8c9ffb640 1 -- 172.21.15.3:0/3919972679 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc010a70 con 0x55b142d05070 2024-09-21T06:22:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.342+0000 7fe8f0daf740 1 -- 172.21.15.3:0/3919972679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142d05070 msgr2=0x55b1430b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.342+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/3919972679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142d05070 0x55b1430b70b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00ade0 tx=0x7fe8dc00af60 comp rx=0 tx=0).stop 2024-09-21T06:22:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.342+0000 7fe8f0daf740 1 -- 172.21.15.3:0/3919972679 shutdown_connections 2024-09-21T06:22:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.342+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/3919972679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142d05070 0x55b1430b70b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.342+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/3919972679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142ce4390 0x55b1430adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.342+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/3919972679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142cdc1b0 0x55b1430bbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.342+0000 7fe8f0daf740 1 -- 172.21.15.3:0/3919972679 >> 172.21.15.3:0/3919972679 conn(0x55b142cf9460 msgr2=0x55b1430ad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.342+0000 7fe8f0daf740 1 -- 172.21.15.3:0/3919972679 shutdown_connections 2024-09-21T06:22:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.342+0000 7fe8f0daf740 1 -- 172.21.15.3:0/3919972679 wait complete. 2024-09-21T06:22:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.343+0000 7fe8f0daf740 1 -- start start 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.343+0000 7fe8f0daf740 1 Processor -- start 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.343+0000 7fe8f0daf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142cdc1b0 0x55b1430c0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.343+0000 7fe8f0daf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430c0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8f0daf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142d05070 0x55b1430c0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8f0daf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e41ae950 con 0x55b142d05070 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8f0daf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e41ae280 con 0x55b142cdc1b0 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8f0daf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e41ab4e0 con 0x55b142ce4390 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8f0daf740 10 client.0 _ll_register_callbacks cb 0x7fe8d800bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8e8dd8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142cdc1b0 0x55b1430c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8e37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8e8dd8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142cdc1b0 0x55b1430c0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:39632/0 (socket says 172.21.15.3:39632) 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8e8dd8640 1 -- 172.21.15.3:0/2338827185 learned_addr learned my addr 172.21.15.3:0/2338827185 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/2338827185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142d05070 0x55b1430c0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8e37fe640 1 -- 172.21.15.3:0/2338827185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142cdc1b0 msgr2=0x55b1430c0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8e37fe640 1 --2- 172.21.15.3:0/2338827185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142cdc1b0 0x55b1430c0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8e37fe640 1 -- 172.21.15.3:0/2338827185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142d05070 msgr2=0x55b1430c0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8e37fe640 1 --2- 172.21.15.3:0/2338827185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142d05070 0x55b1430c0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8e37fe640 1 -- 172.21.15.3:0/2338827185 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1430c6150 con 0x55b142ce4390 2024-09-21T06:22:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8e37fe640 1 --2- 172.21.15.3:0/2338827185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430c0650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800bca0 tx=0x7fe8d8010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6469b9e120af8d48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.344+0000 7fe8c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:22:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8c37fe640 1 -- 172.21.15.3:0/2338827185 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d802f020 con 0x55b142ce4390 2024-09-21T06:22:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8c37fe640 1 -- 172.21.15.3:0/2338827185 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7fe8d80052d0 con 0x55b142ce4390 2024-09-21T06:22:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8f0daf740 1 -- 172.21.15.3:0/2338827185 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8dc004da0 con 0x55b142ce4390 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8c37fe640 1 -- 172.21.15.3:0/2338827185 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe8d803a050 con 0x55b142ce4390 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8f0daf740 10 client.27232 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8f0daf740 1 -- 172.21.15.3:0/2338827185 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b143136b50 con 0x55b142ce4390 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8c37fe640 1 -- 172.21.15.3:0/2338827185 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc004da0 con 0x55b142ce4390 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8c37fe640 1 -- 172.21.15.3:0/2338827185 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fe8d80364b0 con 0x55b142ce4390 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8f0daf740 10 client.27232 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8f0daf740 10 client.27232 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8f0daf740 20 client.27232 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/2338827185 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b142c98da0 0x55b143139390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8f0daf740 4 client.27232 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.345+0000 7fe8f0daf740 1 -- 172.21.15.3:0/2338827185 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1430b1db0 con 0x55b142c98da0 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.346+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/2338827185 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b142c98da0 0x55b143139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.346+0000 7fe8c37fe640 10 client.27232 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:22:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.346+0000 7fe8e8dd8640 1 --2- 172.21.15.3:0/2338827185 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b142c98da0 0x55b143139390 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db33095cfa8ebd37 server_cookie=7c5b70713cfa5d1d in_seq=0 out_seq=0 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.346+0000 7fe8c37fe640 1 -- 172.21.15.3:0/2338827185 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1430b1db0 con 0x55b142c98da0 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.346+0000 7fe8c37fe640 10 client.27232 handle_command_reply: tid=0 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.346+0000 7fe8f0daf740 1 client.27232 shutdown 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8f0daf740 1 -- 172.21.15.3:0/2338827185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 msgr2=0x55b1430c0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/2338827185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430c0650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800bca0 tx=0x7fe8d8010fd0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8f0daf740 1 -- 172.21.15.3:0/2338827185 shutdown_connections 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/2338827185 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55b142c98da0 0x55b143139390 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/2338827185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b142ce4390 0x55b1430c0650 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/2338827185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b142d05070 0x55b1430c0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8c37fe640 0 client.27232 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8f0daf740 1 --2- 172.21.15.3:0/2338827185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b142cdc1b0 0x55b1430c0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8f0daf740 1 -- 172.21.15.3:0/2338827185 >> 172.21.15.3:0/2338827185 conn(0x55b142cf9460 msgr2=0x55b1430c58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8f0daf740 1 -- 172.21.15.3:0/2338827185 shutdown_connections 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8f0daf740 1 -- 172.21.15.3:0/2338827185 wait complete. 2024-09-21T06:22:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8f0daf740 20 client.27232 trim_cache size 0 max 16384 2024-09-21T06:22:45.354 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:22:45.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8ea83b640 1 -- 172.21.15.3:0/2468561288 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe8cc0b6680 msgr2=0x7fe8cc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8ea83b640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe8cc0b6680 0x7fe8cc0b8b90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe8d41057e0 tx=0x7fe8d410bb20 comp rx=0 tx=0).stop 2024-09-21T06:22:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8ea83b640 1 -- 172.21.15.3:0/2468561288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8e4102a60 msgr2=0x7fe8e41993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.347+0000 7fe8ea83b640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8e4102a60 0x7fe8e41993c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4005ab0 tx=0x7fe8d4003cd0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.348+0000 7fe8ea83b640 1 -- 172.21.15.3:0/2468561288 shutdown_connections 2024-09-21T06:22:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.348+0000 7fe8ea83b640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe8cc0b6680 0x7fe8cc0b8b90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.348+0000 7fe8ea83b640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e41083a0 0x7fe8e4199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.348+0000 7fe8ea83b640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8e4108cf0 0x7fe8e419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.348+0000 7fe8ea83b640 1 --2- 172.21.15.3:0/2468561288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8e4102a60 0x7fe8e41993c0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.348+0000 7fe8ea83b640 1 -- 172.21.15.3:0/2468561288 >> 172.21.15.3:0/2468561288 conn(0x7fe8e4079730 msgr2=0x7fe8e4101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.348+0000 7fe8ea83b640 1 -- 172.21.15.3:0/2468561288 shutdown_connections 2024-09-21T06:22:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:45.348+0000 7fe8ea83b640 1 -- 172.21.15.3:0/2468561288 wait complete. 2024-09-21T06:22:45.966 INFO:tasks.workunit.client.0.smithi003.stdout: 1 825084 11.14 MB/sec execute 258 sec latency 15.492 ms 2024-09-21T06:22:46.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:22:46.397 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:46.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.889+0000 7f990e79d640 1 -- 172.21.15.3:0/2974217395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9908108cf0 msgr2=0x7f990810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.889+0000 7f990e79d640 1 --2- 172.21.15.3:0/2974217395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9908108cf0 0x7f990810f520 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f9900009ca0 tx=0x7f990002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.889+0000 7f990e79d640 1 -- 172.21.15.3:0/2974217395 shutdown_connections 2024-09-21T06:22:46.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.889+0000 7f990e79d640 1 --2- 172.21.15.3:0/2974217395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99081083a0 0x7f99081087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.889+0000 7f990e79d640 1 --2- 172.21.15.3:0/2974217395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9908108cf0 0x7f990810f520 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.889+0000 7f990e79d640 1 --2- 172.21.15.3:0/2974217395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9908102a60 0x7f9908102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.889+0000 7f990e79d640 1 -- 172.21.15.3:0/2974217395 >> 172.21.15.3:0/2974217395 conn(0x7f9908079730 msgr2=0x7f9908079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:46.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.889+0000 7f990e79d640 1 -- 172.21.15.3:0/2974217395 shutdown_connections 2024-09-21T06:22:46.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.889+0000 7f990e79d640 1 -- 172.21.15.3:0/2974217395 wait complete. 2024-09-21T06:22:46.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.890+0000 7f990e79d640 1 Processor -- start 2024-09-21T06:22:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.890+0000 7f990e79d640 1 -- start start 2024-09-21T06:22:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.890+0000 7f990e79d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9908102a60 0x7f990819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.890+0000 7f990e79d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99081083a0 0x7f990819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.890+0000 7f990e79d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9908108cf0 0x7f990819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.890+0000 7f990e79d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9908112960 con 0x7f99081083a0 2024-09-21T06:22:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.890+0000 7f990e79d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f99081127e0 con 0x7f9908102a60 2024-09-21T06:22:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.890+0000 7f990e79d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9908112ae0 con 0x7f9908108cf0 2024-09-21T06:22:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.890+0000 7f98fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99081083a0 0x7f990819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.890+0000 7f98fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99081083a0 0x7f990819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51838/0 (socket says 172.21.15.3:51838) 2024-09-21T06:22:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.890+0000 7f98fffff640 1 -- 172.21.15.3:0/2157418162 learned_addr learned my addr 172.21.15.3:0/2157418162 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.890+0000 7f9907fff640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9908102a60 0x7f990819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.890+0000 7f990cd3a640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9908108cf0 0x7f990819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.891+0000 7f98fffff640 1 -- 172.21.15.3:0/2157418162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9908108cf0 msgr2=0x7f990819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.891+0000 7f98fffff640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9908108cf0 0x7f990819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.891+0000 7f98fffff640 1 -- 172.21.15.3:0/2157418162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9908102a60 msgr2=0x7f990819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.891+0000 7f98fffff640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9908102a60 0x7f990819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.891+0000 7f98fffff640 1 -- 172.21.15.3:0/2157418162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99081a0410 con 0x7f99081083a0 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.891+0000 7f98fffff640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99081083a0 0x7f990819aa40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f98f000e9c0 tx=0x7f98f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.891+0000 7f9905ffb640 1 -- 172.21.15.3:0/2157418162 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f000cd80 con 0x7f99081083a0 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.891+0000 7f9905ffb640 1 -- 172.21.15.3:0/2157418162 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f00054d0 con 0x7f99081083a0 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.891+0000 7f990e79d640 1 -- 172.21.15.3:0/2157418162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99081a0700 con 0x7f99081083a0 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.891+0000 7f9905ffb640 1 -- 172.21.15.3:0/2157418162 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0010430 con 0x7f99081083a0 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.891+0000 7f990e79d640 1 -- 172.21.15.3:0/2157418162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99081a6e60 con 0x7f99081083a0 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f9914d11740 1 Processor -- start 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f9914d11740 1 -- start start 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f9914d11740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x56348438b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f9914d11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x56348438bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f9914d11740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x56348438c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f9914d11740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563484301ea0 con 0x563483fe4070 2024-09-21T06:22:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f9914d11740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5634842f12d0 con 0x563483fc3390 2024-09-21T06:22:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f9914d11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5634842f50e0 con 0x563483fbb1b0 2024-09-21T06:22:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f9905ffb640 1 -- 172.21.15.3:0/2157418162 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f98f0005020 con 0x7f99081083a0 2024-09-21T06:22:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f9907fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x56348438bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f9907fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x56348438bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:52704/0 (socket says 172.21.15.3:52704) 2024-09-21T06:22:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f9907fff640 1 -- 172.21.15.3:0/563944608 learned_addr learned my addr 172.21.15.3:0/563944608 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f990cd3a640 1 --2- 172.21.15.3:0/563944608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x56348438b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.894+0000 7f98fffff640 1 --2- 172.21.15.3:0/563944608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x56348438c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.895+0000 7f9907fff640 1 -- 172.21.15.3:0/563944608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 msgr2=0x56348438c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.895+0000 7f9907fff640 1 --2- 172.21.15.3:0/563944608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x56348438c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.895+0000 7f9907fff640 1 -- 172.21.15.3:0/563944608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 msgr2=0x56348438b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.895+0000 7f9907fff640 1 --2- 172.21.15.3:0/563944608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x56348438b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.895+0000 7f9907fff640 1 -- 172.21.15.3:0/563944608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56348404f650 con 0x563483fbb1b0 2024-09-21T06:22:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.896+0000 7f9907fff640 1 --2- 172.21.15.3:0/563944608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x56348438bbe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009cf0 tx=0x7f98f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5d42df802deef92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.896+0000 7f98fe7fc640 1 -- 172.21.15.3:0/563944608 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8004230 con 0x563483fbb1b0 2024-09-21T06:22:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.896+0000 7f98fe7fc640 1 -- 172.21.15.3:0/563944608 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f80043d0 con 0x563483fbb1b0 2024-09-21T06:22:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.896+0000 7f98fe7fc640 1 -- 172.21.15.3:0/563944608 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8019880 con 0x563483fbb1b0 2024-09-21T06:22:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.896+0000 7f9905ffb640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f98f40b6680 0x7f98f40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.897+0000 7f990cd3a640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f98f40b6680 0x7f98f40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.897+0000 7f9905ffb640 1 -- 172.21.15.3:0/2157418162 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f98f00e7ad0 con 0x7f99081083a0 2024-09-21T06:22:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.897+0000 7f9914d11740 1 -- 172.21.15.3:0/563944608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 msgr2=0x56348438bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.897+0000 7f9914d11740 1 --2- 172.21.15.3:0/563944608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x56348438bbe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009cf0 tx=0x7f98f8011810 comp rx=0 tx=0).stop 2024-09-21T06:22:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.897+0000 7f9914d11740 1 -- 172.21.15.3:0/563944608 shutdown_connections 2024-09-21T06:22:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.897+0000 7f9914d11740 1 --2- 172.21.15.3:0/563944608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x56348438bbe0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.897+0000 7f9914d11740 1 --2- 172.21.15.3:0/563944608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x56348438c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.897+0000 7f9914d11740 1 --2- 172.21.15.3:0/563944608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x56348438b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.897+0000 7f9914d11740 1 -- 172.21.15.3:0/563944608 >> 172.21.15.3:0/563944608 conn(0x563483fd8460 msgr2=0x563484389590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.897+0000 7f990cd3a640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f98f40b6680 0x7f98f40b8b90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f990002f500 tx=0x7f990002f310 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.897+0000 7f9914d11740 1 -- 172.21.15.3:0/563944608 shutdown_connections 2024-09-21T06:22:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.897+0000 7f9914d11740 1 -- 172.21.15.3:0/563944608 wait complete. 2024-09-21T06:22:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.898+0000 7f9914d11740 1 -- start start 2024-09-21T06:22:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.898+0000 7f9914d11740 1 Processor -- start 2024-09-21T06:22:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f9914d11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x563484396d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f9914d11740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x563484397250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f9914d11740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x5634843977b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f9914d11740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563484152700 con 0x563483fe4070 2024-09-21T06:22:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f9914d11740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56348428c160 con 0x563483fc3390 2024-09-21T06:22:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f9914d11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563484399e80 con 0x563483fbb1b0 2024-09-21T06:22:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f9914d11740 10 client.0 _ll_register_callbacks cb 0x5634842a59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f990cd3a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x5634843977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f98fffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x563484397250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f9907fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x563484396d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f98fffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x563484397250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:39672/0 (socket says 172.21.15.3:39672) 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f98fffff640 1 -- 172.21.15.3:0/3017477659 learned_addr learned my addr 172.21.15.3:0/3017477659 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f98fffff640 1 -- 172.21.15.3:0/3017477659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 msgr2=0x563484396d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f98fffff640 1 --2- 172.21.15.3:0/3017477659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x563484396d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f98fffff640 1 -- 172.21.15.3:0/3017477659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 msgr2=0x5634843977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f98fffff640 1 --2- 172.21.15.3:0/3017477659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x5634843977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f98fffff640 1 -- 172.21.15.3:0/3017477659 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634842b9c90 con 0x563483fc3390 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f9907fff640 1 --2- 172.21.15.3:0/3017477659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x563484396d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.899+0000 7f990cd3a640 1 --2- 172.21.15.3:0/3017477659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x5634843977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.900+0000 7f98fffff640 1 --2- 172.21.15.3:0/3017477659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x563484397250 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f98f000c6a0 tx=0x7f98f00b7200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b16246f810bc2ed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.900+0000 7f98e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:22:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.900+0000 7f98e77fe640 1 -- 172.21.15.3:0/3017477659 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f00a9f80 con 0x563483fc3390 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.900+0000 7f98e77fe640 1 -- 172.21.15.3:0/3017477659 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f98f00ac600 con 0x563483fc3390 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.900+0000 7f9914d11740 1 -- 172.21.15.3:0/3017477659 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98f80043d0 con 0x563483fc3390 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.900+0000 7f98e77fe640 1 -- 172.21.15.3:0/3017477659 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f98f00ad0a0 con 0x563483fc3390 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.900+0000 7f9914d11740 10 client.27296 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.900+0000 7f9914d11740 1 -- 172.21.15.3:0/3017477659 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5634843a5bc0 con 0x563483fc3390 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.900+0000 7f98e77fe640 1 -- 172.21.15.3:0/3017477659 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f80043d0 con 0x563483fc3390 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.901+0000 7f98e77fe640 1 -- 172.21.15.3:0/3017477659 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f98f00ac800 con 0x563483fc3390 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.901+0000 7f9914d11740 10 client.27296 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.901+0000 7f9914d11740 10 client.27296 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.901+0000 7f9914d11740 20 client.27296 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.901+0000 7f9914d11740 1 --2- 172.21.15.3:0/3017477659 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563483fa29f0 0x5634843a9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.901+0000 7f9914d11740 4 client.27296 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.901+0000 7f9914d11740 1 -- 172.21.15.3:0/3017477659 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563484303df0 con 0x563483fa29f0 2024-09-21T06:22:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.901+0000 7f9907fff640 1 --2- 172.21.15.3:0/3017477659 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563483fa29f0 0x5634843a9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.902+0000 7f9907fff640 1 --2- 172.21.15.3:0/3017477659 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563483fa29f0 0x5634843a9030 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d3185e107d3981b server_cookie=8c8dc709add7c0f6 in_seq=0 out_seq=0 2024-09-21T06:22:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.902+0000 7f98e77fe640 10 client.27296 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:22:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.904+0000 7f98e77fe640 1 -- 172.21.15.3:0/3017477659 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563484303df0 con 0x563483fa29f0 2024-09-21T06:22:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.905+0000 7f98e77fe640 10 client.27296 handle_command_reply: tid=0 2024-09-21T06:22:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.905+0000 7f9914d11740 1 client.27296 shutdown 2024-09-21T06:22:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.919+0000 7f9914d11740 1 -- 172.21.15.3:0/3017477659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 msgr2=0x563484397250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.919+0000 7f9914d11740 1 --2- 172.21.15.3:0/3017477659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x563484397250 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f98f000c6a0 tx=0x7f98f00b7200 comp rx=0 tx=0).stop 2024-09-21T06:22:46.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.923+0000 7f9914d11740 1 -- 172.21.15.3:0/3017477659 shutdown_connections 2024-09-21T06:22:46.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.923+0000 7f9914d11740 1 --2- 172.21.15.3:0/3017477659 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563483fa29f0 0x5634843a9030 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.923+0000 7f9914d11740 1 --2- 172.21.15.3:0/3017477659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x5634843977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.923+0000 7f9914d11740 1 --2- 172.21.15.3:0/3017477659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x563484397250 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.923+0000 7f9914d11740 1 --2- 172.21.15.3:0/3017477659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x563484396d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.923+0000 7f9914d11740 1 -- 172.21.15.3:0/3017477659 >> 172.21.15.3:0/3017477659 conn(0x563483fd8460 msgr2=0x56348438e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.924+0000 7f98e77fe640 0 client.27296 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:22:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.924+0000 7f9914d11740 1 -- 172.21.15.3:0/3017477659 shutdown_connections 2024-09-21T06:22:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.925+0000 7f9914d11740 1 -- 172.21.15.3:0/3017477659 wait complete. 2024-09-21T06:22:46.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.926+0000 7f9914d11740 20 client.27296 trim_cache size 0 max 16384 2024-09-21T06:22:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.956+0000 7f9914d11740 1 Processor -- start 2024-09-21T06:22:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f9914d11740 1 -- start start 2024-09-21T06:22:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f9914d11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x563484396ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f9914d11740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x563484397000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f9914d11740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x563484393f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f9914d11740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f99081ad4d0 con 0x563483fe4070 2024-09-21T06:22:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f9914d11740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f99081af010 con 0x563483fc3390 2024-09-21T06:22:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f9914d11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f99081adba0 con 0x563483fbb1b0 2024-09-21T06:22:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f990cd3a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x563484393f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f9907fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x563484396ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f990cd3a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x563484393f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51864/0 (socket says 172.21.15.3:51864) 2024-09-21T06:22:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f990cd3a640 1 -- 172.21.15.3:0/3554782196 learned_addr learned my addr 172.21.15.3:0/3554782196 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f98fffff640 1 --2- 172.21.15.3:0/3554782196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x563484397000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f9907fff640 1 -- 172.21.15.3:0/3554782196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 msgr2=0x563484397000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f9907fff640 1 --2- 172.21.15.3:0/3554782196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x563484397000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f9907fff640 1 -- 172.21.15.3:0/3554782196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 msgr2=0x563484393f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f9907fff640 1 --2- 172.21.15.3:0/3554782196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x563484393f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f9907fff640 1 -- 172.21.15.3:0/3554782196 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634843a30d0 con 0x563483fbb1b0 2024-09-21T06:22:46.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.957+0000 7f990cd3a640 1 --2- 172.21.15.3:0/3554782196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x563484393f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:46.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.958+0000 7f9907fff640 1 --2- 172.21.15.3:0/3554782196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x563484396ac0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x56348437b7b0 tx=0x7f98f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=236528c27f19f754 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.958+0000 7f98fdffb640 1 -- 172.21.15.3:0/3554782196 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f80035f0 con 0x563483fbb1b0 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.958+0000 7f98fdffb640 1 -- 172.21.15.3:0/3554782196 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f8005040 con 0x563483fbb1b0 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.958+0000 7f98fdffb640 1 -- 172.21.15.3:0/3554782196 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8019710 con 0x563483fbb1b0 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.959+0000 7f9914d11740 1 -- 172.21.15.3:0/3554782196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 msgr2=0x563484396ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.959+0000 7f9914d11740 1 --2- 172.21.15.3:0/3554782196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x563484396ac0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x56348437b7b0 tx=0x7f98f8011810 comp rx=0 tx=0).stop 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.959+0000 7f9914d11740 1 -- 172.21.15.3:0/3554782196 shutdown_connections 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.959+0000 7f9914d11740 1 --2- 172.21.15.3:0/3554782196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fe4070 0x563484393f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.959+0000 7f9914d11740 1 --2- 172.21.15.3:0/3554782196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fc3390 0x563484397000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.959+0000 7f9914d11740 1 --2- 172.21.15.3:0/3554782196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x563484396ac0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.959+0000 7f9914d11740 1 -- 172.21.15.3:0/3554782196 >> 172.21.15.3:0/3554782196 conn(0x563483fd8460 msgr2=0x563484395590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.959+0000 7f9914d11740 1 -- 172.21.15.3:0/3554782196 shutdown_connections 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.959+0000 7f9914d11740 1 -- 172.21.15.3:0/3554782196 wait complete. 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.959+0000 7f9914d11740 1 -- start start 2024-09-21T06:22:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.959+0000 7f9914d11740 1 Processor -- start 2024-09-21T06:22:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f9914d11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x56348439bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f9914d11740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fc3390 0x56348439c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f9914d11740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fe4070 0x56348440ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f9914d11740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f99081ae940 con 0x563483fc3390 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f9914d11740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f99081ae270 con 0x563483fe4070 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f9914d11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f99081ab4d0 con 0x563483fbb1b0 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f9914d11740 10 client.0 _ll_register_callbacks cb 0x7f98f00eb310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f98fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fc3390 0x56348439c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f98fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fc3390 0x56348439c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51872/0 (socket says 172.21.15.3:51872) 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f98fffff640 1 -- 172.21.15.3:0/1018946406 learned_addr learned my addr 172.21.15.3:0/1018946406 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f9907fff640 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x56348439bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f990cd3a640 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fe4070 0x56348440ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f98fffff640 1 -- 172.21.15.3:0/1018946406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 msgr2=0x56348439bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f98fffff640 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x56348439bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f98fffff640 1 -- 172.21.15.3:0/1018946406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fe4070 msgr2=0x56348440ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f98fffff640 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fe4070 0x56348440ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f98fffff640 1 -- 172.21.15.3:0/1018946406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56348440c0d0 con 0x563483fc3390 2024-09-21T06:22:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.960+0000 7f990cd3a640 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fe4070 0x56348440ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f98fffff640 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fc3390 0x56348439c5c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f98f00ad630 tx=0x7f98f00aee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62eb52230341d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f98e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f98e77fe640 1 -- 172.21.15.3:0/1018946406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f00ad280 con 0x563483fc3390 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f98e77fe640 1 -- 172.21.15.3:0/1018946406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f98f00ac600 con 0x563483fc3390 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f9914d11740 1 -- 172.21.15.3:0/1018946406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98f8005040 con 0x563483fc3390 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f98e77fe640 1 -- 172.21.15.3:0/1018946406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f98f00ad700 con 0x563483fc3390 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f9914d11740 10 client.17736 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f9914d11740 1 -- 172.21.15.3:0/1018946406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5634844133f0 con 0x563483fc3390 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f98e77fe640 1 -- 172.21.15.3:0/1018946406 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f8005040 con 0x563483fc3390 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f98e77fe640 1 -- 172.21.15.3:0/1018946406 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f98f00b6890 con 0x563483fc3390 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f9914d11740 10 client.17736 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f9914d11740 10 client.17736 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f9914d11740 20 client.17736 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.961+0000 7f9914d11740 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563483f77da0 0x5634844159c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.962+0000 7f9914d11740 4 client.17736 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:22:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.962+0000 7f9914d11740 1 -- 172.21.15.3:0/1018946406 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634842ff8a0 con 0x563483f77da0 2024-09-21T06:22:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.962+0000 7f9907fff640 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563483f77da0 0x5634844159c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.962+0000 7f9907fff640 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563483f77da0 0x5634844159c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a1359e11097d317 server_cookie=91a8491cb81118b0 in_seq=0 out_seq=0 2024-09-21T06:22:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.962+0000 7f98e77fe640 10 client.17736 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:22:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.962+0000 7f98e77fe640 1 -- 172.21.15.3:0/1018946406 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634842ff8a0 con 0x563483f77da0 2024-09-21T06:22:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.962+0000 7f98e77fe640 10 client.17736 handle_command_reply: tid=0 2024-09-21T06:22:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.962+0000 7f9914d11740 1 client.17736 shutdown 2024-09-21T06:22:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.962+0000 7f9914d11740 1 -- 172.21.15.3:0/1018946406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fc3390 msgr2=0x56348439c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.963+0000 7f9914d11740 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fc3390 0x56348439c5c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f98f00ad630 tx=0x7f98f00aee90 comp rx=0 tx=0).stop 2024-09-21T06:22:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.963+0000 7f9914d11740 1 -- 172.21.15.3:0/1018946406 shutdown_connections 2024-09-21T06:22:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.963+0000 7f9914d11740 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563483f77da0 0x5634844159c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.963+0000 7f9914d11740 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563483fe4070 0x56348440ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.963+0000 7f98e77fe640 0 client.17736 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:22:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.963+0000 7f9914d11740 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563483fc3390 0x56348439c5c0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.963+0000 7f9914d11740 1 --2- 172.21.15.3:0/1018946406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563483fbb1b0 0x56348439bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.963+0000 7f9914d11740 1 -- 172.21.15.3:0/1018946406 >> 172.21.15.3:0/1018946406 conn(0x563483fd8460 msgr2=0x563484395190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.963+0000 7f9914d11740 1 -- 172.21.15.3:0/1018946406 shutdown_connections 2024-09-21T06:22:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.963+0000 7f9914d11740 1 -- 172.21.15.3:0/1018946406 wait complete. 2024-09-21T06:22:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.963+0000 7f9914d11740 20 client.17736 trim_cache size 0 max 16384 2024-09-21T06:22:46.970 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:22:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.964+0000 7f990e79d640 1 -- 172.21.15.3:0/2157418162 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f98f40b6680 msgr2=0x7f98f40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.964+0000 7f990e79d640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f98f40b6680 0x7f98f40b8b90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f990002f500 tx=0x7f990002f310 comp rx=0 tx=0).stop 2024-09-21T06:22:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.964+0000 7f990e79d640 1 -- 172.21.15.3:0/2157418162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99081083a0 msgr2=0x7f990819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.964+0000 7f990e79d640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99081083a0 0x7f990819aa40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f98f000e9c0 tx=0x7f98f000ee80 comp rx=0 tx=0).stop 2024-09-21T06:22:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.964+0000 7f990e79d640 1 -- 172.21.15.3:0/2157418162 shutdown_connections 2024-09-21T06:22:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.964+0000 7f990e79d640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f98f40b6680 0x7f98f40b8b90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.964+0000 7f990e79d640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9908108cf0 0x7f990819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.964+0000 7f990e79d640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99081083a0 0x7f990819aa40 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.964+0000 7f990e79d640 1 --2- 172.21.15.3:0/2157418162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9908102a60 0x7f990819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.964+0000 7f990e79d640 1 -- 172.21.15.3:0/2157418162 >> 172.21.15.3:0/2157418162 conn(0x7f9908079730 msgr2=0x7f990806b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.964+0000 7f990e79d640 1 -- 172.21.15.3:0/2157418162 shutdown_connections 2024-09-21T06:22:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:46.964+0000 7f990e79d640 1 -- 172.21.15.3:0/2157418162 wait complete. 2024-09-21T06:22:46.972 INFO:tasks.workunit.client.0.smithi003.stdout: 1 827338 11.15 MB/sec execute 259 sec latency 40.629 ms 2024-09-21T06:22:47.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:46 smithi003 ceph-mon[31227]: pgmap v561: 97 pgs: 97 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 316 op/s 2024-09-21T06:22:47.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:46 smithi137 ceph-mon[34464]: pgmap v561: 97 pgs: 97 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 316 op/s 2024-09-21T06:22:47.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:46 smithi204 ceph-mon[34659]: pgmap v561: 97 pgs: 97 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 316 op/s 2024-09-21T06:22:47.792 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:22:47.966 INFO:tasks.workunit.client.0.smithi003.stdout: 1 828652 11.13 MB/sec execute 260 sec latency 89.958 ms 2024-09-21T06:22:48.052 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:48.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.566+0000 7fe7117c7640 1 -- 172.21.15.3:0/4267084912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c102a60 msgr2=0x7fe70c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:48.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.566+0000 7fe7117c7640 1 --2- 172.21.15.3:0/4267084912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c102a60 0x7fe70c102e70 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe704009900 tx=0x7fe70402a410 comp rx=0 tx=0).stop 2024-09-21T06:22:48.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.566+0000 7fe7117c7640 1 -- 172.21.15.3:0/4267084912 shutdown_connections 2024-09-21T06:22:48.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.566+0000 7fe7117c7640 1 --2- 172.21.15.3:0/4267084912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe70c108cf0 0x7fe70c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.566+0000 7fe7117c7640 1 --2- 172.21.15.3:0/4267084912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe70c1083a0 0x7fe70c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.566+0000 7fe7117c7640 1 --2- 172.21.15.3:0/4267084912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c102a60 0x7fe70c102e70 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.566+0000 7fe7117c7640 1 -- 172.21.15.3:0/4267084912 >> 172.21.15.3:0/4267084912 conn(0x7fe70c079730 msgr2=0x7fe70c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.566+0000 7fe7117c7640 1 -- 172.21.15.3:0/4267084912 shutdown_connections 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.566+0000 7fe7117c7640 1 -- 172.21.15.3:0/4267084912 wait complete. 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.567+0000 7fe7117c7640 1 Processor -- start 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.567+0000 7fe7117c7640 1 -- start start 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.567+0000 7fe7117c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe70c102a60 0x7fe70c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.567+0000 7fe7117c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe70c1083a0 0x7fe70c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.567+0000 7fe7117c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c108cf0 0x7fe70c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.567+0000 7fe7117c7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c112960 con 0x7fe70c108cf0 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.567+0000 7fe7117c7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1127e0 con 0x7fe70c102a60 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.567+0000 7fe7117c7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c112ae0 con 0x7fe70c1083a0 2024-09-21T06:22:48.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe70a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe70c1083a0 0x7fe70c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe70b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c108cf0 0x7fe70c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe70affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe70c102a60 0x7fe70c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe70a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe70c1083a0 0x7fe70c199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:52742/0 (socket says 172.21.15.3:52742) 2024-09-21T06:22:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe70a7fc640 1 -- 172.21.15.3:0/3121724957 learned_addr learned my addr 172.21.15.3:0/3121724957 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe70a7fc640 1 -- 172.21.15.3:0/3121724957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe70c102a60 msgr2=0x7fe70c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe70a7fc640 1 --2- 172.21.15.3:0/3121724957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe70c102a60 0x7fe70c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe70a7fc640 1 -- 172.21.15.3:0/3121724957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c108cf0 msgr2=0x7fe70c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe70a7fc640 1 --2- 172.21.15.3:0/3121724957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c108cf0 0x7fe70c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe70a7fc640 1 -- 172.21.15.3:0/3121724957 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c19f220 con 0x7fe70c1083a0 2024-09-21T06:22:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe70b7fe640 1 --2- 172.21.15.3:0/3121724957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c108cf0 0x7fe70c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe70a7fc640 1 --2- 172.21.15.3:0/3121724957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe70c1083a0 0x7fe70c199910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4005fd0 tx=0x7fe6f400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe703fff640 1 -- 172.21.15.3:0/3121724957 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4017070 con 0x7fe70c1083a0 2024-09-21T06:22:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe7117c7640 1 -- 172.21.15.3:0/3121724957 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c19f510 con 0x7fe70c1083a0 2024-09-21T06:22:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe703fff640 1 -- 172.21.15.3:0/3121724957 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f4009c30 con 0x7fe70c1083a0 2024-09-21T06:22:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe703fff640 1 -- 172.21.15.3:0/3121724957 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4003d30 con 0x7fe70c1083a0 2024-09-21T06:22:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.568+0000 7fe7117c7640 1 -- 172.21.15.3:0/3121724957 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c1a5d30 con 0x7fe70c1083a0 2024-09-21T06:22:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.571+0000 7fe717d3b740 1 Processor -- start 2024-09-21T06:22:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.571+0000 7fe717d3b740 1 -- start start 2024-09-21T06:22:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.571+0000 7fe703fff640 1 -- 172.21.15.3:0/3121724957 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fe6f4003ed0 con 0x7fe70c1083a0 2024-09-21T06:22:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe717d3b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cda070 0x55e13a0816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe717d3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb11b0 0x55e13a081be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe717d3b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139cb9390 0x55e13a082120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe717d3b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e139ff7ea0 con 0x55e139cda070 2024-09-21T06:22:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe717d3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e139fe72d0 con 0x55e139cb11b0 2024-09-21T06:22:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe717d3b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e139feb0e0 con 0x55e139cb9390 2024-09-21T06:22:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe70b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cda070 0x55e13a0816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe70b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cda070 0x55e13a0816a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51900/0 (socket says 172.21.15.3:51900) 2024-09-21T06:22:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe70b7fe640 1 -- 172.21.15.3:0/3499711110 learned_addr learned my addr 172.21.15.3:0/3499711110 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe70affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb11b0 0x55e13a081be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe70a7fc640 1 --2- 172.21.15.3:0/3499711110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139cb9390 0x55e13a082120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe70b7fe640 1 -- 172.21.15.3:0/3499711110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139cb9390 msgr2=0x55e13a082120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe70b7fe640 1 --2- 172.21.15.3:0/3499711110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139cb9390 0x55e13a082120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe70b7fe640 1 -- 172.21.15.3:0/3499711110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb11b0 msgr2=0x55e13a081be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe70b7fe640 1 --2- 172.21.15.3:0/3499711110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb11b0 0x55e13a081be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe70b7fe640 1 -- 172.21.15.3:0/3499711110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e139d45650 con 0x55e139cda070 2024-09-21T06:22:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe70b7fe640 1 --2- 172.21.15.3:0/3499711110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cda070 0x55e13a0816a0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00f790 tx=0x7fe6fc00fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e13747d466456e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.572+0000 7fe7017fa640 1 -- 172.21.15.3:0/3499711110 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc004d20 con 0x55e139cda070 2024-09-21T06:22:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.573+0000 7fe7017fa640 1 -- 172.21.15.3:0/3499711110 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6fc00ddd0 con 0x55e139cda070 2024-09-21T06:22:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.573+0000 7fe7017fa640 1 -- 172.21.15.3:0/3499711110 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc005910 con 0x55e139cda070 2024-09-21T06:22:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.573+0000 7fe703fff640 1 --2- 172.21.15.3:0/3121724957 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe6f80b66b0 0x7fe6f80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.573+0000 7fe70affd640 1 --2- 172.21.15.3:0/3121724957 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe6f80b66b0 0x7fe6f80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.573+0000 7fe703fff640 1 -- 172.21.15.3:0/3121724957 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7fe6f40e73b0 con 0x7fe70c1083a0 2024-09-21T06:22:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.573+0000 7fe717d3b740 1 -- 172.21.15.3:0/3499711110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cda070 msgr2=0x55e13a0816a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.573+0000 7fe70affd640 1 --2- 172.21.15.3:0/3121724957 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe6f80b66b0 0x7fe6f80b8bc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe704031440 tx=0x7fe704005db0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.573+0000 7fe717d3b740 1 --2- 172.21.15.3:0/3499711110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cda070 0x55e13a0816a0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00f790 tx=0x7fe6fc00fc50 comp rx=0 tx=0).stop 2024-09-21T06:22:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.574+0000 7fe717d3b740 1 -- 172.21.15.3:0/3499711110 shutdown_connections 2024-09-21T06:22:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.574+0000 7fe717d3b740 1 --2- 172.21.15.3:0/3499711110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139cb9390 0x55e13a082120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.574+0000 7fe717d3b740 1 --2- 172.21.15.3:0/3499711110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb11b0 0x55e13a081be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.574+0000 7fe717d3b740 1 --2- 172.21.15.3:0/3499711110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cda070 0x55e13a0816a0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.574+0000 7fe717d3b740 1 -- 172.21.15.3:0/3499711110 >> 172.21.15.3:0/3499711110 conn(0x55e139cce460 msgr2=0x55e13a07f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.574+0000 7fe717d3b740 1 -- 172.21.15.3:0/3499711110 shutdown_connections 2024-09-21T06:22:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.574+0000 7fe717d3b740 1 -- 172.21.15.3:0/3499711110 wait complete. 2024-09-21T06:22:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.574+0000 7fe717d3b740 1 -- start start 2024-09-21T06:22:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.574+0000 7fe717d3b740 1 Processor -- start 2024-09-21T06:22:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.574+0000 7fe717d3b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139cb11b0 0x55e13a08eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.574+0000 7fe717d3b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb9390 0x55e13a08f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.574+0000 7fe717d3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cda070 0x55e13a08f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe717d3b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e139e48700 con 0x55e139cb9390 2024-09-21T06:22:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe717d3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e139f82160 con 0x55e139cda070 2024-09-21T06:22:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe717d3b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e13a091fb0 con 0x55e139cb11b0 2024-09-21T06:22:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe70affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139cb11b0 0x55e13a08eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe70a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb9390 0x55e13a08f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe70affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139cb11b0 0x55e13a08eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:52770/0 (socket says 172.21.15.3:52770) 2024-09-21T06:22:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe70affd640 1 -- 172.21.15.3:0/1801021701 learned_addr learned my addr 172.21.15.3:0/1801021701 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe70b7fe640 1 --2- 172.21.15.3:0/1801021701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cda070 0x55e13a08f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe717d3b740 10 client.0 _ll_register_callbacks cb 0x55e139f9b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe70a7fc640 1 -- 172.21.15.3:0/1801021701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139cb11b0 msgr2=0x55e13a08eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe70a7fc640 1 --2- 172.21.15.3:0/1801021701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139cb11b0 0x55e13a08eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe70a7fc640 1 -- 172.21.15.3:0/1801021701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cda070 msgr2=0x55e13a08f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe70a7fc640 1 --2- 172.21.15.3:0/1801021701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cda070 0x55e13a08f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe70a7fc640 1 -- 172.21.15.3:0/1801021701 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e139fafc90 con 0x55e139cb9390 2024-09-21T06:22:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe70b7fe640 1 --2- 172.21.15.3:0/1801021701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cda070 0x55e13a08f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe70a7fc640 1 --2- 172.21.15.3:0/1801021701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb9390 0x55e13a08f420 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40e4ab0 tx=0x7fe6f40a9170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3f487b8065070 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.575+0000 7fe6f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.576+0000 7fe6f27fc640 1 -- 172.21.15.3:0/1801021701 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40ac920 con 0x55e139cb9390 2024-09-21T06:22:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.576+0000 7fe6f27fc640 1 -- 172.21.15.3:0/1801021701 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7fe6f40eab50 con 0x55e139cb9390 2024-09-21T06:22:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.576+0000 7fe717d3b740 1 -- 172.21.15.3:0/1801021701 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6fc00ddd0 con 0x55e139cb9390 2024-09-21T06:22:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.576+0000 7fe6f27fc640 1 -- 172.21.15.3:0/1801021701 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe6f40e4f40 con 0x55e139cb9390 2024-09-21T06:22:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.576+0000 7fe717d3b740 10 client.17748 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.576+0000 7fe717d3b740 1 -- 172.21.15.3:0/1801021701 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e13a09dc50 con 0x55e139cb9390 2024-09-21T06:22:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.576+0000 7fe6f27fc640 1 -- 172.21.15.3:0/1801021701 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6fc00ddd0 con 0x55e139cb9390 2024-09-21T06:22:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.577+0000 7fe6f27fc640 1 -- 172.21.15.3:0/1801021701 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fe6f40a8b00 con 0x55e139cb9390 2024-09-21T06:22:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.577+0000 7fe717d3b740 10 client.17748 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.577+0000 7fe717d3b740 10 client.17748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.577+0000 7fe717d3b740 20 client.17748 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.577+0000 7fe717d3b740 1 --2- 172.21.15.3:0/1801021701 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e139c989f0 0x55e13a0a1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.577+0000 7fe717d3b740 4 client.17748 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.577+0000 7fe717d3b740 1 -- 172.21.15.3:0/1801021701 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e139ff9df0 con 0x55e139c989f0 2024-09-21T06:22:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.577+0000 7fe70affd640 1 --2- 172.21.15.3:0/1801021701 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e139c989f0 0x55e13a0a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.578+0000 7fe6f27fc640 10 client.17748 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.578+0000 7fe70affd640 1 --2- 172.21.15.3:0/1801021701 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e139c989f0 0x55e13a0a1100 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aad85feb1ba3b75d server_cookie=e23f254f365bf00b in_seq=0 out_seq=0 2024-09-21T06:22:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe6f27fc640 1 -- 172.21.15.3:0/1801021701 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e139ff9df0 con 0x55e139c989f0 2024-09-21T06:22:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe6f27fc640 10 client.17748 handle_command_reply: tid=0 2024-09-21T06:22:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe717d3b740 1 client.17748 shutdown 2024-09-21T06:22:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe717d3b740 1 -- 172.21.15.3:0/1801021701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb9390 msgr2=0x55e13a08f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe717d3b740 1 --2- 172.21.15.3:0/1801021701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb9390 0x55e13a08f420 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40e4ab0 tx=0x7fe6f40a9170 comp rx=0 tx=0).stop 2024-09-21T06:22:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe717d3b740 1 -- 172.21.15.3:0/1801021701 shutdown_connections 2024-09-21T06:22:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe717d3b740 1 --2- 172.21.15.3:0/1801021701 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e139c989f0 0x55e13a0a1100 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe6f27fc640 0 client.17748 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe717d3b740 1 --2- 172.21.15.3:0/1801021701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cda070 0x55e13a08f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe717d3b740 1 --2- 172.21.15.3:0/1801021701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb9390 0x55e13a08f420 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe717d3b740 1 --2- 172.21.15.3:0/1801021701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139cb11b0 0x55e13a08eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe717d3b740 1 -- 172.21.15.3:0/1801021701 >> 172.21.15.3:0/1801021701 conn(0x55e139cce460 msgr2=0x55e13a07dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe717d3b740 1 -- 172.21.15.3:0/1801021701 shutdown_connections 2024-09-21T06:22:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe717d3b740 1 -- 172.21.15.3:0/1801021701 wait complete. 2024-09-21T06:22:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.579+0000 7fe717d3b740 20 client.17748 trim_cache size 0 max 16384 2024-09-21T06:22:48.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.594+0000 7fe717d3b740 1 Processor -- start 2024-09-21T06:22:48.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.594+0000 7fe717d3b740 1 -- start start 2024-09-21T06:22:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe717d3b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139c989f0 0x55e13a090dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe717d3b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb11b0 0x55e13a082de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe717d3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb9390 0x55e13a08c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe717d3b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1ad470 con 0x55e139cb11b0 2024-09-21T06:22:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe717d3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1aefb0 con 0x55e139cb9390 2024-09-21T06:22:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe717d3b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1adb40 con 0x55e139c989f0 2024-09-21T06:22:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe70a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb11b0 0x55e13a082de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe70a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb11b0 0x55e13a082de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51924/0 (socket says 172.21.15.3:51924) 2024-09-21T06:22:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe70affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139c989f0 0x55e13a090dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe70a7fc640 1 -- 172.21.15.3:0/205302395 learned_addr learned my addr 172.21.15.3:0/205302395 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe70b7fe640 1 --2- 172.21.15.3:0/205302395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb9390 0x55e13a08c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe70a7fc640 1 -- 172.21.15.3:0/205302395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139c989f0 msgr2=0x55e13a090dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:48.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe70a7fc640 1 --2- 172.21.15.3:0/205302395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139c989f0 0x55e13a090dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe70a7fc640 1 -- 172.21.15.3:0/205302395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb9390 msgr2=0x55e13a08c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:48.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe70a7fc640 1 --2- 172.21.15.3:0/205302395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb9390 0x55e13a08c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.595+0000 7fe70a7fc640 1 -- 172.21.15.3:0/205302395 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e13a08d3e0 con 0x55e139cb11b0 2024-09-21T06:22:48.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.596+0000 7fe70a7fc640 1 --2- 172.21.15.3:0/205302395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb11b0 0x55e13a082de0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009610 tx=0x7fe6f40a9800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7531981c44e3a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:48.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.596+0000 7fe700ff9640 1 -- 172.21.15.3:0/205302395 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40accc0 con 0x55e139cb11b0 2024-09-21T06:22:48.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.596+0000 7fe700ff9640 1 -- 172.21.15.3:0/205302395 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f40d4040 con 0x55e139cb11b0 2024-09-21T06:22:48.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.596+0000 7fe700ff9640 1 -- 172.21.15.3:0/205302395 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40ac410 con 0x55e139cb11b0 2024-09-21T06:22:48.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.597+0000 7fe717d3b740 1 -- 172.21.15.3:0/205302395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb11b0 msgr2=0x55e13a082de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:48.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.597+0000 7fe717d3b740 1 --2- 172.21.15.3:0/205302395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb11b0 0x55e13a082de0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009610 tx=0x7fe6f40a9800 comp rx=0 tx=0).stop 2024-09-21T06:22:48.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.597+0000 7fe717d3b740 1 -- 172.21.15.3:0/205302395 shutdown_connections 2024-09-21T06:22:48.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.597+0000 7fe717d3b740 1 --2- 172.21.15.3:0/205302395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb9390 0x55e13a08c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.597+0000 7fe717d3b740 1 --2- 172.21.15.3:0/205302395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb11b0 0x55e13a082de0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.597+0000 7fe717d3b740 1 --2- 172.21.15.3:0/205302395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139c989f0 0x55e13a090dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.597+0000 7fe717d3b740 1 -- 172.21.15.3:0/205302395 >> 172.21.15.3:0/205302395 conn(0x55e139cce460 msgr2=0x55e13a0826d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:48.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.597+0000 7fe717d3b740 1 -- 172.21.15.3:0/205302395 shutdown_connections 2024-09-21T06:22:48.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.597+0000 7fe717d3b740 1 -- 172.21.15.3:0/205302395 wait complete. 2024-09-21T06:22:48.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.597+0000 7fe717d3b740 1 -- start start 2024-09-21T06:22:48.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.597+0000 7fe717d3b740 1 Processor -- start 2024-09-21T06:22:48.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe717d3b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139c989f0 0x55e13a095500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe717d3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb11b0 0x55e13a095a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe717d3b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb9390 0x55e13a10aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe717d3b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1ae8e0 con 0x55e139cb9390 2024-09-21T06:22:48.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe717d3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1ae210 con 0x55e139cb11b0 2024-09-21T06:22:48.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe717d3b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1ab470 con 0x55e139c989f0 2024-09-21T06:22:48.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe717d3b740 10 client.0 _ll_register_callbacks cb 0x7fe6f4005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe70b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb9390 0x55e13a10aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe70a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb11b0 0x55e13a095a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe70affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139c989f0 0x55e13a095500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe70b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb9390 0x55e13a10aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51934/0 (socket says 172.21.15.3:51934) 2024-09-21T06:22:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe70b7fe640 1 -- 172.21.15.3:0/3791025381 learned_addr learned my addr 172.21.15.3:0/3791025381 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe70b7fe640 1 -- 172.21.15.3:0/3791025381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139c989f0 msgr2=0x55e13a095500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe70b7fe640 1 --2- 172.21.15.3:0/3791025381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139c989f0 0x55e13a095500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe70b7fe640 1 -- 172.21.15.3:0/3791025381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb11b0 msgr2=0x55e13a095a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe70b7fe640 1 --2- 172.21.15.3:0/3791025381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb11b0 0x55e13a095a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe70b7fe640 1 -- 172.21.15.3:0/3791025381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e13a096290 con 0x55e139cb9390 2024-09-21T06:22:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe70b7fe640 1 --2- 172.21.15.3:0/3791025381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb9390 0x55e13a10aad0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc007cf0 tx=0x7fe6fc005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e94582cc02e48d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.598+0000 7fe6f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.599+0000 7fe6f27fc640 1 -- 172.21.15.3:0/3791025381 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc01d040 con 0x55e139cb9390 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.599+0000 7fe6f27fc640 1 -- 172.21.15.3:0/3791025381 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7fe6fc005820 con 0x55e139cb9390 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.599+0000 7fe717d3b740 1 -- 172.21.15.3:0/3791025381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6f40d4040 con 0x55e139cb9390 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.599+0000 7fe6f27fc640 1 -- 172.21.15.3:0/3791025381 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe6fc005280 con 0x55e139cb9390 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.599+0000 7fe717d3b740 10 client.17754 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.599+0000 7fe717d3b740 1 -- 172.21.15.3:0/3791025381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e13a0879d0 con 0x55e139cb9390 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.599+0000 7fe6f27fc640 1 -- 172.21.15.3:0/3791025381 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f40d4040 con 0x55e139cb9390 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.600+0000 7fe6f27fc640 1 -- 172.21.15.3:0/3791025381 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fe6fc025800 con 0x55e139cb9390 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.600+0000 7fe717d3b740 10 client.17754 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.600+0000 7fe717d3b740 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.600+0000 7fe717d3b740 20 client.17754 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.600+0000 7fe717d3b740 1 --2- 172.21.15.3:0/3791025381 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e139cda070 0x55e13a087e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.600+0000 7fe717d3b740 4 client.17754 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.600+0000 7fe717d3b740 1 -- 172.21.15.3:0/3791025381 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e139ff58a0 con 0x55e139cda070 2024-09-21T06:22:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.600+0000 7fe70affd640 1 --2- 172.21.15.3:0/3791025381 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e139cda070 0x55e13a087e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:48.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.600+0000 7fe6f27fc640 10 client.17754 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:48.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.600+0000 7fe70affd640 1 --2- 172.21.15.3:0/3791025381 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e139cda070 0x55e13a087e90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe28f2d5ec1d8ab3 server_cookie=bef6ee670f62b17e in_seq=0 out_seq=0 2024-09-21T06:22:48.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe6f27fc640 1 -- 172.21.15.3:0/3791025381 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e139ff58a0 con 0x55e139cda070 2024-09-21T06:22:48.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe6f27fc640 10 client.17754 handle_command_reply: tid=0 2024-09-21T06:22:48.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe717d3b740 1 client.17754 shutdown 2024-09-21T06:22:48.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe717d3b740 1 -- 172.21.15.3:0/3791025381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb9390 msgr2=0x55e13a10aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:48.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe717d3b740 1 --2- 172.21.15.3:0/3791025381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb9390 0x55e13a10aad0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc007cf0 tx=0x7fe6fc005470 comp rx=0 tx=0).stop 2024-09-21T06:22:48.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe717d3b740 1 -- 172.21.15.3:0/3791025381 shutdown_connections 2024-09-21T06:22:48.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe717d3b740 1 --2- 172.21.15.3:0/3791025381 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55e139cda070 0x55e13a087e90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe717d3b740 1 --2- 172.21.15.3:0/3791025381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e139cb9390 0x55e13a10aad0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe717d3b740 1 --2- 172.21.15.3:0/3791025381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e139cb11b0 0x55e13a095a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe6f27fc640 0 client.17754 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:48.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe717d3b740 1 --2- 172.21.15.3:0/3791025381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e139c989f0 0x55e13a095500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe717d3b740 1 -- 172.21.15.3:0/3791025381 >> 172.21.15.3:0/3791025381 conn(0x55e139cce460 msgr2=0x55e13a083090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:48.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe717d3b740 1 -- 172.21.15.3:0/3791025381 shutdown_connections 2024-09-21T06:22:48.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.602+0000 7fe717d3b740 1 -- 172.21.15.3:0/3791025381 wait complete. 2024-09-21T06:22:48.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.603+0000 7fe717d3b740 20 client.17754 trim_cache size 0 max 16384 2024-09-21T06:22:48.608 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:22:48.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.603+0000 7fe7117c7640 1 -- 172.21.15.3:0/3121724957 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe6f80b66b0 msgr2=0x7fe6f80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:48.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.603+0000 7fe7117c7640 1 --2- 172.21.15.3:0/3121724957 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe6f80b66b0 0x7fe6f80b8bc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe704031440 tx=0x7fe704005db0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.603+0000 7fe7117c7640 1 -- 172.21.15.3:0/3121724957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe70c1083a0 msgr2=0x7fe70c199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:48.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.603+0000 7fe7117c7640 1 --2- 172.21.15.3:0/3121724957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe70c1083a0 0x7fe70c199910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4005fd0 tx=0x7fe6f400dde0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.603+0000 7fe7117c7640 1 -- 172.21.15.3:0/3121724957 shutdown_connections 2024-09-21T06:22:48.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.603+0000 7fe7117c7640 1 --2- 172.21.15.3:0/3121724957 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe6f80b66b0 0x7fe6f80b8bc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.603+0000 7fe7117c7640 1 --2- 172.21.15.3:0/3121724957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe70c1083a0 0x7fe70c199910 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.603+0000 7fe7117c7640 1 --2- 172.21.15.3:0/3121724957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c108cf0 0x7fe70c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.603+0000 7fe7117c7640 1 --2- 172.21.15.3:0/3121724957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe70c102a60 0x7fe70c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:48.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.603+0000 7fe7117c7640 1 -- 172.21.15.3:0/3121724957 >> 172.21.15.3:0/3121724957 conn(0x7fe70c079730 msgr2=0x7fe70c1014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:48.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.603+0000 7fe7117c7640 1 -- 172.21.15.3:0/3121724957 shutdown_connections 2024-09-21T06:22:48.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:48.603+0000 7fe7117c7640 1 -- 172.21.15.3:0/3121724957 wait complete. 2024-09-21T06:22:48.966 INFO:tasks.workunit.client.0.smithi003.stdout: 1 830931 11.14 MB/sec execute 261 sec latency 50.061 ms 2024-09-21T06:22:49.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:48 smithi137 ceph-mon[34464]: pgmap v562: 97 pgs: 97 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-09-21T06:22:49.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:48 smithi204 ceph-mon[34659]: pgmap v562: 97 pgs: 97 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-09-21T06:22:49.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:48 smithi003 ceph-mon[31227]: pgmap v562: 97 pgs: 97 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-09-21T06:22:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.902+0000 7f863eb6e640 1 -- 172.21.15.3:0/2510833681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638103b00 msgr2=0x7f8638103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.902+0000 7f863eb6e640 1 --2- 172.21.15.3:0/2510833681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638103b00 0x7f8638103f10 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8630009900 tx=0x7f863002a410 comp rx=0 tx=0).stop 2024-09-21T06:22:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.902+0000 7f863eb6e640 1 -- 172.21.15.3:0/2510833681 shutdown_connections 2024-09-21T06:22:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.902+0000 7f863eb6e640 1 --2- 172.21.15.3:0/2510833681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8638109d80 0x7f86381105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.902+0000 7f863eb6e640 1 --2- 172.21.15.3:0/2510833681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8638109430 0x7f8638109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.902+0000 7f863eb6e640 1 --2- 172.21.15.3:0/2510833681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638103b00 0x7f8638103f10 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.902+0000 7f863eb6e640 1 -- 172.21.15.3:0/2510833681 >> 172.21.15.3:0/2510833681 conn(0x7f86380ff500 msgr2=0x7f8638101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.902+0000 7f863eb6e640 1 -- 172.21.15.3:0/2510833681 shutdown_connections 2024-09-21T06:22:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.902+0000 7f863eb6e640 1 -- 172.21.15.3:0/2510833681 wait complete. 2024-09-21T06:22:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.902+0000 7f863eb6e640 1 Processor -- start 2024-09-21T06:22:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.902+0000 7f863eb6e640 1 -- start start 2024-09-21T06:22:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863eb6e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8638103b00 0x7f863819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863eb6e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8638109430 0x7f863819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863eb6e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638109d80 0x7f863819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863eb6e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8638113a40 con 0x7f8638109d80 2024-09-21T06:22:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863eb6e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86381138c0 con 0x7f8638103b00 2024-09-21T06:22:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863eb6e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8638113bc0 con 0x7f8638109430 2024-09-21T06:22:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863d10b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638109d80 0x7f863819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863c90a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8638103b00 0x7f863819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f8637fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8638109430 0x7f863819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863d10b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638109d80 0x7f863819fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51952/0 (socket says 172.21.15.3:51952) 2024-09-21T06:22:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863c90a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8638103b00 0x7f863819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:39774/0 (socket says 172.21.15.3:39774) 2024-09-21T06:22:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863d10b640 1 -- 172.21.15.3:0/378099822 learned_addr learned my addr 172.21.15.3:0/378099822 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863d10b640 1 -- 172.21.15.3:0/378099822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8638109430 msgr2=0x7f863819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863d10b640 1 --2- 172.21.15.3:0/378099822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8638109430 0x7f863819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863d10b640 1 -- 172.21.15.3:0/378099822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8638103b00 msgr2=0x7f863819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863d10b640 1 --2- 172.21.15.3:0/378099822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8638103b00 0x7f863819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863d10b640 1 -- 172.21.15.3:0/378099822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86381a0430 con 0x7f8638109d80 2024-09-21T06:22:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.903+0000 7f863d10b640 1 --2- 172.21.15.3:0/378099822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638109d80 0x7f863819fd30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f862000e970 tx=0x7f862000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.904+0000 7f8635ffb640 1 -- 172.21.15.3:0/378099822 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862000ccc0 con 0x7f8638109d80 2024-09-21T06:22:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.904+0000 7f8635ffb640 1 -- 172.21.15.3:0/378099822 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f862000ce60 con 0x7f8638109d80 2024-09-21T06:22:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.904+0000 7f863eb6e640 1 -- 172.21.15.3:0/378099822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86381a0720 con 0x7f8638109d80 2024-09-21T06:22:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.904+0000 7f8635ffb640 1 -- 172.21.15.3:0/378099822 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620010430 con 0x7f8638109d80 2024-09-21T06:22:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.904+0000 7f863eb6e640 1 -- 172.21.15.3:0/378099822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86381a6eb0 con 0x7f8638109d80 2024-09-21T06:22:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8635ffb640 1 -- 172.21.15.3:0/378099822 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f8620004f90 con 0x7f8638109d80 2024-09-21T06:22:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8645bbb740 1 Processor -- start 2024-09-21T06:22:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8645bbb740 1 -- start start 2024-09-21T06:22:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8645bbb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f7053859b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8645bbb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705385ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8645bbb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 0x55f705386430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8645bbb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f7052fa6f0 con 0x55f704fb72f0 2024-09-21T06:22:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8645bbb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f7052e9b20 con 0x55f704fbf4d0 2024-09-21T06:22:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8645bbb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f7052ed930 con 0x55f704fe01b0 2024-09-21T06:22:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f863c90a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705385ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8637fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f7053859b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8637fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f7053859b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:52826/0 (socket says 172.21.15.3:52826) 2024-09-21T06:22:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8637fff640 1 -- 172.21.15.3:0/3467670720 learned_addr learned my addr 172.21.15.3:0/3467670720 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f863c90a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705385ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51966/0 (socket says 172.21.15.3:51966) 2024-09-21T06:22:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8637fff640 1 --2- 172.21.15.3:0/3467670720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 0x55f705386430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8637fff640 1 -- 172.21.15.3:0/3467670720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 msgr2=0x55f705386430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8637fff640 1 --2- 172.21.15.3:0/3467670720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 0x55f705386430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8637fff640 1 -- 172.21.15.3:0/3467670720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 msgr2=0x55f705385ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8637fff640 1 --2- 172.21.15.3:0/3467670720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705385ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.907+0000 7f8637fff640 1 -- 172.21.15.3:0/3467670720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7052af530 con 0x55f704fe01b0 2024-09-21T06:22:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.908+0000 7f8635ffb640 1 --2- 172.21.15.3:0/378099822 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86240b6700 0x7f86240b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.908+0000 7f8637fff640 1 --2- 172.21.15.3:0/3467670720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f7053859b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8628011b00 tx=0x7f8628009990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2266e9cd9b64647d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.908+0000 7f863c90a640 1 --2- 172.21.15.3:0/378099822 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86240b6700 0x7f86240b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.908+0000 7f862effd640 1 -- 172.21.15.3:0/3467670720 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628011070 con 0x55f704fe01b0 2024-09-21T06:22:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.908+0000 7f862effd640 1 -- 172.21.15.3:0/3467670720 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8628003e20 con 0x55f704fe01b0 2024-09-21T06:22:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.908+0000 7f862effd640 1 -- 172.21.15.3:0/3467670720 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628004050 con 0x55f704fe01b0 2024-09-21T06:22:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.908+0000 7f8635ffb640 1 -- 172.21.15.3:0/378099822 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f86200e7a10 con 0x7f8638109d80 2024-09-21T06:22:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.909+0000 7f8645bbb740 1 -- 172.21.15.3:0/3467670720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 msgr2=0x55f7053859b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.909+0000 7f863c90a640 1 --2- 172.21.15.3:0/378099822 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86240b6700 0x7f86240b8c10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8630031440 tx=0x7f8630009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.909+0000 7f8645bbb740 1 --2- 172.21.15.3:0/3467670720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f7053859b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8628011b00 tx=0x7f8628009990 comp rx=0 tx=0).stop 2024-09-21T06:22:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.909+0000 7f8645bbb740 1 -- 172.21.15.3:0/3467670720 shutdown_connections 2024-09-21T06:22:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.909+0000 7f8645bbb740 1 --2- 172.21.15.3:0/3467670720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 0x55f705386430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.909+0000 7f8645bbb740 1 --2- 172.21.15.3:0/3467670720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705385ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.909+0000 7f8645bbb740 1 --2- 172.21.15.3:0/3467670720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f7053859b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.909+0000 7f8645bbb740 1 -- 172.21.15.3:0/3467670720 >> 172.21.15.3:0/3467670720 conn(0x55f704fd4300 msgr2=0x55f7053838a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.909+0000 7f8645bbb740 1 -- 172.21.15.3:0/3467670720 shutdown_connections 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.909+0000 7f8645bbb740 1 -- 172.21.15.3:0/3467670720 wait complete. 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.909+0000 7f8645bbb740 1 -- start start 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.909+0000 7f8645bbb740 1 Processor -- start 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f8645bbb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fb72f0 0x55f705395400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f8645bbb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fbf4d0 0x55f705395940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f8645bbb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fe01b0 0x55f7053a1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f8645bbb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f705221ab0 con 0x55f704fe01b0 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f8645bbb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f70528f730 con 0x55f704fb72f0 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f8645bbb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f705398510 con 0x55f704fbf4d0 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f8645bbb740 10 client.0 _ll_register_callbacks cb 0x55f7052e36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f8637fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fbf4d0 0x55f705395940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f863d10b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fe01b0 0x55f7053a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f863c90a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fb72f0 0x55f705395400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f8637fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fbf4d0 0x55f705395940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:52834/0 (socket says 172.21.15.3:52834) 2024-09-21T06:22:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f8637fff640 1 -- 172.21.15.3:0/205792576 learned_addr learned my addr 172.21.15.3:0/205792576 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f863d10b640 1 -- 172.21.15.3:0/205792576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fbf4d0 msgr2=0x55f705395940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f863d10b640 1 --2- 172.21.15.3:0/205792576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fbf4d0 0x55f705395940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f863d10b640 1 -- 172.21.15.3:0/205792576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fb72f0 msgr2=0x55f705395400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f863d10b640 1 --2- 172.21.15.3:0/205792576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fb72f0 0x55f705395400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.910+0000 7f863d10b640 1 -- 172.21.15.3:0/205792576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f70504b790 con 0x55f704fe01b0 2024-09-21T06:22:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.911+0000 7f863d10b640 1 --2- 172.21.15.3:0/205792576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fe01b0 0x55f7053a1840 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f86200e5170 tx=0x7f86200b71c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e82c62c5d5dfda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.911+0000 7f8617fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.911+0000 7f8617fff640 1 -- 172.21.15.3:0/205792576 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86200b2020 con 0x55f704fe01b0 2024-09-21T06:22:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.911+0000 7f8645bbb740 1 -- 172.21.15.3:0/205792576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8628003e20 con 0x55f704fe01b0 2024-09-21T06:22:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.911+0000 7f8617fff640 1 -- 172.21.15.3:0/205792576 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f86200aeba0 con 0x55f704fe01b0 2024-09-21T06:22:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.911+0000 7f8617fff640 1 -- 172.21.15.3:0/205792576 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8620005280 con 0x55f704fe01b0 2024-09-21T06:22:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.911+0000 7f8645bbb740 10 client.17766 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.911+0000 7f8645bbb740 1 -- 172.21.15.3:0/205792576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f7050c4620 con 0x55f704fe01b0 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.911+0000 7f8617fff640 1 -- 172.21.15.3:0/205792576 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8628003e20 con 0x55f704fe01b0 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.911+0000 7f8617fff640 1 -- 172.21.15.3:0/205792576 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f86200b34e0 con 0x55f704fe01b0 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.911+0000 7f8645bbb740 10 client.17766 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.911+0000 7f8645bbb740 10 client.17766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.911+0000 7f8645bbb740 20 client.17766 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.912+0000 7f8645bbb740 1 --2- 172.21.15.3:0/205792576 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f704f9ed80 0x55f7053a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.912+0000 7f8645bbb740 4 client.17766 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.912+0000 7f8645bbb740 1 -- 172.21.15.3:0/205792576 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7052fc640 con 0x55f704f9ed80 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.912+0000 7f863c90a640 1 --2- 172.21.15.3:0/205792576 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f704f9ed80 0x55f7053a75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.912+0000 7f8617fff640 10 client.17766 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.912+0000 7f863c90a640 1 --2- 172.21.15.3:0/205792576 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f704f9ed80 0x55f7053a75b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b2de6777869cb9 server_cookie=56a40cd6fdff1a9e in_seq=0 out_seq=0 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8617fff640 1 -- 172.21.15.3:0/205792576 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f7052fc640 con 0x55f704f9ed80 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8617fff640 10 client.17766 handle_command_reply: tid=0 2024-09-21T06:22:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8645bbb740 1 client.17766 shutdown 2024-09-21T06:22:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8645bbb740 1 -- 172.21.15.3:0/205792576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fe01b0 msgr2=0x55f7053a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8645bbb740 1 --2- 172.21.15.3:0/205792576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fe01b0 0x55f7053a1840 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f86200e5170 tx=0x7f86200b71c0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8645bbb740 1 -- 172.21.15.3:0/205792576 shutdown_connections 2024-09-21T06:22:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8645bbb740 1 --2- 172.21.15.3:0/205792576 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f704f9ed80 0x55f7053a75b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8645bbb740 1 --2- 172.21.15.3:0/205792576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fbf4d0 0x55f705395940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8617fff640 0 client.17766 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8645bbb740 1 --2- 172.21.15.3:0/205792576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fe01b0 0x55f7053a1840 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8645bbb740 1 --2- 172.21.15.3:0/205792576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fb72f0 0x55f705395400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8645bbb740 1 -- 172.21.15.3:0/205792576 >> 172.21.15.3:0/205792576 conn(0x55f704fd4300 msgr2=0x55f7053873a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8645bbb740 1 -- 172.21.15.3:0/205792576 shutdown_connections 2024-09-21T06:22:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8645bbb740 1 -- 172.21.15.3:0/205792576 wait complete. 2024-09-21T06:22:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.913+0000 7f8645bbb740 20 client.17766 trim_cache size 0 max 16384 2024-09-21T06:22:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f8645bbb740 1 Processor -- start 2024-09-21T06:22:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f8645bbb740 1 -- start start 2024-09-21T06:22:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f8645bbb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705390ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f8645bbb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 0x55f7053a1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f8645bbb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f7053925d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f8645bbb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86381ad510 con 0x55f704fb72f0 2024-09-21T06:22:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f8645bbb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86381af050 con 0x55f704fbf4d0 2024-09-21T06:22:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f8645bbb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86381adbe0 con 0x55f704fe01b0 2024-09-21T06:22:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f863c90a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705390ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f863c90a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705390ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51994/0 (socket says 172.21.15.3:51994) 2024-09-21T06:22:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f863c90a640 1 -- 172.21.15.3:0/1582078168 learned_addr learned my addr 172.21.15.3:0/1582078168 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f863d10b640 1 --2- 172.21.15.3:0/1582078168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f7053925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f8637fff640 1 --2- 172.21.15.3:0/1582078168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 0x55f7053a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f863c90a640 1 -- 172.21.15.3:0/1582078168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 msgr2=0x55f7053925d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f863c90a640 1 --2- 172.21.15.3:0/1582078168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f7053925d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f863c90a640 1 -- 172.21.15.3:0/1582078168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 msgr2=0x55f7053a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f863c90a640 1 --2- 172.21.15.3:0/1582078168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 0x55f7053a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f863c90a640 1 -- 172.21.15.3:0/1582078168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7053867f0 con 0x55f704fb72f0 2024-09-21T06:22:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.929+0000 7f863d10b640 1 --2- 172.21.15.3:0/1582078168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f7053925d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.930+0000 7f863c90a640 1 --2- 172.21.15.3:0/1582078168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705390ab0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f863002a930 tx=0x7f863003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d32ee6ae0bb01b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.930+0000 7f862e7fc640 1 -- 172.21.15.3:0/1582078168 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630035040 con 0x55f704fb72f0 2024-09-21T06:22:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.930+0000 7f862e7fc640 1 -- 172.21.15.3:0/1582078168 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8630056030 con 0x55f704fb72f0 2024-09-21T06:22:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.930+0000 7f862e7fc640 1 -- 172.21.15.3:0/1582078168 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630004b40 con 0x55f704fb72f0 2024-09-21T06:22:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.931+0000 7f8645bbb740 1 -- 172.21.15.3:0/1582078168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 msgr2=0x55f705390ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.931+0000 7f8645bbb740 1 --2- 172.21.15.3:0/1582078168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705390ab0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f863002a930 tx=0x7f863003f000 comp rx=0 tx=0).stop 2024-09-21T06:22:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.931+0000 7f8645bbb740 1 -- 172.21.15.3:0/1582078168 shutdown_connections 2024-09-21T06:22:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.931+0000 7f8645bbb740 1 --2- 172.21.15.3:0/1582078168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f7053925d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.931+0000 7f8645bbb740 1 --2- 172.21.15.3:0/1582078168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 0x55f7053a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.931+0000 7f8645bbb740 1 --2- 172.21.15.3:0/1582078168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705390ab0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.931+0000 7f8645bbb740 1 -- 172.21.15.3:0/1582078168 >> 172.21.15.3:0/1582078168 conn(0x55f704fd4300 msgr2=0x55f705393c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.931+0000 7f8645bbb740 1 -- 172.21.15.3:0/1582078168 shutdown_connections 2024-09-21T06:22:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.931+0000 7f8645bbb740 1 -- 172.21.15.3:0/1582078168 wait complete. 2024-09-21T06:22:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.931+0000 7f8645bbb740 1 -- start start 2024-09-21T06:22:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.931+0000 7f8645bbb740 1 Processor -- start 2024-09-21T06:22:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f8645bbb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705399fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f8645bbb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 0x55f70539a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f8645bbb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f705410f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f8645bbb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86381ae980 con 0x55f704fb72f0 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f8645bbb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86381ae2b0 con 0x55f704fbf4d0 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f8645bbb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86381ab510 con 0x55f704fe01b0 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f8645bbb740 10 client.0 _ll_register_callbacks cb 0x7f8620009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f863c90a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705399fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f863c90a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705399fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51998/0 (socket says 172.21.15.3:51998) 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f863c90a640 1 -- 172.21.15.3:0/720987515 learned_addr learned my addr 172.21.15.3:0/720987515 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f863d10b640 1 --2- 172.21.15.3:0/720987515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f705410f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f8637fff640 1 --2- 172.21.15.3:0/720987515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 0x55f70539a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f863c90a640 1 -- 172.21.15.3:0/720987515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 msgr2=0x55f705410f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f863c90a640 1 --2- 172.21.15.3:0/720987515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f705410f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f863c90a640 1 -- 172.21.15.3:0/720987515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 msgr2=0x55f70539a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f863c90a640 1 --2- 172.21.15.3:0/720987515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 0x55f70539a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f863c90a640 1 -- 172.21.15.3:0/720987515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7054115b0 con 0x55f704fb72f0 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f863c90a640 1 --2- 172.21.15.3:0/720987515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705399fa0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8630031700 tx=0x7f8630031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82485da230d74802 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.932+0000 7f8617fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8617fff640 1 -- 172.21.15.3:0/720987515 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863006a060 con 0x55f704fb72f0 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8617fff640 1 -- 172.21.15.3:0/720987515 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8507+0+0 (secure 0 0 0) 0x7f86300048c0 con 0x55f704fb72f0 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8645bbb740 1 -- 172.21.15.3:0/720987515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8630056030 con 0x55f704fb72f0 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8617fff640 1 -- 172.21.15.3:0/720987515 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8630004ac0 con 0x55f704fb72f0 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8645bbb740 10 client.17772 fetch_fsmap learned FSMap version 35 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8645bbb740 1 -- 172.21.15.3:0/720987515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f705412820 con 0x55f704fb72f0 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8617fff640 1 -- 172.21.15.3:0/720987515 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8630056030 con 0x55f704fb72f0 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8617fff640 1 -- 172.21.15.3:0/720987515 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f863003d700 con 0x55f704fb72f0 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8645bbb740 10 client.17772 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8645bbb740 10 client.17772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8645bbb740 20 client.17772 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8645bbb740 1 --2- 172.21.15.3:0/720987515 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f704f741c0 0x55f705414fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8645bbb740 4 client.17772 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:22:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.933+0000 7f8645bbb740 1 -- 172.21.15.3:0/720987515 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7052f80f0 con 0x55f704f741c0 2024-09-21T06:22:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.934+0000 7f8637fff640 1 --2- 172.21.15.3:0/720987515 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f704f741c0 0x55f705414fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.934+0000 7f8617fff640 10 client.17772 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:49.934+0000 7f8637fff640 1 --2- 172.21.15.3:0/720987515 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f704f741c0 0x55f705414fe0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=872c89055cff80a4 server_cookie=7a470d1a34401ed5 in_seq=0 out_seq=0 2024-09-21T06:22:49.967 INFO:tasks.workunit.client.0.smithi003.stdout: 1 832337 11.12 MB/sec execute 262 sec latency 26.801 ms 2024-09-21T06:22:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8617fff640 1 -- 172.21.15.3:0/720987515 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7052f80f0 con 0x55f704f741c0 2024-09-21T06:22:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8617fff640 10 client.17772 handle_command_reply: tid=0 2024-09-21T06:22:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8645bbb740 1 client.17772 shutdown 2024-09-21T06:22:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8645bbb740 1 -- 172.21.15.3:0/720987515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 msgr2=0x55f705399fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8645bbb740 1 --2- 172.21.15.3:0/720987515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705399fa0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8630031700 tx=0x7f8630031730 comp rx=0 tx=0).stop 2024-09-21T06:22:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8645bbb740 1 -- 172.21.15.3:0/720987515 shutdown_connections 2024-09-21T06:22:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8645bbb740 1 --2- 172.21.15.3:0/720987515 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55f704f741c0 0x55f705414fe0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8645bbb740 1 --2- 172.21.15.3:0/720987515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f704fe01b0 0x55f705410f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8617fff640 0 client.17772 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8645bbb740 1 --2- 172.21.15.3:0/720987515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f704fbf4d0 0x55f70539a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8645bbb740 1 --2- 172.21.15.3:0/720987515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f704fb72f0 0x55f705399fa0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8645bbb740 1 -- 172.21.15.3:0/720987515 >> 172.21.15.3:0/720987515 conn(0x55f704fd4300 msgr2=0x55f7053869c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8645bbb740 1 -- 172.21.15.3:0/720987515 shutdown_connections 2024-09-21T06:22:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8645bbb740 1 -- 172.21.15.3:0/720987515 wait complete. 2024-09-21T06:22:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.101+0000 7f8645bbb740 20 client.17772 trim_cache size 0 max 16384 2024-09-21T06:22:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:22:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:22:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:22:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:22:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.102+0000 7f863eb6e640 1 -- 172.21.15.3:0/378099822 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86240b6700 msgr2=0x7f86240b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.102+0000 7f863eb6e640 1 --2- 172.21.15.3:0/378099822 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86240b6700 0x7f86240b8c10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8630031440 tx=0x7f8630009560 comp rx=0 tx=0).stop 2024-09-21T06:22:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.102+0000 7f863eb6e640 1 -- 172.21.15.3:0/378099822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638109d80 msgr2=0x7f863819fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.102+0000 7f863eb6e640 1 --2- 172.21.15.3:0/378099822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638109d80 0x7f863819fd30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f862000e970 tx=0x7f862000ed70 comp rx=0 tx=0).stop 2024-09-21T06:22:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.102+0000 7f863eb6e640 1 -- 172.21.15.3:0/378099822 shutdown_connections 2024-09-21T06:22:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.102+0000 7f863eb6e640 1 --2- 172.21.15.3:0/378099822 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f86240b6700 0x7f86240b8c10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.102+0000 7f863eb6e640 1 --2- 172.21.15.3:0/378099822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8638109430 0x7f863819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.102+0000 7f863eb6e640 1 --2- 172.21.15.3:0/378099822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638109d80 0x7f863819fd30 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.102+0000 7f863eb6e640 1 --2- 172.21.15.3:0/378099822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8638103b00 0x7f863819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.102+0000 7f863eb6e640 1 -- 172.21.15.3:0/378099822 >> 172.21.15.3:0/378099822 conn(0x7f86380ff500 msgr2=0x7f8638108a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.102+0000 7f863eb6e640 1 -- 172.21.15.3:0/378099822 shutdown_connections 2024-09-21T06:22:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:22:50.102+0000 7f863eb6e640 1 -- 172.21.15.3:0/378099822 wait complete. 2024-09-21T06:22:50.324 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:22:50.324 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:22:50.592 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:50.851 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:22:50 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e[62522]: ignoring --setuser ceph since I am not root 2024-09-21T06:22:50.852 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:22:50 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e[62522]: ignoring --setgroup ceph since I am not root 2024-09-21T06:22:50.852 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:22:50 smithi137 ceph-mds[62526]: -- 172.21.15.137:0/518530977 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5651498dd860 con 0x56514da85600 2024-09-21T06:22:50.852 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:22:50 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e[62522]: starting mds.e at 2024-09-21T06:22:50.967 INFO:tasks.workunit.client.0.smithi003.stdout: 1 834460 11.13 MB/sec execute 263 sec latency 81.476 ms 2024-09-21T06:22:51.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:50 smithi003 ceph-mon[31227]: pgmap v563: 97 pgs: 97 active+clean; 140 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 356 op/s 2024-09-21T06:22:51.017 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:50 smithi003 ceph-mon[31227]: daemon mds.e restarted 2024-09-21T06:22:51.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.115+0000 7fc76e9b5640 1 -- 172.21.15.3:0/911506591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc768108cf0 msgr2=0x7fc76810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.115+0000 7fc76e9b5640 1 --2- 172.21.15.3:0/911506591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc768108cf0 0x7fc76810f520 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc760009ca0 tx=0x7fc76002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.115+0000 7fc76e9b5640 1 -- 172.21.15.3:0/911506591 shutdown_connections 2024-09-21T06:22:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.115+0000 7fc76e9b5640 1 --2- 172.21.15.3:0/911506591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7681083a0 0x7fc7681087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.115+0000 7fc76e9b5640 1 --2- 172.21.15.3:0/911506591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc768108cf0 0x7fc76810f520 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.115+0000 7fc76e9b5640 1 --2- 172.21.15.3:0/911506591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc768102a60 0x7fc768102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.115+0000 7fc76e9b5640 1 -- 172.21.15.3:0/911506591 >> 172.21.15.3:0/911506591 conn(0x7fc768079730 msgr2=0x7fc768079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.115+0000 7fc76e9b5640 1 -- 172.21.15.3:0/911506591 shutdown_connections 2024-09-21T06:22:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.115+0000 7fc76e9b5640 1 -- 172.21.15.3:0/911506591 wait complete. 2024-09-21T06:22:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.116+0000 7fc76e9b5640 1 Processor -- start 2024-09-21T06:22:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.116+0000 7fc76e9b5640 1 -- start start 2024-09-21T06:22:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.116+0000 7fc76e9b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768102a60 0x7fc76819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.116+0000 7fc76e9b5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7681083a0 0x7fc76819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.116+0000 7fc76e9b5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc768108cf0 0x7fc76819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.116+0000 7fc76e9b5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc768113a80 con 0x7fc7681083a0 2024-09-21T06:22:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.116+0000 7fc76e9b5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc768113900 con 0x7fc768108cf0 2024-09-21T06:22:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.116+0000 7fc76e9b5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc768113c00 con 0x7fc768102a60 2024-09-21T06:22:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.116+0000 7fc7677fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7681083a0 0x7fc76819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.116+0000 7fc7677fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7681083a0 0x7fc76819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52008/0 (socket says 172.21.15.3:52008) 2024-09-21T06:22:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.116+0000 7fc7677fe640 1 -- 172.21.15.3:0/1234810564 learned_addr learned my addr 172.21.15.3:0/1234810564 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.116+0000 7fc767fff640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768102a60 0x7fc76819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.116+0000 7fc76cf52640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc768108cf0 0x7fc76819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.117+0000 7fc7677fe640 1 -- 172.21.15.3:0/1234810564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768102a60 msgr2=0x7fc76819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.117+0000 7fc7677fe640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768102a60 0x7fc76819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.117+0000 7fc7677fe640 1 -- 172.21.15.3:0/1234810564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc768108cf0 msgr2=0x7fc76819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.117+0000 7fc7677fe640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc768108cf0 0x7fc76819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.117+0000 7fc7677fe640 1 -- 172.21.15.3:0/1234810564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7681a0380 con 0x7fc7681083a0 2024-09-21T06:22:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.117+0000 7fc7677fe640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7681083a0 0x7fc76819a9b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00d740 tx=0x7fc75c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.117+0000 7fc7657fa640 1 -- 172.21.15.3:0/1234810564 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c017070 con 0x7fc7681083a0 2024-09-21T06:22:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.117+0000 7fc76e9b5640 1 -- 172.21.15.3:0/1234810564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7681a0670 con 0x7fc7681083a0 2024-09-21T06:22:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.117+0000 7fc7657fa640 1 -- 172.21.15.3:0/1234810564 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc75c00de50 con 0x7fc7681083a0 2024-09-21T06:22:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.117+0000 7fc7657fa640 1 -- 172.21.15.3:0/1234810564 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c004ce0 con 0x7fc7681083a0 2024-09-21T06:22:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.117+0000 7fc76e9b5640 1 -- 172.21.15.3:0/1234810564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7681a6dd0 con 0x7fc7681083a0 2024-09-21T06:22:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.120+0000 7fc774f29740 1 Processor -- start 2024-09-21T06:22:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.120+0000 7fc774f29740 1 -- start start 2024-09-21T06:22:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.120+0000 7fc774f29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b354350 0x55637b6fb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.120+0000 7fc774f29740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b32b490 0x55637b6fb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.120+0000 7fc774f29740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b333670 0x55637b6fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.120+0000 7fc774f29740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55637b671e60 con 0x55637b32b490 2024-09-21T06:22:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.120+0000 7fc774f29740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55637b661290 con 0x55637b333670 2024-09-21T06:22:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.120+0000 7fc774f29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55637b6650a0 con 0x55637b354350 2024-09-21T06:22:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.120+0000 7fc767fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b32b490 0x55637b6fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.120+0000 7fc76cf52640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b354350 0x55637b6fb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.120+0000 7fc76cf52640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b354350 0x55637b6fb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:52868/0 (socket says 172.21.15.3:52868) 2024-09-21T06:22:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.120+0000 7fc767fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b32b490 0x55637b6fb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52022/0 (socket says 172.21.15.3:52022) 2024-09-21T06:22:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.120+0000 7fc76cf52640 1 -- 172.21.15.3:0/894205139 learned_addr learned my addr 172.21.15.3:0/894205139 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.121+0000 7fc76cf52640 1 -- 172.21.15.3:0/894205139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b333670 msgr2=0x55637b6fbeb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:22:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.121+0000 7fc76cf52640 1 --2- 172.21.15.3:0/894205139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b333670 0x55637b6fbeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.121+0000 7fc76cf52640 1 -- 172.21.15.3:0/894205139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b32b490 msgr2=0x55637b6fb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.121+0000 7fc76cf52640 1 --2- 172.21.15.3:0/894205139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b32b490 0x55637b6fb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.121+0000 7fc76cf52640 1 -- 172.21.15.3:0/894205139 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637b3bf930 con 0x55637b354350 2024-09-21T06:22:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.121+0000 7fc7657fa640 1 -- 172.21.15.3:0/1234810564 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fc75c004e80 con 0x7fc7681083a0 2024-09-21T06:22:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.121+0000 7fc76cf52640 1 --2- 172.21.15.3:0/894205139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b354350 0x55637b6fb430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc76002f4c0 tx=0x7fc760009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=360001219c98051 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.121+0000 7fc74e7fc640 1 -- 172.21.15.3:0/894205139 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760031cf0 con 0x55637b354350 2024-09-21T06:22:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.121+0000 7fc74e7fc640 1 -- 172.21.15.3:0/894205139 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc760036030 con 0x55637b354350 2024-09-21T06:22:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.121+0000 7fc74e7fc640 1 -- 172.21.15.3:0/894205139 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7600042a0 con 0x55637b354350 2024-09-21T06:22:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.122+0000 7fc7657fa640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc7500b6680 0x7fc7500b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.122+0000 7fc767fff640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc7500b6680 0x7fc7500b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.122+0000 7fc7657fa640 1 -- 172.21.15.3:0/1234810564 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8463+0+0 (secure 0 0 0) 0x7fc75c0e7030 con 0x7fc7681083a0 2024-09-21T06:22:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.122+0000 7fc774f29740 1 -- 172.21.15.3:0/894205139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b354350 msgr2=0x55637b6fb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.122+0000 7fc767fff640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc7500b6680 0x7fc7500b8b90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55637b6fa0b0 tx=0x7fc758005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.122+0000 7fc774f29740 1 --2- 172.21.15.3:0/894205139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b354350 0x55637b6fb430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc76002f4c0 tx=0x7fc760009800 comp rx=0 tx=0).stop 2024-09-21T06:22:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.122+0000 7fc774f29740 1 -- 172.21.15.3:0/894205139 shutdown_connections 2024-09-21T06:22:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.122+0000 7fc774f29740 1 --2- 172.21.15.3:0/894205139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b333670 0x55637b6fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.122+0000 7fc774f29740 1 --2- 172.21.15.3:0/894205139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b32b490 0x55637b6fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.122+0000 7fc774f29740 1 --2- 172.21.15.3:0/894205139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b354350 0x55637b6fb430 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.122+0000 7fc774f29740 1 -- 172.21.15.3:0/894205139 >> 172.21.15.3:0/894205139 conn(0x55637b348740 msgr2=0x55637b6f9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.122+0000 7fc774f29740 1 -- 172.21.15.3:0/894205139 shutdown_connections 2024-09-21T06:22:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.123+0000 7fc774f29740 1 -- 172.21.15.3:0/894205139 wait complete. 2024-09-21T06:22:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.123+0000 7fc774f29740 1 -- start start 2024-09-21T06:22:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.123+0000 7fc774f29740 1 Processor -- start 2024-09-21T06:22:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.123+0000 7fc774f29740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b32b490 0x55637b707070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.123+0000 7fc774f29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b333670 0x55637b707630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.123+0000 7fc774f29740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b354350 0x55637b712d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.123+0000 7fc774f29740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55637b4c2890 con 0x55637b32b490 2024-09-21T06:22:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.123+0000 7fc774f29740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55637b5fbd40 con 0x55637b354350 2024-09-21T06:22:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.123+0000 7fc774f29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55637b709a30 con 0x55637b333670 2024-09-21T06:22:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc774f29740 10 client.0 _ll_register_callbacks cb 0x55637b65fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc767fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b32b490 0x55637b707070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc7677fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b333670 0x55637b707630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc76cf52640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b354350 0x55637b712d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc7677fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b333670 0x55637b707630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:52878/0 (socket says 172.21.15.3:52878) 2024-09-21T06:22:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc7677fe640 1 -- 172.21.15.3:0/729457856 learned_addr learned my addr 172.21.15.3:0/729457856 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc767fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b32b490 0x55637b707070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52026/0 (socket says 172.21.15.3:52026) 2024-09-21T06:22:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc7677fe640 1 -- 172.21.15.3:0/729457856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b354350 msgr2=0x55637b712d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc7677fe640 1 --2- 172.21.15.3:0/729457856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b354350 0x55637b712d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc7677fe640 1 -- 172.21.15.3:0/729457856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b32b490 msgr2=0x55637b707070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc7677fe640 1 --2- 172.21.15.3:0/729457856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b32b490 0x55637b707070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc7677fe640 1 -- 172.21.15.3:0/729457856 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637b713390 con 0x55637b333670 2024-09-21T06:22:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc76cf52640 1 --2- 172.21.15.3:0/729457856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b354350 0x55637b712d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc767fff640 1 --2- 172.21.15.3:0/729457856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b32b490 0x55637b707070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc7677fe640 1 --2- 172.21.15.3:0/729457856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b333670 0x55637b707630 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0a9540 tx=0x7fc75c0e4cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c56e2c107269d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc7477fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:22:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc7477fe640 1 -- 172.21.15.3:0/729457856 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c020070 con 0x55637b333670 2024-09-21T06:22:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc774f29740 1 -- 172.21.15.3:0/729457856 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc760036030 con 0x55637b333670 2024-09-21T06:22:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.124+0000 7fc7477fe640 1 -- 172.21.15.3:0/729457856 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8463+0+0 (secure 0 0 0) 0x7fc75c0942b0 con 0x55637b333670 2024-09-21T06:22:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.125+0000 7fc7477fe640 1 -- 172.21.15.3:0/729457856 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc75c0b68e0 con 0x55637b333670 2024-09-21T06:22:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.125+0000 7fc774f29740 10 client.27301 fetch_fsmap learned FSMap version 37 2024-09-21T06:22:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.125+0000 7fc774f29740 1 -- 172.21.15.3:0/729457856 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55637b715ca0 con 0x55637b333670 2024-09-21T06:22:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.125+0000 7fc7477fe640 1 -- 172.21.15.3:0/729457856 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc760036030 con 0x55637b333670 2024-09-21T06:22:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.125+0000 7fc7477fe640 1 -- 172.21.15.3:0/729457856 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fc75c0b1020 con 0x55637b333670 2024-09-21T06:22:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.125+0000 7fc774f29740 10 client.27301 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:22:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.125+0000 7fc774f29740 10 client.27301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.125+0000 7fc774f29740 20 client.27301 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.125+0000 7fc774f29740 1 --2- 172.21.15.3:0/729457856 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55637b312cd0 0x55637b719050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.125+0000 7fc774f29740 4 client.27301 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.125+0000 7fc774f29740 1 -- 172.21.15.3:0/729457856 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55637b673db0 con 0x55637b312cd0 2024-09-21T06:22:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.125+0000 7fc767fff640 1 --2- 172.21.15.3:0/729457856 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55637b312cd0 0x55637b719050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.126+0000 7fc767fff640 1 --2- 172.21.15.3:0/729457856 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55637b312cd0 0x55637b719050 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=909fd76899711d99 server_cookie=3f295a1d6b11e91d in_seq=0 out_seq=0 2024-09-21T06:22:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.126+0000 7fc7477fe640 10 client.27301 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.126+0000 7fc7477fe640 1 -- 172.21.15.3:0/729457856 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55637b673db0 con 0x55637b312cd0 2024-09-21T06:22:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.126+0000 7fc7477fe640 10 client.27301 handle_command_reply: tid=0 2024-09-21T06:22:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.126+0000 7fc774f29740 1 client.27301 shutdown 2024-09-21T06:22:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.127+0000 7fc774f29740 1 -- 172.21.15.3:0/729457856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b333670 msgr2=0x55637b707630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.127+0000 7fc774f29740 1 --2- 172.21.15.3:0/729457856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b333670 0x55637b707630 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0a9540 tx=0x7fc75c0e4cf0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.127+0000 7fc774f29740 1 -- 172.21.15.3:0/729457856 shutdown_connections 2024-09-21T06:22:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.127+0000 7fc774f29740 1 --2- 172.21.15.3:0/729457856 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55637b312cd0 0x55637b719050 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.127+0000 7fc774f29740 1 --2- 172.21.15.3:0/729457856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b333670 0x55637b707630 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.127+0000 7fc774f29740 1 --2- 172.21.15.3:0/729457856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b354350 0x55637b712d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.127+0000 7fc774f29740 1 --2- 172.21.15.3:0/729457856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b32b490 0x55637b707070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.127+0000 7fc7477fe640 0 client.27301 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.127+0000 7fc774f29740 1 -- 172.21.15.3:0/729457856 >> 172.21.15.3:0/729457856 conn(0x55637b348740 msgr2=0x55637b6fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.127+0000 7fc774f29740 1 -- 172.21.15.3:0/729457856 shutdown_connections 2024-09-21T06:22:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.127+0000 7fc774f29740 1 -- 172.21.15.3:0/729457856 wait complete. 2024-09-21T06:22:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.127+0000 7fc774f29740 20 client.27301 trim_cache size 0 max 16384 2024-09-21T06:22:51.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:50 smithi137 ceph-mon[34464]: pgmap v563: 97 pgs: 97 active+clean; 140 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 356 op/s 2024-09-21T06:22:51.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:50 smithi137 ceph-mon[34464]: daemon mds.e restarted 2024-09-21T06:22:51.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.145+0000 7fc774f29740 1 Processor -- start 2024-09-21T06:22:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc774f29740 1 -- start start 2024-09-21T06:22:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc774f29740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b312cd0 0x55637b700c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc774f29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b32b490 0x55637b712c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc774f29740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b333670 0x55637b706700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc774f29740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681af510 con 0x55637b333670 2024-09-21T06:22:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc774f29740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681b1050 con 0x55637b312cd0 2024-09-21T06:22:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc774f29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681afbe0 con 0x55637b32b490 2024-09-21T06:22:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc76cf52640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b333670 0x55637b706700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc767fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b312cd0 0x55637b700c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc767fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b312cd0 0x55637b700c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:39862/0 (socket says 172.21.15.3:39862) 2024-09-21T06:22:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc7677fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b32b490 0x55637b712c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc76cf52640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b333670 0x55637b706700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52030/0 (socket says 172.21.15.3:52030) 2024-09-21T06:22:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc767fff640 1 -- 172.21.15.3:0/1716628665 learned_addr learned my addr 172.21.15.3:0/1716628665 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc76cf52640 1 -- 172.21.15.3:0/1716628665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b32b490 msgr2=0x55637b712c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc76cf52640 1 --2- 172.21.15.3:0/1716628665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b32b490 0x55637b712c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc76cf52640 1 -- 172.21.15.3:0/1716628665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b312cd0 msgr2=0x55637b700c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc76cf52640 1 --2- 172.21.15.3:0/1716628665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b312cd0 0x55637b700c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc76cf52640 1 -- 172.21.15.3:0/1716628665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637b6fd720 con 0x55637b333670 2024-09-21T06:22:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.146+0000 7fc767fff640 1 --2- 172.21.15.3:0/1716628665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b312cd0 0x55637b700c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.147+0000 7fc76cf52640 1 --2- 172.21.15.3:0/1716628665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b333670 0x55637b706700 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc76002ade0 tx=0x7fc760003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfdd3e2a4cc75f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.147+0000 7fc74dffb640 1 -- 172.21.15.3:0/1716628665 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76004a040 con 0x55637b333670 2024-09-21T06:22:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.147+0000 7fc74dffb640 1 -- 172.21.15.3:0/1716628665 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc760036040 con 0x55637b333670 2024-09-21T06:22:51.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.147+0000 7fc74dffb640 1 -- 172.21.15.3:0/1716628665 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76002fb30 con 0x55637b333670 2024-09-21T06:22:51.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.148+0000 7fc774f29740 1 -- 172.21.15.3:0/1716628665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b333670 msgr2=0x55637b706700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.148+0000 7fc774f29740 1 --2- 172.21.15.3:0/1716628665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b333670 0x55637b706700 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc76002ade0 tx=0x7fc760003d70 comp rx=0 tx=0).stop 2024-09-21T06:22:51.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.148+0000 7fc774f29740 1 -- 172.21.15.3:0/1716628665 shutdown_connections 2024-09-21T06:22:51.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.148+0000 7fc774f29740 1 --2- 172.21.15.3:0/1716628665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b32b490 0x55637b712c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.148+0000 7fc774f29740 1 --2- 172.21.15.3:0/1716628665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b333670 0x55637b706700 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.148+0000 7fc774f29740 1 --2- 172.21.15.3:0/1716628665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b312cd0 0x55637b700c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.148+0000 7fc774f29740 1 -- 172.21.15.3:0/1716628665 >> 172.21.15.3:0/1716628665 conn(0x55637b348740 msgr2=0x55637b700540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:51.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.148+0000 7fc774f29740 1 -- 172.21.15.3:0/1716628665 shutdown_connections 2024-09-21T06:22:51.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.148+0000 7fc774f29740 1 -- 172.21.15.3:0/1716628665 wait complete. 2024-09-21T06:22:51.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.148+0000 7fc774f29740 1 -- start start 2024-09-21T06:22:51.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc774f29740 1 Processor -- start 2024-09-21T06:22:51.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc774f29740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b312cd0 0x55637b70aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc774f29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b32b490 0x55637b70b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc774f29740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b333670 0x55637b70b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc774f29740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681b0980 con 0x55637b312cd0 2024-09-21T06:22:51.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc774f29740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681b02b0 con 0x55637b333670 2024-09-21T06:22:51.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc774f29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681ad4f0 con 0x55637b32b490 2024-09-21T06:22:51.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc774f29740 10 client.0 _ll_register_callbacks cb 0x7fc75c004700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:51.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc767fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b312cd0 0x55637b70aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc767fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b312cd0 0x55637b70aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52040/0 (socket says 172.21.15.3:52040) 2024-09-21T06:22:51.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc76cf52640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b333670 0x55637b70b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc7677fe640 1 --2- 172.21.15.3:0/370387092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b32b490 0x55637b70b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc767fff640 1 -- 172.21.15.3:0/370387092 learned_addr learned my addr 172.21.15.3:0/370387092 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:51.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc767fff640 1 -- 172.21.15.3:0/370387092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b32b490 msgr2=0x55637b70b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc767fff640 1 --2- 172.21.15.3:0/370387092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b32b490 0x55637b70b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc767fff640 1 -- 172.21.15.3:0/370387092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b333670 msgr2=0x55637b70b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc767fff640 1 --2- 172.21.15.3:0/370387092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b333670 0x55637b70b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.149+0000 7fc767fff640 1 -- 172.21.15.3:0/370387092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637b786de0 con 0x55637b312cd0 2024-09-21T06:22:51.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.150+0000 7fc76cf52640 1 --2- 172.21.15.3:0/370387092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b333670 0x55637b70b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:51.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.150+0000 7fc767fff640 1 --2- 172.21.15.3:0/370387092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b312cd0 0x55637b70aba0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc758003710 tx=0x7fc7580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39986b6a25074aaa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:51.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.150+0000 7fc7477fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:51.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.150+0000 7fc7477fe640 1 -- 172.21.15.3:0/370387092 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758036060 con 0x55637b312cd0 2024-09-21T06:22:51.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.150+0000 7fc7477fe640 1 -- 172.21.15.3:0/370387092 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8463+0+0 (secure 0 0 0) 0x7fc758018030 con 0x55637b312cd0 2024-09-21T06:22:51.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.150+0000 7fc774f29740 1 -- 172.21.15.3:0/370387092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc760036040 con 0x55637b312cd0 2024-09-21T06:22:51.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.150+0000 7fc7477fe640 1 -- 172.21.15.3:0/370387092 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc758005020 con 0x55637b312cd0 2024-09-21T06:22:51.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.150+0000 7fc774f29740 10 client.17796 fetch_fsmap learned FSMap version 37 2024-09-21T06:22:51.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.150+0000 7fc774f29740 1 -- 172.21.15.3:0/370387092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55637b788070 con 0x55637b312cd0 2024-09-21T06:22:51.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.150+0000 7fc7477fe640 1 -- 172.21.15.3:0/370387092 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc760036040 con 0x55637b312cd0 2024-09-21T06:22:51.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.151+0000 7fc7477fe640 1 -- 172.21.15.3:0/370387092 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fc75801f430 con 0x55637b312cd0 2024-09-21T06:22:51.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.151+0000 7fc774f29740 10 client.17796 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:22:51.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.151+0000 7fc774f29740 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:51.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.151+0000 7fc774f29740 20 client.17796 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:51.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.151+0000 7fc774f29740 1 --2- 172.21.15.3:0/370387092 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55637b354350 0x55637b78a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.151+0000 7fc774f29740 4 client.17796 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:22:51.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.151+0000 7fc774f29740 1 -- 172.21.15.3:0/370387092 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55637b66f860 con 0x55637b354350 2024-09-21T06:22:51.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.151+0000 7fc7677fe640 1 --2- 172.21.15.3:0/370387092 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55637b354350 0x55637b78a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.151+0000 7fc7477fe640 10 client.17796 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:51.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.151+0000 7fc7677fe640 1 --2- 172.21.15.3:0/370387092 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55637b354350 0x55637b78a840 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7515f75395eaf8fb server_cookie=9e2107d61c917540 in_seq=0 out_seq=0 2024-09-21T06:22:51.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:50 smithi204 ceph-mon[34659]: pgmap v563: 97 pgs: 97 active+clean; 140 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 356 op/s 2024-09-21T06:22:51.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:50 smithi204 ceph-mon[34659]: daemon mds.e restarted 2024-09-21T06:22:51.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.153+0000 7fc7477fe640 1 -- 172.21.15.3:0/370387092 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55637b66f860 con 0x55637b354350 2024-09-21T06:22:51.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.153+0000 7fc7477fe640 10 client.17796 handle_command_reply: tid=0 2024-09-21T06:22:51.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.153+0000 7fc774f29740 1 client.17796 shutdown 2024-09-21T06:22:51.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.153+0000 7fc774f29740 1 -- 172.21.15.3:0/370387092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b312cd0 msgr2=0x55637b70aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.153+0000 7fc774f29740 1 --2- 172.21.15.3:0/370387092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b312cd0 0x55637b70aba0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc758003710 tx=0x7fc7580046c0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.154+0000 7fc774f29740 1 -- 172.21.15.3:0/370387092 shutdown_connections 2024-09-21T06:22:51.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.154+0000 7fc774f29740 1 --2- 172.21.15.3:0/370387092 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55637b354350 0x55637b78a840 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.154+0000 7fc774f29740 1 --2- 172.21.15.3:0/370387092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55637b32b490 0x55637b70b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.154+0000 7fc774f29740 1 --2- 172.21.15.3:0/370387092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55637b333670 0x55637b70b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.154+0000 7fc774f29740 1 --2- 172.21.15.3:0/370387092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55637b312cd0 0x55637b70aba0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.154+0000 7fc774f29740 1 -- 172.21.15.3:0/370387092 >> 172.21.15.3:0/370387092 conn(0x55637b348740 msgr2=0x55637b708320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:51.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.154+0000 7fc7477fe640 0 client.17796 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:51.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.154+0000 7fc774f29740 1 -- 172.21.15.3:0/370387092 shutdown_connections 2024-09-21T06:22:51.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.154+0000 7fc774f29740 1 -- 172.21.15.3:0/370387092 wait complete. 2024-09-21T06:22:51.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.154+0000 7fc774f29740 20 client.17796 trim_cache size 0 max 16384 2024-09-21T06:22:51.161 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:22:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:22:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "535fb164-fd4f-4d88-b7e0-575559314385", 2024-09-21T06:22:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:22:51.162 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:22:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.155+0000 7fc76e9b5640 1 -- 172.21.15.3:0/1234810564 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc7500b6680 msgr2=0x7fc7500b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.155+0000 7fc76e9b5640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc7500b6680 0x7fc7500b8b90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55637b6fa0b0 tx=0x7fc758005f90 comp rx=0 tx=0).stop 2024-09-21T06:22:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.155+0000 7fc76e9b5640 1 -- 172.21.15.3:0/1234810564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7681083a0 msgr2=0x7fc76819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.155+0000 7fc76e9b5640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7681083a0 0x7fc76819a9b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00d740 tx=0x7fc75c00dc00 comp rx=0 tx=0).stop 2024-09-21T06:22:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.155+0000 7fc76e9b5640 1 -- 172.21.15.3:0/1234810564 shutdown_connections 2024-09-21T06:22:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.155+0000 7fc76e9b5640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc7500b6680 0x7fc7500b8b90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.155+0000 7fc76e9b5640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc768108cf0 0x7fc76819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.155+0000 7fc76e9b5640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7681083a0 0x7fc76819a9b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.155+0000 7fc76e9b5640 1 --2- 172.21.15.3:0/1234810564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768102a60 0x7fc76819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.155+0000 7fc76e9b5640 1 -- 172.21.15.3:0/1234810564 >> 172.21.15.3:0/1234810564 conn(0x7fc768079730 msgr2=0x7fc768105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.155+0000 7fc76e9b5640 1 -- 172.21.15.3:0/1234810564 shutdown_connections 2024-09-21T06:22:51.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:51.155+0000 7fc76e9b5640 1 -- 172.21.15.3:0/1234810564 wait complete. 2024-09-21T06:22:51.967 INFO:tasks.workunit.client.0.smithi003.stdout: 1 835771 11.11 MB/sec execute 264 sec latency 34.691 ms 2024-09-21T06:22:52.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:51 smithi003 ceph-mon[31227]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:22:52.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:51 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.137:6834/3457686047,v1:172.21.15.137:6835/3457686047] up:boot 2024-09-21T06:22:52.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:51 smithi003 ceph-mon[31227]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T06:22:52.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:51 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:22:52.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:51 smithi003 ceph-mon[31227]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:22:52.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:51 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:22:52.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:51 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:22:52.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:51 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:52.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:51 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:52.061 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:22:52.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:51 smithi137 ceph-mon[34464]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:22:52.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:51 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.137:6834/3457686047,v1:172.21.15.137:6835/3457686047] up:boot 2024-09-21T06:22:52.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:51 smithi137 ceph-mon[34464]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T06:22:52.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:51 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:22:52.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:51 smithi137 ceph-mon[34464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:22:52.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:51 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:22:52.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:51 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:22:52.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:51 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:52.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:51 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:52.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:51 smithi204 ceph-mon[34659]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:22:52.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:51 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.137:6834/3457686047,v1:172.21.15.137:6835/3457686047] up:boot 2024-09-21T06:22:52.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:51 smithi204 ceph-mon[34659]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T06:22:52.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:51 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:22:52.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:51 smithi204 ceph-mon[34659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:22:52.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:51 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:22:52.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:51 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:22:52.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:51 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:52.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:51 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:52.341 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:52.967 INFO:tasks.workunit.client.0.smithi003.stdout: 1 837910 11.11 MB/sec execute 265 sec latency 104.930 ms 2024-09-21T06:22:53.116 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:52 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:22:53.116 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:52 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:53.116 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:52 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:22:53.116 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:52 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:53.117 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:52 smithi003 ceph-mon[31227]: pgmap v565: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 279 op/s 2024-09-21T06:22:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.106+0000 7febe7d6a640 1 -- 172.21.15.3:0/3260459436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe0103bc0 msgr2=0x7febe0103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.106+0000 7febe7d6a640 1 --2- 172.21.15.3:0/3260459436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe0103bc0 0x7febe0103fd0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7febd8009ca0 tx=0x7febd802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.106+0000 7febe7d6a640 1 -- 172.21.15.3:0/3260459436 shutdown_connections 2024-09-21T06:22:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.106+0000 7febe7d6a640 1 --2- 172.21.15.3:0/3260459436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febe01094b0 0x7febe01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.106+0000 7febe7d6a640 1 --2- 172.21.15.3:0/3260459436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febe0109e00 0x7febe0110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.106+0000 7febe7d6a640 1 --2- 172.21.15.3:0/3260459436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe0103bc0 0x7febe0103fd0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.106+0000 7febe7d6a640 1 -- 172.21.15.3:0/3260459436 >> 172.21.15.3:0/3260459436 conn(0x7febe007b910 msgr2=0x7febe007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.106+0000 7febe7d6a640 1 -- 172.21.15.3:0/3260459436 shutdown_connections 2024-09-21T06:22:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.106+0000 7febe7d6a640 1 -- 172.21.15.3:0/3260459436 wait complete. 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.107+0000 7febe7d6a640 1 Processor -- start 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.107+0000 7febe7d6a640 1 -- start start 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.107+0000 7febe7d6a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febe0103bc0 0x7febe019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.107+0000 7febe7d6a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febe01094b0 0x7febe019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.107+0000 7febe7d6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe0109e00 0x7febe019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.107+0000 7febe7d6a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7febe0113ac0 con 0x7febe0109e00 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.107+0000 7febe7d6a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febe0113940 con 0x7febe01094b0 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.107+0000 7febe7d6a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7febe0113c40 con 0x7febe0103bc0 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febe6307640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe0109e00 0x7febe019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febe6307640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe0109e00 0x7febe019fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52054/0 (socket says 172.21.15.3:52054) 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febe5b06640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febe0103bc0 0x7febe019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febe6307640 1 -- 172.21.15.3:0/2906359416 learned_addr learned my addr 172.21.15.3:0/2906359416 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febe5305640 1 --2- 172.21.15.3:0/2906359416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febe01094b0 0x7febe019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febe6307640 1 -- 172.21.15.3:0/2906359416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febe0103bc0 msgr2=0x7febe019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febe6307640 1 --2- 172.21.15.3:0/2906359416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febe0103bc0 0x7febe019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febe6307640 1 -- 172.21.15.3:0/2906359416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febe01094b0 msgr2=0x7febe019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febe6307640 1 --2- 172.21.15.3:0/2906359416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febe01094b0 0x7febe019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febe6307640 1 -- 172.21.15.3:0/2906359416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe01a0480 con 0x7febe0109e00 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febe6307640 1 --2- 172.21.15.3:0/2906359416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe0109e00 0x7febe019fd50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7febd000e970 tx=0x7febd000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febdeffd640 1 -- 172.21.15.3:0/2906359416 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd000ccc0 con 0x7febe0109e00 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febdeffd640 1 -- 172.21.15.3:0/2906359416 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd000ce60 con 0x7febe0109e00 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febe7d6a640 1 -- 172.21.15.3:0/2906359416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe01a0770 con 0x7febe0109e00 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febdeffd640 1 -- 172.21.15.3:0/2906359416 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd0010430 con 0x7febe0109e00 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.108+0000 7febe7d6a640 1 -- 172.21.15.3:0/2906359416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe01a6ed0 con 0x7febe0109e00 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.111+0000 7febdeffd640 1 -- 172.21.15.3:0/2906359416 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7febd0004ff0 con 0x7febe0109e00 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.111+0000 7febee2de740 1 Processor -- start 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.111+0000 7febee2de740 1 -- start start 2024-09-21T06:22:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febee2de740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20c4070 0x55b7c246b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febee2de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c246b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febee2de740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c246bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febee2de740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b7c23e1e50 con 0x55b7c20a3390 2024-09-21T06:22:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febee2de740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b7c23d1280 con 0x55b7c20c4070 2024-09-21T06:22:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febee2de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b7c23d5090 con 0x55b7c209b1b0 2024-09-21T06:22:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febe5b06640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c246b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febe5b06640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c246b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:52920/0 (socket says 172.21.15.3:52920) 2024-09-21T06:22:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febe5b06640 1 -- 172.21.15.3:0/2343438709 learned_addr learned my addr 172.21.15.3:0/2343438709 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febe5305640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20c4070 0x55b7c246b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febe5305640 1 --2- 172.21.15.3:0/2343438709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c246bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febdeffd640 1 --2- 172.21.15.3:0/2906359416 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7febc80b6680 0x7febc80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febe5b06640 1 -- 172.21.15.3:0/2343438709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20c4070 msgr2=0x55b7c246b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febe5b06640 1 --2- 172.21.15.3:0/2343438709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20c4070 0x55b7c246b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febe5b06640 1 -- 172.21.15.3:0/2343438709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 msgr2=0x55b7c246bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febe5b06640 1 --2- 172.21.15.3:0/2343438709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c246bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febe5b06640 1 -- 172.21.15.3:0/2343438709 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c212f650 con 0x55b7c209b1b0 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.112+0000 7febe6307640 1 --2- 172.21.15.3:0/2906359416 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7febc80b6680 0x7febc80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.113+0000 7febdeffd640 1 -- 172.21.15.3:0/2906359416 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8463+0+0 (secure 0 0 0) 0x7febd00e7ad0 con 0x7febe0109e00 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.113+0000 7febe5b06640 1 --2- 172.21.15.3:0/2343438709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c246b7b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7febd8035000 tx=0x7febd80093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac8e67de40535e5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.113+0000 7febc7fff640 1 -- 172.21.15.3:0/2343438709 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8031cf0 con 0x55b7c209b1b0 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.113+0000 7febc7fff640 1 -- 172.21.15.3:0/2343438709 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd8036070 con 0x55b7c209b1b0 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.113+0000 7febe6307640 1 --2- 172.21.15.3:0/2906359416 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7febc80b6680 0x7febc80b8b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7febd00e5310 tx=0x7febd00a21c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.113+0000 7febc7fff640 1 -- 172.21.15.3:0/2343438709 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8004260 con 0x55b7c209b1b0 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.114+0000 7febee2de740 1 -- 172.21.15.3:0/2343438709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 msgr2=0x55b7c246b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.114+0000 7febee2de740 1 --2- 172.21.15.3:0/2343438709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c246b7b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7febd8035000 tx=0x7febd80093f0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.114+0000 7febee2de740 1 -- 172.21.15.3:0/2343438709 shutdown_connections 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.114+0000 7febee2de740 1 --2- 172.21.15.3:0/2343438709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c246b7b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.114+0000 7febee2de740 1 --2- 172.21.15.3:0/2343438709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c246bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.114+0000 7febee2de740 1 --2- 172.21.15.3:0/2343438709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20c4070 0x55b7c246b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.114+0000 7febee2de740 1 -- 172.21.15.3:0/2343438709 >> 172.21.15.3:0/2343438709 conn(0x55b7c20b8460 msgr2=0x55b7c2469160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.114+0000 7febee2de740 1 -- 172.21.15.3:0/2343438709 shutdown_connections 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.114+0000 7febee2de740 1 -- 172.21.15.3:0/2343438709 wait complete. 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.115+0000 7febee2de740 1 -- start start 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.115+0000 7febee2de740 1 Processor -- start 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.115+0000 7febee2de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c247f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.115+0000 7febee2de740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c247ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.115+0000 7febee2de740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20c4070 0x55b7c2484f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.115+0000 7febee2de740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b7c2232700 con 0x55b7c20a3390 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.115+0000 7febee2de740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b7c236c110 con 0x55b7c20c4070 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.115+0000 7febee2de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b7c24799f0 con 0x55b7c209b1b0 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.115+0000 7febee2de740 10 client.0 _ll_register_callbacks cb 0x55b7c2385990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febe5b06640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c247ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febe5b06640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c247ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52074/0 (socket says 172.21.15.3:52074) 2024-09-21T06:22:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febe5305640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20c4070 0x55b7c2484f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febe5b06640 1 -- 172.21.15.3:0/4237277013 learned_addr learned my addr 172.21.15.3:0/4237277013 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febe5305640 1 --2- 172.21.15.3:0/4237277013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c247f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febe5b06640 1 -- 172.21.15.3:0/4237277013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 msgr2=0x55b7c247f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febe5b06640 1 --2- 172.21.15.3:0/4237277013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c247f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febe5b06640 1 -- 172.21.15.3:0/4237277013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20c4070 msgr2=0x55b7c2484f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febe5b06640 1 --2- 172.21.15.3:0/4237277013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20c4070 0x55b7c2484f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febe5b06640 1 -- 172.21.15.3:0/4237277013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c2399c40 con 0x55b7c20a3390 2024-09-21T06:22:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febe5b06640 1 --2- 172.21.15.3:0/4237277013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c247ff40 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7febd802a900 tx=0x7febd80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cc00dfdf6e57d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febc57fa640 1 -- 172.21.15.3:0/4237277013 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8003970 con 0x55b7c20a3390 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febc57fa640 1 -- 172.21.15.3:0/4237277013 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8463+0+0 (secure 0 0 0) 0x7febd8031450 con 0x55b7c20a3390 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.116+0000 7febee2de740 1 -- 172.21.15.3:0/4237277013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd8036070 con 0x55b7c20a3390 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.117+0000 7febc57fa640 1 -- 172.21.15.3:0/4237277013 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7febd8031650 con 0x55b7c20a3390 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.117+0000 7febee2de740 10 client.17808 fetch_fsmap learned FSMap version 37 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.117+0000 7febee2de740 1 -- 172.21.15.3:0/4237277013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b7c248deb0 con 0x55b7c20a3390 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.117+0000 7febc57fa640 1 -- 172.21.15.3:0/4237277013 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd8036070 con 0x55b7c20a3390 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.117+0000 7febc57fa640 1 -- 172.21.15.3:0/4237277013 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7febd802faa0 con 0x55b7c20a3390 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.117+0000 7febee2de740 10 client.17808 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.117+0000 7febee2de740 10 client.17808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.117+0000 7febee2de740 20 client.17808 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.117+0000 7febee2de740 1 --2- 172.21.15.3:0/4237277013 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b7c20829f0 0x55b7c2491210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.117+0000 7febee2de740 4 client.17808 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.117+0000 7febee2de740 1 -- 172.21.15.3:0/4237277013 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7c23e3da0 con 0x55b7c20829f0 2024-09-21T06:22:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.117+0000 7febe5305640 1 --2- 172.21.15.3:0/4237277013 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b7c20829f0 0x55b7c2491210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.118+0000 7febc57fa640 10 client.17808 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.118+0000 7febe5305640 1 --2- 172.21.15.3:0/4237277013 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b7c20829f0 0x55b7c2491210 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e8d6539cfc0a47 server_cookie=8f97ce6bb0850c68 in_seq=0 out_seq=0 2024-09-21T06:22:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febc57fa640 1 -- 172.21.15.3:0/4237277013 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b7c23e3da0 con 0x55b7c20829f0 2024-09-21T06:22:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febc57fa640 10 client.17808 handle_command_reply: tid=0 2024-09-21T06:22:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febee2de740 1 client.17808 shutdown 2024-09-21T06:22:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febee2de740 1 -- 172.21.15.3:0/4237277013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 msgr2=0x55b7c247ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febee2de740 1 --2- 172.21.15.3:0/4237277013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c247ff40 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7febd802a900 tx=0x7febd80095c0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febee2de740 1 -- 172.21.15.3:0/4237277013 shutdown_connections 2024-09-21T06:22:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febee2de740 1 --2- 172.21.15.3:0/4237277013 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b7c20829f0 0x55b7c2491210 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febee2de740 1 --2- 172.21.15.3:0/4237277013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20c4070 0x55b7c2484f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febee2de740 1 --2- 172.21.15.3:0/4237277013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c247ff40 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febc57fa640 0 client.17808 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febee2de740 1 --2- 172.21.15.3:0/4237277013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c247f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febee2de740 1 -- 172.21.15.3:0/4237277013 >> 172.21.15.3:0/4237277013 conn(0x55b7c20b8460 msgr2=0x55b7c246e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febee2de740 1 -- 172.21.15.3:0/4237277013 shutdown_connections 2024-09-21T06:22:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febee2de740 1 -- 172.21.15.3:0/4237277013 wait complete. 2024-09-21T06:22:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.119+0000 7febee2de740 20 client.17808 trim_cache size 0 max 16384 2024-09-21T06:22:53.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:52 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:22:53.147 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:52 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:53.147 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:52 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:22:53.147 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:52 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:53.147 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:52 smithi137 ceph-mon[34464]: pgmap v565: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 279 op/s 2024-09-21T06:22:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.138+0000 7febee2de740 1 Processor -- start 2024-09-21T06:22:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.138+0000 7febee2de740 1 -- start start 2024-09-21T06:22:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.138+0000 7febee2de740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20829f0 0x55b7c2480050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.138+0000 7febee2de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c2478d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.138+0000 7febee2de740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20a3390 0x55b7c2484ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febee2de740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7febe01ad420 con 0x55b7c20829f0 2024-09-21T06:22:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febee2de740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febe01aef60 con 0x55b7c20a3390 2024-09-21T06:22:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febee2de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7febe01adaf0 con 0x55b7c209b1b0 2024-09-21T06:22:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febe5305640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20829f0 0x55b7c2480050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febe5305640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20829f0 0x55b7c2480050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52084/0 (socket says 172.21.15.3:52084) 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febe5b06640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c2478d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febe5305640 1 -- 172.21.15.3:0/2906744452 learned_addr learned my addr 172.21.15.3:0/2906744452 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febe5305640 1 --2- 172.21.15.3:0/2906744452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20a3390 0x55b7c2484ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febe5b06640 1 -- 172.21.15.3:0/2906744452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20a3390 msgr2=0x55b7c2484ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febe5b06640 1 --2- 172.21.15.3:0/2906744452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20a3390 0x55b7c2484ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febe5b06640 1 -- 172.21.15.3:0/2906744452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20829f0 msgr2=0x55b7c2480050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febe5b06640 1 --2- 172.21.15.3:0/2906744452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20829f0 0x55b7c2480050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febe5b06640 1 -- 172.21.15.3:0/2906744452 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c247e6c0 con 0x55b7c209b1b0 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febe5b06640 1 --2- 172.21.15.3:0/2906744452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c2478d70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7febd802ade0 tx=0x7febd8009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e53f56abadde4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.139+0000 7febc77fe640 1 -- 172.21.15.3:0/2906744452 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8003970 con 0x55b7c209b1b0 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.140+0000 7febc77fe640 1 -- 172.21.15.3:0/2906744452 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd8036070 con 0x55b7c209b1b0 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.140+0000 7febc77fe640 1 -- 172.21.15.3:0/2906744452 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd80305c0 con 0x55b7c209b1b0 2024-09-21T06:22:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.141+0000 7febee2de740 1 -- 172.21.15.3:0/2906744452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 msgr2=0x55b7c2478d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.141+0000 7febee2de740 1 --2- 172.21.15.3:0/2906744452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c2478d70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7febd802ade0 tx=0x7febd8009360 comp rx=0 tx=0).stop 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.141+0000 7febee2de740 1 -- 172.21.15.3:0/2906744452 shutdown_connections 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.141+0000 7febee2de740 1 --2- 172.21.15.3:0/2906744452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c2478d70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.141+0000 7febee2de740 1 --2- 172.21.15.3:0/2906744452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20a3390 0x55b7c2484ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.141+0000 7febee2de740 1 --2- 172.21.15.3:0/2906744452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20829f0 0x55b7c2480050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.141+0000 7febee2de740 1 -- 172.21.15.3:0/2906744452 >> 172.21.15.3:0/2906744452 conn(0x55b7c20b8460 msgr2=0x55b7c247f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.141+0000 7febee2de740 1 -- 172.21.15.3:0/2906744452 shutdown_connections 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.141+0000 7febee2de740 1 -- 172.21.15.3:0/2906744452 wait complete. 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.141+0000 7febee2de740 1 -- start start 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.141+0000 7febee2de740 1 Processor -- start 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febee2de740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20829f0 0x55b7c248bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febee2de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c2487290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febee2de740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c24879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febee2de740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7febe01ae890 con 0x55b7c20a3390 2024-09-21T06:22:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febee2de740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febe01ae1c0 con 0x55b7c20829f0 2024-09-21T06:22:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febee2de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7febe01ab400 con 0x55b7c209b1b0 2024-09-21T06:22:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febee2de740 10 client.0 _ll_register_callbacks cb 0x7febd4009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febe5305640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20829f0 0x55b7c248bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febe5305640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c24879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febe5b06640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c2487290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febe5305640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c24879a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52090/0 (socket says 172.21.15.3:52090) 2024-09-21T06:22:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febe5b06640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c2487290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:52946/0 (socket says 172.21.15.3:52946) 2024-09-21T06:22:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febe5305640 1 -- 172.21.15.3:0/1938851410 learned_addr learned my addr 172.21.15.3:0/1938851410 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:22:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febe5305640 1 -- 172.21.15.3:0/1938851410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 msgr2=0x55b7c2487290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febe5305640 1 --2- 172.21.15.3:0/1938851410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c2487290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febe5305640 1 -- 172.21.15.3:0/1938851410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20829f0 msgr2=0x55b7c248bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febe5305640 1 --2- 172.21.15.3:0/1938851410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20829f0 0x55b7c248bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.142+0000 7febe5305640 1 -- 172.21.15.3:0/1938851410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c24fa980 con 0x55b7c20a3390 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.143+0000 7febe5b06640 1 --2- 172.21.15.3:0/1938851410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c2487290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.143+0000 7febe5305640 1 --2- 172.21.15.3:0/1938851410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c24879a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7febe006bb30 tx=0x7febd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb24d28c70658f44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.143+0000 7febc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.143+0000 7febc57fa640 1 -- 172.21.15.3:0/1938851410 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4003c00 con 0x55b7c20a3390 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.143+0000 7febee2de740 1 -- 172.21.15.3:0/1938851410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd8036070 con 0x55b7c20a3390 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.143+0000 7febc57fa640 1 -- 172.21.15.3:0/1938851410 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8463+0+0 (secure 0 0 0) 0x7febd401a040 con 0x55b7c20a3390 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.143+0000 7febc57fa640 1 -- 172.21.15.3:0/1938851410 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7febd4003da0 con 0x55b7c20a3390 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.143+0000 7febee2de740 10 client.17814 fetch_fsmap learned FSMap version 37 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.143+0000 7febee2de740 1 -- 172.21.15.3:0/1938851410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b7c24fbb90 con 0x55b7c20a3390 2024-09-21T06:22:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.143+0000 7febc57fa640 1 -- 172.21.15.3:0/1938851410 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd8036070 con 0x55b7c20a3390 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.144+0000 7febc57fa640 1 -- 172.21.15.3:0/1938851410 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7febd4020040 con 0x55b7c20a3390 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.144+0000 7febee2de740 10 client.17814 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.144+0000 7febee2de740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.144+0000 7febee2de740 20 client.17814 populate_metadata read hostname 'smithi003' 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.144+0000 7febee2de740 1 --2- 172.21.15.3:0/1938851410 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b7c20c4070 0x55b7c24fe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.144+0000 7febee2de740 4 client.17814 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.144+0000 7febee2de740 1 -- 172.21.15.3:0/1938851410 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7c23df850 con 0x55b7c20c4070 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.144+0000 7febe5b06640 1 --2- 172.21.15.3:0/1938851410 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b7c20c4070 0x55b7c24fe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.144+0000 7febc57fa640 10 client.17814 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.144+0000 7febe5b06640 1 --2- 172.21.15.3:0/1938851410 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b7c20c4070 0x55b7c24fe360 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53e3f743e7ac2df server_cookie=943977d6c0f20cbc in_seq=0 out_seq=0 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febc57fa640 1 -- 172.21.15.3:0/1938851410 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x55b7c23df850 con 0x55b7c20c4070 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febc57fa640 10 client.17814 handle_command_reply: tid=0 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febee2de740 1 client.17814 shutdown 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febee2de740 1 -- 172.21.15.3:0/1938851410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 msgr2=0x55b7c24879a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:53.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febee2de740 1 --2- 172.21.15.3:0/1938851410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c24879a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7febe006bb30 tx=0x7febd4004730 comp rx=0 tx=0).stop 2024-09-21T06:22:53.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febee2de740 1 -- 172.21.15.3:0/1938851410 shutdown_connections 2024-09-21T06:22:53.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febee2de740 1 --2- 172.21.15.3:0/1938851410 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b7c20c4070 0x55b7c24fe360 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febee2de740 1 --2- 172.21.15.3:0/1938851410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7c209b1b0 0x55b7c2487290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febee2de740 1 --2- 172.21.15.3:0/1938851410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7c20a3390 0x55b7c24879a0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febc57fa640 0 client.17814 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:22:53.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febee2de740 1 --2- 172.21.15.3:0/1938851410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b7c20829f0 0x55b7c248bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febee2de740 1 -- 172.21.15.3:0/1938851410 >> 172.21.15.3:0/1938851410 conn(0x55b7c20b8460 msgr2=0x55b7c2468a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:53.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febee2de740 1 -- 172.21.15.3:0/1938851410 shutdown_connections 2024-09-21T06:22:53.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febee2de740 1 -- 172.21.15.3:0/1938851410 wait complete. 2024-09-21T06:22:53.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.145+0000 7febee2de740 20 client.17814 trim_cache size 0 max 16384 2024-09-21T06:22:53.157 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "535fb164-fd4f-4d88-b7e0-575559314385": { 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "535fb164-fd4f-4d88-b7e0-575559314385", 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "535fb164-fd4f-4d88-b7e0-575559314385": { 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:51.152572+0000", 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:22:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "714b9934-a2d3-4a78-b185-01254776061b": { 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:04.685380+0000", 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58": { 2024-09-21T06:22:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:21:17.396219+0000", 2024-09-21T06:22:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:22:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:22:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:22:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:22:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:22:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:22:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:22:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:22:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:22:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:22:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:22:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:22:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:22:53.166 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:22:53.166 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:22:53.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:52 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:22:53.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:52 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:53.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:52 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:22:53.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:52 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:53.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:52 smithi204 ceph-mon[34659]: pgmap v565: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 279 op/s 2024-09-21T06:22:53.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.146+0000 7febe7d6a640 1 -- 172.21.15.3:0/2906359416 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7febc80b6680 msgr2=0x7febc80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:53.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.146+0000 7febe7d6a640 1 --2- 172.21.15.3:0/2906359416 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7febc80b6680 0x7febc80b8b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7febd00e5310 tx=0x7febd00a21c0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.146+0000 7febe7d6a640 1 -- 172.21.15.3:0/2906359416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe0109e00 msgr2=0x7febe019fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:53.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.146+0000 7febe7d6a640 1 --2- 172.21.15.3:0/2906359416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe0109e00 0x7febe019fd50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7febd000e970 tx=0x7febd000ed70 comp rx=0 tx=0).stop 2024-09-21T06:22:53.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.146+0000 7febe7d6a640 1 -- 172.21.15.3:0/2906359416 shutdown_connections 2024-09-21T06:22:53.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.146+0000 7febe7d6a640 1 --2- 172.21.15.3:0/2906359416 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7febc80b6680 0x7febc80b8b90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.146+0000 7febe7d6a640 1 --2- 172.21.15.3:0/2906359416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe0109e00 0x7febe019fd50 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.146+0000 7febe7d6a640 1 --2- 172.21.15.3:0/2906359416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febe01094b0 0x7febe019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.146+0000 7febe7d6a640 1 --2- 172.21.15.3:0/2906359416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febe0103bc0 0x7febe019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:53.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.146+0000 7febe7d6a640 1 -- 172.21.15.3:0/2906359416 >> 172.21.15.3:0/2906359416 conn(0x7febe007b910 msgr2=0x7febe01075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:53.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.146+0000 7febe7d6a640 1 -- 172.21.15.3:0/2906359416 shutdown_connections 2024-09-21T06:22:53.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:22:53.146+0000 7febe7d6a640 1 -- 172.21.15.3:0/2906359416 wait complete. 2024-09-21T06:22:53.967 INFO:tasks.workunit.client.0.smithi003.stdout: 1 839332 11.10 MB/sec execute 266 sec latency 96.443 ms 2024-09-21T06:22:54.403 INFO:tasks.cephfs.filesystem:scrub status for tag:535fb164-fd4f-4d88-b7e0-575559314385 - {'path': '/', 'tag': '535fb164-fd4f-4d88-b7e0-575559314385', 'options': 'recursive,force'} 2024-09-21T06:22:54.967 INFO:tasks.workunit.client.0.smithi003.stdout: 1 841668 11.11 MB/sec execute 267 sec latency 37.942 ms 2024-09-21T06:22:55.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:54 smithi137 ceph-mon[34464]: pgmap v566: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 297 op/s 2024-09-21T06:22:55.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:54 smithi204 ceph-mon[34659]: pgmap v566: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 297 op/s 2024-09-21T06:22:55.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:54 smithi003 ceph-mon[31227]: pgmap v566: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 297 op/s 2024-09-21T06:22:55.967 INFO:tasks.workunit.client.0.smithi003.stdout: 1 843107 11.09 MB/sec execute 268 sec latency 15.665 ms 2024-09-21T06:22:56.967 INFO:tasks.workunit.client.0.smithi003.stdout: 1 845314 11.10 MB/sec execute 269 sec latency 90.174 ms 2024-09-21T06:22:57.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:56 smithi137 ceph-mon[34464]: pgmap v567: 97 pgs: 97 active+clean; 126 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 359 op/s 2024-09-21T06:22:57.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:56 smithi204 ceph-mon[34659]: pgmap v567: 97 pgs: 97 active+clean; 126 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 359 op/s 2024-09-21T06:22:57.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:56 smithi003 ceph-mon[31227]: pgmap v567: 97 pgs: 97 active+clean; 126 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 359 op/s 2024-09-21T06:22:57.967 INFO:tasks.workunit.client.0.smithi003.stdout: 1 846688 11.08 MB/sec execute 270 sec latency 62.172 ms 2024-09-21T06:22:58.967 INFO:tasks.workunit.client.0.smithi003.stdout: 1 848858 11.08 MB/sec execute 271 sec latency 41.953 ms 2024-09-21T06:22:59.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:22:58 smithi137 ceph-mon[34464]: pgmap v568: 97 pgs: 97 active+clean; 127 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-09-21T06:22:59.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:22:58 smithi204 ceph-mon[34659]: pgmap v568: 97 pgs: 97 active+clean; 127 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-09-21T06:22:59.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:22:58 smithi003 ceph-mon[31227]: pgmap v568: 97 pgs: 97 active+clean; 127 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-09-21T06:22:59.967 INFO:tasks.workunit.client.0.smithi003.stdout: 1 850343 11.06 MB/sec execute 272 sec latency 59.502 ms 2024-09-21T06:23:00.967 INFO:tasks.workunit.client.0.smithi003.stdout: 1 852470 11.07 MB/sec execute 273 sec latency 165.814 ms 2024-09-21T06:23:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:01 smithi137 ceph-mon[34464]: pgmap v569: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 351 op/s 2024-09-21T06:23:01.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:01 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:23:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:01 smithi204 ceph-mon[34659]: pgmap v569: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 351 op/s 2024-09-21T06:23:01.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:01 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:23:01.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:01 smithi003 ceph-mon[31227]: pgmap v569: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 351 op/s 2024-09-21T06:23:01.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:01 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:23:01.967 INFO:tasks.workunit.client.0.smithi003.stdout: 1 852470 11.03 MB/sec execute 274 sec latency 1165.898 ms 2024-09-21T06:23:02.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:02 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:23:02.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:02 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:23:02.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:02 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:23:02.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:02 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:23:02.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:02 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:23:02.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:02 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:23:02.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 852470 10.99 MB/sec execute 275 sec latency 2166.030 ms 2024-09-21T06:23:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.219+0000 7fcc3892b640 1 -- 172.21.15.3:0/285775473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc341029e0 msgr2=0x7fcc34102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.219+0000 7fcc3892b640 1 --2- 172.21.15.3:0/285775473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc341029e0 0x7fcc34102df0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009900 tx=0x7fcc2402a410 comp rx=0 tx=0).stop 2024-09-21T06:23:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.219+0000 7fcc3892b640 1 -- 172.21.15.3:0/285775473 shutdown_connections 2024-09-21T06:23:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.219+0000 7fcc3892b640 1 --2- 172.21.15.3:0/285775473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc34108310 0x7fcc34108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.219+0000 7fcc3892b640 1 --2- 172.21.15.3:0/285775473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc34108c60 0x7fcc3410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.219+0000 7fcc3892b640 1 --2- 172.21.15.3:0/285775473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc341029e0 0x7fcc34102df0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.219+0000 7fcc3892b640 1 -- 172.21.15.3:0/285775473 >> 172.21.15.3:0/285775473 conn(0x7fcc340fe3e0 msgr2=0x7fcc34100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.219+0000 7fcc3892b640 1 -- 172.21.15.3:0/285775473 shutdown_connections 2024-09-21T06:23:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.219+0000 7fcc3892b640 1 -- 172.21.15.3:0/285775473 wait complete. 2024-09-21T06:23:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.220+0000 7fcc3892b640 1 Processor -- start 2024-09-21T06:23:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.220+0000 7fcc3892b640 1 -- start start 2024-09-21T06:23:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.220+0000 7fcc3892b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc341029e0 0x7fcc340fe910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.220+0000 7fcc3892b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc34108310 0x7fcc340fee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.220+0000 7fcc3892b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc34108c60 0x7fcc340ff390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.220+0000 7fcc3892b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc34112920 con 0x7fcc34108c60 2024-09-21T06:23:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.220+0000 7fcc3892b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc341127a0 con 0x7fcc341029e0 2024-09-21T06:23:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.220+0000 7fcc3892b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc34112aa0 con 0x7fcc34108310 2024-09-21T06:23:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc32d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc34108c60 0x7fcc340ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc32d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc34108c60 0x7fcc340ff390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41926/0 (socket says 172.21.15.3:41926) 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc32d9d640 1 -- 172.21.15.3:0/3904063192 learned_addr learned my addr 172.21.15.3:0/3904063192 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc3259c640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc341029e0 0x7fcc340fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc34108310 0x7fcc340fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc32d9d640 1 -- 172.21.15.3:0/3904063192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc34108310 msgr2=0x7fcc340fee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc32d9d640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc34108310 0x7fcc340fee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc32d9d640 1 -- 172.21.15.3:0/3904063192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc341029e0 msgr2=0x7fcc340fe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc32d9d640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc341029e0 0x7fcc340fe910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc32d9d640 1 -- 172.21.15.3:0/3904063192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc341a86c0 con 0x7fcc34108c60 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc32d9d640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc34108c60 0x7fcc340ff390 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00cce0 tx=0x7fcc2c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc237fe640 1 -- 172.21.15.3:0/3904063192 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c00f450 con 0x7fcc34108c60 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc3892b640 1 -- 172.21.15.3:0/3904063192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc341a89b0 con 0x7fcc34108c60 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc237fe640 1 -- 172.21.15.3:0/3904063192 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc2c00faa0 con 0x7fcc34108c60 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.221+0000 7fcc237fe640 1 -- 172.21.15.3:0/3904063192 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c018b70 con 0x7fcc34108c60 2024-09-21T06:23:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.222+0000 7fcc3892b640 1 -- 172.21.15.3:0/3904063192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc341a8f00 con 0x7fcc34108c60 2024-09-21T06:23:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.224+0000 7fcc3f896740 1 Processor -- start 2024-09-21T06:23:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.224+0000 7fcc237fe640 1 -- 172.21.15.3:0/3904063192 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fcc2c00f5f0 con 0x7fcc34108c60 2024-09-21T06:23:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.224+0000 7fcc3f896740 1 -- start start 2024-09-21T06:23:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3f896740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b07301d1b0 0x55b0733c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3f896740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ff42f0 0x55b0733c2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3f896740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ffc4d0 0x55b0733c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3f896740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b0733376f0 con 0x55b072ff42f0 2024-09-21T06:23:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3f896740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b073326b20 con 0x55b07301d1b0 2024-09-21T06:23:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3f896740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b07332a930 con 0x55b072ffc4d0 2024-09-21T06:23:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ff42f0 0x55b0733c2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ff42f0 0x55b0733c2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41928/0 (socket says 172.21.15.3:41928) 2024-09-21T06:23:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3259c640 1 -- 172.21.15.3:0/2179626661 learned_addr learned my addr 172.21.15.3:0/2179626661 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/2179626661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b07301d1b0 0x55b0733c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc237fe640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc1c0b6680 0x7fcc1c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/2179626661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ffc4d0 0x55b0733c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc32d9d640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc1c0b6680 0x7fcc1c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3259c640 1 -- 172.21.15.3:0/2179626661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ffc4d0 msgr2=0x55b0733c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3259c640 1 --2- 172.21.15.3:0/2179626661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ffc4d0 0x55b0733c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3259c640 1 -- 172.21.15.3:0/2179626661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b07301d1b0 msgr2=0x55b0733c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3259c640 1 --2- 172.21.15.3:0/2179626661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b07301d1b0 0x55b0733c29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc3259c640 1 -- 172.21.15.3:0/2179626661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0732ec530 con 0x55b072ff42f0 2024-09-21T06:23:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.225+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/2179626661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ffc4d0 0x55b0733c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.226+0000 7fcc237fe640 1 -- 172.21.15.3:0/3904063192 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8463+0+0 (secure 0 0 0) 0x7fcc2c0e7a00 con 0x7fcc34108c60 2024-09-21T06:23:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.226+0000 7fcc3259c640 1 --2- 172.21.15.3:0/2179626661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ff42f0 0x55b0733c2ef0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009900 tx=0x7fcc24032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e623cf2cd9e2a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.226+0000 7fcc32d9d640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc1c0b6680 0x7fcc1c0b8b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0e5240 tx=0x7fcc2c07a540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.226+0000 7fcc20ff9640 1 -- 172.21.15.3:0/2179626661 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24040070 con 0x55b072ff42f0 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.226+0000 7fcc20ff9640 1 -- 172.21.15.3:0/2179626661 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc24005db0 con 0x55b072ff42f0 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.226+0000 7fcc20ff9640 1 -- 172.21.15.3:0/2179626661 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24035040 con 0x55b072ff42f0 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.227+0000 7fcc3f896740 1 -- 172.21.15.3:0/2179626661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ff42f0 msgr2=0x55b0733c2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.227+0000 7fcc3f896740 1 --2- 172.21.15.3:0/2179626661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ff42f0 0x55b0733c2ef0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009900 tx=0x7fcc24032df0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.227+0000 7fcc3f896740 1 -- 172.21.15.3:0/2179626661 shutdown_connections 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.227+0000 7fcc3f896740 1 --2- 172.21.15.3:0/2179626661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ffc4d0 0x55b0733c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.227+0000 7fcc3f896740 1 --2- 172.21.15.3:0/2179626661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ff42f0 0x55b0733c2ef0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.227+0000 7fcc3f896740 1 --2- 172.21.15.3:0/2179626661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b07301d1b0 0x55b0733c29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.227+0000 7fcc3f896740 1 -- 172.21.15.3:0/2179626661 >> 172.21.15.3:0/2179626661 conn(0x55b073011300 msgr2=0x55b0733c08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.227+0000 7fcc3f896740 1 -- 172.21.15.3:0/2179626661 shutdown_connections 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.227+0000 7fcc3f896740 1 -- 172.21.15.3:0/2179626661 wait complete. 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.227+0000 7fcc3f896740 1 -- start start 2024-09-21T06:23:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.227+0000 7fcc3f896740 1 Processor -- start 2024-09-21T06:23:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc3f896740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ff42f0 0x55b0733ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc3f896740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ffc4d0 0x55b0733ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc3f896740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 0x55b0733da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc3f896740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b07325eab0 con 0x55b07301d1b0 2024-09-21T06:23:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc3f896740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b0732cc730 con 0x55b072ffc4d0 2024-09-21T06:23:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc3f896740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b0733d10d0 con 0x55b072ff42f0 2024-09-21T06:23:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc3f896740 10 client.0 _ll_register_callbacks cb 0x55b0733206b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 0x55b0733da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 0x55b0733da3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41934/0 (socket says 172.21.15.3:41934) 2024-09-21T06:23:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc31d9b640 1 -- 172.21.15.3:0/1967433330 learned_addr learned my addr 172.21.15.3:0/1967433330 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/1967433330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ff42f0 0x55b0733ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc3259c640 1 --2- 172.21.15.3:0/1967433330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ffc4d0 0x55b0733ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc31d9b640 1 -- 172.21.15.3:0/1967433330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ff42f0 msgr2=0x55b0733ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/1967433330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ff42f0 0x55b0733ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc31d9b640 1 -- 172.21.15.3:0/1967433330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ffc4d0 msgr2=0x55b0733ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/1967433330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ffc4d0 0x55b0733ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc31d9b640 1 -- 172.21.15.3:0/1967433330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b073088790 con 0x55b07301d1b0 2024-09-21T06:23:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/1967433330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 0x55b0733da3c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fcc3406b800 tx=0x7fcc28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b2de3b79707bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.228+0000 7fcc19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc19ffb640 1 -- 172.21.15.3:0/1967433330 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2801b040 con 0x55b07301d1b0 2024-09-21T06:23:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc19ffb640 1 -- 172.21.15.3:0/1967433330 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8463+0+0 (secure 0 0 0) 0x7fcc28012030 con 0x55b07301d1b0 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc3f896740 1 -- 172.21.15.3:0/1967433330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc24005db0 con 0x55b07301d1b0 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc19ffb640 1 -- 172.21.15.3:0/1967433330 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcc28029050 con 0x55b07301d1b0 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc3f896740 10 client.17826 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc3f896740 1 -- 172.21.15.3:0/1967433330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b073101620 con 0x55b07301d1b0 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc19ffb640 1 -- 172.21.15.3:0/1967433330 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc24005db0 con 0x55b07301d1b0 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc19ffb640 1 -- 172.21.15.3:0/1967433330 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fcc280037f0 con 0x55b07301d1b0 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc3f896740 10 client.17826 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc3f896740 10 client.17826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc3f896740 20 client.17826 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc3f896740 1 --2- 172.21.15.3:0/1967433330 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b072fdbd80 0x55b0733e00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc3f896740 4 client.17826 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.229+0000 7fcc3f896740 1 -- 172.21.15.3:0/1967433330 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b073339640 con 0x55b072fdbd80 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.230+0000 7fcc3259c640 1 --2- 172.21.15.3:0/1967433330 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b072fdbd80 0x55b0733e00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.230+0000 7fcc3259c640 1 --2- 172.21.15.3:0/1967433330 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b072fdbd80 0x55b0733e00f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4266b9f642dbedd1 server_cookie=23a6a8a11677c968 in_seq=0 out_seq=0 2024-09-21T06:23:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.230+0000 7fcc19ffb640 10 client.17826 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc19ffb640 1 -- 172.21.15.3:0/1967433330 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b073339640 con 0x55b072fdbd80 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc19ffb640 10 client.17826 handle_command_reply: tid=0 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc3f896740 1 client.17826 shutdown 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc3f896740 1 -- 172.21.15.3:0/1967433330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 msgr2=0x55b0733da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc3f896740 1 --2- 172.21.15.3:0/1967433330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 0x55b0733da3c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fcc3406b800 tx=0x7fcc28004730 comp rx=0 tx=0).stop 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc3f896740 1 -- 172.21.15.3:0/1967433330 shutdown_connections 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc3f896740 1 --2- 172.21.15.3:0/1967433330 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b072fdbd80 0x55b0733e00f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc3f896740 1 --2- 172.21.15.3:0/1967433330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 0x55b0733da3c0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc3f896740 1 --2- 172.21.15.3:0/1967433330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ffc4d0 0x55b0733ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc19ffb640 0 client.17826 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc3f896740 1 --2- 172.21.15.3:0/1967433330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ff42f0 0x55b0733ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc3f896740 1 -- 172.21.15.3:0/1967433330 >> 172.21.15.3:0/1967433330 conn(0x55b073011300 msgr2=0x55b0733c39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc3f896740 1 -- 172.21.15.3:0/1967433330 shutdown_connections 2024-09-21T06:23:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc3f896740 1 -- 172.21.15.3:0/1967433330 wait complete. 2024-09-21T06:23:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.231+0000 7fcc3f896740 20 client.17826 trim_cache size 0 max 16384 2024-09-21T06:23:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.246+0000 7fcc3f896740 1 Processor -- start 2024-09-21T06:23:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.246+0000 7fcc3f896740 1 -- start start 2024-09-21T06:23:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.246+0000 7fcc3f896740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ff42f0 0x55b0733c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.246+0000 7fcc3f896740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ffc4d0 0x55b0733cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.246+0000 7fcc3f896740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b07301d1b0 0x55b0733cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.246+0000 7fcc3f896740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc341b1810 con 0x55b072ffc4d0 2024-09-21T06:23:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.246+0000 7fcc3f896740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc341b3350 con 0x55b072ff42f0 2024-09-21T06:23:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.246+0000 7fcc3f896740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc341b1ee0 con 0x55b07301d1b0 2024-09-21T06:23:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.246+0000 7fcc3259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ffc4d0 0x55b0733cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.246+0000 7fcc3259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ffc4d0 0x55b0733cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41942/0 (socket says 172.21.15.3:41942) 2024-09-21T06:23:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.246+0000 7fcc3259c640 1 -- 172.21.15.3:0/4032581956 learned_addr learned my addr 172.21.15.3:0/4032581956 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.247+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/4032581956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ff42f0 0x55b0733c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.247+0000 7fcc3259c640 1 -- 172.21.15.3:0/4032581956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b07301d1b0 msgr2=0x55b0733cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.247+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/4032581956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b07301d1b0 0x55b0733cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.247+0000 7fcc3259c640 1 --2- 172.21.15.3:0/4032581956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b07301d1b0 0x55b0733cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.247+0000 7fcc3259c640 1 -- 172.21.15.3:0/4032581956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ff42f0 msgr2=0x55b0733c43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.247+0000 7fcc3259c640 1 --2- 172.21.15.3:0/4032581956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ff42f0 0x55b0733c43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.247+0000 7fcc3259c640 1 -- 172.21.15.3:0/4032581956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0733d0c60 con 0x55b072ffc4d0 2024-09-21T06:23:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.247+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/4032581956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b07301d1b0 0x55b0733cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:23:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.247+0000 7fcc3259c640 1 --2- 172.21.15.3:0/4032581956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ffc4d0 0x55b0733cfcd0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fcc24005230 tx=0x7fcc24005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f880bcd9b65535 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.247+0000 7fcc1bfff640 1 -- 172.21.15.3:0/4032581956 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24035040 con 0x55b072ffc4d0 2024-09-21T06:23:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.247+0000 7fcc1bfff640 1 -- 172.21.15.3:0/4032581956 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc24031d10 con 0x55b072ffc4d0 2024-09-21T06:23:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.247+0000 7fcc1bfff640 1 -- 172.21.15.3:0/4032581956 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24040040 con 0x55b072ffc4d0 2024-09-21T06:23:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.248+0000 7fcc3f896740 1 -- 172.21.15.3:0/4032581956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ffc4d0 msgr2=0x55b0733cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.248+0000 7fcc3f896740 1 --2- 172.21.15.3:0/4032581956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ffc4d0 0x55b0733cfcd0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fcc24005230 tx=0x7fcc24005300 comp rx=0 tx=0).stop 2024-09-21T06:23:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.248+0000 7fcc3f896740 1 -- 172.21.15.3:0/4032581956 shutdown_connections 2024-09-21T06:23:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.248+0000 7fcc3f896740 1 --2- 172.21.15.3:0/4032581956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b07301d1b0 0x55b0733cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.248+0000 7fcc3f896740 1 --2- 172.21.15.3:0/4032581956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b072ffc4d0 0x55b0733cfcd0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.248+0000 7fcc3f896740 1 --2- 172.21.15.3:0/4032581956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ff42f0 0x55b0733c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.248+0000 7fcc3f896740 1 -- 172.21.15.3:0/4032581956 >> 172.21.15.3:0/4032581956 conn(0x55b073011300 msgr2=0x55b0733cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.248+0000 7fcc3f896740 1 -- 172.21.15.3:0/4032581956 shutdown_connections 2024-09-21T06:23:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.248+0000 7fcc3f896740 1 -- 172.21.15.3:0/4032581956 wait complete. 2024-09-21T06:23:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.248+0000 7fcc3f896740 1 -- start start 2024-09-21T06:23:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.248+0000 7fcc3f896740 1 Processor -- start 2024-09-21T06:23:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc3f896740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ff42f0 0x55b0733d6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc3f896740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ffc4d0 0x55b0733d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc3f896740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 0x55b0733d4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc3f896740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc341b2c80 con 0x55b07301d1b0 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc3f896740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc341b25b0 con 0x55b072ff42f0 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc3f896740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc341af7f0 con 0x55b072ffc4d0 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc3f896740 10 client.0 _ll_register_callbacks cb 0x7fcc2800be40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 0x55b0733d4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 0x55b0733d4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41946/0 (socket says 172.21.15.3:41946) 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc31d9b640 1 -- 172.21.15.3:0/2887878404 learned_addr learned my addr 172.21.15.3:0/2887878404 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ff42f0 0x55b0733d6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc3259c640 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ffc4d0 0x55b0733d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc31d9b640 1 -- 172.21.15.3:0/2887878404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ffc4d0 msgr2=0x55b0733d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ffc4d0 0x55b0733d4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc31d9b640 1 -- 172.21.15.3:0/2887878404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ff42f0 msgr2=0x55b0733d6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ff42f0 0x55b0733d6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc31d9b640 1 -- 172.21.15.3:0/2887878404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0733d4d90 con 0x55b07301d1b0 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc3259c640 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ffc4d0 0x55b0733d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc31d9b640 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 0x55b0733d4600 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fcc280059f0 tx=0x7fcc2800bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91dc46e061f3bf8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.249+0000 7fcc19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc19ffb640 1 -- 172.21.15.3:0/2887878404 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28005020 con 0x55b07301d1b0 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc19ffb640 1 -- 172.21.15.3:0/2887878404 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8463+0+0 (secure 0 0 0) 0x7fcc280051c0 con 0x55b07301d1b0 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc3f896740 1 -- 172.21.15.3:0/2887878404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc24031d10 con 0x55b07301d1b0 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc19ffb640 1 -- 172.21.15.3:0/2887878404 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcc280053c0 con 0x55b07301d1b0 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc3f896740 10 client.17832 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc3f896740 1 -- 172.21.15.3:0/2887878404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b07344aa70 con 0x55b07301d1b0 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc19ffb640 1 -- 172.21.15.3:0/2887878404 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc24031d10 con 0x55b07301d1b0 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc19ffb640 1 -- 172.21.15.3:0/2887878404 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fcc28003de0 con 0x55b07301d1b0 2024-09-21T06:23:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc3f896740 10 client.17832 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc3f896740 10 client.17832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc3f896740 20 client.17832 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc3f896740 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b072fb11c0 0x55b07344d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc3f896740 4 client.17832 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:23:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.250+0000 7fcc3f896740 1 -- 172.21.15.3:0/2887878404 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b0733350f0 con 0x55b072fb11c0 2024-09-21T06:23:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.251+0000 7fcc3259c640 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b072fb11c0 0x55b07344d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.251+0000 7fcc19ffb640 10 client.17832 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.251+0000 7fcc3259c640 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b072fb11c0 0x55b07344d180 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=408a606277534b20 server_cookie=d84ff02e6f06a750 in_seq=0 out_seq=0 2024-09-21T06:23:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.389+0000 7fcc19ffb640 1 -- 172.21.15.3:0/2887878404 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b0733350f0 con 0x55b072fb11c0 2024-09-21T06:23:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.389+0000 7fcc19ffb640 10 client.17832 handle_command_reply: tid=0 2024-09-21T06:23:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.389+0000 7fcc3f896740 1 client.17832 shutdown 2024-09-21T06:23:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3f896740 1 -- 172.21.15.3:0/2887878404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 msgr2=0x55b0733d4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3f896740 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 0x55b0733d4600 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fcc280059f0 tx=0x7fcc2800bbb0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3f896740 1 -- 172.21.15.3:0/2887878404 shutdown_connections 2024-09-21T06:23:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3f896740 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b072fb11c0 0x55b07344d180 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3f896740 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b072ffc4d0 0x55b0733d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3f896740 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b07301d1b0 0x55b0733d4600 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3f896740 1 --2- 172.21.15.3:0/2887878404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b072ff42f0 0x55b0733d6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc19ffb640 0 client.17832 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3f896740 1 -- 172.21.15.3:0/2887878404 >> 172.21.15.3:0/2887878404 conn(0x55b073011300 msgr2=0x55b0733c4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3f896740 1 -- 172.21.15.3:0/2887878404 shutdown_connections 2024-09-21T06:23:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3f896740 1 -- 172.21.15.3:0/2887878404 wait complete. 2024-09-21T06:23:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3f896740 20 client.17832 trim_cache size 0 max 16384 2024-09-21T06:23:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:23:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:23:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:23:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:23:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3892b640 1 -- 172.21.15.3:0/3904063192 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc1c0b6680 msgr2=0x7fcc1c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3892b640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc1c0b6680 0x7fcc1c0b8b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0e5240 tx=0x7fcc2c07a540 comp rx=0 tx=0).stop 2024-09-21T06:23:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3892b640 1 -- 172.21.15.3:0/3904063192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc34108c60 msgr2=0x7fcc340ff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.390+0000 7fcc3892b640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc34108c60 0x7fcc340ff390 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00cce0 tx=0x7fcc2c00a620 comp rx=0 tx=0).stop 2024-09-21T06:23:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.391+0000 7fcc3892b640 1 -- 172.21.15.3:0/3904063192 shutdown_connections 2024-09-21T06:23:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.391+0000 7fcc3892b640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcc1c0b6680 0x7fcc1c0b8b90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.391+0000 7fcc3892b640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc34108310 0x7fcc340fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.391+0000 7fcc3892b640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc34108c60 0x7fcc340ff390 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.391+0000 7fcc3892b640 1 --2- 172.21.15.3:0/3904063192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc341029e0 0x7fcc340fe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.391+0000 7fcc3892b640 1 -- 172.21.15.3:0/3904063192 >> 172.21.15.3:0/3904063192 conn(0x7fcc340fe3e0 msgr2=0x7fcc3406bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.391+0000 7fcc3892b640 1 -- 172.21.15.3:0/3904063192 shutdown_connections 2024-09-21T06:23:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:03.391+0000 7fcc3892b640 1 -- 172.21.15.3:0/3904063192 wait complete. 2024-09-21T06:23:03.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:03 smithi137 ceph-mon[34464]: pgmap v570: 97 pgs: 97 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2024-09-21T06:23:03.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:03 smithi204 ceph-mon[34659]: pgmap v570: 97 pgs: 97 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2024-09-21T06:23:03.741 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:03 smithi003 ceph-mon[31227]: pgmap v570: 97 pgs: 97 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2024-09-21T06:23:03.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 853090 10.96 MB/sec execute 276 sec latency 2664.342 ms 2024-09-21T06:23:04.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 854713 10.94 MB/sec execute 277 sec latency 30.913 ms 2024-09-21T06:23:05.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:05 smithi003 ceph-mon[31227]: pgmap v571: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-09-21T06:23:05.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:05 smithi137 ceph-mon[34464]: pgmap v571: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-09-21T06:23:05.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:05 smithi204 ceph-mon[34659]: pgmap v571: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-09-21T06:23:05.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 856461 10.95 MB/sec execute 278 sec latency 28.740 ms 2024-09-21T06:23:06.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 858054 10.93 MB/sec execute 279 sec latency 50.073 ms 2024-09-21T06:23:07.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:07 smithi137 ceph-mon[34464]: pgmap v572: 97 pgs: 97 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 332 op/s 2024-09-21T06:23:07.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:07 smithi137 ceph-mon[34464]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:23:07.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:07 smithi204 ceph-mon[34659]: pgmap v572: 97 pgs: 97 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 332 op/s 2024-09-21T06:23:07.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:07 smithi204 ceph-mon[34659]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:23:07.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:07 smithi003 ceph-mon[31227]: pgmap v572: 97 pgs: 97 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 332 op/s 2024-09-21T06:23:07.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:07 smithi003 ceph-mon[31227]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:23:07.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 859003 10.92 MB/sec execute 280 sec latency 53.689 ms 2024-09-21T06:23:08.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 859434 10.89 MB/sec execute 281 sec latency 476.129 ms 2024-09-21T06:23:09.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:09 smithi137 ceph-mon[34464]: pgmap v574: 97 pgs: 5 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 84 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 320 op/s 2024-09-21T06:23:09.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:09 smithi204 ceph-mon[34659]: pgmap v574: 97 pgs: 5 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 84 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 320 op/s 2024-09-21T06:23:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:09 smithi003 ceph-mon[31227]: pgmap v574: 97 pgs: 5 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 84 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 320 op/s 2024-09-21T06:23:09.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 860612 10.87 MB/sec execute 282 sec latency 693.463 ms 2024-09-21T06:23:10.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 862690 10.88 MB/sec execute 283 sec latency 226.743 ms 2024-09-21T06:23:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:11 smithi137 ceph-mon[34464]: pgmap v575: 97 pgs: 5 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 84 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-09-21T06:23:11.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:11 smithi204 ceph-mon[34659]: pgmap v575: 97 pgs: 5 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 84 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-09-21T06:23:11.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:11 smithi003 ceph-mon[31227]: pgmap v575: 97 pgs: 5 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 84 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-09-21T06:23:11.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 862690 10.84 MB/sec execute 284 sec latency 1226.833 ms 2024-09-21T06:23:12.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 862690 10.80 MB/sec execute 285 sec latency 2226.927 ms 2024-09-21T06:23:13.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:13 smithi137 ceph-mon[34464]: pgmap v576: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-09-21T06:23:13.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:13 smithi204 ceph-mon[34659]: pgmap v576: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-09-21T06:23:13.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:13 smithi003 ceph-mon[31227]: pgmap v576: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-09-21T06:23:13.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 862690 10.77 MB/sec execute 286 sec latency 3227.038 ms 2024-09-21T06:23:14.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 862690 10.73 MB/sec execute 287 sec latency 4227.150 ms 2024-09-21T06:23:15.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:15 smithi003 ceph-mon[31227]: pgmap v577: 97 pgs: 97 active+clean; 127 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2024-09-21T06:23:15.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:15 smithi137 ceph-mon[34464]: pgmap v577: 97 pgs: 97 active+clean; 127 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2024-09-21T06:23:15.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:15 smithi204 ceph-mon[34659]: pgmap v577: 97 pgs: 97 active+clean; 127 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2024-09-21T06:23:15.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 862690 10.69 MB/sec execute 288 sec latency 5227.189 ms 2024-09-21T06:23:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.494+0000 7f3843301640 1 -- 172.21.15.3:0/1132723125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f383c108c30 msgr2=0x7f383c10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.494+0000 7f3843301640 1 --2- 172.21.15.3:0/1132723125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f383c108c30 0x7f383c10f460 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3834009900 tx=0x7f383402a410 comp rx=0 tx=0).stop 2024-09-21T06:23:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.494+0000 7f3843301640 1 -- 172.21.15.3:0/1132723125 shutdown_connections 2024-09-21T06:23:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.494+0000 7f3843301640 1 --2- 172.21.15.3:0/1132723125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f383c108c30 0x7f383c10f460 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.494+0000 7f3843301640 1 --2- 172.21.15.3:0/1132723125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f383c1082e0 0x7f383c1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.494+0000 7f3843301640 1 --2- 172.21.15.3:0/1132723125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c1029b0 0x7f383c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.494+0000 7f3843301640 1 -- 172.21.15.3:0/1132723125 >> 172.21.15.3:0/1132723125 conn(0x7f383c07a740 msgr2=0x7f383c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.494+0000 7f3843301640 1 -- 172.21.15.3:0/1132723125 shutdown_connections 2024-09-21T06:23:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.495+0000 7f3843301640 1 -- 172.21.15.3:0/1132723125 wait complete. 2024-09-21T06:23:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.495+0000 7f3843301640 1 Processor -- start 2024-09-21T06:23:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.495+0000 7f3843301640 1 -- start start 2024-09-21T06:23:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.495+0000 7f3843301640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c1029b0 0x7f383c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.495+0000 7f3843301640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f383c1082e0 0x7f383c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f3843301640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f383c108c30 0x7f383c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f3843301640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1128f0 con 0x7f383c1082e0 2024-09-21T06:23:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f3843301640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f383c112770 con 0x7f383c108c30 2024-09-21T06:23:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f3843301640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f383c112a70 con 0x7f383c1029b0 2024-09-21T06:23:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f384109d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c1029b0 0x7f383c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f384089c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f383c1082e0 0x7f383c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f384109d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c1029b0 0x7f383c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41380/0 (socket says 172.21.15.3:41380) 2024-09-21T06:23:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f384109d640 1 -- 172.21.15.3:0/3277837432 learned_addr learned my addr 172.21.15.3:0/3277837432 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f384189e640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f383c108c30 0x7f383c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f384089c640 1 -- 172.21.15.3:0/3277837432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c1029b0 msgr2=0x7f383c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f384089c640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c1029b0 0x7f383c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f384089c640 1 -- 172.21.15.3:0/3277837432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f383c108c30 msgr2=0x7f383c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f384089c640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f383c108c30 0x7f383c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f384089c640 1 -- 172.21.15.3:0/3277837432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383c1a0390 con 0x7f383c1082e0 2024-09-21T06:23:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f384109d640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c1029b0 0x7f383c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f384089c640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f383c1082e0 0x7f383c19aa00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f383c06c9c0 tx=0x7f382c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f38327fc640 1 -- 172.21.15.3:0/3277837432 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c00cc40 con 0x7f383c1082e0 2024-09-21T06:23:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f3843301640 1 -- 172.21.15.3:0/3277837432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f383c1a0680 con 0x7f383c1082e0 2024-09-21T06:23:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f38327fc640 1 -- 172.21.15.3:0/3277837432 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f382c00cde0 con 0x7f383c1082e0 2024-09-21T06:23:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f38327fc640 1 -- 172.21.15.3:0/3277837432 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c018760 con 0x7f383c1082e0 2024-09-21T06:23:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.496+0000 7f3843301640 1 -- 172.21.15.3:0/3277837432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f383c1a6e10 con 0x7f383c1082e0 2024-09-21T06:23:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.499+0000 7f384a26f740 1 Processor -- start 2024-09-21T06:23:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.499+0000 7f384a26f740 1 -- start start 2024-09-21T06:23:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.499+0000 7f38327fc640 1 -- 172.21.15.3:0/3277837432 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f382c010a90 con 0x7f383c1082e0 2024-09-21T06:23:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.499+0000 7f384a26f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 0x561fa68979b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.499+0000 7f384a26f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64c92f0 0x561fa6897ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.499+0000 7f384a26f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64d14d0 0x561fa6898430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.499+0000 7f384a26f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561fa680c6f0 con 0x561fa64d14d0 2024-09-21T06:23:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.499+0000 7f384a26f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561fa67fbb20 con 0x561fa64f21b0 2024-09-21T06:23:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.499+0000 7f384a26f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561fa67ff930 con 0x561fa64c92f0 2024-09-21T06:23:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.500+0000 7f384109d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64c92f0 0x561fa6897ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.500+0000 7f384109d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64c92f0 0x561fa6897ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41390/0 (socket says 172.21.15.3:41390) 2024-09-21T06:23:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.500+0000 7f384109d640 1 -- 172.21.15.3:0/2637183301 learned_addr learned my addr 172.21.15.3:0/2637183301 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.500+0000 7f384089c640 1 --2- 172.21.15.3:0/2637183301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64d14d0 0x561fa6898430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.500+0000 7f384189e640 1 --2- 172.21.15.3:0/2637183301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 0x561fa68979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.500+0000 7f384109d640 1 -- 172.21.15.3:0/2637183301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 msgr2=0x561fa68979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.500+0000 7f384109d640 1 --2- 172.21.15.3:0/2637183301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 0x561fa68979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.500+0000 7f384109d640 1 -- 172.21.15.3:0/2637183301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64d14d0 msgr2=0x561fa6898430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.500+0000 7f384109d640 1 --2- 172.21.15.3:0/2637183301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64d14d0 0x561fa6898430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.500+0000 7f384109d640 1 -- 172.21.15.3:0/2637183301 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fa67c1530 con 0x561fa64c92f0 2024-09-21T06:23:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.500+0000 7f384109d640 1 --2- 172.21.15.3:0/2637183301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64c92f0 0x561fa6897ef0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3838011960 tx=0x7f3838011e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2baf5bf8dcc1b116 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.501+0000 7f38327fc640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f38280b6680 0x7f38280b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.501+0000 7f38327fc640 1 -- 172.21.15.3:0/3277837432 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8551+0+0 (secure 0 0 0) 0x7f382c0e7980 con 0x7f383c1082e0 2024-09-21T06:23:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.501+0000 7f384189e640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f38280b6680 0x7f38280b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.501+0000 7f38277fe640 1 -- 172.21.15.3:0/2637183301 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838011310 con 0x561fa64c92f0 2024-09-21T06:23:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.501+0000 7f38277fe640 1 -- 172.21.15.3:0/2637183301 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3838019430 con 0x561fa64c92f0 2024-09-21T06:23:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.501+0000 7f38277fe640 1 -- 172.21.15.3:0/2637183301 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838010670 con 0x561fa64c92f0 2024-09-21T06:23:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.502+0000 7f384189e640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f38280b6680 0x7f38280b8b90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3834006010 tx=0x7f38340058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.502+0000 7f384a26f740 1 -- 172.21.15.3:0/2637183301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64c92f0 msgr2=0x561fa6897ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.502+0000 7f384a26f740 1 --2- 172.21.15.3:0/2637183301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64c92f0 0x561fa6897ef0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3838011960 tx=0x7f3838011e20 comp rx=0 tx=0).stop 2024-09-21T06:23:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.502+0000 7f384a26f740 1 -- 172.21.15.3:0/2637183301 shutdown_connections 2024-09-21T06:23:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.502+0000 7f384a26f740 1 --2- 172.21.15.3:0/2637183301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64c92f0 0x561fa6897ef0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.502+0000 7f384a26f740 1 --2- 172.21.15.3:0/2637183301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64d14d0 0x561fa6898430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.502+0000 7f384a26f740 1 --2- 172.21.15.3:0/2637183301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 0x561fa68979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.502+0000 7f384a26f740 1 -- 172.21.15.3:0/2637183301 >> 172.21.15.3:0/2637183301 conn(0x561fa64e6300 msgr2=0x561fa68958a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.502+0000 7f384a26f740 1 -- 172.21.15.3:0/2637183301 shutdown_connections 2024-09-21T06:23:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.502+0000 7f384a26f740 1 -- 172.21.15.3:0/2637183301 wait complete. 2024-09-21T06:23:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.503+0000 7f384a26f740 1 -- start start 2024-09-21T06:23:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.503+0000 7f384a26f740 1 Processor -- start 2024-09-21T06:23:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384a26f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64c92f0 0x561fa68a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384a26f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64d14d0 0x561fa68a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384a26f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64f21b0 0x561fa68a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384a26f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561fa6733ab0 con 0x561fa64d14d0 2024-09-21T06:23:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384a26f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561fa67a1730 con 0x561fa64c92f0 2024-09-21T06:23:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384a26f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561fa68a6130 con 0x561fa64f21b0 2024-09-21T06:23:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384a26f740 10 client.0 _ll_register_callbacks cb 0x561fa67f56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384189e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64f21b0 0x561fa68a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384089c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64d14d0 0x561fa68a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384189e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64f21b0 0x561fa68a3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:41404/0 (socket says 172.21.15.3:41404) 2024-09-21T06:23:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384109d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64c92f0 0x561fa68a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384189e640 1 -- 172.21.15.3:0/3933688751 learned_addr learned my addr 172.21.15.3:0/3933688751 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384189e640 1 -- 172.21.15.3:0/3933688751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64c92f0 msgr2=0x561fa68a2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384189e640 1 --2- 172.21.15.3:0/3933688751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64c92f0 0x561fa68a2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384189e640 1 -- 172.21.15.3:0/3933688751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64d14d0 msgr2=0x561fa68a34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384189e640 1 --2- 172.21.15.3:0/3933688751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64d14d0 0x561fa68a34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384189e640 1 -- 172.21.15.3:0/3933688751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fa655d790 con 0x561fa64f21b0 2024-09-21T06:23:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384089c640 1 --2- 172.21.15.3:0/3933688751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64d14d0 0x561fa68a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f384189e640 1 --2- 172.21.15.3:0/3933688751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64f21b0 0x561fa68a3a10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3834033000 tx=0x7f3834009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac5de68ae020832d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.504+0000 7f3824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:23:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f3824ff9640 1 -- 172.21.15.3:0/3933688751 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383404d070 con 0x561fa64f21b0 2024-09-21T06:23:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f3824ff9640 1 -- 172.21.15.3:0/3933688751 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8551+0+0 (secure 0 0 0) 0x7f3834005050 con 0x561fa64f21b0 2024-09-21T06:23:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f384a26f740 1 -- 172.21.15.3:0/3933688751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3838019430 con 0x561fa64f21b0 2024-09-21T06:23:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f3824ff9640 1 -- 172.21.15.3:0/3933688751 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3834059050 con 0x561fa64f21b0 2024-09-21T06:23:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f384a26f740 10 client.27352 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f384a26f740 1 -- 172.21.15.3:0/3933688751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561fa65d6620 con 0x561fa64f21b0 2024-09-21T06:23:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f3824ff9640 1 -- 172.21.15.3:0/3933688751 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3838019430 con 0x561fa64f21b0 2024-09-21T06:23:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f3824ff9640 1 -- 172.21.15.3:0/3933688751 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f38340035f0 con 0x561fa64f21b0 2024-09-21T06:23:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f384a26f740 10 client.27352 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f384a26f740 10 client.27352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f384a26f740 20 client.27352 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f384a26f740 1 --2- 172.21.15.3:0/3933688751 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561fa64b0d80 0x561fa68b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f384a26f740 4 client.27352 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.505+0000 7f384a26f740 1 -- 172.21.15.3:0/3933688751 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fa680e640 con 0x561fa64b0d80 2024-09-21T06:23:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.506+0000 7f384109d640 1 --2- 172.21.15.3:0/3933688751 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561fa64b0d80 0x561fa68b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.506+0000 7f384109d640 1 --2- 172.21.15.3:0/3933688751 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561fa64b0d80 0x561fa68b5140 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bebf724c1ad70077 server_cookie=b968102d585ec106 in_seq=0 out_seq=0 2024-09-21T06:23:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.506+0000 7f3824ff9640 10 client.27352 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.507+0000 7f3824ff9640 1 -- 172.21.15.3:0/3933688751 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561fa680e640 con 0x561fa64b0d80 2024-09-21T06:23:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.507+0000 7f3824ff9640 10 client.27352 handle_command_reply: tid=0 2024-09-21T06:23:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.507+0000 7f384a26f740 1 client.27352 shutdown 2024-09-21T06:23:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.507+0000 7f384a26f740 1 -- 172.21.15.3:0/3933688751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64f21b0 msgr2=0x561fa68a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.507+0000 7f384a26f740 1 --2- 172.21.15.3:0/3933688751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64f21b0 0x561fa68a3a10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3834033000 tx=0x7f3834009260 comp rx=0 tx=0).stop 2024-09-21T06:23:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.507+0000 7f384a26f740 1 -- 172.21.15.3:0/3933688751 shutdown_connections 2024-09-21T06:23:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.507+0000 7f384a26f740 1 --2- 172.21.15.3:0/3933688751 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561fa64b0d80 0x561fa68b5140 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.507+0000 7f384a26f740 1 --2- 172.21.15.3:0/3933688751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64f21b0 0x561fa68a3a10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.507+0000 7f384a26f740 1 --2- 172.21.15.3:0/3933688751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64d14d0 0x561fa68a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.507+0000 7f3824ff9640 0 client.27352 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.507+0000 7f384a26f740 1 --2- 172.21.15.3:0/3933688751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64c92f0 0x561fa68a2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.507+0000 7f384a26f740 1 -- 172.21.15.3:0/3933688751 >> 172.21.15.3:0/3933688751 conn(0x561fa64e6300 msgr2=0x561fa68989e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.508+0000 7f384a26f740 1 -- 172.21.15.3:0/3933688751 shutdown_connections 2024-09-21T06:23:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.508+0000 7f384a26f740 1 -- 172.21.15.3:0/3933688751 wait complete. 2024-09-21T06:23:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.508+0000 7f384a26f740 20 client.27352 trim_cache size 0 max 16384 2024-09-21T06:23:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.523+0000 7f384a26f740 1 Processor -- start 2024-09-21T06:23:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.523+0000 7f384a26f740 1 -- start start 2024-09-21T06:23:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384a26f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 0x561fa68990f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384a26f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64d14d0 0x561fa68a0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384a26f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 0x561fa68a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384a26f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1af6e0 con 0x561fa64c92f0 2024-09-21T06:23:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384a26f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b1220 con 0x561fa64f21b0 2024-09-21T06:23:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384a26f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1afdb0 con 0x561fa64d14d0 2024-09-21T06:23:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384109d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 0x561fa68990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384109d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 0x561fa68990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60758/0 (socket says 172.21.15.3:60758) 2024-09-21T06:23:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384109d640 1 -- 172.21.15.3:0/1542081926 learned_addr learned my addr 172.21.15.3:0/1542081926 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384089c640 1 --2- 172.21.15.3:0/1542081926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64d14d0 0x561fa68a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384189e640 1 --2- 172.21.15.3:0/1542081926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 0x561fa68a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384109d640 1 -- 172.21.15.3:0/1542081926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64d14d0 msgr2=0x561fa68a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384109d640 1 --2- 172.21.15.3:0/1542081926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64d14d0 0x561fa68a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384109d640 1 -- 172.21.15.3:0/1542081926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 msgr2=0x561fa68a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384109d640 1 --2- 172.21.15.3:0/1542081926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 0x561fa68a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384109d640 1 -- 172.21.15.3:0/1542081926 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fa68a58e0 con 0x561fa64c92f0 2024-09-21T06:23:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f384109d640 1 --2- 172.21.15.3:0/1542081926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 0x561fa68990f0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f3838011850 tx=0x7f3838019df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b290f3e1cab5e9af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.524+0000 7f3826ffd640 1 -- 172.21.15.3:0/1542081926 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838019860 con 0x561fa64c92f0 2024-09-21T06:23:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.525+0000 7f3826ffd640 1 -- 172.21.15.3:0/1542081926 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3838019a00 con 0x561fa64c92f0 2024-09-21T06:23:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.525+0000 7f3826ffd640 1 -- 172.21.15.3:0/1542081926 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38380112a0 con 0x561fa64c92f0 2024-09-21T06:23:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.525+0000 7f384a26f740 1 -- 172.21.15.3:0/1542081926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 msgr2=0x561fa68990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.525+0000 7f384a26f740 1 --2- 172.21.15.3:0/1542081926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 0x561fa68990f0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f3838011850 tx=0x7f3838019df0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.525+0000 7f384a26f740 1 -- 172.21.15.3:0/1542081926 shutdown_connections 2024-09-21T06:23:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.525+0000 7f384a26f740 1 --2- 172.21.15.3:0/1542081926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64d14d0 0x561fa68a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.525+0000 7f384a26f740 1 --2- 172.21.15.3:0/1542081926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 0x561fa68a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.525+0000 7f384a26f740 1 --2- 172.21.15.3:0/1542081926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 0x561fa68990f0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.525+0000 7f384a26f740 1 -- 172.21.15.3:0/1542081926 >> 172.21.15.3:0/1542081926 conn(0x561fa64e6300 msgr2=0x561fa68989e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.525+0000 7f384a26f740 1 -- 172.21.15.3:0/1542081926 shutdown_connections 2024-09-21T06:23:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.525+0000 7f384a26f740 1 -- 172.21.15.3:0/1542081926 wait complete. 2024-09-21T06:23:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384a26f740 1 -- start start 2024-09-21T06:23:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384a26f740 1 Processor -- start 2024-09-21T06:23:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384a26f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 0x561fa68a8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384a26f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64d14d0 0x561fa68a9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384a26f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 0x561fa6916a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384a26f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b0b50 con 0x561fa64c92f0 2024-09-21T06:23:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384a26f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b0480 con 0x561fa64f21b0 2024-09-21T06:23:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384a26f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1ad6e0 con 0x561fa64d14d0 2024-09-21T06:23:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384a26f740 10 client.0 _ll_register_callbacks cb 0x7f3834009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384109d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 0x561fa68a8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384109d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 0x561fa68a8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60770/0 (socket says 172.21.15.3:60770) 2024-09-21T06:23:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384089c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64d14d0 0x561fa68a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384109d640 1 -- 172.21.15.3:0/2174832839 learned_addr learned my addr 172.21.15.3:0/2174832839 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.526+0000 7f384189e640 1 --2- 172.21.15.3:0/2174832839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 0x561fa6916a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.527+0000 7f384109d640 1 -- 172.21.15.3:0/2174832839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64d14d0 msgr2=0x561fa68a9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.527+0000 7f384109d640 1 --2- 172.21.15.3:0/2174832839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64d14d0 0x561fa68a9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.527+0000 7f384109d640 1 -- 172.21.15.3:0/2174832839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 msgr2=0x561fa6916a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.527+0000 7f384109d640 1 --2- 172.21.15.3:0/2174832839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 0x561fa6916a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.527+0000 7f384109d640 1 -- 172.21.15.3:0/2174832839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fa6917110 con 0x561fa64c92f0 2024-09-21T06:23:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.527+0000 7f384109d640 1 --2- 172.21.15.3:0/2174832839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 0x561fa68a8e30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f383800f930 tx=0x7f3838019cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76dad04254120bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.527+0000 7f3824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.527+0000 7f3824ff9640 1 -- 172.21.15.3:0/2174832839 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838039020 con 0x561fa64c92f0 2024-09-21T06:23:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.527+0000 7f3824ff9640 1 -- 172.21.15.3:0/2174832839 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8551+0+0 (secure 0 0 0) 0x7f38380049d0 con 0x561fa64c92f0 2024-09-21T06:23:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.527+0000 7f384a26f740 1 -- 172.21.15.3:0/2174832839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3838019a00 con 0x561fa64c92f0 2024-09-21T06:23:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.527+0000 7f3824ff9640 1 -- 172.21.15.3:0/2174832839 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3838004bd0 con 0x561fa64c92f0 2024-09-21T06:23:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.527+0000 7f384a26f740 10 client.17850 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.527+0000 7f384a26f740 1 -- 172.21.15.3:0/2174832839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561fa691f610 con 0x561fa64c92f0 2024-09-21T06:23:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.528+0000 7f3824ff9640 1 -- 172.21.15.3:0/2174832839 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3838019a00 con 0x561fa64c92f0 2024-09-21T06:23:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.528+0000 7f3824ff9640 1 -- 172.21.15.3:0/2174832839 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f3838010530 con 0x561fa64c92f0 2024-09-21T06:23:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.528+0000 7f384a26f740 10 client.17850 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.528+0000 7f384a26f740 10 client.17850 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.528+0000 7f384a26f740 20 client.17850 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.528+0000 7f384a26f740 1 --2- 172.21.15.3:0/2174832839 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561fa64861c0 0x561fa6921b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.528+0000 7f384a26f740 4 client.17850 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:23:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.528+0000 7f384a26f740 1 -- 172.21.15.3:0/2174832839 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561fa680a0f0 con 0x561fa64861c0 2024-09-21T06:23:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.528+0000 7f384089c640 1 --2- 172.21.15.3:0/2174832839 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561fa64861c0 0x561fa6921b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.528+0000 7f384089c640 1 --2- 172.21.15.3:0/2174832839 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561fa64861c0 0x561fa6921b70 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1c16846da4843a4 server_cookie=4391953e87756b79 in_seq=0 out_seq=0 2024-09-21T06:23:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.528+0000 7f3824ff9640 10 client.17850 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.699+0000 7f3824ff9640 1 -- 172.21.15.3:0/2174832839 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561fa680a0f0 con 0x561fa64861c0 2024-09-21T06:23:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.699+0000 7f3824ff9640 10 client.17850 handle_command_reply: tid=0 2024-09-21T06:23:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.699+0000 7f384a26f740 1 client.17850 shutdown 2024-09-21T06:23:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.699+0000 7f384a26f740 1 -- 172.21.15.3:0/2174832839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 msgr2=0x561fa68a8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.699+0000 7f384a26f740 1 --2- 172.21.15.3:0/2174832839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 0x561fa68a8e30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f383800f930 tx=0x7f3838019cd0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.699+0000 7f384a26f740 1 -- 172.21.15.3:0/2174832839 shutdown_connections 2024-09-21T06:23:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.699+0000 7f384a26f740 1 --2- 172.21.15.3:0/2174832839 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x561fa64861c0 0x561fa6921b70 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.699+0000 7f3824ff9640 0 client.17850 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.700+0000 7f384a26f740 1 --2- 172.21.15.3:0/2174832839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fa64d14d0 0x561fa68a9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.700+0000 7f384a26f740 1 --2- 172.21.15.3:0/2174832839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561fa64f21b0 0x561fa6916a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.700+0000 7f384a26f740 1 --2- 172.21.15.3:0/2174832839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fa64c92f0 0x561fa68a8e30 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.700+0000 7f384a26f740 1 -- 172.21.15.3:0/2174832839 >> 172.21.15.3:0/2174832839 conn(0x561fa64e6300 msgr2=0x561fa6895880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.700+0000 7f384a26f740 1 -- 172.21.15.3:0/2174832839 shutdown_connections 2024-09-21T06:23:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.700+0000 7f384a26f740 1 -- 172.21.15.3:0/2174832839 wait complete. 2024-09-21T06:23:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.700+0000 7f384a26f740 20 client.17850 trim_cache size 0 max 16384 2024-09-21T06:23:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:23:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:23:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:23:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:23:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.701+0000 7f3843301640 1 -- 172.21.15.3:0/3277837432 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f38280b6680 msgr2=0x7f38280b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.701+0000 7f3843301640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f38280b6680 0x7f38280b8b90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3834006010 tx=0x7f38340058d0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.701+0000 7f3843301640 1 -- 172.21.15.3:0/3277837432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f383c1082e0 msgr2=0x7f383c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.701+0000 7f3843301640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f383c1082e0 0x7f383c19aa00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f383c06c9c0 tx=0x7f382c00ee50 comp rx=0 tx=0).stop 2024-09-21T06:23:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.701+0000 7f3843301640 1 -- 172.21.15.3:0/3277837432 shutdown_connections 2024-09-21T06:23:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.701+0000 7f3843301640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f38280b6680 0x7f38280b8b90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.701+0000 7f3843301640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f383c108c30 0x7f383c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.701+0000 7f3843301640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f383c1082e0 0x7f383c19aa00 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.701+0000 7f3843301640 1 --2- 172.21.15.3:0/3277837432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c1029b0 0x7f383c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.701+0000 7f3843301640 1 -- 172.21.15.3:0/3277837432 >> 172.21.15.3:0/3277837432 conn(0x7f383c07a740 msgr2=0x7f383c1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.701+0000 7f3843301640 1 -- 172.21.15.3:0/3277837432 shutdown_connections 2024-09-21T06:23:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:16.701+0000 7f3843301640 1 -- 172.21.15.3:0/3277837432 wait complete. 2024-09-21T06:23:16.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 863179 10.66 MB/sec execute 289 sec latency 5277.046 ms 2024-09-21T06:23:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:17 smithi204 ceph-mon[34659]: pgmap v578: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-21T06:23:17.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:17 smithi003 ceph-mon[31227]: pgmap v578: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-21T06:23:17.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:17 smithi137 ceph-mon[34464]: pgmap v578: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-21T06:23:17.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 864726 10.64 MB/sec execute 290 sec latency 123.016 ms 2024-09-21T06:23:18.968 INFO:tasks.workunit.client.0.smithi003.stdout: 1 866517 10.65 MB/sec execute 291 sec latency 114.492 ms 2024-09-21T06:23:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:19 smithi204 ceph-mon[34659]: pgmap v579: 97 pgs: 97 active+clean; 136 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-21T06:23:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:19 smithi204 ceph-mon[34659]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:23:19.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:19 smithi003 ceph-mon[31227]: pgmap v579: 97 pgs: 97 active+clean; 136 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-21T06:23:19.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:19 smithi003 ceph-mon[31227]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:23:19.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:19 smithi137 ceph-mon[34464]: pgmap v579: 97 pgs: 97 active+clean; 136 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-21T06:23:19.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:19 smithi137 ceph-mon[34464]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:23:19.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 867961 10.63 MB/sec execute 292 sec latency 59.657 ms 2024-09-21T06:23:20.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 870046 10.64 MB/sec execute 293 sec latency 91.407 ms 2024-09-21T06:23:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:21 smithi204 ceph-mon[34659]: pgmap v581: 97 pgs: 97 active+clean; 133 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2024-09-21T06:23:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:21 smithi003 ceph-mon[31227]: pgmap v581: 97 pgs: 97 active+clean; 133 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2024-09-21T06:23:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:21 smithi137 ceph-mon[34464]: pgmap v581: 97 pgs: 97 active+clean; 133 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2024-09-21T06:23:21.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 871611 10.63 MB/sec execute 294 sec latency 17.546 ms 2024-09-21T06:23:22.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 873610 10.64 MB/sec execute 295 sec latency 108.828 ms 2024-09-21T06:23:23.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:23 smithi204 ceph-mon[34659]: pgmap v582: 97 pgs: 97 active+clean; 125 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 197 op/s 2024-09-21T06:23:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:23 smithi003 ceph-mon[31227]: pgmap v582: 97 pgs: 97 active+clean; 125 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 197 op/s 2024-09-21T06:23:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:23 smithi137 ceph-mon[34464]: pgmap v582: 97 pgs: 97 active+clean; 125 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 197 op/s 2024-09-21T06:23:23.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 874837 10.62 MB/sec execute 296 sec latency 164.594 ms 2024-09-21T06:23:24.404 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:23:24.695 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:24.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 877036 10.63 MB/sec execute 297 sec latency 51.115 ms 2024-09-21T06:23:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.235+0000 7f82f20e8640 1 -- 172.21.15.3:0/977484429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ec105300 msgr2=0x7f82ec10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.235+0000 7f82f20e8640 1 --2- 172.21.15.3:0/977484429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ec105300 0x7f82ec10d850 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009900 tx=0x7f82e002a410 comp rx=0 tx=0).stop 2024-09-21T06:23:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.235+0000 7f82f20e8640 1 -- 172.21.15.3:0/977484429 shutdown_connections 2024-09-21T06:23:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.235+0000 7f82f20e8640 1 --2- 172.21.15.3:0/977484429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82ec10de10 0x7f82ec110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.235+0000 7f82f20e8640 1 --2- 172.21.15.3:0/977484429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ec105300 0x7f82ec10d850 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.235+0000 7f82f20e8640 1 --2- 172.21.15.3:0/977484429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ec1049b0 0x7f82ec104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.235+0000 7f82f20e8640 1 -- 172.21.15.3:0/977484429 >> 172.21.15.3:0/977484429 conn(0x7f82ec079730 msgr2=0x7f82ec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.235+0000 7f82f20e8640 1 -- 172.21.15.3:0/977484429 shutdown_connections 2024-09-21T06:23:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.235+0000 7f82f20e8640 1 -- 172.21.15.3:0/977484429 wait complete. 2024-09-21T06:23:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.236+0000 7f82f20e8640 1 Processor -- start 2024-09-21T06:23:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.236+0000 7f82f20e8640 1 -- start start 2024-09-21T06:23:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82f20e8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ec1049b0 0x7f82ec19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82f20e8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ec105300 0x7f82ec19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82f20e8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82ec10de10 0x7f82ec19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82f20e8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec113a70 con 0x7f82ec1049b0 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82f20e8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec1138f0 con 0x7f82ec10de10 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82f20e8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec113bf0 con 0x7f82ec105300 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ec1049b0 0x7f82ec19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ec1049b0 0x7f82ec19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48690/0 (socket says 172.21.15.3:48690) 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82eaffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ec105300 0x7f82ec19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82eb7fe640 1 -- 172.21.15.3:0/3272913901 learned_addr learned my addr 172.21.15.3:0/3272913901 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82ebfff640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82ec10de10 0x7f82ec19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82eb7fe640 1 -- 172.21.15.3:0/3272913901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ec105300 msgr2=0x7f82ec19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ec105300 0x7f82ec19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82eb7fe640 1 -- 172.21.15.3:0/3272913901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82ec10de10 msgr2=0x7f82ec19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82ec10de10 0x7f82ec19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82eb7fe640 1 -- 172.21.15.3:0/3272913901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec1a0410 con 0x7f82ec1049b0 2024-09-21T06:23:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.237+0000 7f82eaffd640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ec105300 0x7f82ec19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.238+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ec1049b0 0x7f82ec19a4a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00e9c0 tx=0x7f82dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.238+0000 7f82e8ff9640 1 -- 172.21.15.3:0/3272913901 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc00cd80 con 0x7f82ec1049b0 2024-09-21T06:23:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.238+0000 7f82e8ff9640 1 -- 172.21.15.3:0/3272913901 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82dc0035f0 con 0x7f82ec1049b0 2024-09-21T06:23:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.238+0000 7f82e8ff9640 1 -- 172.21.15.3:0/3272913901 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc010690 con 0x7f82ec1049b0 2024-09-21T06:23:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.238+0000 7f82f20e8640 1 -- 172.21.15.3:0/3272913901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ec1a0700 con 0x7f82ec1049b0 2024-09-21T06:23:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.238+0000 7f82f20e8640 1 -- 172.21.15.3:0/3272913901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ec1a6e60 con 0x7f82ec1049b0 2024-09-21T06:23:25.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.241+0000 7f82f865c740 1 Processor -- start 2024-09-21T06:23:25.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.241+0000 7f82f865c740 1 -- start start 2024-09-21T06:23:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82f865c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98aed070 0x55cb98e94650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82f865c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98e94b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82f865c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98acc390 0x55cb98e950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82f865c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cb98e0ae50 con 0x55cb98acc390 2024-09-21T06:23:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82f865c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cb98dfa280 con 0x55cb98ac41b0 2024-09-21T06:23:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82f865c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cb98dfe090 con 0x55cb98aed070 2024-09-21T06:23:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82e8ff9640 1 -- 172.21.15.3:0/3272913901 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f82dc0037a0 con 0x7f82ec1049b0 2024-09-21T06:23:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82ebfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98aed070 0x55cb98e94650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82eaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98acc390 0x55cb98e950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82ebfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98aed070 0x55cb98e94650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:45534/0 (socket says 172.21.15.3:45534) 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82ebfff640 1 -- 172.21.15.3:0/1777062282 learned_addr learned my addr 172.21.15.3:0/1777062282 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/1777062282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98e94b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82eaffd640 1 -- 172.21.15.3:0/1777062282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98aed070 msgr2=0x55cb98e94650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82eaffd640 1 --2- 172.21.15.3:0/1777062282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98aed070 0x55cb98e94650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82eaffd640 1 -- 172.21.15.3:0/1777062282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 msgr2=0x55cb98e94b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82eaffd640 1 --2- 172.21.15.3:0/1777062282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98e94b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82eaffd640 1 -- 172.21.15.3:0/1777062282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb98b58650 con 0x55cb98acc390 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82ebfff640 1 --2- 172.21.15.3:0/1777062282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98aed070 0x55cb98e94650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/1777062282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98e94b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.242+0000 7f82eaffd640 1 --2- 172.21.15.3:0/1777062282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98acc390 0x55cb98e950d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f82e002a930 tx=0x7f82e0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e953c61c509b5990 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.243+0000 7f82d1ffb640 1 -- 172.21.15.3:0/1777062282 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0040070 con 0x55cb98acc390 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.243+0000 7f82d1ffb640 1 -- 172.21.15.3:0/1777062282 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e00317a0 con 0x55cb98acc390 2024-09-21T06:23:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.243+0000 7f82d1ffb640 1 -- 172.21.15.3:0/1777062282 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0035040 con 0x55cb98acc390 2024-09-21T06:23:25.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.243+0000 7f82e8ff9640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82d40b6680 0x7f82d40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.243+0000 7f82ebfff640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82d40b6680 0x7f82d40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.243+0000 7f82e8ff9640 1 -- 172.21.15.3:0/3272913901 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f82dc0e7ac0 con 0x7f82ec1049b0 2024-09-21T06:23:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.244+0000 7f82f865c740 1 -- 172.21.15.3:0/1777062282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98acc390 msgr2=0x55cb98e950d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.244+0000 7f82f865c740 1 --2- 172.21.15.3:0/1777062282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98acc390 0x55cb98e950d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f82e002a930 tx=0x7f82e0000bc0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.244+0000 7f82ebfff640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82d40b6680 0x7f82d40b8b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005f40 tx=0x7f82e40096a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.244+0000 7f82f865c740 1 -- 172.21.15.3:0/1777062282 shutdown_connections 2024-09-21T06:23:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.244+0000 7f82f865c740 1 --2- 172.21.15.3:0/1777062282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98acc390 0x55cb98e950d0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.244+0000 7f82f865c740 1 --2- 172.21.15.3:0/1777062282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98e94b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.244+0000 7f82f865c740 1 --2- 172.21.15.3:0/1777062282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98aed070 0x55cb98e94650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.244+0000 7f82f865c740 1 -- 172.21.15.3:0/1777062282 >> 172.21.15.3:0/1777062282 conn(0x55cb98ae1460 msgr2=0x55cb98e92540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.244+0000 7f82f865c740 1 -- 172.21.15.3:0/1777062282 shutdown_connections 2024-09-21T06:23:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.244+0000 7f82f865c740 1 -- 172.21.15.3:0/1777062282 wait complete. 2024-09-21T06:23:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.244+0000 7f82f865c740 1 -- start start 2024-09-21T06:23:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82f865c740 1 Processor -- start 2024-09-21T06:23:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82f865c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98e9fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82f865c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98acc390 0x55cb98ea0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82f865c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aed070 0x55cb98eac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82f865c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cb98c5b700 con 0x55cb98aed070 2024-09-21T06:23:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82f865c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cb98d95110 con 0x55cb98ac41b0 2024-09-21T06:23:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82f865c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cb98ea2dd0 con 0x55cb98acc390 2024-09-21T06:23:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82f865c740 10 client.0 _ll_register_callbacks cb 0x55cb98dae990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82ebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aed070 0x55cb98eac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82eb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98e9fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82eaffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98acc390 0x55cb98ea0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82ebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aed070 0x55cb98eac0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48710/0 (socket says 172.21.15.3:48710) 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82eb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98e9fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:33144/0 (socket says 172.21.15.3:33144) 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82ebfff640 1 -- 172.21.15.3:0/1278177497 learned_addr learned my addr 172.21.15.3:0/1278177497 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82eb7fe640 1 -- 172.21.15.3:0/1278177497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98acc390 msgr2=0x55cb98ea0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/1278177497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98acc390 0x55cb98ea0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82eb7fe640 1 -- 172.21.15.3:0/1278177497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aed070 msgr2=0x55cb98eac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/1278177497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aed070 0x55cb98eac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82eb7fe640 1 -- 172.21.15.3:0/1278177497 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb98dc2c40 con 0x55cb98ac41b0 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82eaffd640 1 --2- 172.21.15.3:0/1278177497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98acc390 0x55cb98ea0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.245+0000 7f82ebfff640 1 --2- 172.21.15.3:0/1278177497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aed070 0x55cb98eac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.246+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/1278177497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98e9fd30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0e51c0 tx=0x7f82dc0e5740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a24907aa481c58f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.246+0000 7f82caffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:23:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.246+0000 7f82caffd640 1 -- 172.21.15.3:0/1278177497 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0ad210 con 0x55cb98ac41b0 2024-09-21T06:23:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.246+0000 7f82caffd640 1 -- 172.21.15.3:0/1278177497 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f82dc0a9210 con 0x55cb98ac41b0 2024-09-21T06:23:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.246+0000 7f82f865c740 1 -- 172.21.15.3:0/1278177497 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e00317a0 con 0x55cb98ac41b0 2024-09-21T06:23:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.246+0000 7f82caffd640 1 -- 172.21.15.3:0/1278177497 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f82dc0ad030 con 0x55cb98ac41b0 2024-09-21T06:23:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.246+0000 7f82f865c740 10 client.27422 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.246+0000 7f82f865c740 1 -- 172.21.15.3:0/1278177497 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cb98eaeb10 con 0x55cb98ac41b0 2024-09-21T06:23:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.246+0000 7f82caffd640 1 -- 172.21.15.3:0/1278177497 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e00317a0 con 0x55cb98ac41b0 2024-09-21T06:23:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.247+0000 7f82caffd640 1 -- 172.21.15.3:0/1278177497 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f82dc0a9410 con 0x55cb98ac41b0 2024-09-21T06:23:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.247+0000 7f82f865c740 10 client.27422 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.247+0000 7f82f865c740 10 client.27422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.247+0000 7f82f865c740 20 client.27422 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.247+0000 7f82f865c740 1 --2- 172.21.15.3:0/1278177497 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55cb98aab9f0 0x55cb98eb2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.247+0000 7f82f865c740 4 client.27422 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.247+0000 7f82f865c740 1 -- 172.21.15.3:0/1278177497 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb98e0cda0 con 0x55cb98aab9f0 2024-09-21T06:23:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.247+0000 7f82eaffd640 1 --2- 172.21.15.3:0/1278177497 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55cb98aab9f0 0x55cb98eb2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.247+0000 7f82caffd640 10 client.27422 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.247+0000 7f82eaffd640 1 --2- 172.21.15.3:0/1278177497 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55cb98aab9f0 0x55cb98eb2010 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e730df38d65f35c server_cookie=ead87ef7ab911d12 in_seq=0 out_seq=0 2024-09-21T06:23:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.248+0000 7f82caffd640 1 -- 172.21.15.3:0/1278177497 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb98e0cda0 con 0x55cb98aab9f0 2024-09-21T06:23:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.248+0000 7f82caffd640 10 client.27422 handle_command_reply: tid=0 2024-09-21T06:23:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.248+0000 7f82f865c740 1 client.27422 shutdown 2024-09-21T06:23:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.248+0000 7f82f865c740 1 -- 172.21.15.3:0/1278177497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 msgr2=0x55cb98e9fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.248+0000 7f82f865c740 1 --2- 172.21.15.3:0/1278177497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98e9fd30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0e51c0 tx=0x7f82dc0e5740 comp rx=0 tx=0).stop 2024-09-21T06:23:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.249+0000 7f82f865c740 1 -- 172.21.15.3:0/1278177497 shutdown_connections 2024-09-21T06:23:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.249+0000 7f82f865c740 1 --2- 172.21.15.3:0/1278177497 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55cb98aab9f0 0x55cb98eb2010 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.249+0000 7f82f865c740 1 --2- 172.21.15.3:0/1278177497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98acc390 0x55cb98ea0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.249+0000 7f82caffd640 0 client.27422 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.249+0000 7f82f865c740 1 --2- 172.21.15.3:0/1278177497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aed070 0x55cb98eac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.249+0000 7f82f865c740 1 --2- 172.21.15.3:0/1278177497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98e9fd30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.249+0000 7f82f865c740 1 -- 172.21.15.3:0/1278177497 >> 172.21.15.3:0/1278177497 conn(0x55cb98ae1460 msgr2=0x55cb98e977e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.249+0000 7f82f865c740 1 -- 172.21.15.3:0/1278177497 shutdown_connections 2024-09-21T06:23:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.249+0000 7f82f865c740 1 -- 172.21.15.3:0/1278177497 wait complete. 2024-09-21T06:23:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.249+0000 7f82f865c740 20 client.27422 trim_cache size 0 max 16384 2024-09-21T06:23:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.267+0000 7f82f865c740 1 Processor -- start 2024-09-21T06:23:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.267+0000 7f82f865c740 1 -- start start 2024-09-21T06:23:25.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.267+0000 7f82f865c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aab9f0 0x55cb98e96040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.267+0000 7f82f865c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98ea1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.267+0000 7f82f865c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98acc390 0x55cb98e91b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.267+0000 7f82f865c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec1ad530 con 0x55cb98aab9f0 2024-09-21T06:23:25.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.267+0000 7f82f865c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec1af070 con 0x55cb98ac41b0 2024-09-21T06:23:25.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.267+0000 7f82f865c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec1adc00 con 0x55cb98acc390 2024-09-21T06:23:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.267+0000 7f82eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aab9f0 0x55cb98e96040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.267+0000 7f82eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aab9f0 0x55cb98e96040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48726/0 (socket says 172.21.15.3:48726) 2024-09-21T06:23:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.267+0000 7f82eb7fe640 1 -- 172.21.15.3:0/282324475 learned_addr learned my addr 172.21.15.3:0/282324475 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.267+0000 7f82eaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98ea1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.267+0000 7f82ebfff640 1 --2- 172.21.15.3:0/282324475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98acc390 0x55cb98e91b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.268+0000 7f82eb7fe640 1 -- 172.21.15.3:0/282324475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98acc390 msgr2=0x55cb98e91b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:25.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.268+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/282324475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98acc390 0x55cb98e91b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.268+0000 7f82eb7fe640 1 -- 172.21.15.3:0/282324475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 msgr2=0x55cb98ea1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:25.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.268+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/282324475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98ea1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.268+0000 7f82eb7fe640 1 -- 172.21.15.3:0/282324475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb98ea0ff0 con 0x55cb98aab9f0 2024-09-21T06:23:25.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.268+0000 7f82eaffd640 1 --2- 172.21.15.3:0/282324475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98ea1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:25.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.268+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/282324475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aab9f0 0x55cb98e96040 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0aab30 tx=0x7f82dc003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c1de492ccd54c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:25.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.268+0000 7f82d17fa640 1 -- 172.21.15.3:0/282324475 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0ad210 con 0x55cb98aab9f0 2024-09-21T06:23:25.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.268+0000 7f82d17fa640 1 -- 172.21.15.3:0/282324475 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82dc0b23a0 con 0x55cb98aab9f0 2024-09-21T06:23:25.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.268+0000 7f82d17fa640 1 -- 172.21.15.3:0/282324475 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0a9760 con 0x55cb98aab9f0 2024-09-21T06:23:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.269+0000 7f82f865c740 1 -- 172.21.15.3:0/282324475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aab9f0 msgr2=0x55cb98e96040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.269+0000 7f82f865c740 1 --2- 172.21.15.3:0/282324475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aab9f0 0x55cb98e96040 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0aab30 tx=0x7f82dc003cd0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.269+0000 7f82f865c740 1 -- 172.21.15.3:0/282324475 shutdown_connections 2024-09-21T06:23:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.269+0000 7f82f865c740 1 --2- 172.21.15.3:0/282324475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98acc390 0x55cb98e91b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.269+0000 7f82f865c740 1 --2- 172.21.15.3:0/282324475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98ac41b0 0x55cb98ea1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.269+0000 7f82f865c740 1 --2- 172.21.15.3:0/282324475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98aab9f0 0x55cb98e96040 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.269+0000 7f82f865c740 1 -- 172.21.15.3:0/282324475 >> 172.21.15.3:0/282324475 conn(0x55cb98ae1460 msgr2=0x55cb98ea1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:25.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.269+0000 7f82f865c740 1 -- 172.21.15.3:0/282324475 shutdown_connections 2024-09-21T06:23:25.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.269+0000 7f82f865c740 1 -- 172.21.15.3:0/282324475 wait complete. 2024-09-21T06:23:25.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.270+0000 7f82f865c740 1 -- start start 2024-09-21T06:23:25.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.270+0000 7f82f865c740 1 Processor -- start 2024-09-21T06:23:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.270+0000 7f82f865c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98aab9f0 0x55cb98ea84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.270+0000 7f82f865c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98ac41b0 0x55cb98ea8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82f865c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98acc390 0x55cb98ea3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82f865c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec1ae9a0 con 0x55cb98ac41b0 2024-09-21T06:23:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82f865c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec1ae2d0 con 0x55cb98acc390 2024-09-21T06:23:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82f865c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec1ab530 con 0x55cb98aab9f0 2024-09-21T06:23:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82f865c740 10 client.0 _ll_register_callbacks cb 0x7f82dc0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82eaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98ac41b0 0x55cb98ea8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82eaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98ac41b0 0x55cb98ea8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48742/0 (socket says 172.21.15.3:48742) 2024-09-21T06:23:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82eaffd640 1 -- 172.21.15.3:0/681706004 learned_addr learned my addr 172.21.15.3:0/681706004 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82ebfff640 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98acc390 0x55cb98ea3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98aab9f0 0x55cb98ea84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82eaffd640 1 -- 172.21.15.3:0/681706004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98aab9f0 msgr2=0x55cb98ea84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:25.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82eaffd640 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98aab9f0 0x55cb98ea84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82eaffd640 1 -- 172.21.15.3:0/681706004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98acc390 msgr2=0x55cb98ea3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:25.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82eaffd640 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98acc390 0x55cb98ea3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82eaffd640 1 -- 172.21.15.3:0/681706004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb98ea4300 con 0x55cb98ac41b0 2024-09-21T06:23:25.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.271+0000 7f82ebfff640 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98acc390 0x55cb98ea3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:25.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.272+0000 7f82eaffd640 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98ac41b0 0x55cb98ea8a60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009900 tx=0x7f82e0032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81933f8fce42a98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:25.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.272+0000 7f82caffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:25.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.272+0000 7f82caffd640 1 -- 172.21.15.3:0/681706004 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0035040 con 0x55cb98ac41b0 2024-09-21T06:23:25.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.272+0000 7f82caffd640 1 -- 172.21.15.3:0/681706004 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f82e0004040 con 0x55cb98ac41b0 2024-09-21T06:23:25.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.272+0000 7f82f865c740 1 -- 172.21.15.3:0/681706004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cb98e08850 con 0x55cb98ac41b0 2024-09-21T06:23:25.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.272+0000 7f82caffd640 1 -- 172.21.15.3:0/681706004 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f82e0067050 con 0x55cb98ac41b0 2024-09-21T06:23:25.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.272+0000 7f82f865c740 10 client.17868 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:25.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.272+0000 7f82f865c740 1 -- 172.21.15.3:0/681706004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cb98f1c600 con 0x55cb98ac41b0 2024-09-21T06:23:25.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.272+0000 7f82caffd640 1 -- 172.21.15.3:0/681706004 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cb98e08850 con 0x55cb98ac41b0 2024-09-21T06:23:25.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.272+0000 7f82caffd640 1 -- 172.21.15.3:0/681706004 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f82e00035f0 con 0x55cb98ac41b0 2024-09-21T06:23:25.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.272+0000 7f82f865c740 10 client.17868 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:25.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.272+0000 7f82f865c740 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:25.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.272+0000 7f82f865c740 20 client.17868 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:25.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.273+0000 7f82f865c740 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55cb98aed070 0x55cb98f1e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.273+0000 7f82f865c740 4 client.17868 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:23:25.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.273+0000 7f82f865c740 1 -- 172.21.15.3:0/681706004 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cb98e05c80 con 0x55cb98aed070 2024-09-21T06:23:25.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.273+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55cb98aed070 0x55cb98f1e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.273+0000 7f82eb7fe640 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55cb98aed070 0x55cb98f1e950 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bd1f8321cb50b4b server_cookie=f118d3ede5a3d63d in_seq=0 out_seq=0 2024-09-21T06:23:25.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.273+0000 7f82caffd640 10 client.17868 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:25.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.273+0000 7f82caffd640 1 -- 172.21.15.3:0/681706004 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x55cb98e05c80 con 0x55cb98aed070 2024-09-21T06:23:25.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.273+0000 7f82caffd640 10 client.17868 handle_command_reply: tid=0 2024-09-21T06:23:25.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.274+0000 7f82f865c740 1 client.17868 shutdown 2024-09-21T06:23:25.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.274+0000 7f82f865c740 1 -- 172.21.15.3:0/681706004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98ac41b0 msgr2=0x55cb98ea8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.274+0000 7f82f865c740 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98ac41b0 0x55cb98ea8a60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009900 tx=0x7f82e0032f90 comp rx=0 tx=0).stop 2024-09-21T06:23:25.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.274+0000 7f82f865c740 1 -- 172.21.15.3:0/681706004 shutdown_connections 2024-09-21T06:23:25.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.274+0000 7f82f865c740 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55cb98aed070 0x55cb98f1e950 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.274+0000 7f82caffd640 0 client.17868 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:25.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.274+0000 7f82f865c740 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb98acc390 0x55cb98ea3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.274+0000 7f82f865c740 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cb98ac41b0 0x55cb98ea8a60 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.274+0000 7f82f865c740 1 --2- 172.21.15.3:0/681706004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb98aab9f0 0x55cb98ea84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.274+0000 7f82f865c740 1 -- 172.21.15.3:0/681706004 >> 172.21.15.3:0/681706004 conn(0x55cb98ae1460 msgr2=0x55cb98e95e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:25.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.274+0000 7f82f865c740 1 -- 172.21.15.3:0/681706004 shutdown_connections 2024-09-21T06:23:25.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.274+0000 7f82f865c740 1 -- 172.21.15.3:0/681706004 wait complete. 2024-09-21T06:23:25.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.274+0000 7f82f865c740 20 client.17868 trim_cache size 0 max 16384 2024-09-21T06:23:25.291 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:23:25.291 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:23:25.291 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:23:25.292 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:23:25.292 INFO:teuthology.orchestra.run.smithi003.stdout: "535fb164-fd4f-4d88-b7e0-575559314385": { 2024-09-21T06:23:25.292 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:51.152572+0000", 2024-09-21T06:23:25.292 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:25.292 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:25.292 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:25.292 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:25.293 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:25.293 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:25.293 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:23:25.293 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:23:25.293 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:25.293 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:25.294 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:25.294 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:25.294 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:25.294 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:25.294 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:23:25.294 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:23:25.294 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:25.295 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:25.295 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:25.295 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:25.295 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:25.295 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:25.295 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:23:25.296 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:23:25.296 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:25.296 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:25.296 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:25.296 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:25.296 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:25.296 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:25.297 INFO:teuthology.orchestra.run.smithi003.stdout: "714b9934-a2d3-4a78-b185-01254776061b": { 2024-09-21T06:23:25.297 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:04.685380+0000", 2024-09-21T06:23:25.297 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:25.297 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:25.297 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:25.297 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:25.298 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:25.298 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:25.298 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:23:25.298 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:23:25.298 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:25.298 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:25.299 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:25.299 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:25.299 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:25.299 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:25.299 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:23:25.299 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:23:25.299 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:25.300 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:25.300 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:25.300 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:25.300 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:25.300 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:25.300 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:23:25.301 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:23:25.301 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:25.301 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:25.301 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:25.301 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:25.301 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:25.301 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:25.302 INFO:teuthology.orchestra.run.smithi003.stdout: "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58": { 2024-09-21T06:23:25.302 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:21:17.396219+0000", 2024-09-21T06:23:25.302 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:25.302 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:25.302 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:25.302 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:25.302 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:25.303 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:25.303 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:23:25.303 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:23:25.303 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:25.303 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:25.303 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:25.304 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:25.304 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:25.304 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:25.304 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:23:25.304 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:23:25.304 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:25.305 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:25.305 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:25.305 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:25.305 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:25.305 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:25.305 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:23:25.305 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:23:25.305 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:25.306 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:25.306 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:25.306 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:25.306 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:25.306 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:23:25.306 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:23:25.306 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:23:25.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.275+0000 7f82f20e8640 1 -- 172.21.15.3:0/3272913901 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82d40b6680 msgr2=0x7f82d40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.275+0000 7f82f20e8640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82d40b6680 0x7f82d40b8b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005f40 tx=0x7f82e40096a0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.275+0000 7f82f20e8640 1 -- 172.21.15.3:0/3272913901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ec1049b0 msgr2=0x7f82ec19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.275+0000 7f82f20e8640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ec1049b0 0x7f82ec19a4a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00e9c0 tx=0x7f82dc00ee80 comp rx=0 tx=0).stop 2024-09-21T06:23:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.275+0000 7f82f20e8640 1 -- 172.21.15.3:0/3272913901 shutdown_connections 2024-09-21T06:23:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.275+0000 7f82f20e8640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f82d40b6680 0x7f82d40b8b90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.275+0000 7f82f20e8640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82ec105300 0x7f82ec19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.275+0000 7f82f20e8640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82ec10de10 0x7f82ec19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.275+0000 7f82f20e8640 1 --2- 172.21.15.3:0/3272913901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ec1049b0 0x7f82ec19a4a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.275+0000 7f82f20e8640 1 -- 172.21.15.3:0/3272913901 >> 172.21.15.3:0/3272913901 conn(0x7f82ec079730 msgr2=0x7f82ec100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.275+0000 7f82f20e8640 1 -- 172.21.15.3:0/3272913901 shutdown_connections 2024-09-21T06:23:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:25.275+0000 7f82f20e8640 1 -- 172.21.15.3:0/3272913901 wait complete. 2024-09-21T06:23:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:25 smithi003 ceph-mon[31227]: pgmap v583: 97 pgs: 97 active+clean; 130 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-09-21T06:23:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:25 smithi137 ceph-mon[34464]: pgmap v583: 97 pgs: 97 active+clean; 130 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-09-21T06:23:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:25 smithi204 ceph-mon[34659]: pgmap v583: 97 pgs: 97 active+clean; 130 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-09-21T06:23:25.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 878020 10.60 MB/sec execute 298 sec latency 36.919 ms 2024-09-21T06:23:26.454 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:23:26.454 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:23:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:26.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 880436 10.62 MB/sec execute 299 sec latency 19.501 ms 2024-09-21T06:23:27.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.247+0000 7fd0c18ad640 1 -- 172.21.15.3:0/988751673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0bc108480 msgr2=0x7fd0bc10ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.247+0000 7fd0c18ad640 1 --2- 172.21.15.3:0/988751673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0bc108480 0x7fd0bc10ecb0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009ca0 tx=0x7fd0b402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.247+0000 7fd0c18ad640 1 -- 172.21.15.3:0/988751673 shutdown_connections 2024-09-21T06:23:27.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.247+0000 7fd0c18ad640 1 --2- 172.21.15.3:0/988751673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0bc108480 0x7fd0bc10ecb0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.247+0000 7fd0c18ad640 1 --2- 172.21.15.3:0/988751673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0bc107b30 0x7fd0bc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.247+0000 7fd0c18ad640 1 --2- 172.21.15.3:0/988751673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0bc1021f0 0x7fd0bc102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.247+0000 7fd0c18ad640 1 -- 172.21.15.3:0/988751673 >> 172.21.15.3:0/988751673 conn(0x7fd0bc077c60 msgr2=0x7fd0bc0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:27.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.247+0000 7fd0c18ad640 1 -- 172.21.15.3:0/988751673 shutdown_connections 2024-09-21T06:23:27.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.247+0000 7fd0c18ad640 1 -- 172.21.15.3:0/988751673 wait complete. 2024-09-21T06:23:27.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.248+0000 7fd0c18ad640 1 Processor -- start 2024-09-21T06:23:27.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.248+0000 7fd0c18ad640 1 -- start start 2024-09-21T06:23:27.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.248+0000 7fd0c18ad640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0bc1021f0 0x7fd0bc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.248+0000 7fd0c18ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0bc107b30 0x7fd0bc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.248+0000 7fd0c18ad640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0bc108480 0x7fd0bc19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.248+0000 7fd0c18ad640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1118c0 con 0x7fd0bc107b30 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.248+0000 7fd0c18ad640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc111740 con 0x7fd0bc1021f0 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.248+0000 7fd0c18ad640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc111a40 con 0x7fd0bc108480 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0ba7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0bc107b30 0x7fd0bc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0baffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0bc1021f0 0x7fd0bc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0ba7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0bc107b30 0x7fd0bc19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48764/0 (socket says 172.21.15.3:48764) 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0baffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0bc1021f0 0x7fd0bc19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:33200/0 (socket says 172.21.15.3:33200) 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0ba7fc640 1 -- 172.21.15.3:0/1342550833 learned_addr learned my addr 172.21.15.3:0/1342550833 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0bb7fe640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0bc108480 0x7fd0bc19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0ba7fc640 1 -- 172.21.15.3:0/1342550833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0bc108480 msgr2=0x7fd0bc19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0ba7fc640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0bc108480 0x7fd0bc19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0ba7fc640 1 -- 172.21.15.3:0/1342550833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0bc1021f0 msgr2=0x7fd0bc19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0ba7fc640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0bc1021f0 0x7fd0bc19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0ba7fc640 1 -- 172.21.15.3:0/1342550833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc1a0390 con 0x7fd0bc107b30 2024-09-21T06:23:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0baffd640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0bc1021f0 0x7fd0bc19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0ba7fc640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0bc107b30 0x7fd0bc19a9c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000ea90 tx=0x7fd0b000ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0c08ab640 1 -- 172.21.15.3:0/1342550833 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b001b070 con 0x7fd0bc107b30 2024-09-21T06:23:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0c18ad640 1 -- 172.21.15.3:0/1342550833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0bc1a0680 con 0x7fd0bc107b30 2024-09-21T06:23:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0c08ab640 1 -- 172.21.15.3:0/1342550833 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b000cce0 con 0x7fd0bc107b30 2024-09-21T06:23:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0c08ab640 1 -- 172.21.15.3:0/1342550833 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b000b5c0 con 0x7fd0bc107b30 2024-09-21T06:23:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.249+0000 7fd0c18ad640 1 -- 172.21.15.3:0/1342550833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0bc1a6de0 con 0x7fd0bc107b30 2024-09-21T06:23:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.250+0000 7fd0c18ad640 1 -- 172.21.15.3:0/1342550833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0bc102600 con 0x7fd0bc107b30 2024-09-21T06:23:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.252+0000 7fd0c08ab640 1 -- 172.21.15.3:0/1342550833 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fd0b001e070 con 0x7fd0bc107b30 2024-09-21T06:23:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.253+0000 7fd0c08ab640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0a40b6680 0x7fd0a40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.253+0000 7fd0baffd640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0a40b6680 0x7fd0a40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.253+0000 7fd0c08ab640 1 -- 172.21.15.3:0/1342550833 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7fd0b00e76c0 con 0x7fd0bc107b30 2024-09-21T06:23:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.254+0000 7fd0baffd640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0a40b6680 0x7fd0a40b8b90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009fc0 tx=0x7fd0ac005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:27.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.256+0000 7fd0c08ab640 1 -- 172.21.15.3:0/1342550833 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd0b00b2000 con 0x7fd0bc107b30 2024-09-21T06:23:27.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.486+0000 7fd0c18ad640 1 -- 172.21.15.3:0/1342550833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0bc19b550 con 0x7fd0bc107b30 2024-09-21T06:23:27.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.487+0000 7fd0c08ab640 1 -- 172.21.15.3:0/1342550833 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11965 (secure 0 0 0) 0x7fd0bc19b550 con 0x7fd0bc107b30 2024-09-21T06:23:27.490 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:23:27.490 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":37,"btime":"2024-09-21T06:22:50:861894+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:22:50.861867+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27344":{"gid":27344,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3457686047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3457686047},{"type":"v1","addr":"172.21.15.137:6835","nonce":3457686047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:23:27.490 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 37 2024-09-21T06:23:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.490+0000 7fd0c18ad640 1 -- 172.21.15.3:0/1342550833 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0a40b6680 msgr2=0x7fd0a40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.490+0000 7fd0c18ad640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0a40b6680 0x7fd0a40b8b90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009fc0 tx=0x7fd0ac005f90 comp rx=0 tx=0).stop 2024-09-21T06:23:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.490+0000 7fd0c18ad640 1 -- 172.21.15.3:0/1342550833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0bc107b30 msgr2=0x7fd0bc19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.490+0000 7fd0c18ad640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0bc107b30 0x7fd0bc19a9c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000ea90 tx=0x7fd0b000ef50 comp rx=0 tx=0).stop 2024-09-21T06:23:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.490+0000 7fd0c18ad640 1 -- 172.21.15.3:0/1342550833 shutdown_connections 2024-09-21T06:23:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.490+0000 7fd0c18ad640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd0a40b6680 0x7fd0a40b8b90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.490+0000 7fd0c18ad640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0bc108480 0x7fd0bc19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.490+0000 7fd0c18ad640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0bc107b30 0x7fd0bc19a9c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.490+0000 7fd0c18ad640 1 --2- 172.21.15.3:0/1342550833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0bc1021f0 0x7fd0bc19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.490+0000 7fd0c18ad640 1 -- 172.21.15.3:0/1342550833 >> 172.21.15.3:0/1342550833 conn(0x7fd0bc077c60 msgr2=0x7fd0bc103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.490+0000 7fd0c18ad640 1 -- 172.21.15.3:0/1342550833 shutdown_connections 2024-09-21T06:23:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:27.490+0000 7fd0c18ad640 1 -- 172.21.15.3:0/1342550833 wait complete. 2024-09-21T06:23:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:27 smithi003 ceph-mon[31227]: pgmap v584: 97 pgs: 97 active+clean; 130 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-09-21T06:23:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:23:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:23:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:23:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:23:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:27 smithi137 ceph-mon[34464]: pgmap v584: 97 pgs: 97 active+clean; 130 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-09-21T06:23:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:23:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:23:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:23:27.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:23:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:27 smithi204 ceph-mon[34659]: pgmap v584: 97 pgs: 97 active+clean; 130 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-09-21T06:23:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:23:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:23:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:23:27.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:23:27.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 880943 10.59 MB/sec execute 300 sec latency 68.798 ms 2024-09-21T06:23:28.216 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:23:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:28.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:28 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1342550833' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:23:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:28 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1342550833' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:23:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:28 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1342550833' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:23:28.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 882894 10.58 MB/sec execute 301 sec latency 28.655 ms 2024-09-21T06:23:29.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.006+0000 7f57bbfb0640 1 -- 172.21.15.3:0/1584422804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b4108d00 msgr2=0x7f57b410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.006+0000 7f57bbfb0640 1 --2- 172.21.15.3:0/1584422804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b4108d00 0x7f57b410f530 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009ca0 tx=0x7f57ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.006+0000 7f57bbfb0640 1 -- 172.21.15.3:0/1584422804 shutdown_connections 2024-09-21T06:23:29.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.006+0000 7f57bbfb0640 1 --2- 172.21.15.3:0/1584422804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b4108d00 0x7f57b410f530 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.006+0000 7f57bbfb0640 1 --2- 172.21.15.3:0/1584422804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57b41083b0 0x7f57b41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.006+0000 7f57bbfb0640 1 --2- 172.21.15.3:0/1584422804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57b4102ac0 0x7f57b4102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.006+0000 7f57bbfb0640 1 -- 172.21.15.3:0/1584422804 >> 172.21.15.3:0/1584422804 conn(0x7f57b4079720 msgr2=0x7f57b4079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.006+0000 7f57bbfb0640 1 -- 172.21.15.3:0/1584422804 shutdown_connections 2024-09-21T06:23:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.006+0000 7f57bbfb0640 1 -- 172.21.15.3:0/1584422804 wait complete. 2024-09-21T06:23:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.007+0000 7f57bbfb0640 1 Processor -- start 2024-09-21T06:23:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.007+0000 7f57bbfb0640 1 -- start start 2024-09-21T06:23:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57bbfb0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57b4102ac0 0x7f57b4199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57bbfb0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41083b0 0x7f57b4199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57bbfb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57b4108d00 0x7f57b419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57bbfb0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41129c0 con 0x7f57b41083b0 2024-09-21T06:23:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57bbfb0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f57b4112840 con 0x7f57b4108d00 2024-09-21T06:23:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57bbfb0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57b4112b40 con 0x7f57b4102ac0 2024-09-21T06:23:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57b954b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41083b0 0x7f57b4199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57b954b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41083b0 0x7f57b4199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48778/0 (socket says 172.21.15.3:48778) 2024-09-21T06:23:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57b954b640 1 -- 172.21.15.3:0/2833568713 learned_addr learned my addr 172.21.15.3:0/2833568713 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57ba54d640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57b4108d00 0x7f57b419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57b9d4c640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57b4102ac0 0x7f57b4199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57b954b640 1 -- 172.21.15.3:0/2833568713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57b4102ac0 msgr2=0x7f57b4199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57b954b640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57b4102ac0 0x7f57b4199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57b954b640 1 -- 172.21.15.3:0/2833568713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57b4108d00 msgr2=0x7f57b419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57b954b640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57b4108d00 0x7f57b419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.008+0000 7f57b954b640 1 -- 172.21.15.3:0/2833568713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b419f3e0 con 0x7f57b41083b0 2024-09-21T06:23:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.009+0000 7f57b954b640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41083b0 0x7f57b4199a10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f57a400e9f0 tx=0x7f57a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.009+0000 7f57aaffd640 1 -- 172.21.15.3:0/2833568713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a400ccc0 con 0x7f57b41083b0 2024-09-21T06:23:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.009+0000 7f57bbfb0640 1 -- 172.21.15.3:0/2833568713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b419f6d0 con 0x7f57b41083b0 2024-09-21T06:23:29.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.009+0000 7f57aaffd640 1 -- 172.21.15.3:0/2833568713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a40108e0 con 0x7f57b41083b0 2024-09-21T06:23:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.009+0000 7f57aaffd640 1 -- 172.21.15.3:0/2833568713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4018860 con 0x7f57b41083b0 2024-09-21T06:23:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.009+0000 7f57bbfb0640 1 -- 172.21.15.3:0/2833568713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b41a5e30 con 0x7f57b41083b0 2024-09-21T06:23:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57c2524740 1 Processor -- start 2024-09-21T06:23:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57aaffd640 1 -- 172.21.15.3:0/2833568713 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f57a4010430 con 0x7f57b41083b0 2024-09-21T06:23:29.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57c2524740 1 -- start start 2024-09-21T06:23:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57c2524740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f35070 0x5575072dc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57c2524740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072dcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57c2524740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f14390 0x5575072dd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57c2524740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557507252ea0 con 0x557506f0c1b0 2024-09-21T06:23:29.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57c2524740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5575072422d0 con 0x557506f14390 2024-09-21T06:23:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57c2524740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5575072460e0 con 0x557506f35070 2024-09-21T06:23:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57b954b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f14390 0x5575072dd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57ba54d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f35070 0x5575072dc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57b9d4c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072dcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57ba54d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f35070 0x5575072dc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:45620/0 (socket says 172.21.15.3:45620) 2024-09-21T06:23:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57ba54d640 1 -- 172.21.15.3:0/3321489944 learned_addr learned my addr 172.21.15.3:0/3321489944 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57b9d4c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072dcbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48784/0 (socket says 172.21.15.3:48784) 2024-09-21T06:23:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.012+0000 7f57b954b640 1 --2- 172.21.15.3:0/3321489944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f14390 0x5575072dd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:33244/0 (socket says 172.21.15.3:33244) 2024-09-21T06:23:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.013+0000 7f57ba54d640 1 -- 172.21.15.3:0/3321489944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f14390 msgr2=0x5575072dd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.013+0000 7f57ba54d640 1 --2- 172.21.15.3:0/3321489944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f14390 0x5575072dd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.013+0000 7f57ba54d640 1 -- 172.21.15.3:0/3321489944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 msgr2=0x5575072dcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.013+0000 7f57ba54d640 1 --2- 172.21.15.3:0/3321489944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072dcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.013+0000 7f57ba54d640 1 -- 172.21.15.3:0/3321489944 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557506fa0650 con 0x557506f35070 2024-09-21T06:23:29.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.013+0000 7f57b9d4c640 1 --2- 172.21.15.3:0/3321489944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072dcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:29.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.013+0000 7f57aaffd640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f57a00b6680 0x7f57a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.013+0000 7f57ba54d640 1 --2- 172.21.15.3:0/3321489944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f35070 0x5575072dc6a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f57ac02f410 tx=0x7f57ac02f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac9d578a6950d85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.013+0000 7f57b9d4c640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f57a00b6680 0x7f57a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.013+0000 7f57aaffd640 1 -- 172.21.15.3:0/2833568713 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f57a40e7940 con 0x7f57b41083b0 2024-09-21T06:23:29.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.013+0000 7f579ffff640 1 -- 172.21.15.3:0/3321489944 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac031cf0 con 0x557506f35070 2024-09-21T06:23:29.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.013+0000 7f579ffff640 1 -- 172.21.15.3:0/3321489944 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57ac036030 con 0x557506f35070 2024-09-21T06:23:29.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.014+0000 7f579ffff640 1 -- 172.21.15.3:0/3321489944 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0042a0 con 0x557506f35070 2024-09-21T06:23:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.014+0000 7f57b9d4c640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f57a00b6680 0x7f57a00b8b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x5575072db320 tx=0x7f57b0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.014+0000 7f57c2524740 1 -- 172.21.15.3:0/3321489944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f35070 msgr2=0x5575072dc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.014+0000 7f57c2524740 1 --2- 172.21.15.3:0/3321489944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f35070 0x5575072dc6a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f57ac02f410 tx=0x7f57ac02f440 comp rx=0 tx=0).stop 2024-09-21T06:23:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.015+0000 7f57c2524740 1 -- 172.21.15.3:0/3321489944 shutdown_connections 2024-09-21T06:23:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.015+0000 7f57c2524740 1 --2- 172.21.15.3:0/3321489944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f14390 0x5575072dd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.015+0000 7f57c2524740 1 --2- 172.21.15.3:0/3321489944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072dcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.015+0000 7f57c2524740 1 --2- 172.21.15.3:0/3321489944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f35070 0x5575072dc6a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.015+0000 7f57c2524740 1 -- 172.21.15.3:0/3321489944 >> 172.21.15.3:0/3321489944 conn(0x557506f29460 msgr2=0x5575072da590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.015+0000 7f57c2524740 1 -- 172.21.15.3:0/3321489944 shutdown_connections 2024-09-21T06:23:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.015+0000 7f57c2524740 1 -- 172.21.15.3:0/3321489944 wait complete. 2024-09-21T06:23:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.015+0000 7f57c2524740 1 -- start start 2024-09-21T06:23:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.015+0000 7f57c2524740 1 Processor -- start 2024-09-21T06:23:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57c2524740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57c2524740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f14390 0x5575072e8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57c2524740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f35070 0x5575072f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57c2524740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5575070a3700 con 0x557506f0c1b0 2024-09-21T06:23:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57c2524740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5575071dd160 con 0x557506f35070 2024-09-21T06:23:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57c2524740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5575072eceb0 con 0x557506f14390 2024-09-21T06:23:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57b9d4c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57c2524740 10 client.0 _ll_register_callbacks cb 0x5575071f69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57b954b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f14390 0x5575072e8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57ba54d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f35070 0x5575072f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57b954b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f14390 0x5575072e8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:45632/0 (socket says 172.21.15.3:45632) 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57b954b640 1 -- 172.21.15.3:0/604004654 learned_addr learned my addr 172.21.15.3:0/604004654 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57ba54d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f35070 0x5575072f61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:33248/0 (socket says 172.21.15.3:33248) 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57b9d4c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072e7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48796/0 (socket says 172.21.15.3:48796) 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57b954b640 1 -- 172.21.15.3:0/604004654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f35070 msgr2=0x5575072f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57b954b640 1 --2- 172.21.15.3:0/604004654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f35070 0x5575072f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57b954b640 1 -- 172.21.15.3:0/604004654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 msgr2=0x5575072e7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57b954b640 1 --2- 172.21.15.3:0/604004654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072e7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57b954b640 1 -- 172.21.15.3:0/604004654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55750720ac90 con 0x557506f14390 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.016+0000 7f57ba54d640 1 --2- 172.21.15.3:0/604004654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f35070 0x5575072f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f57b954b640 1 --2- 172.21.15.3:0/604004654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f14390 0x5575072e8300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57a40e5100 tx=0x7f57a400ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c32a7e1a28d31a32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f579d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f579d7fa640 1 -- 172.21.15.3:0/604004654 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a40a96f0 con 0x557506f14390 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f579d7fa640 1 -- 172.21.15.3:0/604004654 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f57a40a91a0 con 0x557506f14390 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f57c2524740 1 -- 172.21.15.3:0/604004654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57ac036030 con 0x557506f14390 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f579d7fa640 1 -- 172.21.15.3:0/604004654 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f57a40acfc0 con 0x557506f14390 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f57c2524740 10 client.27394 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f57c2524740 1 -- 172.21.15.3:0/604004654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5575072f8c50 con 0x557506f14390 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f579d7fa640 1 -- 172.21.15.3:0/604004654 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57ac036030 con 0x557506f14390 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f579d7fa640 1 -- 172.21.15.3:0/604004654 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f57a40a93a0 con 0x557506f14390 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f57c2524740 10 client.27394 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f57c2524740 10 client.27394 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f57c2524740 20 client.27394 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f57c2524740 1 --2- 172.21.15.3:0/604004654 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x557506ef39f0 0x5575072fc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f57c2524740 4 client.27394 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:29.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.017+0000 7f57c2524740 1 -- 172.21.15.3:0/604004654 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557507254df0 con 0x557506ef39f0 2024-09-21T06:23:29.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.018+0000 7f57b9d4c640 1 --2- 172.21.15.3:0/604004654 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x557506ef39f0 0x5575072fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.018+0000 7f579d7fa640 10 client.27394 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:23:29.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.018+0000 7f57b9d4c640 1 --2- 172.21.15.3:0/604004654 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x557506ef39f0 0x5575072fc120 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b21132bf45f36db3 server_cookie=9e7a97f40b347848 in_seq=0 out_seq=0 2024-09-21T06:23:29.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f579d7fa640 1 -- 172.21.15.3:0/604004654 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557507254df0 con 0x557506ef39f0 2024-09-21T06:23:29.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f579d7fa640 10 client.27394 handle_command_reply: tid=0 2024-09-21T06:23:29.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f57c2524740 1 client.27394 shutdown 2024-09-21T06:23:29.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f57c2524740 1 -- 172.21.15.3:0/604004654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f14390 msgr2=0x5575072e8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f57c2524740 1 --2- 172.21.15.3:0/604004654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f14390 0x5575072e8300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57a40e5100 tx=0x7f57a400ce60 comp rx=0 tx=0).stop 2024-09-21T06:23:29.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f57c2524740 1 -- 172.21.15.3:0/604004654 shutdown_connections 2024-09-21T06:23:29.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f57c2524740 1 --2- 172.21.15.3:0/604004654 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x557506ef39f0 0x5575072fc120 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f57c2524740 1 --2- 172.21.15.3:0/604004654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f14390 0x5575072e8300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f57c2524740 1 --2- 172.21.15.3:0/604004654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f35070 0x5575072f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f57c2524740 1 --2- 172.21.15.3:0/604004654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072e7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f579d7fa640 0 client.27394 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:23:29.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f57c2524740 1 -- 172.21.15.3:0/604004654 >> 172.21.15.3:0/604004654 conn(0x557506f29460 msgr2=0x5575072df830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f57c2524740 1 -- 172.21.15.3:0/604004654 shutdown_connections 2024-09-21T06:23:29.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f57c2524740 1 -- 172.21.15.3:0/604004654 wait complete. 2024-09-21T06:23:29.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.019+0000 7f57c2524740 20 client.27394 trim_cache size 0 max 16384 2024-09-21T06:23:29.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.033+0000 7f57c2524740 1 Processor -- start 2024-09-21T06:23:29.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.033+0000 7f57c2524740 1 -- start start 2024-09-21T06:23:29.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57c2524740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f0c1b0 0x5575072e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57c2524740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f14390 0x5575072ddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57c2524740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f35070 0x5575072f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57c2524740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41ad540 con 0x557506f35070 2024-09-21T06:23:29.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57c2524740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41af080 con 0x557506f14390 2024-09-21T06:23:29.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57c2524740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41adc10 con 0x557506f0c1b0 2024-09-21T06:23:29.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57ba54d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f35070 0x5575072f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57b954b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f14390 0x5575072ddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57ba54d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f35070 0x5575072f60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48798/0 (socket says 172.21.15.3:48798) 2024-09-21T06:23:29.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57ba54d640 1 -- 172.21.15.3:0/3563400435 learned_addr learned my addr 172.21.15.3:0/3563400435 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:29.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57b9d4c640 1 --2- 172.21.15.3:0/3563400435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f0c1b0 0x5575072e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57b954b640 1 -- 172.21.15.3:0/3563400435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f0c1b0 msgr2=0x5575072e6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57b954b640 1 --2- 172.21.15.3:0/3563400435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f0c1b0 0x5575072e6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57b954b640 1 -- 172.21.15.3:0/3563400435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f35070 msgr2=0x5575072f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57b954b640 1 --2- 172.21.15.3:0/3563400435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f35070 0x5575072f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57b954b640 1 -- 172.21.15.3:0/3563400435 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575072df500 con 0x557506f14390 2024-09-21T06:23:29.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57ba54d640 1 --2- 172.21.15.3:0/3563400435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f35070 0x5575072f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:29.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.034+0000 7f57b954b640 1 --2- 172.21.15.3:0/3563400435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f14390 0x5575072ddde0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f57a40ac520 tx=0x7f57a401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c0ead613ef77187 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.035+0000 7f579f7fe640 1 -- 172.21.15.3:0/3563400435 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a40ad1a0 con 0x557506f14390 2024-09-21T06:23:29.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.035+0000 7f579f7fe640 1 -- 172.21.15.3:0/3563400435 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a4010720 con 0x557506f14390 2024-09-21T06:23:29.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.035+0000 7f579f7fe640 1 -- 172.21.15.3:0/3563400435 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a40ad620 con 0x557506f14390 2024-09-21T06:23:29.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.035+0000 7f57c2524740 1 -- 172.21.15.3:0/3563400435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f14390 msgr2=0x5575072ddde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.035+0000 7f57c2524740 1 --2- 172.21.15.3:0/3563400435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f14390 0x5575072ddde0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f57a40ac520 tx=0x7f57a401f000 comp rx=0 tx=0).stop 2024-09-21T06:23:29.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.035+0000 7f57c2524740 1 -- 172.21.15.3:0/3563400435 shutdown_connections 2024-09-21T06:23:29.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.035+0000 7f57c2524740 1 --2- 172.21.15.3:0/3563400435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f35070 0x5575072f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.036+0000 7f57c2524740 1 --2- 172.21.15.3:0/3563400435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f14390 0x5575072ddde0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.036+0000 7f57c2524740 1 --2- 172.21.15.3:0/3563400435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f0c1b0 0x5575072e6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.036+0000 7f57c2524740 1 -- 172.21.15.3:0/3563400435 >> 172.21.15.3:0/3563400435 conn(0x557506f29460 msgr2=0x5575072dd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.036+0000 7f57c2524740 1 -- 172.21.15.3:0/3563400435 shutdown_connections 2024-09-21T06:23:29.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.036+0000 7f57c2524740 1 -- 172.21.15.3:0/3563400435 wait complete. 2024-09-21T06:23:29.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.036+0000 7f57c2524740 1 -- start start 2024-09-21T06:23:29.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.036+0000 7f57c2524740 1 Processor -- start 2024-09-21T06:23:29.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57c2524740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072f2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57c2524740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f14390 0x5575072eead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57c2524740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f35070 0x5575072ef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57c2524740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41ae9b0 con 0x557506f0c1b0 2024-09-21T06:23:29.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57c2524740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41ae2e0 con 0x557506f35070 2024-09-21T06:23:29.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57c2524740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41ab540 con 0x557506f14390 2024-09-21T06:23:29.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57c2524740 10 client.0 _ll_register_callbacks cb 0x7f57a40a7f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57b9d4c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57b954b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f14390 0x5575072eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57b9d4c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072f2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48812/0 (socket says 172.21.15.3:48812) 2024-09-21T06:23:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57b9d4c640 1 -- 172.21.15.3:0/755202677 learned_addr learned my addr 172.21.15.3:0/755202677 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57ba54d640 1 --2- 172.21.15.3:0/755202677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f35070 0x5575072ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57b954b640 1 -- 172.21.15.3:0/755202677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f35070 msgr2=0x5575072ef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57b954b640 1 --2- 172.21.15.3:0/755202677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f35070 0x5575072ef010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57b954b640 1 -- 172.21.15.3:0/755202677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 msgr2=0x5575072f2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57b954b640 1 --2- 172.21.15.3:0/755202677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072f2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57b954b640 1 -- 172.21.15.3:0/755202677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575072ecdf0 con 0x557506f14390 2024-09-21T06:23:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f57b954b640 1 --2- 172.21.15.3:0/755202677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f14390 0x5575072eead0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57a40ac520 tx=0x7f57a40ed4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd0e7a738edc973e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.037+0000 7f579d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:23:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f579d7fa640 1 -- 172.21.15.3:0/755202677 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a40a96f0 con 0x557506f14390 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f579d7fa640 1 -- 172.21.15.3:0/755202677 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f57a40aead0 con 0x557506f14390 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f57c2524740 1 -- 172.21.15.3:0/755202677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5575072508a0 con 0x557506f14390 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f579d7fa640 1 -- 172.21.15.3:0/755202677 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f57a40ad1a0 con 0x557506f14390 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f57c2524740 10 client.27400 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f57c2524740 1 -- 172.21.15.3:0/755202677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557507367510 con 0x557506f14390 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f579d7fa640 1 -- 172.21.15.3:0/755202677 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5575072508a0 con 0x557506f14390 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f579d7fa640 1 -- 172.21.15.3:0/755202677 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f57a40cf300 con 0x557506f14390 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f57c2524740 10 client.27400 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f57c2524740 10 client.27400 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f57c2524740 20 client.27400 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f57c2524740 1 --2- 172.21.15.3:0/755202677 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x557506ec8da0 0x557507369950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f57c2524740 4 client.27400 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f57c2524740 1 -- 172.21.15.3:0/755202677 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55750724dcd0 con 0x557506ec8da0 2024-09-21T06:23:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.038+0000 7f57b9d4c640 1 --2- 172.21.15.3:0/755202677 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x557506ec8da0 0x557507369950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f579d7fa640 10 client.27400 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f57b9d4c640 1 --2- 172.21.15.3:0/755202677 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x557506ec8da0 0x557507369950 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bcf13cba0fd1cc69 server_cookie=4e0ccc771c55560e in_seq=0 out_seq=0 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f579d7fa640 1 -- 172.21.15.3:0/755202677 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55750724dcd0 con 0x557506ec8da0 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f579d7fa640 10 client.27400 handle_command_reply: tid=0 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f57c2524740 1 client.27400 shutdown 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f57c2524740 1 -- 172.21.15.3:0/755202677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f14390 msgr2=0x5575072eead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f57c2524740 1 --2- 172.21.15.3:0/755202677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f14390 0x5575072eead0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57a40ac520 tx=0x7f57a40ed4f0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f57c2524740 1 -- 172.21.15.3:0/755202677 shutdown_connections 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f57c2524740 1 --2- 172.21.15.3:0/755202677 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x557506ec8da0 0x557507369950 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f57c2524740 1 --2- 172.21.15.3:0/755202677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557506f14390 0x5575072eead0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f57c2524740 1 --2- 172.21.15.3:0/755202677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557506f35070 0x5575072ef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f579d7fa640 0 client.27400 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f57c2524740 1 --2- 172.21.15.3:0/755202677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557506f0c1b0 0x5575072f2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f57c2524740 1 -- 172.21.15.3:0/755202677 >> 172.21.15.3:0/755202677 conn(0x557506f29460 msgr2=0x5575072dd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.039+0000 7f57c2524740 1 -- 172.21.15.3:0/755202677 shutdown_connections 2024-09-21T06:23:29.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57c2524740 1 -- 172.21.15.3:0/755202677 wait complete. 2024-09-21T06:23:29.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57c2524740 20 client.27400 trim_cache size 0 max 16384 2024-09-21T06:23:29.046 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:23:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57bbfb0640 1 -- 172.21.15.3:0/2833568713 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f57a00b6680 msgr2=0x7f57a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57bbfb0640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f57a00b6680 0x7f57a00b8b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x5575072db320 tx=0x7f57b0005f90 comp rx=0 tx=0).stop 2024-09-21T06:23:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57bbfb0640 1 -- 172.21.15.3:0/2833568713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41083b0 msgr2=0x7f57b4199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57bbfb0640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41083b0 0x7f57b4199a10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f57a400e9f0 tx=0x7f57a400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57bbfb0640 1 -- 172.21.15.3:0/2833568713 shutdown_connections 2024-09-21T06:23:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57bbfb0640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f57a00b6680 0x7f57a00b8b90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57bbfb0640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57b4108d00 0x7f57b419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57bbfb0640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41083b0 0x7f57b4199a10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57bbfb0640 1 --2- 172.21.15.3:0/2833568713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57b4102ac0 0x7f57b4199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57bbfb0640 1 -- 172.21.15.3:0/2833568713 >> 172.21.15.3:0/2833568713 conn(0x7f57b4079720 msgr2=0x7f57b4104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57bbfb0640 1 -- 172.21.15.3:0/2833568713 shutdown_connections 2024-09-21T06:23:29.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:29.040+0000 7f57bbfb0640 1 -- 172.21.15.3:0/2833568713 wait complete. 2024-09-21T06:23:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.810+0000 7f0c06b32640 1 -- 172.21.15.3:0/1461186176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c00108c30 msgr2=0x7f0c0010f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.810+0000 7f0c06b32640 1 --2- 172.21.15.3:0/1461186176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c00108c30 0x7f0c0010f460 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009ca0 tx=0x7f0bf802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.810+0000 7f0c06b32640 1 -- 172.21.15.3:0/1461186176 shutdown_connections 2024-09-21T06:23:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.810+0000 7f0c06b32640 1 --2- 172.21.15.3:0/1461186176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c00108c30 0x7f0c0010f460 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.810+0000 7f0c06b32640 1 --2- 172.21.15.3:0/1461186176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c001082e0 0x7f0c001086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.810+0000 7f0c06b32640 1 --2- 172.21.15.3:0/1461186176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c001029b0 0x7f0c00102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.810+0000 7f0c06b32640 1 -- 172.21.15.3:0/1461186176 >> 172.21.15.3:0/1461186176 conn(0x7f0c0007a740 msgr2=0x7f0c0007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.810+0000 7f0c06b32640 1 -- 172.21.15.3:0/1461186176 shutdown_connections 2024-09-21T06:23:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.810+0000 7f0c06b32640 1 -- 172.21.15.3:0/1461186176 wait complete. 2024-09-21T06:23:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c06b32640 1 Processor -- start 2024-09-21T06:23:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c06b32640 1 -- start start 2024-09-21T06:23:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c06b32640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c001029b0 0x7f0c0010ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c06b32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c001082e0 0x7f0c00110480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c06b32640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c00108c30 0x7f0c001109c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c06b32640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00113a40 con 0x7f0c00108c30 2024-09-21T06:23:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c06b32640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001138c0 con 0x7f0c001082e0 2024-09-21T06:23:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c06b32640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00113bc0 con 0x7f0c001029b0 2024-09-21T06:23:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c050cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c00108c30 0x7f0c001109c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c048ce640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c001029b0 0x7f0c0010ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0bfffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c001082e0 0x7f0c00110480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c050cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c00108c30 0x7f0c001109c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48840/0 (socket says 172.21.15.3:48840) 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c050cf640 1 -- 172.21.15.3:0/1031950373 learned_addr learned my addr 172.21.15.3:0/1031950373 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c050cf640 1 -- 172.21.15.3:0/1031950373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c001029b0 msgr2=0x7f0c0010ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.811+0000 7f0c050cf640 1 --2- 172.21.15.3:0/1031950373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c001029b0 0x7f0c0010ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.812+0000 7f0c050cf640 1 -- 172.21.15.3:0/1031950373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c001082e0 msgr2=0x7f0c00110480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.812+0000 7f0c050cf640 1 --2- 172.21.15.3:0/1031950373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c001082e0 0x7f0c00110480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.812+0000 7f0c050cf640 1 -- 172.21.15.3:0/1031950373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c001a8640 con 0x7f0c00108c30 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.812+0000 7f0bfffff640 1 --2- 172.21.15.3:0/1031950373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c001082e0 0x7f0c00110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.812+0000 7f0c050cf640 1 --2- 172.21.15.3:0/1031950373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c00108c30 0x7f0c001109c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f0bf802f540 tx=0x7f0bf80316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.812+0000 7f0bfdffb640 1 -- 172.21.15.3:0/1031950373 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8003b70 con 0x7f0c00108c30 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.812+0000 7f0bfdffb640 1 -- 172.21.15.3:0/1031950373 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf8003d10 con 0x7f0c00108c30 2024-09-21T06:23:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.812+0000 7f0c06b32640 1 -- 172.21.15.3:0/1031950373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c001a88d0 con 0x7f0c00108c30 2024-09-21T06:23:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.812+0000 7f0bfdffb640 1 -- 172.21.15.3:0/1031950373 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf802f990 con 0x7f0c00108c30 2024-09-21T06:23:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.812+0000 7f0c06b32640 1 -- 172.21.15.3:0/1031950373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c001a8e20 con 0x7f0c00108c30 2024-09-21T06:23:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.815+0000 7f0c0daa2740 1 Processor -- start 2024-09-21T06:23:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.815+0000 7f0bfdffb640 1 -- 172.21.15.3:0/1031950373 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f0bf802fb30 con 0x7f0c00108c30 2024-09-21T06:23:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.815+0000 7f0c0daa2740 1 -- start start 2024-09-21T06:23:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.815+0000 7f0c0daa2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b745d1b0 0x5601b78029b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.815+0000 7f0c0daa2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b74342f0 0x5601b7802ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.815+0000 7f0c0daa2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b743c4d0 0x5601b7803430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.815+0000 7f0c0daa2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5601b77776f0 con 0x5601b743c4d0 2024-09-21T06:23:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.815+0000 7f0c0daa2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5601b7766b20 con 0x5601b745d1b0 2024-09-21T06:23:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.815+0000 7f0c0daa2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5601b776a930 con 0x5601b74342f0 2024-09-21T06:23:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.815+0000 7f0c048ce640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b74342f0 0x5601b7802ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.815+0000 7f0bfffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b745d1b0 0x5601b78029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.815+0000 7f0c048ce640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b74342f0 0x5601b7802ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:45680/0 (socket says 172.21.15.3:45680) 2024-09-21T06:23:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.815+0000 7f0c048ce640 1 -- 172.21.15.3:0/2890540988 learned_addr learned my addr 172.21.15.3:0/2890540988 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.816+0000 7f0bfffff640 1 --2- 172.21.15.3:0/2890540988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b743c4d0 0x5601b7803430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.816+0000 7f0c048ce640 1 -- 172.21.15.3:0/2890540988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b745d1b0 msgr2=0x5601b78029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.816+0000 7f0c048ce640 1 --2- 172.21.15.3:0/2890540988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b745d1b0 0x5601b78029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.816+0000 7f0c048ce640 1 -- 172.21.15.3:0/2890540988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b743c4d0 msgr2=0x5601b7803430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.816+0000 7f0c048ce640 1 --2- 172.21.15.3:0/2890540988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b743c4d0 0x5601b7803430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.816+0000 7f0c048ce640 1 -- 172.21.15.3:0/2890540988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b772c530 con 0x5601b74342f0 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.816+0000 7f0bfdffb640 1 --2- 172.21.15.3:0/1031950373 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0bf00b66b0 0x7f0bf00b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.816+0000 7f0bfffff640 1 --2- 172.21.15.3:0/1031950373 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0bf00b66b0 0x7f0bf00b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.816+0000 7f0c048ce640 1 --2- 172.21.15.3:0/2890540988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b74342f0 0x5601b7802ef0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009cd0 tx=0x7f0bf4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc009e8b495256ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.816+0000 7f0bfdffb640 1 -- 172.21.15.3:0/1031950373 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f0bf81088e0 con 0x7f0c00108c30 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.816+0000 7f0beaffd640 1 -- 172.21.15.3:0/2890540988 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4005020 con 0x5601b74342f0 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.816+0000 7f0beaffd640 1 -- 172.21.15.3:0/2890540988 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf40047a0 con 0x5601b74342f0 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.816+0000 7f0beaffd640 1 -- 172.21.15.3:0/2890540988 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4019670 con 0x5601b74342f0 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.817+0000 7f0c0daa2740 1 -- 172.21.15.3:0/2890540988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b74342f0 msgr2=0x5601b7802ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.817+0000 7f0bfffff640 1 --2- 172.21.15.3:0/1031950373 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0bf00b66b0 0x7f0bf00b8bc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0058c0 tx=0x7f0bec005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.817+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/2890540988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b74342f0 0x5601b7802ef0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009cd0 tx=0x7f0bf4011810 comp rx=0 tx=0).stop 2024-09-21T06:23:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.817+0000 7f0c0daa2740 1 -- 172.21.15.3:0/2890540988 shutdown_connections 2024-09-21T06:23:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.817+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/2890540988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b74342f0 0x5601b7802ef0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.817+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/2890540988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b743c4d0 0x5601b7803430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.817+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/2890540988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b745d1b0 0x5601b78029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.817+0000 7f0c0daa2740 1 -- 172.21.15.3:0/2890540988 >> 172.21.15.3:0/2890540988 conn(0x5601b7451300 msgr2=0x5601b78008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.817+0000 7f0c0daa2740 1 -- 172.21.15.3:0/2890540988 shutdown_connections 2024-09-21T06:23:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.817+0000 7f0c0daa2740 1 -- 172.21.15.3:0/2890540988 wait complete. 2024-09-21T06:23:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.818+0000 7f0c0daa2740 1 -- start start 2024-09-21T06:23:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.818+0000 7f0c0daa2740 1 Processor -- start 2024-09-21T06:23:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.818+0000 7f0c0daa2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b74342f0 0x5601b780e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.818+0000 7f0c0daa2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b743c4d0 0x5601b780e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.818+0000 7f0c0daa2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b745d1b0 0x5601b781a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.818+0000 7f0c0daa2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5601b769eab0 con 0x5601b74342f0 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.818+0000 7f0c0daa2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5601b770c730 con 0x5601b745d1b0 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.818+0000 7f0c0daa2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5601b7811130 con 0x5601b743c4d0 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.818+0000 7f0c0daa2740 10 client.0 _ll_register_callbacks cb 0x5601b77606b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0bfffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b743c4d0 0x5601b780e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0c048ce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b74342f0 0x5601b780e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0c050cf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b745d1b0 0x5601b781a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0bfffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b743c4d0 0x5601b780e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:45682/0 (socket says 172.21.15.3:45682) 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0bfffff640 1 -- 172.21.15.3:0/3244519441 learned_addr learned my addr 172.21.15.3:0/3244519441 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0c048ce640 1 -- 172.21.15.3:0/3244519441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b743c4d0 msgr2=0x5601b780e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0c048ce640 1 --2- 172.21.15.3:0/3244519441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b743c4d0 0x5601b780e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0c048ce640 1 -- 172.21.15.3:0/3244519441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b745d1b0 msgr2=0x5601b781a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0c048ce640 1 --2- 172.21.15.3:0/3244519441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b745d1b0 0x5601b781a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0c048ce640 1 -- 172.21.15.3:0/3244519441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b74c8790 con 0x5601b74342f0 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0c050cf640 1 --2- 172.21.15.3:0/3244519441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b745d1b0 0x5601b781a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0bfffff640 1 --2- 172.21.15.3:0/3244519441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b743c4d0 0x5601b780e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0c048ce640 1 --2- 172.21.15.3:0/3244519441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b74342f0 0x5601b780e000 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x5601b7801630 tx=0x7f0bf4005170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8d8fe4c78ed9a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0bdffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0bdffff640 1 -- 172.21.15.3:0/3244519441 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4028020 con 0x5601b74342f0 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0bdffff640 1 -- 172.21.15.3:0/3244519441 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f0bf4005570 con 0x5601b74342f0 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.819+0000 7f0c0daa2740 1 -- 172.21.15.3:0/3244519441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bf40047a0 con 0x5601b74342f0 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.820+0000 7f0bdffff640 1 -- 172.21.15.3:0/3244519441 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0bf4033050 con 0x5601b74342f0 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.820+0000 7f0c0daa2740 10 client.17904 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.820+0000 7f0c0daa2740 1 -- 172.21.15.3:0/3244519441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5601b7541620 con 0x5601b74342f0 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.820+0000 7f0bdffff640 1 -- 172.21.15.3:0/3244519441 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf40047a0 con 0x5601b74342f0 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.820+0000 7f0bdffff640 1 -- 172.21.15.3:0/3244519441 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f0bf4024c50 con 0x5601b74342f0 2024-09-21T06:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.820+0000 7f0c0daa2740 10 client.17904 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.820+0000 7f0c0daa2740 10 client.17904 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.820+0000 7f0c0daa2740 20 client.17904 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.820+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/3244519441 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5601b741bd80 0x5601b7820140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.820+0000 7f0c0daa2740 4 client.17904 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.820+0000 7f0c0daa2740 1 -- 172.21.15.3:0/3244519441 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601b7779640 con 0x5601b741bd80 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.820+0000 7f0bfffff640 1 --2- 172.21.15.3:0/3244519441 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5601b741bd80 0x5601b7820140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.821+0000 7f0bdffff640 10 client.17904 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.821+0000 7f0bfffff640 1 --2- 172.21.15.3:0/3244519441 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5601b741bd80 0x5601b7820140 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b71fe6f044c487 server_cookie=95b0596c02be824e in_seq=0 out_seq=0 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.822+0000 7f0bdffff640 1 -- 172.21.15.3:0/3244519441 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5601b7779640 con 0x5601b741bd80 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.822+0000 7f0bdffff640 10 client.17904 handle_command_reply: tid=0 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.822+0000 7f0c0daa2740 1 client.17904 shutdown 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.822+0000 7f0c0daa2740 1 -- 172.21.15.3:0/3244519441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b74342f0 msgr2=0x5601b780e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.822+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/3244519441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b74342f0 0x5601b780e000 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x5601b7801630 tx=0x7f0bf4005170 comp rx=0 tx=0).stop 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.822+0000 7f0c0daa2740 1 -- 172.21.15.3:0/3244519441 shutdown_connections 2024-09-21T06:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.822+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/3244519441 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5601b741bd80 0x5601b7820140 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.822+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/3244519441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b743c4d0 0x5601b780e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.822+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/3244519441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b745d1b0 0x5601b781a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.822+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/3244519441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b74342f0 0x5601b780e000 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.822+0000 7f0bdffff640 0 client.17904 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.822+0000 7f0c0daa2740 1 -- 172.21.15.3:0/3244519441 >> 172.21.15.3:0/3244519441 conn(0x5601b7451300 msgr2=0x5601b78039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.823+0000 7f0c0daa2740 1 -- 172.21.15.3:0/3244519441 shutdown_connections 2024-09-21T06:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.823+0000 7f0c0daa2740 1 -- 172.21.15.3:0/3244519441 wait complete. 2024-09-21T06:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.823+0000 7f0c0daa2740 20 client.17904 trim_cache size 0 max 16384 2024-09-21T06:23:29.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:29 smithi003 ceph-mon[31227]: pgmap v585: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-09-21T06:23:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.839+0000 7f0c0daa2740 1 Processor -- start 2024-09-21T06:23:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.839+0000 7f0c0daa2740 1 -- start start 2024-09-21T06:23:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.839+0000 7f0c0daa2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b741bd80 0x5601b780fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.839+0000 7f0c0daa2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b74342f0 0x5601b781a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.839+0000 7f0c0daa2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b743c4d0 0x5601b780b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.839+0000 7f0c0daa2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001b1730 con 0x5601b743c4d0 2024-09-21T06:23:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.839+0000 7f0c0daa2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001b3270 con 0x5601b74342f0 2024-09-21T06:23:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.839+0000 7f0c0daa2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001b1e00 con 0x5601b741bd80 2024-09-21T06:23:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.839+0000 7f0c050cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b743c4d0 0x5601b780b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.839+0000 7f0c050cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b743c4d0 0x5601b780b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48872/0 (socket says 172.21.15.3:48872) 2024-09-21T06:23:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.839+0000 7f0c050cf640 1 -- 172.21.15.3:0/2209333074 learned_addr learned my addr 172.21.15.3:0/2209333074 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.839+0000 7f0c048ce640 1 --2- 172.21.15.3:0/2209333074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b741bd80 0x5601b780fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.839+0000 7f0bfffff640 1 --2- 172.21.15.3:0/2209333074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b74342f0 0x5601b781a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.840+0000 7f0c050cf640 1 -- 172.21.15.3:0/2209333074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b741bd80 msgr2=0x5601b780fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.840+0000 7f0c050cf640 1 --2- 172.21.15.3:0/2209333074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b741bd80 0x5601b780fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.840+0000 7f0c050cf640 1 -- 172.21.15.3:0/2209333074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b74342f0 msgr2=0x5601b781a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.840+0000 7f0c050cf640 1 --2- 172.21.15.3:0/2209333074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b74342f0 0x5601b781a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.840+0000 7f0c050cf640 1 -- 172.21.15.3:0/2209333074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b7804cb0 con 0x5601b743c4d0 2024-09-21T06:23:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.840+0000 7f0c050cf640 1 --2- 172.21.15.3:0/2209333074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b743c4d0 0x5601b780b1d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80cb950 tx=0x7f0bf80cae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae8c98fa66fb35a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.840+0000 7f0bea7fc640 1 -- 172.21.15.3:0/2209333074 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80ce1f0 con 0x5601b743c4d0 2024-09-21T06:23:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.840+0000 7f0bea7fc640 1 -- 172.21.15.3:0/2209333074 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf8036040 con 0x5601b743c4d0 2024-09-21T06:23:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.840+0000 7f0bea7fc640 1 -- 172.21.15.3:0/2209333074 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80d7e70 con 0x5601b743c4d0 2024-09-21T06:23:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.841+0000 7f0c0daa2740 1 -- 172.21.15.3:0/2209333074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b743c4d0 msgr2=0x5601b780b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.841+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/2209333074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b743c4d0 0x5601b780b1d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80cb950 tx=0x7f0bf80cae60 comp rx=0 tx=0).stop 2024-09-21T06:23:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.841+0000 7f0c0daa2740 1 -- 172.21.15.3:0/2209333074 shutdown_connections 2024-09-21T06:23:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.841+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/2209333074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b743c4d0 0x5601b780b1d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.841+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/2209333074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b74342f0 0x5601b781a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.841+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/2209333074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b741bd80 0x5601b780fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.841+0000 7f0c0daa2740 1 -- 172.21.15.3:0/2209333074 >> 172.21.15.3:0/2209333074 conn(0x5601b7451300 msgr2=0x5601b780f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.841+0000 7f0c0daa2740 1 -- 172.21.15.3:0/2209333074 shutdown_connections 2024-09-21T06:23:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.841+0000 7f0c0daa2740 1 -- 172.21.15.3:0/2209333074 wait complete. 2024-09-21T06:23:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.841+0000 7f0c0daa2740 1 -- start start 2024-09-21T06:23:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.841+0000 7f0c0daa2740 1 Processor -- start 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0c0daa2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b741bd80 0x5601b78135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0c0daa2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b74342f0 0x5601b7813af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0c0daa2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b743c4d0 0x5601b7814030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0c0daa2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001b2ba0 con 0x5601b74342f0 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0c0daa2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001b24d0 con 0x5601b743c4d0 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0c0daa2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001af730 con 0x5601b741bd80 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0c0daa2740 10 client.0 _ll_register_callbacks cb 0x7f0bec011610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0bfffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b74342f0 0x5601b7813af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0bfffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b74342f0 0x5601b7813af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48884/0 (socket says 172.21.15.3:48884) 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0bfffff640 1 -- 172.21.15.3:0/1368889140 learned_addr learned my addr 172.21.15.3:0/1368889140 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0c048ce640 1 --2- 172.21.15.3:0/1368889140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b741bd80 0x5601b78135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0c050cf640 1 --2- 172.21.15.3:0/1368889140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b743c4d0 0x5601b7814030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0bfffff640 1 -- 172.21.15.3:0/1368889140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b741bd80 msgr2=0x5601b78135b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0bfffff640 1 --2- 172.21.15.3:0/1368889140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b741bd80 0x5601b78135b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0bfffff640 1 -- 172.21.15.3:0/1368889140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b743c4d0 msgr2=0x5601b7814030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0bfffff640 1 --2- 172.21.15.3:0/1368889140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b743c4d0 0x5601b7814030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.842+0000 7f0bfffff640 1 -- 172.21.15.3:0/1368889140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b7888c50 con 0x5601b74342f0 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0bfffff640 1 --2- 172.21.15.3:0/1368889140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b74342f0 0x5601b7813af0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x5601b7820890 tx=0x7f0bec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1291c913f809b07b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0bdffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0bdffff640 1 -- 172.21.15.3:0/1368889140 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec020a60 con 0x5601b74342f0 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0bdffff640 1 -- 172.21.15.3:0/1368889140 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f0bec020c00 con 0x5601b74342f0 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0c0daa2740 1 -- 172.21.15.3:0/1368889140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bf8036040 con 0x5601b74342f0 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0bdffff640 1 -- 172.21.15.3:0/1368889140 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0bec042050 con 0x5601b74342f0 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0c0daa2740 10 client.17910 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0c0daa2740 1 -- 172.21.15.3:0/1368889140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5601b7889f40 con 0x5601b74342f0 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0bdffff640 1 -- 172.21.15.3:0/1368889140 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf8036040 con 0x5601b74342f0 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0bdffff640 1 -- 172.21.15.3:0/1368889140 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f0bec033070 con 0x5601b74342f0 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0c0daa2740 10 client.17910 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0c0daa2740 10 client.17910 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0c0daa2740 20 client.17910 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/1368889140 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5601b745d1b0 0x5601b788c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0c0daa2740 4 client.17910 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:23:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.843+0000 7f0c0daa2740 1 -- 172.21.15.3:0/1368889140 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601b77750f0 con 0x5601b745d1b0 2024-09-21T06:23:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.844+0000 7f0c048ce640 1 --2- 172.21.15.3:0/1368889140 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5601b745d1b0 0x5601b788c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.844+0000 7f0c048ce640 1 --2- 172.21.15.3:0/1368889140 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5601b745d1b0 0x5601b788c760 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94c6083957763145 server_cookie=ce6248d3f3b0ed78 in_seq=0 out_seq=0 2024-09-21T06:23:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.844+0000 7f0bdffff640 10 client.17910 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:29.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:29 smithi137 ceph-mon[34464]: pgmap v585: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-09-21T06:23:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:29 smithi204 ceph-mon[34659]: pgmap v585: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-09-21T06:23:29.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 884329 10.58 MB/sec execute 302 sec latency 68.193 ms 2024-09-21T06:23:29.975 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:23:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.984+0000 7f0bdffff640 1 -- 172.21.15.3:0/1368889140 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601b77750f0 con 0x5601b745d1b0 2024-09-21T06:23:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.984+0000 7f0bdffff640 10 client.17910 handle_command_reply: tid=0 2024-09-21T06:23:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.984+0000 7f0c0daa2740 1 client.17910 shutdown 2024-09-21T06:23:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.984+0000 7f0c0daa2740 1 -- 172.21.15.3:0/1368889140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b74342f0 msgr2=0x5601b7813af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.984+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/1368889140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b74342f0 0x5601b7813af0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x5601b7820890 tx=0x7f0bec004730 comp rx=0 tx=0).stop 2024-09-21T06:23:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.984+0000 7f0c0daa2740 1 -- 172.21.15.3:0/1368889140 shutdown_connections 2024-09-21T06:23:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.984+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/1368889140 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5601b745d1b0 0x5601b788c760 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.984+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/1368889140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b743c4d0 0x5601b7814030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.984+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/1368889140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601b74342f0 0x5601b7813af0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.984+0000 7f0c0daa2740 1 --2- 172.21.15.3:0/1368889140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5601b741bd80 0x5601b78135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.984+0000 7f0c0daa2740 1 -- 172.21.15.3:0/1368889140 >> 172.21.15.3:0/1368889140 conn(0x5601b7451300 msgr2=0x5601b7800880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.984+0000 7f0bdffff640 0 client.17910 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.985+0000 7f0c0daa2740 1 -- 172.21.15.3:0/1368889140 shutdown_connections 2024-09-21T06:23:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.985+0000 7f0c0daa2740 1 -- 172.21.15.3:0/1368889140 wait complete. 2024-09-21T06:23:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.985+0000 7f0c0daa2740 20 client.17910 trim_cache size 0 max 16384 2024-09-21T06:23:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:23:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:23:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:23:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:23:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.985+0000 7f0c06b32640 1 -- 172.21.15.3:0/1031950373 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0bf00b66b0 msgr2=0x7f0bf00b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.985+0000 7f0c06b32640 1 --2- 172.21.15.3:0/1031950373 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0bf00b66b0 0x7f0bf00b8bc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0058c0 tx=0x7f0bec005f90 comp rx=0 tx=0).stop 2024-09-21T06:23:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.986+0000 7f0c06b32640 1 -- 172.21.15.3:0/1031950373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c00108c30 msgr2=0x7f0c001109c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.986+0000 7f0c06b32640 1 --2- 172.21.15.3:0/1031950373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c00108c30 0x7f0c001109c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f0bf802f540 tx=0x7f0bf80316f0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.986+0000 7f0c06b32640 1 -- 172.21.15.3:0/1031950373 shutdown_connections 2024-09-21T06:23:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.986+0000 7f0c06b32640 1 --2- 172.21.15.3:0/1031950373 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0bf00b66b0 0x7f0bf00b8bc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.986+0000 7f0c06b32640 1 --2- 172.21.15.3:0/1031950373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c00108c30 0x7f0c001109c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.986+0000 7f0c06b32640 1 --2- 172.21.15.3:0/1031950373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c001082e0 0x7f0c00110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.986+0000 7f0c06b32640 1 --2- 172.21.15.3:0/1031950373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c001029b0 0x7f0c0010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.986+0000 7f0c06b32640 1 -- 172.21.15.3:0/1031950373 >> 172.21.15.3:0/1031950373 conn(0x7f0c0007a740 msgr2=0x7f0c00104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.986+0000 7f0c06b32640 1 -- 172.21.15.3:0/1031950373 shutdown_connections 2024-09-21T06:23:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:29.986+0000 7f0c06b32640 1 -- 172.21.15.3:0/1031950373 wait complete. 2024-09-21T06:23:30.242 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:30.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.795+0000 7ff28e12f640 1 -- 172.21.15.3:0/1895400764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff288104850 msgr2=0x7ff288104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.795+0000 7ff28e12f640 1 --2- 172.21.15.3:0/1895400764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff288104850 0x7ff288104c60 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff278009900 tx=0x7ff27802a410 comp rx=0 tx=0).stop 2024-09-21T06:23:30.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.795+0000 7ff28e12f640 1 -- 172.21.15.3:0/1895400764 shutdown_connections 2024-09-21T06:23:30.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.795+0000 7ff28e12f640 1 --2- 172.21.15.3:0/1895400764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff288105280 0x7ff28810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.795+0000 7ff28e12f640 1 --2- 172.21.15.3:0/1895400764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff28810de00 0x7ff288110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.795+0000 7ff28e12f640 1 --2- 172.21.15.3:0/1895400764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff288104850 0x7ff288104c60 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.795+0000 7ff28e12f640 1 -- 172.21.15.3:0/1895400764 >> 172.21.15.3:0/1895400764 conn(0x7ff288079730 msgr2=0x7ff288079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:30.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.795+0000 7ff28e12f640 1 -- 172.21.15.3:0/1895400764 shutdown_connections 2024-09-21T06:23:30.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.795+0000 7ff28e12f640 1 -- 172.21.15.3:0/1895400764 wait complete. 2024-09-21T06:23:30.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.795+0000 7ff28e12f640 1 Processor -- start 2024-09-21T06:23:30.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.796+0000 7ff28e12f640 1 -- start start 2024-09-21T06:23:30.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.796+0000 7ff28e12f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff288104850 0x7ff28819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.796+0000 7ff28e12f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff288105280 0x7ff28819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.796+0000 7ff28e12f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff28810de00 0x7ff28819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.796+0000 7ff28e12f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff288113a80 con 0x7ff288105280 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.796+0000 7ff28e12f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff288113900 con 0x7ff288104850 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.796+0000 7ff28e12f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff288113c00 con 0x7ff28810de00 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.796+0000 7ff286ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff288105280 0x7ff28819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.796+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff288104850 0x7ff28819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.796+0000 7ff286ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff288105280 0x7ff28819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48910/0 (socket says 172.21.15.3:48910) 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.796+0000 7ff286ffd640 1 -- 172.21.15.3:0/3086578684 learned_addr learned my addr 172.21.15.3:0/3086578684 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.796+0000 7ff287fff640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff28810de00 0x7ff28819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.797+0000 7ff287fff640 1 -- 172.21.15.3:0/3086578684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff288104850 msgr2=0x7ff28819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.797+0000 7ff287fff640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff288104850 0x7ff28819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.797+0000 7ff287fff640 1 -- 172.21.15.3:0/3086578684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff288105280 msgr2=0x7ff28819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.797+0000 7ff287fff640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff288105280 0x7ff28819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.797+0000 7ff287fff640 1 -- 172.21.15.3:0/3086578684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2881a0420 con 0x7ff28810de00 2024-09-21T06:23:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.797+0000 7ff2877fe640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff288104850 0x7ff28819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.797+0000 7ff286ffd640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff288105280 0x7ff28819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.797+0000 7ff287fff640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff28810de00 0x7ff28819fcf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff280009fd0 tx=0x7ff28000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.797+0000 7ff284ff9640 1 -- 172.21.15.3:0/3086578684 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28000ce20 con 0x7ff28810de00 2024-09-21T06:23:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.797+0000 7ff28e12f640 1 -- 172.21.15.3:0/3086578684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2881a0710 con 0x7ff28810de00 2024-09-21T06:23:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.797+0000 7ff284ff9640 1 -- 172.21.15.3:0/3086578684 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff280005020 con 0x7ff28810de00 2024-09-21T06:23:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.797+0000 7ff284ff9640 1 -- 172.21.15.3:0/3086578684 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280010690 con 0x7ff28810de00 2024-09-21T06:23:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.797+0000 7ff28e12f640 1 -- 172.21.15.3:0/3086578684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2881a6e70 con 0x7ff28810de00 2024-09-21T06:23:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff2946a3740 1 Processor -- start 2024-09-21T06:23:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff284ff9640 1 -- 172.21.15.3:0/3086578684 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7ff280010830 con 0x7ff28810de00 2024-09-21T06:23:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff2946a3740 1 -- start start 2024-09-21T06:23:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff2946a3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 0x55c8e0dff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff2946a3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0dffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff2946a3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a37390 0x55c8e0e00120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff2946a3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c8e0d75ea0 con 0x55c8e0a37390 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff2946a3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c8e0d652d0 con 0x55c8e0a58070 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff2946a3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c8e0d690e0 con 0x55c8e0a2f1b0 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff286ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a37390 0x55c8e0e00120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0dffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0dffbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:45726/0 (socket says 172.21.15.3:45726) 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff286ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a37390 0x55c8e0e00120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48912/0 (socket says 172.21.15.3:48912) 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff2877fe640 1 -- 172.21.15.3:0/4001523539 learned_addr learned my addr 172.21.15.3:0/4001523539 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.801+0000 7ff286ffd640 1 --2- 172.21.15.3:0/4001523539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 0x55c8e0dff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff284ff9640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff2700b6680 0x7ff2700b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff2877fe640 1 -- 172.21.15.3:0/4001523539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 msgr2=0x55c8e0dff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff2877fe640 1 --2- 172.21.15.3:0/4001523539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 0x55c8e0dff6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff2877fe640 1 -- 172.21.15.3:0/4001523539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a37390 msgr2=0x55c8e0e00120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff2877fe640 1 --2- 172.21.15.3:0/4001523539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a37390 0x55c8e0e00120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff2877fe640 1 -- 172.21.15.3:0/4001523539 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e0ac3650 con 0x55c8e0a2f1b0 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff287fff640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff2700b6680 0x7ff2700b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff286ffd640 1 --2- 172.21.15.3:0/4001523539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a37390 0x55c8e0e00120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff284ff9640 1 -- 172.21.15.3:0/3086578684 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7ff2800e7e00 con 0x7ff28810de00 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff2877fe640 1 --2- 172.21.15.3:0/4001523539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0dffbe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff28806c880 tx=0x7ff278005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e45fb30d7c4d66cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff26dffb640 1 -- 172.21.15.3:0/4001523539 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278040070 con 0x55c8e0a2f1b0 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff287fff640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff2700b6680 0x7ff2700b8b90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff2800e5610 tx=0x7ff2800053b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff26dffb640 1 -- 172.21.15.3:0/4001523539 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff278031720 con 0x55c8e0a2f1b0 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.802+0000 7ff26dffb640 1 -- 172.21.15.3:0/4001523539 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278035040 con 0x55c8e0a2f1b0 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.803+0000 7ff2946a3740 1 -- 172.21.15.3:0/4001523539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 msgr2=0x55c8e0dffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.803+0000 7ff2946a3740 1 --2- 172.21.15.3:0/4001523539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0dffbe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff28806c880 tx=0x7ff278005e20 comp rx=0 tx=0).stop 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.803+0000 7ff2946a3740 1 -- 172.21.15.3:0/4001523539 shutdown_connections 2024-09-21T06:23:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.803+0000 7ff2946a3740 1 --2- 172.21.15.3:0/4001523539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0dffbe0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.803+0000 7ff2946a3740 1 --2- 172.21.15.3:0/4001523539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a37390 0x55c8e0e00120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.803+0000 7ff2946a3740 1 --2- 172.21.15.3:0/4001523539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 0x55c8e0dff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.803+0000 7ff2946a3740 1 -- 172.21.15.3:0/4001523539 >> 172.21.15.3:0/4001523539 conn(0x55c8e0a4c460 msgr2=0x55c8e0dfd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.803+0000 7ff2946a3740 1 -- 172.21.15.3:0/4001523539 shutdown_connections 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.803+0000 7ff2946a3740 1 -- 172.21.15.3:0/4001523539 wait complete. 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.804+0000 7ff2946a3740 1 -- start start 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.804+0000 7ff2946a3740 1 Processor -- start 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.804+0000 7ff2946a3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e0ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.804+0000 7ff2946a3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a37390 0x55c8e0e0b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.804+0000 7ff2946a3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 0x55c8e0e17140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.804+0000 7ff2946a3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c8e0bc6700 con 0x55c8e0a2f1b0 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.804+0000 7ff2946a3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c8e0d00160 con 0x55c8e0a58070 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.804+0000 7ff2946a3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c8e0e0de20 con 0x55c8e0a37390 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.804+0000 7ff2946a3740 10 client.0 _ll_register_callbacks cb 0x55c8e0d199e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff286ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e0ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a37390 0x55c8e0e0b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff286ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e0ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48926/0 (socket says 172.21.15.3:48926) 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff286ffd640 1 -- 172.21.15.3:0/1862769094 learned_addr learned my addr 172.21.15.3:0/1862769094 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff286ffd640 1 --2- 172.21.15.3:0/1862769094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 0x55c8e0e17140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff2877fe640 1 -- 172.21.15.3:0/1862769094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 msgr2=0x55c8e0e17140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff2877fe640 1 --2- 172.21.15.3:0/1862769094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 0x55c8e0e17140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff2877fe640 1 -- 172.21.15.3:0/1862769094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a2f1b0 msgr2=0x55c8e0e0ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff2877fe640 1 --2- 172.21.15.3:0/1862769094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e0ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff2877fe640 1 -- 172.21.15.3:0/1862769094 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e0d2dc90 con 0x55c8e0a37390 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff286ffd640 1 --2- 172.21.15.3:0/1862769094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e0ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff2877fe640 1 --2- 172.21.15.3:0/1862769094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a37390 0x55c8e0e0b250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff278006010 tx=0x7ff27802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1b53ed17d8d16b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff266ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff266ffd640 1 -- 172.21.15.3:0/1862769094 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27804b070 con 0x55c8e0a37390 2024-09-21T06:23:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff266ffd640 1 -- 172.21.15.3:0/1862769094 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7ff278004250 con 0x55c8e0a37390 2024-09-21T06:23:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.805+0000 7ff2946a3740 1 -- 172.21.15.3:0/1862769094 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff278031720 con 0x55c8e0a37390 2024-09-21T06:23:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.806+0000 7ff266ffd640 1 -- 172.21.15.3:0/1862769094 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff278057050 con 0x55c8e0a37390 2024-09-21T06:23:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.806+0000 7ff2946a3740 10 client.27430 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.806+0000 7ff2946a3740 1 -- 172.21.15.3:0/1862769094 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c8e0e19b20 con 0x55c8e0a37390 2024-09-21T06:23:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.806+0000 7ff266ffd640 1 -- 172.21.15.3:0/1862769094 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff278031720 con 0x55c8e0a37390 2024-09-21T06:23:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.806+0000 7ff266ffd640 1 -- 172.21.15.3:0/1862769094 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff278004770 con 0x55c8e0a37390 2024-09-21T06:23:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.806+0000 7ff2946a3740 10 client.27430 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.806+0000 7ff2946a3740 10 client.27430 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:23:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.806+0000 7ff2946a3740 20 client.27430 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.806+0000 7ff2946a3740 1 --2- 172.21.15.3:0/1862769094 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55c8e0a169f0 0x55c8e0e1ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.806+0000 7ff2946a3740 4 client.27430 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.806+0000 7ff2946a3740 1 -- 172.21.15.3:0/1862769094 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8e0d77df0 con 0x55c8e0a169f0 2024-09-21T06:23:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.806+0000 7ff286ffd640 1 --2- 172.21.15.3:0/1862769094 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55c8e0a169f0 0x55c8e0e1ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.807+0000 7ff266ffd640 10 client.27430 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:23:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.807+0000 7ff286ffd640 1 --2- 172.21.15.3:0/1862769094 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55c8e0a169f0 0x55c8e0e1ce80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4986ba07b224eeaf server_cookie=ab6d20c165513977 in_seq=0 out_seq=0 2024-09-21T06:23:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.807+0000 7ff266ffd640 1 -- 172.21.15.3:0/1862769094 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c8e0d77df0 con 0x55c8e0a169f0 2024-09-21T06:23:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.807+0000 7ff266ffd640 10 client.27430 handle_command_reply: tid=0 2024-09-21T06:23:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.807+0000 7ff2946a3740 1 client.27430 shutdown 2024-09-21T06:23:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.808+0000 7ff2946a3740 1 -- 172.21.15.3:0/1862769094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a37390 msgr2=0x55c8e0e0b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.808+0000 7ff2946a3740 1 --2- 172.21.15.3:0/1862769094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a37390 0x55c8e0e0b250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff278006010 tx=0x7ff27802a410 comp rx=0 tx=0).stop 2024-09-21T06:23:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.808+0000 7ff2946a3740 1 -- 172.21.15.3:0/1862769094 shutdown_connections 2024-09-21T06:23:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.808+0000 7ff2946a3740 1 --2- 172.21.15.3:0/1862769094 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55c8e0a169f0 0x55c8e0e1ce80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.808+0000 7ff2946a3740 1 --2- 172.21.15.3:0/1862769094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a37390 0x55c8e0e0b250 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.808+0000 7ff2946a3740 1 --2- 172.21.15.3:0/1862769094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 0x55c8e0e17140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.808+0000 7ff266ffd640 0 client.27430 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:23:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.808+0000 7ff2946a3740 1 --2- 172.21.15.3:0/1862769094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e0ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.808+0000 7ff2946a3740 1 -- 172.21.15.3:0/1862769094 >> 172.21.15.3:0/1862769094 conn(0x55c8e0a4c460 msgr2=0x55c8e0e02830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:30.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.808+0000 7ff2946a3740 1 -- 172.21.15.3:0/1862769094 shutdown_connections 2024-09-21T06:23:30.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.808+0000 7ff2946a3740 1 -- 172.21.15.3:0/1862769094 wait complete. 2024-09-21T06:23:30.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.808+0000 7ff2946a3740 20 client.27430 trim_cache size 0 max 16384 2024-09-21T06:23:30.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff2946a3740 1 Processor -- start 2024-09-21T06:23:30.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff2946a3740 1 -- start start 2024-09-21T06:23:30.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff2946a3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e0b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff2946a3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a37390 0x55c8e0e07f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff2946a3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a58070 0x55c8e0e08440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff2946a3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881af620 con 0x55c8e0a58070 2024-09-21T06:23:30.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff2946a3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b1160 con 0x55c8e0a37390 2024-09-21T06:23:30.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff2946a3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881afcf0 con 0x55c8e0a2f1b0 2024-09-21T06:23:30.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff286ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a58070 0x55c8e0e08440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a37390 0x55c8e0e07f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff286ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a58070 0x55c8e0e08440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48930/0 (socket says 172.21.15.3:48930) 2024-09-21T06:23:30.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff286ffd640 1 -- 172.21.15.3:0/1436223169 learned_addr learned my addr 172.21.15.3:0/1436223169 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:30.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a37390 0x55c8e0e07f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:33346/0 (socket says 172.21.15.3:33346) 2024-09-21T06:23:30.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.823+0000 7ff286ffd640 1 --2- 172.21.15.3:0/1436223169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e0b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.824+0000 7ff286ffd640 1 -- 172.21.15.3:0/1436223169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 msgr2=0x55c8e0e0b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.824+0000 7ff286ffd640 1 --2- 172.21.15.3:0/1436223169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e0b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.824+0000 7ff286ffd640 1 -- 172.21.15.3:0/1436223169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a37390 msgr2=0x55c8e0e07f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.824+0000 7ff286ffd640 1 --2- 172.21.15.3:0/1436223169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a37390 0x55c8e0e07f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.824+0000 7ff286ffd640 1 -- 172.21.15.3:0/1436223169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e0e09210 con 0x55c8e0a58070 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.824+0000 7ff2877fe640 1 --2- 172.21.15.3:0/1436223169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a37390 0x55c8e0e07f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.824+0000 7ff286ffd640 1 --2- 172.21.15.3:0/1436223169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a58070 0x55c8e0e08440 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff27c000c00 tx=0x7ff27c009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f7a7a8a26309db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.824+0000 7ff26d7fa640 1 -- 172.21.15.3:0/1436223169 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c01a040 con 0x55c8e0a58070 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.824+0000 7ff26d7fa640 1 -- 172.21.15.3:0/1436223169 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff27c0054d0 con 0x55c8e0a58070 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.824+0000 7ff26d7fa640 1 -- 172.21.15.3:0/1436223169 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c029c70 con 0x55c8e0a58070 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.825+0000 7ff2946a3740 1 -- 172.21.15.3:0/1436223169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a58070 msgr2=0x55c8e0e08440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.825+0000 7ff2946a3740 1 --2- 172.21.15.3:0/1436223169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a58070 0x55c8e0e08440 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff27c000c00 tx=0x7ff27c009530 comp rx=0 tx=0).stop 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.825+0000 7ff2946a3740 1 -- 172.21.15.3:0/1436223169 shutdown_connections 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.825+0000 7ff2946a3740 1 --2- 172.21.15.3:0/1436223169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a58070 0x55c8e0e08440 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.825+0000 7ff2946a3740 1 --2- 172.21.15.3:0/1436223169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a37390 0x55c8e0e07f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.825+0000 7ff2946a3740 1 --2- 172.21.15.3:0/1436223169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e0b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.825+0000 7ff2946a3740 1 -- 172.21.15.3:0/1436223169 >> 172.21.15.3:0/1436223169 conn(0x55c8e0a4c460 msgr2=0x55c8e0e0aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.825+0000 7ff2946a3740 1 -- 172.21.15.3:0/1436223169 shutdown_connections 2024-09-21T06:23:30.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.825+0000 7ff2946a3740 1 -- 172.21.15.3:0/1436223169 wait complete. 2024-09-21T06:23:30.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.825+0000 7ff2946a3740 1 -- start start 2024-09-21T06:23:30.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.826+0000 7ff2946a3740 1 Processor -- start 2024-09-21T06:23:30.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.826+0000 7ff2946a3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e1de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.826+0000 7ff2946a3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a37390 0x55c8e0e1e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.826+0000 7ff2946a3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 0x55c8e0e1e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.826+0000 7ff2946a3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b0a90 con 0x55c8e0a37390 2024-09-21T06:23:30.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.826+0000 7ff2946a3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b03c0 con 0x55c8e0a58070 2024-09-21T06:23:30.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.826+0000 7ff2946a3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881ad620 con 0x55c8e0a2f1b0 2024-09-21T06:23:30.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.826+0000 7ff2946a3740 10 client.0 _ll_register_callbacks cb 0x7ff27c009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:30.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.826+0000 7ff286ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e1de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.826+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a37390 0x55c8e0e1e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.826+0000 7ff286ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e1de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:45758/0 (socket says 172.21.15.3:45758) 2024-09-21T06:23:30.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.826+0000 7ff286ffd640 1 -- 172.21.15.3:0/2243171473 learned_addr learned my addr 172.21.15.3:0/2243171473 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.826+0000 7ff286ffd640 1 --2- 172.21.15.3:0/2243171473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 0x55c8e0e1e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff2877fe640 1 -- 172.21.15.3:0/2243171473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 msgr2=0x55c8e0e1de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff2877fe640 1 --2- 172.21.15.3:0/2243171473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e1de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff2877fe640 1 -- 172.21.15.3:0/2243171473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 msgr2=0x55c8e0e1e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff2877fe640 1 --2- 172.21.15.3:0/2243171473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 0x55c8e0e1e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff2877fe640 1 -- 172.21.15.3:0/2243171473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e0e86600 con 0x55c8e0a37390 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff286ffd640 1 --2- 172.21.15.3:0/2243171473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e1de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff2877fe640 1 --2- 172.21.15.3:0/2243171473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a37390 0x55c8e0e1e3a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff278004b40 tx=0x7ff278004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926082e632e3ea2c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff266ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff266ffd640 1 -- 172.21.15.3:0/2243171473 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278060060 con 0x55c8e0a37390 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff266ffd640 1 -- 172.21.15.3:0/2243171473 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7ff278004250 con 0x55c8e0a37390 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff2946a3740 1 -- 172.21.15.3:0/2243171473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff27c0054d0 con 0x55c8e0a37390 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff266ffd640 1 -- 172.21.15.3:0/2243171473 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff27804b040 con 0x55c8e0a37390 2024-09-21T06:23:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff2946a3740 10 client.17928 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.827+0000 7ff2946a3740 1 -- 172.21.15.3:0/2243171473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c8e0e87830 con 0x55c8e0a37390 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.828+0000 7ff266ffd640 1 -- 172.21.15.3:0/2243171473 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff27c0054d0 con 0x55c8e0a37390 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.828+0000 7ff266ffd640 1 -- 172.21.15.3:0/2243171473 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff278003aa0 con 0x55c8e0a37390 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.828+0000 7ff2946a3740 10 client.17928 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.828+0000 7ff2946a3740 10 client.17928 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.828+0000 7ff2946a3740 20 client.17928 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.828+0000 7ff2946a3740 1 --2- 172.21.15.3:0/2243171473 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55c8e09ebda0 0x55c8e0e89f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.828+0000 7ff2946a3740 4 client.17928 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.828+0000 7ff2946a3740 1 -- 172.21.15.3:0/2243171473 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8e0d738a0 con 0x55c8e09ebda0 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.828+0000 7ff286ffd640 1 --2- 172.21.15.3:0/2243171473 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55c8e09ebda0 0x55c8e0e89f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff266ffd640 10 client.17928 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff286ffd640 1 --2- 172.21.15.3:0/2243171473 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55c8e09ebda0 0x55c8e0e89f60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=367224c1a5ce5f94 server_cookie=e03bcf79ec78ddc3 in_seq=0 out_seq=0 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff266ffd640 1 -- 172.21.15.3:0/2243171473 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8e0d738a0 con 0x55c8e09ebda0 2024-09-21T06:23:30.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff266ffd640 10 client.17928 handle_command_reply: tid=0 2024-09-21T06:23:30.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff2946a3740 1 client.17928 shutdown 2024-09-21T06:23:30.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff2946a3740 1 -- 172.21.15.3:0/2243171473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a37390 msgr2=0x55c8e0e1e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff2946a3740 1 --2- 172.21.15.3:0/2243171473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a37390 0x55c8e0e1e3a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff278004b40 tx=0x7ff278004a00 comp rx=0 tx=0).stop 2024-09-21T06:23:30.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff2946a3740 1 -- 172.21.15.3:0/2243171473 shutdown_connections 2024-09-21T06:23:30.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff2946a3740 1 --2- 172.21.15.3:0/2243171473 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55c8e09ebda0 0x55c8e0e89f60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff2946a3740 1 --2- 172.21.15.3:0/2243171473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8e0a58070 0x55c8e0e1e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff2946a3740 1 --2- 172.21.15.3:0/2243171473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8e0a37390 0x55c8e0e1e3a0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff2946a3740 1 --2- 172.21.15.3:0/2243171473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8e0a2f1b0 0x55c8e0e1de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff2946a3740 1 -- 172.21.15.3:0/2243171473 >> 172.21.15.3:0/2243171473 conn(0x55c8e0a4c460 msgr2=0x55c8e0e006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:30.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.829+0000 7ff266ffd640 0 client.17928 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:23:30.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.830+0000 7ff2946a3740 1 -- 172.21.15.3:0/2243171473 shutdown_connections 2024-09-21T06:23:30.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.830+0000 7ff2946a3740 1 -- 172.21.15.3:0/2243171473 wait complete. 2024-09-21T06:23:30.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.830+0000 7ff2946a3740 20 client.17928 trim_cache size 0 max 16384 2024-09-21T06:23:30.836 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:23:30.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.830+0000 7ff28e12f640 1 -- 172.21.15.3:0/3086578684 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff2700b6680 msgr2=0x7ff2700b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.830+0000 7ff28e12f640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff2700b6680 0x7ff2700b8b90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff2800e5610 tx=0x7ff2800053b0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.830+0000 7ff28e12f640 1 -- 172.21.15.3:0/3086578684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff28810de00 msgr2=0x7ff28819fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.830+0000 7ff28e12f640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff28810de00 0x7ff28819fcf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff280009fd0 tx=0x7ff28000ef10 comp rx=0 tx=0).stop 2024-09-21T06:23:30.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.830+0000 7ff28e12f640 1 -- 172.21.15.3:0/3086578684 shutdown_connections 2024-09-21T06:23:30.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.830+0000 7ff28e12f640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff2700b6680 0x7ff2700b8b90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.830+0000 7ff28e12f640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff28810de00 0x7ff28819fcf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.830+0000 7ff28e12f640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff288105280 0x7ff28819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.830+0000 7ff28e12f640 1 --2- 172.21.15.3:0/3086578684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff288104850 0x7ff28819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.831+0000 7ff28e12f640 1 -- 172.21.15.3:0/3086578684 >> 172.21.15.3:0/3086578684 conn(0x7ff288079730 msgr2=0x7ff288100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:30.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.831+0000 7ff28e12f640 1 -- 172.21.15.3:0/3086578684 shutdown_connections 2024-09-21T06:23:30.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:30.831+0000 7ff28e12f640 1 -- 172.21.15.3:0/3086578684 wait complete. 2024-09-21T06:23:30.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 885911 10.56 MB/sec execute 303 sec latency 78.276 ms 2024-09-21T06:23:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:31 smithi003 ceph-mon[31227]: pgmap v586: 97 pgs: 97 active+clean; 139 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.9 MiB/s wr, 288 op/s 2024-09-21T06:23:31.621 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:23:31.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:31 smithi137 ceph-mon[34464]: pgmap v586: 97 pgs: 97 active+clean; 139 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.9 MiB/s wr, 288 op/s 2024-09-21T06:23:31.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:31 smithi204 ceph-mon[34659]: pgmap v586: 97 pgs: 97 active+clean; 139 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.9 MiB/s wr, 288 op/s 2024-09-21T06:23:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:31.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 887804 10.57 MB/sec execute 304 sec latency 19.533 ms 2024-09-21T06:23:32.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.441+0000 7f4e755a1640 1 -- 172.21.15.3:0/1060110280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70103b80 msgr2=0x7f4e70103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.441+0000 7f4e755a1640 1 --2- 172.21.15.3:0/1060110280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70103b80 0x7f4e70103f90 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009900 tx=0x7f4e6002a410 comp rx=0 tx=0).stop 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.441+0000 7f4e755a1640 1 -- 172.21.15.3:0/1060110280 shutdown_connections 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.442+0000 7f4e755a1640 1 --2- 172.21.15.3:0/1060110280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e70109e10 0x7f4e70110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.442+0000 7f4e755a1640 1 --2- 172.21.15.3:0/1060110280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e701094c0 0x7f4e701098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.442+0000 7f4e755a1640 1 --2- 172.21.15.3:0/1060110280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70103b80 0x7f4e70103f90 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.442+0000 7f4e755a1640 1 -- 172.21.15.3:0/1060110280 >> 172.21.15.3:0/1060110280 conn(0x7f4e700ff610 msgr2=0x7f4e70101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.442+0000 7f4e755a1640 1 -- 172.21.15.3:0/1060110280 shutdown_connections 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.442+0000 7f4e755a1640 1 -- 172.21.15.3:0/1060110280 wait complete. 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.442+0000 7f4e755a1640 1 Processor -- start 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.442+0000 7f4e755a1640 1 -- start start 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e755a1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70103b80 0x7f4e7019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e755a1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e701094c0 0x7f4e7019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e755a1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e70109e10 0x7f4e701a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e755a1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70113ad0 con 0x7f4e70103b80 2024-09-21T06:23:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e755a1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70113950 con 0x7f4e70109e10 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e755a1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70113c50 con 0x7f4e701094c0 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e6effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70103b80 0x7f4e7019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e6effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70103b80 0x7f4e7019c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48968/0 (socket says 172.21.15.3:48968) 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e6effd640 1 -- 172.21.15.3:0/842736231 learned_addr learned my addr 172.21.15.3:0/842736231 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e67fff640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e701094c0 0x7f4e7019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e6f7fe640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e70109e10 0x7f4e701a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e6effd640 1 -- 172.21.15.3:0/842736231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e701094c0 msgr2=0x7f4e7019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e6effd640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e701094c0 0x7f4e7019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e6effd640 1 -- 172.21.15.3:0/842736231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e70109e10 msgr2=0x7f4e701a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e6effd640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e70109e10 0x7f4e701a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.443+0000 7f4e6effd640 1 -- 172.21.15.3:0/842736231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e701a2610 con 0x7f4e70103b80 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.444+0000 7f4e6effd640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70103b80 0x7f4e7019c6a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4e60005940 tx=0x7f4e60004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.444+0000 7f4e6cff9640 1 -- 172.21.15.3:0/842736231 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6003f030 con 0x7f4e70103b80 2024-09-21T06:23:32.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.444+0000 7f4e6cff9640 1 -- 172.21.15.3:0/842736231 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e60003aa0 con 0x7f4e70103b80 2024-09-21T06:23:32.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.444+0000 7f4e755a1640 1 -- 172.21.15.3:0/842736231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e701a28a0 con 0x7f4e70103b80 2024-09-21T06:23:32.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.444+0000 7f4e6cff9640 1 -- 172.21.15.3:0/842736231 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60035070 con 0x7f4e70103b80 2024-09-21T06:23:32.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.444+0000 7f4e755a1640 1 -- 172.21.15.3:0/842736231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e701a9060 con 0x7f4e70103b80 2024-09-21T06:23:32.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.446+0000 7f4e6cff9640 1 -- 172.21.15.3:0/842736231 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f4e60042030 con 0x7f4e70103b80 2024-09-21T06:23:32.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.447+0000 7f4e7bb15740 1 Processor -- start 2024-09-21T06:23:32.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.447+0000 7f4e7bb15740 1 -- start start 2024-09-21T06:23:32.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.447+0000 7f4e7bb15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94c9070 0x555fa98702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.447+0000 7f4e7bb15740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a01b0 0x555fa9870800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.447+0000 7f4e7bb15740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94a8390 0x555fa9870d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.447+0000 7f4e7bb15740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555fa97e6ea0 con 0x555fa94a01b0 2024-09-21T06:23:32.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.447+0000 7f4e7bb15740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555fa97d62d0 con 0x555fa94a8390 2024-09-21T06:23:32.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.447+0000 7f4e7bb15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555fa97da0e0 con 0x555fa94c9070 2024-09-21T06:23:32.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.447+0000 7f4e6effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a01b0 0x555fa9870800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.447+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94a8390 0x555fa9870d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.447+0000 7f4e6effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a01b0 0x555fa9870800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48974/0 (socket says 172.21.15.3:48974) 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.447+0000 7f4e6effd640 1 -- 172.21.15.3:0/1969390963 learned_addr learned my addr 172.21.15.3:0/1969390963 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.448+0000 7f4e6f7fe640 1 --2- 172.21.15.3:0/1969390963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94c9070 0x555fa98702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.448+0000 7f4e6cff9640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4e5c0b6680 0x7f4e5c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.448+0000 7f4e67fff640 1 -- 172.21.15.3:0/1969390963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94c9070 msgr2=0x555fa98702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.448+0000 7f4e6f7fe640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4e5c0b6680 0x7f4e5c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.448+0000 7f4e67fff640 1 --2- 172.21.15.3:0/1969390963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94c9070 0x555fa98702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.448+0000 7f4e67fff640 1 -- 172.21.15.3:0/1969390963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a01b0 msgr2=0x555fa9870800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.448+0000 7f4e67fff640 1 --2- 172.21.15.3:0/1969390963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a01b0 0x555fa9870800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.448+0000 7f4e67fff640 1 -- 172.21.15.3:0/1969390963 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa9534650 con 0x555fa94a8390 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.448+0000 7f4e6cff9640 1 -- 172.21.15.3:0/842736231 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f4e60107d70 con 0x7f4e70103b80 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.448+0000 7f4e67fff640 1 --2- 172.21.15.3:0/1969390963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94a8390 0x555fa9870d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4e7006c970 tx=0x7f4e58011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2f21d7a3f154bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:32.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.448+0000 7f4e657fa640 1 -- 172.21.15.3:0/1969390963 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58004050 con 0x555fa94a8390 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.448+0000 7f4e657fa640 1 -- 172.21.15.3:0/1969390963 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e580054d0 con 0x555fa94a8390 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.448+0000 7f4e657fa640 1 -- 172.21.15.3:0/1969390963 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58004c80 con 0x555fa94a8390 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.449+0000 7f4e7bb15740 1 -- 172.21.15.3:0/1969390963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94a8390 msgr2=0x555fa9870d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.449+0000 7f4e6f7fe640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4e5c0b6680 0x7f4e5c0b8b90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4e680098b0 tx=0x7f4e68004ab0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.449+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/1969390963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94a8390 0x555fa9870d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4e7006c970 tx=0x7f4e58011810 comp rx=0 tx=0).stop 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.449+0000 7f4e7bb15740 1 -- 172.21.15.3:0/1969390963 shutdown_connections 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.449+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/1969390963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94a8390 0x555fa9870d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.449+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/1969390963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a01b0 0x555fa9870800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.449+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/1969390963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94c9070 0x555fa98702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.449+0000 7f4e7bb15740 1 -- 172.21.15.3:0/1969390963 >> 172.21.15.3:0/1969390963 conn(0x555fa94bd460 msgr2=0x555fa986e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.449+0000 7f4e7bb15740 1 -- 172.21.15.3:0/1969390963 shutdown_connections 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.449+0000 7f4e7bb15740 1 -- 172.21.15.3:0/1969390963 wait complete. 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.450+0000 7f4e7bb15740 1 -- start start 2024-09-21T06:23:32.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.450+0000 7f4e7bb15740 1 Processor -- start 2024-09-21T06:23:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.450+0000 7f4e7bb15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94a01b0 0x555fa987b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.450+0000 7f4e7bb15740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 0x555fa987be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.450+0000 7f4e7bb15740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94c9070 0x555fa987c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.450+0000 7f4e7bb15740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555fa9637700 con 0x555fa94a8390 2024-09-21T06:23:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.450+0000 7f4e7bb15740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555fa9771160 con 0x555fa94c9070 2024-09-21T06:23:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.450+0000 7f4e7bb15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555fa987eaa0 con 0x555fa94a01b0 2024-09-21T06:23:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.450+0000 7f4e7bb15740 10 client.0 _ll_register_callbacks cb 0x555fa978a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.451+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 0x555fa987be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.451+0000 7f4e6f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94c9070 0x555fa987c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.451+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 0x555fa987be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48986/0 (socket says 172.21.15.3:48986) 2024-09-21T06:23:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.451+0000 7f4e67fff640 1 -- 172.21.15.3:0/1607057694 learned_addr learned my addr 172.21.15.3:0/1607057694 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.451+0000 7f4e6effd640 1 --2- 172.21.15.3:0/1607057694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94a01b0 0x555fa987b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.451+0000 7f4e67fff640 1 -- 172.21.15.3:0/1607057694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94a01b0 msgr2=0x555fa987b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.451+0000 7f4e67fff640 1 --2- 172.21.15.3:0/1607057694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94a01b0 0x555fa987b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.451+0000 7f4e67fff640 1 -- 172.21.15.3:0/1607057694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94c9070 msgr2=0x555fa987c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.451+0000 7f4e67fff640 1 --2- 172.21.15.3:0/1607057694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94c9070 0x555fa987c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.451+0000 7f4e67fff640 1 -- 172.21.15.3:0/1607057694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa979ec90 con 0x555fa94a8390 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.451+0000 7f4e67fff640 1 --2- 172.21.15.3:0/1607057694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 0x555fa987be20 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4e58004920 tx=0x7f4e58009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ee9834d6ddea3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.451+0000 7f4e6f7fe640 1 --2- 172.21.15.3:0/1607057694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94c9070 0x555fa987c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.451+0000 7f4e567fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.452+0000 7f4e567fc640 1 -- 172.21.15.3:0/1607057694 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58027030 con 0x555fa94a8390 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.452+0000 7f4e567fc640 1 -- 172.21.15.3:0/1607057694 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f4e580035f0 con 0x555fa94a8390 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.452+0000 7f4e7bb15740 1 -- 172.21.15.3:0/1607057694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e580054d0 con 0x555fa94a8390 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.452+0000 7f4e567fc640 1 -- 172.21.15.3:0/1607057694 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4e5801d040 con 0x555fa94a8390 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.452+0000 7f4e7bb15740 10 client.17940 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.452+0000 7f4e7bb15740 1 -- 172.21.15.3:0/1607057694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555fa988a720 con 0x555fa94a8390 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.452+0000 7f4e567fc640 1 -- 172.21.15.3:0/1607057694 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e580054d0 con 0x555fa94a8390 2024-09-21T06:23:32.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.452+0000 7f4e567fc640 1 -- 172.21.15.3:0/1607057694 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f4e58004c20 con 0x555fa94a8390 2024-09-21T06:23:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.452+0000 7f4e7bb15740 10 client.17940 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.452+0000 7f4e7bb15740 10 client.17940 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:23:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.452+0000 7f4e7bb15740 20 client.17940 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.452+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/1607057694 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555fa94879f0 0x555fa988da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.453+0000 7f4e7bb15740 4 client.17940 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.453+0000 7f4e7bb15740 1 -- 172.21.15.3:0/1607057694 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fa97e8df0 con 0x555fa94879f0 2024-09-21T06:23:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.453+0000 7f4e6effd640 1 --2- 172.21.15.3:0/1607057694 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555fa94879f0 0x555fa988da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.453+0000 7f4e567fc640 10 client.17940 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:23:32.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.453+0000 7f4e6effd640 1 --2- 172.21.15.3:0/1607057694 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555fa94879f0 0x555fa988da80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd6bf07afd464aa3 server_cookie=99f3494f7ed948f0 in_seq=0 out_seq=0 2024-09-21T06:23:32.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e567fc640 1 -- 172.21.15.3:0/1607057694 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555fa97e8df0 con 0x555fa94879f0 2024-09-21T06:23:32.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e567fc640 10 client.17940 handle_command_reply: tid=0 2024-09-21T06:23:32.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e7bb15740 1 client.17940 shutdown 2024-09-21T06:23:32.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e7bb15740 1 -- 172.21.15.3:0/1607057694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 msgr2=0x555fa987be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/1607057694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 0x555fa987be20 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4e58004920 tx=0x7f4e58009f80 comp rx=0 tx=0).stop 2024-09-21T06:23:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e7bb15740 1 -- 172.21.15.3:0/1607057694 shutdown_connections 2024-09-21T06:23:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/1607057694 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555fa94879f0 0x555fa988da80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e567fc640 0 client.17940 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:23:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/1607057694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94c9070 0x555fa987c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/1607057694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 0x555fa987be20 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/1607057694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94a01b0 0x555fa987b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e7bb15740 1 -- 172.21.15.3:0/1607057694 >> 172.21.15.3:0/1607057694 conn(0x555fa94bd460 msgr2=0x555fa9873450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e7bb15740 1 -- 172.21.15.3:0/1607057694 shutdown_connections 2024-09-21T06:23:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e7bb15740 1 -- 172.21.15.3:0/1607057694 wait complete. 2024-09-21T06:23:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.454+0000 7f4e7bb15740 20 client.17940 trim_cache size 0 max 16384 2024-09-21T06:23:32.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.468+0000 7f4e7bb15740 1 Processor -- start 2024-09-21T06:23:32.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.468+0000 7f4e7bb15740 1 -- start start 2024-09-21T06:23:32.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.469+0000 7f4e7bb15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94879f0 0x555fa987d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.469+0000 7f4e7bb15740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a01b0 0x555fa9871a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.469+0000 7f4e7bb15740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94a8390 0x555fa9878ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.469+0000 7f4e7bb15740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4e701b18f0 con 0x555fa94a01b0 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.469+0000 7f4e7bb15740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4e701b3430 con 0x555fa94a8390 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.469+0000 7f4e7bb15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e701b1fc0 con 0x555fa94879f0 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.469+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a01b0 0x555fa9871a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.469+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a01b0 0x555fa9871a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49002/0 (socket says 172.21.15.3:49002) 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.469+0000 7f4e6effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94879f0 0x555fa987d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.469+0000 7f4e67fff640 1 -- 172.21.15.3:0/3947091970 learned_addr learned my addr 172.21.15.3:0/3947091970 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.469+0000 7f4e6f7fe640 1 --2- 172.21.15.3:0/3947091970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94a8390 0x555fa9878ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.470+0000 7f4e6effd640 1 -- 172.21.15.3:0/3947091970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94a8390 msgr2=0x555fa9878ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.470+0000 7f4e6effd640 1 --2- 172.21.15.3:0/3947091970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94a8390 0x555fa9878ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.470+0000 7f4e6effd640 1 -- 172.21.15.3:0/3947091970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a01b0 msgr2=0x555fa9871a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.470+0000 7f4e6effd640 1 --2- 172.21.15.3:0/3947091970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a01b0 0x555fa9871a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.470+0000 7f4e6effd640 1 -- 172.21.15.3:0/3947091970 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa9873120 con 0x555fa94879f0 2024-09-21T06:23:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.470+0000 7f4e6effd640 1 --2- 172.21.15.3:0/3947091970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94879f0 0x555fa987d7c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4e600cd980 tx=0x7f4e600f1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23ea6d0ed79299d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.470+0000 7f4e64ff9640 1 -- 172.21.15.3:0/3947091970 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60049040 con 0x555fa94879f0 2024-09-21T06:23:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.470+0000 7f4e64ff9640 1 -- 172.21.15.3:0/3947091970 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e600cd790 con 0x555fa94879f0 2024-09-21T06:23:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.470+0000 7f4e64ff9640 1 -- 172.21.15.3:0/3947091970 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60105750 con 0x555fa94879f0 2024-09-21T06:23:32.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.471+0000 7f4e7bb15740 1 -- 172.21.15.3:0/3947091970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94879f0 msgr2=0x555fa987d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.471+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/3947091970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94879f0 0x555fa987d7c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4e600cd980 tx=0x7f4e600f1040 comp rx=0 tx=0).stop 2024-09-21T06:23:32.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.471+0000 7f4e7bb15740 1 -- 172.21.15.3:0/3947091970 shutdown_connections 2024-09-21T06:23:32.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.471+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/3947091970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94a8390 0x555fa9878ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.471+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/3947091970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a01b0 0x555fa9871a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.471+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/3947091970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94879f0 0x555fa987d7c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.471+0000 7f4e7bb15740 1 -- 172.21.15.3:0/3947091970 >> 172.21.15.3:0/3947091970 conn(0x555fa94bd460 msgr2=0x555fa98712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.471+0000 7f4e7bb15740 1 -- 172.21.15.3:0/3947091970 shutdown_connections 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.471+0000 7f4e7bb15740 1 -- 172.21.15.3:0/3947091970 wait complete. 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e7bb15740 1 -- start start 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e7bb15740 1 Processor -- start 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e7bb15740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94879f0 0x555fa98f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e7bb15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94a01b0 0x555fa98f9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e7bb15740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 0x555fa987f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e7bb15740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4e701b2d60 con 0x555fa94a8390 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e7bb15740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4e701b2690 con 0x555fa94879f0 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e7bb15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e701af8f0 con 0x555fa94a01b0 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e7bb15740 10 client.0 _ll_register_callbacks cb 0x7f4e600c9b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94a01b0 0x555fa98f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e6f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 0x555fa987f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e6effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94879f0 0x555fa98f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e6f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 0x555fa987f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49012/0 (socket says 172.21.15.3:49012) 2024-09-21T06:23:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.472+0000 7f4e6f7fe640 1 -- 172.21.15.3:0/2607190712 learned_addr learned my addr 172.21.15.3:0/2607190712 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e6effd640 1 -- 172.21.15.3:0/2607190712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94a01b0 msgr2=0x555fa98f9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e6effd640 1 --2- 172.21.15.3:0/2607190712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94a01b0 0x555fa98f9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e6effd640 1 -- 172.21.15.3:0/2607190712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 msgr2=0x555fa987f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e6effd640 1 --2- 172.21.15.3:0/2607190712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 0x555fa987f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e6effd640 1 -- 172.21.15.3:0/2607190712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa9887c90 con 0x555fa94879f0 2024-09-21T06:23:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e6f7fe640 1 --2- 172.21.15.3:0/2607190712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 0x555fa987f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e67fff640 1 --2- 172.21.15.3:0/2607190712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94a01b0 0x555fa98f9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e6effd640 1 --2- 172.21.15.3:0/2607190712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94879f0 0x555fa98f8e90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4e60005a90 tx=0x7f4e60003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b83e9b524b96e787 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e567fc640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:23:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e567fc640 1 -- 172.21.15.3:0/2607190712 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60049070 con 0x555fa94879f0 2024-09-21T06:23:32.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e7bb15740 1 -- 172.21.15.3:0/2607190712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e600cd790 con 0x555fa94879f0 2024-09-21T06:23:32.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e567fc640 1 -- 172.21.15.3:0/2607190712 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f4e6003bcc0 con 0x555fa94879f0 2024-09-21T06:23:32.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e567fc640 1 -- 172.21.15.3:0/2607190712 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4e600cd3f0 con 0x555fa94879f0 2024-09-21T06:23:32.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e7bb15740 10 client.27506 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:32.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.473+0000 7f4e7bb15740 1 -- 172.21.15.3:0/2607190712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555fa9900480 con 0x555fa94879f0 2024-09-21T06:23:32.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.474+0000 7f4e567fc640 1 -- 172.21.15.3:0/2607190712 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e600cd790 con 0x555fa94879f0 2024-09-21T06:23:32.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.474+0000 7f4e567fc640 1 -- 172.21.15.3:0/2607190712 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f4e600f10b0 con 0x555fa94879f0 2024-09-21T06:23:32.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.474+0000 7f4e7bb15740 10 client.27506 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.474+0000 7f4e7bb15740 10 client.27506 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.474+0000 7f4e7bb15740 20 client.27506 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.474+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/2607190712 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555fa94c9070 0x555fa9902b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.474+0000 7f4e7bb15740 4 client.27506 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.474+0000 7f4e7bb15740 1 -- 172.21.15.3:0/2607190712 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555fa97e48a0 con 0x555fa94c9070 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.474+0000 7f4e67fff640 1 --2- 172.21.15.3:0/2607190712 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555fa94c9070 0x555fa9902b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e67fff640 1 --2- 172.21.15.3:0/2607190712 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555fa94c9070 0x555fa9902b60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9ccdbc749b20e11 server_cookie=b3254c0b5eb4a2c6 in_seq=0 out_seq=0 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e567fc640 10 client.27506 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e567fc640 1 -- 172.21.15.3:0/2607190712 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555fa97e48a0 con 0x555fa94c9070 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e567fc640 10 client.27506 handle_command_reply: tid=0 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e7bb15740 1 client.27506 shutdown 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e7bb15740 1 -- 172.21.15.3:0/2607190712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94879f0 msgr2=0x555fa98f8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/2607190712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94879f0 0x555fa98f8e90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4e60005a90 tx=0x7f4e60003e80 comp rx=0 tx=0).stop 2024-09-21T06:23:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e7bb15740 1 -- 172.21.15.3:0/2607190712 shutdown_connections 2024-09-21T06:23:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/2607190712 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x555fa94c9070 0x555fa9902b60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/2607190712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fa94a01b0 0x555fa98f9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/2607190712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555fa94a8390 0x555fa987f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e567fc640 0 client.27506 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:23:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e7bb15740 1 --2- 172.21.15.3:0/2607190712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fa94879f0 0x555fa98f8e90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e7bb15740 1 -- 172.21.15.3:0/2607190712 >> 172.21.15.3:0/2607190712 conn(0x555fa94bd460 msgr2=0x555fa9887410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e7bb15740 1 -- 172.21.15.3:0/2607190712 shutdown_connections 2024-09-21T06:23:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e7bb15740 1 -- 172.21.15.3:0/2607190712 wait complete. 2024-09-21T06:23:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.475+0000 7f4e7bb15740 20 client.27506 trim_cache size 0 max 16384 2024-09-21T06:23:32.481 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:23:32.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.476+0000 7f4e755a1640 1 -- 172.21.15.3:0/842736231 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4e5c0b6680 msgr2=0x7f4e5c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.476+0000 7f4e755a1640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4e5c0b6680 0x7f4e5c0b8b90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4e680098b0 tx=0x7f4e68004ab0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.476+0000 7f4e755a1640 1 -- 172.21.15.3:0/842736231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70103b80 msgr2=0x7f4e7019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.476+0000 7f4e755a1640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70103b80 0x7f4e7019c6a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4e60005940 tx=0x7f4e60004270 comp rx=0 tx=0).stop 2024-09-21T06:23:32.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.476+0000 7f4e755a1640 1 -- 172.21.15.3:0/842736231 shutdown_connections 2024-09-21T06:23:32.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.476+0000 7f4e755a1640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4e5c0b6680 0x7f4e5c0b8b90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.476+0000 7f4e755a1640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e701094c0 0x7f4e7019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.476+0000 7f4e755a1640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e70109e10 0x7f4e701a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.476+0000 7f4e755a1640 1 --2- 172.21.15.3:0/842736231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70103b80 0x7f4e7019c6a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.476+0000 7f4e755a1640 1 -- 172.21.15.3:0/842736231 >> 172.21.15.3:0/842736231 conn(0x7f4e700ff610 msgr2=0x7f4e70108b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:32.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.476+0000 7f4e755a1640 1 -- 172.21.15.3:0/842736231 shutdown_connections 2024-09-21T06:23:32.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:32.476+0000 7f4e755a1640 1 -- 172.21.15.3:0/842736231 wait complete. 2024-09-21T06:23:32.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 889258 10.56 MB/sec execute 305 sec latency 20.330 ms 2024-09-21T06:23:33.289 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:23:33.554 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:33.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:33 smithi003 ceph-mon[31227]: pgmap v587: 97 pgs: 97 active+clean; 135 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-09-21T06:23:33.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:33 smithi137 ceph-mon[34464]: pgmap v587: 97 pgs: 97 active+clean; 135 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-09-21T06:23:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:33 smithi204 ceph-mon[34659]: pgmap v587: 97 pgs: 97 active+clean; 135 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-09-21T06:23:33.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 891219 10.56 MB/sec execute 306 sec latency 64.088 ms 2024-09-21T06:23:34.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.059+0000 7f1c4d830640 1 -- 172.21.15.3:0/3980708552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c48107b30 msgr2=0x7f1c48107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.059+0000 7f1c4d830640 1 --2- 172.21.15.3:0/3980708552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c48107b30 0x7f1c48107f40 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009900 tx=0x7f1c3002a410 comp rx=0 tx=0).stop 2024-09-21T06:23:34.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.059+0000 7f1c4d830640 1 -- 172.21.15.3:0/3980708552 shutdown_connections 2024-09-21T06:23:34.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.059+0000 7f1c4d830640 1 --2- 172.21.15.3:0/3980708552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c48108480 0x7f1c4810ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.059+0000 7f1c4d830640 1 --2- 172.21.15.3:0/3980708552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c48107b30 0x7f1c48107f40 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.059+0000 7f1c4d830640 1 --2- 172.21.15.3:0/3980708552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c481021f0 0x7f1c48102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.059+0000 7f1c4d830640 1 -- 172.21.15.3:0/3980708552 >> 172.21.15.3:0/3980708552 conn(0x7f1c48077c60 msgr2=0x7f1c480780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.059+0000 7f1c4d830640 1 -- 172.21.15.3:0/3980708552 shutdown_connections 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.059+0000 7f1c4d830640 1 -- 172.21.15.3:0/3980708552 wait complete. 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.060+0000 7f1c4d830640 1 Processor -- start 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.060+0000 7f1c4d830640 1 -- start start 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c4d830640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c481021f0 0x7f1c4819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c4d830640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c48107b30 0x7f1c4819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c4d830640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c48108480 0x7f1c4819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c4d830640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48113ab0 con 0x7f1c48107b30 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c4d830640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48113930 con 0x7f1c48108480 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c4d830640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48113c30 con 0x7f1c481021f0 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c467fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c48107b30 0x7f1c4819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c467fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c48107b30 0x7f1c4819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49016/0 (socket says 172.21.15.3:49016) 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c46ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c481021f0 0x7f1c4819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c467fc640 1 -- 172.21.15.3:0/1474946826 learned_addr learned my addr 172.21.15.3:0/1474946826 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c477fe640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c48108480 0x7f1c4819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c467fc640 1 -- 172.21.15.3:0/1474946826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c481021f0 msgr2=0x7f1c4819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c467fc640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c481021f0 0x7f1c4819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c467fc640 1 -- 172.21.15.3:0/1474946826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c48108480 msgr2=0x7f1c4819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c467fc640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c48108480 0x7f1c4819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c467fc640 1 -- 172.21.15.3:0/1474946826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c481a0490 con 0x7f1c48107b30 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c46ffd640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c481021f0 0x7f1c4819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.061+0000 7f1c467fc640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c48107b30 0x7f1c4819aac0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1c3002a930 tx=0x7f1c30031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.062+0000 7f1c4c82e640 1 -- 172.21.15.3:0/1474946826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3003f030 con 0x7f1c48107b30 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.062+0000 7f1c4c82e640 1 -- 172.21.15.3:0/1474946826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c30004490 con 0x7f1c48107b30 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.062+0000 7f1c4d830640 1 -- 172.21.15.3:0/1474946826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c481a0720 con 0x7f1c48107b30 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.062+0000 7f1c4c82e640 1 -- 172.21.15.3:0/1474946826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30035070 con 0x7f1c48107b30 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.062+0000 7f1c4d830640 1 -- 172.21.15.3:0/1474946826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c481a6ee0 con 0x7f1c48107b30 2024-09-21T06:23:34.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.064+0000 7f1c53da4740 1 Processor -- start 2024-09-21T06:23:34.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.064+0000 7f1c4c82e640 1 -- 172.21.15.3:0/1474946826 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f1c30042030 con 0x7f1c48107b30 2024-09-21T06:23:34.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.064+0000 7f1c53da4740 1 -- start start 2024-09-21T06:23:34.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c53da4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467f18070 0x5624682bf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c53da4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467eef1b0 0x5624682bfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c53da4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467ef7390 0x5624682c0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c53da4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562468235ea0 con 0x562467f18070 2024-09-21T06:23:34.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c53da4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5624682252d0 con 0x562467ef7390 2024-09-21T06:23:34.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c53da4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5624682290e0 con 0x562467eef1b0 2024-09-21T06:23:34.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467f18070 0x5624682bf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467f18070 0x5624682bf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49020/0 (socket says 172.21.15.3:49020) 2024-09-21T06:23:34.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c46ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467eef1b0 0x5624682bfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c477fe640 1 -- 172.21.15.3:0/1993358315 learned_addr learned my addr 172.21.15.3:0/1993358315 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:34.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c467fc640 1 --2- 172.21.15.3:0/1993358315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467ef7390 0x5624682c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c46ffd640 1 -- 172.21.15.3:0/1993358315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467ef7390 msgr2=0x5624682c0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:34.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c46ffd640 1 --2- 172.21.15.3:0/1993358315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467ef7390 0x5624682c0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c46ffd640 1 -- 172.21.15.3:0/1993358315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467f18070 msgr2=0x5624682bf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:34.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c46ffd640 1 --2- 172.21.15.3:0/1993358315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467f18070 0x5624682bf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c46ffd640 1 -- 172.21.15.3:0/1993358315 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562467f83650 con 0x562467eef1b0 2024-09-21T06:23:34.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c477fe640 1 --2- 172.21.15.3:0/1993358315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467f18070 0x5624682bf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:34.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.065+0000 7f1c4c82e640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1c340b6680 0x7f1c340b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.066+0000 7f1c46ffd640 1 --2- 172.21.15.3:0/1993358315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467eef1b0 0x5624682bfbe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005fd0 tx=0x7f1c40011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7468adc337300b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:34.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.066+0000 7f1c477fe640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1c340b6680 0x7f1c340b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.066+0000 7f1c4c82e640 1 -- 172.21.15.3:0/1474946826 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f1c301079d0 con 0x7f1c48107b30 2024-09-21T06:23:34.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.066+0000 7f1c3dffb640 1 -- 172.21.15.3:0/1993358315 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40004d20 con 0x562467eef1b0 2024-09-21T06:23:34.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.066+0000 7f1c3dffb640 1 -- 172.21.15.3:0/1993358315 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c40003970 con 0x562467eef1b0 2024-09-21T06:23:34.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.066+0000 7f1c3dffb640 1 -- 172.21.15.3:0/1993358315 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c400043c0 con 0x562467eef1b0 2024-09-21T06:23:34.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.067+0000 7f1c53da4740 1 -- 172.21.15.3:0/1993358315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467eef1b0 msgr2=0x5624682bfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.067+0000 7f1c53da4740 1 --2- 172.21.15.3:0/1993358315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467eef1b0 0x5624682bfbe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005fd0 tx=0x7f1c40011c70 comp rx=0 tx=0).stop 2024-09-21T06:23:34.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.067+0000 7f1c477fe640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1c340b6680 0x7f1c340b8b90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1c380124a0 tx=0x7f1c38005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:34.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.067+0000 7f1c53da4740 1 -- 172.21.15.3:0/1993358315 shutdown_connections 2024-09-21T06:23:34.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.067+0000 7f1c53da4740 1 --2- 172.21.15.3:0/1993358315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467eef1b0 0x5624682bfbe0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.067+0000 7f1c53da4740 1 --2- 172.21.15.3:0/1993358315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467ef7390 0x5624682c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.067+0000 7f1c53da4740 1 --2- 172.21.15.3:0/1993358315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467f18070 0x5624682bf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.067+0000 7f1c53da4740 1 -- 172.21.15.3:0/1993358315 >> 172.21.15.3:0/1993358315 conn(0x562467f0c460 msgr2=0x5624682bd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:34.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.067+0000 7f1c53da4740 1 -- 172.21.15.3:0/1993358315 shutdown_connections 2024-09-21T06:23:34.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.067+0000 7f1c53da4740 1 -- 172.21.15.3:0/1993358315 wait complete. 2024-09-21T06:23:34.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.067+0000 7f1c53da4740 1 -- start start 2024-09-21T06:23:34.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c53da4740 1 Processor -- start 2024-09-21T06:23:34.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c53da4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467eef1b0 0x5624682cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c53da4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467ef7390 0x5624682cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c53da4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467f18070 0x5624682cb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c53da4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562468086700 con 0x562467ef7390 2024-09-21T06:23:34.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c53da4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5624681c0160 con 0x562467eef1b0 2024-09-21T06:23:34.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c53da4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5624682cde80 con 0x562467f18070 2024-09-21T06:23:34.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c53da4740 10 client.0 _ll_register_callbacks cb 0x5624681d99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:34.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c467fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467ef7390 0x5624682cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c46ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467eef1b0 0x5624682cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c477fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467f18070 0x5624682cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c467fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467ef7390 0x5624682cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49032/0 (socket says 172.21.15.3:49032) 2024-09-21T06:23:34.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c467fc640 1 -- 172.21.15.3:0/3893474858 learned_addr learned my addr 172.21.15.3:0/3893474858 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:34.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c467fc640 1 -- 172.21.15.3:0/3893474858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467f18070 msgr2=0x5624682cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:34.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.068+0000 7f1c467fc640 1 --2- 172.21.15.3:0/3893474858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467f18070 0x5624682cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.069+0000 7f1c467fc640 1 -- 172.21.15.3:0/3893474858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467eef1b0 msgr2=0x5624682cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:34.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.069+0000 7f1c467fc640 1 --2- 172.21.15.3:0/3893474858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467eef1b0 0x5624682cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.069+0000 7f1c467fc640 1 -- 172.21.15.3:0/3893474858 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624681edc90 con 0x562467ef7390 2024-09-21T06:23:34.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.069+0000 7f1c467fc640 1 --2- 172.21.15.3:0/3893474858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467ef7390 0x5624682cb1d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1c30105190 tx=0x7f1c301055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2714c9346ef8b210 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:34.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.069+0000 7f1c26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:34.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.069+0000 7f1c26ffd640 1 -- 172.21.15.3:0/3893474858 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30049070 con 0x562467ef7390 2024-09-21T06:23:34.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.069+0000 7f1c26ffd640 1 -- 172.21.15.3:0/3893474858 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f1c300d6fc0 con 0x562467ef7390 2024-09-21T06:23:34.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.069+0000 7f1c53da4740 1 -- 172.21.15.3:0/3893474858 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c40003970 con 0x562467ef7390 2024-09-21T06:23:34.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.069+0000 7f1c26ffd640 1 -- 172.21.15.3:0/3893474858 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1c300c9f30 con 0x562467ef7390 2024-09-21T06:23:34.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.069+0000 7f1c53da4740 10 client.17958 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:34.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.069+0000 7f1c53da4740 1 -- 172.21.15.3:0/3893474858 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5624682d9a90 con 0x562467ef7390 2024-09-21T06:23:34.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.069+0000 7f1c26ffd640 1 -- 172.21.15.3:0/3893474858 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c40003970 con 0x562467ef7390 2024-09-21T06:23:34.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.070+0000 7f1c26ffd640 1 -- 172.21.15.3:0/3893474858 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f1c300ceb60 con 0x562467ef7390 2024-09-21T06:23:34.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.070+0000 7f1c53da4740 10 client.17958 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:34.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.070+0000 7f1c53da4740 10 client.17958 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:23:34.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.070+0000 7f1c53da4740 20 client.17958 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:34.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.070+0000 7f1c53da4740 1 --2- 172.21.15.3:0/3893474858 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562467ed69f0 0x5624682dd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.070+0000 7f1c53da4740 4 client.17958 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:34.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.070+0000 7f1c53da4740 1 -- 172.21.15.3:0/3893474858 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562468237df0 con 0x562467ed69f0 2024-09-21T06:23:34.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.070+0000 7f1c46ffd640 1 --2- 172.21.15.3:0/3893474858 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562467ed69f0 0x5624682dd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.070+0000 7f1c26ffd640 10 client.17958 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:23:34.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.070+0000 7f1c46ffd640 1 --2- 172.21.15.3:0/3893474858 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562467ed69f0 0x5624682dd050 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=221f136143e6e356 server_cookie=f307a985abfa1c83 in_seq=0 out_seq=0 2024-09-21T06:23:34.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.071+0000 7f1c26ffd640 1 -- 172.21.15.3:0/3893474858 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562468237df0 con 0x562467ed69f0 2024-09-21T06:23:34.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.071+0000 7f1c26ffd640 10 client.17958 handle_command_reply: tid=0 2024-09-21T06:23:34.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.071+0000 7f1c53da4740 1 client.17958 shutdown 2024-09-21T06:23:34.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.071+0000 7f1c53da4740 1 -- 172.21.15.3:0/3893474858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467ef7390 msgr2=0x5624682cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.071+0000 7f1c53da4740 1 --2- 172.21.15.3:0/3893474858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467ef7390 0x5624682cb1d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1c30105190 tx=0x7f1c301055e0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.071+0000 7f1c53da4740 1 -- 172.21.15.3:0/3893474858 shutdown_connections 2024-09-21T06:23:34.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.071+0000 7f1c53da4740 1 --2- 172.21.15.3:0/3893474858 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562467ed69f0 0x5624682dd050 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.071+0000 7f1c53da4740 1 --2- 172.21.15.3:0/3893474858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467f18070 0x5624682cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.071+0000 7f1c53da4740 1 --2- 172.21.15.3:0/3893474858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467ef7390 0x5624682cb1d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.071+0000 7f1c53da4740 1 --2- 172.21.15.3:0/3893474858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467eef1b0 0x5624682cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.071+0000 7f1c26ffd640 0 client.17958 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:23:34.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.071+0000 7f1c53da4740 1 -- 172.21.15.3:0/3893474858 >> 172.21.15.3:0/3893474858 conn(0x562467f0c460 msgr2=0x5624682c2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:34.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.071+0000 7f1c53da4740 1 -- 172.21.15.3:0/3893474858 shutdown_connections 2024-09-21T06:23:34.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.072+0000 7f1c53da4740 1 -- 172.21.15.3:0/3893474858 wait complete. 2024-09-21T06:23:34.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.072+0000 7f1c53da4740 20 client.17958 trim_cache size 0 max 16384 2024-09-21T06:23:34.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.085+0000 7f1c53da4740 1 Processor -- start 2024-09-21T06:23:34.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.085+0000 7f1c53da4740 1 -- start start 2024-09-21T06:23:34.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.085+0000 7f1c53da4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 0x5624682c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.085+0000 7f1c53da4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467ef7390 0x5624682c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.085+0000 7f1c53da4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467f18070 0x5624682c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.085+0000 7f1c53da4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481af760 con 0x562467eef1b0 2024-09-21T06:23:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.085+0000 7f1c53da4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b12a0 con 0x562467f18070 2024-09-21T06:23:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.085+0000 7f1c53da4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481afe30 con 0x562467ef7390 2024-09-21T06:23:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.085+0000 7f1c46ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 0x5624682c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.086+0000 7f1c46ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 0x5624682c0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49034/0 (socket says 172.21.15.3:49034) 2024-09-21T06:23:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.086+0000 7f1c467fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467ef7390 0x5624682c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.086+0000 7f1c46ffd640 1 -- 172.21.15.3:0/484671381 learned_addr learned my addr 172.21.15.3:0/484671381 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.086+0000 7f1c477fe640 1 --2- 172.21.15.3:0/484671381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467f18070 0x5624682c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.086+0000 7f1c46ffd640 1 -- 172.21.15.3:0/484671381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467ef7390 msgr2=0x5624682c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.086+0000 7f1c46ffd640 1 --2- 172.21.15.3:0/484671381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467ef7390 0x5624682c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.086+0000 7f1c46ffd640 1 -- 172.21.15.3:0/484671381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467f18070 msgr2=0x5624682c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.086+0000 7f1c46ffd640 1 --2- 172.21.15.3:0/484671381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467f18070 0x5624682c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.086+0000 7f1c46ffd640 1 -- 172.21.15.3:0/484671381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624682bd6e0 con 0x562467eef1b0 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.086+0000 7f1c46ffd640 1 --2- 172.21.15.3:0/484671381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 0x5624682c0de0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1c40014020 tx=0x7f1c40000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928d389020c66b7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.086+0000 7f1c3d7fa640 1 -- 172.21.15.3:0/484671381 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4001d050 con 0x562467eef1b0 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.086+0000 7f1c3d7fa640 1 -- 172.21.15.3:0/484671381 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c4001e040 con 0x562467eef1b0 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.086+0000 7f1c3d7fa640 1 -- 172.21.15.3:0/484671381 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40004160 con 0x562467eef1b0 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.087+0000 7f1c53da4740 1 -- 172.21.15.3:0/484671381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 msgr2=0x5624682c0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.087+0000 7f1c53da4740 1 --2- 172.21.15.3:0/484671381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 0x5624682c0de0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1c40014020 tx=0x7f1c40000f40 comp rx=0 tx=0).stop 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.087+0000 7f1c53da4740 1 -- 172.21.15.3:0/484671381 shutdown_connections 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.087+0000 7f1c53da4740 1 --2- 172.21.15.3:0/484671381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467ef7390 0x5624682c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.087+0000 7f1c53da4740 1 --2- 172.21.15.3:0/484671381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467f18070 0x5624682c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.087+0000 7f1c53da4740 1 --2- 172.21.15.3:0/484671381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 0x5624682c0de0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.087+0000 7f1c53da4740 1 -- 172.21.15.3:0/484671381 >> 172.21.15.3:0/484671381 conn(0x562467f0c460 msgr2=0x5624682c06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.087+0000 7f1c53da4740 1 -- 172.21.15.3:0/484671381 shutdown_connections 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.087+0000 7f1c53da4740 1 -- 172.21.15.3:0/484671381 wait complete. 2024-09-21T06:23:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c53da4740 1 -- start start 2024-09-21T06:23:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c53da4740 1 Processor -- start 2024-09-21T06:23:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c53da4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 0x5624682cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c53da4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467ef7390 0x5624682cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c53da4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467f18070 0x5624682cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c53da4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b0bd0 con 0x562467eef1b0 2024-09-21T06:23:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c53da4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b0500 con 0x562467ef7390 2024-09-21T06:23:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c53da4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481ad760 con 0x562467f18070 2024-09-21T06:23:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c53da4740 10 client.0 _ll_register_callbacks cb 0x7f1c30005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c46ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 0x5624682cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c46ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 0x5624682cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49038/0 (socket says 172.21.15.3:49038) 2024-09-21T06:23:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c46ffd640 1 -- 172.21.15.3:0/1047985038 learned_addr learned my addr 172.21.15.3:0/1047985038 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c467fc640 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467ef7390 0x5624682cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.088+0000 7f1c477fe640 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467f18070 0x5624682cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c467fc640 1 -- 172.21.15.3:0/1047985038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467f18070 msgr2=0x5624682cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c467fc640 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467f18070 0x5624682cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c467fc640 1 -- 172.21.15.3:0/1047985038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 msgr2=0x5624682cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c467fc640 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 0x5624682cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c467fc640 1 -- 172.21.15.3:0/1047985038 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562468346b90 con 0x562467ef7390 2024-09-21T06:23:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c46ffd640 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 0x5624682cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c467fc640 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467ef7390 0x5624682cf7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1c300c9eb0 tx=0x7f1c300d73b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cb63292088cb9e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:23:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c26ffd640 1 -- 172.21.15.3:0/1047985038 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30049070 con 0x562467ef7390 2024-09-21T06:23:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c53da4740 1 -- 172.21.15.3:0/1047985038 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c4001e040 con 0x562467ef7390 2024-09-21T06:23:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c26ffd640 1 -- 172.21.15.3:0/1047985038 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f1c300ccb40 con 0x562467ef7390 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c26ffd640 1 -- 172.21.15.3:0/1047985038 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1c3007b050 con 0x562467ef7390 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c53da4740 10 client.27524 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.089+0000 7f1c53da4740 1 -- 172.21.15.3:0/1047985038 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562468347e10 con 0x562467ef7390 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.090+0000 7f1c26ffd640 1 -- 172.21.15.3:0/1047985038 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c4001e040 con 0x562467ef7390 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.090+0000 7f1c26ffd640 1 -- 172.21.15.3:0/1047985038 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f1c300cc5b0 con 0x562467ef7390 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.090+0000 7f1c53da4740 10 client.27524 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.090+0000 7f1c53da4740 10 client.27524 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.090+0000 7f1c53da4740 20 client.27524 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.090+0000 7f1c53da4740 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562467eabda0 0x56246834a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.090+0000 7f1c53da4740 4 client.27524 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.090+0000 7f1c53da4740 1 -- 172.21.15.3:0/1047985038 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624682338a0 con 0x562467eabda0 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.090+0000 7f1c46ffd640 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562467eabda0 0x56246834a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c26ffd640 10 client.27524 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:23:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c46ffd640 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562467eabda0 0x56246834a440 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d60d8bb1ce69e382 server_cookie=7167971bca46beaa in_seq=0 out_seq=0 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c26ffd640 1 -- 172.21.15.3:0/1047985038 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624682338a0 con 0x562467eabda0 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c26ffd640 10 client.27524 handle_command_reply: tid=0 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c53da4740 1 client.27524 shutdown 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c53da4740 1 -- 172.21.15.3:0/1047985038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467ef7390 msgr2=0x5624682cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c53da4740 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467ef7390 0x5624682cf7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1c300c9eb0 tx=0x7f1c300d73b0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c53da4740 1 -- 172.21.15.3:0/1047985038 shutdown_connections 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c53da4740 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x562467eabda0 0x56246834a440 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c53da4740 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562467f18070 0x5624682cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c53da4740 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562467ef7390 0x5624682cf7a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c26ffd640 0 client.27524 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c53da4740 1 --2- 172.21.15.3:0/1047985038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562467eef1b0 0x5624682cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c53da4740 1 -- 172.21.15.3:0/1047985038 >> 172.21.15.3:0/1047985038 conn(0x562467f0c460 msgr2=0x5624682d6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c53da4740 1 -- 172.21.15.3:0/1047985038 shutdown_connections 2024-09-21T06:23:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c53da4740 1 -- 172.21.15.3:0/1047985038 wait complete. 2024-09-21T06:23:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.091+0000 7f1c53da4740 20 client.27524 trim_cache size 0 max 16384 2024-09-21T06:23:34.104 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:23:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.092+0000 7f1c4d830640 1 -- 172.21.15.3:0/1474946826 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1c340b6680 msgr2=0x7f1c340b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.092+0000 7f1c4d830640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1c340b6680 0x7f1c340b8b90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1c380124a0 tx=0x7f1c38005f90 comp rx=0 tx=0).stop 2024-09-21T06:23:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.092+0000 7f1c4d830640 1 -- 172.21.15.3:0/1474946826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c48107b30 msgr2=0x7f1c4819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.092+0000 7f1c4d830640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c48107b30 0x7f1c4819aac0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1c3002a930 tx=0x7f1c30031a20 comp rx=0 tx=0).stop 2024-09-21T06:23:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.092+0000 7f1c4d830640 1 -- 172.21.15.3:0/1474946826 shutdown_connections 2024-09-21T06:23:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.092+0000 7f1c4d830640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1c340b6680 0x7f1c340b8b90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.092+0000 7f1c4d830640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c48108480 0x7f1c4819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.092+0000 7f1c4d830640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c48107b30 0x7f1c4819aac0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.092+0000 7f1c4d830640 1 --2- 172.21.15.3:0/1474946826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c481021f0 0x7f1c4819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.092+0000 7f1c4d830640 1 -- 172.21.15.3:0/1474946826 >> 172.21.15.3:0/1474946826 conn(0x7f1c48077c60 msgr2=0x7f1c48105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.092+0000 7f1c4d830640 1 -- 172.21.15.3:0/1474946826 shutdown_connections 2024-09-21T06:23:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:34.092+0000 7f1c4d830640 1 -- 172.21.15.3:0/1474946826 wait complete. 2024-09-21T06:23:34.920 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:23:34.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 892316 10.54 MB/sec execute 307 sec latency 28.590 ms 2024-09-21T06:23:35.189 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:35.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:35 smithi003 ceph-mon[31227]: pgmap v588: 97 pgs: 97 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 234 op/s 2024-09-21T06:23:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.732+0000 7f3c8ab8a640 1 -- 172.21.15.3:0/2444161323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c84108cf0 msgr2=0x7f3c8410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.732+0000 7f3c8ab8a640 1 --2- 172.21.15.3:0/2444161323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c84108cf0 0x7f3c8410f520 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009900 tx=0x7f3c7c02a410 comp rx=0 tx=0).stop 2024-09-21T06:23:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.732+0000 7f3c8ab8a640 1 -- 172.21.15.3:0/2444161323 shutdown_connections 2024-09-21T06:23:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.732+0000 7f3c8ab8a640 1 --2- 172.21.15.3:0/2444161323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c841083a0 0x7f3c841087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.732+0000 7f3c8ab8a640 1 --2- 172.21.15.3:0/2444161323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c84108cf0 0x7f3c8410f520 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.732+0000 7f3c8ab8a640 1 --2- 172.21.15.3:0/2444161323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c84102a60 0x7f3c84102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.732+0000 7f3c8ab8a640 1 -- 172.21.15.3:0/2444161323 >> 172.21.15.3:0/2444161323 conn(0x7f3c84079730 msgr2=0x7f3c84079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.732+0000 7f3c8ab8a640 1 -- 172.21.15.3:0/2444161323 shutdown_connections 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.732+0000 7f3c8ab8a640 1 -- 172.21.15.3:0/2444161323 wait complete. 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.733+0000 7f3c8ab8a640 1 Processor -- start 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.733+0000 7f3c8ab8a640 1 -- start start 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c8ab8a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c84102a60 0x7f3c84199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c8ab8a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c841083a0 0x7f3c841998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c8ab8a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c84108cf0 0x7f3c8419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c8ab8a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84112960 con 0x7f3c841083a0 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c8ab8a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841127e0 con 0x7f3c84102a60 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c8ab8a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84112ae0 con 0x7f3c84108cf0 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c83fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c841083a0 0x7f3c841998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c83fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c841083a0 0x7f3c841998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59870/0 (socket says 172.21.15.3:59870) 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c83fff640 1 -- 172.21.15.3:0/3640495272 learned_addr learned my addr 172.21.15.3:0/3640495272 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c88926640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c84102a60 0x7f3c84199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c89127640 1 --2- 172.21.15.3:0/3640495272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c84108cf0 0x7f3c8419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c83fff640 1 -- 172.21.15.3:0/3640495272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c84108cf0 msgr2=0x7f3c8419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c83fff640 1 --2- 172.21.15.3:0/3640495272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c84108cf0 0x7f3c8419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c83fff640 1 -- 172.21.15.3:0/3640495272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c84102a60 msgr2=0x7f3c84199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c83fff640 1 --2- 172.21.15.3:0/3640495272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c84102a60 0x7f3c84199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c83fff640 1 -- 172.21.15.3:0/3640495272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8419f280 con 0x7f3c841083a0 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.734+0000 7f3c83fff640 1 --2- 172.21.15.3:0/3640495272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c841083a0 0x7f3c841998b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00d740 tx=0x7f3c6c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.735+0000 7f3c81ffb640 1 -- 172.21.15.3:0/3640495272 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c017070 con 0x7f3c841083a0 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.735+0000 7f3c81ffb640 1 -- 172.21.15.3:0/3640495272 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c6c00de50 con 0x7f3c841083a0 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.735+0000 7f3c8ab8a640 1 -- 172.21.15.3:0/3640495272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8419f570 con 0x7f3c841083a0 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.735+0000 7f3c81ffb640 1 -- 172.21.15.3:0/3640495272 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c004ce0 con 0x7f3c841083a0 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.735+0000 7f3c8ab8a640 1 -- 172.21.15.3:0/3640495272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c841a5cd0 con 0x7f3c841083a0 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c910fe740 1 Processor -- start 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c81ffb640 1 -- 172.21.15.3:0/3640495272 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f3c6c004e80 con 0x7f3c841083a0 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c910fe740 1 -- start start 2024-09-21T06:23:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c910fe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb5a070 0x55dacbf016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c910fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbf01be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c910fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf02120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c910fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dacbe77ea0 con 0x55dacbb39390 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c910fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dacbe672d0 con 0x55dacbb311b0 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c910fe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dacbe6b0e0 con 0x55dacbb5a070 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c83fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf02120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c89127640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb5a070 0x55dacbf016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c89127640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb5a070 0x55dacbf016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46666/0 (socket says 172.21.15.3:46666) 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c89127640 1 -- 172.21.15.3:0/2257387986 learned_addr learned my addr 172.21.15.3:0/2257387986 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c88926640 1 --2- 172.21.15.3:0/2257387986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbf01be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c89127640 1 -- 172.21.15.3:0/2257387986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 msgr2=0x55dacbf01be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c89127640 1 --2- 172.21.15.3:0/2257387986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbf01be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c89127640 1 -- 172.21.15.3:0/2257387986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 msgr2=0x55dacbf02120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c89127640 1 --2- 172.21.15.3:0/2257387986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf02120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.738+0000 7f3c89127640 1 -- 172.21.15.3:0/2257387986 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dacbbc5650 con 0x55dacbb5a070 2024-09-21T06:23:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.739+0000 7f3c81ffb640 1 --2- 172.21.15.3:0/3640495272 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3c700b6680 0x7f3c700b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.739+0000 7f3c88926640 1 --2- 172.21.15.3:0/3640495272 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3c700b6680 0x7f3c700b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.739+0000 7f3c89127640 1 --2- 172.21.15.3:0/2257387986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb5a070 0x55dacbf016a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009900 tx=0x7f3c7c032bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=419b75a58424fc3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.739+0000 7f3c81ffb640 1 -- 172.21.15.3:0/3640495272 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f3c6c0e7060 con 0x7f3c841083a0 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.739+0000 7f3c7affd640 1 -- 172.21.15.3:0/2257387986 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c040070 con 0x55dacbb5a070 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.739+0000 7f3c7affd640 1 -- 172.21.15.3:0/2257387986 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c7c004030 con 0x55dacbb5a070 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.739+0000 7f3c7affd640 1 -- 172.21.15.3:0/2257387986 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c035040 con 0x55dacbb5a070 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.740+0000 7f3c910fe740 1 -- 172.21.15.3:0/2257387986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb5a070 msgr2=0x55dacbf016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.740+0000 7f3c88926640 1 --2- 172.21.15.3:0/3640495272 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3c700b6680 0x7f3c700b8b90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55dacbf00320 tx=0x7f3c74009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.740+0000 7f3c910fe740 1 --2- 172.21.15.3:0/2257387986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb5a070 0x55dacbf016a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009900 tx=0x7f3c7c032bd0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.740+0000 7f3c910fe740 1 -- 172.21.15.3:0/2257387986 shutdown_connections 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.740+0000 7f3c910fe740 1 --2- 172.21.15.3:0/2257387986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf02120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.740+0000 7f3c910fe740 1 --2- 172.21.15.3:0/2257387986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbf01be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.740+0000 7f3c910fe740 1 --2- 172.21.15.3:0/2257387986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb5a070 0x55dacbf016a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.740+0000 7f3c910fe740 1 -- 172.21.15.3:0/2257387986 >> 172.21.15.3:0/2257387986 conn(0x55dacbb4e460 msgr2=0x55dacbeff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.740+0000 7f3c910fe740 1 -- 172.21.15.3:0/2257387986 shutdown_connections 2024-09-21T06:23:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.740+0000 7f3c910fe740 1 -- 172.21.15.3:0/2257387986 wait complete. 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.740+0000 7f3c910fe740 1 -- start start 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.740+0000 7f3c910fe740 1 Processor -- start 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.741+0000 7f3c910fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbf0efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.741+0000 7f3c910fe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb39390 0x55dacbf0f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.741+0000 7f3c910fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb5a070 0x55dacbf1b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.741+0000 7f3c910fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dacbcc8700 con 0x55dacbb5a070 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.741+0000 7f3c910fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dacbe02160 con 0x55dacbb311b0 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.741+0000 7f3c910fe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dacbf11f80 con 0x55dacbb39390 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.741+0000 7f3c910fe740 10 client.0 _ll_register_callbacks cb 0x55dacbe1b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.741+0000 7f3c88926640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbf0efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.741+0000 7f3c89127640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb5a070 0x55dacbf1b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.741+0000 7f3c88926640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbf0efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:47658/0 (socket says 172.21.15.3:47658) 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.741+0000 7f3c83fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb39390 0x55dacbf0f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.741+0000 7f3c89127640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb5a070 0x55dacbf1b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59878/0 (socket says 172.21.15.3:59878) 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.741+0000 7f3c89127640 1 -- 172.21.15.3:0/2558705088 learned_addr learned my addr 172.21.15.3:0/2558705088 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c89127640 1 -- 172.21.15.3:0/2558705088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb39390 msgr2=0x55dacbf0f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c89127640 1 --2- 172.21.15.3:0/2558705088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb39390 0x55dacbf0f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c89127640 1 -- 172.21.15.3:0/2558705088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 msgr2=0x55dacbf0efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c89127640 1 --2- 172.21.15.3:0/2558705088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbf0efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c89127640 1 -- 172.21.15.3:0/2558705088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dacbe2fc90 con 0x55dacbb5a070 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c83fff640 1 --2- 172.21.15.3:0/2558705088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb39390 0x55dacbf0f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c89127640 1 --2- 172.21.15.3:0/2558705088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb5a070 0x55dacbf1b2e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c006010 tx=0x7f3c7c005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ef13030dd75633 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c63fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c63fff640 1 -- 172.21.15.3:0/2558705088 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c04c020 con 0x55dacbb5a070 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c63fff640 1 -- 172.21.15.3:0/2558705088 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f3c7c0043e0 con 0x55dacbb5a070 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c910fe740 1 -- 172.21.15.3:0/2558705088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c7c004030 con 0x55dacbb5a070 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c63fff640 1 -- 172.21.15.3:0/2558705088 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3c7c057050 con 0x55dacbb5a070 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c910fe740 10 client.17976 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c910fe740 1 -- 172.21.15.3:0/2558705088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55dacbf1dd30 con 0x55dacbb5a070 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.742+0000 7f3c63fff640 1 -- 172.21.15.3:0/2558705088 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c7c004030 con 0x55dacbb5a070 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.743+0000 7f3c63fff640 1 -- 172.21.15.3:0/2558705088 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f3c7c003c40 con 0x55dacbb5a070 2024-09-21T06:23:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.743+0000 7f3c910fe740 10 client.17976 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.743+0000 7f3c910fe740 10 client.17976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.743+0000 7f3c910fe740 20 client.17976 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.743+0000 7f3c910fe740 1 --2- 172.21.15.3:0/2558705088 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dacbb189f0 0x55dacbf21090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.743+0000 7f3c910fe740 4 client.17976 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.743+0000 7f3c910fe740 1 -- 172.21.15.3:0/2558705088 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dacbe79df0 con 0x55dacbb189f0 2024-09-21T06:23:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.743+0000 7f3c88926640 1 --2- 172.21.15.3:0/2558705088 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dacbb189f0 0x55dacbf21090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.743+0000 7f3c63fff640 10 client.17976 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.743+0000 7f3c88926640 1 --2- 172.21.15.3:0/2558705088 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dacbb189f0 0x55dacbf21090 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8f52bca4c50ef1 server_cookie=895269c65df57f48 in_seq=0 out_seq=0 2024-09-21T06:23:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.745+0000 7f3c63fff640 1 -- 172.21.15.3:0/2558705088 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dacbe79df0 con 0x55dacbb189f0 2024-09-21T06:23:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.745+0000 7f3c63fff640 10 client.17976 handle_command_reply: tid=0 2024-09-21T06:23:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.745+0000 7f3c910fe740 1 client.17976 shutdown 2024-09-21T06:23:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.745+0000 7f3c910fe740 1 -- 172.21.15.3:0/2558705088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb5a070 msgr2=0x55dacbf1b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.745+0000 7f3c910fe740 1 --2- 172.21.15.3:0/2558705088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb5a070 0x55dacbf1b2e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c006010 tx=0x7f3c7c005150 comp rx=0 tx=0).stop 2024-09-21T06:23:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.745+0000 7f3c910fe740 1 -- 172.21.15.3:0/2558705088 shutdown_connections 2024-09-21T06:23:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.745+0000 7f3c910fe740 1 --2- 172.21.15.3:0/2558705088 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dacbb189f0 0x55dacbf21090 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.745+0000 7f3c910fe740 1 --2- 172.21.15.3:0/2558705088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb39390 0x55dacbf0f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.745+0000 7f3c910fe740 1 --2- 172.21.15.3:0/2558705088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb5a070 0x55dacbf1b2e0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.745+0000 7f3c910fe740 1 --2- 172.21.15.3:0/2558705088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbf0efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.745+0000 7f3c910fe740 1 -- 172.21.15.3:0/2558705088 >> 172.21.15.3:0/2558705088 conn(0x55dacbb4e460 msgr2=0x55dacbf03090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.746+0000 7f3c63fff640 0 client.17976 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.746+0000 7f3c910fe740 1 -- 172.21.15.3:0/2558705088 shutdown_connections 2024-09-21T06:23:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.746+0000 7f3c910fe740 1 -- 172.21.15.3:0/2558705088 wait complete. 2024-09-21T06:23:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.746+0000 7f3c910fe740 20 client.17976 trim_cache size 0 max 16384 2024-09-21T06:23:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.760+0000 7f3c910fe740 1 Processor -- start 2024-09-21T06:23:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.760+0000 7f3c910fe740 1 -- start start 2024-09-21T06:23:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.760+0000 7f3c910fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbeff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.760+0000 7f3c910fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf1b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.760+0000 7f3c910fe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb5a070 0x55dacbf0ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.760+0000 7f3c910fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841ad410 con 0x55dacbb39390 2024-09-21T06:23:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.760+0000 7f3c910fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841aef50 con 0x55dacbb311b0 2024-09-21T06:23:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.760+0000 7f3c910fe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841adae0 con 0x55dacbb5a070 2024-09-21T06:23:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.760+0000 7f3c83fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf1b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.760+0000 7f3c88926640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbeff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.760+0000 7f3c89127640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb5a070 0x55dacbf0ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.760+0000 7f3c83fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf1b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59888/0 (socket says 172.21.15.3:59888) 2024-09-21T06:23:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.760+0000 7f3c83fff640 1 -- 172.21.15.3:0/484862566 learned_addr learned my addr 172.21.15.3:0/484862566 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.761+0000 7f3c83fff640 1 -- 172.21.15.3:0/484862566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb5a070 msgr2=0x55dacbf0ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.761+0000 7f3c83fff640 1 --2- 172.21.15.3:0/484862566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb5a070 0x55dacbf0ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.761+0000 7f3c83fff640 1 -- 172.21.15.3:0/484862566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 msgr2=0x55dacbeff5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.761+0000 7f3c83fff640 1 --2- 172.21.15.3:0/484862566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbeff5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.761+0000 7f3c83fff640 1 -- 172.21.15.3:0/484862566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dacbf044c0 con 0x55dacbb39390 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.761+0000 7f3c88926640 1 --2- 172.21.15.3:0/484862566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbeff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.761+0000 7f3c89127640 1 --2- 172.21.15.3:0/484862566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb5a070 0x55dacbf0ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.761+0000 7f3c83fff640 1 --2- 172.21.15.3:0/484862566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf1b250 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0e4820 tx=0x7f3c6c0d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3d66636126dc9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.761+0000 7f3c7a7fc640 1 -- 172.21.15.3:0/484862566 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c020040 con 0x55dacbb39390 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.761+0000 7f3c7a7fc640 1 -- 172.21.15.3:0/484862566 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c6c0aca80 con 0x55dacbb39390 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.761+0000 7f3c7a7fc640 1 -- 172.21.15.3:0/484862566 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0a91e0 con 0x55dacbb39390 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.762+0000 7f3c910fe740 1 -- 172.21.15.3:0/484862566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 msgr2=0x55dacbf1b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.762+0000 7f3c910fe740 1 --2- 172.21.15.3:0/484862566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf1b250 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0e4820 tx=0x7f3c6c0d0040 comp rx=0 tx=0).stop 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.762+0000 7f3c910fe740 1 -- 172.21.15.3:0/484862566 shutdown_connections 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.762+0000 7f3c910fe740 1 --2- 172.21.15.3:0/484862566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb5a070 0x55dacbf0ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.762+0000 7f3c910fe740 1 --2- 172.21.15.3:0/484862566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf1b250 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.762+0000 7f3c910fe740 1 --2- 172.21.15.3:0/484862566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb311b0 0x55dacbeff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.762+0000 7f3c910fe740 1 -- 172.21.15.3:0/484862566 >> 172.21.15.3:0/484862566 conn(0x55dacbb4e460 msgr2=0x55dacbefeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.762+0000 7f3c910fe740 1 -- 172.21.15.3:0/484862566 shutdown_connections 2024-09-21T06:23:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.762+0000 7f3c910fe740 1 -- 172.21.15.3:0/484862566 wait complete. 2024-09-21T06:23:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.762+0000 7f3c910fe740 1 -- start start 2024-09-21T06:23:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c910fe740 1 Processor -- start 2024-09-21T06:23:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c910fe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb311b0 0x55dacbf249c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c910fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf867e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c910fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb5a070 0x55dacbf86e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c910fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841ae880 con 0x55dacbb39390 2024-09-21T06:23:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c910fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841ae1b0 con 0x55dacbb5a070 2024-09-21T06:23:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c910fe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841ab410 con 0x55dacbb311b0 2024-09-21T06:23:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c910fe740 10 client.0 _ll_register_callbacks cb 0x7f3c6c0eaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c83fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c88926640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb311b0 0x55dacbf249c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c89127640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb5a070 0x55dacbf86e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c88926640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb311b0 0x55dacbf249c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46680/0 (socket says 172.21.15.3:46680) 2024-09-21T06:23:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c88926640 1 -- 172.21.15.3:0/3537196005 learned_addr learned my addr 172.21.15.3:0/3537196005 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.763+0000 7f3c83fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf867e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59894/0 (socket says 172.21.15.3:59894) 2024-09-21T06:23:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c83fff640 1 -- 172.21.15.3:0/3537196005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb311b0 msgr2=0x55dacbf249c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c83fff640 1 --2- 172.21.15.3:0/3537196005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb311b0 0x55dacbf249c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c83fff640 1 -- 172.21.15.3:0/3537196005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb5a070 msgr2=0x55dacbf86e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c83fff640 1 --2- 172.21.15.3:0/3537196005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb5a070 0x55dacbf86e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c83fff640 1 -- 172.21.15.3:0/3537196005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dacbf03bc0 con 0x55dacbb39390 2024-09-21T06:23:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c88926640 1 --2- 172.21.15.3:0/3537196005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb311b0 0x55dacbf249c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c83fff640 1 --2- 172.21.15.3:0/3537196005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf867e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c005c60 tx=0x7f3c6c0e47b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51da8c057ed4a4a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c63fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c63fff640 1 -- 172.21.15.3:0/3537196005 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0ac6e0 con 0x55dacbb39390 2024-09-21T06:23:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c63fff640 1 -- 172.21.15.3:0/3537196005 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f3c6c0e92d0 con 0x55dacbb39390 2024-09-21T06:23:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c910fe740 1 -- 172.21.15.3:0/3537196005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c6c0aca80 con 0x55dacbb39390 2024-09-21T06:23:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c63fff640 1 -- 172.21.15.3:0/3537196005 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3c6c0b19a0 con 0x55dacbb39390 2024-09-21T06:23:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c910fe740 10 client.17982 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c910fe740 1 -- 172.21.15.3:0/3537196005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55dacbf8dc00 con 0x55dacbb39390 2024-09-21T06:23:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c63fff640 1 -- 172.21.15.3:0/3537196005 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c6c0aca80 con 0x55dacbb39390 2024-09-21T06:23:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.764+0000 7f3c63fff640 1 -- 172.21.15.3:0/3537196005 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f3c6c0d4170 con 0x55dacbb39390 2024-09-21T06:23:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.765+0000 7f3c910fe740 10 client.17982 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.765+0000 7f3c910fe740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.765+0000 7f3c910fe740 20 client.17982 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.765+0000 7f3c910fe740 1 --2- 172.21.15.3:0/3537196005 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dacbaedda0 0x55dacbf90130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.765+0000 7f3c910fe740 4 client.17982 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:23:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.765+0000 7f3c910fe740 1 -- 172.21.15.3:0/3537196005 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dacbe758a0 con 0x55dacbaedda0 2024-09-21T06:23:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.765+0000 7f3c88926640 1 --2- 172.21.15.3:0/3537196005 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dacbaedda0 0x55dacbf90130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.765+0000 7f3c88926640 1 --2- 172.21.15.3:0/3537196005 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dacbaedda0 0x55dacbf90130 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=437d197b8634dc06 server_cookie=4d1840d2b22fca06 in_seq=0 out_seq=0 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.765+0000 7f3c63fff640 10 client.17982 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.766+0000 7f3c63fff640 1 -- 172.21.15.3:0/3537196005 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dacbe758a0 con 0x55dacbaedda0 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.766+0000 7f3c63fff640 10 client.17982 handle_command_reply: tid=0 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.766+0000 7f3c910fe740 1 client.17982 shutdown 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.766+0000 7f3c910fe740 1 -- 172.21.15.3:0/3537196005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 msgr2=0x55dacbf867e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.766+0000 7f3c910fe740 1 --2- 172.21.15.3:0/3537196005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf867e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c005c60 tx=0x7f3c6c0e47b0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.766+0000 7f3c910fe740 1 -- 172.21.15.3:0/3537196005 shutdown_connections 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.766+0000 7f3c910fe740 1 --2- 172.21.15.3:0/3537196005 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55dacbaedda0 0x55dacbf90130 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.766+0000 7f3c63fff640 0 client.17982 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.766+0000 7f3c910fe740 1 --2- 172.21.15.3:0/3537196005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dacbb5a070 0x55dacbf86e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.766+0000 7f3c910fe740 1 --2- 172.21.15.3:0/3537196005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dacbb39390 0x55dacbf867e0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.766+0000 7f3c910fe740 1 --2- 172.21.15.3:0/3537196005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dacbb311b0 0x55dacbf249c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.766+0000 7f3c910fe740 1 -- 172.21.15.3:0/3537196005 >> 172.21.15.3:0/3537196005 conn(0x55dacbb4e460 msgr2=0x55dacbf0ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.766+0000 7f3c910fe740 1 -- 172.21.15.3:0/3537196005 shutdown_connections 2024-09-21T06:23:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.767+0000 7f3c910fe740 1 -- 172.21.15.3:0/3537196005 wait complete. 2024-09-21T06:23:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.767+0000 7f3c910fe740 20 client.17982 trim_cache size 0 max 16384 2024-09-21T06:23:35.773 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:23:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.767+0000 7f3c8ab8a640 1 -- 172.21.15.3:0/3640495272 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3c700b6680 msgr2=0x7f3c700b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.767+0000 7f3c8ab8a640 1 --2- 172.21.15.3:0/3640495272 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3c700b6680 0x7f3c700b8b90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55dacbf00320 tx=0x7f3c74009500 comp rx=0 tx=0).stop 2024-09-21T06:23:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.767+0000 7f3c8ab8a640 1 -- 172.21.15.3:0/3640495272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c841083a0 msgr2=0x7f3c841998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.767+0000 7f3c8ab8a640 1 --2- 172.21.15.3:0/3640495272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c841083a0 0x7f3c841998b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00d740 tx=0x7f3c6c00dc00 comp rx=0 tx=0).stop 2024-09-21T06:23:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.767+0000 7f3c8ab8a640 1 -- 172.21.15.3:0/3640495272 shutdown_connections 2024-09-21T06:23:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.768+0000 7f3c8ab8a640 1 --2- 172.21.15.3:0/3640495272 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f3c700b6680 0x7f3c700b8b90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.768+0000 7f3c8ab8a640 1 --2- 172.21.15.3:0/3640495272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c84108cf0 0x7f3c8419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.768+0000 7f3c8ab8a640 1 --2- 172.21.15.3:0/3640495272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c841083a0 0x7f3c841998b0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.768+0000 7f3c8ab8a640 1 --2- 172.21.15.3:0/3640495272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c84102a60 0x7f3c84199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.768+0000 7f3c8ab8a640 1 -- 172.21.15.3:0/3640495272 >> 172.21.15.3:0/3640495272 conn(0x7f3c84079730 msgr2=0x7f3c841043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.768+0000 7f3c8ab8a640 1 -- 172.21.15.3:0/3640495272 shutdown_connections 2024-09-21T06:23:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:35.768+0000 7f3c8ab8a640 1 -- 172.21.15.3:0/3640495272 wait complete. 2024-09-21T06:23:35.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:35 smithi137 ceph-mon[34464]: pgmap v588: 97 pgs: 97 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 234 op/s 2024-09-21T06:23:35.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:35 smithi204 ceph-mon[34659]: pgmap v588: 97 pgs: 97 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 234 op/s 2024-09-21T06:23:35.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 894651 10.56 MB/sec execute 308 sec latency 27.596 ms 2024-09-21T06:23:36.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 895396 10.53 MB/sec execute 309 sec latency 17.275 ms 2024-09-21T06:23:37.544 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:23:37.544 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:23:37.821 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:37.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:37 smithi003 ceph-mon[31227]: pgmap v589: 97 pgs: 97 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 287 op/s 2024-09-21T06:23:37.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:37 smithi137 ceph-mon[34464]: pgmap v589: 97 pgs: 97 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 287 op/s 2024-09-21T06:23:37.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:37 smithi204 ceph-mon[34659]: pgmap v589: 97 pgs: 97 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 287 op/s 2024-09-21T06:23:37.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 898019 10.54 MB/sec execute 310 sec latency 53.449 ms 2024-09-21T06:23:38.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.333+0000 7fa3ce74b640 1 -- 172.21.15.3:0/2904778212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c8108cf0 msgr2=0x7fa3c810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:38.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.333+0000 7fa3ce74b640 1 --2- 172.21.15.3:0/2904778212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c8108cf0 0x7fa3c810f520 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009ca0 tx=0x7fa3c002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.334+0000 7fa3ce74b640 1 -- 172.21.15.3:0/2904778212 shutdown_connections 2024-09-21T06:23:38.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.334+0000 7fa3ce74b640 1 --2- 172.21.15.3:0/2904778212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c81083a0 0x7fa3c81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.334+0000 7fa3ce74b640 1 --2- 172.21.15.3:0/2904778212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c8108cf0 0x7fa3c810f520 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.334+0000 7fa3ce74b640 1 --2- 172.21.15.3:0/2904778212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c8102a60 0x7fa3c8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.334+0000 7fa3ce74b640 1 -- 172.21.15.3:0/2904778212 >> 172.21.15.3:0/2904778212 conn(0x7fa3c8079730 msgr2=0x7fa3c8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:38.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.334+0000 7fa3ce74b640 1 -- 172.21.15.3:0/2904778212 shutdown_connections 2024-09-21T06:23:38.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.334+0000 7fa3ce74b640 1 -- 172.21.15.3:0/2904778212 wait complete. 2024-09-21T06:23:38.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.334+0000 7fa3ce74b640 1 Processor -- start 2024-09-21T06:23:38.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.334+0000 7fa3ce74b640 1 -- start start 2024-09-21T06:23:38.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3ce74b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c8102a60 0x7fa3c819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3ce74b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c81083a0 0x7fa3c819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3ce74b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c8108cf0 0x7fa3c819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3ce74b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8112960 con 0x7fa3c8108cf0 2024-09-21T06:23:38.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3ce74b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81127e0 con 0x7fa3c81083a0 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3ce74b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8112ae0 con 0x7fa3c8102a60 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c8102a60 0x7fa3c819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3ccce8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c8108cf0 0x7fa3c819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c8102a60 0x7fa3c819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46710/0 (socket says 172.21.15.3:46710) 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3c77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c81083a0 0x7fa3c819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3c7fff640 1 -- 172.21.15.3:0/226982570 learned_addr learned my addr 172.21.15.3:0/226982570 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3ccce8640 1 -- 172.21.15.3:0/226982570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c8102a60 msgr2=0x7fa3c819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3ccce8640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c8102a60 0x7fa3c819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3ccce8640 1 -- 172.21.15.3:0/226982570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c81083a0 msgr2=0x7fa3c819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.335+0000 7fa3ccce8640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c81083a0 0x7fa3c819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.336+0000 7fa3ccce8640 1 -- 172.21.15.3:0/226982570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c81a0410 con 0x7fa3c8108cf0 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.336+0000 7fa3c7fff640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c8102a60 0x7fa3c819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.336+0000 7fa3c77fe640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c81083a0 0x7fa3c819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:38.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.336+0000 7fa3ccce8640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c8108cf0 0x7fa3c819fce0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa3c002ade0 tx=0x7fa3c0031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.336+0000 7fa3c57fa640 1 -- 172.21.15.3:0/226982570 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0003b70 con 0x7fa3c8108cf0 2024-09-21T06:23:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.336+0000 7fa3ce74b640 1 -- 172.21.15.3:0/226982570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c81a06a0 con 0x7fa3c8108cf0 2024-09-21T06:23:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.336+0000 7fa3c57fa640 1 -- 172.21.15.3:0/226982570 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c0031910 con 0x7fa3c8108cf0 2024-09-21T06:23:38.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.336+0000 7fa3c57fa640 1 -- 172.21.15.3:0/226982570 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c002f950 con 0x7fa3c8108cf0 2024-09-21T06:23:38.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.336+0000 7fa3ce74b640 1 -- 172.21.15.3:0/226982570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c81a6e60 con 0x7fa3c8108cf0 2024-09-21T06:23:38.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3d4cbf740 1 Processor -- start 2024-09-21T06:23:38.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3c57fa640 1 -- 172.21.15.3:0/226982570 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fa3c002faf0 con 0x7fa3c8108cf0 2024-09-21T06:23:38.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3d4cbf740 1 -- start start 2024-09-21T06:23:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3d4cbf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea722350 0x5635eaac9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3d4cbf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaac9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3d4cbf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaac9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3d4cbf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5635eaa3fe60 con 0x5635ea6f9490 2024-09-21T06:23:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3d4cbf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5635eaa2f290 con 0x5635ea722350 2024-09-21T06:23:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3d4cbf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5635eaa330a0 con 0x5635ea701670 2024-09-21T06:23:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaac9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaac9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59928/0 (socket says 172.21.15.3:59928) 2024-09-21T06:23:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3c77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea722350 0x5635eaac9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3c7fff640 1 -- 172.21.15.3:0/426037258 learned_addr learned my addr 172.21.15.3:0/426037258 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.339+0000 7fa3c77fe640 1 --2- 172.21.15.3:0/426037258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaac9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.340+0000 7fa3c7fff640 1 -- 172.21.15.3:0/426037258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 msgr2=0x5635eaac9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.340+0000 7fa3c7fff640 1 --2- 172.21.15.3:0/426037258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaac9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.340+0000 7fa3c7fff640 1 -- 172.21.15.3:0/426037258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea722350 msgr2=0x5635eaac9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.340+0000 7fa3c7fff640 1 --2- 172.21.15.3:0/426037258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea722350 0x5635eaac9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.340+0000 7fa3c7fff640 1 -- 172.21.15.3:0/426037258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ea78d930 con 0x5635ea6f9490 2024-09-21T06:23:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.340+0000 7fa3c77fe640 1 --2- 172.21.15.3:0/426037258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaac9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.340+0000 7fa3c57fa640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa3b00b6680 0x7fa3b00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.340+0000 7fa3c7fff640 1 --2- 172.21.15.3:0/426037258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaac9970 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8011650 tx=0x7fa3b8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b5a5f238708eca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.340+0000 7fa3c77fe640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa3b00b6680 0x7fa3b00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.340+0000 7fa3ae7fc640 1 -- 172.21.15.3:0/426037258 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b801e070 con 0x5635ea6f9490 2024-09-21T06:23:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.340+0000 7fa3c57fa640 1 -- 172.21.15.3:0/226982570 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7fa3c01087b0 con 0x7fa3c8108cf0 2024-09-21T06:23:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.340+0000 7fa3ae7fc640 1 -- 172.21.15.3:0/426037258 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b800fb70 con 0x5635ea6f9490 2024-09-21T06:23:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.340+0000 7fa3ae7fc640 1 -- 172.21.15.3:0/426037258 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8019c30 con 0x5635ea6f9490 2024-09-21T06:23:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.341+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/426037258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 msgr2=0x5635eaac9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.341+0000 7fa3c77fe640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa3b00b6680 0x7fa3b00b8b90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009fd0 tx=0x7fa3bc0096a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.341+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/426037258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaac9970 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8011650 tx=0x7fa3b8009f90 comp rx=0 tx=0).stop 2024-09-21T06:23:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.341+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/426037258 shutdown_connections 2024-09-21T06:23:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.341+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/426037258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaac9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.341+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/426037258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaac9970 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.341+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/426037258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea722350 0x5635eaac9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.341+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/426037258 >> 172.21.15.3:0/426037258 conn(0x5635ea716740 msgr2=0x5635eaac7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.341+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/426037258 shutdown_connections 2024-09-21T06:23:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.341+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/426037258 wait complete. 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.342+0000 7fa3d4cbf740 1 -- start start 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.342+0000 7fa3d4cbf740 1 Processor -- start 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.342+0000 7fa3d4cbf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaad5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.342+0000 7fa3d4cbf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaad5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.342+0000 7fa3d4cbf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea722350 0x5635eaae5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.342+0000 7fa3d4cbf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5635ea890890 con 0x5635ea6f9490 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.342+0000 7fa3d4cbf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5635ea9c9d40 con 0x5635ea722350 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.342+0000 7fa3d4cbf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5635eaadc020 con 0x5635ea701670 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.342+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaad5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaad5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59930/0 (socket says 172.21.15.3:59930) 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3c7fff640 1 -- 172.21.15.3:0/2810929923 learned_addr learned my addr 172.21.15.3:0/2810929923 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3c77fe640 1 --2- 172.21.15.3:0/2810929923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaad5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3d4cbf740 10 client.0 _ll_register_callbacks cb 0x5635eaa2dc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3ccce8640 1 --2- 172.21.15.3:0/2810929923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea722350 0x5635eaae5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3c7fff640 1 -- 172.21.15.3:0/2810929923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 msgr2=0x5635eaad5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3c7fff640 1 --2- 172.21.15.3:0/2810929923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaad5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3c7fff640 1 -- 172.21.15.3:0/2810929923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea722350 msgr2=0x5635eaae5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3c7fff640 1 --2- 172.21.15.3:0/2810929923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea722350 0x5635eaae5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3c7fff640 1 -- 172.21.15.3:0/2810929923 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635eaae58d0 con 0x5635ea6f9490 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3c7fff640 1 --2- 172.21.15.3:0/2810929923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaad5030 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8019770 tx=0x7fa3b800ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ff9eee4a386512 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3a77fe640 1 -- 172.21.15.3:0/2810929923 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b801e040 con 0x5635ea6f9490 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/2810929923 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b800fb70 con 0x5635ea6f9490 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3a77fe640 1 -- 172.21.15.3:0/2810929923 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7fa3b8009cc0 con 0x5635ea6f9490 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.343+0000 7fa3a77fe640 1 -- 172.21.15.3:0/2810929923 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa3b8030050 con 0x5635ea6f9490 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.344+0000 7fa3d4cbf740 10 client.17994 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.344+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/2810929923 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5635eaae81e0 con 0x5635ea6f9490 2024-09-21T06:23:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.344+0000 7fa3a77fe640 1 -- 172.21.15.3:0/2810929923 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b800fb70 con 0x5635ea6f9490 2024-09-21T06:23:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.344+0000 7fa3a77fe640 1 -- 172.21.15.3:0/2810929923 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fa3b8021070 con 0x5635ea6f9490 2024-09-21T06:23:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.344+0000 7fa3d4cbf740 10 client.17994 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.344+0000 7fa3d4cbf740 10 client.17994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.344+0000 7fa3d4cbf740 20 client.17994 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.344+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/2810929923 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5635ea6e0cd0 0x5635eaaeb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.344+0000 7fa3d4cbf740 4 client.17994 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.344+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/2810929923 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635eaa41db0 con 0x5635ea6e0cd0 2024-09-21T06:23:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.344+0000 7fa3c77fe640 1 --2- 172.21.15.3:0/2810929923 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5635ea6e0cd0 0x5635eaaeb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.345+0000 7fa3c77fe640 1 --2- 172.21.15.3:0/2810929923 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5635ea6e0cd0 0x5635eaaeb540 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34a1a4156db12800 server_cookie=67460b853226159f in_seq=0 out_seq=0 2024-09-21T06:23:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.345+0000 7fa3a77fe640 10 client.17994 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.345+0000 7fa3a77fe640 1 -- 172.21.15.3:0/2810929923 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5635eaa41db0 con 0x5635ea6e0cd0 2024-09-21T06:23:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.345+0000 7fa3a77fe640 10 client.17994 handle_command_reply: tid=0 2024-09-21T06:23:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.346+0000 7fa3d4cbf740 1 client.17994 shutdown 2024-09-21T06:23:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.346+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/2810929923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 msgr2=0x5635eaad5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.346+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/2810929923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaad5030 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8019770 tx=0x7fa3b800ff50 comp rx=0 tx=0).stop 2024-09-21T06:23:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.346+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/2810929923 shutdown_connections 2024-09-21T06:23:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.346+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/2810929923 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5635ea6e0cd0 0x5635eaaeb540 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.346+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/2810929923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaad5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.346+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/2810929923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea722350 0x5635eaae5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.346+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/2810929923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaad5030 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.346+0000 7fa3a77fe640 0 client.17994 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.346+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/2810929923 >> 172.21.15.3:0/2810929923 conn(0x5635ea716740 msgr2=0x5635eaacc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.346+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/2810929923 shutdown_connections 2024-09-21T06:23:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.346+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/2810929923 wait complete. 2024-09-21T06:23:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.346+0000 7fa3d4cbf740 20 client.17994 trim_cache size 0 max 16384 2024-09-21T06:23:38.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.364+0000 7fa3d4cbf740 1 Processor -- start 2024-09-21T06:23:38.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.364+0000 7fa3d4cbf740 1 -- start start 2024-09-21T06:23:38.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.364+0000 7fa3d4cbf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaad4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.364+0000 7fa3d4cbf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea701670 0x5635eaacab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.364+0000 7fa3d4cbf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea722350 0x5635eaace510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.364+0000 7fa3d4cbf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81ad520 con 0x5635ea6f9490 2024-09-21T06:23:38.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.364+0000 7fa3d4cbf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81af060 con 0x5635ea701670 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.364+0000 7fa3d4cbf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81adbf0 con 0x5635ea722350 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaad4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaad4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59942/0 (socket says 172.21.15.3:59942) 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3c7fff640 1 -- 172.21.15.3:0/1770420736 learned_addr learned my addr 172.21.15.3:0/1770420736 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3c77fe640 1 --2- 172.21.15.3:0/1770420736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea701670 0x5635eaacab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3ccce8640 1 --2- 172.21.15.3:0/1770420736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea722350 0x5635eaace510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3c7fff640 1 -- 172.21.15.3:0/1770420736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea722350 msgr2=0x5635eaace510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3c7fff640 1 --2- 172.21.15.3:0/1770420736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea722350 0x5635eaace510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3c7fff640 1 -- 172.21.15.3:0/1770420736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea701670 msgr2=0x5635eaacab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3c7fff640 1 --2- 172.21.15.3:0/1770420736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea701670 0x5635eaacab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3c7fff640 1 -- 172.21.15.3:0/1770420736 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ea6e0f00 con 0x5635ea6f9490 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3c7fff640 1 --2- 172.21.15.3:0/1770420736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaad4590 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80057c0 tx=0x7fa3b8011550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7a46a14b80956 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3adffb640 1 -- 172.21.15.3:0/1770420736 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b801e030 con 0x5635ea6f9490 2024-09-21T06:23:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3adffb640 1 -- 172.21.15.3:0/1770420736 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b8027070 con 0x5635ea6f9490 2024-09-21T06:23:38.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.365+0000 7fa3adffb640 1 -- 172.21.15.3:0/1770420736 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80197d0 con 0x5635ea6f9490 2024-09-21T06:23:38.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.366+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/1770420736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 msgr2=0x5635eaad4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:38.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.366+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/1770420736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaad4590 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80057c0 tx=0x7fa3b8011550 comp rx=0 tx=0).stop 2024-09-21T06:23:38.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.366+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/1770420736 shutdown_connections 2024-09-21T06:23:38.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.366+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/1770420736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea722350 0x5635eaace510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.366+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/1770420736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea701670 0x5635eaacab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.366+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/1770420736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea6f9490 0x5635eaad4590 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.366+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/1770420736 >> 172.21.15.3:0/1770420736 conn(0x5635ea716740 msgr2=0x5635eaaca460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:38.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.366+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/1770420736 shutdown_connections 2024-09-21T06:23:38.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.366+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/1770420736 wait complete. 2024-09-21T06:23:38.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.367+0000 7fa3d4cbf740 1 -- start start 2024-09-21T06:23:38.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.367+0000 7fa3d4cbf740 1 Processor -- start 2024-09-21T06:23:38.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.367+0000 7fa3d4cbf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea6f9490 0x5635eaac7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.367+0000 7fa3d4cbf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaae1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.367+0000 7fa3d4cbf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea722350 0x5635eaae17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.367+0000 7fa3d4cbf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81ae990 con 0x5635ea722350 2024-09-21T06:23:38.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.367+0000 7fa3d4cbf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81ae2c0 con 0x5635ea6f9490 2024-09-21T06:23:38.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.367+0000 7fa3d4cbf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81ab520 con 0x5635ea701670 2024-09-21T06:23:38.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.367+0000 7fa3d4cbf740 10 client.0 _ll_register_callbacks cb 0x7fa3b801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.367+0000 7fa3ccce8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea722350 0x5635eaae17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.367+0000 7fa3c77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaae1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.368+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea6f9490 0x5635eaac7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.368+0000 7fa3ccce8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea722350 0x5635eaae17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59956/0 (socket says 172.21.15.3:59956) 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.368+0000 7fa3ccce8640 1 -- 172.21.15.3:0/766244952 learned_addr learned my addr 172.21.15.3:0/766244952 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.368+0000 7fa3c77fe640 1 -- 172.21.15.3:0/766244952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea6f9490 msgr2=0x5635eaac7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.368+0000 7fa3c77fe640 1 --2- 172.21.15.3:0/766244952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea6f9490 0x5635eaac7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.368+0000 7fa3c77fe640 1 -- 172.21.15.3:0/766244952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea722350 msgr2=0x5635eaae17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.368+0000 7fa3c77fe640 1 --2- 172.21.15.3:0/766244952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea722350 0x5635eaae17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.368+0000 7fa3c77fe640 1 -- 172.21.15.3:0/766244952 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635eaae1ee0 con 0x5635ea701670 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.368+0000 7fa3c77fe640 1 --2- 172.21.15.3:0/766244952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaae1260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5635eaaebbb0 tx=0x7fa3bc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a78a2bea226c28af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.368+0000 7fa3a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.368+0000 7fa3a77fe640 1 -- 172.21.15.3:0/766244952 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc01e720 con 0x5635ea701670 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.368+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/766244952 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b8027070 con 0x5635ea701670 2024-09-21T06:23:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.368+0000 7fa3a77fe640 1 -- 172.21.15.3:0/766244952 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7fa3bc01a030 con 0x5635ea701670 2024-09-21T06:23:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.369+0000 7fa3a77fe640 1 -- 172.21.15.3:0/766244952 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa3bc03d050 con 0x5635ea701670 2024-09-21T06:23:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.369+0000 7fa3d4cbf740 10 client.27508 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.369+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/766244952 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5635eab5a600 con 0x5635ea701670 2024-09-21T06:23:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.369+0000 7fa3a77fe640 1 -- 172.21.15.3:0/766244952 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b8027070 con 0x5635ea701670 2024-09-21T06:23:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.369+0000 7fa3a77fe640 1 -- 172.21.15.3:0/766244952 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fa3bc010970 con 0x5635ea701670 2024-09-21T06:23:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.369+0000 7fa3d4cbf740 10 client.27508 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.369+0000 7fa3d4cbf740 10 client.27508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.369+0000 7fa3d4cbf740 20 client.27508 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.369+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/766244952 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5635ea6b6080 0x5635eab5cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.369+0000 7fa3d4cbf740 4 client.27508 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:23:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.369+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/766244952 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5635eaa3d860 con 0x5635ea6b6080 2024-09-21T06:23:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.369+0000 7fa3c7fff640 1 --2- 172.21.15.3:0/766244952 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5635ea6b6080 0x5635eab5cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:38.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.370+0000 7fa3a77fe640 10 client.27508 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.370+0000 7fa3c7fff640 1 --2- 172.21.15.3:0/766244952 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5635ea6b6080 0x5635eab5cd90 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60e31f6dbabdc9c7 server_cookie=36040a19ae56f439 in_seq=0 out_seq=0 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3a77fe640 1 -- 172.21.15.3:0/766244952 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5635eaa3d860 con 0x5635ea6b6080 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3a77fe640 10 client.27508 handle_command_reply: tid=0 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3d4cbf740 1 client.27508 shutdown 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/766244952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 msgr2=0x5635eaae1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/766244952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaae1260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5635eaaebbb0 tx=0x7fa3bc004730 comp rx=0 tx=0).stop 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/766244952 shutdown_connections 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/766244952 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5635ea6b6080 0x5635eab5cd90 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/766244952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635ea701670 0x5635eaae1260 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/766244952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5635ea722350 0x5635eaae17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3d4cbf740 1 --2- 172.21.15.3:0/766244952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ea6f9490 0x5635eaac7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/766244952 >> 172.21.15.3:0/766244952 conn(0x5635ea716740 msgr2=0x5635eaacabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3a77fe640 0 client.27508 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/766244952 shutdown_connections 2024-09-21T06:23:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.373+0000 7fa3d4cbf740 1 -- 172.21.15.3:0/766244952 wait complete. 2024-09-21T06:23:38.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.374+0000 7fa3d4cbf740 20 client.27508 trim_cache size 0 max 16384 2024-09-21T06:23:38.377 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:23:38.377 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:23:38.377 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "cbb5d4a7-b96d-4c2c-b7ab-7a3716c17f5b", 2024-09-21T06:23:38.377 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:23:38.377 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:23:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.374+0000 7fa3ce74b640 1 -- 172.21.15.3:0/226982570 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa3b00b6680 msgr2=0x7fa3b00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.374+0000 7fa3ce74b640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa3b00b6680 0x7fa3b00b8b90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009fd0 tx=0x7fa3bc0096a0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.374+0000 7fa3ce74b640 1 -- 172.21.15.3:0/226982570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c8108cf0 msgr2=0x7fa3c819fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.374+0000 7fa3ce74b640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c8108cf0 0x7fa3c819fce0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa3c002ade0 tx=0x7fa3c0031660 comp rx=0 tx=0).stop 2024-09-21T06:23:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.374+0000 7fa3ce74b640 1 -- 172.21.15.3:0/226982570 shutdown_connections 2024-09-21T06:23:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.374+0000 7fa3ce74b640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fa3b00b6680 0x7fa3b00b8b90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.374+0000 7fa3ce74b640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c8108cf0 0x7fa3c819fce0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.374+0000 7fa3ce74b640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c81083a0 0x7fa3c819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.374+0000 7fa3ce74b640 1 --2- 172.21.15.3:0/226982570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c8102a60 0x7fa3c819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.374+0000 7fa3ce74b640 1 -- 172.21.15.3:0/226982570 >> 172.21.15.3:0/226982570 conn(0x7fa3c8079730 msgr2=0x7fa3c806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.374+0000 7fa3ce74b640 1 -- 172.21.15.3:0/226982570 shutdown_connections 2024-09-21T06:23:38.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:38.374+0000 7fa3ce74b640 1 -- 172.21.15.3:0/226982570 wait complete. 2024-09-21T06:23:38.969 INFO:tasks.workunit.client.0.smithi003.stdout: 1 898653 10.52 MB/sec execute 311 sec latency 96.789 ms 2024-09-21T06:23:39.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:23:39.359 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:39.772 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:39 smithi003 ceph-mon[31227]: pgmap v590: 97 pgs: 97 active+clean; 132 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 313 op/s 2024-09-21T06:23:39.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:39 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:23:39.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:39 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:39.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:39 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:23:39.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:39 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:23:39.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:39 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:39.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:39 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:23:39.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:39 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:39.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:39 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:39.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.869+0000 7f1784694640 1 -- 172.21.15.3:0/2833660401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f177c109e10 msgr2=0x7f177c110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.869+0000 7f1784694640 1 --2- 172.21.15.3:0/2833660401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f177c109e10 0x7f177c110640 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1774009ca0 tx=0x7f177402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.869+0000 7f1784694640 1 -- 172.21.15.3:0/2833660401 shutdown_connections 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.869+0000 7f1784694640 1 --2- 172.21.15.3:0/2833660401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f177c109e10 0x7f177c110640 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.869+0000 7f1784694640 1 --2- 172.21.15.3:0/2833660401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f177c1094c0 0x7f177c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.869+0000 7f1784694640 1 --2- 172.21.15.3:0/2833660401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f177c101fe0 0x7f177c1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.869+0000 7f1784694640 1 -- 172.21.15.3:0/2833660401 >> 172.21.15.3:0/2833660401 conn(0x7f177c077bc0 msgr2=0x7f177c078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.869+0000 7f1784694640 1 -- 172.21.15.3:0/2833660401 shutdown_connections 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.869+0000 7f1784694640 1 -- 172.21.15.3:0/2833660401 wait complete. 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.870+0000 7f1784694640 1 Processor -- start 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.870+0000 7f1784694640 1 -- start start 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.870+0000 7f1784694640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f177c101fe0 0x7f177c19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.870+0000 7f1784694640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f177c1094c0 0x7f177c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.870+0000 7f1784694640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f177c109e10 0x7f177c1a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.870+0000 7f1784694640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f177c113a80 con 0x7f177c1094c0 2024-09-21T06:23:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.870+0000 7f1784694640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f177c113900 con 0x7f177c109e10 2024-09-21T06:23:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.870+0000 7f1784694640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f177c113c00 con 0x7f177c101fe0 2024-09-21T06:23:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.870+0000 7f1781c2f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f177c1094c0 0x7f177c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.870+0000 7f1782430640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f177c101fe0 0x7f177c19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.870+0000 7f1781c2f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f177c1094c0 0x7f177c19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59976/0 (socket says 172.21.15.3:59976) 2024-09-21T06:23:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.870+0000 7f1781c2f640 1 -- 172.21.15.3:0/4246119591 learned_addr learned my addr 172.21.15.3:0/4246119591 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.870+0000 7f1782c31640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f177c109e10 0x7f177c1a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.871+0000 7f1782430640 1 -- 172.21.15.3:0/4246119591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f177c109e10 msgr2=0x7f177c1a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.871+0000 7f1782430640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f177c109e10 0x7f177c1a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.871+0000 7f1782430640 1 -- 172.21.15.3:0/4246119591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f177c1094c0 msgr2=0x7f177c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.871+0000 7f1782430640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f177c1094c0 0x7f177c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.871+0000 7f1782430640 1 -- 172.21.15.3:0/4246119591 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c1a2620 con 0x7f177c101fe0 2024-09-21T06:23:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.871+0000 7f1781c2f640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f177c1094c0 0x7f177c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.871+0000 7f1782430640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f177c101fe0 0x7f177c19c6b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f177800e9f0 tx=0x7f177800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:39.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.871+0000 7f17737fe640 1 -- 172.21.15.3:0/4246119591 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177800ccc0 con 0x7f177c101fe0 2024-09-21T06:23:39.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.871+0000 7f1784694640 1 -- 172.21.15.3:0/4246119591 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c1a2910 con 0x7f177c101fe0 2024-09-21T06:23:39.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.871+0000 7f17737fe640 1 -- 172.21.15.3:0/4246119591 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17780108e0 con 0x7f177c101fe0 2024-09-21T06:23:39.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.871+0000 7f17737fe640 1 -- 172.21.15.3:0/4246119591 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778018880 con 0x7f177c101fe0 2024-09-21T06:23:39.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.871+0000 7f1784694640 1 -- 172.21.15.3:0/4246119591 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c1a9070 con 0x7f177c101fe0 2024-09-21T06:23:39.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f178ac08740 1 Processor -- start 2024-09-21T06:23:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f178ac08740 1 -- start start 2024-09-21T06:23:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f178ac08740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22d0070 0x55b6b2677650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f178ac08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22a71b0 0x55b6b2677b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f178ac08740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 0x55b6b26780d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f178ac08740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b6b25ede50 con 0x55b6b22af390 2024-09-21T06:23:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f178ac08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b6b25dd280 con 0x55b6b22a71b0 2024-09-21T06:23:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f178ac08740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b6b25e1090 con 0x55b6b22d0070 2024-09-21T06:23:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f17737fe640 1 -- 172.21.15.3:0/4246119591 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f1778010430 con 0x7f177c101fe0 2024-09-21T06:23:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f1781c2f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 0x55b6b26780d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f1782430640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22a71b0 0x55b6b2677b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f1782c31640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22d0070 0x55b6b2677650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f1781c2f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 0x55b6b26780d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59986/0 (socket says 172.21.15.3:59986) 2024-09-21T06:23:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.874+0000 7f1781c2f640 1 -- 172.21.15.3:0/1973289366 learned_addr learned my addr 172.21.15.3:0/1973289366 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.875+0000 7f1781c2f640 1 -- 172.21.15.3:0/1973289366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22d0070 msgr2=0x55b6b2677650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.875+0000 7f1781c2f640 1 --2- 172.21.15.3:0/1973289366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22d0070 0x55b6b2677650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.875+0000 7f1781c2f640 1 -- 172.21.15.3:0/1973289366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22a71b0 msgr2=0x55b6b2677b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.875+0000 7f1781c2f640 1 --2- 172.21.15.3:0/1973289366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22a71b0 0x55b6b2677b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.875+0000 7f1781c2f640 1 -- 172.21.15.3:0/1973289366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b233b650 con 0x55b6b22af390 2024-09-21T06:23:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.875+0000 7f1781c2f640 1 --2- 172.21.15.3:0/1973289366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 0x55b6b26780d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f176c0058c0 tx=0x7f176c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea9c50e5ced28a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.875+0000 7f1770ff9640 1 -- 172.21.15.3:0/1973289366 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c005020 con 0x55b6b22af390 2024-09-21T06:23:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.875+0000 7f1770ff9640 1 -- 172.21.15.3:0/1973289366 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176c0047a0 con 0x55b6b22af390 2024-09-21T06:23:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.875+0000 7f1770ff9640 1 -- 172.21.15.3:0/1973289366 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c00f860 con 0x55b6b22af390 2024-09-21T06:23:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.875+0000 7f17737fe640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f17680b6680 0x7f17680b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.876+0000 7f1782c31640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f17680b6680 0x7f17680b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.876+0000 7f17737fe640 1 -- 172.21.15.3:0/4246119591 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f17780e7a40 con 0x7f177c101fe0 2024-09-21T06:23:39.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.876+0000 7f1782c31640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f17680b6680 0x7f17680b8b90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f177402f500 tx=0x7f177402f310 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:39.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.876+0000 7f178ac08740 1 -- 172.21.15.3:0/1973289366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 msgr2=0x55b6b26780d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.876+0000 7f178ac08740 1 --2- 172.21.15.3:0/1973289366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 0x55b6b26780d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f176c0058c0 tx=0x7f176c0119a0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.876+0000 7f178ac08740 1 -- 172.21.15.3:0/1973289366 shutdown_connections 2024-09-21T06:23:39.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.876+0000 7f178ac08740 1 --2- 172.21.15.3:0/1973289366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 0x55b6b26780d0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.876+0000 7f178ac08740 1 --2- 172.21.15.3:0/1973289366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22a71b0 0x55b6b2677b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.876+0000 7f178ac08740 1 --2- 172.21.15.3:0/1973289366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22d0070 0x55b6b2677650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.876+0000 7f178ac08740 1 -- 172.21.15.3:0/1973289366 >> 172.21.15.3:0/1973289366 conn(0x55b6b22c4460 msgr2=0x55b6b2675540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:39.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.877+0000 7f178ac08740 1 -- 172.21.15.3:0/1973289366 shutdown_connections 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.877+0000 7f178ac08740 1 -- 172.21.15.3:0/1973289366 wait complete. 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.877+0000 7f178ac08740 1 -- start start 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.877+0000 7f178ac08740 1 Processor -- start 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.877+0000 7f178ac08740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22a71b0 0x55b6b2682d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.877+0000 7f178ac08740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22af390 0x55b6b2683270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.877+0000 7f178ac08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22d0070 0x55b6b268f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.877+0000 7f178ac08740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b6b243e700 con 0x55b6b22a71b0 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.877+0000 7f178ac08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b6b2578110 con 0x55b6b22d0070 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.877+0000 7f178ac08740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b6b2685e30 con 0x55b6b22af390 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f178ac08740 10 client.0 _ll_register_callbacks cb 0x55b6b2591990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1782430640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22a71b0 0x55b6b2682d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1782430640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22a71b0 0x55b6b2682d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59998/0 (socket says 172.21.15.3:59998) 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1782430640 1 -- 172.21.15.3:0/4109525381 learned_addr learned my addr 172.21.15.3:0/4109525381 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1782c31640 1 --2- 172.21.15.3:0/4109525381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22d0070 0x55b6b268f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1781c2f640 1 --2- 172.21.15.3:0/4109525381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22af390 0x55b6b2683270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1782430640 1 -- 172.21.15.3:0/4109525381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22af390 msgr2=0x55b6b2683270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1782430640 1 --2- 172.21.15.3:0/4109525381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22af390 0x55b6b2683270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1782430640 1 -- 172.21.15.3:0/4109525381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22d0070 msgr2=0x55b6b268f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1782430640 1 --2- 172.21.15.3:0/4109525381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22d0070 0x55b6b268f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1782430640 1 -- 172.21.15.3:0/4109525381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b25a5c40 con 0x55b6b22a71b0 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1782430640 1 --2- 172.21.15.3:0/4109525381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22a71b0 0x55b6b2682d30 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f17780aabc0 tx=0x7f17780eb140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d4d00d49fd73b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1765ffb640 1 -- 172.21.15.3:0/4109525381 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17780ad0c0 con 0x55b6b22a71b0 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f178ac08740 1 -- 172.21.15.3:0/4109525381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f176c0047a0 con 0x55b6b22a71b0 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.878+0000 7f1765ffb640 1 -- 172.21.15.3:0/4109525381 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f17780acbb0 con 0x55b6b22a71b0 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.879+0000 7f1765ffb640 1 -- 172.21.15.3:0/4109525381 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f17780a90c0 con 0x55b6b22a71b0 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.879+0000 7f178ac08740 10 client.18012 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.879+0000 7f178ac08740 1 -- 172.21.15.3:0/4109525381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b6b2691bc0 con 0x55b6b22a71b0 2024-09-21T06:23:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.879+0000 7f1765ffb640 1 -- 172.21.15.3:0/4109525381 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176c0047a0 con 0x55b6b22a71b0 2024-09-21T06:23:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.879+0000 7f1765ffb640 1 -- 172.21.15.3:0/4109525381 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f17780ac620 con 0x55b6b22a71b0 2024-09-21T06:23:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.879+0000 7f178ac08740 10 client.18012 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.879+0000 7f178ac08740 10 client.18012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.879+0000 7f178ac08740 20 client.18012 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.879+0000 7f178ac08740 1 --2- 172.21.15.3:0/4109525381 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6b228e9f0 0x55b6b26950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.879+0000 7f178ac08740 4 client.18012 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.879+0000 7f178ac08740 1 -- 172.21.15.3:0/4109525381 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6b25efda0 con 0x55b6b228e9f0 2024-09-21T06:23:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.879+0000 7f1781c2f640 1 --2- 172.21.15.3:0/4109525381 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6b228e9f0 0x55b6b26950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.880+0000 7f1765ffb640 10 client.18012 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.880+0000 7f1781c2f640 1 --2- 172.21.15.3:0/4109525381 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6b228e9f0 0x55b6b26950c0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492eea9da99fbbb1 server_cookie=c2543f28c8af5307 in_seq=0 out_seq=0 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.880+0000 7f1765ffb640 1 -- 172.21.15.3:0/4109525381 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6b25efda0 con 0x55b6b228e9f0 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.880+0000 7f1765ffb640 10 client.18012 handle_command_reply: tid=0 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.881+0000 7f178ac08740 1 client.18012 shutdown 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.881+0000 7f178ac08740 1 -- 172.21.15.3:0/4109525381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22a71b0 msgr2=0x55b6b2682d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.881+0000 7f178ac08740 1 --2- 172.21.15.3:0/4109525381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22a71b0 0x55b6b2682d30 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f17780aabc0 tx=0x7f17780eb140 comp rx=0 tx=0).stop 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.881+0000 7f178ac08740 1 -- 172.21.15.3:0/4109525381 shutdown_connections 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.881+0000 7f178ac08740 1 --2- 172.21.15.3:0/4109525381 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6b228e9f0 0x55b6b26950c0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.881+0000 7f178ac08740 1 --2- 172.21.15.3:0/4109525381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22af390 0x55b6b2683270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.881+0000 7f178ac08740 1 --2- 172.21.15.3:0/4109525381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22d0070 0x55b6b268f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.881+0000 7f178ac08740 1 --2- 172.21.15.3:0/4109525381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22a71b0 0x55b6b2682d30 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.881+0000 7f178ac08740 1 -- 172.21.15.3:0/4109525381 >> 172.21.15.3:0/4109525381 conn(0x55b6b22c4460 msgr2=0x55b6b267a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.881+0000 7f1765ffb640 0 client.18012 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.881+0000 7f178ac08740 1 -- 172.21.15.3:0/4109525381 shutdown_connections 2024-09-21T06:23:39.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.881+0000 7f178ac08740 1 -- 172.21.15.3:0/4109525381 wait complete. 2024-09-21T06:23:39.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.881+0000 7f178ac08740 20 client.18012 trim_cache size 0 max 16384 2024-09-21T06:23:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:39 smithi137 ceph-mon[34464]: pgmap v590: 97 pgs: 97 active+clean; 132 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 313 op/s 2024-09-21T06:23:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:39 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:23:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:39 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:39 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:23:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:39 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:23:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:39 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:39 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:23:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:39 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:39 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f178ac08740 1 Processor -- start 2024-09-21T06:23:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f178ac08740 1 -- start start 2024-09-21T06:23:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f178ac08740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22c4460 0x55b6b2684250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f178ac08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22a71b0 0x55b6b2674b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f178ac08740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 0x55b6b26750b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f178ac08740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1b17f0 con 0x55b6b22af390 2024-09-21T06:23:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f178ac08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1b3330 con 0x55b6b22a71b0 2024-09-21T06:23:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f178ac08740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1b1ec0 con 0x55b6b22c4460 2024-09-21T06:23:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f1782c31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 0x55b6b26750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f1782c31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 0x55b6b26750b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60012/0 (socket says 172.21.15.3:60012) 2024-09-21T06:23:39.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f1782c31640 1 -- 172.21.15.3:0/2818262443 learned_addr learned my addr 172.21.15.3:0/2818262443 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:39.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f1781c2f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22a71b0 0x55b6b2674b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f1782430640 1 --2- 172.21.15.3:0/2818262443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22c4460 0x55b6b2684250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f1782c31640 1 -- 172.21.15.3:0/2818262443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22c4460 msgr2=0x55b6b2684250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f1782c31640 1 --2- 172.21.15.3:0/2818262443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22c4460 0x55b6b2684250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f1782c31640 1 -- 172.21.15.3:0/2818262443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22a71b0 msgr2=0x55b6b2674b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f1782c31640 1 --2- 172.21.15.3:0/2818262443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22a71b0 0x55b6b2674b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f1782c31640 1 -- 172.21.15.3:0/2818262443 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b2685100 con 0x55b6b22af390 2024-09-21T06:23:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.900+0000 7f1782430640 1 --2- 172.21.15.3:0/2818262443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22c4460 0x55b6b2684250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.901+0000 7f1782c31640 1 --2- 172.21.15.3:0/2818262443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 0x55b6b26750b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1774009ca0 tx=0x7f1774044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd6451d3b4fdb53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.901+0000 7f1767fff640 1 -- 172.21.15.3:0/2818262443 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774003970 con 0x55b6b22af390 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.901+0000 7f1767fff640 1 -- 172.21.15.3:0/2818262443 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1774036050 con 0x55b6b22af390 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.901+0000 7f1767fff640 1 -- 172.21.15.3:0/2818262443 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177402f8c0 con 0x55b6b22af390 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.902+0000 7f178ac08740 1 -- 172.21.15.3:0/2818262443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 msgr2=0x55b6b26750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.902+0000 7f178ac08740 1 --2- 172.21.15.3:0/2818262443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 0x55b6b26750b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1774009ca0 tx=0x7f1774044000 comp rx=0 tx=0).stop 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.902+0000 7f178ac08740 1 -- 172.21.15.3:0/2818262443 shutdown_connections 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.902+0000 7f178ac08740 1 --2- 172.21.15.3:0/2818262443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22af390 0x55b6b26750b0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.902+0000 7f178ac08740 1 --2- 172.21.15.3:0/2818262443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22a71b0 0x55b6b2674b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.902+0000 7f178ac08740 1 --2- 172.21.15.3:0/2818262443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22c4460 0x55b6b2684250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.902+0000 7f178ac08740 1 -- 172.21.15.3:0/2818262443 >> 172.21.15.3:0/2818262443 conn(0x55b6b26833f0 msgr2=0x55b6b2679040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.902+0000 7f178ac08740 1 -- 172.21.15.3:0/2818262443 shutdown_connections 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.902+0000 7f178ac08740 1 -- 172.21.15.3:0/2818262443 wait complete. 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.902+0000 7f178ac08740 1 -- start start 2024-09-21T06:23:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f178ac08740 1 Processor -- start 2024-09-21T06:23:39.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:39 smithi204 ceph-mon[34659]: pgmap v590: 97 pgs: 97 active+clean; 132 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 313 op/s 2024-09-21T06:23:39.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:39 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:23:39.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:39 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:39.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:39 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:23:39.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:39 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:23:39.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:39 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:39.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:39 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:23:39.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:39 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:39.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:39 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f178ac08740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22a71b0 0x55b6b26874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f178ac08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22af390 0x55b6b2687ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f178ac08740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22c4460 0x55b6b26fea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f178ac08740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1b2c60 con 0x55b6b22c4460 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f178ac08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1b2590 con 0x55b6b22af390 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f178ac08740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1af7f0 con 0x55b6b22a71b0 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f178ac08740 10 client.0 _ll_register_callbacks cb 0x7f17780056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f1781c2f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22af390 0x55b6b2687ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f1782c31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22c4460 0x55b6b26fea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f1781c2f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22af390 0x55b6b2687ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:47770/0 (socket says 172.21.15.3:47770) 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f1781c2f640 1 -- 172.21.15.3:0/2064844023 learned_addr learned my addr 172.21.15.3:0/2064844023 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f1782430640 1 --2- 172.21.15.3:0/2064844023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22a71b0 0x55b6b26874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f1782c31640 1 -- 172.21.15.3:0/2064844023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22a71b0 msgr2=0x55b6b26874d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f1782c31640 1 --2- 172.21.15.3:0/2064844023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22a71b0 0x55b6b26874d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f1782c31640 1 -- 172.21.15.3:0/2064844023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22af390 msgr2=0x55b6b2687ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f1782c31640 1 --2- 172.21.15.3:0/2064844023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22af390 0x55b6b2687ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.903+0000 7f1782c31640 1 -- 172.21.15.3:0/2064844023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b26fefc0 con 0x55b6b22c4460 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.904+0000 7f1782c31640 1 --2- 172.21.15.3:0/2064844023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22c4460 0x55b6b26fea80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1774043040 tx=0x7f177402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7858cf86bbdfd5b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.904+0000 7f1765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.904+0000 7f1765ffb640 1 -- 172.21.15.3:0/2064844023 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177405b020 con 0x55b6b22c4460 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.904+0000 7f1765ffb640 1 -- 172.21.15.3:0/2064844023 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f1774003b70 con 0x55b6b22c4460 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.904+0000 7f178ac08740 1 -- 172.21.15.3:0/2064844023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1774036050 con 0x55b6b22c4460 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.904+0000 7f1765ffb640 1 -- 172.21.15.3:0/2064844023 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1774066050 con 0x55b6b22c4460 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.904+0000 7f178ac08740 10 client.18018 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.904+0000 7f178ac08740 1 -- 172.21.15.3:0/2064844023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b6b2700180 con 0x55b6b22c4460 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.904+0000 7f1765ffb640 1 -- 172.21.15.3:0/2064844023 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1774036050 con 0x55b6b22c4460 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.905+0000 7f1765ffb640 1 -- 172.21.15.3:0/2064844023 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f17740305c0 con 0x55b6b22c4460 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.905+0000 7f178ac08740 10 client.18018 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.905+0000 7f178ac08740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.905+0000 7f178ac08740 20 client.18018 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.905+0000 7f178ac08740 1 --2- 172.21.15.3:0/2064844023 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6b22d0070 0x55b6b27028b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.905+0000 7f178ac08740 4 client.18018 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.905+0000 7f178ac08740 1 -- 172.21.15.3:0/2064844023 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6b25eb850 con 0x55b6b22d0070 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.905+0000 7f1782430640 1 --2- 172.21.15.3:0/2064844023 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6b22d0070 0x55b6b27028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.905+0000 7f1782430640 1 --2- 172.21.15.3:0/2064844023 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6b22d0070 0x55b6b27028b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d5a9e395b2988e2 server_cookie=2038a9d0cf476c1e in_seq=0 out_seq=0 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.905+0000 7f1765ffb640 10 client.18018 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f1765ffb640 1 -- 172.21.15.3:0/2064844023 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x55b6b25eb850 con 0x55b6b22d0070 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f1765ffb640 10 client.18018 handle_command_reply: tid=0 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f178ac08740 1 client.18018 shutdown 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f178ac08740 1 -- 172.21.15.3:0/2064844023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22c4460 msgr2=0x55b6b26fea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f178ac08740 1 --2- 172.21.15.3:0/2064844023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22c4460 0x55b6b26fea80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1774043040 tx=0x7f177402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f178ac08740 1 -- 172.21.15.3:0/2064844023 shutdown_connections 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f178ac08740 1 --2- 172.21.15.3:0/2064844023 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b6b22d0070 0x55b6b27028b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f178ac08740 1 --2- 172.21.15.3:0/2064844023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6b22c4460 0x55b6b26fea80 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f178ac08740 1 --2- 172.21.15.3:0/2064844023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6b22af390 0x55b6b2687ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f178ac08740 1 --2- 172.21.15.3:0/2064844023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6b22a71b0 0x55b6b26874d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f178ac08740 1 -- 172.21.15.3:0/2064844023 >> 172.21.15.3:0/2064844023 conn(0x55b6b26833f0 msgr2=0x55b6b2679040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:39.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f1765ffb640 0 client.18018 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:39.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f178ac08740 1 -- 172.21.15.3:0/2064844023 shutdown_connections 2024-09-21T06:23:39.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f178ac08740 1 -- 172.21.15.3:0/2064844023 wait complete. 2024-09-21T06:23:39.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.907+0000 7f178ac08740 20 client.18018 trim_cache size 0 max 16384 2024-09-21T06:23:39.913 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:23:39.913 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:23:39.913 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-21T06:23:39.913 INFO:teuthology.orchestra.run.smithi003.stdout: "cbb5d4a7-b96d-4c2c-b7ab-7a3716c17f5b": { 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "cbb5d4a7-b96d-4c2c-b7ab-7a3716c17f5b", 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: "535fb164-fd4f-4d88-b7e0-575559314385": { 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:51.152572+0000", 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:39.914 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:39.915 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: "714b9934-a2d3-4a78-b185-01254776061b": { 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:04.685380+0000", 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:39.916 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: "cbb5d4a7-b96d-4c2c-b7ab-7a3716c17f5b": { 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:23:38.372765+0000", 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.917 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:39.918 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:39.918 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:39.918 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:39.918 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:39.918 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:23:39.918 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:23:39.918 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.918 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:39.918 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:39.918 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:39.918 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:39.918 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:39.918 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58": { 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:21:17.396219+0000", 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:39.919 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:39.920 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:39.921 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:39.921 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:23:39.921 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:23:39.921 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:23:39.921 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:23:39.921 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:23:39.921 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:23:39.921 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:23:39.921 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:23:39.921 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:23:39.921 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:23:39.921 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:23:39.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.908+0000 7f1784694640 1 -- 172.21.15.3:0/4246119591 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f17680b6680 msgr2=0x7f17680b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.908+0000 7f1784694640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f17680b6680 0x7f17680b8b90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f177402f500 tx=0x7f177402f310 comp rx=0 tx=0).stop 2024-09-21T06:23:39.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.908+0000 7f1784694640 1 -- 172.21.15.3:0/4246119591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f177c101fe0 msgr2=0x7f177c19c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.908+0000 7f1784694640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f177c101fe0 0x7f177c19c6b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f177800e9f0 tx=0x7f177800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.908+0000 7f1784694640 1 -- 172.21.15.3:0/4246119591 shutdown_connections 2024-09-21T06:23:39.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.908+0000 7f1784694640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f17680b6680 0x7f17680b8b90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.908+0000 7f1784694640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f177c109e10 0x7f177c1a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.908+0000 7f1784694640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f177c1094c0 0x7f177c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.908+0000 7f1784694640 1 --2- 172.21.15.3:0/4246119591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f177c101fe0 0x7f177c19c6b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.908+0000 7f1784694640 1 -- 172.21.15.3:0/4246119591 >> 172.21.15.3:0/4246119591 conn(0x7f177c077bc0 msgr2=0x7f177c10f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:39.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.908+0000 7f1784694640 1 -- 172.21.15.3:0/4246119591 shutdown_connections 2024-09-21T06:23:39.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:23:39.908+0000 7f1784694640 1 -- 172.21.15.3:0/4246119591 wait complete. 2024-09-21T06:23:39.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 900667 10.51 MB/sec execute 312 sec latency 34.808 ms 2024-09-21T06:23:40.639 INFO:tasks.cephfs.filesystem:scrub status for tag:cbb5d4a7-b96d-4c2c-b7ab-7a3716c17f5b - {'path': '/', 'tag': 'cbb5d4a7-b96d-4c2c-b7ab-7a3716c17f5b', 'options': 'recursive,force'} 2024-09-21T06:23:40.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 902115 10.51 MB/sec execute 313 sec latency 22.669 ms 2024-09-21T06:23:41.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:41 smithi003 ceph-mon[31227]: pgmap v591: 97 pgs: 97 active+clean; 132 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-09-21T06:23:41.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:41 smithi137 ceph-mon[34464]: pgmap v591: 97 pgs: 97 active+clean; 132 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-09-21T06:23:41.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:41 smithi204 ceph-mon[34659]: pgmap v591: 97 pgs: 97 active+clean; 132 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-09-21T06:23:41.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 903201 10.50 MB/sec execute 314 sec latency 175.708 ms 2024-09-21T06:23:42.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 905380 10.50 MB/sec execute 315 sec latency 17.754 ms 2024-09-21T06:23:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.091+0000 7fc258733640 1 -- 172.21.15.3:0/1118426137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc250108c60 msgr2=0x7fc25010f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.091+0000 7fc258733640 1 --2- 172.21.15.3:0/1118426137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc250108c60 0x7fc25010f490 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fc248009ca0 tx=0x7fc24802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.091+0000 7fc258733640 1 -- 172.21.15.3:0/1118426137 shutdown_connections 2024-09-21T06:23:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.091+0000 7fc258733640 1 --2- 172.21.15.3:0/1118426137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc250108c60 0x7fc25010f490 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.091+0000 7fc258733640 1 --2- 172.21.15.3:0/1118426137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc250108310 0x7fc250108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.091+0000 7fc258733640 1 --2- 172.21.15.3:0/1118426137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2501029e0 0x7fc250102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.091+0000 7fc258733640 1 -- 172.21.15.3:0/1118426137 >> 172.21.15.3:0/1118426137 conn(0x7fc2500fe3c0 msgr2=0x7fc250100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.091+0000 7fc258733640 1 -- 172.21.15.3:0/1118426137 shutdown_connections 2024-09-21T06:23:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.091+0000 7fc258733640 1 -- 172.21.15.3:0/1118426137 wait complete. 2024-09-21T06:23:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.091+0000 7fc258733640 1 Processor -- start 2024-09-21T06:23:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc258733640 1 -- start start 2024-09-21T06:23:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc258733640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2501029e0 0x7fc25019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc258733640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc250108310 0x7fc25019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc258733640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc250108c60 0x7fc25019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc258733640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc250112970 con 0x7fc2501029e0 2024-09-21T06:23:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc258733640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501127f0 con 0x7fc250108310 2024-09-21T06:23:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc258733640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc250112af0 con 0x7fc250108c60 2024-09-21T06:23:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc2564cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2501029e0 0x7fc25019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc2564cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2501029e0 0x7fc25019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60042/0 (socket says 172.21.15.3:60042) 2024-09-21T06:23:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc2564cf640 1 -- 172.21.15.3:0/92724382 learned_addr learned my addr 172.21.15.3:0/92724382 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc256cd0640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc250108c60 0x7fc25019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc255cce640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc250108310 0x7fc25019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc2564cf640 1 -- 172.21.15.3:0/92724382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc250108c60 msgr2=0x7fc25019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc2564cf640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc250108c60 0x7fc25019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc2564cf640 1 -- 172.21.15.3:0/92724382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc250108310 msgr2=0x7fc25019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc2564cf640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc250108310 0x7fc25019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc2564cf640 1 -- 172.21.15.3:0/92724382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2501a0390 con 0x7fc2501029e0 2024-09-21T06:23:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc256cd0640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc250108c60 0x7fc25019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.092+0000 7fc2564cf640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2501029e0 0x7fc25019a460 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00e9f0 tx=0x7fc24c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.093+0000 7fc2477fe640 1 -- 172.21.15.3:0/92724382 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c00cd80 con 0x7fc2501029e0 2024-09-21T06:23:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.093+0000 7fc2477fe640 1 -- 172.21.15.3:0/92724382 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc24c0054d0 con 0x7fc2501029e0 2024-09-21T06:23:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.093+0000 7fc2477fe640 1 -- 172.21.15.3:0/92724382 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c010690 con 0x7fc2501029e0 2024-09-21T06:23:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.093+0000 7fc258733640 1 -- 172.21.15.3:0/92724382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2501a0680 con 0x7fc2501029e0 2024-09-21T06:23:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.093+0000 7fc258733640 1 -- 172.21.15.3:0/92724382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2501a6e10 con 0x7fc2501029e0 2024-09-21T06:23:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.095+0000 7fc25f69d740 1 Processor -- start 2024-09-21T06:23:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc25f69d740 1 -- start start 2024-09-21T06:23:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc25f69d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e04a1b0 0x55659e3ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc25f69d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0212f0 0x55659e3efef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc25f69d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0294d0 0x55659e3f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc25f69d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55659e3646f0 con 0x55659e04a1b0 2024-09-21T06:23:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc25f69d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55659e353b20 con 0x55659e0294d0 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc25f69d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55659e357930 con 0x55659e0212f0 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc2477fe640 1 -- 172.21.15.3:0/92724382 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fc24c004e90 con 0x7fc2501029e0 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc256cd0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e04a1b0 0x55659e3ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc256cd0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e04a1b0 0x55659e3ef9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60054/0 (socket says 172.21.15.3:60054) 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc255cce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0294d0 0x55659e3f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc256cd0640 1 -- 172.21.15.3:0/3884871978 learned_addr learned my addr 172.21.15.3:0/3884871978 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc2564cf640 1 --2- 172.21.15.3:0/3884871978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0212f0 0x55659e3efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc256cd0640 1 -- 172.21.15.3:0/3884871978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0212f0 msgr2=0x55659e3efef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.096+0000 7fc256cd0640 1 --2- 172.21.15.3:0/3884871978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0212f0 0x55659e3efef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.097+0000 7fc256cd0640 1 -- 172.21.15.3:0/3884871978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0294d0 msgr2=0x55659e3f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.097+0000 7fc256cd0640 1 --2- 172.21.15.3:0/3884871978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0294d0 0x55659e3f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.097+0000 7fc256cd0640 1 -- 172.21.15.3:0/3884871978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55659e319530 con 0x55659e04a1b0 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.097+0000 7fc255cce640 1 --2- 172.21.15.3:0/3884871978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0294d0 0x55659e3f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.097+0000 7fc256cd0640 1 --2- 172.21.15.3:0/3884871978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e04a1b0 0x55659e3ef9b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fc248009ca0 tx=0x7fc248009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1dc1a3e4b10f952 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.097+0000 7fc244ff9640 1 -- 172.21.15.3:0/3884871978 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24802f3d0 con 0x55659e04a1b0 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.097+0000 7fc244ff9640 1 -- 172.21.15.3:0/3884871978 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc248036040 con 0x55659e04a1b0 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.097+0000 7fc244ff9640 1 -- 172.21.15.3:0/3884871978 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248004440 con 0x55659e04a1b0 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.097+0000 7fc2477fe640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc23c0b6700 0x7fc23c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.097+0000 7fc2564cf640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc23c0b6700 0x7fc23c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.098+0000 7fc2477fe640 1 -- 172.21.15.3:0/92724382 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7fc24c0e7a80 con 0x7fc2501029e0 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.098+0000 7fc25f69d740 1 -- 172.21.15.3:0/3884871978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e04a1b0 msgr2=0x55659e3ef9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.098+0000 7fc25f69d740 1 --2- 172.21.15.3:0/3884871978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e04a1b0 0x55659e3ef9b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fc248009ca0 tx=0x7fc248009490 comp rx=0 tx=0).stop 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.098+0000 7fc2564cf640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc23c0b6700 0x7fc23c0b8c10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0e5290 tx=0x7fc24c0eb5d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.098+0000 7fc25f69d740 1 -- 172.21.15.3:0/3884871978 shutdown_connections 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.098+0000 7fc25f69d740 1 --2- 172.21.15.3:0/3884871978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0212f0 0x55659e3efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.098+0000 7fc25f69d740 1 --2- 172.21.15.3:0/3884871978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0294d0 0x55659e3f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.098+0000 7fc25f69d740 1 --2- 172.21.15.3:0/3884871978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e04a1b0 0x55659e3ef9b0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.098+0000 7fc25f69d740 1 -- 172.21.15.3:0/3884871978 >> 172.21.15.3:0/3884871978 conn(0x55659e03e300 msgr2=0x55659e3ed8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.098+0000 7fc25f69d740 1 -- 172.21.15.3:0/3884871978 shutdown_connections 2024-09-21T06:23:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.098+0000 7fc25f69d740 1 -- 172.21.15.3:0/3884871978 wait complete. 2024-09-21T06:23:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.098+0000 7fc25f69d740 1 -- start start 2024-09-21T06:23:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc25f69d740 1 Processor -- start 2024-09-21T06:23:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc25f69d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0212f0 0x55659e3fb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc25f69d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0294d0 0x55659e3fb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc25f69d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e04a1b0 0x55659e407240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc25f69d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55659e28bab0 con 0x55659e04a1b0 2024-09-21T06:23:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc25f69d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55659e2f9730 con 0x55659e0212f0 2024-09-21T06:23:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc25f69d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55659e3fdf50 con 0x55659e0294d0 2024-09-21T06:23:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc25f69d740 10 client.0 _ll_register_callbacks cb 0x55659e34d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc256cd0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0212f0 0x55659e3fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc255cce640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0294d0 0x55659e3fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc256cd0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e04a1b0 0x55659e407240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc255cce640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0294d0 0x55659e3fb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:46838/0 (socket says 172.21.15.3:46838) 2024-09-21T06:23:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc255cce640 1 -- 172.21.15.3:0/3347003237 learned_addr learned my addr 172.21.15.3:0/3347003237 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc256cd0640 1 -- 172.21.15.3:0/3347003237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0294d0 msgr2=0x55659e3fb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc256cd0640 1 --2- 172.21.15.3:0/3347003237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0294d0 0x55659e3fb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc256cd0640 1 -- 172.21.15.3:0/3347003237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0212f0 msgr2=0x55659e3fb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc256cd0640 1 --2- 172.21.15.3:0/3347003237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0212f0 0x55659e3fb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.099+0000 7fc256cd0640 1 -- 172.21.15.3:0/3347003237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55659e0b5790 con 0x55659e04a1b0 2024-09-21T06:23:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc256cd0640 1 --2- 172.21.15.3:0/3347003237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e04a1b0 0x55659e407240 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fc24802ade0 tx=0x7fc248004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0244986dc1fc5d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc239ffb640 1 -- 172.21.15.3:0/3347003237 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248004d40 con 0x55659e04a1b0 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc239ffb640 1 -- 172.21.15.3:0/3347003237 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7fc248003970 con 0x55659e04a1b0 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc25f69d740 1 -- 172.21.15.3:0/3347003237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc248036040 con 0x55659e04a1b0 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc239ffb640 1 -- 172.21.15.3:0/3347003237 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc248003b70 con 0x55659e04a1b0 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc25f69d740 10 client.18030 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc25f69d740 1 -- 172.21.15.3:0/3347003237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55659e12e620 con 0x55659e04a1b0 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc239ffb640 1 -- 172.21.15.3:0/3347003237 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc248036040 con 0x55659e04a1b0 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc239ffb640 1 -- 172.21.15.3:0/3347003237 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fc248044070 con 0x55659e04a1b0 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc25f69d740 10 client.18030 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc25f69d740 10 client.18030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc25f69d740 20 client.18030 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc25f69d740 1 --2- 172.21.15.3:0/3347003237 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55659e008d80 0x55659e40cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc25f69d740 4 client.18030 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.100+0000 7fc25f69d740 1 -- 172.21.15.3:0/3347003237 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55659e366640 con 0x55659e008d80 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.101+0000 7fc255cce640 1 --2- 172.21.15.3:0/3347003237 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55659e008d80 0x55659e40cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.101+0000 7fc255cce640 1 --2- 172.21.15.3:0/3347003237 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55659e008d80 0x55659e40cdc0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39190466ce7cb8a4 server_cookie=166f1cfc57f3ef27 in_seq=0 out_seq=0 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.101+0000 7fc239ffb640 10 client.18030 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.102+0000 7fc239ffb640 1 -- 172.21.15.3:0/3347003237 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55659e366640 con 0x55659e008d80 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.102+0000 7fc239ffb640 10 client.18030 handle_command_reply: tid=0 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.102+0000 7fc25f69d740 1 client.18030 shutdown 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.102+0000 7fc25f69d740 1 -- 172.21.15.3:0/3347003237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e04a1b0 msgr2=0x55659e407240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.102+0000 7fc25f69d740 1 --2- 172.21.15.3:0/3347003237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e04a1b0 0x55659e407240 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fc24802ade0 tx=0x7fc248004040 comp rx=0 tx=0).stop 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.102+0000 7fc25f69d740 1 -- 172.21.15.3:0/3347003237 shutdown_connections 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.102+0000 7fc25f69d740 1 --2- 172.21.15.3:0/3347003237 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55659e008d80 0x55659e40cdc0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.102+0000 7fc25f69d740 1 --2- 172.21.15.3:0/3347003237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0294d0 0x55659e3fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.102+0000 7fc25f69d740 1 --2- 172.21.15.3:0/3347003237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e04a1b0 0x55659e407240 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.102+0000 7fc25f69d740 1 --2- 172.21.15.3:0/3347003237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0212f0 0x55659e3fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.102+0000 7fc239ffb640 0 client.18030 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.102+0000 7fc25f69d740 1 -- 172.21.15.3:0/3347003237 >> 172.21.15.3:0/3347003237 conn(0x55659e03e300 msgr2=0x55659e3f09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.103+0000 7fc25f69d740 1 -- 172.21.15.3:0/3347003237 shutdown_connections 2024-09-21T06:23:43.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.103+0000 7fc25f69d740 1 -- 172.21.15.3:0/3347003237 wait complete. 2024-09-21T06:23:43.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.103+0000 7fc25f69d740 20 client.18030 trim_cache size 0 max 16384 2024-09-21T06:23:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.118+0000 7fc25f69d740 1 Processor -- start 2024-09-21T06:23:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.118+0000 7fc25f69d740 1 -- start start 2024-09-21T06:23:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.118+0000 7fc25f69d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0212f0 0x55659e3ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.118+0000 7fc25f69d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0294d0 0x55659e407140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.118+0000 7fc25f69d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e04a1b0 0x55659e3f8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.118+0000 7fc25f69d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501af680 con 0x55659e0212f0 2024-09-21T06:23:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.118+0000 7fc25f69d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501b11c0 con 0x55659e0294d0 2024-09-21T06:23:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.118+0000 7fc25f69d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501afd50 con 0x55659e04a1b0 2024-09-21T06:23:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.118+0000 7fc256cd0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0212f0 0x55659e3ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.118+0000 7fc256cd0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0212f0 0x55659e3ec5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60080/0 (socket says 172.21.15.3:60080) 2024-09-21T06:23:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.118+0000 7fc256cd0640 1 -- 172.21.15.3:0/2033934011 learned_addr learned my addr 172.21.15.3:0/2033934011 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.118+0000 7fc255cce640 1 --2- 172.21.15.3:0/2033934011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0294d0 0x55659e407140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.118+0000 7fc256cd0640 1 --2- 172.21.15.3:0/2033934011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e04a1b0 0x55659e3f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.119+0000 7fc256cd0640 1 -- 172.21.15.3:0/2033934011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e04a1b0 msgr2=0x55659e3f8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.119+0000 7fc256cd0640 1 --2- 172.21.15.3:0/2033934011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e04a1b0 0x55659e3f8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.119+0000 7fc256cd0640 1 -- 172.21.15.3:0/2033934011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0294d0 msgr2=0x55659e407140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.119+0000 7fc256cd0640 1 --2- 172.21.15.3:0/2033934011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0294d0 0x55659e407140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.119+0000 7fc256cd0640 1 -- 172.21.15.3:0/2033934011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55659e40d0f0 con 0x55659e0212f0 2024-09-21T06:23:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.119+0000 7fc256cd0640 1 --2- 172.21.15.3:0/2033934011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0212f0 0x55659e3ec5b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc248009ca0 tx=0x7fc24802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff938e5a6293b78 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.119+0000 7fc23bfff640 1 -- 172.21.15.3:0/2033934011 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248003970 con 0x55659e0212f0 2024-09-21T06:23:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.119+0000 7fc23bfff640 1 -- 172.21.15.3:0/2033934011 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc248064030 con 0x55659e0212f0 2024-09-21T06:23:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.119+0000 7fc23bfff640 1 -- 172.21.15.3:0/2033934011 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2480043a0 con 0x55659e0212f0 2024-09-21T06:23:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.120+0000 7fc25f69d740 1 -- 172.21.15.3:0/2033934011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0212f0 msgr2=0x55659e3ec5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.120+0000 7fc25f69d740 1 --2- 172.21.15.3:0/2033934011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0212f0 0x55659e3ec5b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc248009ca0 tx=0x7fc24802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.120+0000 7fc25f69d740 1 -- 172.21.15.3:0/2033934011 shutdown_connections 2024-09-21T06:23:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.120+0000 7fc25f69d740 1 --2- 172.21.15.3:0/2033934011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e04a1b0 0x55659e3f8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.120+0000 7fc25f69d740 1 --2- 172.21.15.3:0/2033934011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e0294d0 0x55659e407140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.120+0000 7fc25f69d740 1 --2- 172.21.15.3:0/2033934011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0212f0 0x55659e3ec5b0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.120+0000 7fc25f69d740 1 -- 172.21.15.3:0/2033934011 >> 172.21.15.3:0/2033934011 conn(0x55659e03e300 msgr2=0x55659e3ebea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.120+0000 7fc25f69d740 1 -- 172.21.15.3:0/2033934011 shutdown_connections 2024-09-21T06:23:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.120+0000 7fc25f69d740 1 -- 172.21.15.3:0/2033934011 wait complete. 2024-09-21T06:23:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.120+0000 7fc25f69d740 1 -- start start 2024-09-21T06:23:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.120+0000 7fc25f69d740 1 Processor -- start 2024-09-21T06:23:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc25f69d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0212f0 0x55659e400950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc25f69d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0294d0 0x55659e400e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc25f69d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e04a1b0 0x55659e475560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc25f69d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501b0af0 con 0x55659e0294d0 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc25f69d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501b0420 con 0x55659e04a1b0 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc25f69d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501ad680 con 0x55659e0212f0 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc25f69d740 10 client.0 _ll_register_callbacks cb 0x7fc2400097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc255cce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0294d0 0x55659e400e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc255cce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0294d0 0x55659e400e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60090/0 (socket says 172.21.15.3:60090) 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc255cce640 1 -- 172.21.15.3:0/271907396 learned_addr learned my addr 172.21.15.3:0/271907396 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc256cd0640 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0212f0 0x55659e400950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc256cd0640 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e04a1b0 0x55659e475560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc255cce640 1 -- 172.21.15.3:0/271907396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0212f0 msgr2=0x55659e400950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc255cce640 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0212f0 0x55659e400950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc255cce640 1 -- 172.21.15.3:0/271907396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e04a1b0 msgr2=0x55659e475560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc255cce640 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e04a1b0 0x55659e475560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc255cce640 1 -- 172.21.15.3:0/271907396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55659e475b20 con 0x55659e0294d0 2024-09-21T06:23:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc256cd0640 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e04a1b0 0x55659e475560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc255cce640 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0294d0 0x55659e400e90 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc240009cd0 tx=0x7fc240010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04e05c7438638bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc239ffb640 1 -- 172.21.15.3:0/271907396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240028020 con 0x55659e0294d0 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc239ffb640 1 -- 172.21.15.3:0/271907396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7fc24001fa50 con 0x55659e0294d0 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.121+0000 7fc25f69d740 1 -- 172.21.15.3:0/271907396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc248064030 con 0x55659e0294d0 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.122+0000 7fc239ffb640 1 -- 172.21.15.3:0/271907396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc24001fc50 con 0x55659e0294d0 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.122+0000 7fc25f69d740 10 client.18036 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.122+0000 7fc25f69d740 1 -- 172.21.15.3:0/271907396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55659e476e40 con 0x55659e0294d0 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.122+0000 7fc239ffb640 1 -- 172.21.15.3:0/271907396 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc248064030 con 0x55659e0294d0 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.122+0000 7fc239ffb640 1 -- 172.21.15.3:0/271907396 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fc240023070 con 0x55659e0294d0 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.122+0000 7fc25f69d740 10 client.18036 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.122+0000 7fc25f69d740 10 client.18036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.122+0000 7fc25f69d740 20 client.18036 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.122+0000 7fc25f69d740 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55659dfde1c0 0x55659e4795e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.122+0000 7fc25f69d740 4 client.18036 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:23:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.122+0000 7fc25f69d740 1 -- 172.21.15.3:0/271907396 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55659e3620f0 con 0x55659dfde1c0 2024-09-21T06:23:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.122+0000 7fc256cd0640 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55659dfde1c0 0x55659e4795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.123+0000 7fc256cd0640 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55659dfde1c0 0x55659e4795e0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1964ab51dc9541 server_cookie=9578155527a30e6e in_seq=0 out_seq=0 2024-09-21T06:23:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.123+0000 7fc239ffb640 10 client.18036 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc239ffb640 1 -- 172.21.15.3:0/271907396 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55659e3620f0 con 0x55659dfde1c0 2024-09-21T06:23:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc239ffb640 10 client.18036 handle_command_reply: tid=0 2024-09-21T06:23:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc25f69d740 1 client.18036 shutdown 2024-09-21T06:23:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc25f69d740 1 -- 172.21.15.3:0/271907396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0294d0 msgr2=0x55659e400e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc25f69d740 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0294d0 0x55659e400e90 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc240009cd0 tx=0x7fc240010b30 comp rx=0 tx=0).stop 2024-09-21T06:23:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc25f69d740 1 -- 172.21.15.3:0/271907396 shutdown_connections 2024-09-21T06:23:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc25f69d740 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55659dfde1c0 0x55659e4795e0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc25f69d740 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55659e04a1b0 0x55659e475560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc25f69d740 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55659e0294d0 0x55659e400e90 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc239ffb640 0 client.18036 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc25f69d740 1 --2- 172.21.15.3:0/271907396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659e0212f0 0x55659e400950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc25f69d740 1 -- 172.21.15.3:0/271907396 >> 172.21.15.3:0/271907396 conn(0x55659e03e300 msgr2=0x55659e3ed880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc25f69d740 1 -- 172.21.15.3:0/271907396 shutdown_connections 2024-09-21T06:23:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc25f69d740 1 -- 172.21.15.3:0/271907396 wait complete. 2024-09-21T06:23:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.301+0000 7fc25f69d740 20 client.18036 trim_cache size 0 max 16384 2024-09-21T06:23:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:23:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:23:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:23:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:23:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.302+0000 7fc258733640 1 -- 172.21.15.3:0/92724382 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc23c0b6700 msgr2=0x7fc23c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.302+0000 7fc258733640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc23c0b6700 0x7fc23c0b8c10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0e5290 tx=0x7fc24c0eb5d0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.302+0000 7fc258733640 1 -- 172.21.15.3:0/92724382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2501029e0 msgr2=0x7fc25019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.302+0000 7fc258733640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2501029e0 0x7fc25019a460 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00e9f0 tx=0x7fc24c00edf0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.302+0000 7fc258733640 1 -- 172.21.15.3:0/92724382 shutdown_connections 2024-09-21T06:23:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.302+0000 7fc258733640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc23c0b6700 0x7fc23c0b8c10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.302+0000 7fc258733640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc250108c60 0x7fc25019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.302+0000 7fc258733640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc250108310 0x7fc25019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.302+0000 7fc258733640 1 --2- 172.21.15.3:0/92724382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2501029e0 0x7fc25019a460 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.302+0000 7fc258733640 1 -- 172.21.15.3:0/92724382 >> 172.21.15.3:0/92724382 conn(0x7fc2500fe3c0 msgr2=0x7fc250101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.303+0000 7fc258733640 1 -- 172.21.15.3:0/92724382 shutdown_connections 2024-09-21T06:23:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:43.303+0000 7fc258733640 1 -- 172.21.15.3:0/92724382 wait complete. 2024-09-21T06:23:43.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:43 smithi003 ceph-mon[31227]: pgmap v592: 97 pgs: 97 active+clean; 133 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 258 op/s 2024-09-21T06:23:43.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:43 smithi137 ceph-mon[34464]: pgmap v592: 97 pgs: 97 active+clean; 133 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 258 op/s 2024-09-21T06:23:43.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:43 smithi204 ceph-mon[34659]: pgmap v592: 97 pgs: 97 active+clean; 133 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 258 op/s 2024-09-21T06:23:43.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 905870 10.48 MB/sec execute 316 sec latency 107.297 ms 2024-09-21T06:23:44.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 908184 10.48 MB/sec execute 317 sec latency 18.044 ms 2024-09-21T06:23:45.696 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:45 smithi137 ceph-mon[34464]: pgmap v593: 97 pgs: 97 active+clean; 140 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 275 op/s 2024-09-21T06:23:45.706 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:45 smithi204 ceph-mon[34659]: pgmap v593: 97 pgs: 97 active+clean; 140 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 275 op/s 2024-09-21T06:23:45.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:45 smithi003 ceph-mon[31227]: pgmap v593: 97 pgs: 97 active+clean; 140 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 275 op/s 2024-09-21T06:23:45.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 909266 10.47 MB/sec execute 318 sec latency 20.942 ms 2024-09-21T06:23:46.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:46 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:23:46.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:46 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:23:46.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:46 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:23:46.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 911274 10.46 MB/sec execute 319 sec latency 22.655 ms 2024-09-21T06:23:47.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:47 smithi003 ceph-mon[31227]: pgmap v594: 97 pgs: 97 active+clean; 137 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 333 op/s 2024-09-21T06:23:47.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:47 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:23:47.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:47 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:23:47.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:47 smithi137 ceph-mon[34464]: pgmap v594: 97 pgs: 97 active+clean; 137 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 333 op/s 2024-09-21T06:23:47.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:47 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:23:47.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:47 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:23:47.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:47 smithi204 ceph-mon[34659]: pgmap v594: 97 pgs: 97 active+clean; 137 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 333 op/s 2024-09-21T06:23:47.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:47 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:23:47.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:47 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:23:47.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 912772 10.46 MB/sec execute 320 sec latency 16.486 ms 2024-09-21T06:23:48.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 914298 10.45 MB/sec execute 321 sec latency 31.536 ms 2024-09-21T06:23:49.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:49 smithi003 ceph-mon[31227]: pgmap v595: 97 pgs: 97 active+clean; 130 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 281 op/s 2024-09-21T06:23:49.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:49 smithi137 ceph-mon[34464]: pgmap v595: 97 pgs: 97 active+clean; 130 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 281 op/s 2024-09-21T06:23:49.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:49 smithi204 ceph-mon[34659]: pgmap v595: 97 pgs: 97 active+clean; 130 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 281 op/s 2024-09-21T06:23:49.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 916138 10.46 MB/sec execute 322 sec latency 20.555 ms 2024-09-21T06:23:50.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 916943 10.46 MB/sec execute 323 sec latency 76.656 ms 2024-09-21T06:23:51.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:51 smithi003 ceph-mon[31227]: pgmap v596: 97 pgs: 97 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 259 op/s 2024-09-21T06:23:51.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:51 smithi137 ceph-mon[34464]: pgmap v596: 97 pgs: 97 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 259 op/s 2024-09-21T06:23:51.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:51 smithi204 ceph-mon[34659]: pgmap v596: 97 pgs: 97 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 259 op/s 2024-09-21T06:23:51.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 918388 10.48 MB/sec execute 324 sec latency 30.373 ms 2024-09-21T06:23:52.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 920608 10.49 MB/sec execute 325 sec latency 14.925 ms 2024-09-21T06:23:53.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:53 smithi003 ceph-mon[31227]: pgmap v597: 97 pgs: 97 active+clean; 137 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-09-21T06:23:53.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:53 smithi137 ceph-mon[34464]: pgmap v597: 97 pgs: 97 active+clean; 137 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-09-21T06:23:53.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:53 smithi204 ceph-mon[34659]: pgmap v597: 97 pgs: 97 active+clean; 137 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-09-21T06:23:53.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 921942 10.47 MB/sec execute 326 sec latency 82.664 ms 2024-09-21T06:23:54.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 924129 10.48 MB/sec execute 327 sec latency 16.322 ms 2024-09-21T06:23:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:55 smithi003 ceph-mon[31227]: pgmap v598: 97 pgs: 97 active+clean; 138 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 266 op/s 2024-09-21T06:23:55.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:55 smithi137 ceph-mon[34464]: pgmap v598: 97 pgs: 97 active+clean; 138 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 266 op/s 2024-09-21T06:23:55.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:55 smithi204 ceph-mon[34659]: pgmap v598: 97 pgs: 97 active+clean; 138 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 266 op/s 2024-09-21T06:23:55.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 926051 10.47 MB/sec execute 328 sec latency 17.156 ms 2024-09-21T06:23:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.405+0000 7f094de25640 1 -- 172.21.15.3:0/1937052742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481061d0 msgr2=0x7f09481065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.405+0000 7f094de25640 1 --2- 172.21.15.3:0/1937052742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481061d0 0x7f09481065e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f093c009900 tx=0x7f093c02a410 comp rx=0 tx=0).stop 2024-09-21T06:23:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.405+0000 7f094de25640 1 -- 172.21.15.3:0/1937052742 shutdown_connections 2024-09-21T06:23:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.405+0000 7f094de25640 1 --2- 172.21.15.3:0/1937052742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0948106b20 0x7f0948110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.405+0000 7f094de25640 1 --2- 172.21.15.3:0/1937052742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481061d0 0x7f09481065e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.405+0000 7f094de25640 1 --2- 172.21.15.3:0/1937052742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0948102060 0x7f0948102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.405+0000 7f094de25640 1 -- 172.21.15.3:0/1937052742 >> 172.21.15.3:0/1937052742 conn(0x7f0948079e90 msgr2=0x7f094807a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.405+0000 7f094de25640 1 -- 172.21.15.3:0/1937052742 shutdown_connections 2024-09-21T06:23:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.405+0000 7f094de25640 1 -- 172.21.15.3:0/1937052742 wait complete. 2024-09-21T06:23:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.405+0000 7f094de25640 1 Processor -- start 2024-09-21T06:23:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f094de25640 1 -- start start 2024-09-21T06:23:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f094de25640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0948102060 0x7f0948102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f094de25640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481061d0 0x7f0948103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f094de25640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0948106b20 0x7f0948103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f094de25640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0948113a20 con 0x7f09481061d0 2024-09-21T06:23:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f094de25640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09481138a0 con 0x7f0948102060 2024-09-21T06:23:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f094de25640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0948113ba0 con 0x7f0948106b20 2024-09-21T06:23:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f0946ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481061d0 0x7f0948103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f09477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0948102060 0x7f0948102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f0946ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481061d0 0x7f0948103530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39628/0 (socket says 172.21.15.3:39628) 2024-09-21T06:23:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f0946ffd640 1 -- 172.21.15.3:0/605809942 learned_addr learned my addr 172.21.15.3:0/605809942 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f0947fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0948106b20 0x7f0948103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f09477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0948102060 0x7f0948102ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:36790/0 (socket says 172.21.15.3:36790) 2024-09-21T06:23:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f0946ffd640 1 -- 172.21.15.3:0/605809942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0948106b20 msgr2=0x7f0948103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f0946ffd640 1 --2- 172.21.15.3:0/605809942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0948106b20 0x7f0948103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f0946ffd640 1 -- 172.21.15.3:0/605809942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0948102060 msgr2=0x7f0948102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f0946ffd640 1 --2- 172.21.15.3:0/605809942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0948102060 0x7f0948102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f0946ffd640 1 -- 172.21.15.3:0/605809942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09481a86d0 con 0x7f09481061d0 2024-09-21T06:23:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.406+0000 7f09477fe640 1 --2- 172.21.15.3:0/605809942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0948102060 0x7f0948102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.407+0000 7f0946ffd640 1 --2- 172.21.15.3:0/605809942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481061d0 0x7f0948103530 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f093c02a930 tx=0x7f093c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.407+0000 7f0944ff9640 1 -- 172.21.15.3:0/605809942 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c03f030 con 0x7f09481061d0 2024-09-21T06:23:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.407+0000 7f094de25640 1 -- 172.21.15.3:0/605809942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09481a8960 con 0x7f09481061d0 2024-09-21T06:23:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.407+0000 7f0944ff9640 1 -- 172.21.15.3:0/605809942 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f093c0043d0 con 0x7f09481061d0 2024-09-21T06:23:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.407+0000 7f0944ff9640 1 -- 172.21.15.3:0/605809942 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c035070 con 0x7f09481061d0 2024-09-21T06:23:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.407+0000 7f094de25640 1 -- 172.21.15.3:0/605809942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09481a8e50 con 0x7f09481061d0 2024-09-21T06:23:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.410+0000 7f0954e71740 1 Processor -- start 2024-09-21T06:23:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.410+0000 7f0954e71740 1 -- start start 2024-09-21T06:23:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.410+0000 7f0954e71740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c6389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.410+0000 7f0954e71740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 0x55672c638ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.410+0000 7f0954e71740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2724d0 0x55672c639430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.410+0000 7f0954e71740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55672c5ad6f0 con 0x55672c2931b0 2024-09-21T06:23:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.410+0000 7f0954e71740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55672c59cb20 con 0x55672c26a2f0 2024-09-21T06:23:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.410+0000 7f0954e71740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55672c5a0930 con 0x55672c2724d0 2024-09-21T06:23:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f0947fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c6389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f09477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 0x55672c638ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f09477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 0x55672c638ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:36796/0 (socket says 172.21.15.3:36796) 2024-09-21T06:23:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f09477fe640 1 -- 172.21.15.3:0/3668728329 learned_addr learned my addr 172.21.15.3:0/3668728329 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f0947fff640 1 --2- 172.21.15.3:0/3668728329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c6389b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39644/0 (socket says 172.21.15.3:39644) 2024-09-21T06:23:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f0944ff9640 1 -- 172.21.15.3:0/605809942 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f093c042030 con 0x7f09481061d0 2024-09-21T06:23:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f09477fe640 1 -- 172.21.15.3:0/3668728329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2724d0 msgr2=0x55672c639430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f09477fe640 1 --2- 172.21.15.3:0/3668728329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2724d0 0x55672c639430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f09477fe640 1 -- 172.21.15.3:0/3668728329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 msgr2=0x55672c6389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f09477fe640 1 --2- 172.21.15.3:0/3668728329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c6389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f09477fe640 1 -- 172.21.15.3:0/3668728329 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55672c562530 con 0x55672c26a2f0 2024-09-21T06:23:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f0947fff640 1 --2- 172.21.15.3:0/3668728329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c6389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f09477fe640 1 --2- 172.21.15.3:0/3668728329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 0x55672c638ef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0938011520 tx=0x7f09380119e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7d3d24a27da04da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f092dffb640 1 -- 172.21.15.3:0/3668728329 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938005020 con 0x55672c26a2f0 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f092dffb640 1 -- 172.21.15.3:0/3668728329 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09380047a0 con 0x55672c26a2f0 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f092dffb640 1 -- 172.21.15.3:0/3668728329 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093800f860 con 0x55672c26a2f0 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.411+0000 7f0944ff9640 1 --2- 172.21.15.3:0/605809942 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09300b66b0 0x7f09300b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.412+0000 7f0944ff9640 1 -- 172.21.15.3:0/605809942 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f093c1079c0 con 0x7f09481061d0 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.412+0000 7f0947fff640 1 --2- 172.21.15.3:0/605809942 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09300b66b0 0x7f09300b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.412+0000 7f0947fff640 1 --2- 172.21.15.3:0/605809942 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09300b66b0 0x7f09300b8bc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0940005fd0 tx=0x7f0940009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.412+0000 7f0954e71740 1 -- 172.21.15.3:0/3668728329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 msgr2=0x55672c638ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.412+0000 7f0954e71740 1 --2- 172.21.15.3:0/3668728329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 0x55672c638ef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0938011520 tx=0x7f09380119e0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.412+0000 7f0954e71740 1 -- 172.21.15.3:0/3668728329 shutdown_connections 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.412+0000 7f0954e71740 1 --2- 172.21.15.3:0/3668728329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2724d0 0x55672c639430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.412+0000 7f0954e71740 1 --2- 172.21.15.3:0/3668728329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 0x55672c638ef0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.412+0000 7f0954e71740 1 --2- 172.21.15.3:0/3668728329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c6389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.412+0000 7f0954e71740 1 -- 172.21.15.3:0/3668728329 >> 172.21.15.3:0/3668728329 conn(0x55672c287300 msgr2=0x55672c6368a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.412+0000 7f0954e71740 1 -- 172.21.15.3:0/3668728329 shutdown_connections 2024-09-21T06:23:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.412+0000 7f0954e71740 1 -- 172.21.15.3:0/3668728329 wait complete. 2024-09-21T06:23:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.413+0000 7f0954e71740 1 -- start start 2024-09-21T06:23:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.413+0000 7f0954e71740 1 Processor -- start 2024-09-21T06:23:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.413+0000 7f0954e71740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 0x55672c644130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.413+0000 7f0954e71740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2724d0 0x55672c644670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.413+0000 7f0954e71740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c650420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.413+0000 7f0954e71740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55672c4d4ab0 con 0x55672c2931b0 2024-09-21T06:23:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.413+0000 7f0954e71740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55672c542730 con 0x55672c26a2f0 2024-09-21T06:23:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.413+0000 7f0954e71740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55672c647130 con 0x55672c2724d0 2024-09-21T06:23:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.413+0000 7f0954e71740 10 client.0 _ll_register_callbacks cb 0x55672c5966b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0947fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c650420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f09477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 0x55672c644130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0946ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2724d0 0x55672c644670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0947fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c650420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39648/0 (socket says 172.21.15.3:39648) 2024-09-21T06:23:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f09477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 0x55672c644130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:36800/0 (socket says 172.21.15.3:36800) 2024-09-21T06:23:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0947fff640 1 -- 172.21.15.3:0/1077212929 learned_addr learned my addr 172.21.15.3:0/1077212929 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0947fff640 1 -- 172.21.15.3:0/1077212929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2724d0 msgr2=0x55672c644670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0947fff640 1 --2- 172.21.15.3:0/1077212929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2724d0 0x55672c644670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0947fff640 1 -- 172.21.15.3:0/1077212929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 msgr2=0x55672c644130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0947fff640 1 --2- 172.21.15.3:0/1077212929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 0x55672c644130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0947fff640 1 -- 172.21.15.3:0/1077212929 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55672c2fe790 con 0x55672c2931b0 2024-09-21T06:23:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0946ffd640 1 --2- 172.21.15.3:0/1077212929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2724d0 0x55672c644670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f09477fe640 1 --2- 172.21.15.3:0/1077212929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 0x55672c644130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0947fff640 1 --2- 172.21.15.3:0/1077212929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c650420 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f094000f6c0 tx=0x7f0940004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eced660977da2069 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0926ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:23:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0926ffd640 1 -- 172.21.15.3:0/1077212929 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0940003b80 con 0x55672c2931b0 2024-09-21T06:23:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0926ffd640 1 -- 172.21.15.3:0/1077212929 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f0940003d20 con 0x55672c2931b0 2024-09-21T06:23:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.414+0000 7f0954e71740 1 -- 172.21.15.3:0/1077212929 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09380047a0 con 0x55672c2931b0 2024-09-21T06:23:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.415+0000 7f0926ffd640 1 -- 172.21.15.3:0/1077212929 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0940004050 con 0x55672c2931b0 2024-09-21T06:23:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.415+0000 7f0954e71740 10 client.18048 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.415+0000 7f0954e71740 1 -- 172.21.15.3:0/1077212929 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55672c377620 con 0x55672c2931b0 2024-09-21T06:23:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.415+0000 7f0926ffd640 1 -- 172.21.15.3:0/1077212929 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09380047a0 con 0x55672c2931b0 2024-09-21T06:23:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.415+0000 7f0926ffd640 1 -- 172.21.15.3:0/1077212929 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f0940027020 con 0x55672c2931b0 2024-09-21T06:23:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.415+0000 7f0954e71740 10 client.18048 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.415+0000 7f0954e71740 10 client.18048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.415+0000 7f0954e71740 20 client.18048 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.415+0000 7f0954e71740 1 --2- 172.21.15.3:0/1077212929 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55672c251d80 0x55672c6562a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.415+0000 7f0954e71740 4 client.18048 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.415+0000 7f0954e71740 1 -- 172.21.15.3:0/1077212929 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55672c5af640 con 0x55672c251d80 2024-09-21T06:23:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.415+0000 7f09477fe640 1 --2- 172.21.15.3:0/1077212929 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55672c251d80 0x55672c6562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.416+0000 7f09477fe640 1 --2- 172.21.15.3:0/1077212929 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55672c251d80 0x55672c6562a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68e2533ca205b8d1 server_cookie=845599888e518e3e in_seq=0 out_seq=0 2024-09-21T06:23:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.416+0000 7f0926ffd640 10 client.18048 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.417+0000 7f0926ffd640 1 -- 172.21.15.3:0/1077212929 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55672c5af640 con 0x55672c251d80 2024-09-21T06:23:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.417+0000 7f0926ffd640 10 client.18048 handle_command_reply: tid=0 2024-09-21T06:23:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.417+0000 7f0954e71740 1 client.18048 shutdown 2024-09-21T06:23:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.418+0000 7f0954e71740 1 -- 172.21.15.3:0/1077212929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 msgr2=0x55672c650420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.418+0000 7f0954e71740 1 --2- 172.21.15.3:0/1077212929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c650420 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f094000f6c0 tx=0x7f0940004ab0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.418+0000 7f0954e71740 1 -- 172.21.15.3:0/1077212929 shutdown_connections 2024-09-21T06:23:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.418+0000 7f0954e71740 1 --2- 172.21.15.3:0/1077212929 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55672c251d80 0x55672c6562a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.418+0000 7f0954e71740 1 --2- 172.21.15.3:0/1077212929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2724d0 0x55672c644670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.418+0000 7f0954e71740 1 --2- 172.21.15.3:0/1077212929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c650420 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.418+0000 7f0926ffd640 0 client.18048 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.418+0000 7f0954e71740 1 --2- 172.21.15.3:0/1077212929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c26a2f0 0x55672c644130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.418+0000 7f0954e71740 1 -- 172.21.15.3:0/1077212929 >> 172.21.15.3:0/1077212929 conn(0x55672c287300 msgr2=0x55672c6399e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.418+0000 7f0954e71740 1 -- 172.21.15.3:0/1077212929 shutdown_connections 2024-09-21T06:23:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.418+0000 7f0954e71740 1 -- 172.21.15.3:0/1077212929 wait complete. 2024-09-21T06:23:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.418+0000 7f0954e71740 20 client.18048 trim_cache size 0 max 16384 2024-09-21T06:23:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0954e71740 1 Processor -- start 2024-09-21T06:23:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0954e71740 1 -- start start 2024-09-21T06:23:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0954e71740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c26a2f0 0x55672c63a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0954e71740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c2724d0 0x55672c644610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0954e71740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c645d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0954e71740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f09481b18a0 con 0x55672c2931b0 2024-09-21T06:23:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0954e71740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09481b33e0 con 0x55672c2724d0 2024-09-21T06:23:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0954e71740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f09481b1f70 con 0x55672c26a2f0 2024-09-21T06:23:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0947fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c645d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f09477fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c26a2f0 0x55672c63a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0947fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c645d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39658/0 (socket says 172.21.15.3:39658) 2024-09-21T06:23:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0947fff640 1 -- 172.21.15.3:0/3674829728 learned_addr learned my addr 172.21.15.3:0/3674829728 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0946ffd640 1 --2- 172.21.15.3:0/3674829728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c2724d0 0x55672c644610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0947fff640 1 -- 172.21.15.3:0/3674829728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c26a2f0 msgr2=0x55672c63a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0947fff640 1 --2- 172.21.15.3:0/3674829728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c26a2f0 0x55672c63a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0947fff640 1 -- 172.21.15.3:0/3674829728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c2724d0 msgr2=0x55672c644610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0947fff640 1 --2- 172.21.15.3:0/3674829728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c2724d0 0x55672c644610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f0947fff640 1 -- 172.21.15.3:0/3674829728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55672c63aee0 con 0x55672c2931b0 2024-09-21T06:23:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.433+0000 7f09477fe640 1 --2- 172.21.15.3:0/3674829728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c26a2f0 0x55672c63a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.434+0000 7f0947fff640 1 --2- 172.21.15.3:0/3674829728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c645d60 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f0940011cb0 tx=0x7f0940012410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9712c53e465e6613 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.434+0000 7f092d7fa640 1 -- 172.21.15.3:0/3674829728 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0940036070 con 0x55672c2931b0 2024-09-21T06:23:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.434+0000 7f092d7fa640 1 -- 172.21.15.3:0/3674829728 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0940003d80 con 0x55672c2931b0 2024-09-21T06:23:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.434+0000 7f092d7fa640 1 -- 172.21.15.3:0/3674829728 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094001f4a0 con 0x55672c2931b0 2024-09-21T06:23:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.435+0000 7f0954e71740 1 -- 172.21.15.3:0/3674829728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 msgr2=0x55672c645d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.435+0000 7f0954e71740 1 --2- 172.21.15.3:0/3674829728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c645d60 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f0940011cb0 tx=0x7f0940012410 comp rx=0 tx=0).stop 2024-09-21T06:23:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.435+0000 7f0954e71740 1 -- 172.21.15.3:0/3674829728 shutdown_connections 2024-09-21T06:23:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.435+0000 7f0954e71740 1 --2- 172.21.15.3:0/3674829728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c2931b0 0x55672c645d60 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.435+0000 7f0954e71740 1 --2- 172.21.15.3:0/3674829728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c2724d0 0x55672c644610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.435+0000 7f0954e71740 1 --2- 172.21.15.3:0/3674829728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c26a2f0 0x55672c63a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.435+0000 7f0954e71740 1 -- 172.21.15.3:0/3674829728 >> 172.21.15.3:0/3674829728 conn(0x55672c287300 msgr2=0x55672c645650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.435+0000 7f0954e71740 1 -- 172.21.15.3:0/3674829728 shutdown_connections 2024-09-21T06:23:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.435+0000 7f0954e71740 1 -- 172.21.15.3:0/3674829728 wait complete. 2024-09-21T06:23:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.435+0000 7f0954e71740 1 -- start start 2024-09-21T06:23:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.435+0000 7f0954e71740 1 Processor -- start 2024-09-21T06:23:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.435+0000 7f0954e71740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c26a2f0 0x55672c647b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.435+0000 7f0954e71740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c2724d0 0x55672c6480c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0954e71740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2931b0 0x55672c6beca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0954e71740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f09481b2d10 con 0x55672c26a2f0 2024-09-21T06:23:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0954e71740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09481b2640 con 0x55672c2724d0 2024-09-21T06:23:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0954e71740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f09481af8a0 con 0x55672c2931b0 2024-09-21T06:23:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0954e71740 10 client.0 _ll_register_callbacks cb 0x7f0940009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f09477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c26a2f0 0x55672c647b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0946ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c2724d0 0x55672c6480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f09477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c26a2f0 0x55672c647b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39674/0 (socket says 172.21.15.3:39674) 2024-09-21T06:23:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f09477fe640 1 -- 172.21.15.3:0/3325102552 learned_addr learned my addr 172.21.15.3:0/3325102552 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0947fff640 1 --2- 172.21.15.3:0/3325102552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2931b0 0x55672c6beca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0946ffd640 1 -- 172.21.15.3:0/3325102552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2931b0 msgr2=0x55672c6beca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0946ffd640 1 --2- 172.21.15.3:0/3325102552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2931b0 0x55672c6beca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0946ffd640 1 -- 172.21.15.3:0/3325102552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c26a2f0 msgr2=0x55672c647b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0946ffd640 1 --2- 172.21.15.3:0/3325102552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c26a2f0 0x55672c647b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0946ffd640 1 -- 172.21.15.3:0/3325102552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55672c6bf260 con 0x55672c2724d0 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f09477fe640 1 --2- 172.21.15.3:0/3325102552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c26a2f0 0x55672c647b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0946ffd640 1 --2- 172.21.15.3:0/3325102552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c2724d0 0x55672c6480c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f093c105120 tx=0x7f093c003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c20eb78fa035a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0926ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0926ffd640 1 -- 172.21.15.3:0/3325102552 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c049040 con 0x55672c2724d0 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0926ffd640 1 -- 172.21.15.3:0/3325102552 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8507+0+0 (secure 0 0 0) 0x7f093c0cca80 con 0x55672c2724d0 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.436+0000 7f0954e71740 1 -- 172.21.15.3:0/3325102552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0940003d80 con 0x55672c2724d0 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.437+0000 7f0926ffd640 1 -- 172.21.15.3:0/3325102552 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f093c0cd170 con 0x55672c2724d0 2024-09-21T06:23:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.437+0000 7f0954e71740 10 client.27614 fetch_fsmap learned FSMap version 37 2024-09-21T06:23:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.437+0000 7f0954e71740 1 -- 172.21.15.3:0/3325102552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55672c6c0550 con 0x55672c2724d0 2024-09-21T06:23:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.437+0000 7f0926ffd640 1 -- 172.21.15.3:0/3325102552 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0940003d80 con 0x55672c2724d0 2024-09-21T06:23:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.437+0000 7f0926ffd640 1 -- 172.21.15.3:0/3325102552 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f093c031ac0 con 0x55672c2724d0 2024-09-21T06:23:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.437+0000 7f0954e71740 10 client.27614 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:23:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.437+0000 7f0954e71740 10 client.27614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.437+0000 7f0954e71740 20 client.27614 populate_metadata read hostname 'smithi003' 2024-09-21T06:23:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.437+0000 7f0954e71740 1 --2- 172.21.15.3:0/3325102552 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55672c2271c0 0x55672c6c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.437+0000 7f0954e71740 4 client.27614 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:23:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.437+0000 7f0954e71740 1 -- 172.21.15.3:0/3325102552 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55672c5ab0f0 con 0x55672c2271c0 2024-09-21T06:23:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.437+0000 7f09477fe640 1 --2- 172.21.15.3:0/3325102552 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55672c2271c0 0x55672c6c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.438+0000 7f0926ffd640 10 client.27614 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.438+0000 7f09477fe640 1 --2- 172.21.15.3:0/3325102552 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55672c2271c0 0x55672c6c2d60 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f690aec84f46ee82 server_cookie=96cb4fe620aa30b4 in_seq=0 out_seq=0 2024-09-21T06:23:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.596+0000 7f0926ffd640 1 -- 172.21.15.3:0/3325102552 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55672c5ab0f0 con 0x55672c2271c0 2024-09-21T06:23:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.596+0000 7f0926ffd640 10 client.27614 handle_command_reply: tid=0 2024-09-21T06:23:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.596+0000 7f0954e71740 1 client.27614 shutdown 2024-09-21T06:23:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.596+0000 7f0954e71740 1 -- 172.21.15.3:0/3325102552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c2724d0 msgr2=0x55672c6480c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.596+0000 7f0954e71740 1 --2- 172.21.15.3:0/3325102552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c2724d0 0x55672c6480c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f093c105120 tx=0x7f093c003eb0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.596+0000 7f0954e71740 1 -- 172.21.15.3:0/3325102552 shutdown_connections 2024-09-21T06:23:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.596+0000 7f0954e71740 1 --2- 172.21.15.3:0/3325102552 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55672c2271c0 0x55672c6c2d60 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.596+0000 7f0954e71740 1 --2- 172.21.15.3:0/3325102552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55672c2931b0 0x55672c6beca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.596+0000 7f0954e71740 1 --2- 172.21.15.3:0/3325102552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55672c2724d0 0x55672c6480c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.596+0000 7f0954e71740 1 --2- 172.21.15.3:0/3325102552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55672c26a2f0 0x55672c647b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.596+0000 7f0926ffd640 0 client.27614 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:23:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.596+0000 7f0954e71740 1 -- 172.21.15.3:0/3325102552 >> 172.21.15.3:0/3325102552 conn(0x55672c287300 msgr2=0x55672c63a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.597+0000 7f0954e71740 1 -- 172.21.15.3:0/3325102552 shutdown_connections 2024-09-21T06:23:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.597+0000 7f0954e71740 1 -- 172.21.15.3:0/3325102552 wait complete. 2024-09-21T06:23:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.597+0000 7f0954e71740 20 client.27614 trim_cache size 0 max 16384 2024-09-21T06:23:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-21T06:23:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-21T06:23:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-21T06:23:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-21T06:23:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.597+0000 7f094de25640 1 -- 172.21.15.3:0/605809942 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09300b66b0 msgr2=0x7f09300b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.597+0000 7f094de25640 1 --2- 172.21.15.3:0/605809942 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09300b66b0 0x7f09300b8bc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0940005fd0 tx=0x7f0940009500 comp rx=0 tx=0).stop 2024-09-21T06:23:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.597+0000 7f094de25640 1 -- 172.21.15.3:0/605809942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481061d0 msgr2=0x7f0948103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.597+0000 7f094de25640 1 --2- 172.21.15.3:0/605809942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481061d0 0x7f0948103530 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f093c02a930 tx=0x7f093c031a20 comp rx=0 tx=0).stop 2024-09-21T06:23:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.597+0000 7f094de25640 1 -- 172.21.15.3:0/605809942 shutdown_connections 2024-09-21T06:23:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.597+0000 7f094de25640 1 --2- 172.21.15.3:0/605809942 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09300b66b0 0x7f09300b8bc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.597+0000 7f094de25640 1 --2- 172.21.15.3:0/605809942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0948106b20 0x7f0948103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.597+0000 7f094de25640 1 --2- 172.21.15.3:0/605809942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09481061d0 0x7f0948103530 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.597+0000 7f094de25640 1 --2- 172.21.15.3:0/605809942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0948102060 0x7f0948102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.597+0000 7f094de25640 1 -- 172.21.15.3:0/605809942 >> 172.21.15.3:0/605809942 conn(0x7f0948079e90 msgr2=0x7f0948107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.598+0000 7f094de25640 1 -- 172.21.15.3:0/605809942 shutdown_connections 2024-09-21T06:23:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-21T06:23:56.598+0000 7f094de25640 1 -- 172.21.15.3:0/605809942 wait complete. 2024-09-21T06:23:56.970 INFO:tasks.workunit.client.0.smithi003.stdout: 1 927698 10.47 MB/sec execute 329 sec latency 85.521 ms 2024-09-21T06:23:57.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:57 smithi003 ceph-mon[31227]: pgmap v599: 97 pgs: 97 active+clean; 141 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 260 op/s 2024-09-21T06:23:57.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:57 smithi003 ceph-mon[31227]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:23:57.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:57 smithi137 ceph-mon[34464]: pgmap v599: 97 pgs: 97 active+clean; 141 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 260 op/s 2024-09-21T06:23:57.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:57 smithi137 ceph-mon[34464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:23:57.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:57 smithi204 ceph-mon[34659]: pgmap v599: 97 pgs: 97 active+clean; 141 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 260 op/s 2024-09-21T06:23:57.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:57 smithi204 ceph-mon[34659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:23:57.971 INFO:tasks.workunit.client.0.smithi003.stdout: 1 929695 10.46 MB/sec execute 330 sec latency 14.935 ms 2024-09-21T06:23:58.971 INFO:tasks.workunit.client.0.smithi003.stdout: 1 931254 10.47 MB/sec execute 331 sec latency 59.297 ms 2024-09-21T06:23:59.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:23:59 smithi003 ceph-mon[31227]: pgmap v600: 97 pgs: 97 active+clean; 142 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 209 op/s 2024-09-21T06:23:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:23:59 smithi137 ceph-mon[34464]: pgmap v600: 97 pgs: 97 active+clean; 142 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 209 op/s 2024-09-21T06:23:59.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:23:59 smithi204 ceph-mon[34659]: pgmap v600: 97 pgs: 97 active+clean; 142 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 209 op/s 2024-09-21T06:23:59.971 INFO:tasks.workunit.client.0.smithi003.stdout: 1 932974 10.45 MB/sec execute 332 sec latency 16.900 ms 2024-09-21T06:24:00.971 INFO:tasks.workunit.client.0.smithi003.stdout: 1 934624 10.46 MB/sec execute 333 sec latency 28.744 ms 2024-09-21T06:24:01.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:01 smithi003 ceph-mon[31227]: pgmap v601: 97 pgs: 97 active+clean; 138 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 294 op/s 2024-09-21T06:24:01.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:01 smithi137 ceph-mon[34464]: pgmap v601: 97 pgs: 97 active+clean; 138 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 294 op/s 2024-09-21T06:24:01.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:01 smithi204 ceph-mon[34659]: pgmap v601: 97 pgs: 97 active+clean; 138 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 294 op/s 2024-09-21T06:24:01.971 INFO:tasks.workunit.client.0.smithi003.stdout: 1 935958 10.45 MB/sec execute 334 sec latency 25.590 ms 2024-09-21T06:24:02.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:24:02.931 INFO:tasks.workunit.client.0.smithi003.stdout:[938167] write failed on handle 10128 (Resource temporarily unavailable) 2024-09-21T06:24:02.931 INFO:tasks.workunit.client.0.smithi003.stdout:Child failed with status 1 2024-09-21T06:24:02.932 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-21T06:24:02.932 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T06:24:03.265 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi003 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-21T06:24:03.453 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a3e885ac48dd4740a77456faadbfa1b6 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi003 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-21T06:24:03.455 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-21T06:24:03.465 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi003.front.sepia.ceph.com 2024-09-21T06:24:03.476 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-21T06:24:03.725 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-21T06:24:03.725 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T06:24:03.725 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-09-21T06:24:03.726 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-21T06:24:03.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:03 smithi003 ceph-mon[31227]: pgmap v602: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 333 op/s 2024-09-21T06:24:03.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-21T06:24:03.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:03 smithi137 ceph-mon[34464]: pgmap v602: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 333 op/s 2024-09-21T06:24:03.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:03 smithi204 ceph-mon[34659]: pgmap v602: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 333 op/s 2024-09-21T06:24:05.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:05 smithi003 ceph-mon[31227]: pgmap v603: 97 pgs: 97 active+clean; 134 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 295 op/s 2024-09-21T06:24:05.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:05 smithi137 ceph-mon[34464]: pgmap v603: 97 pgs: 97 active+clean; 134 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 295 op/s 2024-09-21T06:24:05.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:05 smithi204 ceph-mon[34659]: pgmap v603: 97 pgs: 97 active+clean; 134 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 295 op/s 2024-09-21T06:24:07.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:07 smithi003 ceph-mon[31227]: pgmap v604: 97 pgs: 97 active+clean; 139 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 318 op/s 2024-09-21T06:24:07.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:07 smithi003 ceph-mon[31227]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:24:07.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:07 smithi137 ceph-mon[34464]: pgmap v604: 97 pgs: 97 active+clean; 139 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 318 op/s 2024-09-21T06:24:07.898 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:07 smithi137 ceph-mon[34464]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:24:07.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:07 smithi204 ceph-mon[34659]: pgmap v604: 97 pgs: 97 active+clean; 139 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 318 op/s 2024-09-21T06:24:07.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:07 smithi204 ceph-mon[34659]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:24:09.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:09 smithi003 ceph-mon[31227]: pgmap v606: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 142 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 313 op/s 2024-09-21T06:24:09.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:09 smithi137 ceph-mon[34464]: pgmap v606: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 142 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 313 op/s 2024-09-21T06:24:09.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:09 smithi204 ceph-mon[34659]: pgmap v606: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 142 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 313 op/s 2024-09-21T06:24:10.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:24:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:11.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:11 smithi003 ceph-mon[31227]: pgmap v607: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 141 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-21T06:24:11.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.422+0000 7f6273e5f640 1 -- 172.21.15.3:0/2437931212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626c109df0 msgr2=0x7f626c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:11.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.422+0000 7f6273e5f640 1 --2- 172.21.15.3:0/2437931212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626c109df0 0x7f626c110620 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6264009ca0 tx=0x7f626402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.422+0000 7f6273e5f640 1 -- 172.21.15.3:0/2437931212 shutdown_connections 2024-09-21T06:24:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.422+0000 7f6273e5f640 1 --2- 172.21.15.3:0/2437931212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626c109df0 0x7f626c110620 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.422+0000 7f6273e5f640 1 --2- 172.21.15.3:0/2437931212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f626c1094a0 0x7f626c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.422+0000 7f6273e5f640 1 --2- 172.21.15.3:0/2437931212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f626c103b60 0x7f626c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.422+0000 7f6273e5f640 1 -- 172.21.15.3:0/2437931212 >> 172.21.15.3:0/2437931212 conn(0x7f626c07b850 msgr2=0x7f626c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.422+0000 7f6273e5f640 1 -- 172.21.15.3:0/2437931212 shutdown_connections 2024-09-21T06:24:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.422+0000 7f6273e5f640 1 -- 172.21.15.3:0/2437931212 wait complete. 2024-09-21T06:24:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.423+0000 7f6273e5f640 1 Processor -- start 2024-09-21T06:24:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.423+0000 7f6273e5f640 1 -- start start 2024-09-21T06:24:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.423+0000 7f6273e5f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f626c103b60 0x7f626c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.423+0000 7f6273e5f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f626c1094a0 0x7f626c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.423+0000 7f6273e5f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626c109df0 0x7f626c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.423+0000 7f6273e5f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f626c113ab0 con 0x7f626c109df0 2024-09-21T06:24:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.423+0000 7f6273e5f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f626c113930 con 0x7f626c103b60 2024-09-21T06:24:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.423+0000 7f6273e5f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f626c113c30 con 0x7f626c1094a0 2024-09-21T06:24:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.423+0000 7f62723fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626c109df0 0x7f626c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.423+0000 7f62723fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626c109df0 0x7f626c19fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57394/0 (socket says 172.21.15.3:57394) 2024-09-21T06:24:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.423+0000 7f62723fc640 1 -- 172.21.15.3:0/3529068557 learned_addr learned my addr 172.21.15.3:0/3529068557 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.424+0000 7f62713fa640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f626c1094a0 0x7f626c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.424+0000 7f6271bfb640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f626c103b60 0x7f626c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.424+0000 7f62723fc640 1 -- 172.21.15.3:0/3529068557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f626c1094a0 msgr2=0x7f626c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.424+0000 7f62723fc640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f626c1094a0 0x7f626c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.424+0000 7f62723fc640 1 -- 172.21.15.3:0/3529068557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f626c103b60 msgr2=0x7f626c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.424+0000 7f62723fc640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f626c103b60 0x7f626c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.424+0000 7f62723fc640 1 -- 172.21.15.3:0/3529068557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c1a03b0 con 0x7f626c109df0 2024-09-21T06:24:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.424+0000 7f62723fc640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626c109df0 0x7f626c19fc80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6264005cc0 tx=0x7f626402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.424+0000 7f6262ffd640 1 -- 172.21.15.3:0/3529068557 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264003bf0 con 0x7f626c109df0 2024-09-21T06:24:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.424+0000 7f6262ffd640 1 -- 172.21.15.3:0/3529068557 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6264003d90 con 0x7f626c109df0 2024-09-21T06:24:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.424+0000 7f6262ffd640 1 -- 172.21.15.3:0/3529068557 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626402fbc0 con 0x7f626c109df0 2024-09-21T06:24:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.424+0000 7f6273e5f640 1 -- 172.21.15.3:0/3529068557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626c1a0640 con 0x7f626c109df0 2024-09-21T06:24:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.424+0000 7f6273e5f640 1 -- 172.21.15.3:0/3529068557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626c1a6e00 con 0x7f626c109df0 2024-09-21T06:24:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.427+0000 7f627a3d3740 1 Processor -- start 2024-09-21T06:24:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.427+0000 7f6262ffd640 1 -- 172.21.15.3:0/3529068557 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f626402fd60 con 0x7f626c109df0 2024-09-21T06:24:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.428+0000 7f627a3d3740 1 -- start start 2024-09-21T06:24:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.428+0000 7f627a3d3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbdd2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.428+0000 7f627a3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba021b0 0x559ecbdd27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.428+0000 7f627a3d3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba0a390 0x559ecbdd2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.428+0000 7f627a3d3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559ecbd48e50 con 0x559ecba2b070 2024-09-21T06:24:11.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.428+0000 7f627a3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559ecbd38280 con 0x559ecba021b0 2024-09-21T06:24:11.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.428+0000 7f627a3d3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559ecbd3c090 con 0x559ecba0a390 2024-09-21T06:24:11.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.428+0000 7f62713fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbdd2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.428+0000 7f62713fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbdd2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57404/0 (socket says 172.21.15.3:57404) 2024-09-21T06:24:11.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.428+0000 7f62713fa640 1 -- 172.21.15.3:0/4236333424 learned_addr learned my addr 172.21.15.3:0/4236333424 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:11.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.428+0000 7f6271bfb640 1 --2- 172.21.15.3:0/4236333424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba021b0 0x559ecbdd27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.428+0000 7f62713fa640 1 --2- 172.21.15.3:0/4236333424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba0a390 0x559ecbdd2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.428+0000 7f6262ffd640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f62580b6700 0x7f62580b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.429+0000 7f62713fa640 1 -- 172.21.15.3:0/4236333424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba0a390 msgr2=0x559ecbdd2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.429+0000 7f62713fa640 1 --2- 172.21.15.3:0/4236333424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba0a390 0x559ecbdd2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.429+0000 7f62713fa640 1 -- 172.21.15.3:0/4236333424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba021b0 msgr2=0x559ecbdd27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.429+0000 7f62713fa640 1 --2- 172.21.15.3:0/4236333424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba021b0 0x559ecbdd27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.429+0000 7f62713fa640 1 -- 172.21.15.3:0/4236333424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ecba96650 con 0x559ecba2b070 2024-09-21T06:24:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.429+0000 7f62723fc640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f62580b6700 0x7f62580b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.429+0000 7f6262ffd640 1 -- 172.21.15.3:0/3529068557 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8551+0+0 (secure 0 0 0) 0x7f6264108be0 con 0x7f626c109df0 2024-09-21T06:24:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.429+0000 7f62713fa640 1 --2- 172.21.15.3:0/4236333424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbdd2270 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f625c012440 tx=0x7f625c00b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cf9fc684c86f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.429+0000 7f6257fff640 1 -- 172.21.15.3:0/4236333424 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c004030 con 0x559ecba2b070 2024-09-21T06:24:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.429+0000 7f6257fff640 1 -- 172.21.15.3:0/4236333424 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f625c0054d0 con 0x559ecba2b070 2024-09-21T06:24:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.429+0000 7f6257fff640 1 -- 172.21.15.3:0/4236333424 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c004ae0 con 0x559ecba2b070 2024-09-21T06:24:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.430+0000 7f627a3d3740 1 -- 172.21.15.3:0/4236333424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 msgr2=0x559ecbdd2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.430+0000 7f62723fc640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f62580b6700 0x7f62580b8c10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f62640ce9a0 tx=0x7f62640f2040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.430+0000 7f627a3d3740 1 --2- 172.21.15.3:0/4236333424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbdd2270 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f625c012440 tx=0x7f625c00b740 comp rx=0 tx=0).stop 2024-09-21T06:24:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.430+0000 7f627a3d3740 1 -- 172.21.15.3:0/4236333424 shutdown_connections 2024-09-21T06:24:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.430+0000 7f627a3d3740 1 --2- 172.21.15.3:0/4236333424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba0a390 0x559ecbdd2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.430+0000 7f627a3d3740 1 --2- 172.21.15.3:0/4236333424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba021b0 0x559ecbdd27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.430+0000 7f627a3d3740 1 --2- 172.21.15.3:0/4236333424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbdd2270 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.430+0000 7f627a3d3740 1 -- 172.21.15.3:0/4236333424 >> 172.21.15.3:0/4236333424 conn(0x559ecba1f460 msgr2=0x559ecbdd0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.430+0000 7f627a3d3740 1 -- 172.21.15.3:0/4236333424 shutdown_connections 2024-09-21T06:24:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.430+0000 7f627a3d3740 1 -- 172.21.15.3:0/4236333424 wait complete. 2024-09-21T06:24:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.431+0000 7f627a3d3740 1 -- start start 2024-09-21T06:24:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.431+0000 7f627a3d3740 1 Processor -- start 2024-09-21T06:24:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.431+0000 7f627a3d3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba021b0 0x559ecbddd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.431+0000 7f627a3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba0a390 0x559ecbddde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.431+0000 7f627a3d3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbdde340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.431+0000 7f627a3d3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559ecbb99700 con 0x559ecba2b070 2024-09-21T06:24:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.431+0000 7f627a3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559ecbcd3110 con 0x559ecba0a390 2024-09-21T06:24:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.431+0000 7f627a3d3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559ecbde0a50 con 0x559ecba021b0 2024-09-21T06:24:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.431+0000 7f627a3d3740 10 client.0 _ll_register_callbacks cb 0x559ecbcec990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.431+0000 7f62713fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbdde340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.431+0000 7f62713fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbdde340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57410/0 (socket says 172.21.15.3:57410) 2024-09-21T06:24:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.431+0000 7f62713fa640 1 -- 172.21.15.3:0/3602238799 learned_addr learned my addr 172.21.15.3:0/3602238799 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f6271bfb640 1 --2- 172.21.15.3:0/3602238799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba0a390 0x559ecbddde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f62713fa640 1 --2- 172.21.15.3:0/3602238799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba021b0 0x559ecbddd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f62713fa640 1 -- 172.21.15.3:0/3602238799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba021b0 msgr2=0x559ecbddd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f62713fa640 1 --2- 172.21.15.3:0/3602238799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba021b0 0x559ecbddd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f62713fa640 1 -- 172.21.15.3:0/3602238799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba0a390 msgr2=0x559ecbddde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f62713fa640 1 --2- 172.21.15.3:0/3602238799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba0a390 0x559ecbddde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f62713fa640 1 -- 172.21.15.3:0/3602238799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ecbd00c40 con 0x559ecba2b070 2024-09-21T06:24:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f62713fa640 1 --2- 172.21.15.3:0/3602238799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbdde340 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f625c004650 tx=0x7f625c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866a6e57513f91c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f62557fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f62557fa640 1 -- 172.21.15.3:0/3602238799 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c023060 con 0x559ecba2b070 2024-09-21T06:24:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f62557fa640 1 -- 172.21.15.3:0/3602238799 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8551+0+0 (secure 0 0 0) 0x7f625c004880 con 0x559ecba2b070 2024-09-21T06:24:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f627a3d3740 1 -- 172.21.15.3:0/3602238799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f625c0054d0 con 0x559ecba2b070 2024-09-21T06:24:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f62557fa640 1 -- 172.21.15.3:0/3602238799 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f625c004a80 con 0x559ecba2b070 2024-09-21T06:24:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f627a3d3740 10 client.18066 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.432+0000 7f627a3d3740 1 -- 172.21.15.3:0/3602238799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559ecbdec6d0 con 0x559ecba2b070 2024-09-21T06:24:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.433+0000 7f62557fa640 1 -- 172.21.15.3:0/3602238799 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f625c0054d0 con 0x559ecba2b070 2024-09-21T06:24:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.433+0000 7f62557fa640 1 -- 172.21.15.3:0/3602238799 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f625c01c020 con 0x559ecba2b070 2024-09-21T06:24:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.433+0000 7f627a3d3740 10 client.18066 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.433+0000 7f627a3d3740 10 client.18066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.433+0000 7f627a3d3740 20 client.18066 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.433+0000 7f627a3d3740 1 --2- 172.21.15.3:0/3602238799 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559ecb9e99f0 0x559ecbdefa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.433+0000 7f627a3d3740 4 client.18066 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.433+0000 7f627a3d3740 1 -- 172.21.15.3:0/3602238799 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ecbd4ada0 con 0x559ecb9e99f0 2024-09-21T06:24:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.433+0000 7f6271bfb640 1 --2- 172.21.15.3:0/3602238799 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559ecb9e99f0 0x559ecbdefa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.434+0000 7f6271bfb640 1 --2- 172.21.15.3:0/3602238799 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559ecb9e99f0 0x559ecbdefa30 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ebcea2dae71ace4 server_cookie=cb669385c9275637 in_seq=0 out_seq=0 2024-09-21T06:24:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.434+0000 7f62557fa640 10 client.18066 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f62557fa640 1 -- 172.21.15.3:0/3602238799 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559ecbd4ada0 con 0x559ecb9e99f0 2024-09-21T06:24:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f62557fa640 10 client.18066 handle_command_reply: tid=0 2024-09-21T06:24:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f627a3d3740 1 client.18066 shutdown 2024-09-21T06:24:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f627a3d3740 1 -- 172.21.15.3:0/3602238799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 msgr2=0x559ecbdde340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f627a3d3740 1 --2- 172.21.15.3:0/3602238799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbdde340 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f625c004650 tx=0x7f625c004530 comp rx=0 tx=0).stop 2024-09-21T06:24:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f627a3d3740 1 -- 172.21.15.3:0/3602238799 shutdown_connections 2024-09-21T06:24:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f627a3d3740 1 --2- 172.21.15.3:0/3602238799 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559ecb9e99f0 0x559ecbdefa30 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f627a3d3740 1 --2- 172.21.15.3:0/3602238799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbdde340 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f627a3d3740 1 --2- 172.21.15.3:0/3602238799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba0a390 0x559ecbddde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f627a3d3740 1 --2- 172.21.15.3:0/3602238799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba021b0 0x559ecbddd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f62557fa640 0 client.18066 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:11.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f627a3d3740 1 -- 172.21.15.3:0/3602238799 >> 172.21.15.3:0/3602238799 conn(0x559ecba1f460 msgr2=0x559ecbdd5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:11.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f627a3d3740 1 -- 172.21.15.3:0/3602238799 shutdown_connections 2024-09-21T06:24:11.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f627a3d3740 1 -- 172.21.15.3:0/3602238799 wait complete. 2024-09-21T06:24:11.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.435+0000 7f627a3d3740 20 client.18066 trim_cache size 0 max 16384 2024-09-21T06:24:11.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.453+0000 7f627a3d3740 1 Processor -- start 2024-09-21T06:24:11.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.453+0000 7f627a3d3740 1 -- start start 2024-09-21T06:24:11.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.453+0000 7f627a3d3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba021b0 0x559ecbddf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.453+0000 7f627a3d3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba0a390 0x559ecbde9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.453+0000 7f627a3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba2b070 0x559ecbde98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.453+0000 7f627a3d3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1af7b0 con 0x559ecba0a390 2024-09-21T06:24:11.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.453+0000 7f627a3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1b12f0 con 0x559ecba2b070 2024-09-21T06:24:11.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.453+0000 7f627a3d3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1afe80 con 0x559ecba021b0 2024-09-21T06:24:11.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.453+0000 7f6271bfb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba0a390 0x559ecbde9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.453+0000 7f6271bfb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba0a390 0x559ecbde9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57422/0 (socket says 172.21.15.3:57422) 2024-09-21T06:24:11.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.453+0000 7f6271bfb640 1 -- 172.21.15.3:0/2212307582 learned_addr learned my addr 172.21.15.3:0/2212307582 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:11.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.454+0000 7f62713fa640 1 --2- 172.21.15.3:0/2212307582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba021b0 0x559ecbddf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.454+0000 7f6271bfb640 1 -- 172.21.15.3:0/2212307582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba021b0 msgr2=0x559ecbddf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:11.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.454+0000 7f6271bfb640 1 --2- 172.21.15.3:0/2212307582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba021b0 0x559ecbddf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.454+0000 7f6271bfb640 1 -- 172.21.15.3:0/2212307582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba2b070 msgr2=0x559ecbde98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:11.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.454+0000 7f6271bfb640 1 --2- 172.21.15.3:0/2212307582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba2b070 0x559ecbde98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.454+0000 7f6271bfb640 1 -- 172.21.15.3:0/2212307582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ecbde08a0 con 0x559ecba0a390 2024-09-21T06:24:11.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.454+0000 7f62713fa640 1 --2- 172.21.15.3:0/2212307582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba021b0 0x559ecbddf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:24:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.454+0000 7f6271bfb640 1 --2- 172.21.15.3:0/2212307582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba0a390 0x559ecbde9370 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x559ecbdf00b0 tx=0x7f62680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f696d3a675ed3df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.454+0000 7f62577fe640 1 -- 172.21.15.3:0/2212307582 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268011440 con 0x559ecba0a390 2024-09-21T06:24:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.454+0000 7f62577fe640 1 -- 172.21.15.3:0/2212307582 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6268011a90 con 0x559ecba0a390 2024-09-21T06:24:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.454+0000 7f62577fe640 1 -- 172.21.15.3:0/2212307582 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626801c900 con 0x559ecba0a390 2024-09-21T06:24:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.455+0000 7f627a3d3740 1 -- 172.21.15.3:0/2212307582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba0a390 msgr2=0x559ecbde9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.455+0000 7f627a3d3740 1 --2- 172.21.15.3:0/2212307582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba0a390 0x559ecbde9370 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x559ecbdf00b0 tx=0x7f62680046c0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.455+0000 7f627a3d3740 1 -- 172.21.15.3:0/2212307582 shutdown_connections 2024-09-21T06:24:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.455+0000 7f627a3d3740 1 --2- 172.21.15.3:0/2212307582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba2b070 0x559ecbde98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.455+0000 7f627a3d3740 1 --2- 172.21.15.3:0/2212307582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba0a390 0x559ecbde9370 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.455+0000 7f627a3d3740 1 --2- 172.21.15.3:0/2212307582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba021b0 0x559ecbddf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.455+0000 7f627a3d3740 1 -- 172.21.15.3:0/2212307582 >> 172.21.15.3:0/2212307582 conn(0x559ecba1f460 msgr2=0x559ecbde8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:11.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.455+0000 7f627a3d3740 1 -- 172.21.15.3:0/2212307582 shutdown_connections 2024-09-21T06:24:11.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.455+0000 7f627a3d3740 1 -- 172.21.15.3:0/2212307582 wait complete. 2024-09-21T06:24:11.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.455+0000 7f627a3d3740 1 -- start start 2024-09-21T06:24:11.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.455+0000 7f627a3d3740 1 Processor -- start 2024-09-21T06:24:11.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f627a3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba021b0 0x559ecbde63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f627a3d3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba0a390 0x559ecbde1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f627a3d3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbde1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f627a3d3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1b0c20 con 0x559ecba2b070 2024-09-21T06:24:11.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f627a3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1b0550 con 0x559ecba021b0 2024-09-21T06:24:11.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f627a3d3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1ad7b0 con 0x559ecba0a390 2024-09-21T06:24:11.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f627a3d3740 10 client.0 _ll_register_callbacks cb 0x7f626800d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:11.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f62713fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbde1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f62713fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbde1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57428/0 (socket says 172.21.15.3:57428) 2024-09-21T06:24:11.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f62713fa640 1 -- 172.21.15.3:0/4096746869 learned_addr learned my addr 172.21.15.3:0/4096746869 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:11.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f6271bfb640 1 --2- 172.21.15.3:0/4096746869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba0a390 0x559ecbde1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f62713fa640 1 --2- 172.21.15.3:0/4096746869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba021b0 0x559ecbde63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f62713fa640 1 -- 172.21.15.3:0/4096746869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba0a390 msgr2=0x559ecbde1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f62713fa640 1 --2- 172.21.15.3:0/4096746869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba0a390 0x559ecbde1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f62713fa640 1 -- 172.21.15.3:0/4096746869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba021b0 msgr2=0x559ecbde63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f62713fa640 1 --2- 172.21.15.3:0/4096746869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba021b0 0x559ecbde63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.456+0000 7f62713fa640 1 -- 172.21.15.3:0/4096746869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ecbde2400 con 0x559ecba2b070 2024-09-21T06:24:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f62713fa640 1 --2- 172.21.15.3:0/4096746869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbde1d90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f625c004620 tx=0x7f625c020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c729ae9e666af219 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f62557fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f62557fa640 1 -- 172.21.15.3:0/4096746869 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c023040 con 0x559ecba2b070 2024-09-21T06:24:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f62557fa640 1 -- 172.21.15.3:0/4096746869 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8551+0+0 (secure 0 0 0) 0x7f625c004c20 con 0x559ecba2b070 2024-09-21T06:24:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f627a3d3740 1 -- 172.21.15.3:0/4096746869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6268011a90 con 0x559ecba2b070 2024-09-21T06:24:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f62557fa640 1 -- 172.21.15.3:0/4096746869 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f625c03e050 con 0x559ecba2b070 2024-09-21T06:24:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f627a3d3740 10 client.18072 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f627a3d3740 1 -- 172.21.15.3:0/4096746869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559ecbe5a0c0 con 0x559ecba2b070 2024-09-21T06:24:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f62557fa640 1 -- 172.21.15.3:0/4096746869 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6268011a90 con 0x559ecba2b070 2024-09-21T06:24:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f62557fa640 1 -- 172.21.15.3:0/4096746869 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f625c01c070 con 0x559ecba2b070 2024-09-21T06:24:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f627a3d3740 10 client.18072 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f627a3d3740 10 client.18072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:11.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f627a3d3740 20 client.18072 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:11.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f627a3d3740 1 --2- 172.21.15.3:0/4096746869 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559ecb9beda0 0x559ecbe5c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:11.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f627a3d3740 4 client.18072 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:24:11.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.457+0000 7f627a3d3740 1 -- 172.21.15.3:0/4096746869 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ecbd46850 con 0x559ecb9beda0 2024-09-21T06:24:11.725 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:24:11.725 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:24:11.726 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:24:11.726 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:24:11.726 INFO:teuthology.orchestra.run.smithi003.stdout: "535fb164-fd4f-4d88-b7e0-575559314385": { 2024-09-21T06:24:11.726 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:51.152572+0000", 2024-09-21T06:24:11.726 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:11.726 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:11.726 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:11.726 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:11.727 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:11.727 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:11.727 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:24:11.727 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:24:11.727 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:11.727 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:11.727 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:11.727 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:11.728 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:11.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:11.728 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:24:11.728 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:24:11.728 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:11.728 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:11.728 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:11.728 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:11.728 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:11.729 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:11.729 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:24:11.729 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:24:11.729 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:11.729 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:11.729 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:11.729 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:11.729 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:11.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:11.730 INFO:teuthology.orchestra.run.smithi003.stdout: "714b9934-a2d3-4a78-b185-01254776061b": { 2024-09-21T06:24:11.730 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:04.685380+0000", 2024-09-21T06:24:11.730 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:11.730 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:11.730 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:11.730 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:11.730 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:11.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:11.731 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:24:11.731 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:24:11.731 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:11.731 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:11.731 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:11.731 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:11.731 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:11.731 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:11.731 INFO:teuthology.orchestra.run.smithi003.stdout: "cbb5d4a7-b96d-4c2c-b7ab-7a3716c17f5b": { 2024-09-21T06:24:11.732 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:23:38.372765+0000", 2024-09-21T06:24:11.732 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:11.732 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:11.732 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:11.732 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:11.732 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:11.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:11.732 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:24:11.732 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:24:11.733 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:11.733 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:11.733 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:11.733 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:11.733 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:11.733 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:11.733 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:24:11.733 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:24:11.733 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:11.734 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:11.734 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:11.734 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:11.734 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:11.734 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:11.734 INFO:teuthology.orchestra.run.smithi003.stdout: "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58": { 2024-09-21T06:24:11.734 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:21:17.396219+0000", 2024-09-21T06:24:11.734 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:11.734 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:11.735 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:11.735 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:11.735 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:11.735 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:11.735 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:24:11.735 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:24:11.735 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:11.735 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:11.735 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:11.736 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:11.736 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:11.736 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:11.736 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:24:11.736 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:24:11.736 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:11.737 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:11.737 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:11.737 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:11.737 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:11.737 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:11.737 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:24:11.737 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:24:11.737 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:11.737 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:11.738 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:11.738 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:11.738 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:11.738 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:24:11.738 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:24:11.738 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:24:11.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.458+0000 7f6271bfb640 1 --2- 172.21.15.3:0/4096746869 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559ecb9beda0 0x559ecbe5c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:11.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.458+0000 7f6271bfb640 1 --2- 172.21.15.3:0/4096746869 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559ecb9beda0 0x559ecbe5c790 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6936a741618d996 server_cookie=56c66c5e9ed10e6c in_seq=0 out_seq=0 2024-09-21T06:24:11.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.458+0000 7f62557fa640 10 client.18072 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:11.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.458+0000 7f62557fa640 1 -- 172.21.15.3:0/4096746869 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x559ecbd46850 con 0x559ecb9beda0 2024-09-21T06:24:11.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.458+0000 7f62557fa640 10 client.18072 handle_command_reply: tid=0 2024-09-21T06:24:11.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.458+0000 7f627a3d3740 1 client.18072 shutdown 2024-09-21T06:24:11.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.459+0000 7f627a3d3740 1 -- 172.21.15.3:0/4096746869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 msgr2=0x559ecbde1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:11.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.459+0000 7f627a3d3740 1 --2- 172.21.15.3:0/4096746869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbde1d90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f625c004620 tx=0x7f625c020040 comp rx=0 tx=0).stop 2024-09-21T06:24:11.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.459+0000 7f627a3d3740 1 -- 172.21.15.3:0/4096746869 shutdown_connections 2024-09-21T06:24:11.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.459+0000 7f627a3d3740 1 --2- 172.21.15.3:0/4096746869 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x559ecb9beda0 0x559ecbe5c790 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.459+0000 7f627a3d3740 1 --2- 172.21.15.3:0/4096746869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ecba0a390 0x559ecbde1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.459+0000 7f627a3d3740 1 --2- 172.21.15.3:0/4096746869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ecba2b070 0x559ecbde1d90 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.459+0000 7f62557fa640 0 client.18072 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:11.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.459+0000 7f627a3d3740 1 --2- 172.21.15.3:0/4096746869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ecba021b0 0x559ecbde63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.459+0000 7f627a3d3740 1 -- 172.21.15.3:0/4096746869 >> 172.21.15.3:0/4096746869 conn(0x559ecba1f460 msgr2=0x559ecbde9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:11.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.459+0000 7f627a3d3740 1 -- 172.21.15.3:0/4096746869 shutdown_connections 2024-09-21T06:24:11.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.459+0000 7f627a3d3740 1 -- 172.21.15.3:0/4096746869 wait complete. 2024-09-21T06:24:11.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.459+0000 7f627a3d3740 20 client.18072 trim_cache size 0 max 16384 2024-09-21T06:24:11.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.460+0000 7f6273e5f640 1 -- 172.21.15.3:0/3529068557 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f62580b6700 msgr2=0x7f62580b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:11.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.460+0000 7f6273e5f640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f62580b6700 0x7f62580b8c10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f62640ce9a0 tx=0x7f62640f2040 comp rx=0 tx=0).stop 2024-09-21T06:24:11.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.460+0000 7f6273e5f640 1 -- 172.21.15.3:0/3529068557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626c109df0 msgr2=0x7f626c19fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:11.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.460+0000 7f6273e5f640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626c109df0 0x7f626c19fc80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6264005cc0 tx=0x7f626402f380 comp rx=0 tx=0).stop 2024-09-21T06:24:11.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.460+0000 7f6273e5f640 1 -- 172.21.15.3:0/3529068557 shutdown_connections 2024-09-21T06:24:11.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.460+0000 7f6273e5f640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f62580b6700 0x7f62580b8c10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.460+0000 7f6273e5f640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f626c1094a0 0x7f626c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.460+0000 7f6273e5f640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626c109df0 0x7f626c19fc80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.460+0000 7f6273e5f640 1 --2- 172.21.15.3:0/3529068557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f626c103b60 0x7f626c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.460+0000 7f6273e5f640 1 -- 172.21.15.3:0/3529068557 >> 172.21.15.3:0/3529068557 conn(0x7f626c07b850 msgr2=0x7f626c1074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:11.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.460+0000 7f6273e5f640 1 -- 172.21.15.3:0/3529068557 shutdown_connections 2024-09-21T06:24:11.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:11.460+0000 7f6273e5f640 1 -- 172.21.15.3:0/3529068557 wait complete. 2024-09-21T06:24:11.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:11 smithi137 ceph-mon[34464]: pgmap v607: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 141 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-21T06:24:11.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:11 smithi204 ceph-mon[34659]: pgmap v607: 97 pgs: 1 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 94 active+clean; 141 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-21T06:24:12.777 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:24:12.777 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:24:13.041 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:13.744 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:13 smithi003 ceph-mon[31227]: pgmap v608: 97 pgs: 97 active+clean; 142 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 2.2 MiB/s wr, 111 op/s 2024-09-21T06:24:13.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.742+0000 7f1eb2af4640 1 -- 172.21.15.3:0/4036545497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eac1094b0 msgr2=0x7f1eac1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:13.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.742+0000 7f1eb2af4640 1 --2- 172.21.15.3:0/4036545497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eac1094b0 0x7f1eac1098c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009900 tx=0x7f1e9402a410 comp rx=0 tx=0).stop 2024-09-21T06:24:13.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.742+0000 7f1eb2af4640 1 -- 172.21.15.3:0/4036545497 shutdown_connections 2024-09-21T06:24:13.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.742+0000 7f1eb2af4640 1 --2- 172.21.15.3:0/4036545497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1eac109e00 0x7f1eac110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.742+0000 7f1eb2af4640 1 --2- 172.21.15.3:0/4036545497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eac1094b0 0x7f1eac1098c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.742+0000 7f1eb2af4640 1 --2- 172.21.15.3:0/4036545497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1eac103bc0 0x7f1eac103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.742+0000 7f1eb2af4640 1 -- 172.21.15.3:0/4036545497 >> 172.21.15.3:0/4036545497 conn(0x7f1eac07b910 msgr2=0x7f1eac07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:13.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.742+0000 7f1eb2af4640 1 -- 172.21.15.3:0/4036545497 shutdown_connections 2024-09-21T06:24:13.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.742+0000 7f1eb2af4640 1 -- 172.21.15.3:0/4036545497 wait complete. 2024-09-21T06:24:13.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.743+0000 7f1eb2af4640 1 Processor -- start 2024-09-21T06:24:13.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.743+0000 7f1eb2af4640 1 -- start start 2024-09-21T06:24:13.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.743+0000 7f1eb2af4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eac103bc0 0x7f1eac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.743+0000 7f1eb2af4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1eac1094b0 0x7f1eac075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.743+0000 7f1eb2af4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1eac109e00 0x7f1eac076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.743+0000 7f1eb2af4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac113ac0 con 0x7f1eac103bc0 2024-09-21T06:24:13.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eb2af4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac113940 con 0x7f1eac1094b0 2024-09-21T06:24:13.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eb2af4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac113c40 con 0x7f1eac109e00 2024-09-21T06:24:13.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eb0890640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eac103bc0 0x7f1eac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eb0890640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eac103bc0 0x7f1eac075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57452/0 (socket says 172.21.15.3:57452) 2024-09-21T06:24:13.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eb0890640 1 -- 172.21.15.3:0/605512216 learned_addr learned my addr 172.21.15.3:0/605512216 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:13.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eb1091640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1eac109e00 0x7f1eac076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eabfff640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1eac1094b0 0x7f1eac075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eb0890640 1 -- 172.21.15.3:0/605512216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1eac109e00 msgr2=0x7f1eac076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:13.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eb0890640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1eac109e00 0x7f1eac076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eb0890640 1 -- 172.21.15.3:0/605512216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1eac1094b0 msgr2=0x7f1eac075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:13.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eb0890640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1eac1094b0 0x7f1eac075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eb0890640 1 -- 172.21.15.3:0/605512216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac1a87d0 con 0x7f1eac103bc0 2024-09-21T06:24:13.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eabfff640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1eac1094b0 0x7f1eac075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:13.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eb0890640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eac103bc0 0x7f1eac075710 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00c890 tx=0x7f1e9c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:13.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1ea9ffb640 1 -- 172.21.15.3:0/605512216 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c004220 con 0x7f1eac103bc0 2024-09-21T06:24:13.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1ea9ffb640 1 -- 172.21.15.3:0/605512216 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e9c0043c0 con 0x7f1eac103bc0 2024-09-21T06:24:13.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1ea9ffb640 1 -- 172.21.15.3:0/605512216 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c004a00 con 0x7f1eac103bc0 2024-09-21T06:24:13.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.744+0000 7f1eb2af4640 1 -- 172.21.15.3:0/605512216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eac1a8ac0 con 0x7f1eac103bc0 2024-09-21T06:24:13.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.745+0000 7f1eb2af4640 1 -- 172.21.15.3:0/605512216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eac1a8fd0 con 0x7f1eac103bc0 2024-09-21T06:24:13.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.746+0000 7f1eb2af4640 1 -- 172.21.15.3:0/605512216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eac103fd0 con 0x7f1eac103bc0 2024-09-21T06:24:13.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.747+0000 7f1ea9ffb640 1 -- 172.21.15.3:0/605512216 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f1e9c0079b0 con 0x7f1eac103bc0 2024-09-21T06:24:13.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.748+0000 7f1ea9ffb640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1e980b6680 0x7f1e980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.748+0000 7f1eabfff640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1e980b6680 0x7f1e980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.748+0000 7f1ea9ffb640 1 -- 172.21.15.3:0/605512216 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8551+0+0 (secure 0 0 0) 0x7f1e9c0e78c0 con 0x7f1eac103bc0 2024-09-21T06:24:13.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.749+0000 7f1eabfff640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1e980b6680 0x7f1e980b8b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009230 tx=0x7f1e94000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:13.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:13.751+0000 7f1ea9ffb640 1 -- 172.21.15.3:0/605512216 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1e9c014030 con 0x7f1eac103bc0 2024-09-21T06:24:13.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:13 smithi137 ceph-mon[34464]: pgmap v608: 97 pgs: 97 active+clean; 142 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 2.2 MiB/s wr, 111 op/s 2024-09-21T06:24:13.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:13 smithi204 ceph-mon[34659]: pgmap v608: 97 pgs: 97 active+clean; 142 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 2.2 MiB/s wr, 111 op/s 2024-09-21T06:24:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.006+0000 7f1eb2af4640 1 -- 172.21.15.3:0/605512216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1eac076a10 con 0x7f1eac103bc0 2024-09-21T06:24:14.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.007+0000 7f1ea9ffb640 1 -- 172.21.15.3:0/605512216 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11966 (secure 0 0 0) 0x7f1eac076a10 con 0x7f1eac103bc0 2024-09-21T06:24:14.009 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:24:14.009 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"btime":"2024-09-21T06:23:56:445581+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:23:55.736497+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27344":{"gid":27344,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3457686047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3457686047},{"type":"v1","addr":"172.21.15.137:6835","nonce":3457686047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24670,24674,24679,24691,24704]},"id":1}]} 2024-09-21T06:24:14.010 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-21T06:24:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.010+0000 7f1eb2af4640 1 -- 172.21.15.3:0/605512216 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1e980b6680 msgr2=0x7f1e980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.010+0000 7f1eb2af4640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1e980b6680 0x7f1e980b8b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009230 tx=0x7f1e94000bc0 comp rx=0 tx=0).stop 2024-09-21T06:24:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.010+0000 7f1eb2af4640 1 -- 172.21.15.3:0/605512216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eac103bc0 msgr2=0x7f1eac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.010+0000 7f1eb2af4640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eac103bc0 0x7f1eac075710 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00c890 tx=0x7f1e9c00cd50 comp rx=0 tx=0).stop 2024-09-21T06:24:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.010+0000 7f1eb2af4640 1 -- 172.21.15.3:0/605512216 shutdown_connections 2024-09-21T06:24:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.010+0000 7f1eb2af4640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1e980b6680 0x7f1e980b8b90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.010+0000 7f1eb2af4640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1eac109e00 0x7f1eac076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.010+0000 7f1eb2af4640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1eac1094b0 0x7f1eac075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:14.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.010+0000 7f1eb2af4640 1 --2- 172.21.15.3:0/605512216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eac103bc0 0x7f1eac075710 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:14.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.010+0000 7f1eb2af4640 1 -- 172.21.15.3:0/605512216 >> 172.21.15.3:0/605512216 conn(0x7f1eac07b910 msgr2=0x7f1eac1075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:14.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.010+0000 7f1eb2af4640 1 -- 172.21.15.3:0/605512216 shutdown_connections 2024-09-21T06:24:14.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:14.010+0000 7f1eb2af4640 1 -- 172.21.15.3:0/605512216 wait complete. 2024-09-21T06:24:14.702 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:14 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/605512216' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:24:14.703 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:14 smithi003 ceph-mon[31227]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:24:14.731 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:24:14.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:14 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/605512216' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:24:14.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:14 smithi137 ceph-mon[34464]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:24:14.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:14 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/605512216' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:24:14.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:14 smithi204 ceph-mon[34659]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:24:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:15.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.489+0000 7fcd46558640 1 -- 172.21.15.3:0/1156380587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40102270 msgr2=0x7fcd40102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.489+0000 7fcd46558640 1 --2- 172.21.15.3:0/1156380587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40102270 0x7fcd40102680 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009900 tx=0x7fcd3002a410 comp rx=0 tx=0).stop 2024-09-21T06:24:15.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.489+0000 7fcd46558640 1 -- 172.21.15.3:0/1156380587 shutdown_connections 2024-09-21T06:24:15.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.489+0000 7fcd46558640 1 --2- 172.21.15.3:0/1156380587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd401084b0 0x7fcd4010ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.489+0000 7fcd46558640 1 --2- 172.21.15.3:0/1156380587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd40107b60 0x7fcd40107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.489+0000 7fcd46558640 1 --2- 172.21.15.3:0/1156380587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40102270 0x7fcd40102680 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.489+0000 7fcd46558640 1 -- 172.21.15.3:0/1156380587 >> 172.21.15.3:0/1156380587 conn(0x7fcd40077cc0 msgr2=0x7fcd40078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:15.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.489+0000 7fcd46558640 1 -- 172.21.15.3:0/1156380587 shutdown_connections 2024-09-21T06:24:15.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.489+0000 7fcd46558640 1 -- 172.21.15.3:0/1156380587 wait complete. 2024-09-21T06:24:15.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.490+0000 7fcd46558640 1 Processor -- start 2024-09-21T06:24:15.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.490+0000 7fcd46558640 1 -- start start 2024-09-21T06:24:15.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd46558640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd40102270 0x7fcd4019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd46558640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40107b60 0x7fcd4019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd46558640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd401084b0 0x7fcd4019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd46558640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcd401118f0 con 0x7fcd40107b60 2024-09-21T06:24:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd46558640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd40111770 con 0x7fcd40102270 2024-09-21T06:24:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd46558640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd40111a70 con 0x7fcd401084b0 2024-09-21T06:24:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd3f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40107b60 0x7fcd4019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd3f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40107b60 0x7fcd4019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57632/0 (socket says 172.21.15.3:57632) 2024-09-21T06:24:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd3f7fe640 1 -- 172.21.15.3:0/589078998 learned_addr learned my addr 172.21.15.3:0/589078998 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd3ffff640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd40102270 0x7fcd4019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd44af5640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd401084b0 0x7fcd4019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd3f7fe640 1 -- 172.21.15.3:0/589078998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd401084b0 msgr2=0x7fcd4019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd3f7fe640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd401084b0 0x7fcd4019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd3f7fe640 1 -- 172.21.15.3:0/589078998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd40102270 msgr2=0x7fcd4019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd3f7fe640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd40102270 0x7fcd4019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd3f7fe640 1 -- 172.21.15.3:0/589078998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd401a04f0 con 0x7fcd40107b60 2024-09-21T06:24:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.491+0000 7fcd3f7fe640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40107b60 0x7fcd4019ab20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400d740 tx=0x7fcd3400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.492+0000 7fcd3d7fa640 1 -- 172.21.15.3:0/589078998 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34017070 con 0x7fcd40107b60 2024-09-21T06:24:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.492+0000 7fcd3d7fa640 1 -- 172.21.15.3:0/589078998 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd3400de50 con 0x7fcd40107b60 2024-09-21T06:24:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.492+0000 7fcd46558640 1 -- 172.21.15.3:0/589078998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd401a07e0 con 0x7fcd40107b60 2024-09-21T06:24:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.492+0000 7fcd3d7fa640 1 -- 172.21.15.3:0/589078998 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400f620 con 0x7fcd40107b60 2024-09-21T06:24:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.492+0000 7fcd46558640 1 -- 172.21.15.3:0/589078998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd401a6f40 con 0x7fcd40107b60 2024-09-21T06:24:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.494+0000 7fcd3d7fa640 1 -- 172.21.15.3:0/589078998 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fcd3401a070 con 0x7fcd40107b60 2024-09-21T06:24:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.494+0000 7fcd4cacc740 1 Processor -- start 2024-09-21T06:24:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd4cacc740 1 -- start start 2024-09-21T06:24:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd4cacc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 0x55aef587d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd4cacc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54ad1b0 0x55aef587d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd4cacc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef587dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd4cacc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55aef57f3ea0 con 0x55aef54d6070 2024-09-21T06:24:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd4cacc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aef57e32d0 con 0x55aef54ad1b0 2024-09-21T06:24:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd4cacc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aef57e70e0 con 0x55aef54b5390 2024-09-21T06:24:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd44af5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 0x55aef587d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd44af5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 0x55aef587d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57638/0 (socket says 172.21.15.3:57638) 2024-09-21T06:24:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd44af5640 1 -- 172.21.15.3:0/1840740989 learned_addr learned my addr 172.21.15.3:0/1840740989 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd3f7fe640 1 --2- 172.21.15.3:0/1840740989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef587dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd3ffff640 1 --2- 172.21.15.3:0/1840740989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54ad1b0 0x55aef587d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd44af5640 1 -- 172.21.15.3:0/1840740989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 msgr2=0x55aef587dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd44af5640 1 --2- 172.21.15.3:0/1840740989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef587dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd44af5640 1 -- 172.21.15.3:0/1840740989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54ad1b0 msgr2=0x55aef587d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd44af5640 1 --2- 172.21.15.3:0/1840740989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54ad1b0 0x55aef587d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd3d7fa640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcd280b6700 0x7fcd280b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd44af5640 1 -- 172.21.15.3:0/1840740989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aef5541650 con 0x55aef54d6070 2024-09-21T06:24:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd3ffff640 1 --2- 172.21.15.3:0/1840740989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54ad1b0 0x55aef587d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.495+0000 7fcd3f7fe640 1 --2- 172.21.15.3:0/1840740989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef587dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.496+0000 7fcd3f7fe640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcd280b6700 0x7fcd280b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.496+0000 7fcd44af5640 1 --2- 172.21.15.3:0/1840740989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 0x55aef587d2c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800aea0 tx=0x7fcd3800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61accbbea8370d00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.496+0000 7fcd3d7fa640 1 -- 172.21.15.3:0/589078998 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fcd340e7050 con 0x7fcd40107b60 2024-09-21T06:24:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.496+0000 7fcd267fc640 1 -- 172.21.15.3:0/1840740989 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38004b20 con 0x55aef54d6070 2024-09-21T06:24:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.496+0000 7fcd267fc640 1 -- 172.21.15.3:0/1840740989 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd38004cc0 con 0x55aef54d6070 2024-09-21T06:24:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.496+0000 7fcd267fc640 1 -- 172.21.15.3:0/1840740989 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd380053e0 con 0x55aef54d6070 2024-09-21T06:24:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.497+0000 7fcd4cacc740 1 -- 172.21.15.3:0/1840740989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 msgr2=0x55aef587d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.497+0000 7fcd3f7fe640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcd280b6700 0x7fcd280b8c10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcd340e4860 tx=0x7fcd34010c70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.497+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/1840740989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 0x55aef587d2c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800aea0 tx=0x7fcd3800af90 comp rx=0 tx=0).stop 2024-09-21T06:24:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.497+0000 7fcd4cacc740 1 -- 172.21.15.3:0/1840740989 shutdown_connections 2024-09-21T06:24:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.497+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/1840740989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef587dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.497+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/1840740989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54ad1b0 0x55aef587d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.497+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/1840740989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 0x55aef587d2c0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.497+0000 7fcd4cacc740 1 -- 172.21.15.3:0/1840740989 >> 172.21.15.3:0/1840740989 conn(0x55aef54ca460 msgr2=0x55aef587b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.497+0000 7fcd4cacc740 1 -- 172.21.15.3:0/1840740989 shutdown_connections 2024-09-21T06:24:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.497+0000 7fcd4cacc740 1 -- 172.21.15.3:0/1840740989 wait complete. 2024-09-21T06:24:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.497+0000 7fcd4cacc740 1 -- start start 2024-09-21T06:24:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.497+0000 7fcd4cacc740 1 Processor -- start 2024-09-21T06:24:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd4cacc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 0x55aef58889c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd4cacc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54b5390 0x55aef5888f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd4cacc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54d6070 0x55aef58991a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd4cacc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55aef5644700 con 0x55aef54ad1b0 2024-09-21T06:24:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd4cacc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aef577e160 con 0x55aef54b5390 2024-09-21T06:24:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd4cacc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aef588feb0 con 0x55aef54d6070 2024-09-21T06:24:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd4cacc740 10 client.0 _ll_register_callbacks cb 0x55aef57979e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd44af5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 0x55aef58889c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd44af5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 0x55aef58889c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57654/0 (socket says 172.21.15.3:57654) 2024-09-21T06:24:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd44af5640 1 -- 172.21.15.3:0/3253214166 learned_addr learned my addr 172.21.15.3:0/3253214166 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd3ffff640 1 --2- 172.21.15.3:0/3253214166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54b5390 0x55aef5888f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd44af5640 1 --2- 172.21.15.3:0/3253214166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54d6070 0x55aef58991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd44af5640 1 -- 172.21.15.3:0/3253214166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54d6070 msgr2=0x55aef58991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd44af5640 1 --2- 172.21.15.3:0/3253214166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54d6070 0x55aef58991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd44af5640 1 -- 172.21.15.3:0/3253214166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54b5390 msgr2=0x55aef5888f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd44af5640 1 --2- 172.21.15.3:0/3253214166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54b5390 0x55aef5888f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.498+0000 7fcd44af5640 1 -- 172.21.15.3:0/3253214166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aef57abc90 con 0x55aef54ad1b0 2024-09-21T06:24:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.499+0000 7fcd44af5640 1 --2- 172.21.15.3:0/3253214166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 0x55aef58889c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800dea0 tx=0x7fcd38005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6a90635eb25eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.499+0000 7fcd1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.499+0000 7fcd1f7fe640 1 -- 172.21.15.3:0/3253214166 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3800db60 con 0x55aef54ad1b0 2024-09-21T06:24:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.499+0000 7fcd1f7fe640 1 -- 172.21.15.3:0/3253214166 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fcd38003970 con 0x55aef54ad1b0 2024-09-21T06:24:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.499+0000 7fcd4cacc740 1 -- 172.21.15.3:0/3253214166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd38004cc0 con 0x55aef54ad1b0 2024-09-21T06:24:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.499+0000 7fcd1f7fe640 1 -- 172.21.15.3:0/3253214166 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcd38005120 con 0x55aef54ad1b0 2024-09-21T06:24:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.499+0000 7fcd4cacc740 10 client.18090 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.499+0000 7fcd4cacc740 1 -- 172.21.15.3:0/3253214166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55aef589bba0 con 0x55aef54ad1b0 2024-09-21T06:24:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.499+0000 7fcd1f7fe640 1 -- 172.21.15.3:0/3253214166 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd38004cc0 con 0x55aef54ad1b0 2024-09-21T06:24:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.499+0000 7fcd1f7fe640 1 -- 172.21.15.3:0/3253214166 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fcd3800b040 con 0x55aef54ad1b0 2024-09-21T06:24:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.499+0000 7fcd4cacc740 10 client.18090 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.499+0000 7fcd4cacc740 10 client.18090 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:24:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.499+0000 7fcd4cacc740 20 client.18090 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.500+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/3253214166 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55aef54949f0 0x55aef589f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.500+0000 7fcd4cacc740 4 client.18090 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.500+0000 7fcd4cacc740 1 -- 172.21.15.3:0/3253214166 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aef57f5df0 con 0x55aef54949f0 2024-09-21T06:24:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.500+0000 7fcd3ffff640 1 --2- 172.21.15.3:0/3253214166 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55aef54949f0 0x55aef589f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.500+0000 7fcd1f7fe640 10 client.18090 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:24:15.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.500+0000 7fcd3ffff640 1 --2- 172.21.15.3:0/3253214166 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55aef54949f0 0x55aef589f010 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=702c03b0729e1dd3 server_cookie=c7b337976f43f5a0 in_seq=0 out_seq=0 2024-09-21T06:24:15.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.501+0000 7fcd1f7fe640 1 -- 172.21.15.3:0/3253214166 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aef57f5df0 con 0x55aef54949f0 2024-09-21T06:24:15.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.501+0000 7fcd1f7fe640 10 client.18090 handle_command_reply: tid=0 2024-09-21T06:24:15.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.501+0000 7fcd4cacc740 1 client.18090 shutdown 2024-09-21T06:24:15.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.501+0000 7fcd4cacc740 1 -- 172.21.15.3:0/3253214166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 msgr2=0x55aef58889c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.501+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/3253214166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 0x55aef58889c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800dea0 tx=0x7fcd38005f30 comp rx=0 tx=0).stop 2024-09-21T06:24:15.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.501+0000 7fcd4cacc740 1 -- 172.21.15.3:0/3253214166 shutdown_connections 2024-09-21T06:24:15.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.501+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/3253214166 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55aef54949f0 0x55aef589f010 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.501+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/3253214166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54d6070 0x55aef58991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.501+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/3253214166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54b5390 0x55aef5888f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.501+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/3253214166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 0x55aef58889c0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.501+0000 7fcd1f7fe640 0 client.18090 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:24:15.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.501+0000 7fcd4cacc740 1 -- 172.21.15.3:0/3253214166 >> 172.21.15.3:0/3253214166 conn(0x55aef54ca460 msgr2=0x55aef58804a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:15.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.502+0000 7fcd4cacc740 1 -- 172.21.15.3:0/3253214166 shutdown_connections 2024-09-21T06:24:15.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.502+0000 7fcd4cacc740 1 -- 172.21.15.3:0/3253214166 wait complete. 2024-09-21T06:24:15.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.502+0000 7fcd4cacc740 20 client.18090 trim_cache size 0 max 16384 2024-09-21T06:24:15.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.515+0000 7fcd4cacc740 1 Processor -- start 2024-09-21T06:24:15.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.515+0000 7fcd4cacc740 1 -- start start 2024-09-21T06:24:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.515+0000 7fcd4cacc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 0x55aef588a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.515+0000 7fcd4cacc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef587e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.515+0000 7fcd4cacc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54d6070 0x55aef5899110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.515+0000 7fcd4cacc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcd401ad6b0 con 0x55aef54ad1b0 2024-09-21T06:24:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.515+0000 7fcd4cacc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd401af1f0 con 0x55aef54d6070 2024-09-21T06:24:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.515+0000 7fcd4cacc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd401add80 con 0x55aef54b5390 2024-09-21T06:24:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.515+0000 7fcd44af5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 0x55aef588a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.515+0000 7fcd44af5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 0x55aef588a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57668/0 (socket says 172.21.15.3:57668) 2024-09-21T06:24:15.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.515+0000 7fcd44af5640 1 -- 172.21.15.3:0/2153093828 learned_addr learned my addr 172.21.15.3:0/2153093828 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:15.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.515+0000 7fcd3ffff640 1 --2- 172.21.15.3:0/2153093828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef587e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.515+0000 7fcd44af5640 1 -- 172.21.15.3:0/2153093828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 msgr2=0x55aef587e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:15.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.516+0000 7fcd44af5640 1 --2- 172.21.15.3:0/2153093828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef587e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.516+0000 7fcd44af5640 1 -- 172.21.15.3:0/2153093828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54d6070 msgr2=0x55aef5899110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:15.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.516+0000 7fcd44af5640 1 --2- 172.21.15.3:0/2153093828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54d6070 0x55aef5899110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.516+0000 7fcd44af5640 1 -- 172.21.15.3:0/2153093828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aef5880120 con 0x55aef54ad1b0 2024-09-21T06:24:15.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.516+0000 7fcd3ffff640 1 --2- 172.21.15.3:0/2153093828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef587e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:15.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.516+0000 7fcd44af5640 1 --2- 172.21.15.3:0/2153093828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 0x55aef588a8a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800aa10 tx=0x7fcd38004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d458c98d50d115b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:15.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.516+0000 7fcd25ffb640 1 -- 172.21.15.3:0/2153093828 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38004c00 con 0x55aef54ad1b0 2024-09-21T06:24:15.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.516+0000 7fcd25ffb640 1 -- 172.21.15.3:0/2153093828 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd3800db60 con 0x55aef54ad1b0 2024-09-21T06:24:15.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.516+0000 7fcd25ffb640 1 -- 172.21.15.3:0/2153093828 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd380059d0 con 0x55aef54ad1b0 2024-09-21T06:24:15.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.517+0000 7fcd4cacc740 1 -- 172.21.15.3:0/2153093828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 msgr2=0x55aef588a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.517+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/2153093828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 0x55aef588a8a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800aa10 tx=0x7fcd38004b10 comp rx=0 tx=0).stop 2024-09-21T06:24:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.517+0000 7fcd4cacc740 1 -- 172.21.15.3:0/2153093828 shutdown_connections 2024-09-21T06:24:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.517+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/2153093828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef587e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.517+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/2153093828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54d6070 0x55aef5899110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.517+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/2153093828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54ad1b0 0x55aef588a8a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.517+0000 7fcd4cacc740 1 -- 172.21.15.3:0/2153093828 >> 172.21.15.3:0/2153093828 conn(0x55aef54ca460 msgr2=0x55aef58823d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.517+0000 7fcd4cacc740 1 -- 172.21.15.3:0/2153093828 shutdown_connections 2024-09-21T06:24:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.517+0000 7fcd4cacc740 1 -- 172.21.15.3:0/2153093828 wait complete. 2024-09-21T06:24:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.517+0000 7fcd4cacc740 1 -- start start 2024-09-21T06:24:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.517+0000 7fcd4cacc740 1 Processor -- start 2024-09-21T06:24:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd4cacc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54ad1b0 0x55aef58a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd4cacc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef58a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd4cacc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 0x55aef5902760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd4cacc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcd401aeb20 con 0x55aef54d6070 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd4cacc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd401ae450 con 0x55aef54ad1b0 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd4cacc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd401ab6b0 con 0x55aef54b5390 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd4cacc740 10 client.0 _ll_register_callbacks cb 0x7fcd3800ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd44af5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 0x55aef5902760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd44af5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 0x55aef5902760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57682/0 (socket says 172.21.15.3:57682) 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd44af5640 1 -- 172.21.15.3:0/238084674 learned_addr learned my addr 172.21.15.3:0/238084674 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:15.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd44af5640 1 --2- 172.21.15.3:0/238084674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54ad1b0 0x55aef58a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd3ffff640 1 --2- 172.21.15.3:0/238084674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef58a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd44af5640 1 -- 172.21.15.3:0/238084674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 msgr2=0x55aef58a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:15.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd44af5640 1 --2- 172.21.15.3:0/238084674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef58a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd44af5640 1 -- 172.21.15.3:0/238084674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54ad1b0 msgr2=0x55aef58a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:15.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd44af5640 1 --2- 172.21.15.3:0/238084674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54ad1b0 0x55aef58a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.518+0000 7fcd44af5640 1 -- 172.21.15.3:0/238084674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aef5902dd0 con 0x55aef54d6070 2024-09-21T06:24:15.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd44af5640 1 --2- 172.21.15.3:0/238084674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 0x55aef5902760 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800d920 tx=0x7fcd3802fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aed248d6241b925 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd1f7fe640 1 -- 172.21.15.3:0/238084674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38004b80 con 0x55aef54d6070 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd4cacc740 1 -- 172.21.15.3:0/238084674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd3800db60 con 0x55aef54d6070 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd1f7fe640 1 -- 172.21.15.3:0/238084674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fcd38014040 con 0x55aef54d6070 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd1f7fe640 1 -- 172.21.15.3:0/238084674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcd38004d20 con 0x55aef54d6070 2024-09-21T06:24:15.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd4cacc740 10 client.18096 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:15.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd4cacc740 1 -- 172.21.15.3:0/238084674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55aef590a0f0 con 0x55aef54d6070 2024-09-21T06:24:15.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd1f7fe640 1 -- 172.21.15.3:0/238084674 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd3800db60 con 0x55aef54d6070 2024-09-21T06:24:15.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd1f7fe640 1 -- 172.21.15.3:0/238084674 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fcd380058d0 con 0x55aef54d6070 2024-09-21T06:24:15.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd4cacc740 10 client.18096 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:15.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd4cacc740 10 client.18096 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd4cacc740 20 client.18096 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/238084674 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55aef5469da0 0x55aef590c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd4cacc740 4 client.18096 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.519+0000 7fcd4cacc740 1 -- 172.21.15.3:0/238084674 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aef57f18a0 con 0x55aef5469da0 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.520+0000 7fcd3ffff640 1 --2- 172.21.15.3:0/238084674 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55aef5469da0 0x55aef590c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.520+0000 7fcd1f7fe640 10 client.18096 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:24:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.520+0000 7fcd3ffff640 1 --2- 172.21.15.3:0/238084674 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55aef5469da0 0x55aef590c670 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e04af46ab0c0de7d server_cookie=271c636c5515aca2 in_seq=0 out_seq=0 2024-09-21T06:24:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.520+0000 7fcd1f7fe640 1 -- 172.21.15.3:0/238084674 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aef57f18a0 con 0x55aef5469da0 2024-09-21T06:24:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.520+0000 7fcd1f7fe640 10 client.18096 handle_command_reply: tid=0 2024-09-21T06:24:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.520+0000 7fcd4cacc740 1 client.18096 shutdown 2024-09-21T06:24:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd4cacc740 1 -- 172.21.15.3:0/238084674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 msgr2=0x55aef5902760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/238084674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 0x55aef5902760 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800d920 tx=0x7fcd3802fa40 comp rx=0 tx=0).stop 2024-09-21T06:24:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd4cacc740 1 -- 172.21.15.3:0/238084674 shutdown_connections 2024-09-21T06:24:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/238084674 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x55aef5469da0 0x55aef590c670 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/238084674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef54b5390 0x55aef58a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/238084674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aef54d6070 0x55aef5902760 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd4cacc740 1 --2- 172.21.15.3:0/238084674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aef54ad1b0 0x55aef58a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd4cacc740 1 -- 172.21.15.3:0/238084674 >> 172.21.15.3:0/238084674 conn(0x55aef54ca460 msgr2=0x55aef58823d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd1f7fe640 0 client.18096 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:24:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd4cacc740 1 -- 172.21.15.3:0/238084674 shutdown_connections 2024-09-21T06:24:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd4cacc740 1 -- 172.21.15.3:0/238084674 wait complete. 2024-09-21T06:24:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd4cacc740 20 client.18096 trim_cache size 0 max 16384 2024-09-21T06:24:15.537 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:24:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd46558640 1 -- 172.21.15.3:0/589078998 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcd280b6700 msgr2=0x7fcd280b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd46558640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcd280b6700 0x7fcd280b8c10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcd340e4860 tx=0x7fcd34010c70 comp rx=0 tx=0).stop 2024-09-21T06:24:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd46558640 1 -- 172.21.15.3:0/589078998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40107b60 msgr2=0x7fcd4019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.521+0000 7fcd46558640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40107b60 0x7fcd4019ab20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400d740 tx=0x7fcd3400dc00 comp rx=0 tx=0).stop 2024-09-21T06:24:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.522+0000 7fcd46558640 1 -- 172.21.15.3:0/589078998 shutdown_connections 2024-09-21T06:24:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.522+0000 7fcd46558640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcd280b6700 0x7fcd280b8c10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.522+0000 7fcd46558640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd401084b0 0x7fcd4019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.522+0000 7fcd46558640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40107b60 0x7fcd4019ab20 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.522+0000 7fcd46558640 1 --2- 172.21.15.3:0/589078998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd40102270 0x7fcd4019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.522+0000 7fcd46558640 1 -- 172.21.15.3:0/589078998 >> 172.21.15.3:0/589078998 conn(0x7fcd40077cc0 msgr2=0x7fcd401060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.522+0000 7fcd46558640 1 -- 172.21.15.3:0/589078998 shutdown_connections 2024-09-21T06:24:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:15.522+0000 7fcd46558640 1 -- 172.21.15.3:0/589078998 wait complete. 2024-09-21T06:24:15.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:15 smithi137 ceph-mon[34464]: pgmap v609: 97 pgs: 97 active+clean; 142 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.9 MiB/s wr, 94 op/s 2024-09-21T06:24:15.805 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:15 smithi204 ceph-mon[34659]: pgmap v609: 97 pgs: 97 active+clean; 142 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.9 MiB/s wr, 94 op/s 2024-09-21T06:24:15.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:15 smithi003 ceph-mon[31227]: pgmap v609: 97 pgs: 97 active+clean; 142 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.9 MiB/s wr, 94 op/s 2024-09-21T06:24:16.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:24:16.589 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:17.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.079+0000 7fcaccc48640 1 -- 172.21.15.3:0/2174594898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac8101e30 msgr2=0x7fcac8102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.079+0000 7fcaccc48640 1 --2- 172.21.15.3:0/2174594898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac8101e30 0x7fcac8102240 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009900 tx=0x7fcab002a410 comp rx=0 tx=0).stop 2024-09-21T06:24:17.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.079+0000 7fcaccc48640 1 -- 172.21.15.3:0/2174594898 shutdown_connections 2024-09-21T06:24:17.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.079+0000 7fcaccc48640 1 --2- 172.21.15.3:0/2174594898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8102780 0x7fcac81105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.079+0000 7fcaccc48640 1 --2- 172.21.15.3:0/2174594898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac8101e30 0x7fcac8102240 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.079+0000 7fcaccc48640 1 --2- 172.21.15.3:0/2174594898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcac8108740 0x7fcac8108b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.079+0000 7fcaccc48640 1 -- 172.21.15.3:0/2174594898 >> 172.21.15.3:0/2174594898 conn(0x7fcac8079f90 msgr2=0x7fcac807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.079+0000 7fcaccc48640 1 -- 172.21.15.3:0/2174594898 shutdown_connections 2024-09-21T06:24:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.079+0000 7fcaccc48640 1 -- 172.21.15.3:0/2174594898 wait complete. 2024-09-21T06:24:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.080+0000 7fcaccc48640 1 Processor -- start 2024-09-21T06:24:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.080+0000 7fcaccc48640 1 -- start start 2024-09-21T06:24:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.080+0000 7fcaccc48640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac8101e30 0x7fcac8103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.080+0000 7fcaccc48640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcac8102780 0x7fcac81035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.080+0000 7fcaccc48640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108740 0x7fcac8103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.080+0000 7fcaccc48640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8113af0 con 0x7fcac8101e30 2024-09-21T06:24:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.080+0000 7fcaccc48640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8113970 con 0x7fcac8108740 2024-09-21T06:24:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.080+0000 7fcaccc48640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8113c70 con 0x7fcac8102780 2024-09-21T06:24:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.080+0000 7fcac659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac8101e30 0x7fcac8103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.080+0000 7fcac659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac8101e30 0x7fcac8103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57696/0 (socket says 172.21.15.3:57696) 2024-09-21T06:24:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.080+0000 7fcac659c640 1 -- 172.21.15.3:0/2083887085 learned_addr learned my addr 172.21.15.3:0/2083887085 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.080+0000 7fcac6d9d640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108740 0x7fcac8103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.080+0000 7fcabffff640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcac8102780 0x7fcac81035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.081+0000 7fcac659c640 1 -- 172.21.15.3:0/2083887085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcac8102780 msgr2=0x7fcac81035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.081+0000 7fcac659c640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcac8102780 0x7fcac81035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.081+0000 7fcac659c640 1 -- 172.21.15.3:0/2083887085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108740 msgr2=0x7fcac8103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.081+0000 7fcac659c640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108740 0x7fcac8103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.081+0000 7fcac659c640 1 -- 172.21.15.3:0/2083887085 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac81a8780 con 0x7fcac8101e30 2024-09-21T06:24:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.081+0000 7fcac659c640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac8101e30 0x7fcac8103060 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fcab800e9f0 tx=0x7fcab800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.081+0000 7fcabf7fe640 1 -- 172.21.15.3:0/2083887085 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab800cd80 con 0x7fcac8101e30 2024-09-21T06:24:17.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.081+0000 7fcabf7fe640 1 -- 172.21.15.3:0/2083887085 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcab80054d0 con 0x7fcac8101e30 2024-09-21T06:24:17.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.081+0000 7fcaccc48640 1 -- 172.21.15.3:0/2083887085 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac81a8a70 con 0x7fcac8101e30 2024-09-21T06:24:17.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.081+0000 7fcabf7fe640 1 -- 172.21.15.3:0/2083887085 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8010690 con 0x7fcac8101e30 2024-09-21T06:24:17.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.081+0000 7fcaccc48640 1 -- 172.21.15.3:0/2083887085 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac81a8f80 con 0x7fcac8101e30 2024-09-21T06:24:17.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.084+0000 7fcad31bc740 1 Processor -- start 2024-09-21T06:24:17.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.084+0000 7fcad31bc740 1 -- start start 2024-09-21T06:24:17.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.084+0000 7fcad31bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559309395070 0x55930973c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.084+0000 7fcad31bc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55930936c1b0 0x55930973cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.084+0000 7fcad31bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x55930973d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.084+0000 7fcad31bc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5593096b2ea0 con 0x55930936c1b0 2024-09-21T06:24:17.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.084+0000 7fcad31bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5593096a22d0 con 0x559309374390 2024-09-21T06:24:17.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.084+0000 7fcad31bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5593096a60e0 con 0x559309395070 2024-09-21T06:24:17.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.084+0000 7fcabf7fe640 1 -- 172.21.15.3:0/2083887085 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fcab8010830 con 0x7fcac8101e30 2024-09-21T06:24:17.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.084+0000 7fcac659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55930936c1b0 0x55930973cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcac659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55930936c1b0 0x55930973cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57704/0 (socket says 172.21.15.3:57704) 2024-09-21T06:24:17.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcac659c640 1 -- 172.21.15.3:0/2731019001 learned_addr learned my addr 172.21.15.3:0/2731019001 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:17.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcabffff640 1 --2- 172.21.15.3:0/2731019001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x55930973d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcac6d9d640 1 --2- 172.21.15.3:0/2731019001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559309395070 0x55930973c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcac659c640 1 -- 172.21.15.3:0/2731019001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559309395070 msgr2=0x55930973c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcac659c640 1 --2- 172.21.15.3:0/2731019001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559309395070 0x55930973c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcac659c640 1 -- 172.21.15.3:0/2731019001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 msgr2=0x55930973d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcac659c640 1 --2- 172.21.15.3:0/2731019001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x55930973d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcac659c640 1 -- 172.21.15.3:0/2731019001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559309400650 con 0x55930936c1b0 2024-09-21T06:24:17.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcabffff640 1 --2- 172.21.15.3:0/2731019001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x55930973d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:17.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcac659c640 1 --2- 172.21.15.3:0/2731019001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55930936c1b0 0x55930973cbe0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fcab80e5130 tx=0x7fcab80ebb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee80df12e380442 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:17.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcabf7fe640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcab40b6700 0x7fcab40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcabcff9640 1 -- 172.21.15.3:0/2731019001 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80a9f40 con 0x55930936c1b0 2024-09-21T06:24:17.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.085+0000 7fcac6d9d640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcab40b6700 0x7fcab40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.086+0000 7fcabcff9640 1 -- 172.21.15.3:0/2731019001 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcab80f9070 con 0x55930936c1b0 2024-09-21T06:24:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.086+0000 7fcabcff9640 1 -- 172.21.15.3:0/2731019001 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80ad240 con 0x55930936c1b0 2024-09-21T06:24:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.086+0000 7fcabf7fe640 1 -- 172.21.15.3:0/2083887085 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fcab80e7af0 con 0x7fcac8101e30 2024-09-21T06:24:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.087+0000 7fcad31bc740 1 -- 172.21.15.3:0/2731019001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55930936c1b0 msgr2=0x55930973cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.087+0000 7fcac6d9d640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcab40b6700 0x7fcab40b8c10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcac0005fd0 tx=0x7fcac0009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.087+0000 7fcad31bc740 1 --2- 172.21.15.3:0/2731019001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55930936c1b0 0x55930973cbe0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fcab80e5130 tx=0x7fcab80ebb90 comp rx=0 tx=0).stop 2024-09-21T06:24:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.087+0000 7fcad31bc740 1 -- 172.21.15.3:0/2731019001 shutdown_connections 2024-09-21T06:24:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.087+0000 7fcad31bc740 1 --2- 172.21.15.3:0/2731019001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x55930973d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.087+0000 7fcad31bc740 1 --2- 172.21.15.3:0/2731019001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55930936c1b0 0x55930973cbe0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.087+0000 7fcad31bc740 1 --2- 172.21.15.3:0/2731019001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559309395070 0x55930973c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.087+0000 7fcad31bc740 1 -- 172.21.15.3:0/2731019001 >> 172.21.15.3:0/2731019001 conn(0x559309389460 msgr2=0x55930973a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.087+0000 7fcad31bc740 1 -- 172.21.15.3:0/2731019001 shutdown_connections 2024-09-21T06:24:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.087+0000 7fcad31bc740 1 -- 172.21.15.3:0/2731019001 wait complete. 2024-09-21T06:24:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.087+0000 7fcad31bc740 1 -- start start 2024-09-21T06:24:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.087+0000 7fcad31bc740 1 Processor -- start 2024-09-21T06:24:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcad31bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930936c1b0 0x559309747d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcad31bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x5593097482a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcad31bc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x559309754140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcad31bc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559309503700 con 0x559309395070 2024-09-21T06:24:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcad31bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55930963d160 con 0x559309374390 2024-09-21T06:24:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcad31bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55930974ae20 con 0x55930936c1b0 2024-09-21T06:24:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcad31bc740 10 client.0 _ll_register_callbacks cb 0x5593096569e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcabffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x5593097482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcac6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x559309754140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcac659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930936c1b0 0x559309747d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcabffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x5593097482a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:43082/0 (socket says 172.21.15.3:43082) 2024-09-21T06:24:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcac6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x559309754140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57720/0 (socket says 172.21.15.3:57720) 2024-09-21T06:24:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcabffff640 1 -- 172.21.15.3:0/1846200776 learned_addr learned my addr 172.21.15.3:0/1846200776 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcac6d9d640 1 -- 172.21.15.3:0/1846200776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930936c1b0 msgr2=0x559309747d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcac6d9d640 1 --2- 172.21.15.3:0/1846200776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930936c1b0 0x559309747d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcac6d9d640 1 -- 172.21.15.3:0/1846200776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 msgr2=0x5593097482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcac6d9d640 1 --2- 172.21.15.3:0/1846200776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x5593097482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcac6d9d640 1 -- 172.21.15.3:0/1846200776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55930966ac90 con 0x559309395070 2024-09-21T06:24:17.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcac659c640 1 --2- 172.21.15.3:0/1846200776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930936c1b0 0x559309747d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:17.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcabffff640 1 --2- 172.21.15.3:0/1846200776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x5593097482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:17.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcac6d9d640 1 --2- 172.21.15.3:0/1846200776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x559309754140 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fcac0011d10 tx=0x7fcac0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8494f2f68ec984 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:17.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.088+0000 7fcaadffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:17.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcaadffb640 1 -- 172.21.15.3:0/1846200776 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0020620 con 0x559309395070 2024-09-21T06:24:17.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcaadffb640 1 -- 172.21.15.3:0/1846200776 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fcac00207c0 con 0x559309395070 2024-09-21T06:24:17.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcad31bc740 1 -- 172.21.15.3:0/1846200776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab80f9070 con 0x559309395070 2024-09-21T06:24:17.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcaadffb640 1 -- 172.21.15.3:0/1846200776 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcac0004b20 con 0x559309395070 2024-09-21T06:24:17.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcad31bc740 10 client.18108 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:17.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcad31bc740 1 -- 172.21.15.3:0/1846200776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559309756b60 con 0x559309395070 2024-09-21T06:24:17.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcaadffb640 1 -- 172.21.15.3:0/1846200776 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcab80f9070 con 0x559309395070 2024-09-21T06:24:17.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcaadffb640 1 -- 172.21.15.3:0/1846200776 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fcac000f040 con 0x559309395070 2024-09-21T06:24:17.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcad31bc740 10 client.18108 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:17.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcad31bc740 10 client.18108 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:24:17.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcad31bc740 20 client.18108 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:17.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcad31bc740 1 --2- 172.21.15.3:0/1846200776 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5593093539f0 0x55930975a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcad31bc740 4 client.18108 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:17.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.089+0000 7fcad31bc740 1 -- 172.21.15.3:0/1846200776 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593096b4df0 con 0x5593093539f0 2024-09-21T06:24:17.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.090+0000 7fcac659c640 1 --2- 172.21.15.3:0/1846200776 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5593093539f0 0x55930975a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.090+0000 7fcac659c640 1 --2- 172.21.15.3:0/1846200776 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5593093539f0 0x55930975a100 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8ba9195ce6de46af server_cookie=df53d68ff50b2b31 in_seq=0 out_seq=0 2024-09-21T06:24:17.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.090+0000 7fcaadffb640 10 client.18108 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:24:17.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcaadffb640 1 -- 172.21.15.3:0/1846200776 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5593096b4df0 con 0x5593093539f0 2024-09-21T06:24:17.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcaadffb640 10 client.18108 handle_command_reply: tid=0 2024-09-21T06:24:17.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcad31bc740 1 client.18108 shutdown 2024-09-21T06:24:17.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcad31bc740 1 -- 172.21.15.3:0/1846200776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 msgr2=0x559309754140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcad31bc740 1 --2- 172.21.15.3:0/1846200776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x559309754140 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fcac0011d10 tx=0x7fcac0004a40 comp rx=0 tx=0).stop 2024-09-21T06:24:17.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcad31bc740 1 -- 172.21.15.3:0/1846200776 shutdown_connections 2024-09-21T06:24:17.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcad31bc740 1 --2- 172.21.15.3:0/1846200776 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5593093539f0 0x55930975a100 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcad31bc740 1 --2- 172.21.15.3:0/1846200776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x559309754140 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcad31bc740 1 --2- 172.21.15.3:0/1846200776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x5593097482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcad31bc740 1 --2- 172.21.15.3:0/1846200776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930936c1b0 0x559309747d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcaadffb640 0 client.18108 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:24:17.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcad31bc740 1 -- 172.21.15.3:0/1846200776 >> 172.21.15.3:0/1846200776 conn(0x559309389460 msgr2=0x55930973f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:17.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcad31bc740 1 -- 172.21.15.3:0/1846200776 shutdown_connections 2024-09-21T06:24:17.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcad31bc740 1 -- 172.21.15.3:0/1846200776 wait complete. 2024-09-21T06:24:17.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.091+0000 7fcad31bc740 20 client.18108 trim_cache size 0 max 16384 2024-09-21T06:24:17.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcad31bc740 1 Processor -- start 2024-09-21T06:24:17.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcad31bc740 1 -- start start 2024-09-21T06:24:17.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcad31bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930936c1b0 0x559309749c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcad31bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559309374390 0x55930973dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcad31bc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x559309754040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcad31bc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b1910 con 0x559309395070 2024-09-21T06:24:17.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcad31bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b3450 con 0x55930936c1b0 2024-09-21T06:24:17.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcad31bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b1fe0 con 0x559309374390 2024-09-21T06:24:17.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcac6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x559309754040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcac6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x559309754040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57730/0 (socket says 172.21.15.3:57730) 2024-09-21T06:24:17.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcac6d9d640 1 -- 172.21.15.3:0/4073697276 learned_addr learned my addr 172.21.15.3:0/4073697276 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:17.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcac6d9d640 1 -- 172.21.15.3:0/4073697276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559309374390 msgr2=0x55930973dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcac659c640 1 --2- 172.21.15.3:0/4073697276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930936c1b0 0x559309749c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcabffff640 1 --2- 172.21.15.3:0/4073697276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559309374390 0x55930973dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcac6d9d640 1 --2- 172.21.15.3:0/4073697276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559309374390 0x55930973dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcac6d9d640 1 -- 172.21.15.3:0/4073697276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930936c1b0 msgr2=0x559309749c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcac6d9d640 1 --2- 172.21.15.3:0/4073697276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930936c1b0 0x559309749c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcac6d9d640 1 -- 172.21.15.3:0/4073697276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55930974a4a0 con 0x559309395070 2024-09-21T06:24:17.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcac659c640 1 --2- 172.21.15.3:0/4073697276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930936c1b0 0x559309749c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:24:17.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcabffff640 1 --2- 172.21.15.3:0/4073697276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559309374390 0x55930973dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:24:17.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.105+0000 7fcac6d9d640 1 --2- 172.21.15.3:0/4073697276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x559309754040 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcac0013e40 tx=0x7fcac0011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1778f4bfae61332 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:17.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.106+0000 7fcaaffff640 1 -- 172.21.15.3:0/4073697276 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac001fe80 con 0x559309395070 2024-09-21T06:24:17.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.106+0000 7fcaaffff640 1 -- 172.21.15.3:0/4073697276 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcac001de20 con 0x559309395070 2024-09-21T06:24:17.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.106+0000 7fcaaffff640 1 -- 172.21.15.3:0/4073697276 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0030400 con 0x559309395070 2024-09-21T06:24:17.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.106+0000 7fcad31bc740 1 -- 172.21.15.3:0/4073697276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 msgr2=0x559309754040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.106+0000 7fcad31bc740 1 --2- 172.21.15.3:0/4073697276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x559309754040 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcac0013e40 tx=0x7fcac0011fd0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.106+0000 7fcad31bc740 1 -- 172.21.15.3:0/4073697276 shutdown_connections 2024-09-21T06:24:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.106+0000 7fcad31bc740 1 --2- 172.21.15.3:0/4073697276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559309374390 0x55930973dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.106+0000 7fcad31bc740 1 --2- 172.21.15.3:0/4073697276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x559309754040 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.106+0000 7fcad31bc740 1 --2- 172.21.15.3:0/4073697276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930936c1b0 0x559309749c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.106+0000 7fcad31bc740 1 -- 172.21.15.3:0/4073697276 >> 172.21.15.3:0/4073697276 conn(0x559309389460 msgr2=0x55930973d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.106+0000 7fcad31bc740 1 -- 172.21.15.3:0/4073697276 shutdown_connections 2024-09-21T06:24:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcad31bc740 1 -- 172.21.15.3:0/4073697276 wait complete. 2024-09-21T06:24:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcad31bc740 1 -- start start 2024-09-21T06:24:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcad31bc740 1 Processor -- start 2024-09-21T06:24:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcad31bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930936c1b0 0x559309750a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcad31bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x55930974bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcad31bc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x55930974c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcad31bc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b2d80 con 0x559309395070 2024-09-21T06:24:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcad31bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b26b0 con 0x559309374390 2024-09-21T06:24:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcad31bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81af910 con 0x55930936c1b0 2024-09-21T06:24:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcad31bc740 10 client.0 _ll_register_callbacks cb 0x7fcac00119c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcac6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x55930974c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcac6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x55930974c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57736/0 (socket says 172.21.15.3:57736) 2024-09-21T06:24:17.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcac6d9d640 1 -- 172.21.15.3:0/2530883972 learned_addr learned my addr 172.21.15.3:0/2530883972 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:17.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcabffff640 1 --2- 172.21.15.3:0/2530883972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x55930974bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.107+0000 7fcac659c640 1 --2- 172.21.15.3:0/2530883972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930936c1b0 0x559309750a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcac6d9d640 1 -- 172.21.15.3:0/2530883972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930936c1b0 msgr2=0x559309750a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcac6d9d640 1 --2- 172.21.15.3:0/2530883972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930936c1b0 0x559309750a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcac6d9d640 1 -- 172.21.15.3:0/2530883972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 msgr2=0x55930974bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcac6d9d640 1 --2- 172.21.15.3:0/2530883972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x55930974bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcac6d9d640 1 -- 172.21.15.3:0/2530883972 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593097c3ac0 con 0x559309395070 2024-09-21T06:24:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcac6d9d640 1 --2- 172.21.15.3:0/2530883972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x55930974c640 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fcac0004a80 tx=0x7fcac001d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82dbec3a5da3ac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcaadffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcaadffb640 1 -- 172.21.15.3:0/2530883972 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0048060 con 0x559309395070 2024-09-21T06:24:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcaadffb640 1 -- 172.21.15.3:0/2530883972 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fcac001dc20 con 0x559309395070 2024-09-21T06:24:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcad31bc740 1 -- 172.21.15.3:0/2530883972 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcac001de20 con 0x559309395070 2024-09-21T06:24:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcaadffb640 1 -- 172.21.15.3:0/2530883972 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcac0039640 con 0x559309395070 2024-09-21T06:24:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcad31bc740 10 client.18114 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcad31bc740 1 -- 172.21.15.3:0/2530883972 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5593097c4a60 con 0x559309395070 2024-09-21T06:24:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.108+0000 7fcaadffb640 1 -- 172.21.15.3:0/2530883972 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcac001de20 con 0x559309395070 2024-09-21T06:24:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.109+0000 7fcaadffb640 1 -- 172.21.15.3:0/2530883972 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fcac000f040 con 0x559309395070 2024-09-21T06:24:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.109+0000 7fcad31bc740 10 client.18114 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.109+0000 7fcad31bc740 10 client.18114 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:24:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.109+0000 7fcad31bc740 20 client.18114 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.109+0000 7fcad31bc740 1 --2- 172.21.15.3:0/2530883972 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x559309328da0 0x5593097c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.109+0000 7fcad31bc740 4 client.18114 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:24:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.109+0000 7fcad31bc740 1 -- 172.21.15.3:0/2530883972 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593096b08a0 con 0x559309328da0 2024-09-21T06:24:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.109+0000 7fcac659c640 1 --2- 172.21.15.3:0/2530883972 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x559309328da0 0x5593097c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.109+0000 7fcaadffb640 10 client.18114 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:24:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.109+0000 7fcac659c640 1 --2- 172.21.15.3:0/2530883972 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x559309328da0 0x5593097c70e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bce6142a56d662bd server_cookie=f7f58ada530bc33 in_seq=0 out_seq=0 2024-09-21T06:24:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcaadffb640 1 -- 172.21.15.3:0/2530883972 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593096b08a0 con 0x559309328da0 2024-09-21T06:24:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcaadffb640 10 client.18114 handle_command_reply: tid=0 2024-09-21T06:24:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcad31bc740 1 client.18114 shutdown 2024-09-21T06:24:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcad31bc740 1 -- 172.21.15.3:0/2530883972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 msgr2=0x55930974c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcad31bc740 1 --2- 172.21.15.3:0/2530883972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x55930974c640 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fcac0004a80 tx=0x7fcac001d970 comp rx=0 tx=0).stop 2024-09-21T06:24:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcad31bc740 1 -- 172.21.15.3:0/2530883972 shutdown_connections 2024-09-21T06:24:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcad31bc740 1 --2- 172.21.15.3:0/2530883972 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x559309328da0 0x5593097c70e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcad31bc740 1 --2- 172.21.15.3:0/2530883972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559309395070 0x55930974c640 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcad31bc740 1 --2- 172.21.15.3:0/2530883972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559309374390 0x55930974bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcad31bc740 1 --2- 172.21.15.3:0/2530883972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930936c1b0 0x559309750a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcad31bc740 1 -- 172.21.15.3:0/2530883972 >> 172.21.15.3:0/2530883972 conn(0x559309389460 msgr2=0x5593097499a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:17.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcaadffb640 0 client.18114 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:24:17.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcad31bc740 1 -- 172.21.15.3:0/2530883972 shutdown_connections 2024-09-21T06:24:17.325 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:24:17.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcad31bc740 1 -- 172.21.15.3:0/2530883972 wait complete. 2024-09-21T06:24:17.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcad31bc740 20 client.18114 trim_cache size 0 max 16384 2024-09-21T06:24:17.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcaccc48640 1 -- 172.21.15.3:0/2083887085 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcab40b6700 msgr2=0x7fcab40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.110+0000 7fcaccc48640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcab40b6700 0x7fcab40b8c10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcac0005fd0 tx=0x7fcac0009600 comp rx=0 tx=0).stop 2024-09-21T06:24:17.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.111+0000 7fcaccc48640 1 -- 172.21.15.3:0/2083887085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac8101e30 msgr2=0x7fcac8103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.111+0000 7fcaccc48640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac8101e30 0x7fcac8103060 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fcab800e9f0 tx=0x7fcab800edf0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.111+0000 7fcaccc48640 1 -- 172.21.15.3:0/2083887085 shutdown_connections 2024-09-21T06:24:17.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.111+0000 7fcaccc48640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcab40b6700 0x7fcab40b8c10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.111+0000 7fcaccc48640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcac8102780 0x7fcac81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.111+0000 7fcaccc48640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108740 0x7fcac8103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.111+0000 7fcaccc48640 1 --2- 172.21.15.3:0/2083887085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac8101e30 0x7fcac8103060 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.111+0000 7fcaccc48640 1 -- 172.21.15.3:0/2083887085 >> 172.21.15.3:0/2083887085 conn(0x7fcac8079f90 msgr2=0x7fcac8105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:17.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.111+0000 7fcaccc48640 1 -- 172.21.15.3:0/2083887085 shutdown_connections 2024-09-21T06:24:17.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:17.111+0000 7fcaccc48640 1 -- 172.21.15.3:0/2083887085 wait complete. 2024-09-21T06:24:17.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:17 smithi003 ceph-mon[31227]: pgmap v611: 97 pgs: 97 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.4 MiB/s wr, 59 op/s 2024-09-21T06:24:17.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:17 smithi137 ceph-mon[34464]: pgmap v611: 97 pgs: 97 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.4 MiB/s wr, 59 op/s 2024-09-21T06:24:17.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:17 smithi204 ceph-mon[34659]: pgmap v611: 97 pgs: 97 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.4 MiB/s wr, 59 op/s 2024-09-21T06:24:18.145 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:24:18.411 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.915+0000 7fec913c6640 1 -- 172.21.15.3:0/1759779234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec8c109e40 msgr2=0x7fec8c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.915+0000 7fec913c6640 1 --2- 172.21.15.3:0/1759779234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec8c109e40 0x7fec8c110670 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fec84009ca0 tx=0x7fec8402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.915+0000 7fec913c6640 1 -- 172.21.15.3:0/1759779234 shutdown_connections 2024-09-21T06:24:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.915+0000 7fec913c6640 1 --2- 172.21.15.3:0/1759779234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec8c109e40 0x7fec8c110670 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.915+0000 7fec913c6640 1 --2- 172.21.15.3:0/1759779234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec8c1094f0 0x7fec8c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.915+0000 7fec913c6640 1 --2- 172.21.15.3:0/1759779234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec8c103bb0 0x7fec8c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.915+0000 7fec913c6640 1 -- 172.21.15.3:0/1759779234 >> 172.21.15.3:0/1759779234 conn(0x7fec8c0ff600 msgr2=0x7fec8c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.915+0000 7fec913c6640 1 -- 172.21.15.3:0/1759779234 shutdown_connections 2024-09-21T06:24:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.915+0000 7fec913c6640 1 -- 172.21.15.3:0/1759779234 wait complete. 2024-09-21T06:24:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.915+0000 7fec913c6640 1 Processor -- start 2024-09-21T06:24:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec913c6640 1 -- start start 2024-09-21T06:24:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec913c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec8c103bb0 0x7fec8c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec913c6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec8c1094f0 0x7fec8c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec913c6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec8c109e40 0x7fec8c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec913c6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c113b00 con 0x7fec8c1094f0 2024-09-21T06:24:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec913c6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c113980 con 0x7fec8c109e40 2024-09-21T06:24:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec913c6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c113c80 con 0x7fec8c103bb0 2024-09-21T06:24:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec8a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec8c1094f0 0x7fec8c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec8a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec8c1094f0 0x7fec8c19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57758/0 (socket says 172.21.15.3:57758) 2024-09-21T06:24:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec8a7fc640 1 -- 172.21.15.3:0/3312806097 learned_addr learned my addr 172.21.15.3:0/3312806097 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec8affd640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec8c103bb0 0x7fec8c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec8b7fe640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec8c109e40 0x7fec8c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec8a7fc640 1 -- 172.21.15.3:0/3312806097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec8c103bb0 msgr2=0x7fec8c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec8a7fc640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec8c103bb0 0x7fec8c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec8a7fc640 1 -- 172.21.15.3:0/3312806097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec8c109e40 msgr2=0x7fec8c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec8a7fc640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec8c109e40 0x7fec8c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.916+0000 7fec8a7fc640 1 -- 172.21.15.3:0/3312806097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8c1a0340 con 0x7fec8c1094f0 2024-09-21T06:24:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.917+0000 7fec8a7fc640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec8c1094f0 0x7fec8c19aa30 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fec8000c8c0 tx=0x7fec8000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.917+0000 7fec73fff640 1 -- 172.21.15.3:0/3312806097 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80007bb0 con 0x7fec8c1094f0 2024-09-21T06:24:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.917+0000 7fec73fff640 1 -- 172.21.15.3:0/3312806097 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec80007d50 con 0x7fec8c1094f0 2024-09-21T06:24:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.917+0000 7fec73fff640 1 -- 172.21.15.3:0/3312806097 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec800187c0 con 0x7fec8c1094f0 2024-09-21T06:24:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.917+0000 7fec913c6640 1 -- 172.21.15.3:0/3312806097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8c1a0630 con 0x7fec8c1094f0 2024-09-21T06:24:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.917+0000 7fec913c6640 1 -- 172.21.15.3:0/3312806097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec8c1a6e50 con 0x7fec8c1094f0 2024-09-21T06:24:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec9793a740 1 Processor -- start 2024-09-21T06:24:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec9793a740 1 -- start start 2024-09-21T06:24:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec73fff640 1 -- 172.21.15.3:0/3312806097 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fec8000fa20 con 0x7fec8c1094f0 2024-09-21T06:24:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec9793a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b1a8070 0x55873b54f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec9793a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b54fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec9793a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 0x55873b550120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec9793a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55873b4c5ea0 con 0x55873b17f1b0 2024-09-21T06:24:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec9793a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55873b4b52d0 con 0x55873b1a8070 2024-09-21T06:24:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec9793a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55873b4b90e0 con 0x55873b187390 2024-09-21T06:24:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec8affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b54fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec8affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b54fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57762/0 (socket says 172.21.15.3:57762) 2024-09-21T06:24:18.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec8affd640 1 -- 172.21.15.3:0/2778090607 learned_addr learned my addr 172.21.15.3:0/2778090607 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:18.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec8b7fe640 1 --2- 172.21.15.3:0/2778090607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b1a8070 0x55873b54f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec8a7fc640 1 --2- 172.21.15.3:0/2778090607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 0x55873b550120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec8affd640 1 -- 172.21.15.3:0/2778090607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 msgr2=0x55873b550120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:18.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec8affd640 1 --2- 172.21.15.3:0/2778090607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 0x55873b550120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec8affd640 1 -- 172.21.15.3:0/2778090607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b1a8070 msgr2=0x55873b54f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:18.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.920+0000 7fec8affd640 1 --2- 172.21.15.3:0/2778090607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b1a8070 0x55873b54f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.921+0000 7fec8affd640 1 -- 172.21.15.3:0/2778090607 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873b213650 con 0x55873b17f1b0 2024-09-21T06:24:18.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.921+0000 7fec8affd640 1 --2- 172.21.15.3:0/2778090607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b54fbe0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009c90 tx=0x7fec7c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6895bdeadbac5f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:18.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.921+0000 7fec717fa640 1 -- 172.21.15.3:0/2778090607 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c012450 con 0x55873b17f1b0 2024-09-21T06:24:18.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.921+0000 7fec717fa640 1 -- 172.21.15.3:0/2778090607 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec7c012aa0 con 0x55873b17f1b0 2024-09-21T06:24:18.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.921+0000 7fec717fa640 1 -- 172.21.15.3:0/2778090607 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c00fae0 con 0x55873b17f1b0 2024-09-21T06:24:18.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.921+0000 7fec73fff640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fec740b66b0 0x7fec740b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.921+0000 7fec8b7fe640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fec740b66b0 0x7fec740b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.921+0000 7fec73fff640 1 -- 172.21.15.3:0/3312806097 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fec800e7860 con 0x7fec8c1094f0 2024-09-21T06:24:18.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.922+0000 7fec8b7fe640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fec740b66b0 0x7fec740b8bc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fec84005cc0 tx=0x7fec84005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:18.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.922+0000 7fec9793a740 1 -- 172.21.15.3:0/2778090607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 msgr2=0x55873b54fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:18.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.922+0000 7fec9793a740 1 --2- 172.21.15.3:0/2778090607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b54fbe0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009c90 tx=0x7fec7c0117b0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.922+0000 7fec9793a740 1 -- 172.21.15.3:0/2778090607 shutdown_connections 2024-09-21T06:24:18.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.922+0000 7fec9793a740 1 --2- 172.21.15.3:0/2778090607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 0x55873b550120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.922+0000 7fec9793a740 1 --2- 172.21.15.3:0/2778090607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b54fbe0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.922+0000 7fec9793a740 1 --2- 172.21.15.3:0/2778090607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b1a8070 0x55873b54f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.922+0000 7fec9793a740 1 -- 172.21.15.3:0/2778090607 >> 172.21.15.3:0/2778090607 conn(0x55873b19c460 msgr2=0x55873b54d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:18.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.922+0000 7fec9793a740 1 -- 172.21.15.3:0/2778090607 shutdown_connections 2024-09-21T06:24:18.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.922+0000 7fec9793a740 1 -- 172.21.15.3:0/2778090607 wait complete. 2024-09-21T06:24:18.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.922+0000 7fec9793a740 1 -- start start 2024-09-21T06:24:18.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.922+0000 7fec9793a740 1 Processor -- start 2024-09-21T06:24:18.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec9793a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b55ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec9793a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b187390 0x55873b55b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec9793a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b1a8070 0x55873b55b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec9793a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55873b316700 con 0x55873b17f1b0 2024-09-21T06:24:18.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec9793a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55873b450160 con 0x55873b187390 2024-09-21T06:24:18.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec9793a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55873b55de80 con 0x55873b1a8070 2024-09-21T06:24:18.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec9793a740 10 client.0 _ll_register_callbacks cb 0x55873b4699e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:18.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec8affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b55ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec8affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b55ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57764/0 (socket says 172.21.15.3:57764) 2024-09-21T06:24:18.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec8affd640 1 -- 172.21.15.3:0/3507221354 learned_addr learned my addr 172.21.15.3:0/3507221354 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:18.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec8a7fc640 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b187390 0x55873b55b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec8b7fe640 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b1a8070 0x55873b55b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec8affd640 1 -- 172.21.15.3:0/3507221354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b1a8070 msgr2=0x55873b55b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:18.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec8affd640 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b1a8070 0x55873b55b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec8affd640 1 -- 172.21.15.3:0/3507221354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b187390 msgr2=0x55873b55b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:18.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec8affd640 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b187390 0x55873b55b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec8affd640 1 -- 172.21.15.3:0/3507221354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873b47dc90 con 0x55873b17f1b0 2024-09-21T06:24:18.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec8a7fc640 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b187390 0x55873b55b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:18.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec8affd640 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b55ad20 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00f480 tx=0x7fec7c009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85d32981bd7c5f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.923+0000 7fec6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec6eff5640 1 -- 172.21.15.3:0/3507221354 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c029030 con 0x55873b17f1b0 2024-09-21T06:24:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec6eff5640 1 -- 172.21.15.3:0/3507221354 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fec7c020040 con 0x55873b17f1b0 2024-09-21T06:24:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec9793a740 1 -- 172.21.15.3:0/3507221354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec7c012aa0 con 0x55873b17f1b0 2024-09-21T06:24:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec6eff5640 1 -- 172.21.15.3:0/3507221354 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fec7c033050 con 0x55873b17f1b0 2024-09-21T06:24:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec9793a740 10 client.18126 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec9793a740 1 -- 172.21.15.3:0/3507221354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55873b569b30 con 0x55873b17f1b0 2024-09-21T06:24:18.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec6eff5640 1 -- 172.21.15.3:0/3507221354 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec7c012aa0 con 0x55873b17f1b0 2024-09-21T06:24:18.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec6eff5640 1 -- 172.21.15.3:0/3507221354 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fec7c0046c0 con 0x55873b17f1b0 2024-09-21T06:24:18.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec9793a740 10 client.18126 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:18.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec9793a740 10 client.18126 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:24:18.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec9793a740 20 client.18126 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:18.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec9793a740 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55873b1669f0 0x55873b56cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec9793a740 4 client.18126 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.924+0000 7fec9793a740 1 -- 172.21.15.3:0/3507221354 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55873b4c7df0 con 0x55873b1669f0 2024-09-21T06:24:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.925+0000 7fec8a7fc640 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55873b1669f0 0x55873b56cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.925+0000 7fec6eff5640 10 client.18126 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:24:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.925+0000 7fec8a7fc640 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55873b1669f0 0x55873b56cfa0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6efdd2c7f877362 server_cookie=7963cd01a8c30d57 in_seq=0 out_seq=0 2024-09-21T06:24:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec6eff5640 1 -- 172.21.15.3:0/3507221354 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55873b4c7df0 con 0x55873b1669f0 2024-09-21T06:24:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec6eff5640 10 client.18126 handle_command_reply: tid=0 2024-09-21T06:24:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec9793a740 1 client.18126 shutdown 2024-09-21T06:24:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec9793a740 1 -- 172.21.15.3:0/3507221354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 msgr2=0x55873b55ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec9793a740 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b55ad20 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00f480 tx=0x7fec7c009fb0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec9793a740 1 -- 172.21.15.3:0/3507221354 shutdown_connections 2024-09-21T06:24:18.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec9793a740 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55873b1669f0 0x55873b56cfa0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec9793a740 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b1a8070 0x55873b55b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec9793a740 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b187390 0x55873b55b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec6eff5640 0 client.18126 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:24:18.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec9793a740 1 --2- 172.21.15.3:0/3507221354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b55ad20 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec9793a740 1 -- 172.21.15.3:0/3507221354 >> 172.21.15.3:0/3507221354 conn(0x55873b19c460 msgr2=0x55873b552830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:18.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec9793a740 1 -- 172.21.15.3:0/3507221354 shutdown_connections 2024-09-21T06:24:18.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.926+0000 7fec9793a740 1 -- 172.21.15.3:0/3507221354 wait complete. 2024-09-21T06:24:18.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.927+0000 7fec9793a740 20 client.18126 trim_cache size 0 max 16384 2024-09-21T06:24:18.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.940+0000 7fec9793a740 1 Processor -- start 2024-09-21T06:24:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.940+0000 7fec9793a740 1 -- start start 2024-09-21T06:24:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.940+0000 7fec9793a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b55b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.940+0000 7fec9793a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 0x55873b54cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.940+0000 7fec9793a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b1a8070 0x55873b54d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.940+0000 7fec9793a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c1af660 con 0x55873b17f1b0 2024-09-21T06:24:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.940+0000 7fec9793a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c1b11a0 con 0x55873b1a8070 2024-09-21T06:24:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.940+0000 7fec9793a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c1afd30 con 0x55873b187390 2024-09-21T06:24:18.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec8affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b55b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec8affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b55b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57774/0 (socket says 172.21.15.3:57774) 2024-09-21T06:24:18.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec8affd640 1 -- 172.21.15.3:0/3575214010 learned_addr learned my addr 172.21.15.3:0/3575214010 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:18.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec8a7fc640 1 --2- 172.21.15.3:0/3575214010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 0x55873b54cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec8affd640 1 -- 172.21.15.3:0/3575214010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 msgr2=0x55873b54cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:18.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec8affd640 1 --2- 172.21.15.3:0/3575214010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 0x55873b54cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec8affd640 1 -- 172.21.15.3:0/3575214010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b1a8070 msgr2=0x55873b54d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:18.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec8b7fe640 1 --2- 172.21.15.3:0/3575214010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b1a8070 0x55873b54d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec8affd640 1 --2- 172.21.15.3:0/3575214010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b1a8070 0x55873b54d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec8affd640 1 -- 172.21.15.3:0/3575214010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873b54e3c0 con 0x55873b17f1b0 2024-09-21T06:24:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec8a7fc640 1 --2- 172.21.15.3:0/3575214010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 0x55873b54cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec8affd640 1 --2- 172.21.15.3:0/3575214010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b55b1e0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fec7c001eb0 tx=0x7fec7c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bae106792b8614c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec70ff9640 1 -- 172.21.15.3:0/3575214010 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c03c070 con 0x55873b17f1b0 2024-09-21T06:24:18.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec70ff9640 1 -- 172.21.15.3:0/3575214010 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec7c00f4b0 con 0x55873b17f1b0 2024-09-21T06:24:18.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.941+0000 7fec70ff9640 1 -- 172.21.15.3:0/3575214010 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c029070 con 0x55873b17f1b0 2024-09-21T06:24:18.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.942+0000 7fec9793a740 1 -- 172.21.15.3:0/3575214010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 msgr2=0x55873b55b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:18.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.942+0000 7fec9793a740 1 --2- 172.21.15.3:0/3575214010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b55b1e0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fec7c001eb0 tx=0x7fec7c009b60 comp rx=0 tx=0).stop 2024-09-21T06:24:18.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.942+0000 7fec9793a740 1 -- 172.21.15.3:0/3575214010 shutdown_connections 2024-09-21T06:24:18.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.942+0000 7fec9793a740 1 --2- 172.21.15.3:0/3575214010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 0x55873b54cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.942+0000 7fec9793a740 1 --2- 172.21.15.3:0/3575214010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b1a8070 0x55873b54d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.942+0000 7fec9793a740 1 --2- 172.21.15.3:0/3575214010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b17f1b0 0x55873b55b1e0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.942+0000 7fec9793a740 1 -- 172.21.15.3:0/3575214010 >> 172.21.15.3:0/3575214010 conn(0x55873b19c460 msgr2=0x55873b55aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:18.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.942+0000 7fec9793a740 1 -- 172.21.15.3:0/3575214010 shutdown_connections 2024-09-21T06:24:18.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.942+0000 7fec9793a740 1 -- 172.21.15.3:0/3575214010 wait complete. 2024-09-21T06:24:18.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec9793a740 1 -- start start 2024-09-21T06:24:18.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec9793a740 1 Processor -- start 2024-09-21T06:24:18.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec9793a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b17f1b0 0x55873b55f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec9793a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 0x55873b55f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec9793a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b1a8070 0x55873b55fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec9793a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c1b0ad0 con 0x55873b1a8070 2024-09-21T06:24:18.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec9793a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c1b0400 con 0x55873b17f1b0 2024-09-21T06:24:18.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec9793a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c1ad660 con 0x55873b187390 2024-09-21T06:24:18.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec9793a740 10 client.0 _ll_register_callbacks cb 0x7fec7c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:18.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec8b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b1a8070 0x55873b55fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec8b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b1a8070 0x55873b55fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57776/0 (socket says 172.21.15.3:57776) 2024-09-21T06:24:18.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec8b7fe640 1 -- 172.21.15.3:0/3501661503 learned_addr learned my addr 172.21.15.3:0/3501661503 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:18.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec8a7fc640 1 --2- 172.21.15.3:0/3501661503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 0x55873b55f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.943+0000 7fec8affd640 1 --2- 172.21.15.3:0/3501661503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b17f1b0 0x55873b55f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.944+0000 7fec8b7fe640 1 -- 172.21.15.3:0/3501661503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 msgr2=0x55873b55f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.944+0000 7fec8b7fe640 1 --2- 172.21.15.3:0/3501661503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 0x55873b55f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.944+0000 7fec8b7fe640 1 -- 172.21.15.3:0/3501661503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b17f1b0 msgr2=0x55873b55f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.944+0000 7fec8b7fe640 1 --2- 172.21.15.3:0/3501661503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b17f1b0 0x55873b55f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.944+0000 7fec8b7fe640 1 -- 172.21.15.3:0/3501661503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873b5670a0 con 0x55873b1a8070 2024-09-21T06:24:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.944+0000 7fec8b7fe640 1 --2- 172.21.15.3:0/3501661503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b1a8070 0x55873b55fbd0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fec84031590 tx=0x7fec8402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a59221e7a3a3777 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.944+0000 7fec6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.944+0000 7fec6eff5640 1 -- 172.21.15.3:0/3501661503 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8402f9f0 con 0x55873b1a8070 2024-09-21T06:24:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.944+0000 7fec6eff5640 1 -- 172.21.15.3:0/3501661503 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fec84042030 con 0x55873b1a8070 2024-09-21T06:24:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.944+0000 7fec9793a740 1 -- 172.21.15.3:0/3501661503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec7c00f4b0 con 0x55873b1a8070 2024-09-21T06:24:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.944+0000 7fec6eff5640 1 -- 172.21.15.3:0/3501661503 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fec8402fb90 con 0x55873b1a8070 2024-09-21T06:24:18.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.944+0000 7fec9793a740 10 client.18132 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:18.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.944+0000 7fec9793a740 1 -- 172.21.15.3:0/3501661503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55873b5d7900 con 0x55873b1a8070 2024-09-21T06:24:18.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.945+0000 7fec6eff5640 1 -- 172.21.15.3:0/3501661503 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec7c00f4b0 con 0x55873b1a8070 2024-09-21T06:24:18.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.945+0000 7fec6eff5640 1 -- 172.21.15.3:0/3501661503 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fec84053420 con 0x55873b1a8070 2024-09-21T06:24:18.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.945+0000 7fec9793a740 10 client.18132 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:18.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.945+0000 7fec9793a740 10 client.18132 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:24:18.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.945+0000 7fec9793a740 20 client.18132 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:18.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.945+0000 7fec9793a740 1 --2- 172.21.15.3:0/3501661503 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55873b13bda0 0x55873b5da150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:18.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.945+0000 7fec9793a740 4 client.18132 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:24:18.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.945+0000 7fec9793a740 1 -- 172.21.15.3:0/3501661503 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55873b4c38a0 con 0x55873b13bda0 2024-09-21T06:24:18.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.945+0000 7fec8affd640 1 --2- 172.21.15.3:0/3501661503 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55873b13bda0 0x55873b5da150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:18.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec6eff5640 10 client.18132 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:24:18.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec8affd640 1 --2- 172.21.15.3:0/3501661503 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55873b13bda0 0x55873b5da150 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c0e9685333e41ef server_cookie=8d48592adc8d23e5 in_seq=0 out_seq=0 2024-09-21T06:24:18.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec6eff5640 1 -- 172.21.15.3:0/3501661503 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55873b4c38a0 con 0x55873b13bda0 2024-09-21T06:24:18.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec6eff5640 10 client.18132 handle_command_reply: tid=0 2024-09-21T06:24:18.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec9793a740 1 client.18132 shutdown 2024-09-21T06:24:18.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec9793a740 1 -- 172.21.15.3:0/3501661503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b1a8070 msgr2=0x55873b55fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:18.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec9793a740 1 --2- 172.21.15.3:0/3501661503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b1a8070 0x55873b55fbd0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fec84031590 tx=0x7fec8402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec9793a740 1 -- 172.21.15.3:0/3501661503 shutdown_connections 2024-09-21T06:24:18.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec9793a740 1 --2- 172.21.15.3:0/3501661503 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55873b13bda0 0x55873b5da150 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec9793a740 1 --2- 172.21.15.3:0/3501661503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b187390 0x55873b55f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec9793a740 1 --2- 172.21.15.3:0/3501661503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55873b1a8070 0x55873b55fbd0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec9793a740 1 --2- 172.21.15.3:0/3501661503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55873b17f1b0 0x55873b55f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec9793a740 1 -- 172.21.15.3:0/3501661503 >> 172.21.15.3:0/3501661503 conn(0x55873b19c460 msgr2=0x55873b5506d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:18.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec6eff5640 0 client.18132 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:24:18.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec9793a740 1 -- 172.21.15.3:0/3501661503 shutdown_connections 2024-09-21T06:24:18.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec9793a740 1 -- 172.21.15.3:0/3501661503 wait complete. 2024-09-21T06:24:18.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.946+0000 7fec9793a740 20 client.18132 trim_cache size 0 max 16384 2024-09-21T06:24:18.965 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:24:18.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.947+0000 7fec913c6640 1 -- 172.21.15.3:0/3312806097 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fec740b66b0 msgr2=0x7fec740b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:18.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.947+0000 7fec913c6640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fec740b66b0 0x7fec740b8bc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fec84005cc0 tx=0x7fec84005c30 comp rx=0 tx=0).stop 2024-09-21T06:24:18.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.947+0000 7fec913c6640 1 -- 172.21.15.3:0/3312806097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec8c1094f0 msgr2=0x7fec8c19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:18.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.947+0000 7fec913c6640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec8c1094f0 0x7fec8c19aa30 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fec8000c8c0 tx=0x7fec8000cd80 comp rx=0 tx=0).stop 2024-09-21T06:24:18.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.947+0000 7fec913c6640 1 -- 172.21.15.3:0/3312806097 shutdown_connections 2024-09-21T06:24:18.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.947+0000 7fec913c6640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fec740b66b0 0x7fec740b8bc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.947+0000 7fec913c6640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec8c109e40 0x7fec8c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.947+0000 7fec913c6640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec8c1094f0 0x7fec8c19aa30 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.947+0000 7fec913c6640 1 --2- 172.21.15.3:0/3312806097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec8c103bb0 0x7fec8c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:18.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.947+0000 7fec913c6640 1 -- 172.21.15.3:0/3312806097 >> 172.21.15.3:0/3312806097 conn(0x7fec8c0ff600 msgr2=0x7fec8c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:18.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.947+0000 7fec913c6640 1 -- 172.21.15.3:0/3312806097 shutdown_connections 2024-09-21T06:24:18.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:18.947+0000 7fec913c6640 1 -- 172.21.15.3:0/3312806097 wait complete. 2024-09-21T06:24:19.703 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:19 smithi003 ceph-mon[31227]: pgmap v612: 97 pgs: 97 active+clean; 141 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.3 MiB/s wr, 59 op/s 2024-09-21T06:24:19.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:24:19.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:19 smithi137 ceph-mon[34464]: pgmap v612: 97 pgs: 97 active+clean; 141 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.3 MiB/s wr, 59 op/s 2024-09-21T06:24:19.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:19 smithi204 ceph-mon[34659]: pgmap v612: 97 pgs: 97 active+clean; 141 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.3 MiB/s wr, 59 op/s 2024-09-21T06:24:19.992 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:20.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.473+0000 7fe2b016a640 1 -- 172.21.15.3:0/2841200756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a8108d70 msgr2=0x7fe2a810f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.473+0000 7fe2b016a640 1 --2- 172.21.15.3:0/2841200756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a8108d70 0x7fe2a810f5a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009ca0 tx=0x7fe2a002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.473+0000 7fe2b016a640 1 -- 172.21.15.3:0/2841200756 shutdown_connections 2024-09-21T06:24:20.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.473+0000 7fe2b016a640 1 --2- 172.21.15.3:0/2841200756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2a8108420 0x7fe2a8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.473+0000 7fe2b016a640 1 --2- 172.21.15.3:0/2841200756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a8108d70 0x7fe2a810f5a0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.473+0000 7fe2b016a640 1 --2- 172.21.15.3:0/2841200756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a8102ae0 0x7fe2a8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.473+0000 7fe2b016a640 1 -- 172.21.15.3:0/2841200756 >> 172.21.15.3:0/2841200756 conn(0x7fe2a8079770 msgr2=0x7fe2a8079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.474+0000 7fe2b016a640 1 -- 172.21.15.3:0/2841200756 shutdown_connections 2024-09-21T06:24:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.474+0000 7fe2b016a640 1 -- 172.21.15.3:0/2841200756 wait complete. 2024-09-21T06:24:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.474+0000 7fe2b016a640 1 Processor -- start 2024-09-21T06:24:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.474+0000 7fe2b016a640 1 -- start start 2024-09-21T06:24:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2b016a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2a8102ae0 0x7fe2a819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2b016a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a8108420 0x7fe2a819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2b016a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a8108d70 0x7fe2a819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2b016a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a8112a30 con 0x7fe2a8108420 2024-09-21T06:24:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2b016a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a81128b0 con 0x7fe2a8108d70 2024-09-21T06:24:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2b016a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a8112bb0 con 0x7fe2a8102ae0 2024-09-21T06:24:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2ad705640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a8108420 0x7fe2a819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2ad705640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a8108420 0x7fe2a819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57804/0 (socket says 172.21.15.3:57804) 2024-09-21T06:24:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2ad705640 1 -- 172.21.15.3:0/2939722002 learned_addr learned my addr 172.21.15.3:0/2939722002 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2adf06640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2a8102ae0 0x7fe2a819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2ae707640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a8108d70 0x7fe2a819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2ad705640 1 -- 172.21.15.3:0/2939722002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2a8102ae0 msgr2=0x7fe2a819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2ad705640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2a8102ae0 0x7fe2a819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2ad705640 1 -- 172.21.15.3:0/2939722002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a8108d70 msgr2=0x7fe2a819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2ad705640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a8108d70 0x7fe2a819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2ad705640 1 -- 172.21.15.3:0/2939722002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a81a0340 con 0x7fe2a8108420 2024-09-21T06:24:20.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe2ad705640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a8108420 0x7fe2a819aa30 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe29800c970 tx=0x7fe29800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:20.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe29effd640 1 -- 172.21.15.3:0/2939722002 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298004220 con 0x7fe2a8108420 2024-09-21T06:24:20.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.475+0000 7fe29effd640 1 -- 172.21.15.3:0/2939722002 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2980043c0 con 0x7fe2a8108420 2024-09-21T06:24:20.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.476+0000 7fe29effd640 1 -- 172.21.15.3:0/2939722002 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298004a00 con 0x7fe2a8108420 2024-09-21T06:24:20.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.476+0000 7fe2b016a640 1 -- 172.21.15.3:0/2939722002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a81a0630 con 0x7fe2a8108420 2024-09-21T06:24:20.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.476+0000 7fe2b016a640 1 -- 172.21.15.3:0/2939722002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a81a0b20 con 0x7fe2a8108420 2024-09-21T06:24:20.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.478+0000 7fe2b66de740 1 Processor -- start 2024-09-21T06:24:20.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.478+0000 7fe2b66de740 1 -- start start 2024-09-21T06:24:20.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe29effd640 1 -- 172.21.15.3:0/2939722002 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fe2980079b0 con 0x7fe2a8108420 2024-09-21T06:24:20.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2b66de740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db884b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2b66de740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 0x563db884b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2b66de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db884bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2b66de740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563db87c1ea0 con 0x563db84a4070 2024-09-21T06:24:20.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2b66de740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563db87b12d0 con 0x563db847b1b0 2024-09-21T06:24:20.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2b66de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563db87b50e0 con 0x563db8483390 2024-09-21T06:24:20.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2ae707640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db884b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2ae707640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db884b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57812/0 (socket says 172.21.15.3:57812) 2024-09-21T06:24:20.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2ae707640 1 -- 172.21.15.3:0/192378225 learned_addr learned my addr 172.21.15.3:0/192378225 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:20.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2ad705640 1 --2- 172.21.15.3:0/192378225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db884bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2adf06640 1 --2- 172.21.15.3:0/192378225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 0x563db884b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2ae707640 1 -- 172.21.15.3:0/192378225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 msgr2=0x563db884bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2ae707640 1 --2- 172.21.15.3:0/192378225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db884bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2ae707640 1 -- 172.21.15.3:0/192378225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 msgr2=0x563db884b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2ae707640 1 --2- 172.21.15.3:0/192378225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 0x563db884b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2ae707640 1 -- 172.21.15.3:0/192378225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563db850f650 con 0x563db84a4070 2024-09-21T06:24:20.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2ad705640 1 --2- 172.21.15.3:0/192378225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db884bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:20.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.479+0000 7fe2adf06640 1 --2- 172.21.15.3:0/192378225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 0x563db884b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:20.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.480+0000 7fe2ae707640 1 --2- 172.21.15.3:0/192378225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db884b2c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fe2a002f4c0 tx=0x7fe2a0031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcdf6fbf6edb14d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:20.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.480+0000 7fe29effd640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2940b66b0 0x7fe2940b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.480+0000 7fe293fff640 1 -- 172.21.15.3:0/192378225 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0031450 con 0x563db84a4070 2024-09-21T06:24:20.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.480+0000 7fe293fff640 1 -- 172.21.15.3:0/192378225 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a0036030 con 0x563db84a4070 2024-09-21T06:24:20.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.480+0000 7fe293fff640 1 -- 172.21.15.3:0/192378225 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0004380 con 0x563db84a4070 2024-09-21T06:24:20.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.480+0000 7fe2adf06640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2940b66b0 0x7fe2940b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.480+0000 7fe29effd640 1 -- 172.21.15.3:0/2939722002 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fe2980e7330 con 0x7fe2a8108420 2024-09-21T06:24:20.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.481+0000 7fe2b66de740 1 -- 172.21.15.3:0/192378225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 msgr2=0x563db884b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.481+0000 7fe2b66de740 1 --2- 172.21.15.3:0/192378225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db884b2c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fe2a002f4c0 tx=0x7fe2a0031af0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.481+0000 7fe2adf06640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2940b66b0 0x7fe2940b8bc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x563db8849f40 tx=0x7fe2a400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:20.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.481+0000 7fe2b66de740 1 -- 172.21.15.3:0/192378225 shutdown_connections 2024-09-21T06:24:20.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.481+0000 7fe2b66de740 1 --2- 172.21.15.3:0/192378225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db884bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.481+0000 7fe2b66de740 1 --2- 172.21.15.3:0/192378225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 0x563db884b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.481+0000 7fe2b66de740 1 --2- 172.21.15.3:0/192378225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db884b2c0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.481+0000 7fe2b66de740 1 -- 172.21.15.3:0/192378225 >> 172.21.15.3:0/192378225 conn(0x563db8498460 msgr2=0x563db88491b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:20.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.481+0000 7fe2b66de740 1 -- 172.21.15.3:0/192378225 shutdown_connections 2024-09-21T06:24:20.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.481+0000 7fe2b66de740 1 -- 172.21.15.3:0/192378225 wait complete. 2024-09-21T06:24:20.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.482+0000 7fe2b66de740 1 -- start start 2024-09-21T06:24:20.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.482+0000 7fe2b66de740 1 Processor -- start 2024-09-21T06:24:20.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.482+0000 7fe2b66de740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 0x563db88569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.482+0000 7fe2b66de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db8856f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.482+0000 7fe2b66de740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db8862be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.482+0000 7fe2b66de740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563db8612700 con 0x563db84a4070 2024-09-21T06:24:20.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.482+0000 7fe2b66de740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563db874c160 con 0x563db847b1b0 2024-09-21T06:24:20.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.482+0000 7fe2b66de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563db88598c0 con 0x563db8483390 2024-09-21T06:24:20.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.482+0000 7fe2b66de740 10 client.0 _ll_register_callbacks cb 0x563db87659e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:20.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.482+0000 7fe2ae707640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db8862be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.482+0000 7fe2ae707640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db8862be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57820/0 (socket says 172.21.15.3:57820) 2024-09-21T06:24:20.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.482+0000 7fe2ae707640 1 -- 172.21.15.3:0/4045828477 learned_addr learned my addr 172.21.15.3:0/4045828477 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:20.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2ad705640 1 --2- 172.21.15.3:0/4045828477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db8856f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2adf06640 1 --2- 172.21.15.3:0/4045828477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 0x563db88569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2ae707640 1 -- 172.21.15.3:0/4045828477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 msgr2=0x563db8856f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2ae707640 1 --2- 172.21.15.3:0/4045828477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db8856f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2ae707640 1 -- 172.21.15.3:0/4045828477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 msgr2=0x563db88569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2ae707640 1 --2- 172.21.15.3:0/4045828477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 0x563db88569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2ae707640 1 -- 172.21.15.3:0/4045828477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563db8779c90 con 0x563db84a4070 2024-09-21T06:24:20.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2ae707640 1 --2- 172.21.15.3:0/4045828477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db8862be0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0004080 tx=0x7fe2a0031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0aa967fbc5677f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:20.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2917fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:20.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2917fa640 1 -- 172.21.15.3:0/4045828477 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a004a040 con 0x563db84a4070 2024-09-21T06:24:20.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2917fa640 1 -- 172.21.15.3:0/4045828477 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fe2a0004120 con 0x563db84a4070 2024-09-21T06:24:20.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2b66de740 1 -- 172.21.15.3:0/4045828477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2a0036030 con 0x563db84a4070 2024-09-21T06:24:20.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2917fa640 1 -- 172.21.15.3:0/4045828477 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe2a0004320 con 0x563db84a4070 2024-09-21T06:24:20.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2b66de740 10 client.18144 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:20.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.483+0000 7fe2b66de740 1 -- 172.21.15.3:0/4045828477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563db88655e0 con 0x563db84a4070 2024-09-21T06:24:20.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.484+0000 7fe2917fa640 1 -- 172.21.15.3:0/4045828477 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a0036030 con 0x563db84a4070 2024-09-21T06:24:20.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.484+0000 7fe2917fa640 1 -- 172.21.15.3:0/4045828477 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fe2a004d030 con 0x563db84a4070 2024-09-21T06:24:20.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.484+0000 7fe2b66de740 10 client.18144 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:20.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.484+0000 7fe2b66de740 10 client.18144 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:24:20.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.484+0000 7fe2b66de740 20 client.18144 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:20.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.484+0000 7fe2b66de740 1 --2- 172.21.15.3:0/4045828477 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563db84629f0 0x563db8868940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.484+0000 7fe2b66de740 4 client.18144 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:20.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.484+0000 7fe2b66de740 1 -- 172.21.15.3:0/4045828477 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563db87c3df0 con 0x563db84629f0 2024-09-21T06:24:20.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.484+0000 7fe2adf06640 1 --2- 172.21.15.3:0/4045828477 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563db84629f0 0x563db8868940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.485+0000 7fe2917fa640 10 client.18144 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:24:20.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.485+0000 7fe2adf06640 1 --2- 172.21.15.3:0/4045828477 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563db84629f0 0x563db8868940 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2754ef4cd501fd41 server_cookie=6d5836f3bfa13bf in_seq=0 out_seq=0 2024-09-21T06:24:20.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2917fa640 1 -- 172.21.15.3:0/4045828477 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563db87c3df0 con 0x563db84629f0 2024-09-21T06:24:20.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2917fa640 10 client.18144 handle_command_reply: tid=0 2024-09-21T06:24:20.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2b66de740 1 client.18144 shutdown 2024-09-21T06:24:20.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2b66de740 1 -- 172.21.15.3:0/4045828477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 msgr2=0x563db8862be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2b66de740 1 --2- 172.21.15.3:0/4045828477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db8862be0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0004080 tx=0x7fe2a0031700 comp rx=0 tx=0).stop 2024-09-21T06:24:20.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2b66de740 1 -- 172.21.15.3:0/4045828477 shutdown_connections 2024-09-21T06:24:20.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2b66de740 1 --2- 172.21.15.3:0/4045828477 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563db84629f0 0x563db8868940 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2b66de740 1 --2- 172.21.15.3:0/4045828477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db8856f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2b66de740 1 --2- 172.21.15.3:0/4045828477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db8862be0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2b66de740 1 --2- 172.21.15.3:0/4045828477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 0x563db88569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2917fa640 0 client.18144 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:24:20.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2b66de740 1 -- 172.21.15.3:0/4045828477 >> 172.21.15.3:0/4045828477 conn(0x563db8498460 msgr2=0x563db884e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:20.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2b66de740 1 -- 172.21.15.3:0/4045828477 shutdown_connections 2024-09-21T06:24:20.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2b66de740 1 -- 172.21.15.3:0/4045828477 wait complete. 2024-09-21T06:24:20.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.486+0000 7fe2b66de740 20 client.18144 trim_cache size 0 max 16384 2024-09-21T06:24:20.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2b66de740 1 Processor -- start 2024-09-21T06:24:20.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2b66de740 1 -- start start 2024-09-21T06:24:20.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2b66de740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db847b1b0 0x563db884ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2b66de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db88585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2b66de740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db84a4070 0x563db8848760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2b66de740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a81af580 con 0x563db847b1b0 2024-09-21T06:24:20.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2b66de740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a81b10c0 con 0x563db84a4070 2024-09-21T06:24:20.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2b66de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a81afc50 con 0x563db8483390 2024-09-21T06:24:20.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2adf06640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db847b1b0 0x563db884ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2adf06640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db847b1b0 0x563db884ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57836/0 (socket says 172.21.15.3:57836) 2024-09-21T06:24:20.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2adf06640 1 -- 172.21.15.3:0/3527477284 learned_addr learned my addr 172.21.15.3:0/3527477284 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:20.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2ad705640 1 --2- 172.21.15.3:0/3527477284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db88585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2ae707640 1 --2- 172.21.15.3:0/3527477284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db84a4070 0x563db8848760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2adf06640 1 -- 172.21.15.3:0/3527477284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 msgr2=0x563db88585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.500+0000 7fe2adf06640 1 --2- 172.21.15.3:0/3527477284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db88585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.501+0000 7fe2adf06640 1 -- 172.21.15.3:0/3527477284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db84a4070 msgr2=0x563db8848760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.501+0000 7fe2adf06640 1 --2- 172.21.15.3:0/3527477284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db84a4070 0x563db8848760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.501+0000 7fe2adf06640 1 -- 172.21.15.3:0/3527477284 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563db8849f70 con 0x563db847b1b0 2024-09-21T06:24:20.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.501+0000 7fe2ad705640 1 --2- 172.21.15.3:0/3527477284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db88585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:20.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.501+0000 7fe2adf06640 1 --2- 172.21.15.3:0/3527477284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db847b1b0 0x563db884ccb0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4005940 tx=0x7fe2a4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec90a312e545a149 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:20.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.501+0000 7fe2937fe640 1 -- 172.21.15.3:0/3527477284 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a401eb70 con 0x563db847b1b0 2024-09-21T06:24:20.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.501+0000 7fe2937fe640 1 -- 172.21.15.3:0/3527477284 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a401ed10 con 0x563db847b1b0 2024-09-21T06:24:20.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.501+0000 7fe2937fe640 1 -- 172.21.15.3:0/3527477284 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4004d80 con 0x563db847b1b0 2024-09-21T06:24:20.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.502+0000 7fe2b66de740 1 -- 172.21.15.3:0/3527477284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db847b1b0 msgr2=0x563db884ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.502+0000 7fe2b66de740 1 --2- 172.21.15.3:0/3527477284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db847b1b0 0x563db884ccb0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4005940 tx=0x7fe2a4015000 comp rx=0 tx=0).stop 2024-09-21T06:24:20.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.502+0000 7fe2b66de740 1 -- 172.21.15.3:0/3527477284 shutdown_connections 2024-09-21T06:24:20.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.502+0000 7fe2b66de740 1 --2- 172.21.15.3:0/3527477284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db88585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.502+0000 7fe2b66de740 1 --2- 172.21.15.3:0/3527477284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db84a4070 0x563db8848760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.502+0000 7fe2b66de740 1 --2- 172.21.15.3:0/3527477284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db847b1b0 0x563db884ccb0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.502+0000 7fe2b66de740 1 -- 172.21.15.3:0/3527477284 >> 172.21.15.3:0/3527477284 conn(0x563db8498460 msgr2=0x563db8857ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:20.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.502+0000 7fe2b66de740 1 -- 172.21.15.3:0/3527477284 shutdown_connections 2024-09-21T06:24:20.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.502+0000 7fe2b66de740 1 -- 172.21.15.3:0/3527477284 wait complete. 2024-09-21T06:24:20.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.502+0000 7fe2b66de740 1 -- start start 2024-09-21T06:24:20.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.502+0000 7fe2b66de740 1 Processor -- start 2024-09-21T06:24:20.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.503+0000 7fe2b66de740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 0x563db885f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.503+0000 7fe2b66de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db885a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.503+0000 7fe2b66de740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db885ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.503+0000 7fe2b66de740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a81b09f0 con 0x563db84a4070 2024-09-21T06:24:20.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.503+0000 7fe2b66de740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a81b0320 con 0x563db847b1b0 2024-09-21T06:24:20.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.503+0000 7fe2b66de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a81ad580 con 0x563db8483390 2024-09-21T06:24:20.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.503+0000 7fe2b66de740 10 client.0 _ll_register_callbacks cb 0x7fe2a002f410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:20.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.503+0000 7fe2ae707640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db885ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.503+0000 7fe2ae707640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db885ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57848/0 (socket says 172.21.15.3:57848) 2024-09-21T06:24:20.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.503+0000 7fe2ae707640 1 -- 172.21.15.3:0/1481891369 learned_addr learned my addr 172.21.15.3:0/1481891369 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:20.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.503+0000 7fe2ad705640 1 --2- 172.21.15.3:0/1481891369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db885a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.503+0000 7fe2adf06640 1 --2- 172.21.15.3:0/1481891369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 0x563db885f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.504+0000 7fe2ae707640 1 -- 172.21.15.3:0/1481891369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 msgr2=0x563db885a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.504+0000 7fe2ae707640 1 --2- 172.21.15.3:0/1481891369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db885a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.504+0000 7fe2ae707640 1 -- 172.21.15.3:0/1481891369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 msgr2=0x563db885f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.504+0000 7fe2ae707640 1 --2- 172.21.15.3:0/1481891369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 0x563db885f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.504+0000 7fe2ae707640 1 -- 172.21.15.3:0/1481891369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563db885b4a0 con 0x563db84a4070 2024-09-21T06:24:20.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.504+0000 7fe2ae707640 1 --2- 172.21.15.3:0/1481891369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db885ad10 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe2a002ade0 tx=0x7fe2a00319e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc232d15a264203 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:20.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.504+0000 7fe2917fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:20.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.504+0000 7fe2917fa640 1 -- 172.21.15.3:0/1481891369 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0003c80 con 0x563db84a4070 2024-09-21T06:24:20.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.504+0000 7fe2917fa640 1 -- 172.21.15.3:0/1481891369 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fe2a0003e20 con 0x563db84a4070 2024-09-21T06:24:20.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.504+0000 7fe2b66de740 1 -- 172.21.15.3:0/1481891369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2a401ed10 con 0x563db84a4070 2024-09-21T06:24:20.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.505+0000 7fe2917fa640 1 -- 172.21.15.3:0/1481891369 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe2a00044c0 con 0x563db84a4070 2024-09-21T06:24:20.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.505+0000 7fe2b66de740 10 client.18150 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.505+0000 7fe2b66de740 1 -- 172.21.15.3:0/1481891369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563db88d7440 con 0x563db84a4070 2024-09-21T06:24:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.505+0000 7fe2917fa640 1 -- 172.21.15.3:0/1481891369 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a401ed10 con 0x563db84a4070 2024-09-21T06:24:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.505+0000 7fe2917fa640 1 -- 172.21.15.3:0/1481891369 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fe2a006a030 con 0x563db84a4070 2024-09-21T06:24:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.505+0000 7fe2b66de740 10 client.18150 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.505+0000 7fe2b66de740 10 client.18150 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:24:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.505+0000 7fe2b66de740 20 client.18150 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.505+0000 7fe2b66de740 1 --2- 172.21.15.3:0/1481891369 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563db8437da0 0x563db88d9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.505+0000 7fe2b66de740 4 client.18150 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:24:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.505+0000 7fe2b66de740 1 -- 172.21.15.3:0/1481891369 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563db87bf8a0 con 0x563db8437da0 2024-09-21T06:24:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.505+0000 7fe2adf06640 1 --2- 172.21.15.3:0/1481891369 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563db8437da0 0x563db88d9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2917fa640 10 client.18150 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:24:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2adf06640 1 --2- 172.21.15.3:0/1481891369 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563db8437da0 0x563db88d9a90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bdc28f16e978f17c server_cookie=c929c48d05aa5697 in_seq=0 out_seq=0 2024-09-21T06:24:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2917fa640 1 -- 172.21.15.3:0/1481891369 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563db87bf8a0 con 0x563db8437da0 2024-09-21T06:24:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2917fa640 10 client.18150 handle_command_reply: tid=0 2024-09-21T06:24:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2b66de740 1 client.18150 shutdown 2024-09-21T06:24:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2b66de740 1 -- 172.21.15.3:0/1481891369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 msgr2=0x563db885ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2b66de740 1 --2- 172.21.15.3:0/1481891369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db885ad10 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe2a002ade0 tx=0x7fe2a00319e0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2b66de740 1 -- 172.21.15.3:0/1481891369 shutdown_connections 2024-09-21T06:24:20.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2b66de740 1 --2- 172.21.15.3:0/1481891369 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x563db8437da0 0x563db88d9a90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2b66de740 1 --2- 172.21.15.3:0/1481891369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563db8483390 0x563db885a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2b66de740 1 --2- 172.21.15.3:0/1481891369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563db84a4070 0x563db885ad10 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2917fa640 0 client.18150 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:24:20.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2b66de740 1 --2- 172.21.15.3:0/1481891369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563db847b1b0 0x563db885f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2b66de740 1 -- 172.21.15.3:0/1481891369 >> 172.21.15.3:0/1481891369 conn(0x563db8498460 msgr2=0x563db8858760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:20.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2b66de740 1 -- 172.21.15.3:0/1481891369 shutdown_connections 2024-09-21T06:24:20.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.506+0000 7fe2b66de740 1 -- 172.21.15.3:0/1481891369 wait complete. 2024-09-21T06:24:20.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.507+0000 7fe2b66de740 20 client.18150 trim_cache size 0 max 16384 2024-09-21T06:24:20.529 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:24:20.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.507+0000 7fe2b016a640 1 -- 172.21.15.3:0/2939722002 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2940b66b0 msgr2=0x7fe2940b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.507+0000 7fe2b016a640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2940b66b0 0x7fe2940b8bc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x563db8849f40 tx=0x7fe2a400a2d0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.507+0000 7fe2b016a640 1 -- 172.21.15.3:0/2939722002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a8108420 msgr2=0x7fe2a819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.507+0000 7fe2b016a640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a8108420 0x7fe2a819aa30 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe29800c970 tx=0x7fe29800cd70 comp rx=0 tx=0).stop 2024-09-21T06:24:20.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.507+0000 7fe2b016a640 1 -- 172.21.15.3:0/2939722002 shutdown_connections 2024-09-21T06:24:20.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.507+0000 7fe2b016a640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fe2940b66b0 0x7fe2940b8bc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.507+0000 7fe2b016a640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a8108d70 0x7fe2a819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.507+0000 7fe2b016a640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a8108420 0x7fe2a819aa30 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.507+0000 7fe2b016a640 1 --2- 172.21.15.3:0/2939722002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2a8102ae0 0x7fe2a819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.507+0000 7fe2b016a640 1 -- 172.21.15.3:0/2939722002 >> 172.21.15.3:0/2939722002 conn(0x7fe2a8079770 msgr2=0x7fe2a8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:20.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.507+0000 7fe2b016a640 1 -- 172.21.15.3:0/2939722002 shutdown_connections 2024-09-21T06:24:20.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:20.507+0000 7fe2b016a640 1 -- 172.21.15.3:0/2939722002 wait complete. 2024-09-21T06:24:21.404 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:24:21.661 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:21.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:21 smithi003 ceph-mon[31227]: pgmap v613: 97 pgs: 97 active+clean; 143 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.3 MiB/s wr, 58 op/s 2024-09-21T06:24:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:21 smithi137 ceph-mon[34464]: pgmap v613: 97 pgs: 97 active+clean; 143 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.3 MiB/s wr, 58 op/s 2024-09-21T06:24:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:21 smithi204 ceph-mon[34659]: pgmap v613: 97 pgs: 97 active+clean; 143 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.3 MiB/s wr, 58 op/s 2024-09-21T06:24:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.142+0000 7f653663f640 1 -- 172.21.15.3:0/3354291982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6530105270 msgr2=0x7f653010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.142+0000 7f653663f640 1 --2- 172.21.15.3:0/3354291982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6530105270 0x7f653010d8b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f6518009900 tx=0x7f651802a410 comp rx=0 tx=0).stop 2024-09-21T06:24:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.142+0000 7f653663f640 1 -- 172.21.15.3:0/3354291982 shutdown_connections 2024-09-21T06:24:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.142+0000 7f653663f640 1 --2- 172.21.15.3:0/3354291982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f653010ddf0 0x7f6530110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.142+0000 7f653663f640 1 --2- 172.21.15.3:0/3354291982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6530105270 0x7f653010d8b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.142+0000 7f653663f640 1 --2- 172.21.15.3:0/3354291982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6530104840 0x7f6530104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.142+0000 7f653663f640 1 -- 172.21.15.3:0/3354291982 >> 172.21.15.3:0/3354291982 conn(0x7f6530079730 msgr2=0x7f6530079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.142+0000 7f653663f640 1 -- 172.21.15.3:0/3354291982 shutdown_connections 2024-09-21T06:24:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.143+0000 7f653663f640 1 -- 172.21.15.3:0/3354291982 wait complete. 2024-09-21T06:24:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.143+0000 7f653663f640 1 Processor -- start 2024-09-21T06:24:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.143+0000 7f653663f640 1 -- start start 2024-09-21T06:24:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.143+0000 7f653663f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6530104840 0x7f653019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.143+0000 7f653663f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6530105270 0x7f653019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.143+0000 7f653663f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f653010ddf0 0x7f653019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.143+0000 7f653663f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6530113a70 con 0x7f6530105270 2024-09-21T06:24:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.143+0000 7f653663f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65301138f0 con 0x7f653010ddf0 2024-09-21T06:24:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.143+0000 7f653663f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6530113bf0 con 0x7f6530104840 2024-09-21T06:24:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.144+0000 7f6527fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6530105270 0x7f653019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.144+0000 7f6527fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6530105270 0x7f653019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57868/0 (socket says 172.21.15.3:57868) 2024-09-21T06:24:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.144+0000 7f6527fff640 1 -- 172.21.15.3:0/2135799168 learned_addr learned my addr 172.21.15.3:0/2135799168 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.144+0000 7f652ffff640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6530104840 0x7f653019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.144+0000 7f6534bdc640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f653010ddf0 0x7f653019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.144+0000 7f6527fff640 1 -- 172.21.15.3:0/2135799168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6530104840 msgr2=0x7f653019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.144+0000 7f6527fff640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6530104840 0x7f653019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.144+0000 7f6527fff640 1 -- 172.21.15.3:0/2135799168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f653010ddf0 msgr2=0x7f653019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.144+0000 7f6527fff640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f653010ddf0 0x7f653019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.144+0000 7f6527fff640 1 -- 172.21.15.3:0/2135799168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65301a0370 con 0x7f6530105270 2024-09-21T06:24:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.144+0000 7f652ffff640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6530104840 0x7f653019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.144+0000 7f6527fff640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6530105270 0x7f653019aa60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6518006010 tx=0x7f6518004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.144+0000 7f652dffb640 1 -- 172.21.15.3:0/2135799168 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651803f030 con 0x7f6530105270 2024-09-21T06:24:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.145+0000 7f652dffb640 1 -- 172.21.15.3:0/2135799168 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6518003aa0 con 0x7f6530105270 2024-09-21T06:24:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.145+0000 7f652dffb640 1 -- 172.21.15.3:0/2135799168 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518035070 con 0x7f6530105270 2024-09-21T06:24:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.145+0000 7f653663f640 1 -- 172.21.15.3:0/2135799168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65301a0600 con 0x7f6530105270 2024-09-21T06:24:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.145+0000 7f653663f640 1 -- 172.21.15.3:0/2135799168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65301a0ae0 con 0x7f6530105270 2024-09-21T06:24:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.147+0000 7f653cbb3740 1 Processor -- start 2024-09-21T06:24:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.147+0000 7f653cbb3740 1 -- start start 2024-09-21T06:24:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.147+0000 7f652dffb640 1 -- 172.21.15.3:0/2135799168 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f6518042030 con 0x7f6530105270 2024-09-21T06:24:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.147+0000 7f653cbb3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a4420070 0x55d3a47c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.147+0000 7f653cbb3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.147+0000 7f653cbb3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a43ff390 0x55d3a47c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f653cbb3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d3a473dea0 con 0x55d3a43f71b0 2024-09-21T06:24:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f653cbb3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d3a472d2d0 con 0x55d3a43ff390 2024-09-21T06:24:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f653cbb3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d3a47310e0 con 0x55d3a4420070 2024-09-21T06:24:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f652ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f652ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47c7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57878/0 (socket says 172.21.15.3:57878) 2024-09-21T06:24:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f652ffff640 1 -- 172.21.15.3:0/1744064171 learned_addr learned my addr 172.21.15.3:0/1744064171 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f6534bdc640 1 --2- 172.21.15.3:0/1744064171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a4420070 0x55d3a47c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f6527fff640 1 --2- 172.21.15.3:0/1744064171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a43ff390 0x55d3a47c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f652ffff640 1 -- 172.21.15.3:0/1744064171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a4420070 msgr2=0x55d3a47c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f652ffff640 1 --2- 172.21.15.3:0/1744064171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a4420070 0x55d3a47c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f652ffff640 1 -- 172.21.15.3:0/1744064171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a43ff390 msgr2=0x55d3a47c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f652ffff640 1 --2- 172.21.15.3:0/1744064171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a43ff390 0x55d3a47c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f652ffff640 1 -- 172.21.15.3:0/1744064171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a448b650 con 0x55d3a43f71b0 2024-09-21T06:24:22.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f6527fff640 1 --2- 172.21.15.3:0/1744064171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a43ff390 0x55d3a47c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:22.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f652ffff640 1 --2- 172.21.15.3:0/1744064171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47c7be0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f65200114f0 tx=0x7f65200119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1551c282002b1719 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:22.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f65267fc640 1 -- 172.21.15.3:0/1744064171 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652001e070 con 0x55d3a43f71b0 2024-09-21T06:24:22.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f652dffb640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f651c0b66b0 0x7f651c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f65267fc640 1 -- 172.21.15.3:0/1744064171 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65200035f0 con 0x55d3a43f71b0 2024-09-21T06:24:22.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.148+0000 7f65267fc640 1 -- 172.21.15.3:0/1744064171 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652000f820 con 0x55d3a43f71b0 2024-09-21T06:24:22.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.149+0000 7f6534bdc640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f651c0b66b0 0x7f651c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.149+0000 7f652dffb640 1 -- 172.21.15.3:0/2135799168 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f6518107bd0 con 0x7f6530105270 2024-09-21T06:24:22.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.149+0000 7f653cbb3740 1 -- 172.21.15.3:0/1744064171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 msgr2=0x55d3a47c7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.149+0000 7f6534bdc640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f651c0b66b0 0x7f651c0b8bc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6528007f80 tx=0x7f6528007520 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:22.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.149+0000 7f653cbb3740 1 --2- 172.21.15.3:0/1744064171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47c7be0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f65200114f0 tx=0x7f65200119b0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.149+0000 7f653cbb3740 1 -- 172.21.15.3:0/1744064171 shutdown_connections 2024-09-21T06:24:22.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.149+0000 7f653cbb3740 1 --2- 172.21.15.3:0/1744064171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a43ff390 0x55d3a47c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.149+0000 7f653cbb3740 1 --2- 172.21.15.3:0/1744064171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47c7be0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.149+0000 7f653cbb3740 1 --2- 172.21.15.3:0/1744064171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a4420070 0x55d3a47c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.149+0000 7f653cbb3740 1 -- 172.21.15.3:0/1744064171 >> 172.21.15.3:0/1744064171 conn(0x55d3a4414460 msgr2=0x55d3a47c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:22.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.149+0000 7f653cbb3740 1 -- 172.21.15.3:0/1744064171 shutdown_connections 2024-09-21T06:24:22.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.149+0000 7f653cbb3740 1 -- 172.21.15.3:0/1744064171 wait complete. 2024-09-21T06:24:22.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.150+0000 7f653cbb3740 1 -- start start 2024-09-21T06:24:22.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.150+0000 7f653cbb3740 1 Processor -- start 2024-09-21T06:24:22.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.150+0000 7f653cbb3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.150+0000 7f653cbb3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43ff390 0x55d3a47d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.150+0000 7f653cbb3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 0x55d3a47d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.150+0000 7f653cbb3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d3a458e700 con 0x55d3a43f71b0 2024-09-21T06:24:22.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.150+0000 7f653cbb3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d3a46c8160 con 0x55d3a4420070 2024-09-21T06:24:22.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.150+0000 7f653cbb3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d3a47d5e80 con 0x55d3a43ff390 2024-09-21T06:24:22.191 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:24:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.150+0000 7f653cbb3740 10 client.0 _ll_register_callbacks cb 0x55d3a46e19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f652ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f652ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57888/0 (socket says 172.21.15.3:57888) 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f652ffff640 1 -- 172.21.15.3:0/2890637290 learned_addr learned my addr 172.21.15.3:0/2890637290 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f6527fff640 1 --2- 172.21.15.3:0/2890637290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43ff390 0x55d3a47d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f6534bdc640 1 --2- 172.21.15.3:0/2890637290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 0x55d3a47d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f652ffff640 1 -- 172.21.15.3:0/2890637290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43ff390 msgr2=0x55d3a47d3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f652ffff640 1 --2- 172.21.15.3:0/2890637290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43ff390 0x55d3a47d3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f652ffff640 1 -- 172.21.15.3:0/2890637290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 msgr2=0x55d3a47d3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f652ffff640 1 --2- 172.21.15.3:0/2890637290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 0x55d3a47d3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f652ffff640 1 -- 172.21.15.3:0/2890637290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a46f5c90 con 0x55d3a43f71b0 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f652ffff640 1 --2- 172.21.15.3:0/2890637290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47d2cd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f65200114c0 tx=0x7f6520009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a8df8a353694a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f650f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f650f7fe640 1 -- 172.21.15.3:0/2890637290 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520026020 con 0x55d3a43f71b0 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f650f7fe640 1 -- 172.21.15.3:0/2890637290 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f65200039e0 con 0x55d3a43f71b0 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f653cbb3740 1 -- 172.21.15.3:0/2890637290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65200035f0 con 0x55d3a43f71b0 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f650f7fe640 1 -- 172.21.15.3:0/2890637290 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6520031050 con 0x55d3a43f71b0 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.151+0000 7f653cbb3740 10 client.18162 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.152+0000 7f653cbb3740 1 -- 172.21.15.3:0/2890637290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d3a47e1ae0 con 0x55d3a43f71b0 2024-09-21T06:24:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.152+0000 7f650f7fe640 1 -- 172.21.15.3:0/2890637290 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65200035f0 con 0x55d3a43f71b0 2024-09-21T06:24:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.152+0000 7f650f7fe640 1 -- 172.21.15.3:0/2890637290 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f6520021030 con 0x55d3a43f71b0 2024-09-21T06:24:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.152+0000 7f653cbb3740 10 client.18162 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.152+0000 7f653cbb3740 10 client.18162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.152+0000 7f653cbb3740 20 client.18162 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.152+0000 7f653cbb3740 1 --2- 172.21.15.3:0/2890637290 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d3a43de9f0 0x55d3a47e4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.152+0000 7f653cbb3740 4 client.18162 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.152+0000 7f653cbb3740 1 -- 172.21.15.3:0/2890637290 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3a473fdf0 con 0x55d3a43de9f0 2024-09-21T06:24:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.152+0000 7f6527fff640 1 --2- 172.21.15.3:0/2890637290 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d3a43de9f0 0x55d3a47e4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.153+0000 7f650f7fe640 10 client.18162 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.153+0000 7f6527fff640 1 --2- 172.21.15.3:0/2890637290 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d3a43de9f0 0x55d3a47e4ef0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54786ada097fdb8 server_cookie=498f89efb714bfc8 in_seq=0 out_seq=0 2024-09-21T06:24:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.153+0000 7f650f7fe640 1 -- 172.21.15.3:0/2890637290 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d3a473fdf0 con 0x55d3a43de9f0 2024-09-21T06:24:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.153+0000 7f650f7fe640 10 client.18162 handle_command_reply: tid=0 2024-09-21T06:24:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.154+0000 7f653cbb3740 1 client.18162 shutdown 2024-09-21T06:24:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.154+0000 7f653cbb3740 1 -- 172.21.15.3:0/2890637290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 msgr2=0x55d3a47d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.154+0000 7f653cbb3740 1 --2- 172.21.15.3:0/2890637290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47d2cd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f65200114c0 tx=0x7f6520009da0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.154+0000 7f653cbb3740 1 -- 172.21.15.3:0/2890637290 shutdown_connections 2024-09-21T06:24:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.154+0000 7f653cbb3740 1 --2- 172.21.15.3:0/2890637290 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d3a43de9f0 0x55d3a47e4ef0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.154+0000 7f653cbb3740 1 --2- 172.21.15.3:0/2890637290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43ff390 0x55d3a47d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.154+0000 7f653cbb3740 1 --2- 172.21.15.3:0/2890637290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 0x55d3a47d3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.154+0000 7f653cbb3740 1 --2- 172.21.15.3:0/2890637290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47d2cd0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.154+0000 7f653cbb3740 1 -- 172.21.15.3:0/2890637290 >> 172.21.15.3:0/2890637290 conn(0x55d3a4414460 msgr2=0x55d3a47ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.154+0000 7f650f7fe640 0 client.18162 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.154+0000 7f653cbb3740 1 -- 172.21.15.3:0/2890637290 shutdown_connections 2024-09-21T06:24:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.154+0000 7f653cbb3740 1 -- 172.21.15.3:0/2890637290 wait complete. 2024-09-21T06:24:22.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.154+0000 7f653cbb3740 20 client.18162 trim_cache size 0 max 16384 2024-09-21T06:24:22.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.167+0000 7f653cbb3740 1 Processor -- start 2024-09-21T06:24:22.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f653cbb3740 1 -- start start 2024-09-21T06:24:22.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f653cbb3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f653cbb3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43ff390 0x55d3a47ea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f653cbb3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 0x55d3a47eaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f653cbb3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f65301af670 con 0x55d3a43f71b0 2024-09-21T06:24:22.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f653cbb3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65301b11b0 con 0x55d3a4420070 2024-09-21T06:24:22.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f653cbb3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65301afd40 con 0x55d3a43ff390 2024-09-21T06:24:22.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f652ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f652ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57890/0 (socket says 172.21.15.3:57890) 2024-09-21T06:24:22.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f652ffff640 1 -- 172.21.15.3:0/747790445 learned_addr learned my addr 172.21.15.3:0/747790445 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:22.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f6527fff640 1 --2- 172.21.15.3:0/747790445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43ff390 0x55d3a47ea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f652ffff640 1 -- 172.21.15.3:0/747790445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43ff390 msgr2=0x55d3a47ea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f652ffff640 1 --2- 172.21.15.3:0/747790445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43ff390 0x55d3a47ea550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f652ffff640 1 -- 172.21.15.3:0/747790445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 msgr2=0x55d3a47eaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f652ffff640 1 --2- 172.21.15.3:0/747790445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 0x55d3a47eaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f652ffff640 1 -- 172.21.15.3:0/747790445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a47d3d70 con 0x55d3a43f71b0 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f6527fff640 1 --2- 172.21.15.3:0/747790445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43ff390 0x55d3a47ea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f652ffff640 1 --2- 172.21.15.3:0/747790445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47c8de0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f6520009710 tx=0x7f6520003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc275a6d06b2b90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.168+0000 7f6525ffb640 1 -- 172.21.15.3:0/747790445 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520026040 con 0x55d3a43f71b0 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.169+0000 7f6525ffb640 1 -- 172.21.15.3:0/747790445 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6520004840 con 0x55d3a43f71b0 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.169+0000 7f6525ffb640 1 -- 172.21.15.3:0/747790445 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652001e040 con 0x55d3a43f71b0 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.169+0000 7f653cbb3740 1 -- 172.21.15.3:0/747790445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 msgr2=0x55d3a47c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.169+0000 7f653cbb3740 1 --2- 172.21.15.3:0/747790445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47c8de0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f6520009710 tx=0x7f6520003880 comp rx=0 tx=0).stop 2024-09-21T06:24:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.169+0000 7f653cbb3740 1 -- 172.21.15.3:0/747790445 shutdown_connections 2024-09-21T06:24:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.169+0000 7f653cbb3740 1 --2- 172.21.15.3:0/747790445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43ff390 0x55d3a47ea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.169+0000 7f653cbb3740 1 --2- 172.21.15.3:0/747790445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 0x55d3a47eaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.169+0000 7f653cbb3740 1 --2- 172.21.15.3:0/747790445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43f71b0 0x55d3a47c8de0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.169+0000 7f653cbb3740 1 -- 172.21.15.3:0/747790445 >> 172.21.15.3:0/747790445 conn(0x55d3a4414460 msgr2=0x55d3a47c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.169+0000 7f653cbb3740 1 -- 172.21.15.3:0/747790445 shutdown_connections 2024-09-21T06:24:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.169+0000 7f653cbb3740 1 -- 172.21.15.3:0/747790445 wait complete. 2024-09-21T06:24:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.170+0000 7f653cbb3740 1 -- start start 2024-09-21T06:24:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.170+0000 7f653cbb3740 1 Processor -- start 2024-09-21T06:24:22.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.170+0000 7f653cbb3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43f71b0 0x55d3a47d8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.170+0000 7f653cbb3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43ff390 0x55d3a47d9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.170+0000 7f653cbb3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 0x55d3a4847910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.170+0000 7f653cbb3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f65301b0ae0 con 0x55d3a43ff390 2024-09-21T06:24:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.170+0000 7f653cbb3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65301b0410 con 0x55d3a4420070 2024-09-21T06:24:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.170+0000 7f653cbb3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65301ad670 con 0x55d3a43f71b0 2024-09-21T06:24:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.170+0000 7f653cbb3740 10 client.0 _ll_register_callbacks cb 0x7f652001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.170+0000 7f6527fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43ff390 0x55d3a47d9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.170+0000 7f6527fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43ff390 0x55d3a47d9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57904/0 (socket says 172.21.15.3:57904) 2024-09-21T06:24:22.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.170+0000 7f6527fff640 1 -- 172.21.15.3:0/1793707568 learned_addr learned my addr 172.21.15.3:0/1793707568 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:22.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.170+0000 7f652ffff640 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43f71b0 0x55d3a47d8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f6534bdc640 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 0x55d3a4847910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f6527fff640 1 -- 172.21.15.3:0/1793707568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43f71b0 msgr2=0x55d3a47d8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f6527fff640 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43f71b0 0x55d3a47d8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f6527fff640 1 -- 172.21.15.3:0/1793707568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 msgr2=0x55d3a4847910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f6527fff640 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 0x55d3a4847910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f6527fff640 1 -- 172.21.15.3:0/1793707568 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a4847f80 con 0x55d3a43ff390 2024-09-21T06:24:22.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f6534bdc640 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 0x55d3a4847910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:22.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f6527fff640 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43ff390 0x55d3a47d9270 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f65180cd780 tx=0x7f651802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cd51c2c632308d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:22.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f650f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:22.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f650f7fe640 1 -- 172.21.15.3:0/1793707568 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518049040 con 0x55d3a43ff390 2024-09-21T06:24:22.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f650f7fe640 1 -- 172.21.15.3:0/1793707568 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f65180d6f60 con 0x55d3a43ff390 2024-09-21T06:24:22.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f653cbb3740 1 -- 172.21.15.3:0/1793707568 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6520004840 con 0x55d3a43ff390 2024-09-21T06:24:22.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f650f7fe640 1 -- 172.21.15.3:0/1793707568 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f65180c91f0 con 0x55d3a43ff390 2024-09-21T06:24:22.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f653cbb3740 10 client.18168 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:22.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f653cbb3740 1 -- 172.21.15.3:0/1793707568 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d3a484f370 con 0x55d3a43ff390 2024-09-21T06:24:22.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.171+0000 7f650f7fe640 1 -- 172.21.15.3:0/1793707568 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6520004840 con 0x55d3a43ff390 2024-09-21T06:24:22.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.172+0000 7f650f7fe640 1 -- 172.21.15.3:0/1793707568 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f65180ccce0 con 0x55d3a43ff390 2024-09-21T06:24:22.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.172+0000 7f653cbb3740 10 client.18168 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:22.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.172+0000 7f653cbb3740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:22.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.172+0000 7f653cbb3740 20 client.18168 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:22.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.172+0000 7f653cbb3740 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d3a43b3da0 0x55d3a4851ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.172+0000 7f653cbb3740 4 client.18168 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:24:22.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.172+0000 7f653cbb3740 1 -- 172.21.15.3:0/1793707568 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3a473b8a0 con 0x55d3a43b3da0 2024-09-21T06:24:22.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.172+0000 7f652ffff640 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d3a43b3da0 0x55d3a4851ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f652ffff640 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d3a43b3da0 0x55d3a4851ae0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=148213bb7db2fd63 server_cookie=eaea02cd2717795b in_seq=0 out_seq=0 2024-09-21T06:24:22.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f650f7fe640 10 client.18168 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:22.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f650f7fe640 1 -- 172.21.15.3:0/1793707568 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3a473b8a0 con 0x55d3a43b3da0 2024-09-21T06:24:22.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f650f7fe640 10 client.18168 handle_command_reply: tid=0 2024-09-21T06:24:22.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f653cbb3740 1 client.18168 shutdown 2024-09-21T06:24:22.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f653cbb3740 1 -- 172.21.15.3:0/1793707568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43ff390 msgr2=0x55d3a47d9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f653cbb3740 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43ff390 0x55d3a47d9270 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f65180cd780 tx=0x7f651802a410 comp rx=0 tx=0).stop 2024-09-21T06:24:22.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f653cbb3740 1 -- 172.21.15.3:0/1793707568 shutdown_connections 2024-09-21T06:24:22.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f653cbb3740 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55d3a43b3da0 0x55d3a4851ae0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f653cbb3740 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a4420070 0x55d3a4847910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f653cbb3740 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3a43ff390 0x55d3a47d9270 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f653cbb3740 1 --2- 172.21.15.3:0/1793707568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3a43f71b0 0x55d3a47d8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f653cbb3740 1 -- 172.21.15.3:0/1793707568 >> 172.21.15.3:0/1793707568 conn(0x55d3a4414460 msgr2=0x55d3a47de820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:22.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f650f7fe640 0 client.18168 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:22.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f653cbb3740 1 -- 172.21.15.3:0/1793707568 shutdown_connections 2024-09-21T06:24:22.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f653cbb3740 1 -- 172.21.15.3:0/1793707568 wait complete. 2024-09-21T06:24:22.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.173+0000 7f653cbb3740 20 client.18168 trim_cache size 0 max 16384 2024-09-21T06:24:22.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.174+0000 7f653663f640 1 -- 172.21.15.3:0/2135799168 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f651c0b66b0 msgr2=0x7f651c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.174+0000 7f653663f640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f651c0b66b0 0x7f651c0b8bc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6528007f80 tx=0x7f6528007520 comp rx=0 tx=0).stop 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.174+0000 7f653663f640 1 -- 172.21.15.3:0/2135799168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6530105270 msgr2=0x7f653019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.174+0000 7f653663f640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6530105270 0x7f653019aa60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6518006010 tx=0x7f6518004060 comp rx=0 tx=0).stop 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.174+0000 7f653663f640 1 -- 172.21.15.3:0/2135799168 shutdown_connections 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.174+0000 7f653663f640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f651c0b66b0 0x7f651c0b8bc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.174+0000 7f653663f640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f653010ddf0 0x7f653019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.174+0000 7f653663f640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6530105270 0x7f653019aa60 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.174+0000 7f653663f640 1 --2- 172.21.15.3:0/2135799168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6530104840 0x7f653019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.174+0000 7f653663f640 1 -- 172.21.15.3:0/2135799168 >> 172.21.15.3:0/2135799168 conn(0x7f6530079730 msgr2=0x7f6530100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.174+0000 7f653663f640 1 -- 172.21.15.3:0/2135799168 shutdown_connections 2024-09-21T06:24:22.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:22.174+0000 7f653663f640 1 -- 172.21.15.3:0/2135799168 wait complete. 2024-09-21T06:24:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:23 smithi003 ceph-mon[31227]: pgmap v614: 97 pgs: 97 active+clean; 142 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.4 MiB/s wr, 63 op/s 2024-09-21T06:24:23.878 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:24:23.879 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:24:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:23 smithi137 ceph-mon[34464]: pgmap v614: 97 pgs: 97 active+clean; 142 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.4 MiB/s wr, 63 op/s 2024-09-21T06:24:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:23 smithi204 ceph-mon[34659]: pgmap v614: 97 pgs: 97 active+clean; 142 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.4 MiB/s wr, 63 op/s 2024-09-21T06:24:24.154 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.747+0000 7f365c36b640 1 -- 172.21.15.3:0/3987521863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3654108f60 msgr2=0x7f365410f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.747+0000 7f365c36b640 1 --2- 172.21.15.3:0/3987521863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3654108f60 0x7f365410f790 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f364c009ca0 tx=0x7f364c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.747+0000 7f365c36b640 1 -- 172.21.15.3:0/3987521863 shutdown_connections 2024-09-21T06:24:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.747+0000 7f365c36b640 1 --2- 172.21.15.3:0/3987521863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3654108f60 0x7f365410f790 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.747+0000 7f365c36b640 1 --2- 172.21.15.3:0/3987521863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3654108610 0x7f3654108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.747+0000 7f365c36b640 1 --2- 172.21.15.3:0/3987521863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36541031d0 0x7f36541035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.747+0000 7f365c36b640 1 -- 172.21.15.3:0/3987521863 >> 172.21.15.3:0/3987521863 conn(0x7f3654079e40 msgr2=0x7f365407a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.747+0000 7f365c36b640 1 -- 172.21.15.3:0/3987521863 shutdown_connections 2024-09-21T06:24:24.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.747+0000 7f365c36b640 1 -- 172.21.15.3:0/3987521863 wait complete. 2024-09-21T06:24:24.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.748+0000 7f365c36b640 1 Processor -- start 2024-09-21T06:24:24.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.748+0000 7f365c36b640 1 -- start start 2024-09-21T06:24:24.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.748+0000 7f365c36b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36541031d0 0x7f36541995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.748+0000 7f365c36b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3654108610 0x7f3654199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.748+0000 7f365c36b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3654108f60 0x7f365419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.748+0000 7f365c36b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3654112c20 con 0x7f36541031d0 2024-09-21T06:24:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.748+0000 7f365c36b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3654112aa0 con 0x7f3654108f60 2024-09-21T06:24:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.748+0000 7f365c36b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3654112da0 con 0x7f3654108610 2024-09-21T06:24:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.748+0000 7f365a107640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36541031d0 0x7f36541995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.748+0000 7f365a107640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36541031d0 0x7f36541995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57922/0 (socket says 172.21.15.3:57922) 2024-09-21T06:24:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.748+0000 7f365a107640 1 -- 172.21.15.3:0/4213725818 learned_addr learned my addr 172.21.15.3:0/4213725818 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.748+0000 7f3659906640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3654108610 0x7f3654199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.749+0000 7f365a908640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3654108f60 0x7f365419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.749+0000 7f365a107640 1 -- 172.21.15.3:0/4213725818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3654108610 msgr2=0x7f3654199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.749+0000 7f365a107640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3654108610 0x7f3654199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.749+0000 7f365a107640 1 -- 172.21.15.3:0/4213725818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3654108f60 msgr2=0x7f365419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.749+0000 7f365a107640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3654108f60 0x7f365419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.749+0000 7f365a107640 1 -- 172.21.15.3:0/4213725818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365419f4a0 con 0x7f36541031d0 2024-09-21T06:24:24.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.749+0000 7f365a107640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36541031d0 0x7f36541995f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f365000d950 tx=0x7f365000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.749+0000 7f364b7fe640 1 -- 172.21.15.3:0/4213725818 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650017070 con 0x7f36541031d0 2024-09-21T06:24:24.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.749+0000 7f364b7fe640 1 -- 172.21.15.3:0/4213725818 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f365000de80 con 0x7f36541031d0 2024-09-21T06:24:24.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.749+0000 7f365c36b640 1 -- 172.21.15.3:0/4213725818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365419f790 con 0x7f36541031d0 2024-09-21T06:24:24.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.749+0000 7f364b7fe640 1 -- 172.21.15.3:0/4213725818 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365000f620 con 0x7f36541031d0 2024-09-21T06:24:24.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.749+0000 7f365c36b640 1 -- 172.21.15.3:0/4213725818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36541a5fb0 con 0x7f36541031d0 2024-09-21T06:24:24.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.752+0000 7f364b7fe640 1 -- 172.21.15.3:0/4213725818 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f3650009c30 con 0x7f36541031d0 2024-09-21T06:24:24.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.752+0000 7f36628df740 1 Processor -- start 2024-09-21T06:24:24.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.752+0000 7f36628df740 1 -- start start 2024-09-21T06:24:24.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.752+0000 7f36628df740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528de350 0x556352c85430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.752+0000 7f36628df740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528b5490 0x556352c85970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.752+0000 7f36628df740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563528bd670 0x556352c85eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.752+0000 7f36628df740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556352bfbe60 con 0x5563528de350 2024-09-21T06:24:24.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.752+0000 7f36628df740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556352beb290 con 0x5563528bd670 2024-09-21T06:24:24.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.752+0000 7f36628df740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556352bef0a0 con 0x5563528b5490 2024-09-21T06:24:24.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f365a908640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528de350 0x556352c85430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f365a908640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528de350 0x556352c85430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57938/0 (socket says 172.21.15.3:57938) 2024-09-21T06:24:24.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f365a908640 1 -- 172.21.15.3:0/3632076379 learned_addr learned my addr 172.21.15.3:0/3632076379 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:24.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f365a107640 1 --2- 172.21.15.3:0/3632076379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528b5490 0x556352c85970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f3659906640 1 --2- 172.21.15.3:0/3632076379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563528bd670 0x556352c85eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f365a908640 1 -- 172.21.15.3:0/3632076379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528b5490 msgr2=0x556352c85970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f365a908640 1 --2- 172.21.15.3:0/3632076379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528b5490 0x556352c85970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f365a908640 1 -- 172.21.15.3:0/3632076379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563528bd670 msgr2=0x556352c85eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f365a908640 1 --2- 172.21.15.3:0/3632076379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563528bd670 0x556352c85eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f364b7fe640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f36400b66b0 0x7f36400b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f365a908640 1 -- 172.21.15.3:0/3632076379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556352949930 con 0x5563528de350 2024-09-21T06:24:24.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f3659906640 1 --2- 172.21.15.3:0/3632076379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563528bd670 0x556352c85eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:24.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f364b7fe640 1 -- 172.21.15.3:0/4213725818 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f36500e72f0 con 0x7f36541031d0 2024-09-21T06:24:24.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f365a908640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f36400b66b0 0x7f36400b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.753+0000 7f365a908640 1 --2- 172.21.15.3:0/3632076379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528de350 0x556352c85430 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f364c009930 tx=0x7f364c009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac8ab780dc3bba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.754+0000 7f3648d9a640 1 -- 172.21.15.3:0/3632076379 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c031510 con 0x5563528de350 2024-09-21T06:24:24.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.754+0000 7f3648d9a640 1 -- 172.21.15.3:0/3632076379 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f364c036040 con 0x5563528de350 2024-09-21T06:24:24.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.754+0000 7f3648d9a640 1 -- 172.21.15.3:0/3632076379 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c004800 con 0x5563528de350 2024-09-21T06:24:24.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.755+0000 7f36628df740 1 -- 172.21.15.3:0/3632076379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528de350 msgr2=0x556352c85430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.755+0000 7f36628df740 1 --2- 172.21.15.3:0/3632076379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528de350 0x556352c85430 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f364c009930 tx=0x7f364c009960 comp rx=0 tx=0).stop 2024-09-21T06:24:24.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.755+0000 7f365a908640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f36400b66b0 0x7f36400b8bc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f364c04b000 tx=0x7f364c02fa70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.755+0000 7f36628df740 1 -- 172.21.15.3:0/3632076379 shutdown_connections 2024-09-21T06:24:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.755+0000 7f36628df740 1 --2- 172.21.15.3:0/3632076379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528b5490 0x556352c85970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.755+0000 7f36628df740 1 --2- 172.21.15.3:0/3632076379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563528bd670 0x556352c85eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.755+0000 7f36628df740 1 --2- 172.21.15.3:0/3632076379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528de350 0x556352c85430 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.755+0000 7f36628df740 1 -- 172.21.15.3:0/3632076379 >> 172.21.15.3:0/3632076379 conn(0x5563528d2740 msgr2=0x556352c83320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.755+0000 7f36628df740 1 -- 172.21.15.3:0/3632076379 shutdown_connections 2024-09-21T06:24:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.755+0000 7f36628df740 1 -- 172.21.15.3:0/3632076379 wait complete. 2024-09-21T06:24:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.755+0000 7f36628df740 1 -- start start 2024-09-21T06:24:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.755+0000 7f36628df740 1 Processor -- start 2024-09-21T06:24:24.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f36628df740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 0x556352c90ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f36628df740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528bd670 0x556352c91530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f36628df740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563528de350 0x556352c9cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f36628df740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556352a4c890 con 0x5563528b5490 2024-09-21T06:24:24.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f36628df740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556352b85d40 con 0x5563528de350 2024-09-21T06:24:24.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f36628df740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556352c93a90 con 0x5563528bd670 2024-09-21T06:24:24.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f36628df740 10 client.0 _ll_register_callbacks cb 0x556352be9c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:24.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f365a107640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 0x556352c90ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f365a107640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 0x556352c90ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57944/0 (socket says 172.21.15.3:57944) 2024-09-21T06:24:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f365a107640 1 -- 172.21.15.3:0/1676012749 learned_addr learned my addr 172.21.15.3:0/1676012749 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f3659906640 1 --2- 172.21.15.3:0/1676012749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528bd670 0x556352c91530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f365a908640 1 --2- 172.21.15.3:0/1676012749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563528de350 0x556352c9cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f365a107640 1 -- 172.21.15.3:0/1676012749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528bd670 msgr2=0x556352c91530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f365a107640 1 --2- 172.21.15.3:0/1676012749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528bd670 0x556352c91530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f365a107640 1 -- 172.21.15.3:0/1676012749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563528de350 msgr2=0x556352c9cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f365a107640 1 --2- 172.21.15.3:0/1676012749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563528de350 0x556352c9cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f365a107640 1 -- 172.21.15.3:0/1676012749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556352c9d370 con 0x5563528b5490 2024-09-21T06:24:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f365a107640 1 --2- 172.21.15.3:0/1676012749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 0x556352c90ff0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f36500aa3c0 tx=0x7f36500eaa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b66924328a7937 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f363dffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.756+0000 7f363dffb640 1 -- 172.21.15.3:0/1676012749 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650020070 con 0x5563528b5490 2024-09-21T06:24:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.757+0000 7f363dffb640 1 -- 172.21.15.3:0/1676012749 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f36500abe20 con 0x5563528b5490 2024-09-21T06:24:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.757+0000 7f36628df740 1 -- 172.21.15.3:0/1676012749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f364c036040 con 0x5563528b5490 2024-09-21T06:24:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.757+0000 7f363dffb640 1 -- 172.21.15.3:0/1676012749 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f36500a8ff0 con 0x5563528b5490 2024-09-21T06:24:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.757+0000 7f36628df740 10 client.18180 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.757+0000 7f36628df740 1 -- 172.21.15.3:0/1676012749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556352c9fcd0 con 0x5563528b5490 2024-09-21T06:24:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.757+0000 7f363dffb640 1 -- 172.21.15.3:0/1676012749 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f364c036040 con 0x5563528b5490 2024-09-21T06:24:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.757+0000 7f363dffb640 1 -- 172.21.15.3:0/1676012749 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f36500ac020 con 0x5563528b5490 2024-09-21T06:24:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.757+0000 7f36628df740 10 client.18180 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.757+0000 7f36628df740 10 client.18180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.757+0000 7f36628df740 20 client.18180 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.757+0000 7f36628df740 1 --2- 172.21.15.3:0/1676012749 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55635289ccd0 0x556352ca30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.757+0000 7f36628df740 4 client.18180 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.757+0000 7f36628df740 1 -- 172.21.15.3:0/1676012749 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556352bfddb0 con 0x55635289ccd0 2024-09-21T06:24:24.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.758+0000 7f3659906640 1 --2- 172.21.15.3:0/1676012749 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55635289ccd0 0x556352ca30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.758+0000 7f363dffb640 10 client.18180 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.758+0000 7f3659906640 1 --2- 172.21.15.3:0/1676012749 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55635289ccd0 0x556352ca30a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84464b27b4c94324 server_cookie=403bec543b4294c4 in_seq=0 out_seq=0 2024-09-21T06:24:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f363dffb640 1 -- 172.21.15.3:0/1676012749 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556352bfddb0 con 0x55635289ccd0 2024-09-21T06:24:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f363dffb640 10 client.18180 handle_command_reply: tid=0 2024-09-21T06:24:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f36628df740 1 client.18180 shutdown 2024-09-21T06:24:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f36628df740 1 -- 172.21.15.3:0/1676012749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 msgr2=0x556352c90ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f36628df740 1 --2- 172.21.15.3:0/1676012749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 0x556352c90ff0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f36500aa3c0 tx=0x7f36500eaa30 comp rx=0 tx=0).stop 2024-09-21T06:24:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f36628df740 1 -- 172.21.15.3:0/1676012749 shutdown_connections 2024-09-21T06:24:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f36628df740 1 --2- 172.21.15.3:0/1676012749 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55635289ccd0 0x556352ca30a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f363dffb640 0 client.18180 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f36628df740 1 --2- 172.21.15.3:0/1676012749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528bd670 0x556352c91530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f36628df740 1 --2- 172.21.15.3:0/1676012749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563528de350 0x556352c9cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f36628df740 1 --2- 172.21.15.3:0/1676012749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 0x556352c90ff0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f36628df740 1 -- 172.21.15.3:0/1676012749 >> 172.21.15.3:0/1676012749 conn(0x5563528d2740 msgr2=0x556352c88610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f36628df740 1 -- 172.21.15.3:0/1676012749 shutdown_connections 2024-09-21T06:24:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f36628df740 1 -- 172.21.15.3:0/1676012749 wait complete. 2024-09-21T06:24:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.759+0000 7f36628df740 20 client.18180 trim_cache size 0 max 16384 2024-09-21T06:24:24.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.777+0000 7f36628df740 1 Processor -- start 2024-09-21T06:24:24.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.777+0000 7f36628df740 1 -- start start 2024-09-21T06:24:24.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.777+0000 7f36628df740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55635289ccd0 0x556352c9cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.777+0000 7f36628df740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528b5490 0x556352c90680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.777+0000 7f36628df740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528bd670 0x556352c92980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.777+0000 7f36628df740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f36541ad4c0 con 0x5563528bd670 2024-09-21T06:24:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.777+0000 7f36628df740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f36541af000 con 0x55635289ccd0 2024-09-21T06:24:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.777+0000 7f36628df740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36541adb90 con 0x5563528b5490 2024-09-21T06:24:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f365a908640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528bd670 0x556352c92980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f365a908640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528bd670 0x556352c92980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57948/0 (socket says 172.21.15.3:57948) 2024-09-21T06:24:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f365a908640 1 -- 172.21.15.3:0/3055729051 learned_addr learned my addr 172.21.15.3:0/3055729051 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f365a107640 1 --2- 172.21.15.3:0/3055729051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55635289ccd0 0x556352c9cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f3659906640 1 --2- 172.21.15.3:0/3055729051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528b5490 0x556352c90680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f365a908640 1 -- 172.21.15.3:0/3055729051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528b5490 msgr2=0x556352c90680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f365a908640 1 --2- 172.21.15.3:0/3055729051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528b5490 0x556352c90680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f365a908640 1 -- 172.21.15.3:0/3055729051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55635289ccd0 msgr2=0x556352c9cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f365a908640 1 --2- 172.21.15.3:0/3055729051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55635289ccd0 0x556352c9cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f365a908640 1 -- 172.21.15.3:0/3055729051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556352c87720 con 0x5563528bd670 2024-09-21T06:24:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f365a107640 1 --2- 172.21.15.3:0/3055729051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55635289ccd0 0x556352c9cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f365a908640 1 --2- 172.21.15.3:0/3055729051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528bd670 0x556352c92980 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f364c009930 tx=0x7f364c053000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41285142773d8c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f363ffff640 1 -- 172.21.15.3:0/3055729051 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c05a070 con 0x5563528bd670 2024-09-21T06:24:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f363ffff640 1 -- 172.21.15.3:0/3055729051 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f364c036050 con 0x5563528bd670 2024-09-21T06:24:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.778+0000 7f363ffff640 1 -- 172.21.15.3:0/3055729051 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c03c6e0 con 0x5563528bd670 2024-09-21T06:24:24.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.779+0000 7f36628df740 1 -- 172.21.15.3:0/3055729051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528bd670 msgr2=0x556352c92980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.779+0000 7f36628df740 1 --2- 172.21.15.3:0/3055729051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528bd670 0x556352c92980 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f364c009930 tx=0x7f364c053000 comp rx=0 tx=0).stop 2024-09-21T06:24:24.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.779+0000 7f36628df740 1 -- 172.21.15.3:0/3055729051 shutdown_connections 2024-09-21T06:24:24.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.779+0000 7f36628df740 1 --2- 172.21.15.3:0/3055729051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528b5490 0x556352c90680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.779+0000 7f36628df740 1 --2- 172.21.15.3:0/3055729051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528bd670 0x556352c92980 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.779+0000 7f36628df740 1 --2- 172.21.15.3:0/3055729051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55635289ccd0 0x556352c9cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.779+0000 7f36628df740 1 -- 172.21.15.3:0/3055729051 >> 172.21.15.3:0/3055729051 conn(0x5563528d2740 msgr2=0x556352c86e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.779+0000 7f36628df740 1 -- 172.21.15.3:0/3055729051 shutdown_connections 2024-09-21T06:24:24.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.779+0000 7f36628df740 1 -- 172.21.15.3:0/3055729051 wait complete. 2024-09-21T06:24:24.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f36628df740 1 -- start start 2024-09-21T06:24:24.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f36628df740 1 Processor -- start 2024-09-21T06:24:24.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f36628df740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55635289ccd0 0x556352c95100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f36628df740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 0x556352c95680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f36628df740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528bd670 0x556352d0ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f36628df740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f36541ae930 con 0x5563528b5490 2024-09-21T06:24:24.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f36628df740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f36541ae260 con 0x55635289ccd0 2024-09-21T06:24:24.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f36628df740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36541ab4c0 con 0x5563528bd670 2024-09-21T06:24:24.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f36628df740 10 client.0 _ll_register_callbacks cb 0x7f36500e4f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f3659906640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 0x556352c95680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f3659906640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 0x556352c95680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57954/0 (socket says 172.21.15.3:57954) 2024-09-21T06:24:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f3659906640 1 -- 172.21.15.3:0/3361939129 learned_addr learned my addr 172.21.15.3:0/3361939129 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f365a107640 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55635289ccd0 0x556352c95100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.780+0000 7f365a908640 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528bd670 0x556352d0ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f3659906640 1 -- 172.21.15.3:0/3361939129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528bd670 msgr2=0x556352d0ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f3659906640 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528bd670 0x556352d0ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f3659906640 1 -- 172.21.15.3:0/3361939129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55635289ccd0 msgr2=0x556352c95100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f3659906640 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55635289ccd0 0x556352c95100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f3659906640 1 -- 172.21.15.3:0/3361939129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556352d0cfa0 con 0x5563528b5490 2024-09-21T06:24:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f365a908640 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528bd670 0x556352d0ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f3659906640 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 0x556352c95680 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f36541060e0 tx=0x7f3644011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ca6c1329d1a4a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f363dffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f363dffb640 1 -- 172.21.15.3:0/3361939129 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364400fb60 con 0x5563528b5490 2024-09-21T06:24:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f363dffb640 1 -- 172.21.15.3:0/3361939129 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f364400fd00 con 0x5563528b5490 2024-09-21T06:24:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f36628df740 1 -- 172.21.15.3:0/3361939129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f364c036050 con 0x5563528b5490 2024-09-21T06:24:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f363dffb640 1 -- 172.21.15.3:0/3361939129 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3644005020 con 0x5563528b5490 2024-09-21T06:24:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f36628df740 10 client.18186 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:24.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.781+0000 7f36628df740 1 -- 172.21.15.3:0/3361939129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556352d0e190 con 0x5563528b5490 2024-09-21T06:24:24.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.782+0000 7f363dffb640 1 -- 172.21.15.3:0/3361939129 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f364c036050 con 0x5563528b5490 2024-09-21T06:24:24.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.782+0000 7f363dffb640 1 -- 172.21.15.3:0/3361939129 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f3644023440 con 0x5563528b5490 2024-09-21T06:24:24.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.782+0000 7f36628df740 10 client.18186 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:24.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.782+0000 7f36628df740 10 client.18186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:24.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.782+0000 7f36628df740 20 client.18186 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:24.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.782+0000 7f36628df740 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5563528de350 0x556352d10980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.782+0000 7f36628df740 4 client.18186 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:24:24.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.782+0000 7f36628df740 1 -- 172.21.15.3:0/3361939129 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556352bf9860 con 0x5563528de350 2024-09-21T06:24:24.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.782+0000 7f365a107640 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5563528de350 0x556352d10980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.783+0000 7f363dffb640 10 client.18186 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:24.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.783+0000 7f365a107640 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5563528de350 0x556352d10980 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14363e1d197721c4 server_cookie=17983a8f37f8b5d6 in_seq=0 out_seq=0 2024-09-21T06:24:24.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.784+0000 7f363dffb640 1 -- 172.21.15.3:0/3361939129 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556352bf9860 con 0x5563528de350 2024-09-21T06:24:24.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.784+0000 7f363dffb640 10 client.18186 handle_command_reply: tid=0 2024-09-21T06:24:24.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.784+0000 7f36628df740 1 client.18186 shutdown 2024-09-21T06:24:24.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.784+0000 7f36628df740 1 -- 172.21.15.3:0/3361939129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 msgr2=0x556352c95680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.784+0000 7f36628df740 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 0x556352c95680 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f36541060e0 tx=0x7f3644011b50 comp rx=0 tx=0).stop 2024-09-21T06:24:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f36628df740 1 -- 172.21.15.3:0/3361939129 shutdown_connections 2024-09-21T06:24:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f36628df740 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x5563528de350 0x556352d10980 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f36628df740 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563528bd670 0x556352d0ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f363dffb640 0 client.18186 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f36628df740 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5563528b5490 0x556352c95680 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f36628df740 1 --2- 172.21.15.3:0/3361939129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55635289ccd0 0x556352c95100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f36628df740 1 -- 172.21.15.3:0/3361939129 >> 172.21.15.3:0/3361939129 conn(0x5563528d2740 msgr2=0x556352c86e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f36628df740 1 -- 172.21.15.3:0/3361939129 shutdown_connections 2024-09-21T06:24:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f36628df740 1 -- 172.21.15.3:0/3361939129 wait complete. 2024-09-21T06:24:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f36628df740 20 client.18186 trim_cache size 0 max 16384 2024-09-21T06:24:24.793 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:24:24.793 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:24:24.793 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "0bf99782-6ab2-4000-ba98-9ddb00a60402", 2024-09-21T06:24:24.793 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:24:24.793 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:24:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f365c36b640 1 -- 172.21.15.3:0/4213725818 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f36400b66b0 msgr2=0x7f36400b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f365c36b640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f36400b66b0 0x7f36400b8bc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f364c04b000 tx=0x7f364c02fa70 comp rx=0 tx=0).stop 2024-09-21T06:24:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f365c36b640 1 -- 172.21.15.3:0/4213725818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36541031d0 msgr2=0x7f36541995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f365c36b640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36541031d0 0x7f36541995f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f365000d950 tx=0x7f365000dd50 comp rx=0 tx=0).stop 2024-09-21T06:24:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.785+0000 7f365c36b640 1 -- 172.21.15.3:0/4213725818 shutdown_connections 2024-09-21T06:24:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.786+0000 7f365c36b640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f36400b66b0 0x7f36400b8bc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.786+0000 7f365c36b640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3654108610 0x7f3654199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.786+0000 7f365c36b640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3654108f60 0x7f365419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.786+0000 7f365c36b640 1 --2- 172.21.15.3:0/4213725818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36541031d0 0x7f36541995f0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.786+0000 7f365c36b640 1 -- 172.21.15.3:0/4213725818 >> 172.21.15.3:0/4213725818 conn(0x7f3654079e40 msgr2=0x7f36541045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.786+0000 7f365c36b640 1 -- 172.21.15.3:0/4213725818 shutdown_connections 2024-09-21T06:24:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:24.786+0000 7f365c36b640 1 -- 172.21.15.3:0/4213725818 wait complete. 2024-09-21T06:24:25.435 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:24:25.693 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:25.814 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:25 smithi003 ceph-mon[31227]: pgmap v615: 97 pgs: 97 active+clean; 141 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-09-21T06:24:25.814 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:25 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:24:25.814 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:25 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:25.815 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:25 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:24:25.815 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:25 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:24:25.815 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:25 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:25.815 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:25 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:25 smithi137 ceph-mon[34464]: pgmap v615: 97 pgs: 97 active+clean; 141 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-09-21T06:24:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:25 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:24:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:25 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:25 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:24:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:25 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:24:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:25 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:25.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:25 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:25 smithi204 ceph-mon[34659]: pgmap v615: 97 pgs: 97 active+clean; 141 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-09-21T06:24:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:25 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:24:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:25 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:25 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:24:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:25 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:24:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:25 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:25.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:25 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:26.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.295+0000 7f01677d0640 1 -- 172.21.15.3:0/1456450577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01600757b0 msgr2=0x7f0160075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.295+0000 7f01677d0640 1 --2- 172.21.15.3:0/1456450577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01600757b0 0x7f0160075bc0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f015c009900 tx=0x7f015c02a410 comp rx=0 tx=0).stop 2024-09-21T06:24:26.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.295+0000 7f01677d0640 1 -- 172.21.15.3:0/1456450577 shutdown_connections 2024-09-21T06:24:26.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.295+0000 7f01677d0640 1 --2- 172.21.15.3:0/1456450577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01601100e0 0x7f0160112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.295+0000 7f01677d0640 1 --2- 172.21.15.3:0/1456450577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0160076100 0x7f016010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.295+0000 7f01677d0640 1 --2- 172.21.15.3:0/1456450577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01600757b0 0x7f0160075bc0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.295+0000 7f01677d0640 1 -- 172.21.15.3:0/1456450577 >> 172.21.15.3:0/1456450577 conn(0x7f01600ff5d0 msgr2=0x7f0160101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:26.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.295+0000 7f01677d0640 1 -- 172.21.15.3:0/1456450577 shutdown_connections 2024-09-21T06:24:26.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.295+0000 7f01677d0640 1 -- 172.21.15.3:0/1456450577 wait complete. 2024-09-21T06:24:26.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f01677d0640 1 Processor -- start 2024-09-21T06:24:26.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f01677d0640 1 -- start start 2024-09-21T06:24:26.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f01677d0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01600757b0 0x7f016019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f01677d0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0160076100 0x7f016019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f01677d0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01601100e0 0x7f01601a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f01677d0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0160115c90 con 0x7f0160076100 2024-09-21T06:24:26.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f01677d0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0160115b10 con 0x7f01601100e0 2024-09-21T06:24:26.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f01677d0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0160115e10 con 0x7f01600757b0 2024-09-21T06:24:26.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f0164d6b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0160076100 0x7f016019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f0164d6b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0160076100 0x7f016019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44894/0 (socket says 172.21.15.3:44894) 2024-09-21T06:24:26.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f0164d6b640 1 -- 172.21.15.3:0/4212493448 learned_addr learned my addr 172.21.15.3:0/4212493448 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:26.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f016556c640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01600757b0 0x7f016019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f0165d6d640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01601100e0 0x7f01601a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f0164d6b640 1 -- 172.21.15.3:0/4212493448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01600757b0 msgr2=0x7f016019c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.296+0000 7f0164d6b640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01600757b0 0x7f016019c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.297+0000 7f0164d6b640 1 -- 172.21.15.3:0/4212493448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01601100e0 msgr2=0x7f01601a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.297+0000 7f0164d6b640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01601100e0 0x7f01601a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.297+0000 7f0164d6b640 1 -- 172.21.15.3:0/4212493448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01601a2530 con 0x7f0160076100 2024-09-21T06:24:26.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.297+0000 7f0164d6b640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0160076100 0x7f016019cc20 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f015000d770 tx=0x7f015000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:26.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.297+0000 7f01567fc640 1 -- 172.21.15.3:0/4212493448 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150017070 con 0x7f0160076100 2024-09-21T06:24:26.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.297+0000 7f01567fc640 1 -- 172.21.15.3:0/4212493448 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0150009c30 con 0x7f0160076100 2024-09-21T06:24:26.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.297+0000 7f01567fc640 1 -- 172.21.15.3:0/4212493448 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150003cc0 con 0x7f0160076100 2024-09-21T06:24:26.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.297+0000 7f01677d0640 1 -- 172.21.15.3:0/4212493448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01601a2820 con 0x7f0160076100 2024-09-21T06:24:26.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.297+0000 7f01677d0640 1 -- 172.21.15.3:0/4212493448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01601a9040 con 0x7f0160076100 2024-09-21T06:24:26.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f016dd44740 1 Processor -- start 2024-09-21T06:24:26.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f016dd44740 1 -- start start 2024-09-21T06:24:26.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f01567fc640 1 -- 172.21.15.3:0/4212493448 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f0150003e60 con 0x7f0160076100 2024-09-21T06:24:26.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f016dd44740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb62070 0x55b1dff09650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f016dd44740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb391b0 0x55b1dff09b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f016dd44740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb41390 0x55b1dff0a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f016dd44740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b1dfe7fe50 con 0x55b1dfb62070 2024-09-21T06:24:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f016dd44740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b1dfe6f280 con 0x55b1dfb391b0 2024-09-21T06:24:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f016dd44740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b1dfe73090 con 0x55b1dfb41390 2024-09-21T06:24:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f0165d6d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb62070 0x55b1dff09650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f0165d6d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb62070 0x55b1dff09650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44906/0 (socket says 172.21.15.3:44906) 2024-09-21T06:24:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f0165d6d640 1 -- 172.21.15.3:0/547247999 learned_addr learned my addr 172.21.15.3:0/547247999 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f016556c640 1 --2- 172.21.15.3:0/547247999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb391b0 0x55b1dff09b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.300+0000 7f0164d6b640 1 --2- 172.21.15.3:0/547247999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb41390 0x55b1dff0a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.301+0000 7f0165d6d640 1 -- 172.21.15.3:0/547247999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb41390 msgr2=0x55b1dff0a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.301+0000 7f0165d6d640 1 --2- 172.21.15.3:0/547247999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb41390 0x55b1dff0a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.301+0000 7f0165d6d640 1 -- 172.21.15.3:0/547247999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb391b0 msgr2=0x55b1dff09b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.301+0000 7f0165d6d640 1 --2- 172.21.15.3:0/547247999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb391b0 0x55b1dff09b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.301+0000 7f0165d6d640 1 -- 172.21.15.3:0/547247999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1dfbcd650 con 0x55b1dfb62070 2024-09-21T06:24:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.301+0000 7f0165d6d640 1 --2- 172.21.15.3:0/547247999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb62070 0x55b1dff09650 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f0158008da0 tx=0x7f015800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb958d1e609f17a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.301+0000 7f01567fc640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f014c0b6730 0x7f014c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.301+0000 7f014b7fe640 1 -- 172.21.15.3:0/547247999 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015800f830 con 0x55b1dfb62070 2024-09-21T06:24:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.301+0000 7f014b7fe640 1 -- 172.21.15.3:0/547247999 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f015800de20 con 0x55b1dfb62070 2024-09-21T06:24:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.301+0000 7f014b7fe640 1 -- 172.21.15.3:0/547247999 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0158018e30 con 0x55b1dfb62070 2024-09-21T06:24:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.301+0000 7f016556c640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f014c0b6730 0x7f014c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.302+0000 7f01567fc640 1 -- 172.21.15.3:0/4212493448 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f01500e6ff0 con 0x7f0160076100 2024-09-21T06:24:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.302+0000 7f016dd44740 1 -- 172.21.15.3:0/547247999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb62070 msgr2=0x55b1dff09650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.302+0000 7f016556c640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f014c0b6730 0x7f014c0b8c40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f015c031440 tx=0x7f015c005db0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.302+0000 7f016dd44740 1 --2- 172.21.15.3:0/547247999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb62070 0x55b1dff09650 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f0158008da0 tx=0x7f015800d710 comp rx=0 tx=0).stop 2024-09-21T06:24:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.302+0000 7f016dd44740 1 -- 172.21.15.3:0/547247999 shutdown_connections 2024-09-21T06:24:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.302+0000 7f016dd44740 1 --2- 172.21.15.3:0/547247999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb41390 0x55b1dff0a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.302+0000 7f016dd44740 1 --2- 172.21.15.3:0/547247999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb391b0 0x55b1dff09b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.302+0000 7f016dd44740 1 --2- 172.21.15.3:0/547247999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb62070 0x55b1dff09650 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.302+0000 7f016dd44740 1 -- 172.21.15.3:0/547247999 >> 172.21.15.3:0/547247999 conn(0x55b1dfb56460 msgr2=0x55b1dff07540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.302+0000 7f016dd44740 1 -- 172.21.15.3:0/547247999 shutdown_connections 2024-09-21T06:24:26.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.302+0000 7f016dd44740 1 -- 172.21.15.3:0/547247999 wait complete. 2024-09-21T06:24:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.303+0000 7f016dd44740 1 -- start start 2024-09-21T06:24:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.303+0000 7f016dd44740 1 Processor -- start 2024-09-21T06:24:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.303+0000 7f016dd44740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb391b0 0x55b1dff14dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.303+0000 7f016dd44740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 0x55b1dff15310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.303+0000 7f016dd44740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 0x55b1dff21120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.303+0000 7f016dd44740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b1dfcd0700 con 0x55b1dfb41390 2024-09-21T06:24:26.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.303+0000 7f016dd44740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b1dfe0a110 con 0x55b1dfb62070 2024-09-21T06:24:26.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.303+0000 7f016dd44740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b1dff17e30 con 0x55b1dfb391b0 2024-09-21T06:24:26.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.303+0000 7f016dd44740 10 client.0 _ll_register_callbacks cb 0x55b1dfe23990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:26.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.303+0000 7f0164d6b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 0x55b1dff15310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f016556c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb391b0 0x55b1dff14dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f0164d6b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 0x55b1dff15310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44922/0 (socket says 172.21.15.3:44922) 2024-09-21T06:24:26.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f0164d6b640 1 -- 172.21.15.3:0/201372938 learned_addr learned my addr 172.21.15.3:0/201372938 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:26.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f0165d6d640 1 --2- 172.21.15.3:0/201372938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 0x55b1dff21120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f0164d6b640 1 -- 172.21.15.3:0/201372938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb391b0 msgr2=0x55b1dff14dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f0164d6b640 1 --2- 172.21.15.3:0/201372938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb391b0 0x55b1dff14dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f0164d6b640 1 -- 172.21.15.3:0/201372938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 msgr2=0x55b1dff21120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f0164d6b640 1 --2- 172.21.15.3:0/201372938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 0x55b1dff21120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f0164d6b640 1 -- 172.21.15.3:0/201372938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1dfe37c40 con 0x55b1dfb41390 2024-09-21T06:24:26.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f016556c640 1 --2- 172.21.15.3:0/201372938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb391b0 0x55b1dff14dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:26.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f0164d6b640 1 --2- 172.21.15.3:0/201372938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 0x55b1dff15310 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0150000c00 tx=0x7f015000df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8957186144a90775 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:26.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f0148ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:26.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f0148ff9640 1 -- 172.21.15.3:0/201372938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150020030 con 0x55b1dfb41390 2024-09-21T06:24:26.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f0148ff9640 1 -- 172.21.15.3:0/201372938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f01500e90d0 con 0x55b1dfb41390 2024-09-21T06:24:26.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f016dd44740 1 -- 172.21.15.3:0/201372938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f015800de20 con 0x55b1dfb41390 2024-09-21T06:24:26.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f0148ff9640 1 -- 172.21.15.3:0/201372938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f01500cee60 con 0x55b1dfb41390 2024-09-21T06:24:26.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f016dd44740 10 client.18198 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:26.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.304+0000 7f016dd44740 1 -- 172.21.15.3:0/201372938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b1dff23ba0 con 0x55b1dfb41390 2024-09-21T06:24:26.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.305+0000 7f0148ff9640 1 -- 172.21.15.3:0/201372938 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01500af080 con 0x55b1dfb41390 2024-09-21T06:24:26.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.305+0000 7f0148ff9640 1 -- 172.21.15.3:0/201372938 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f01500a87a0 con 0x55b1dfb41390 2024-09-21T06:24:26.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.305+0000 7f016dd44740 10 client.18198 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:26.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.305+0000 7f016dd44740 10 client.18198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:26.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.305+0000 7f016dd44740 20 client.18198 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:26.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.305+0000 7f016dd44740 1 --2- 172.21.15.3:0/201372938 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b1dfb209f0 0x55b1dff27070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.305+0000 7f016dd44740 4 client.18198 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:26.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.305+0000 7f016dd44740 1 -- 172.21.15.3:0/201372938 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1dfe81da0 con 0x55b1dfb209f0 2024-09-21T06:24:26.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.305+0000 7f016556c640 1 --2- 172.21.15.3:0/201372938 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b1dfb209f0 0x55b1dff27070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.306+0000 7f016556c640 1 --2- 172.21.15.3:0/201372938 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b1dfb209f0 0x55b1dff27070 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d8236e749bdcc58 server_cookie=f6266b74d01bbd72 in_seq=0 out_seq=0 2024-09-21T06:24:26.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.306+0000 7f0148ff9640 10 client.18198 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:26.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.306+0000 7f0148ff9640 1 -- 172.21.15.3:0/201372938 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b1dfe81da0 con 0x55b1dfb209f0 2024-09-21T06:24:26.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.306+0000 7f0148ff9640 10 client.18198 handle_command_reply: tid=0 2024-09-21T06:24:26.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.307+0000 7f016dd44740 1 client.18198 shutdown 2024-09-21T06:24:26.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.307+0000 7f016dd44740 1 -- 172.21.15.3:0/201372938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 msgr2=0x55b1dff15310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.307+0000 7f016dd44740 1 --2- 172.21.15.3:0/201372938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 0x55b1dff15310 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0150000c00 tx=0x7f015000df60 comp rx=0 tx=0).stop 2024-09-21T06:24:26.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.307+0000 7f016dd44740 1 -- 172.21.15.3:0/201372938 shutdown_connections 2024-09-21T06:24:26.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.307+0000 7f016dd44740 1 --2- 172.21.15.3:0/201372938 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b1dfb209f0 0x55b1dff27070 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.307+0000 7f016dd44740 1 --2- 172.21.15.3:0/201372938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 0x55b1dff21120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.307+0000 7f016dd44740 1 --2- 172.21.15.3:0/201372938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 0x55b1dff15310 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.307+0000 7f016dd44740 1 --2- 172.21.15.3:0/201372938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb391b0 0x55b1dff14dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.307+0000 7f016dd44740 1 -- 172.21.15.3:0/201372938 >> 172.21.15.3:0/201372938 conn(0x55b1dfb56460 msgr2=0x55b1dff0c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:26.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.307+0000 7f0148ff9640 0 client.18198 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:26.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.307+0000 7f016dd44740 1 -- 172.21.15.3:0/201372938 shutdown_connections 2024-09-21T06:24:26.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.307+0000 7f016dd44740 1 -- 172.21.15.3:0/201372938 wait complete. 2024-09-21T06:24:26.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.307+0000 7f016dd44740 20 client.18198 trim_cache size 0 max 16384 2024-09-21T06:24:26.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016dd44740 1 Processor -- start 2024-09-21T06:24:26.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016dd44740 1 -- start start 2024-09-21T06:24:26.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016dd44740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb391b0 0x55b1dff0ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016dd44740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb41390 0x55b1dff14b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016dd44740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 0x55b1dff150c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016dd44740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f01601b1820 con 0x55b1dfb391b0 2024-09-21T06:24:26.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016dd44740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01601b3360 con 0x55b1dfb62070 2024-09-21T06:24:26.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016dd44740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01601b1ef0 con 0x55b1dfb41390 2024-09-21T06:24:26.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016556c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb391b0 0x55b1dff0ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016556c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb391b0 0x55b1dff0ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44926/0 (socket says 172.21.15.3:44926) 2024-09-21T06:24:26.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016556c640 1 -- 172.21.15.3:0/1159984036 learned_addr learned my addr 172.21.15.3:0/1159984036 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:26.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f0165d6d640 1 --2- 172.21.15.3:0/1159984036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 0x55b1dff150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f0164d6b640 1 --2- 172.21.15.3:0/1159984036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb41390 0x55b1dff14b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016556c640 1 -- 172.21.15.3:0/1159984036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb41390 msgr2=0x55b1dff14b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016556c640 1 --2- 172.21.15.3:0/1159984036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb41390 0x55b1dff14b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016556c640 1 -- 172.21.15.3:0/1159984036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 msgr2=0x55b1dff150c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016556c640 1 --2- 172.21.15.3:0/1159984036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 0x55b1dff150c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.325+0000 7f016556c640 1 -- 172.21.15.3:0/1159984036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1dff07e30 con 0x55b1dfb391b0 2024-09-21T06:24:26.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.326+0000 7f016556c640 1 --2- 172.21.15.3:0/1159984036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb391b0 0x55b1dff0ce60 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f015c02a930 tx=0x7f015c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d557a18e83040a04 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:26.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.326+0000 7f014affd640 1 -- 172.21.15.3:0/1159984036 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c035040 con 0x55b1dfb391b0 2024-09-21T06:24:26.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.326+0000 7f014affd640 1 -- 172.21.15.3:0/1159984036 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f015c056030 con 0x55b1dfb391b0 2024-09-21T06:24:26.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.326+0000 7f014affd640 1 -- 172.21.15.3:0/1159984036 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c004e60 con 0x55b1dfb391b0 2024-09-21T06:24:26.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.327+0000 7f016dd44740 1 -- 172.21.15.3:0/1159984036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb391b0 msgr2=0x55b1dff0ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.327+0000 7f016dd44740 1 --2- 172.21.15.3:0/1159984036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb391b0 0x55b1dff0ce60 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f015c02a930 tx=0x7f015c03f000 comp rx=0 tx=0).stop 2024-09-21T06:24:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.327+0000 7f016dd44740 1 -- 172.21.15.3:0/1159984036 shutdown_connections 2024-09-21T06:24:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.327+0000 7f016dd44740 1 --2- 172.21.15.3:0/1159984036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb41390 0x55b1dff14b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.327+0000 7f016dd44740 1 --2- 172.21.15.3:0/1159984036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 0x55b1dff150c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.327+0000 7f016dd44740 1 --2- 172.21.15.3:0/1159984036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb391b0 0x55b1dff0ce60 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.327+0000 7f016dd44740 1 -- 172.21.15.3:0/1159984036 >> 172.21.15.3:0/1159984036 conn(0x55b1dfb56460 msgr2=0x55b1dff0c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.327+0000 7f016dd44740 1 -- 172.21.15.3:0/1159984036 shutdown_connections 2024-09-21T06:24:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.327+0000 7f016dd44740 1 -- 172.21.15.3:0/1159984036 wait complete. 2024-09-21T06:24:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.327+0000 7f016dd44740 1 -- start start 2024-09-21T06:24:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.327+0000 7f016dd44740 1 Processor -- start 2024-09-21T06:24:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.328+0000 7f016dd44740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb391b0 0x55b1dff18fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.328+0000 7f016dd44740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 0x55b1dff19520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.328+0000 7f016dd44740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 0x55b1dff19a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.328+0000 7f016dd44740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f01601b2c90 con 0x55b1dfb41390 2024-09-21T06:24:26.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.328+0000 7f016dd44740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01601b25c0 con 0x55b1dfb62070 2024-09-21T06:24:26.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.328+0000 7f016dd44740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01601af820 con 0x55b1dfb391b0 2024-09-21T06:24:26.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.328+0000 7f016dd44740 10 client.0 _ll_register_callbacks cb 0x7f015000d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:26.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.328+0000 7f0164d6b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 0x55b1dff19520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.328+0000 7f016556c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb391b0 0x55b1dff18fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.328+0000 7f0164d6b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 0x55b1dff19520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44934/0 (socket says 172.21.15.3:44934) 2024-09-21T06:24:26.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.328+0000 7f0164d6b640 1 -- 172.21.15.3:0/3761670621 learned_addr learned my addr 172.21.15.3:0/3761670621 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:26.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.328+0000 7f0165d6d640 1 --2- 172.21.15.3:0/3761670621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 0x55b1dff19a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.328+0000 7f016556c640 1 -- 172.21.15.3:0/3761670621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 msgr2=0x55b1dff19a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.329+0000 7f016556c640 1 --2- 172.21.15.3:0/3761670621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 0x55b1dff19a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.329+0000 7f016556c640 1 -- 172.21.15.3:0/3761670621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 msgr2=0x55b1dff19520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.329+0000 7f016556c640 1 --2- 172.21.15.3:0/3761670621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 0x55b1dff19520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.329+0000 7f016556c640 1 -- 172.21.15.3:0/3761670621 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1dff90a30 con 0x55b1dfb391b0 2024-09-21T06:24:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.329+0000 7f0164d6b640 1 --2- 172.21.15.3:0/3761670621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 0x55b1dff19520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.329+0000 7f016556c640 1 --2- 172.21.15.3:0/3761670621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb391b0 0x55b1dff18fe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f015c006010 tx=0x7f015c031a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26396e5570104fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.329+0000 7f0148ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-21T06:24:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.329+0000 7f0148ff9640 1 -- 172.21.15.3:0/3761670621 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c004750 con 0x55b1dfb391b0 2024-09-21T06:24:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.329+0000 7f016dd44740 1 -- 172.21.15.3:0/3761670621 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f015c056030 con 0x55b1dfb391b0 2024-09-21T06:24:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.329+0000 7f0148ff9640 1 -- 172.21.15.3:0/3761670621 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f015c004ae0 con 0x55b1dfb391b0 2024-09-21T06:24:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.329+0000 7f0148ff9640 1 -- 172.21.15.3:0/3761670621 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f015c004ce0 con 0x55b1dfb391b0 2024-09-21T06:24:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.329+0000 7f016dd44740 10 client.27691 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.329+0000 7f016dd44740 1 -- 172.21.15.3:0/3761670621 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b1dff91bf0 con 0x55b1dfb391b0 2024-09-21T06:24:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.330+0000 7f0148ff9640 1 -- 172.21.15.3:0/3761670621 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f015c056030 con 0x55b1dfb391b0 2024-09-21T06:24:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.330+0000 7f0148ff9640 1 -- 172.21.15.3:0/3761670621 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f015c0038c0 con 0x55b1dfb391b0 2024-09-21T06:24:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.330+0000 7f016dd44740 10 client.27691 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.330+0000 7f016dd44740 10 client.27691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.330+0000 7f016dd44740 20 client.27691 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.330+0000 7f016dd44740 1 --2- 172.21.15.3:0/3761670621 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b1dfaf5da0 0x55b1dff94360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.330+0000 7f016dd44740 4 client.27691 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:24:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.330+0000 7f016dd44740 1 -- 172.21.15.3:0/3761670621 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1dfe7d850 con 0x55b1dfaf5da0 2024-09-21T06:24:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.330+0000 7f0164d6b640 1 --2- 172.21.15.3:0/3761670621 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b1dfaf5da0 0x55b1dff94360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f0164d6b640 1 --2- 172.21.15.3:0/3761670621 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b1dfaf5da0 0x55b1dff94360 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e53e0ba703e9908e server_cookie=7556f0c2e85721f9 in_seq=0 out_seq=0 2024-09-21T06:24:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f0148ff9640 10 client.27691 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f0148ff9640 1 -- 172.21.15.3:0/3761670621 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x55b1dfe7d850 con 0x55b1dfaf5da0 2024-09-21T06:24:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f0148ff9640 10 client.27691 handle_command_reply: tid=0 2024-09-21T06:24:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f016dd44740 1 client.27691 shutdown 2024-09-21T06:24:26.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f016dd44740 1 -- 172.21.15.3:0/3761670621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb391b0 msgr2=0x55b1dff18fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f016dd44740 1 --2- 172.21.15.3:0/3761670621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb391b0 0x55b1dff18fe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f015c006010 tx=0x7f015c031a50 comp rx=0 tx=0).stop 2024-09-21T06:24:26.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f016dd44740 1 -- 172.21.15.3:0/3761670621 shutdown_connections 2024-09-21T06:24:26.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f016dd44740 1 --2- 172.21.15.3:0/3761670621 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55b1dfaf5da0 0x55b1dff94360 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f016dd44740 1 --2- 172.21.15.3:0/3761670621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1dfb62070 0x55b1dff19a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f016dd44740 1 --2- 172.21.15.3:0/3761670621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1dfb41390 0x55b1dff19520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f0148ff9640 0 client.27691 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:26.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f016dd44740 1 --2- 172.21.15.3:0/3761670621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1dfb391b0 0x55b1dff18fe0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f016dd44740 1 -- 172.21.15.3:0/3761670621 >> 172.21.15.3:0/3761670621 conn(0x55b1dfb56460 msgr2=0x55b1dff14b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:26.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f016dd44740 1 -- 172.21.15.3:0/3761670621 shutdown_connections 2024-09-21T06:24:26.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f016dd44740 1 -- 172.21.15.3:0/3761670621 wait complete. 2024-09-21T06:24:26.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.331+0000 7f016dd44740 20 client.27691 trim_cache size 0 max 16384 2024-09-21T06:24:26.343 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:24:26.344 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:24:26.344 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-21T06:24:26.344 INFO:teuthology.orchestra.run.smithi003.stdout: "0bf99782-6ab2-4000-ba98-9ddb00a60402": { 2024-09-21T06:24:26.344 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.344 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "0bf99782-6ab2-4000-ba98-9ddb00a60402", 2024-09-21T06:24:26.344 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-21T06:24:26.344 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:24:26.345 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.345 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:24:26.345 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:24:26.345 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:24:26.345 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.345 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.345 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.345 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.346 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.346 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.346 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:24:26.346 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:24:26.346 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.346 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.346 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.347 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.347 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.347 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.347 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:24:26.347 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:24:26.347 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.347 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.348 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.348 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.348 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.348 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.348 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:24:26.348 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:24:26.348 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.348 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.349 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.349 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.349 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.349 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.349 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:24:26.349 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:24:26.349 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.349 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.350 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.350 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.350 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.350 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.350 INFO:teuthology.orchestra.run.smithi003.stdout: "cbb5d4a7-b96d-4c2c-b7ab-7a3716c17f5b": { 2024-09-21T06:24:26.350 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:23:38.372765+0000", 2024-09-21T06:24:26.350 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.351 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.351 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.351 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.351 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.351 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.351 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:24:26.351 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:24:26.351 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.352 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.352 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.352 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.352 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.352 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.352 INFO:teuthology.orchestra.run.smithi003.stdout: "0bf99782-6ab2-4000-ba98-9ddb00a60402": { 2024-09-21T06:24:26.352 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:24:24.782991+0000", 2024-09-21T06:24:26.353 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.353 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.353 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.353 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.353 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.353 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.353 INFO:teuthology.orchestra.run.smithi003.stdout: "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58": { 2024-09-21T06:24:26.353 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:21:17.396219+0000", 2024-09-21T06:24:26.354 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.354 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.354 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.354 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.354 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.354 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.354 INFO:teuthology.orchestra.run.smithi003.stdout: "714b9934-a2d3-4a78-b185-01254776061b": { 2024-09-21T06:24:26.355 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:04.685380+0000", 2024-09-21T06:24:26.355 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.355 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.355 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.355 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.355 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.355 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.355 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:24:26.356 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:24:26.356 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.356 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.356 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.356 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.356 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.356 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.356 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:24:26.357 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:24:26.357 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.357 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.357 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.357 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.357 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.357 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.358 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:24:26.358 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:24:26.358 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.358 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.358 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.358 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.358 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.359 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:26.359 INFO:teuthology.orchestra.run.smithi003.stdout: "535fb164-fd4f-4d88-b7e0-575559314385": { 2024-09-21T06:24:26.359 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:51.152572+0000", 2024-09-21T06:24:26.359 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:26.359 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:26.359 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:26.359 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:26.359 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:26.360 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:24:26.360 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:24:26.360 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:24:26.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.332+0000 7f01677d0640 1 -- 172.21.15.3:0/4212493448 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f014c0b6730 msgr2=0x7f014c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.332+0000 7f01677d0640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f014c0b6730 0x7f014c0b8c40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f015c031440 tx=0x7f015c005db0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.332+0000 7f01677d0640 1 -- 172.21.15.3:0/4212493448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0160076100 msgr2=0x7f016019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.332+0000 7f01677d0640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0160076100 0x7f016019cc20 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f015000d770 tx=0x7f015000dc30 comp rx=0 tx=0).stop 2024-09-21T06:24:26.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.332+0000 7f01677d0640 1 -- 172.21.15.3:0/4212493448 shutdown_connections 2024-09-21T06:24:26.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.332+0000 7f01677d0640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f014c0b6730 0x7f014c0b8c40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.332+0000 7f01677d0640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01601100e0 0x7f01601a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.332+0000 7f01677d0640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0160076100 0x7f016019cc20 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.332+0000 7f01677d0640 1 --2- 172.21.15.3:0/4212493448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01600757b0 0x7f016019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.332+0000 7f01677d0640 1 -- 172.21.15.3:0/4212493448 >> 172.21.15.3:0/4212493448 conn(0x7f01600ff5d0 msgr2=0x7f0160109c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:26.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.332+0000 7f01677d0640 1 -- 172.21.15.3:0/4212493448 shutdown_connections 2024-09-21T06:24:26.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:26.332+0000 7f01677d0640 1 -- 172.21.15.3:0/4212493448 wait complete. 2024-09-21T06:24:26.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:26 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:24:26.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:26 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:26 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:24:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:26 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:26 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:24:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:26 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:27.285 INFO:tasks.cephfs.filesystem:scrub status for tag:0bf99782-6ab2-4000-ba98-9ddb00a60402 - {'path': '/', 'tag': '0bf99782-6ab2-4000-ba98-9ddb00a60402', 'options': 'recursive,force'} 2024-09-21T06:24:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:27 smithi003 ceph-mon[31227]: pgmap v616: 97 pgs: 97 active+clean; 148 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.3 MiB/s wr, 70 op/s 2024-09-21T06:24:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:27 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:24:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:27 smithi137 ceph-mon[34464]: pgmap v616: 97 pgs: 97 active+clean; 148 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.3 MiB/s wr, 70 op/s 2024-09-21T06:24:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:27 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:24:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:27 smithi204 ceph-mon[34659]: pgmap v616: 97 pgs: 97 active+clean; 148 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.3 MiB/s wr, 70 op/s 2024-09-21T06:24:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:27 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:24:28.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:24:28.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:24:28.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:24:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:24:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:24:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:24:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:24:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:24:28.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:24:29.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:29 smithi003 ceph-mon[31227]: pgmap v617: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.7 MiB/s wr, 75 op/s 2024-09-21T06:24:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:29 smithi137 ceph-mon[34464]: pgmap v617: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.7 MiB/s wr, 75 op/s 2024-09-21T06:24:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:29 smithi204 ceph-mon[34659]: pgmap v617: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.7 MiB/s wr, 75 op/s 2024-09-21T06:24:31.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:31 smithi003 ceph-mon[31227]: pgmap v618: 97 pgs: 97 active+clean; 161 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 3.4 MiB/s wr, 89 op/s 2024-09-21T06:24:31.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:31 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:24:31.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:31 smithi137 ceph-mon[34464]: pgmap v618: 97 pgs: 97 active+clean; 161 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 3.4 MiB/s wr, 89 op/s 2024-09-21T06:24:31.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:31 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:24:31.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:31 smithi204 ceph-mon[34659]: pgmap v618: 97 pgs: 97 active+clean; 161 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 3.4 MiB/s wr, 89 op/s 2024-09-21T06:24:31.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:31 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:24:32.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:32 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:24:32.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:32 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:24:32.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:32 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:24:32.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:32 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:24:32.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:32 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:24:32.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:32 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:24:33.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:33 smithi003 ceph-mon[31227]: pgmap v619: 97 pgs: 97 active+clean; 161 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 3.1 MiB/s wr, 78 op/s 2024-09-21T06:24:33.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:33 smithi137 ceph-mon[34464]: pgmap v619: 97 pgs: 97 active+clean; 161 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 3.1 MiB/s wr, 78 op/s 2024-09-21T06:24:33.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:33 smithi204 ceph-mon[34659]: pgmap v619: 97 pgs: 97 active+clean; 161 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 3.1 MiB/s wr, 78 op/s 2024-09-21T06:24:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:35 smithi003 ceph-mon[31227]: pgmap v620: 97 pgs: 97 active+clean; 161 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 2.9 MiB/s wr, 70 op/s 2024-09-21T06:24:35.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:35 smithi137 ceph-mon[34464]: pgmap v620: 97 pgs: 97 active+clean; 161 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 2.9 MiB/s wr, 70 op/s 2024-09-21T06:24:35.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:35 smithi204 ceph-mon[34659]: pgmap v620: 97 pgs: 97 active+clean; 161 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 2.9 MiB/s wr, 70 op/s 2024-09-21T06:24:37.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:37 smithi003 ceph-mon[31227]: pgmap v621: 97 pgs: 97 active+clean; 158 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 3.2 MiB/s wr, 81 op/s 2024-09-21T06:24:37.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:37 smithi137 ceph-mon[34464]: pgmap v621: 97 pgs: 97 active+clean; 158 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 3.2 MiB/s wr, 81 op/s 2024-09-21T06:24:37.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:37 smithi204 ceph-mon[34659]: pgmap v621: 97 pgs: 97 active+clean; 158 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 3.2 MiB/s wr, 81 op/s 2024-09-21T06:24:39.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:39 smithi003 ceph-mon[31227]: pgmap v622: 97 pgs: 97 active+clean; 157 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 2.2 MiB/s wr, 70 op/s 2024-09-21T06:24:39.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:39 smithi137 ceph-mon[34464]: pgmap v622: 97 pgs: 97 active+clean; 157 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 2.2 MiB/s wr, 70 op/s 2024-09-21T06:24:39.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:39 smithi204 ceph-mon[34659]: pgmap v622: 97 pgs: 97 active+clean; 157 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 2.2 MiB/s wr, 70 op/s 2024-09-21T06:24:41.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:41 smithi003 ceph-mon[31227]: pgmap v623: 97 pgs: 97 active+clean; 156 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 1.5 MiB/s wr, 59 op/s 2024-09-21T06:24:41.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:41 smithi137 ceph-mon[34464]: pgmap v623: 97 pgs: 97 active+clean; 156 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 1.5 MiB/s wr, 59 op/s 2024-09-21T06:24:41.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:41 smithi204 ceph-mon[34659]: pgmap v623: 97 pgs: 97 active+clean; 156 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 1.5 MiB/s wr, 59 op/s 2024-09-21T06:24:43.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:43 smithi003 ceph-mon[31227]: pgmap v624: 97 pgs: 97 active+clean; 156 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 630 KiB/s wr, 41 op/s 2024-09-21T06:24:43.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:43 smithi137 ceph-mon[34464]: pgmap v624: 97 pgs: 97 active+clean; 156 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 630 KiB/s wr, 41 op/s 2024-09-21T06:24:43.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:43 smithi204 ceph-mon[34659]: pgmap v624: 97 pgs: 97 active+clean; 156 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 630 KiB/s wr, 41 op/s 2024-09-21T06:24:45.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:45 smithi003 ceph-mon[31227]: pgmap v625: 97 pgs: 97 active+clean; 156 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 442 KiB/s wr, 36 op/s 2024-09-21T06:24:45.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:45 smithi137 ceph-mon[34464]: pgmap v625: 97 pgs: 97 active+clean; 156 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 442 KiB/s wr, 36 op/s 2024-09-21T06:24:45.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:45 smithi204 ceph-mon[34659]: pgmap v625: 97 pgs: 97 active+clean; 156 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 442 KiB/s wr, 36 op/s 2024-09-21T06:24:47.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:47 smithi003 ceph-mon[31227]: pgmap v626: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 391 KiB/s wr, 35 op/s 2024-09-21T06:24:47.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:47 smithi137 ceph-mon[34464]: pgmap v626: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 391 KiB/s wr, 35 op/s 2024-09-21T06:24:47.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:47 smithi204 ceph-mon[34659]: pgmap v626: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 391 KiB/s wr, 35 op/s 2024-09-21T06:24:49.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:49 smithi003 ceph-mon[31227]: pgmap v627: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-09-21T06:24:49.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:49 smithi137 ceph-mon[34464]: pgmap v627: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-09-21T06:24:49.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:49 smithi204 ceph-mon[34659]: pgmap v627: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-09-21T06:24:51.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:51 smithi003 ceph-mon[31227]: pgmap v628: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-09-21T06:24:51.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:51 smithi137 ceph-mon[34464]: pgmap v628: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-09-21T06:24:51.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:51 smithi204 ceph-mon[34659]: pgmap v628: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-09-21T06:24:53.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:53 smithi003 ceph-mon[31227]: pgmap v629: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2024-09-21T06:24:53.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:53 smithi137 ceph-mon[34464]: pgmap v629: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2024-09-21T06:24:53.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:53 smithi204 ceph-mon[34659]: pgmap v629: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2024-09-21T06:24:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:55 smithi003 ceph-mon[31227]: pgmap v630: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-21T06:24:55.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:55 smithi137 ceph-mon[34464]: pgmap v630: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-21T06:24:55.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:55 smithi204 ceph-mon[34659]: pgmap v630: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-21T06:24:57.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:24:57.560 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:57.829 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:57 smithi003 ceph-mon[31227]: pgmap v631: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-09-21T06:24:57.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:57 smithi137 ceph-mon[34464]: pgmap v631: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-09-21T06:24:57.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:57 smithi204 ceph-mon[34659]: pgmap v631: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-09-21T06:24:58.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.052+0000 7f2a6f5cc640 1 -- 172.21.15.3:0/1232887417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a68114450 msgr2=0x7f2a68116920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:58.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.052+0000 7f2a6f5cc640 1 --2- 172.21.15.3:0/1232887417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a68114450 0x7f2a68116920 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009ca0 tx=0x7f2a6002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.052+0000 7f2a6f5cc640 1 -- 172.21.15.3:0/1232887417 shutdown_connections 2024-09-21T06:24:58.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.052+0000 7f2a6f5cc640 1 --2- 172.21.15.3:0/1232887417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a68075dd0 0x7f2a680761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.052+0000 7f2a6f5cc640 1 --2- 172.21.15.3:0/1232887417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a68114450 0x7f2a68116920 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.052+0000 7f2a6f5cc640 1 --2- 172.21.15.3:0/1232887417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a68077380 0x7f2a680757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.052+0000 7f2a6f5cc640 1 -- 172.21.15.3:0/1232887417 >> 172.21.15.3:0/1232887417 conn(0x7f2a680ff5d0 msgr2=0x7f2a68101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:58.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.052+0000 7f2a6f5cc640 1 -- 172.21.15.3:0/1232887417 shutdown_connections 2024-09-21T06:24:58.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.052+0000 7f2a6f5cc640 1 -- 172.21.15.3:0/1232887417 wait complete. 2024-09-21T06:24:58.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.053+0000 7f2a6f5cc640 1 Processor -- start 2024-09-21T06:24:58.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.053+0000 7f2a6f5cc640 1 -- start start 2024-09-21T06:24:58.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.053+0000 7f2a6f5cc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a68075dd0 0x7f2a681a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.053+0000 7f2a6f5cc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a68077380 0x7f2a681a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.053+0000 7f2a6f5cc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a68114450 0x7f2a681a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.053+0000 7f2a6f5cc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6811a080 con 0x7f2a68077380 2024-09-21T06:24:58.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.053+0000 7f2a6f5cc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68119f00 con 0x7f2a68075dd0 2024-09-21T06:24:58.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.053+0000 7f2a6f5cc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6811a200 con 0x7f2a68114450 2024-09-21T06:24:58.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.053+0000 7f2a6cb67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a68077380 0x7f2a681a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.053+0000 7f2a6cb67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a68077380 0x7f2a681a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47250/0 (socket says 172.21.15.3:47250) 2024-09-21T06:24:58.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.053+0000 7f2a6cb67640 1 -- 172.21.15.3:0/1472467955 learned_addr learned my addr 172.21.15.3:0/1472467955 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:58.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.053+0000 7f2a6d368640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a68075dd0 0x7f2a681a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.054+0000 7f2a6db69640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a68114450 0x7f2a681a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.054+0000 7f2a6cb67640 1 -- 172.21.15.3:0/1472467955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a68114450 msgr2=0x7f2a681a62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:58.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.054+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a68114450 0x7f2a681a62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.054+0000 7f2a6cb67640 1 -- 172.21.15.3:0/1472467955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a68075dd0 msgr2=0x7f2a681a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:58.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.054+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a68075dd0 0x7f2a681a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.054+0000 7f2a6cb67640 1 -- 172.21.15.3:0/1472467955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a681a6960 con 0x7f2a68077380 2024-09-21T06:24:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.054+0000 7f2a6d368640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a68075dd0 0x7f2a681a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.054+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a68077380 0x7f2a681a1050 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800d770 tx=0x7f2a5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.054+0000 7f2a5e7fc640 1 -- 172.21.15.3:0/1472467955 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58017070 con 0x7f2a68077380 2024-09-21T06:24:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.054+0000 7f2a6f5cc640 1 -- 172.21.15.3:0/1472467955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a681a6c50 con 0x7f2a68077380 2024-09-21T06:24:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.054+0000 7f2a5e7fc640 1 -- 172.21.15.3:0/1472467955 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a58009c30 con 0x7f2a68077380 2024-09-21T06:24:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.054+0000 7f2a5e7fc640 1 -- 172.21.15.3:0/1472467955 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5800f620 con 0x7f2a68077380 2024-09-21T06:24:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.054+0000 7f2a6f5cc640 1 -- 172.21.15.3:0/1472467955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a681ad470 con 0x7f2a68077380 2024-09-21T06:24:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.057+0000 7f2a5e7fc640 1 -- 172.21.15.3:0/1472467955 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f2a5801a070 con 0x7f2a68077380 2024-09-21T06:24:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.057+0000 7f2a75b40740 1 Processor -- start 2024-09-21T06:24:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.057+0000 7f2a75b40740 1 -- start start 2024-09-21T06:24:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.057+0000 7f2a75b40740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 0x55aa7397c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.057+0000 7f2a75b40740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735ac1b0 0x55aa7397c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.057+0000 7f2a75b40740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 0x55aa7397ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.057+0000 7f2a75b40740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55aa738f2e50 con 0x55aa735b4390 2024-09-21T06:24:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.057+0000 7f2a75b40740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aa738e2280 con 0x55aa735ac1b0 2024-09-21T06:24:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.057+0000 7f2a75b40740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aa738e6090 con 0x55aa735d5070 2024-09-21T06:24:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.058+0000 7f2a6cb67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 0x55aa7397ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.058+0000 7f2a6cb67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 0x55aa7397ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47256/0 (socket says 172.21.15.3:47256) 2024-09-21T06:24:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.058+0000 7f2a6cb67640 1 -- 172.21.15.3:0/2039496635 learned_addr learned my addr 172.21.15.3:0/2039496635 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.058+0000 7f2a6db69640 1 --2- 172.21.15.3:0/2039496635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 0x55aa7397c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.058+0000 7f2a6d368640 1 --2- 172.21.15.3:0/2039496635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735ac1b0 0x55aa7397c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.058+0000 7f2a6cb67640 1 -- 172.21.15.3:0/2039496635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 msgr2=0x55aa7397c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.058+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/2039496635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 0x55aa7397c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.058+0000 7f2a6cb67640 1 -- 172.21.15.3:0/2039496635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735ac1b0 msgr2=0x55aa7397c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.058+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/2039496635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735ac1b0 0x55aa7397c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.058+0000 7f2a6cb67640 1 -- 172.21.15.3:0/2039496635 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa73640650 con 0x55aa735b4390 2024-09-21T06:24:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.058+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/2039496635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 0x55aa7397ccf0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f2a580e4990 tx=0x7f2a580ef9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bde9d7059203c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.058+0000 7f2a5e7fc640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a540b66b0 0x7f2a540b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.058+0000 7f2a6db69640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a540b66b0 0x7f2a540b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.059+0000 7f2a537fe640 1 -- 172.21.15.3:0/2039496635 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a580fa070 con 0x55aa735b4390 2024-09-21T06:24:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.059+0000 7f2a5e7fc640 1 -- 172.21.15.3:0/1472467955 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f2a580e72c0 con 0x7f2a68077380 2024-09-21T06:24:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.059+0000 7f2a537fe640 1 -- 172.21.15.3:0/2039496635 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a580acc30 con 0x55aa735b4390 2024-09-21T06:24:58.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.059+0000 7f2a537fe640 1 -- 172.21.15.3:0/2039496635 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a580acef0 con 0x55aa735b4390 2024-09-21T06:24:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.060+0000 7f2a75b40740 1 -- 172.21.15.3:0/2039496635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 msgr2=0x55aa7397ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.060+0000 7f2a75b40740 1 --2- 172.21.15.3:0/2039496635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 0x55aa7397ccf0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f2a580e4990 tx=0x7f2a580ef9b0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.060+0000 7f2a6db69640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a540b66b0 0x7f2a540b8bc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2a60005cc0 tx=0x7f2a60005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.060+0000 7f2a75b40740 1 -- 172.21.15.3:0/2039496635 shutdown_connections 2024-09-21T06:24:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.060+0000 7f2a75b40740 1 --2- 172.21.15.3:0/2039496635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 0x55aa7397ccf0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.060+0000 7f2a75b40740 1 --2- 172.21.15.3:0/2039496635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735ac1b0 0x55aa7397c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.060+0000 7f2a75b40740 1 --2- 172.21.15.3:0/2039496635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 0x55aa7397c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.060+0000 7f2a75b40740 1 -- 172.21.15.3:0/2039496635 >> 172.21.15.3:0/2039496635 conn(0x55aa735c9460 msgr2=0x55aa7397a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.060+0000 7f2a75b40740 1 -- 172.21.15.3:0/2039496635 shutdown_connections 2024-09-21T06:24:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.060+0000 7f2a75b40740 1 -- 172.21.15.3:0/2039496635 wait complete. 2024-09-21T06:24:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.061+0000 7f2a75b40740 1 -- start start 2024-09-21T06:24:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.061+0000 7f2a75b40740 1 Processor -- start 2024-09-21T06:24:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.061+0000 7f2a75b40740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 0x55aa739879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.061+0000 7f2a75b40740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735b4390 0x55aa73987f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.061+0000 7f2a75b40740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735d5070 0x55aa73993d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.061+0000 7f2a75b40740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55aa73743700 con 0x55aa735ac1b0 2024-09-21T06:24:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.061+0000 7f2a75b40740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aa7387d110 con 0x55aa735d5070 2024-09-21T06:24:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.061+0000 7f2a75b40740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aa7398aaa0 con 0x55aa735b4390 2024-09-21T06:24:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.061+0000 7f2a75b40740 10 client.0 _ll_register_callbacks cb 0x55aa73896990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.061+0000 7f2a6d368640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 0x55aa739879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.061+0000 7f2a6d368640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 0x55aa739879e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47260/0 (socket says 172.21.15.3:47260) 2024-09-21T06:24:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.061+0000 7f2a6d368640 1 -- 172.21.15.3:0/3111291186 learned_addr learned my addr 172.21.15.3:0/3111291186 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735b4390 0x55aa73987f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a6db69640 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735d5070 0x55aa73993d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a6d368640 1 -- 172.21.15.3:0/3111291186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735b4390 msgr2=0x55aa73987f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a6d368640 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735b4390 0x55aa73987f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a6d368640 1 -- 172.21.15.3:0/3111291186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735d5070 msgr2=0x55aa73993d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a6d368640 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735d5070 0x55aa73993d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a6d368640 1 -- 172.21.15.3:0/3111291186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa738aac40 con 0x55aa735ac1b0 2024-09-21T06:24:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a6db69640 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735d5070 0x55aa73993d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a6d368640 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 0x55aa739879e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f2a64011800 tx=0x7f2a64011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb459473fd48703 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a50ff9640 1 -- 172.21.15.3:0/3111291186 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64004460 con 0x55aa735ac1b0 2024-09-21T06:24:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a50ff9640 1 -- 172.21.15.3:0/3111291186 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f2a6400f9a0 con 0x55aa735ac1b0 2024-09-21T06:24:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a75b40740 1 -- 172.21.15.3:0/3111291186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a580acc30 con 0x55aa735ac1b0 2024-09-21T06:24:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a50ff9640 1 -- 172.21.15.3:0/3111291186 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2a6400fba0 con 0x55aa735ac1b0 2024-09-21T06:24:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a75b40740 10 client.18216 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.062+0000 7f2a75b40740 1 -- 172.21.15.3:0/3111291186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55aa739968d0 con 0x55aa735ac1b0 2024-09-21T06:24:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.063+0000 7f2a50ff9640 1 -- 172.21.15.3:0/3111291186 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a580acc30 con 0x55aa735ac1b0 2024-09-21T06:24:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.063+0000 7f2a50ff9640 1 -- 172.21.15.3:0/3111291186 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f2a64018020 con 0x55aa735ac1b0 2024-09-21T06:24:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.063+0000 7f2a75b40740 10 client.18216 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.063+0000 7f2a75b40740 10 client.18216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.063+0000 7f2a75b40740 20 client.18216 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.063+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55aa735939f0 0x55aa73999d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.063+0000 7f2a75b40740 4 client.18216 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.063+0000 7f2a75b40740 1 -- 172.21.15.3:0/3111291186 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa738f4da0 con 0x55aa735939f0 2024-09-21T06:24:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.063+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55aa735939f0 0x55aa73999d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.064+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55aa735939f0 0x55aa73999d80 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8660c4b4dca38f0e server_cookie=664edebaeec7c43e in_seq=0 out_seq=0 2024-09-21T06:24:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.064+0000 7f2a50ff9640 10 client.18216 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.064+0000 7f2a50ff9640 1 -- 172.21.15.3:0/3111291186 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aa738f4da0 con 0x55aa735939f0 2024-09-21T06:24:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.064+0000 7f2a50ff9640 10 client.18216 handle_command_reply: tid=0 2024-09-21T06:24:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.065+0000 7f2a75b40740 1 client.18216 shutdown 2024-09-21T06:24:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.065+0000 7f2a75b40740 1 -- 172.21.15.3:0/3111291186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 msgr2=0x55aa739879e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.065+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 0x55aa739879e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f2a64011800 tx=0x7f2a64011cc0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.065+0000 7f2a75b40740 1 -- 172.21.15.3:0/3111291186 shutdown_connections 2024-09-21T06:24:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.065+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55aa735939f0 0x55aa73999d80 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.065+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735b4390 0x55aa73987f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.065+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735d5070 0x55aa73993d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.065+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3111291186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 0x55aa739879e0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.065+0000 7f2a75b40740 1 -- 172.21.15.3:0/3111291186 >> 172.21.15.3:0/3111291186 conn(0x55aa735c9460 msgr2=0x55aa7397f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:58.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.065+0000 7f2a50ff9640 0 client.18216 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:58.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.065+0000 7f2a75b40740 1 -- 172.21.15.3:0/3111291186 shutdown_connections 2024-09-21T06:24:58.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.065+0000 7f2a75b40740 1 -- 172.21.15.3:0/3111291186 wait complete. 2024-09-21T06:24:58.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.065+0000 7f2a75b40740 20 client.18216 trim_cache size 0 max 16384 2024-09-21T06:24:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.083+0000 7f2a75b40740 1 Processor -- start 2024-09-21T06:24:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.083+0000 7f2a75b40740 1 -- start start 2024-09-21T06:24:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.083+0000 7f2a75b40740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 0x55aa7397f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.083+0000 7f2a75b40740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735b4390 0x55aa7397f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.083+0000 7f2a75b40740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 0x55aa73984bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.083+0000 7f2a75b40740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2a681b5dc0 con 0x55aa735ac1b0 2024-09-21T06:24:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.083+0000 7f2a75b40740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2a681b7900 con 0x55aa735b4390 2024-09-21T06:24:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.083+0000 7f2a75b40740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a681b6490 con 0x55aa735d5070 2024-09-21T06:24:58.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.083+0000 7f2a6d368640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 0x55aa7397f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.083+0000 7f2a6d368640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 0x55aa7397f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47266/0 (socket says 172.21.15.3:47266) 2024-09-21T06:24:58.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.083+0000 7f2a6d368640 1 -- 172.21.15.3:0/3283258564 learned_addr learned my addr 172.21.15.3:0/3283258564 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:58.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.083+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/3283258564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735b4390 0x55aa7397f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.084+0000 7f2a6d368640 1 -- 172.21.15.3:0/3283258564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 msgr2=0x55aa73984bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:58.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.084+0000 7f2a6d368640 1 --2- 172.21.15.3:0/3283258564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 0x55aa73984bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.084+0000 7f2a6d368640 1 -- 172.21.15.3:0/3283258564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735b4390 msgr2=0x55aa7397f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:58.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.084+0000 7f2a6d368640 1 --2- 172.21.15.3:0/3283258564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735b4390 0x55aa7397f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.084+0000 7f2a6d368640 1 -- 172.21.15.3:0/3283258564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa73593c20 con 0x55aa735ac1b0 2024-09-21T06:24:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.084+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/3283258564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735b4390 0x55aa7397f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.084+0000 7f2a6d368640 1 --2- 172.21.15.3:0/3283258564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 0x55aa7397f370 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2a64023630 tx=0x7f2a6400f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc2f2f2dcadfa04 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.084+0000 7f2a52ffd640 1 -- 172.21.15.3:0/3283258564 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64029070 con 0x55aa735ac1b0 2024-09-21T06:24:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.084+0000 7f2a52ffd640 1 -- 172.21.15.3:0/3283258564 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a64023820 con 0x55aa735ac1b0 2024-09-21T06:24:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.084+0000 7f2a52ffd640 1 -- 172.21.15.3:0/3283258564 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64003830 con 0x55aa735ac1b0 2024-09-21T06:24:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.085+0000 7f2a75b40740 1 -- 172.21.15.3:0/3283258564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 msgr2=0x55aa7397f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.085+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3283258564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 0x55aa7397f370 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2a64023630 tx=0x7f2a6400f9a0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.085+0000 7f2a75b40740 1 -- 172.21.15.3:0/3283258564 shutdown_connections 2024-09-21T06:24:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.085+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3283258564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 0x55aa73984bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.085+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3283258564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735b4390 0x55aa7397f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.085+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3283258564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735ac1b0 0x55aa7397f370 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.085+0000 7f2a75b40740 1 -- 172.21.15.3:0/3283258564 >> 172.21.15.3:0/3283258564 conn(0x55aa735c9460 msgr2=0x55aa73986260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.085+0000 7f2a75b40740 1 -- 172.21.15.3:0/3283258564 shutdown_connections 2024-09-21T06:24:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.085+0000 7f2a75b40740 1 -- 172.21.15.3:0/3283258564 wait complete. 2024-09-21T06:24:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.085+0000 7f2a75b40740 1 -- start start 2024-09-21T06:24:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.085+0000 7f2a75b40740 1 Processor -- start 2024-09-21T06:24:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.086+0000 7f2a75b40740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735ac1b0 0x55aa7398c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.086+0000 7f2a75b40740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 0x55aa7398c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.086+0000 7f2a75b40740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 0x55aa73a03780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.086+0000 7f2a75b40740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2a681b7230 con 0x55aa735b4390 2024-09-21T06:24:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.086+0000 7f2a75b40740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2a681b6b60 con 0x55aa735ac1b0 2024-09-21T06:24:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.086+0000 7f2a75b40740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a681b3dc0 con 0x55aa735d5070 2024-09-21T06:24:58.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.086+0000 7f2a75b40740 10 client.0 _ll_register_callbacks cb 0x7f2a64008760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:58.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.086+0000 7f2a6cb67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 0x55aa7398c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.086+0000 7f2a6cb67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 0x55aa7398c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47276/0 (socket says 172.21.15.3:47276) 2024-09-21T06:24:58.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.086+0000 7f2a6cb67640 1 -- 172.21.15.3:0/3695941425 learned_addr learned my addr 172.21.15.3:0/3695941425 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:58.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.086+0000 7f2a6d368640 1 --2- 172.21.15.3:0/3695941425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735ac1b0 0x55aa7398c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.086+0000 7f2a6db69640 1 --2- 172.21.15.3:0/3695941425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 0x55aa73a03780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a6cb67640 1 -- 172.21.15.3:0/3695941425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 msgr2=0x55aa73a03780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:58.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/3695941425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 0x55aa73a03780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a6cb67640 1 -- 172.21.15.3:0/3695941425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735ac1b0 msgr2=0x55aa7398c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:58.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/3695941425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735ac1b0 0x55aa7398c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a6cb67640 1 -- 172.21.15.3:0/3695941425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa73a03cc0 con 0x55aa735b4390 2024-09-21T06:24:58.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a6cb67640 1 --2- 172.21.15.3:0/3695941425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 0x55aa7398c8b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f2a58000c00 tx=0x7f2a5800df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a832b83c94d7de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:58.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:24:58.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a50ff9640 1 -- 172.21.15.3:0/3695941425 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a580e6040 con 0x55aa735b4390 2024-09-21T06:24:58.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a50ff9640 1 -- 172.21.15.3:0/3695941425 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f2a580eabe0 con 0x55aa735b4390 2024-09-21T06:24:58.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a75b40740 1 -- 172.21.15.3:0/3695941425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a64023820 con 0x55aa735b4390 2024-09-21T06:24:58.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a50ff9640 1 -- 172.21.15.3:0/3695941425 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2a58021040 con 0x55aa735b4390 2024-09-21T06:24:58.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a75b40740 10 client.18222 fetch_fsmap learned FSMap version 38 2024-09-21T06:24:58.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a75b40740 1 -- 172.21.15.3:0/3695941425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55aa73a04eb0 con 0x55aa735b4390 2024-09-21T06:24:58.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.087+0000 7f2a50ff9640 1 -- 172.21.15.3:0/3695941425 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a64023820 con 0x55aa735b4390 2024-09-21T06:24:58.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.088+0000 7f2a50ff9640 1 -- 172.21.15.3:0/3695941425 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f2a580e4ca0 con 0x55aa735b4390 2024-09-21T06:24:58.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.088+0000 7f2a75b40740 10 client.18222 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:24:58.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.088+0000 7f2a75b40740 10 client.18222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:58.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.088+0000 7f2a75b40740 20 client.18222 populate_metadata read hostname 'smithi003' 2024-09-21T06:24:58.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.088+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3695941425 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55aa73568da0 0x55aa73a070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:58.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.088+0000 7f2a75b40740 4 client.18222 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:24:58.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.088+0000 7f2a75b40740 1 -- 172.21.15.3:0/3695941425 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aa738f0850 con 0x55aa73568da0 2024-09-21T06:24:58.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.088+0000 7f2a6d368640 1 --2- 172.21.15.3:0/3695941425 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55aa73568da0 0x55aa73a070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:58.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a6d368640 1 --2- 172.21.15.3:0/3695941425 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55aa73568da0 0x55aa73a070f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c76d6aeea3a87c server_cookie=d46969975786424 in_seq=0 out_seq=0 2024-09-21T06:24:58.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a50ff9640 10 client.18222 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:58.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a50ff9640 1 -- 172.21.15.3:0/3695941425 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x55aa738f0850 con 0x55aa73568da0 2024-09-21T06:24:58.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a50ff9640 10 client.18222 handle_command_reply: tid=0 2024-09-21T06:24:58.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a75b40740 1 client.18222 shutdown 2024-09-21T06:24:58.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a75b40740 1 -- 172.21.15.3:0/3695941425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 msgr2=0x55aa7398c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:58.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3695941425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 0x55aa7398c8b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f2a58000c00 tx=0x7f2a5800df10 comp rx=0 tx=0).stop 2024-09-21T06:24:58.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a75b40740 1 -- 172.21.15.3:0/3695941425 shutdown_connections 2024-09-21T06:24:58.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3695941425 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55aa73568da0 0x55aa73a070f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3695941425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa735d5070 0x55aa73a03780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3695941425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aa735b4390 0x55aa7398c8b0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a75b40740 1 --2- 172.21.15.3:0/3695941425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa735ac1b0 0x55aa7398c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a75b40740 1 -- 172.21.15.3:0/3695941425 >> 172.21.15.3:0/3695941425 conn(0x55aa735c9460 msgr2=0x55aa7397f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:58.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a50ff9640 0 client.18222 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:24:58.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a75b40740 1 -- 172.21.15.3:0/3695941425 shutdown_connections 2024-09-21T06:24:58.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a75b40740 1 -- 172.21.15.3:0/3695941425 wait complete. 2024-09-21T06:24:58.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.089+0000 7f2a75b40740 20 client.18222 trim_cache size 0 max 16384 2024-09-21T06:24:58.100 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:24:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:24:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:24:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:24:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:24:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:24:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:24:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:24:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:24:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:24:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:24:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:24:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:24:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:24:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "cbb5d4a7-b96d-4c2c-b7ab-7a3716c17f5b": { 2024-09-21T06:24:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:23:38.372765+0000", 2024-09-21T06:24:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:24:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:24:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "0bf99782-6ab2-4000-ba98-9ddb00a60402": { 2024-09-21T06:24:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:24:24.782991+0000", 2024-09-21T06:24:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58": { 2024-09-21T06:24:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:21:17.396219+0000", 2024-09-21T06:24:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "714b9934-a2d3-4a78-b185-01254776061b": { 2024-09-21T06:24:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:04.685380+0000", 2024-09-21T06:24:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:24:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:24:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:24:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:24:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:24:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:24:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:24:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "535fb164-fd4f-4d88-b7e0-575559314385": { 2024-09-21T06:24:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:51.152572+0000", 2024-09-21T06:24:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:24:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:24:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:24:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:24:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:24:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:24:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:24:58.113 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:24:58.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.090+0000 7f2a6f5cc640 1 -- 172.21.15.3:0/1472467955 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a540b66b0 msgr2=0x7f2a540b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:58.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.090+0000 7f2a6f5cc640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a540b66b0 0x7f2a540b8bc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2a60005cc0 tx=0x7f2a60005c30 comp rx=0 tx=0).stop 2024-09-21T06:24:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.090+0000 7f2a6f5cc640 1 -- 172.21.15.3:0/1472467955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a68077380 msgr2=0x7f2a681a1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.090+0000 7f2a6f5cc640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a68077380 0x7f2a681a1050 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800d770 tx=0x7f2a5800dc30 comp rx=0 tx=0).stop 2024-09-21T06:24:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.090+0000 7f2a6f5cc640 1 -- 172.21.15.3:0/1472467955 shutdown_connections 2024-09-21T06:24:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.090+0000 7f2a6f5cc640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a540b66b0 0x7f2a540b8bc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.090+0000 7f2a6f5cc640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a68114450 0x7f2a681a62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.090+0000 7f2a6f5cc640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a68077380 0x7f2a681a1050 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.090+0000 7f2a6f5cc640 1 --2- 172.21.15.3:0/1472467955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a68075dd0 0x7f2a681a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:58.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.090+0000 7f2a6f5cc640 1 -- 172.21.15.3:0/1472467955 >> 172.21.15.3:0/1472467955 conn(0x7f2a680ff5d0 msgr2=0x7f2a68110350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.090+0000 7f2a6f5cc640 1 -- 172.21.15.3:0/1472467955 shutdown_connections 2024-09-21T06:24:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:58.090+0000 7f2a6f5cc640 1 -- 172.21.15.3:0/1472467955 wait complete. 2024-09-21T06:24:59.070 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:24:59.070 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:24:59.336 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:59.748 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:24:59 smithi003 ceph-mon[31227]: pgmap v632: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T06:24:59.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.837+0000 7efc4359a640 1 -- 172.21.15.3:0/3525220335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc3c109df0 msgr2=0x7efc3c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:59.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.837+0000 7efc4359a640 1 --2- 172.21.15.3:0/3525220335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc3c109df0 0x7efc3c110620 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7efc34009ca0 tx=0x7efc3402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:24:59.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.837+0000 7efc4359a640 1 -- 172.21.15.3:0/3525220335 shutdown_connections 2024-09-21T06:24:59.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.837+0000 7efc4359a640 1 --2- 172.21.15.3:0/3525220335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc3c109df0 0x7efc3c110620 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:59.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.837+0000 7efc4359a640 1 --2- 172.21.15.3:0/3525220335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc3c1094a0 0x7efc3c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:59.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.837+0000 7efc4359a640 1 --2- 172.21.15.3:0/3525220335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc3c103b60 0x7efc3c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:59.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.837+0000 7efc4359a640 1 -- 172.21.15.3:0/3525220335 >> 172.21.15.3:0/3525220335 conn(0x7efc3c07b850 msgr2=0x7efc3c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:59.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.837+0000 7efc4359a640 1 -- 172.21.15.3:0/3525220335 shutdown_connections 2024-09-21T06:24:59.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.837+0000 7efc4359a640 1 -- 172.21.15.3:0/3525220335 wait complete. 2024-09-21T06:24:59.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.838+0000 7efc4359a640 1 Processor -- start 2024-09-21T06:24:59.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.838+0000 7efc4359a640 1 -- start start 2024-09-21T06:24:59.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.838+0000 7efc4359a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc3c103b60 0x7efc3c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:59.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.838+0000 7efc4359a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc3c1094a0 0x7efc3c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:59.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.838+0000 7efc4359a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc3c109df0 0x7efc3c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:59.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.838+0000 7efc4359a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c113ab0 con 0x7efc3c103b60 2024-09-21T06:24:59.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.838+0000 7efc4359a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c113930 con 0x7efc3c1094a0 2024-09-21T06:24:59.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.838+0000 7efc4359a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c113c30 con 0x7efc3c109df0 2024-09-21T06:24:59.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.838+0000 7efc41336640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc3c103b60 0x7efc3c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:59.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc41336640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc3c103b60 0x7efc3c19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47292/0 (socket says 172.21.15.3:47292) 2024-09-21T06:24:59.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc41336640 1 -- 172.21.15.3:0/3703334324 learned_addr learned my addr 172.21.15.3:0/3703334324 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:24:59.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc40b35640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc3c1094a0 0x7efc3c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:59.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc41b37640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc3c109df0 0x7efc3c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:59.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc41336640 1 -- 172.21.15.3:0/3703334324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc3c109df0 msgr2=0x7efc3c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:59.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc41336640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc3c109df0 0x7efc3c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:59.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc41336640 1 -- 172.21.15.3:0/3703334324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc3c1094a0 msgr2=0x7efc3c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:59.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc41336640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc3c1094a0 0x7efc3c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:59.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc41336640 1 -- 172.21.15.3:0/3703334324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c1a0370 con 0x7efc3c103b60 2024-09-21T06:24:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc41b37640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc3c109df0 0x7efc3c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc40b35640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc3c1094a0 0x7efc3c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc41336640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc3c103b60 0x7efc3c19a400 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7efc3800d950 tx=0x7efc3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc327fc640 1 -- 172.21.15.3:0/3703334324 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38017070 con 0x7efc3c103b60 2024-09-21T06:24:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc327fc640 1 -- 172.21.15.3:0/3703334324 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc38009c30 con 0x7efc3c103b60 2024-09-21T06:24:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc4359a640 1 -- 172.21.15.3:0/3703334324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3c1a0660 con 0x7efc3c103b60 2024-09-21T06:24:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc327fc640 1 -- 172.21.15.3:0/3703334324 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3800f620 con 0x7efc3c103b60 2024-09-21T06:24:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.839+0000 7efc4359a640 1 -- 172.21.15.3:0/3703334324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3c1a6dc0 con 0x7efc3c103b60 2024-09-21T06:24:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.840+0000 7efc4359a640 1 -- 172.21.15.3:0/3703334324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3c103f70 con 0x7efc3c103b60 2024-09-21T06:24:59.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.842+0000 7efc327fc640 1 -- 172.21.15.3:0/3703334324 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7efc38005030 con 0x7efc3c103b60 2024-09-21T06:24:59.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.843+0000 7efc327fc640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7efc280b6680 0x7efc280b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:59.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.843+0000 7efc327fc640 1 -- 172.21.15.3:0/3703334324 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7efc380e7410 con 0x7efc3c103b60 2024-09-21T06:24:59.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.843+0000 7efc40b35640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7efc280b6680 0x7efc280b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:59.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.844+0000 7efc40b35640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7efc280b6680 0x7efc280b8b90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0058c0 tx=0x7efc2c005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:59.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:24:59.846+0000 7efc327fc640 1 -- 172.21.15.3:0/3703334324 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efc380ad910 con 0x7efc3c103b60 2024-09-21T06:24:59.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:24:59 smithi137 ceph-mon[34464]: pgmap v632: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T06:24:59.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:24:59 smithi204 ceph-mon[34659]: pgmap v632: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T06:25:00.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.075+0000 7efc4359a640 1 -- 172.21.15.3:0/3703334324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc3c02e640 con 0x7efc3c103b60 2024-09-21T06:25:00.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.076+0000 7efc327fc640 1 -- 172.21.15.3:0/3703334324 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11966 (secure 0 0 0) 0x7efc38009de0 con 0x7efc3c103b60 2024-09-21T06:25:00.078 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:25:00.078 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-21T06:25:00.078 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"btime":"2024-09-21T06:23:56:445581+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:23:55.736497+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27344":{"gid":27344,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3457686047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3457686047},{"type":"v1","addr":"172.21.15.137:6835","nonce":3457686047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24670,24674,24679,24691,24704]},"id":1}]} 2024-09-21T06:25:00.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.078+0000 7efc4359a640 1 -- 172.21.15.3:0/3703334324 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7efc280b6680 msgr2=0x7efc280b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:00.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.078+0000 7efc4359a640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7efc280b6680 0x7efc280b8b90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0058c0 tx=0x7efc2c005850 comp rx=0 tx=0).stop 2024-09-21T06:25:00.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.078+0000 7efc4359a640 1 -- 172.21.15.3:0/3703334324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc3c103b60 msgr2=0x7efc3c19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:00.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.078+0000 7efc4359a640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc3c103b60 0x7efc3c19a400 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7efc3800d950 tx=0x7efc3800de10 comp rx=0 tx=0).stop 2024-09-21T06:25:00.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.078+0000 7efc4359a640 1 -- 172.21.15.3:0/3703334324 shutdown_connections 2024-09-21T06:25:00.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.078+0000 7efc4359a640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7efc280b6680 0x7efc280b8b90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:00.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.078+0000 7efc4359a640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc3c109df0 0x7efc3c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:00.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.078+0000 7efc4359a640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc3c1094a0 0x7efc3c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:00.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.078+0000 7efc4359a640 1 --2- 172.21.15.3:0/3703334324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc3c103b60 0x7efc3c19a400 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:00.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.078+0000 7efc4359a640 1 -- 172.21.15.3:0/3703334324 >> 172.21.15.3:0/3703334324 conn(0x7efc3c07b850 msgr2=0x7efc3c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:00.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.078+0000 7efc4359a640 1 -- 172.21.15.3:0/3703334324 shutdown_connections 2024-09-21T06:25:00.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:00.078+0000 7efc4359a640 1 -- 172.21.15.3:0/3703334324 wait complete. 2024-09-21T06:25:00.679 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:00 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3703334324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:25:00.717 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:25:00.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:00 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3703334324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:25:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:00 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3703334324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:25:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:01.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.474+0000 7f09e9da1640 1 -- 172.21.15.3:0/56790610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e4108b10 msgr2=0x7f09e410f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:01.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.474+0000 7f09e9da1640 1 --2- 172.21.15.3:0/56790610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e4108b10 0x7f09e410f340 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009ca0 tx=0x7f09dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.474+0000 7f09e9da1640 1 -- 172.21.15.3:0/56790610 shutdown_connections 2024-09-21T06:25:01.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.474+0000 7f09e9da1640 1 --2- 172.21.15.3:0/56790610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e4108b10 0x7f09e410f340 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.474+0000 7f09e9da1640 1 --2- 172.21.15.3:0/56790610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09e41081c0 0x7f09e41085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.474+0000 7f09e9da1640 1 --2- 172.21.15.3:0/56790610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09e4102880 0x7f09e4102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.474+0000 7f09e9da1640 1 -- 172.21.15.3:0/56790610 >> 172.21.15.3:0/56790610 conn(0x7f09e40fe2d0 msgr2=0x7f09e4100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:01.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.474+0000 7f09e9da1640 1 -- 172.21.15.3:0/56790610 shutdown_connections 2024-09-21T06:25:01.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.474+0000 7f09e9da1640 1 -- 172.21.15.3:0/56790610 wait complete. 2024-09-21T06:25:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.475+0000 7f09e9da1640 1 Processor -- start 2024-09-21T06:25:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.475+0000 7f09e9da1640 1 -- start start 2024-09-21T06:25:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.475+0000 7f09e9da1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09e4102880 0x7f09e4199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.475+0000 7f09e9da1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09e41081c0 0x7f09e4199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.475+0000 7f09e9da1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e4108b10 0x7f09e419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.475+0000 7f09e9da1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f09e41127c0 con 0x7f09e4108b10 2024-09-21T06:25:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.475+0000 7f09e9da1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09e4112640 con 0x7f09e4102880 2024-09-21T06:25:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.475+0000 7f09e9da1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f09e4112940 con 0x7f09e41081c0 2024-09-21T06:25:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.475+0000 7f09e95a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e4108b10 0x7f09e419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.475+0000 7f09e95a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e4108b10 0x7f09e419e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47310/0 (socket says 172.21.15.3:47310) 2024-09-21T06:25:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.475+0000 7f09e95a0640 1 -- 172.21.15.3:0/1407658003 learned_addr learned my addr 172.21.15.3:0/1407658003 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e8d9f640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09e4102880 0x7f09e4199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e3fff640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09e41081c0 0x7f09e4199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e95a0640 1 -- 172.21.15.3:0/1407658003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09e41081c0 msgr2=0x7f09e4199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e95a0640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09e41081c0 0x7f09e4199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e95a0640 1 -- 172.21.15.3:0/1407658003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09e4102880 msgr2=0x7f09e4199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e95a0640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09e4102880 0x7f09e4199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e95a0640 1 -- 172.21.15.3:0/1407658003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e419f0e0 con 0x7f09e4108b10 2024-09-21T06:25:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e3fff640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09e41081c0 0x7f09e4199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e8d9f640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09e4102880 0x7f09e4199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e95a0640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e4108b10 0x7f09e419e9b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f09dc005cc0 tx=0x7f09dc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e1ffb640 1 -- 172.21.15.3:0/1407658003 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc003bf0 con 0x7f09e4108b10 2024-09-21T06:25:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e9da1640 1 -- 172.21.15.3:0/1407658003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e419f370 con 0x7f09e4108b10 2024-09-21T06:25:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e1ffb640 1 -- 172.21.15.3:0/1407658003 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09dc003d90 con 0x7f09e4108b10 2024-09-21T06:25:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e1ffb640 1 -- 172.21.15.3:0/1407658003 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc02fbc0 con 0x7f09e4108b10 2024-09-21T06:25:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.476+0000 7f09e9da1640 1 -- 172.21.15.3:0/1407658003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e41a5b30 con 0x7f09e4108b10 2024-09-21T06:25:01.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.479+0000 7f09f1577740 1 Processor -- start 2024-09-21T06:25:01.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.479+0000 7f09e1ffb640 1 -- 172.21.15.3:0/1407658003 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f09dc02fd60 con 0x7f09e4108b10 2024-09-21T06:25:01.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.479+0000 7f09f1577740 1 -- start start 2024-09-21T06:25:01.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09f1577740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db139f070 0x555db1746860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09f1577740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db13761b0 0x555db1746da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09f1577740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 0x555db17472e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09f1577740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555db16bcea0 con 0x555db13761b0 2024-09-21T06:25:01.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09f1577740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555db16ac2d0 con 0x555db139f070 2024-09-21T06:25:01.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09f1577740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555db16b00e0 con 0x555db137e390 2024-09-21T06:25:01.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09e8d9f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db13761b0 0x555db1746da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09e8d9f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db13761b0 0x555db1746da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47322/0 (socket says 172.21.15.3:47322) 2024-09-21T06:25:01.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09e8d9f640 1 -- 172.21.15.3:0/1125430658 learned_addr learned my addr 172.21.15.3:0/1125430658 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:01.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09e3fff640 1 --2- 172.21.15.3:0/1125430658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db139f070 0x555db1746860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09e8d9f640 1 -- 172.21.15.3:0/1125430658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 msgr2=0x555db17472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:01.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09e8d9f640 1 --2- 172.21.15.3:0/1125430658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 0x555db17472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09e8d9f640 1 -- 172.21.15.3:0/1125430658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db139f070 msgr2=0x555db1746860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:01.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09e8d9f640 1 --2- 172.21.15.3:0/1125430658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db139f070 0x555db1746860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09e8d9f640 1 -- 172.21.15.3:0/1125430658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db140a650 con 0x555db13761b0 2024-09-21T06:25:01.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09e3fff640 1 --2- 172.21.15.3:0/1125430658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db139f070 0x555db1746860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:01.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09e1ffb640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09cc0b6680 0x7f09cc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09e8d9f640 1 --2- 172.21.15.3:0/1125430658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db13761b0 0x555db1746da0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f09d400bd40 tx=0x7f09d400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2eb9b2f88b0593 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:01.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09e3fff640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09cc0b6680 0x7f09cc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.480+0000 7f09caffd640 1 -- 172.21.15.3:0/1125430658 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4015070 con 0x555db13761b0 2024-09-21T06:25:01.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.481+0000 7f09e1ffb640 1 -- 172.21.15.3:0/1407658003 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f09dc108d30 con 0x7f09e4108b10 2024-09-21T06:25:01.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.481+0000 7f09caffd640 1 -- 172.21.15.3:0/1125430658 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09d400be30 con 0x555db13761b0 2024-09-21T06:25:01.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.481+0000 7f09caffd640 1 -- 172.21.15.3:0/1125430658 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4011920 con 0x555db13761b0 2024-09-21T06:25:01.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.481+0000 7f09f1577740 1 -- 172.21.15.3:0/1125430658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db13761b0 msgr2=0x555db1746da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:01.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.481+0000 7f09f1577740 1 --2- 172.21.15.3:0/1125430658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db13761b0 0x555db1746da0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f09d400bd40 tx=0x7f09d400f730 comp rx=0 tx=0).stop 2024-09-21T06:25:01.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.481+0000 7f09e3fff640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09cc0b6680 0x7f09cc0b8b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f09d8005fd0 tx=0x7f09d800b2c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:01.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.481+0000 7f09f1577740 1 -- 172.21.15.3:0/1125430658 shutdown_connections 2024-09-21T06:25:01.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.481+0000 7f09f1577740 1 --2- 172.21.15.3:0/1125430658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 0x555db17472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.481+0000 7f09f1577740 1 --2- 172.21.15.3:0/1125430658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db13761b0 0x555db1746da0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.481+0000 7f09f1577740 1 --2- 172.21.15.3:0/1125430658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db139f070 0x555db1746860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.481+0000 7f09f1577740 1 -- 172.21.15.3:0/1125430658 >> 172.21.15.3:0/1125430658 conn(0x555db1393460 msgr2=0x555db1744750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:01.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.482+0000 7f09f1577740 1 -- 172.21.15.3:0/1125430658 shutdown_connections 2024-09-21T06:25:01.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.482+0000 7f09f1577740 1 -- 172.21.15.3:0/1125430658 wait complete. 2024-09-21T06:25:01.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.482+0000 7f09f1577740 1 -- start start 2024-09-21T06:25:01.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.482+0000 7f09f1577740 1 Processor -- start 2024-09-21T06:25:01.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09f1577740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 0x555db1754160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09f1577740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 0x555db17546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09f1577740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 0x555db1760500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09f1577740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555db150d700 con 0x555db139f070 2024-09-21T06:25:01.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09f1577740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555db1647160 con 0x555db13761b0 2024-09-21T06:25:01.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09f1577740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555db17571a0 con 0x555db137e390 2024-09-21T06:25:01.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09f1577740 10 client.0 _ll_register_callbacks cb 0x555db16609e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:01.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09e95a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 0x555db1760500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09e95a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 0x555db1760500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47328/0 (socket says 172.21.15.3:47328) 2024-09-21T06:25:01.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09e95a0640 1 -- 172.21.15.3:0/1956745079 learned_addr learned my addr 172.21.15.3:0/1956745079 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:01.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09e8d9f640 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 0x555db1754160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09e3fff640 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 0x555db17546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09e95a0640 1 -- 172.21.15.3:0/1956745079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 msgr2=0x555db17546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:01.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09e95a0640 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 0x555db17546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09e95a0640 1 -- 172.21.15.3:0/1956745079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 msgr2=0x555db1754160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09e95a0640 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 0x555db1754160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09e95a0640 1 -- 172.21.15.3:0/1956745079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db1674c90 con 0x555db139f070 2024-09-21T06:25:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09e8d9f640 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 0x555db1754160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09e95a0640 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 0x555db1760500 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0ce890 tx=0x7f09dc0ce610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61781c5aa37a3cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09c3fff640 1 -- 172.21.15.3:0/1956745079 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0cf290 con 0x555db139f070 2024-09-21T06:25:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09f1577740 1 -- 172.21.15.3:0/1956745079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09d400be30 con 0x555db139f070 2024-09-21T06:25:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.483+0000 7f09c3fff640 1 -- 172.21.15.3:0/1956745079 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f09dc004610 con 0x555db139f070 2024-09-21T06:25:01.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.484+0000 7f09c3fff640 1 -- 172.21.15.3:0/1956745079 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f09dc0098b0 con 0x555db139f070 2024-09-21T06:25:01.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.484+0000 7f09f1577740 10 client.18240 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:01.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.484+0000 7f09f1577740 1 -- 172.21.15.3:0/1956745079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555db1762e90 con 0x555db139f070 2024-09-21T06:25:01.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.484+0000 7f09c3fff640 1 -- 172.21.15.3:0/1956745079 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09d400be30 con 0x555db139f070 2024-09-21T06:25:01.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.484+0000 7f09c3fff640 1 -- 172.21.15.3:0/1956745079 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f09dc0d8120 con 0x555db139f070 2024-09-21T06:25:01.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.484+0000 7f09f1577740 10 client.18240 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:01.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.484+0000 7f09f1577740 10 client.18240 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:25:01.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.484+0000 7f09f1577740 20 client.18240 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:01.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.484+0000 7f09f1577740 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x555db135d9f0 0x555db1766410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.484+0000 7f09f1577740 4 client.18240 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.484+0000 7f09f1577740 1 -- 172.21.15.3:0/1956745079 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555db16bedf0 con 0x555db135d9f0 2024-09-21T06:25:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.485+0000 7f09e8d9f640 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x555db135d9f0 0x555db1766410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.485+0000 7f09c3fff640 10 client.18240 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:25:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.485+0000 7f09e8d9f640 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x555db135d9f0 0x555db1766410 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=960eee0e8424a128 server_cookie=ca55ed56f54d802f in_seq=0 out_seq=0 2024-09-21T06:25:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09c3fff640 1 -- 172.21.15.3:0/1956745079 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555db16bedf0 con 0x555db135d9f0 2024-09-21T06:25:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09c3fff640 10 client.18240 handle_command_reply: tid=0 2024-09-21T06:25:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09f1577740 1 client.18240 shutdown 2024-09-21T06:25:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09f1577740 1 -- 172.21.15.3:0/1956745079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 msgr2=0x555db1760500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09f1577740 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 0x555db1760500 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0ce890 tx=0x7f09dc0ce610 comp rx=0 tx=0).stop 2024-09-21T06:25:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09f1577740 1 -- 172.21.15.3:0/1956745079 shutdown_connections 2024-09-21T06:25:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09f1577740 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x555db135d9f0 0x555db1766410 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09f1577740 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 0x555db17546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09f1577740 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 0x555db1760500 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09f1577740 1 --2- 172.21.15.3:0/1956745079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 0x555db1754160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09f1577740 1 -- 172.21.15.3:0/1956745079 >> 172.21.15.3:0/1956745079 conn(0x555db1393460 msgr2=0x555db1748250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09c3fff640 0 client.18240 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:25:01.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09f1577740 1 -- 172.21.15.3:0/1956745079 shutdown_connections 2024-09-21T06:25:01.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09f1577740 1 -- 172.21.15.3:0/1956745079 wait complete. 2024-09-21T06:25:01.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.486+0000 7f09f1577740 20 client.18240 trim_cache size 0 max 16384 2024-09-21T06:25:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.500+0000 7f09f1577740 1 Processor -- start 2024-09-21T06:25:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.500+0000 7f09f1577740 1 -- start start 2024-09-21T06:25:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.500+0000 7f09f1577740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 0x555db1753f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.500+0000 7f09f1577740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db137e390 0x555db1754450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.500+0000 7f09f1577740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db139f070 0x555db1751350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.500+0000 7f09f1577740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f09e41ad360 con 0x555db137e390 2024-09-21T06:25:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.500+0000 7f09f1577740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09e41aeea0 con 0x555db13761b0 2024-09-21T06:25:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.500+0000 7f09f1577740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f09e41ada30 con 0x555db139f070 2024-09-21T06:25:01.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.500+0000 7f09e3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db137e390 0x555db1754450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.500+0000 7f09e3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db137e390 0x555db1754450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47330/0 (socket says 172.21.15.3:47330) 2024-09-21T06:25:01.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.500+0000 7f09e3fff640 1 -- 172.21.15.3:0/1201908377 learned_addr learned my addr 172.21.15.3:0/1201908377 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:01.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.500+0000 7f09e8d9f640 1 --2- 172.21.15.3:0/1201908377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 0x555db1753f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.501+0000 7f09e95a0640 1 --2- 172.21.15.3:0/1201908377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db139f070 0x555db1751350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.501+0000 7f09e3fff640 1 -- 172.21.15.3:0/1201908377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db139f070 msgr2=0x555db1751350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:01.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.501+0000 7f09e3fff640 1 --2- 172.21.15.3:0/1201908377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db139f070 0x555db1751350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.501+0000 7f09e3fff640 1 -- 172.21.15.3:0/1201908377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 msgr2=0x555db1753f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:01.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.501+0000 7f09e3fff640 1 --2- 172.21.15.3:0/1201908377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 0x555db1753f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.501+0000 7f09e3fff640 1 -- 172.21.15.3:0/1201908377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db1752660 con 0x555db137e390 2024-09-21T06:25:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.501+0000 7f09e95a0640 1 --2- 172.21.15.3:0/1201908377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db139f070 0x555db1751350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.501+0000 7f09e3fff640 1 --2- 172.21.15.3:0/1201908377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db137e390 0x555db1754450 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f09e406b4a0 tx=0x7f09d8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9020faac8b0e11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.501+0000 7f09ca7fc640 1 -- 172.21.15.3:0/1201908377 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8004220 con 0x555db137e390 2024-09-21T06:25:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.501+0000 7f09ca7fc640 1 -- 172.21.15.3:0/1201908377 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09d8011e50 con 0x555db137e390 2024-09-21T06:25:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.501+0000 7f09ca7fc640 1 -- 172.21.15.3:0/1201908377 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d801f7b0 con 0x555db137e390 2024-09-21T06:25:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.502+0000 7f09f1577740 1 -- 172.21.15.3:0/1201908377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db137e390 msgr2=0x555db1754450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.502+0000 7f09f1577740 1 --2- 172.21.15.3:0/1201908377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db137e390 0x555db1754450 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f09e406b4a0 tx=0x7f09d8012000 comp rx=0 tx=0).stop 2024-09-21T06:25:01.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.502+0000 7f09f1577740 1 -- 172.21.15.3:0/1201908377 shutdown_connections 2024-09-21T06:25:01.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.502+0000 7f09f1577740 1 --2- 172.21.15.3:0/1201908377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db139f070 0x555db1751350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.502+0000 7f09f1577740 1 --2- 172.21.15.3:0/1201908377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db137e390 0x555db1754450 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.502+0000 7f09f1577740 1 --2- 172.21.15.3:0/1201908377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 0x555db1753f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.502+0000 7f09f1577740 1 -- 172.21.15.3:0/1201908377 >> 172.21.15.3:0/1201908377 conn(0x555db1393460 msgr2=0x555db17529e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:01.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.502+0000 7f09f1577740 1 -- 172.21.15.3:0/1201908377 shutdown_connections 2024-09-21T06:25:01.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.502+0000 7f09f1577740 1 -- 172.21.15.3:0/1201908377 wait complete. 2024-09-21T06:25:01.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.503+0000 7f09f1577740 1 -- start start 2024-09-21T06:25:01.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.503+0000 7f09f1577740 1 Processor -- start 2024-09-21T06:25:01.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.503+0000 7f09f1577740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 0x555db1758530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.503+0000 7f09f1577740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 0x555db1758a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.503+0000 7f09f1577740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 0x555db1758fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.503+0000 7f09f1577740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f09e41ae7d0 con 0x555db139f070 2024-09-21T06:25:01.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.503+0000 7f09f1577740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09e41ae100 con 0x555db13761b0 2024-09-21T06:25:01.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.503+0000 7f09f1577740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f09e41ab360 con 0x555db137e390 2024-09-21T06:25:01.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.503+0000 7f09f1577740 10 client.0 _ll_register_callbacks cb 0x7f09dc009430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:01.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.503+0000 7f09e95a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 0x555db1758fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09e95a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 0x555db1758fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47340/0 (socket says 172.21.15.3:47340) 2024-09-21T06:25:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09e95a0640 1 -- 172.21.15.3:0/2020317882 learned_addr learned my addr 172.21.15.3:0/2020317882 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09e8d9f640 1 --2- 172.21.15.3:0/2020317882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 0x555db1758530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09e3fff640 1 --2- 172.21.15.3:0/2020317882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 0x555db1758a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09e95a0640 1 -- 172.21.15.3:0/2020317882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 msgr2=0x555db1758a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09e95a0640 1 --2- 172.21.15.3:0/2020317882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 0x555db1758a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09e95a0640 1 -- 172.21.15.3:0/2020317882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 msgr2=0x555db1758530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09e95a0640 1 --2- 172.21.15.3:0/2020317882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 0x555db1758530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09e95a0640 1 -- 172.21.15.3:0/2020317882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db17cfed0 con 0x555db139f070 2024-09-21T06:25:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09e95a0640 1 --2- 172.21.15.3:0/2020317882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 0x555db1758fb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0cb160 tx=0x7f09dc009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98c0414e8934c4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09c3fff640 1 -- 172.21.15.3:0/2020317882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0e8020 con 0x555db139f070 2024-09-21T06:25:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09c3fff640 1 -- 172.21.15.3:0/2020317882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f09dc0f20b0 con 0x555db139f070 2024-09-21T06:25:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.504+0000 7f09f1577740 1 -- 172.21.15.3:0/2020317882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09d8011e50 con 0x555db139f070 2024-09-21T06:25:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.505+0000 7f09c3fff640 1 -- 172.21.15.3:0/2020317882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f09dc0cf290 con 0x555db139f070 2024-09-21T06:25:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.505+0000 7f09f1577740 10 client.18246 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.505+0000 7f09f1577740 1 -- 172.21.15.3:0/2020317882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555db17d1150 con 0x555db139f070 2024-09-21T06:25:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.505+0000 7f09c3fff640 1 -- 172.21.15.3:0/2020317882 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09d8011e50 con 0x555db139f070 2024-09-21T06:25:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.505+0000 7f09c3fff640 1 -- 172.21.15.3:0/2020317882 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f09dc0d31c0 con 0x555db139f070 2024-09-21T06:25:01.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.505+0000 7f09f1577740 10 client.18246 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:01.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.505+0000 7f09f1577740 10 client.18246 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:25:01.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.505+0000 7f09f1577740 20 client.18246 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:01.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.505+0000 7f09f1577740 1 --2- 172.21.15.3:0/2020317882 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x555db1332da0 0x555db17d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:01.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.505+0000 7f09f1577740 4 client.18246 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:25:01.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.505+0000 7f09f1577740 1 -- 172.21.15.3:0/2020317882 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555db16ba8a0 con 0x555db1332da0 2024-09-21T06:25:01.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.506+0000 7f09e8d9f640 1 --2- 172.21.15.3:0/2020317882 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x555db1332da0 0x555db17d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:01.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.506+0000 7f09c3fff640 10 client.18246 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:25:01.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.506+0000 7f09e8d9f640 1 --2- 172.21.15.3:0/2020317882 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x555db1332da0 0x555db17d36f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70a2bc6755752833 server_cookie=36b789c93c90ac2c in_seq=0 out_seq=0 2024-09-21T06:25:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.506+0000 7f09c3fff640 1 -- 172.21.15.3:0/2020317882 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555db16ba8a0 con 0x555db1332da0 2024-09-21T06:25:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.506+0000 7f09c3fff640 10 client.18246 handle_command_reply: tid=0 2024-09-21T06:25:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.506+0000 7f09f1577740 1 client.18246 shutdown 2024-09-21T06:25:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09f1577740 1 -- 172.21.15.3:0/2020317882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 msgr2=0x555db1758fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09f1577740 1 --2- 172.21.15.3:0/2020317882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 0x555db1758fb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0cb160 tx=0x7f09dc009650 comp rx=0 tx=0).stop 2024-09-21T06:25:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09f1577740 1 -- 172.21.15.3:0/2020317882 shutdown_connections 2024-09-21T06:25:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09f1577740 1 --2- 172.21.15.3:0/2020317882 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x555db1332da0 0x555db17d36f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09f1577740 1 --2- 172.21.15.3:0/2020317882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555db137e390 0x555db1758a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09f1577740 1 --2- 172.21.15.3:0/2020317882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555db139f070 0x555db1758fb0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09f1577740 1 --2- 172.21.15.3:0/2020317882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555db13761b0 0x555db1758530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09c3fff640 0 client.18246 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:25:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09f1577740 1 -- 172.21.15.3:0/2020317882 >> 172.21.15.3:0/2020317882 conn(0x555db1393460 msgr2=0x555db1752870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09f1577740 1 -- 172.21.15.3:0/2020317882 shutdown_connections 2024-09-21T06:25:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09f1577740 1 -- 172.21.15.3:0/2020317882 wait complete. 2024-09-21T06:25:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09f1577740 20 client.18246 trim_cache size 0 max 16384 2024-09-21T06:25:01.521 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:25:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09e9da1640 1 -- 172.21.15.3:0/1407658003 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09cc0b6680 msgr2=0x7f09cc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.507+0000 7f09e9da1640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09cc0b6680 0x7f09cc0b8b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f09d8005fd0 tx=0x7f09d800b2c0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.508+0000 7f09e9da1640 1 -- 172.21.15.3:0/1407658003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e4108b10 msgr2=0x7f09e419e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.508+0000 7f09e9da1640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e4108b10 0x7f09e419e9b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f09dc005cc0 tx=0x7f09dc02f380 comp rx=0 tx=0).stop 2024-09-21T06:25:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.508+0000 7f09e9da1640 1 -- 172.21.15.3:0/1407658003 shutdown_connections 2024-09-21T06:25:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.508+0000 7f09e9da1640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f09cc0b6680 0x7f09cc0b8b90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.508+0000 7f09e9da1640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09e41081c0 0x7f09e4199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.508+0000 7f09e9da1640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e4108b10 0x7f09e419e9b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.508+0000 7f09e9da1640 1 --2- 172.21.15.3:0/1407658003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09e4102880 0x7f09e4199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.508+0000 7f09e9da1640 1 -- 172.21.15.3:0/1407658003 >> 172.21.15.3:0/1407658003 conn(0x7f09e40fe2d0 msgr2=0x7f09e41006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.508+0000 7f09e9da1640 1 -- 172.21.15.3:0/1407658003 shutdown_connections 2024-09-21T06:25:01.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:01.508+0000 7f09e9da1640 1 -- 172.21.15.3:0/1407658003 wait complete. 2024-09-21T06:25:01.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:01 smithi003 ceph-mon[31227]: pgmap v633: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2024-09-21T06:25:01.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:01 smithi137 ceph-mon[34464]: pgmap v633: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2024-09-21T06:25:01.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:01 smithi204 ceph-mon[34659]: pgmap v633: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2024-09-21T06:25:02.221 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:25:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.985+0000 7f18b99fa640 1 -- 172.21.15.3:0/1050747879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b4108d10 msgr2=0x7f18b410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.985+0000 7f18b99fa640 1 --2- 172.21.15.3:0/1050747879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b4108d10 0x7f18b410f540 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009ca0 tx=0x7f18ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:25:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.985+0000 7f18b99fa640 1 -- 172.21.15.3:0/1050747879 shutdown_connections 2024-09-21T06:25:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.985+0000 7f18b99fa640 1 --2- 172.21.15.3:0/1050747879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18b41083c0 0x7f18b41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.985+0000 7f18b99fa640 1 --2- 172.21.15.3:0/1050747879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b4108d10 0x7f18b410f540 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.985+0000 7f18b99fa640 1 --2- 172.21.15.3:0/1050747879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b4102a80 0x7f18b4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.985+0000 7f18b99fa640 1 -- 172.21.15.3:0/1050747879 >> 172.21.15.3:0/1050747879 conn(0x7f18b4079740 msgr2=0x7f18b4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.985+0000 7f18b99fa640 1 -- 172.21.15.3:0/1050747879 shutdown_connections 2024-09-21T06:25:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.985+0000 7f18b99fa640 1 -- 172.21.15.3:0/1050747879 wait complete. 2024-09-21T06:25:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.986+0000 7f18b99fa640 1 Processor -- start 2024-09-21T06:25:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b99fa640 1 -- start start 2024-09-21T06:25:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b99fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b4102a80 0x7f18b419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b99fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b41083c0 0x7f18b419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b99fa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18b4108d10 0x7f18b419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b99fa640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18b4113aa0 con 0x7f18b41083c0 2024-09-21T06:25:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b99fa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f18b4113920 con 0x7f18b4102a80 2024-09-21T06:25:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b99fa640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18b4113c20 con 0x7f18b4108d10 2024-09-21T06:25:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b41083c0 0x7f18b419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b41083c0 0x7f18b419aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47360/0 (socket says 172.21.15.3:47360) 2024-09-21T06:25:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b27fc640 1 -- 172.21.15.3:0/4036230257 learned_addr learned my addr 172.21.15.3:0/4036230257 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b37fe640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18b4108d10 0x7f18b419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b2ffd640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b4102a80 0x7f18b419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b27fc640 1 -- 172.21.15.3:0/4036230257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18b4108d10 msgr2=0x7f18b419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b27fc640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18b4108d10 0x7f18b419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b27fc640 1 -- 172.21.15.3:0/4036230257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b4102a80 msgr2=0x7f18b419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b27fc640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b4102a80 0x7f18b419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.987+0000 7f18b27fc640 1 -- 172.21.15.3:0/4036230257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b41a0360 con 0x7f18b41083c0 2024-09-21T06:25:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.988+0000 7f18b27fc640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b41083c0 0x7f18b419aa50 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c970 tx=0x7f18a800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.988+0000 7f18b89f8640 1 -- 172.21.15.3:0/4036230257 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8014070 con 0x7f18b41083c0 2024-09-21T06:25:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.988+0000 7f18b99fa640 1 -- 172.21.15.3:0/4036230257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b41a0650 con 0x7f18b41083c0 2024-09-21T06:25:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.988+0000 7f18b89f8640 1 -- 172.21.15.3:0/4036230257 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a8004440 con 0x7f18b41083c0 2024-09-21T06:25:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.988+0000 7f18b89f8640 1 -- 172.21.15.3:0/4036230257 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8003e50 con 0x7f18b41083c0 2024-09-21T06:25:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.988+0000 7f18b99fa640 1 -- 172.21.15.3:0/4036230257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b41a6e70 con 0x7f18b41083c0 2024-09-21T06:25:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.991+0000 7f18bff6e740 1 Processor -- start 2024-09-21T06:25:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.991+0000 7f18bff6e740 1 -- start start 2024-09-21T06:25:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.991+0000 7f18b89f8640 1 -- 172.21.15.3:0/4036230257 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f18a8017070 con 0x7f18b41083c0 2024-09-21T06:25:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.991+0000 7f18bff6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a36070 0x559544ddd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.991+0000 7f18bff6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a0d1b0 0x559544ddd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.991+0000 7f18bff6e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a15390 0x559544dddd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.991+0000 7f18bff6e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559544d53ea0 con 0x559544a15390 2024-09-21T06:25:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.991+0000 7f18bff6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559544d432d0 con 0x559544a36070 2024-09-21T06:25:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.991+0000 7f18bff6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559544d470e0 con 0x559544a0d1b0 2024-09-21T06:25:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.991+0000 7f18b27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a15390 0x559544dddd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.991+0000 7f18b27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a15390 0x559544dddd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47374/0 (socket says 172.21.15.3:47374) 2024-09-21T06:25:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.991+0000 7f18b27fc640 1 -- 172.21.15.3:0/2997057012 learned_addr learned my addr 172.21.15.3:0/2997057012 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.991+0000 7f18b2ffd640 1 --2- 172.21.15.3:0/2997057012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a0d1b0 0x559544ddd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.992+0000 7f18b27fc640 1 -- 172.21.15.3:0/2997057012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a0d1b0 msgr2=0x559544ddd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.992+0000 7f18b27fc640 1 --2- 172.21.15.3:0/2997057012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a0d1b0 0x559544ddd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.992+0000 7f18b37fe640 1 --2- 172.21.15.3:0/2997057012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a36070 0x559544ddd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.992+0000 7f18b27fc640 1 -- 172.21.15.3:0/2997057012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a36070 msgr2=0x559544ddd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.992+0000 7f18b27fc640 1 --2- 172.21.15.3:0/2997057012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a36070 0x559544ddd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.992+0000 7f18b27fc640 1 -- 172.21.15.3:0/2997057012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559544aa1650 con 0x559544a15390 2024-09-21T06:25:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.992+0000 7f18b37fe640 1 --2- 172.21.15.3:0/2997057012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a36070 0x559544ddd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.992+0000 7f18b89f8640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f189c0b6850 0x7f189c0b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.992+0000 7f18b27fc640 1 --2- 172.21.15.3:0/2997057012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a15390 0x559544dddd40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f18a80e4840 tx=0x7f18a80ef850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc67328a84caba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.992+0000 7f18b37fe640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f189c0b6850 0x7f189c0b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.992+0000 7f18b89f8640 1 -- 172.21.15.3:0/4036230257 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f18a80e70d0 con 0x7f18b41083c0 2024-09-21T06:25:03.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.993+0000 7f1899ffb640 1 -- 172.21.15.3:0/2997057012 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80a9620 con 0x559544a15390 2024-09-21T06:25:03.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.993+0000 7f1899ffb640 1 -- 172.21.15.3:0/2997057012 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a80acae0 con 0x559544a15390 2024-09-21T06:25:03.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.993+0000 7f1899ffb640 1 -- 172.21.15.3:0/2997057012 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80eada0 con 0x559544a15390 2024-09-21T06:25:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.993+0000 7f18bff6e740 1 -- 172.21.15.3:0/2997057012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a15390 msgr2=0x559544dddd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.993+0000 7f18b37fe640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f189c0b6850 0x7f189c0b8d60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f18ac005cc0 tx=0x7f18ac005c50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.993+0000 7f18bff6e740 1 --2- 172.21.15.3:0/2997057012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a15390 0x559544dddd40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f18a80e4840 tx=0x7f18a80ef850 comp rx=0 tx=0).stop 2024-09-21T06:25:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.994+0000 7f18bff6e740 1 -- 172.21.15.3:0/2997057012 shutdown_connections 2024-09-21T06:25:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.994+0000 7f18bff6e740 1 --2- 172.21.15.3:0/2997057012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a0d1b0 0x559544ddd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.994+0000 7f18bff6e740 1 --2- 172.21.15.3:0/2997057012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a15390 0x559544dddd40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.994+0000 7f18bff6e740 1 --2- 172.21.15.3:0/2997057012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a36070 0x559544ddd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.994+0000 7f18bff6e740 1 -- 172.21.15.3:0/2997057012 >> 172.21.15.3:0/2997057012 conn(0x559544a2a460 msgr2=0x559544ddb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.994+0000 7f18bff6e740 1 -- 172.21.15.3:0/2997057012 shutdown_connections 2024-09-21T06:25:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.994+0000 7f18bff6e740 1 -- 172.21.15.3:0/2997057012 wait complete. 2024-09-21T06:25:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.994+0000 7f18bff6e740 1 -- start start 2024-09-21T06:25:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.994+0000 7f18bff6e740 1 Processor -- start 2024-09-21T06:25:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18bff6e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 0x559544de8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18bff6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a15390 0x559544de8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18bff6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a36070 0x559544df4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18bff6e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559544ba4700 con 0x559544a0d1b0 2024-09-21T06:25:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18bff6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559544cde160 con 0x559544a15390 2024-09-21T06:25:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18bff6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559544debaa0 con 0x559544a36070 2024-09-21T06:25:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18bff6e740 10 client.0 _ll_register_callbacks cb 0x559544cf79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18b2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 0x559544de8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18b2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 0x559544de8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47376/0 (socket says 172.21.15.3:47376) 2024-09-21T06:25:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18b2ffd640 1 -- 172.21.15.3:0/2558231071 learned_addr learned my addr 172.21.15.3:0/2558231071 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18b27fc640 1 --2- 172.21.15.3:0/2558231071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a15390 0x559544de8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18b37fe640 1 --2- 172.21.15.3:0/2558231071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a36070 0x559544df4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18b2ffd640 1 -- 172.21.15.3:0/2558231071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a36070 msgr2=0x559544df4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18b2ffd640 1 --2- 172.21.15.3:0/2558231071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a36070 0x559544df4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18b2ffd640 1 -- 172.21.15.3:0/2558231071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a15390 msgr2=0x559544de8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18b2ffd640 1 --2- 172.21.15.3:0/2558231071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a15390 0x559544de8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.995+0000 7f18b2ffd640 1 -- 172.21.15.3:0/2558231071 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559544d0bc90 con 0x559544a0d1b0 2024-09-21T06:25:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.996+0000 7f18b2ffd640 1 --2- 172.21.15.3:0/2558231071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 0x559544de8920 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f18a400ce00 tx=0x7f18a40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1893a2d3a6ffe825 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.996+0000 7f1892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.996+0000 7f1892ffd640 1 -- 172.21.15.3:0/2558231071 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4011970 con 0x559544a0d1b0 2024-09-21T06:25:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.996+0000 7f1892ffd640 1 -- 172.21.15.3:0/2558231071 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f18a4011b10 con 0x559544a0d1b0 2024-09-21T06:25:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.996+0000 7f18bff6e740 1 -- 172.21.15.3:0/2558231071 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a80acae0 con 0x559544a0d1b0 2024-09-21T06:25:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.996+0000 7f1892ffd640 1 -- 172.21.15.3:0/2558231071 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f18a4011d10 con 0x559544a0d1b0 2024-09-21T06:25:03.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.996+0000 7f18bff6e740 10 client.18258 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:03.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.996+0000 7f18bff6e740 1 -- 172.21.15.3:0/2558231071 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559544df7720 con 0x559544a0d1b0 2024-09-21T06:25:03.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.996+0000 7f1892ffd640 1 -- 172.21.15.3:0/2558231071 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a80acae0 con 0x559544a0d1b0 2024-09-21T06:25:03.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.996+0000 7f1892ffd640 1 -- 172.21.15.3:0/2558231071 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f18a4018070 con 0x559544a0d1b0 2024-09-21T06:25:03.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.996+0000 7f18bff6e740 10 client.18258 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:03.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.996+0000 7f18bff6e740 10 client.18258 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:25:03.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.996+0000 7f18bff6e740 20 client.18258 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:03.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.997+0000 7f18bff6e740 1 --2- 172.21.15.3:0/2558231071 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5595449f49f0 0x559544dfae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.997+0000 7f18bff6e740 4 client.18258 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:03.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.997+0000 7f18bff6e740 1 -- 172.21.15.3:0/2558231071 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559544d55df0 con 0x5595449f49f0 2024-09-21T06:25:03.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.997+0000 7f18b27fc640 1 --2- 172.21.15.3:0/2558231071 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5595449f49f0 0x559544dfae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.997+0000 7f18b27fc640 1 --2- 172.21.15.3:0/2558231071 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5595449f49f0 0x559544dfae80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bca7065a008ab2b7 server_cookie=e09f16695677ede4 in_seq=0 out_seq=0 2024-09-21T06:25:03.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.997+0000 7f1892ffd640 10 client.18258 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:25:03.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f1892ffd640 1 -- 172.21.15.3:0/2558231071 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559544d55df0 con 0x5595449f49f0 2024-09-21T06:25:03.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f1892ffd640 10 client.18258 handle_command_reply: tid=0 2024-09-21T06:25:03.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f18bff6e740 1 client.18258 shutdown 2024-09-21T06:25:03.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f18bff6e740 1 -- 172.21.15.3:0/2558231071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 msgr2=0x559544de8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f18bff6e740 1 --2- 172.21.15.3:0/2558231071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 0x559544de8920 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f18a400ce00 tx=0x7f18a40039c0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f18bff6e740 1 -- 172.21.15.3:0/2558231071 shutdown_connections 2024-09-21T06:25:03.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f18bff6e740 1 --2- 172.21.15.3:0/2558231071 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5595449f49f0 0x559544dfae80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f18bff6e740 1 --2- 172.21.15.3:0/2558231071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a36070 0x559544df4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f18bff6e740 1 --2- 172.21.15.3:0/2558231071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a15390 0x559544de8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f1892ffd640 0 client.18258 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:25:03.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f18bff6e740 1 --2- 172.21.15.3:0/2558231071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 0x559544de8920 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f18bff6e740 1 -- 172.21.15.3:0/2558231071 >> 172.21.15.3:0/2558231071 conn(0x559544a2a460 msgr2=0x559544de04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:03.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f18bff6e740 1 -- 172.21.15.3:0/2558231071 shutdown_connections 2024-09-21T06:25:03.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.998+0000 7f18bff6e740 1 -- 172.21.15.3:0/2558231071 wait complete. 2024-09-21T06:25:03.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:02.999+0000 7f18bff6e740 20 client.18258 trim_cache size 0 max 16384 2024-09-21T06:25:03.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.012+0000 7f18bff6e740 1 Processor -- start 2024-09-21T06:25:03.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.012+0000 7f18bff6e740 1 -- start start 2024-09-21T06:25:03.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.012+0000 7f18bff6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a0d1b0 0x559544de7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.012+0000 7f18bff6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a15390 0x559544dea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.012+0000 7f18bff6e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a36070 0x559544de5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.012+0000 7f18bff6e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18b41af740 con 0x559544a36070 2024-09-21T06:25:03.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.012+0000 7f18bff6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f18b41b1280 con 0x559544a0d1b0 2024-09-21T06:25:03.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.012+0000 7f18bff6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18b41afe10 con 0x559544a15390 2024-09-21T06:25:03.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.012+0000 7f18b37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a36070 0x559544de5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.012+0000 7f18b37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a36070 0x559544de5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47384/0 (socket says 172.21.15.3:47384) 2024-09-21T06:25:03.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.012+0000 7f18b37fe640 1 -- 172.21.15.3:0/827820496 learned_addr learned my addr 172.21.15.3:0/827820496 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.012+0000 7f18b27fc640 1 --2- 172.21.15.3:0/827820496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a15390 0x559544dea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.013+0000 7f18b2ffd640 1 --2- 172.21.15.3:0/827820496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a0d1b0 0x559544de7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.013+0000 7f18b37fe640 1 -- 172.21.15.3:0/827820496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a15390 msgr2=0x559544dea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.013+0000 7f18b37fe640 1 --2- 172.21.15.3:0/827820496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a15390 0x559544dea550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.013+0000 7f18b37fe640 1 -- 172.21.15.3:0/827820496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a0d1b0 msgr2=0x559544de7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.013+0000 7f18b37fe640 1 --2- 172.21.15.3:0/827820496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a0d1b0 0x559544de7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.013+0000 7f18b37fe640 1 -- 172.21.15.3:0/827820496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559544dd5460 con 0x559544a36070 2024-09-21T06:25:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.013+0000 7f18b27fc640 1 --2- 172.21.15.3:0/827820496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a15390 0x559544dea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:03.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.013+0000 7f18b37fe640 1 --2- 172.21.15.3:0/827820496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a36070 0x559544de5b10 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f18ac02ade0 tx=0x7f18ac039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fdb5c56f60b8f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:03.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.013+0000 7f18997fa640 1 -- 172.21.15.3:0/827820496 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac056070 con 0x559544a36070 2024-09-21T06:25:03.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.013+0000 7f18997fa640 1 -- 172.21.15.3:0/827820496 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18ac036040 con 0x559544a36070 2024-09-21T06:25:03.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.013+0000 7f18997fa640 1 -- 172.21.15.3:0/827820496 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac02f990 con 0x559544a36070 2024-09-21T06:25:03.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.014+0000 7f18bff6e740 1 -- 172.21.15.3:0/827820496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a36070 msgr2=0x559544de5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.014+0000 7f18bff6e740 1 --2- 172.21.15.3:0/827820496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a36070 0x559544de5b10 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f18ac02ade0 tx=0x7f18ac039000 comp rx=0 tx=0).stop 2024-09-21T06:25:03.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.014+0000 7f18bff6e740 1 -- 172.21.15.3:0/827820496 shutdown_connections 2024-09-21T06:25:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.014+0000 7f18bff6e740 1 --2- 172.21.15.3:0/827820496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a15390 0x559544dea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.014+0000 7f18bff6e740 1 --2- 172.21.15.3:0/827820496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a36070 0x559544de5b10 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.014+0000 7f18bff6e740 1 --2- 172.21.15.3:0/827820496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a0d1b0 0x559544de7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.014+0000 7f18bff6e740 1 -- 172.21.15.3:0/827820496 >> 172.21.15.3:0/827820496 conn(0x559544a2a460 msgr2=0x559544de9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.014+0000 7f18bff6e740 1 -- 172.21.15.3:0/827820496 shutdown_connections 2024-09-21T06:25:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.014+0000 7f18bff6e740 1 -- 172.21.15.3:0/827820496 wait complete. 2024-09-21T06:25:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.015+0000 7f18bff6e740 1 -- start start 2024-09-21T06:25:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.015+0000 7f18bff6e740 1 Processor -- start 2024-09-21T06:25:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.015+0000 7f18bff6e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 0x559544decca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.015+0000 7f18bff6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a15390 0x559544ded1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.015+0000 7f18bff6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a36070 0x559544ded720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.015+0000 7f18bff6e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18b41b0bb0 con 0x559544a0d1b0 2024-09-21T06:25:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.015+0000 7f18bff6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f18b41b04e0 con 0x559544a36070 2024-09-21T06:25:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.015+0000 7f18bff6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18b41ad740 con 0x559544a15390 2024-09-21T06:25:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.015+0000 7f18bff6e740 10 client.0 _ll_register_callbacks cb 0x7f18a4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.015+0000 7f18b2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 0x559544decca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.015+0000 7f18b2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 0x559544decca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47398/0 (socket says 172.21.15.3:47398) 2024-09-21T06:25:03.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.015+0000 7f18b2ffd640 1 -- 172.21.15.3:0/190098627 learned_addr learned my addr 172.21.15.3:0/190098627 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f18b37fe640 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a36070 0x559544ded720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f18b27fc640 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a15390 0x559544ded1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f18b2ffd640 1 -- 172.21.15.3:0/190098627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a15390 msgr2=0x559544ded1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f18b2ffd640 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a15390 0x559544ded1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f18b2ffd640 1 -- 172.21.15.3:0/190098627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a36070 msgr2=0x559544ded720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f18b2ffd640 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a36070 0x559544ded720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f18b2ffd640 1 -- 172.21.15.3:0/190098627 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559544e64840 con 0x559544a0d1b0 2024-09-21T06:25:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f18b37fe640 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a36070 0x559544ded720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f18b2ffd640 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 0x559544decca0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f18a4005b90 tx=0x7f18a4023a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b486b5289da6dd92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f1892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:03.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f1892ffd640 1 -- 172.21.15.3:0/190098627 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a402f060 con 0x559544a0d1b0 2024-09-21T06:25:03.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f1892ffd640 1 -- 172.21.15.3:0/190098627 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f18a4005030 con 0x559544a0d1b0 2024-09-21T06:25:03.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f18bff6e740 1 -- 172.21.15.3:0/190098627 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18ac036040 con 0x559544a0d1b0 2024-09-21T06:25:03.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f1892ffd640 1 -- 172.21.15.3:0/190098627 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f18a4005270 con 0x559544a0d1b0 2024-09-21T06:25:03.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f18bff6e740 10 client.18264 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:03.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.016+0000 7f18bff6e740 1 -- 172.21.15.3:0/190098627 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559544e65a80 con 0x559544a0d1b0 2024-09-21T06:25:03.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.017+0000 7f1892ffd640 1 -- 172.21.15.3:0/190098627 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18ac036040 con 0x559544a0d1b0 2024-09-21T06:25:03.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.017+0000 7f1892ffd640 1 -- 172.21.15.3:0/190098627 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f18a4018070 con 0x559544a0d1b0 2024-09-21T06:25:03.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.017+0000 7f18bff6e740 10 client.18264 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:03.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.017+0000 7f18bff6e740 10 client.18264 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:25:03.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.017+0000 7f18bff6e740 20 client.18264 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:03.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.017+0000 7f18bff6e740 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5595449c9da0 0x559544e68270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.017+0000 7f18bff6e740 4 client.18264 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:25:03.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.017+0000 7f18bff6e740 1 -- 172.21.15.3:0/190098627 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559544d518a0 con 0x5595449c9da0 2024-09-21T06:25:03.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.017+0000 7f18b27fc640 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5595449c9da0 0x559544e68270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f18b27fc640 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5595449c9da0 0x559544e68270 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3196911061f7af0 server_cookie=2ca36542040cbc2a in_seq=0 out_seq=0 2024-09-21T06:25:03.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f1892ffd640 10 client.18264 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:25:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f1892ffd640 1 -- 172.21.15.3:0/190098627 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559544d518a0 con 0x5595449c9da0 2024-09-21T06:25:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f1892ffd640 10 client.18264 handle_command_reply: tid=0 2024-09-21T06:25:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f18bff6e740 1 client.18264 shutdown 2024-09-21T06:25:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f18bff6e740 1 -- 172.21.15.3:0/190098627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 msgr2=0x559544decca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f18bff6e740 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 0x559544decca0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f18a4005b90 tx=0x7f18a4023a20 comp rx=0 tx=0).stop 2024-09-21T06:25:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f18bff6e740 1 -- 172.21.15.3:0/190098627 shutdown_connections 2024-09-21T06:25:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f18bff6e740 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x5595449c9da0 0x559544e68270 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f18bff6e740 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559544a15390 0x559544ded1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f18bff6e740 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559544a36070 0x559544ded720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f18bff6e740 1 --2- 172.21.15.3:0/190098627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559544a0d1b0 0x559544decca0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f18bff6e740 1 -- 172.21.15.3:0/190098627 >> 172.21.15.3:0/190098627 conn(0x559544a2a460 msgr2=0x559544dde2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:03.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f1892ffd640 0 client.18264 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:25:03.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f18bff6e740 1 -- 172.21.15.3:0/190098627 shutdown_connections 2024-09-21T06:25:03.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f18bff6e740 1 -- 172.21.15.3:0/190098627 wait complete. 2024-09-21T06:25:03.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.018+0000 7f18bff6e740 20 client.18264 trim_cache size 0 max 16384 2024-09-21T06:25:03.028 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:25:03.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.019+0000 7f18b99fa640 1 -- 172.21.15.3:0/4036230257 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f189c0b6850 msgr2=0x7f189c0b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.019+0000 7f18b99fa640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f189c0b6850 0x7f189c0b8d60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f18ac005cc0 tx=0x7f18ac005c50 comp rx=0 tx=0).stop 2024-09-21T06:25:03.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.019+0000 7f18b99fa640 1 -- 172.21.15.3:0/4036230257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b41083c0 msgr2=0x7f18b419aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.019+0000 7f18b99fa640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b41083c0 0x7f18b419aa50 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c970 tx=0x7f18a800ce30 comp rx=0 tx=0).stop 2024-09-21T06:25:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.019+0000 7f18b99fa640 1 -- 172.21.15.3:0/4036230257 shutdown_connections 2024-09-21T06:25:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.019+0000 7f18b99fa640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f189c0b6850 0x7f189c0b8d60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.019+0000 7f18b99fa640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18b4108d10 0x7f18b419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.019+0000 7f18b99fa640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b41083c0 0x7f18b419aa50 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.019+0000 7f18b99fa640 1 --2- 172.21.15.3:0/4036230257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18b4102a80 0x7f18b419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.019+0000 7f18b99fa640 1 -- 172.21.15.3:0/4036230257 >> 172.21.15.3:0/4036230257 conn(0x7f18b4079740 msgr2=0x7f18b41052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.019+0000 7f18b99fa640 1 -- 172.21.15.3:0/4036230257 shutdown_connections 2024-09-21T06:25:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:03.019+0000 7f18b99fa640 1 -- 172.21.15.3:0/4036230257 wait complete. 2024-09-21T06:25:03.775 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:03 smithi003 ceph-mon[31227]: pgmap v634: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-09-21T06:25:03.808 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:25:03.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:03 smithi137 ceph-mon[34464]: pgmap v634: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-09-21T06:25:03.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:03 smithi204 ceph-mon[34659]: pgmap v634: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-09-21T06:25:04.069 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:04.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.562+0000 7fca02125640 1 -- 172.21.15.3:0/1092536822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9fc108d20 msgr2=0x7fc9fc10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.562+0000 7fca02125640 1 --2- 172.21.15.3:0/1092536822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9fc108d20 0x7fc9fc10f550 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009ca0 tx=0x7fc9f402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.562+0000 7fca02125640 1 -- 172.21.15.3:0/1092536822 shutdown_connections 2024-09-21T06:25:04.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.562+0000 7fca02125640 1 --2- 172.21.15.3:0/1092536822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9fc1083d0 0x7fc9fc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.562+0000 7fca02125640 1 --2- 172.21.15.3:0/1092536822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9fc108d20 0x7fc9fc10f550 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.562+0000 7fca02125640 1 --2- 172.21.15.3:0/1092536822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9fc102ae0 0x7fc9fc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.562+0000 7fca02125640 1 -- 172.21.15.3:0/1092536822 >> 172.21.15.3:0/1092536822 conn(0x7fc9fc079760 msgr2=0x7fc9fc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:04.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.562+0000 7fca02125640 1 -- 172.21.15.3:0/1092536822 shutdown_connections 2024-09-21T06:25:04.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.562+0000 7fca02125640 1 -- 172.21.15.3:0/1092536822 wait complete. 2024-09-21T06:25:04.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.563+0000 7fca02125640 1 Processor -- start 2024-09-21T06:25:04.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.563+0000 7fca02125640 1 -- start start 2024-09-21T06:25:04.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fca02125640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9fc102ae0 0x7fc9fc0ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fca02125640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9fc1083d0 0x7fc9fc0ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fca02125640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9fc108d20 0x7fc9fc1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fca02125640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc9fc112990 con 0x7fc9fc1083d0 2024-09-21T06:25:04.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fca02125640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9fc112810 con 0x7fc9fc102ae0 2024-09-21T06:25:04.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fca02125640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc9fc112b10 con 0x7fc9fc108d20 2024-09-21T06:25:04.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fc9faffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9fc1083d0 0x7fc9fc0ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fc9faffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9fc1083d0 0x7fc9fc0ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47412/0 (socket says 172.21.15.3:47412) 2024-09-21T06:25:04.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fc9faffd640 1 -- 172.21.15.3:0/1758301969 learned_addr learned my addr 172.21.15.3:0/1758301969 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:04.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fc9fb7fe640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9fc102ae0 0x7fc9fc0ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fc9fbfff640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9fc108d20 0x7fc9fc1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fc9faffd640 1 -- 172.21.15.3:0/1758301969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9fc108d20 msgr2=0x7fc9fc1000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fc9faffd640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9fc108d20 0x7fc9fc1000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fc9faffd640 1 -- 172.21.15.3:0/1758301969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9fc102ae0 msgr2=0x7fc9fc0ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fc9faffd640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9fc102ae0 0x7fc9fc0ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fc9faffd640 1 -- 172.21.15.3:0/1758301969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9fc1a86f0 con 0x7fc9fc1083d0 2024-09-21T06:25:04.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.564+0000 7fc9faffd640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9fc1083d0 0x7fc9fc0ffbb0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000d740 tx=0x7fc9f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:04.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.565+0000 7fc9f8ff9640 1 -- 172.21.15.3:0/1758301969 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0017070 con 0x7fc9fc1083d0 2024-09-21T06:25:04.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.565+0000 7fc9f8ff9640 1 -- 172.21.15.3:0/1758301969 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9f000de50 con 0x7fc9fc1083d0 2024-09-21T06:25:04.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.565+0000 7fc9f8ff9640 1 -- 172.21.15.3:0/1758301969 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0004ce0 con 0x7fc9fc1083d0 2024-09-21T06:25:04.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.565+0000 7fca02125640 1 -- 172.21.15.3:0/1758301969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9fc1a89e0 con 0x7fc9fc1083d0 2024-09-21T06:25:04.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.565+0000 7fca02125640 1 -- 172.21.15.3:0/1758301969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9fc1a8ed0 con 0x7fc9fc1083d0 2024-09-21T06:25:04.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.568+0000 7fca08699740 1 Processor -- start 2024-09-21T06:25:04.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.568+0000 7fca08699740 1 -- start start 2024-09-21T06:25:04.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.568+0000 7fca08699740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a35f070 0x562b8a7062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.568+0000 7fca08699740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a3361b0 0x562b8a706800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.568+0000 7fc9f8ff9640 1 -- 172.21.15.3:0/1758301969 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fc9f0004e80 con 0x7fc9fc1083d0 2024-09-21T06:25:04.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.568+0000 7fca08699740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a33e390 0x562b8a706d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.568+0000 7fca08699740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562b8a67cea0 con 0x562b8a33e390 2024-09-21T06:25:04.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.568+0000 7fca08699740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562b8a66c2d0 con 0x562b8a35f070 2024-09-21T06:25:04.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.568+0000 7fca08699740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562b8a6700e0 con 0x562b8a3361b0 2024-09-21T06:25:04.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.569+0000 7fc9faffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a33e390 0x562b8a706d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.569+0000 7fc9fbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a35f070 0x562b8a7062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.569+0000 7fc9fbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a35f070 0x562b8a7062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:34894/0 (socket says 172.21.15.3:34894) 2024-09-21T06:25:04.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.569+0000 7fc9faffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a33e390 0x562b8a706d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47426/0 (socket says 172.21.15.3:47426) 2024-09-21T06:25:04.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.569+0000 7fc9fbfff640 1 -- 172.21.15.3:0/1798420741 learned_addr learned my addr 172.21.15.3:0/1798420741 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:04.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.569+0000 7fc9fb7fe640 1 --2- 172.21.15.3:0/1798420741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a3361b0 0x562b8a706800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.569+0000 7fc9fbfff640 1 -- 172.21.15.3:0/1798420741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a3361b0 msgr2=0x562b8a706800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.569+0000 7fc9fbfff640 1 --2- 172.21.15.3:0/1798420741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a3361b0 0x562b8a706800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.569+0000 7fc9fbfff640 1 -- 172.21.15.3:0/1798420741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a33e390 msgr2=0x562b8a706d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.569+0000 7fc9fbfff640 1 --2- 172.21.15.3:0/1798420741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a33e390 0x562b8a706d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.569+0000 7fc9fbfff640 1 -- 172.21.15.3:0/1798420741 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8a3ca650 con 0x562b8a35f070 2024-09-21T06:25:04.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.569+0000 7fc9fbfff640 1 --2- 172.21.15.3:0/1798420741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a35f070 0x562b8a7062c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4005cc0 tx=0x7fc9f4009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf78e98d3879bcff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:04.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.570+0000 7fc9f8ff9640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc9e40b6680 0x7fc9e40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.570+0000 7fc9e1ffb640 1 -- 172.21.15.3:0/1798420741 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4031cf0 con 0x562b8a35f070 2024-09-21T06:25:04.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.570+0000 7fc9e1ffb640 1 -- 172.21.15.3:0/1798420741 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9f4036040 con 0x562b8a35f070 2024-09-21T06:25:04.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.570+0000 7fc9e1ffb640 1 -- 172.21.15.3:0/1798420741 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4004240 con 0x562b8a35f070 2024-09-21T06:25:04.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.570+0000 7fc9fb7fe640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc9e40b6680 0x7fc9e40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.570+0000 7fc9f8ff9640 1 -- 172.21.15.3:0/1758301969 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fc9f00e7110 con 0x7fc9fc1083d0 2024-09-21T06:25:04.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.571+0000 7fca08699740 1 -- 172.21.15.3:0/1798420741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a35f070 msgr2=0x562b8a7062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.571+0000 7fc9fb7fe640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc9e40b6680 0x7fc9e40b8b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x562b8a704f40 tx=0x7fc9ec00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:04.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.571+0000 7fca08699740 1 --2- 172.21.15.3:0/1798420741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a35f070 0x562b8a7062c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4005cc0 tx=0x7fc9f4009800 comp rx=0 tx=0).stop 2024-09-21T06:25:04.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.571+0000 7fca08699740 1 -- 172.21.15.3:0/1798420741 shutdown_connections 2024-09-21T06:25:04.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.571+0000 7fca08699740 1 --2- 172.21.15.3:0/1798420741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a3361b0 0x562b8a706800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.571+0000 7fca08699740 1 --2- 172.21.15.3:0/1798420741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a33e390 0x562b8a706d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.571+0000 7fca08699740 1 --2- 172.21.15.3:0/1798420741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a35f070 0x562b8a7062c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.571+0000 7fca08699740 1 -- 172.21.15.3:0/1798420741 >> 172.21.15.3:0/1798420741 conn(0x562b8a353460 msgr2=0x562b8a7041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:04.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.571+0000 7fca08699740 1 -- 172.21.15.3:0/1798420741 shutdown_connections 2024-09-21T06:25:04.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.571+0000 7fca08699740 1 -- 172.21.15.3:0/1798420741 wait complete. 2024-09-21T06:25:04.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.571+0000 7fca08699740 1 -- start start 2024-09-21T06:25:04.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.571+0000 7fca08699740 1 Processor -- start 2024-09-21T06:25:04.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fca08699740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 0x562b8a711940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fca08699740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a33e390 0x562b8a711e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fca08699740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a35f070 0x562b8a71dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fca08699740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562b8a4cd700 con 0x562b8a3361b0 2024-09-21T06:25:04.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fca08699740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562b8a607160 con 0x562b8a33e390 2024-09-21T06:25:04.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fca08699740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562b8a714a40 con 0x562b8a35f070 2024-09-21T06:25:04.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fca08699740 10 client.0 _ll_register_callbacks cb 0x562b8a6209e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:04.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fc9fb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 0x562b8a711940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fc9fb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 0x562b8a711940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47428/0 (socket says 172.21.15.3:47428) 2024-09-21T06:25:04.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fc9fb7fe640 1 -- 172.21.15.3:0/2386832675 learned_addr learned my addr 172.21.15.3:0/2386832675 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:04.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fc9faffd640 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a33e390 0x562b8a711e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fc9fbfff640 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a35f070 0x562b8a71dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fc9fb7fe640 1 -- 172.21.15.3:0/2386832675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a35f070 msgr2=0x562b8a71dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fc9fb7fe640 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a35f070 0x562b8a71dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fc9fb7fe640 1 -- 172.21.15.3:0/2386832675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a33e390 msgr2=0x562b8a711e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fc9fb7fe640 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a33e390 0x562b8a711e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fc9fb7fe640 1 -- 172.21.15.3:0/2386832675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8a634c90 con 0x562b8a3361b0 2024-09-21T06:25:04.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fc9fbfff640 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a35f070 0x562b8a71dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:04.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.572+0000 7fc9faffd640 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a33e390 0x562b8a711e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:04.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fc9fb7fe640 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 0x562b8a711940 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec003d00 tx=0x7fc9ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad13fbf541b7f28b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:04.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fc9daffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:04.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fc9daffd640 1 -- 172.21.15.3:0/2386832675 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec02c060 con 0x562b8a3361b0 2024-09-21T06:25:04.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fca08699740 1 -- 172.21.15.3:0/2386832675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9f4036040 con 0x562b8a3361b0 2024-09-21T06:25:04.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fc9daffd640 1 -- 172.21.15.3:0/2386832675 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fc9ec017030 con 0x562b8a3361b0 2024-09-21T06:25:04.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fc9daffd640 1 -- 172.21.15.3:0/2386832675 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc9ec027410 con 0x562b8a3361b0 2024-09-21T06:25:04.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fca08699740 10 client.18276 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:04.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fca08699740 1 -- 172.21.15.3:0/2386832675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562b8a720770 con 0x562b8a3361b0 2024-09-21T06:25:04.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fc9daffd640 1 -- 172.21.15.3:0/2386832675 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9f4036040 con 0x562b8a3361b0 2024-09-21T06:25:04.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fc9daffd640 1 -- 172.21.15.3:0/2386832675 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fc9ec01c3f0 con 0x562b8a3361b0 2024-09-21T06:25:04.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fca08699740 10 client.18276 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:04.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fca08699740 10 client.18276 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:25:04.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fca08699740 20 client.18276 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:04.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.573+0000 7fca08699740 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562b8a31d9f0 0x562b8a723d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.574+0000 7fca08699740 4 client.18276 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:04.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.574+0000 7fca08699740 1 -- 172.21.15.3:0/2386832675 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b8a67edf0 con 0x562b8a31d9f0 2024-09-21T06:25:04.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.574+0000 7fc9faffd640 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562b8a31d9f0 0x562b8a723d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.574+0000 7fc9faffd640 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562b8a31d9f0 0x562b8a723d20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ddfa65e34c8e386 server_cookie=f93d557eee7e8581 in_seq=0 out_seq=0 2024-09-21T06:25:04.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.574+0000 7fc9daffd640 10 client.18276 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:25:04.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.575+0000 7fc9daffd640 1 -- 172.21.15.3:0/2386832675 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562b8a67edf0 con 0x562b8a31d9f0 2024-09-21T06:25:04.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.575+0000 7fc9daffd640 10 client.18276 handle_command_reply: tid=0 2024-09-21T06:25:04.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.575+0000 7fca08699740 1 client.18276 shutdown 2024-09-21T06:25:04.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.575+0000 7fca08699740 1 -- 172.21.15.3:0/2386832675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 msgr2=0x562b8a711940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.575+0000 7fca08699740 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 0x562b8a711940 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec003d00 tx=0x7fc9ec0046c0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.575+0000 7fca08699740 1 -- 172.21.15.3:0/2386832675 shutdown_connections 2024-09-21T06:25:04.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.575+0000 7fca08699740 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562b8a31d9f0 0x562b8a723d20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.575+0000 7fca08699740 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a35f070 0x562b8a71dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.575+0000 7fca08699740 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a33e390 0x562b8a711e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.575+0000 7fca08699740 1 --2- 172.21.15.3:0/2386832675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 0x562b8a711940 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.575+0000 7fca08699740 1 -- 172.21.15.3:0/2386832675 >> 172.21.15.3:0/2386832675 conn(0x562b8a353460 msgr2=0x562b8a7094a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.576+0000 7fc9daffd640 0 client.18276 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:25:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.576+0000 7fca08699740 1 -- 172.21.15.3:0/2386832675 shutdown_connections 2024-09-21T06:25:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.576+0000 7fca08699740 1 -- 172.21.15.3:0/2386832675 wait complete. 2024-09-21T06:25:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.576+0000 7fca08699740 20 client.18276 trim_cache size 0 max 16384 2024-09-21T06:25:04.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.591+0000 7fca08699740 1 Processor -- start 2024-09-21T06:25:04.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.591+0000 7fca08699740 1 -- start start 2024-09-21T06:25:04.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.591+0000 7fca08699740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 0x562b8a711e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.591+0000 7fca08699740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a33e390 0x562b8a71dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.591+0000 7fca08699740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a35f070 0x562b8a7093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.591+0000 7fca08699740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc9fc1b1950 con 0x562b8a3361b0 2024-09-21T06:25:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.591+0000 7fca08699740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9fc1b3490 con 0x562b8a33e390 2024-09-21T06:25:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.591+0000 7fca08699740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc9fc1b2020 con 0x562b8a35f070 2024-09-21T06:25:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9fb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 0x562b8a711e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9fb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 0x562b8a711e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47440/0 (socket says 172.21.15.3:47440) 2024-09-21T06:25:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9fb7fe640 1 -- 172.21.15.3:0/3952515645 learned_addr learned my addr 172.21.15.3:0/3952515645 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9faffd640 1 --2- 172.21.15.3:0/3952515645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a33e390 0x562b8a71dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9fbfff640 1 --2- 172.21.15.3:0/3952515645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a35f070 0x562b8a7093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9fb7fe640 1 -- 172.21.15.3:0/3952515645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a35f070 msgr2=0x562b8a7093c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9fb7fe640 1 --2- 172.21.15.3:0/3952515645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a35f070 0x562b8a7093c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9fb7fe640 1 -- 172.21.15.3:0/3952515645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a33e390 msgr2=0x562b8a71dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9fb7fe640 1 --2- 172.21.15.3:0/3952515645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a33e390 0x562b8a71dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9fb7fe640 1 -- 172.21.15.3:0/3952515645 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8a70fe40 con 0x562b8a3361b0 2024-09-21T06:25:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9fb7fe640 1 --2- 172.21.15.3:0/3952515645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 0x562b8a711e00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0059f0 tx=0x7fc9ec01cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3d09923b8b29da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9e17fa640 1 -- 172.21.15.3:0/3952515645 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec01c5f0 con 0x562b8a3361b0 2024-09-21T06:25:04.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9e17fa640 1 -- 172.21.15.3:0/3952515645 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9ec01ce50 con 0x562b8a3361b0 2024-09-21T06:25:04.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.592+0000 7fc9e17fa640 1 -- 172.21.15.3:0/3952515645 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec004a30 con 0x562b8a3361b0 2024-09-21T06:25:04.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.593+0000 7fca08699740 1 -- 172.21.15.3:0/3952515645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 msgr2=0x562b8a711e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.593+0000 7fca08699740 1 --2- 172.21.15.3:0/3952515645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 0x562b8a711e00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0059f0 tx=0x7fc9ec01cce0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.593+0000 7fca08699740 1 -- 172.21.15.3:0/3952515645 shutdown_connections 2024-09-21T06:25:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.593+0000 7fca08699740 1 --2- 172.21.15.3:0/3952515645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a35f070 0x562b8a7093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.593+0000 7fca08699740 1 --2- 172.21.15.3:0/3952515645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a33e390 0x562b8a71dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.593+0000 7fca08699740 1 --2- 172.21.15.3:0/3952515645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a3361b0 0x562b8a711e00 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.593+0000 7fca08699740 1 -- 172.21.15.3:0/3952515645 >> 172.21.15.3:0/3952515645 conn(0x562b8a353460 msgr2=0x562b8a7116f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.593+0000 7fca08699740 1 -- 172.21.15.3:0/3952515645 shutdown_connections 2024-09-21T06:25:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.593+0000 7fca08699740 1 -- 172.21.15.3:0/3952515645 wait complete. 2024-09-21T06:25:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.594+0000 7fca08699740 1 -- start start 2024-09-21T06:25:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.594+0000 7fca08699740 1 Processor -- start 2024-09-21T06:25:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.594+0000 7fca08699740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a3361b0 0x562b8a71a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.594+0000 7fca08699740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a33e390 0x562b8a7158f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.594+0000 7fca08699740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a35f070 0x562b8a715e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.594+0000 7fca08699740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc9fc1b2dc0 con 0x562b8a33e390 2024-09-21T06:25:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.594+0000 7fca08699740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9fc1b26f0 con 0x562b8a35f070 2024-09-21T06:25:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.594+0000 7fca08699740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc9fc1af950 con 0x562b8a3361b0 2024-09-21T06:25:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.594+0000 7fca08699740 10 client.0 _ll_register_callbacks cb 0x7fc9ec004300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.594+0000 7fc9faffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a33e390 0x562b8a7158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.594+0000 7fc9faffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a33e390 0x562b8a7158f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47456/0 (socket says 172.21.15.3:47456) 2024-09-21T06:25:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.594+0000 7fc9faffd640 1 -- 172.21.15.3:0/2225312099 learned_addr learned my addr 172.21.15.3:0/2225312099 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fc9fb7fe640 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a3361b0 0x562b8a71a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fc9fbfff640 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a35f070 0x562b8a715e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fc9faffd640 1 -- 172.21.15.3:0/2225312099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a3361b0 msgr2=0x562b8a71a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fc9faffd640 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a3361b0 0x562b8a71a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fc9faffd640 1 -- 172.21.15.3:0/2225312099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a35f070 msgr2=0x562b8a715e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fc9faffd640 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a35f070 0x562b8a715e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fc9faffd640 1 -- 172.21.15.3:0/2225312099 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8a7165a0 con 0x562b8a33e390 2024-09-21T06:25:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fc9fbfff640 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a35f070 0x562b8a715e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fc9faffd640 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a33e390 0x562b8a7158f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00e4920 tx=0x7fc9f00eac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca33adb8e4f5fb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:04.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fc9daffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:04.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fc9daffd640 1 -- 172.21.15.3:0/2225312099 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0020040 con 0x562b8a33e390 2024-09-21T06:25:04.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fc9daffd640 1 -- 172.21.15.3:0/2225312099 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7fc9f00940b0 con 0x562b8a33e390 2024-09-21T06:25:04.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fca08699740 1 -- 172.21.15.3:0/2225312099 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9ec01ce50 con 0x562b8a33e390 2024-09-21T06:25:04.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fc9daffd640 1 -- 172.21.15.3:0/2225312099 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc9f00ac8c0 con 0x562b8a33e390 2024-09-21T06:25:04.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fca08699740 10 client.18282 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:04.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.595+0000 7fca08699740 1 -- 172.21.15.3:0/2225312099 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562b8a78e640 con 0x562b8a33e390 2024-09-21T06:25:04.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.596+0000 7fc9daffd640 1 -- 172.21.15.3:0/2225312099 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9ec01ce50 con 0x562b8a33e390 2024-09-21T06:25:04.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.596+0000 7fc9daffd640 1 -- 172.21.15.3:0/2225312099 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fc9f00e90b0 con 0x562b8a33e390 2024-09-21T06:25:04.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.596+0000 7fca08699740 10 client.18282 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:04.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.596+0000 7fca08699740 10 client.18282 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:25:04.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.596+0000 7fca08699740 20 client.18282 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:04.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.596+0000 7fca08699740 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562b8a2f2da0 0x562b8a790da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.596+0000 7fca08699740 4 client.18282 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:25:04.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.596+0000 7fca08699740 1 -- 172.21.15.3:0/2225312099 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b8a67a8a0 con 0x562b8a2f2da0 2024-09-21T06:25:04.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.596+0000 7fc9fb7fe640 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562b8a2f2da0 0x562b8a790da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fc9daffd640 10 client.18282 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:25:04.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fc9fb7fe640 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562b8a2f2da0 0x562b8a790da0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f032a127573ff16 server_cookie=c87da9b0422b7b04 in_seq=0 out_seq=0 2024-09-21T06:25:04.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fc9daffd640 1 -- 172.21.15.3:0/2225312099 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b8a67a8a0 con 0x562b8a2f2da0 2024-09-21T06:25:04.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fc9daffd640 10 client.18282 handle_command_reply: tid=0 2024-09-21T06:25:04.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fca08699740 1 client.18282 shutdown 2024-09-21T06:25:04.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fca08699740 1 -- 172.21.15.3:0/2225312099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a33e390 msgr2=0x562b8a7158f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fca08699740 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a33e390 0x562b8a7158f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00e4920 tx=0x7fc9f00eac60 comp rx=0 tx=0).stop 2024-09-21T06:25:04.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fca08699740 1 -- 172.21.15.3:0/2225312099 shutdown_connections 2024-09-21T06:25:04.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fca08699740 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x562b8a2f2da0 0x562b8a790da0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fc9daffd640 0 client.18282 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:25:04.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fca08699740 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b8a35f070 0x562b8a715e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fca08699740 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b8a33e390 0x562b8a7158f0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fca08699740 1 --2- 172.21.15.3:0/2225312099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b8a3361b0 0x562b8a71a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fca08699740 1 -- 172.21.15.3:0/2225312099 >> 172.21.15.3:0/2225312099 conn(0x562b8a353460 msgr2=0x562b8a7093c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:04.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fca08699740 1 -- 172.21.15.3:0/2225312099 shutdown_connections 2024-09-21T06:25:04.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fca08699740 1 -- 172.21.15.3:0/2225312099 wait complete. 2024-09-21T06:25:04.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.597+0000 7fca08699740 20 client.18282 trim_cache size 0 max 16384 2024-09-21T06:25:04.607 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:25:04.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.598+0000 7fca02125640 1 -- 172.21.15.3:0/1758301969 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc9e40b6680 msgr2=0x7fc9e40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.598+0000 7fca02125640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc9e40b6680 0x7fc9e40b8b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x562b8a704f40 tx=0x7fc9ec00b390 comp rx=0 tx=0).stop 2024-09-21T06:25:04.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.598+0000 7fca02125640 1 -- 172.21.15.3:0/1758301969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9fc1083d0 msgr2=0x7fc9fc0ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.598+0000 7fca02125640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9fc1083d0 0x7fc9fc0ffbb0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000d740 tx=0x7fc9f000dc00 comp rx=0 tx=0).stop 2024-09-21T06:25:04.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.598+0000 7fca02125640 1 -- 172.21.15.3:0/1758301969 shutdown_connections 2024-09-21T06:25:04.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.598+0000 7fca02125640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fc9e40b6680 0x7fc9e40b8b90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.598+0000 7fca02125640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9fc108d20 0x7fc9fc1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.598+0000 7fca02125640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9fc1083d0 0x7fc9fc0ffbb0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.598+0000 7fca02125640 1 --2- 172.21.15.3:0/1758301969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9fc102ae0 0x7fc9fc0ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.598+0000 7fca02125640 1 -- 172.21.15.3:0/1758301969 >> 172.21.15.3:0/1758301969 conn(0x7fc9fc079760 msgr2=0x7fc9fc06be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:04.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.598+0000 7fca02125640 1 -- 172.21.15.3:0/1758301969 shutdown_connections 2024-09-21T06:25:04.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:04.598+0000 7fca02125640 1 -- 172.21.15.3:0/1758301969 wait complete. 2024-09-21T06:25:05.345 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:25:05.610 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:05.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:05 smithi003 ceph-mon[31227]: pgmap v635: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 26 op/s 2024-09-21T06:25:05.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:05 smithi137 ceph-mon[34464]: pgmap v635: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 26 op/s 2024-09-21T06:25:05.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:05 smithi204 ceph-mon[34659]: pgmap v635: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 26 op/s 2024-09-21T06:25:06.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.114+0000 7f1824b4d640 1 -- 172.21.15.3:0/4067998770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820104a30 msgr2=0x7f1820104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.114+0000 7f1824b4d640 1 --2- 172.21.15.3:0/4067998770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820104a30 0x7f1820104e40 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1810009900 tx=0x7f181002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.115+0000 7f1824b4d640 1 -- 172.21.15.3:0/4067998770 shutdown_connections 2024-09-21T06:25:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.115+0000 7f1824b4d640 1 --2- 172.21.15.3:0/4067998770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f182010de60 0x7f1820110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.115+0000 7f1824b4d640 1 --2- 172.21.15.3:0/4067998770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1820105380 0x7f182010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.115+0000 7f1824b4d640 1 --2- 172.21.15.3:0/4067998770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820104a30 0x7f1820104e40 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.115+0000 7f1824b4d640 1 -- 172.21.15.3:0/4067998770 >> 172.21.15.3:0/4067998770 conn(0x7f1820079760 msgr2=0x7f1820079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.115+0000 7f1824b4d640 1 -- 172.21.15.3:0/4067998770 shutdown_connections 2024-09-21T06:25:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.115+0000 7f1824b4d640 1 -- 172.21.15.3:0/4067998770 wait complete. 2024-09-21T06:25:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.116+0000 7f1824b4d640 1 Processor -- start 2024-09-21T06:25:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.116+0000 7f1824b4d640 1 -- start start 2024-09-21T06:25:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.116+0000 7f1824b4d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820104a30 0x7f1820102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.116+0000 7f1824b4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1820105380 0x7f1820102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.116+0000 7f1824b4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f182010de60 0x7f18201031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.116+0000 7f1824b4d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1820113ac0 con 0x7f1820104a30 2024-09-21T06:25:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.116+0000 7f1824b4d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1820113940 con 0x7f182010de60 2024-09-21T06:25:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.116+0000 7f1824b4d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1820113c40 con 0x7f1820105380 2024-09-21T06:25:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.116+0000 7f181e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820104a30 0x7f1820102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.116+0000 7f181e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820104a30 0x7f1820102770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44356/0 (socket says 172.21.15.3:44356) 2024-09-21T06:25:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.116+0000 7f181e59c640 1 -- 172.21.15.3:0/2207683856 learned_addr learned my addr 172.21.15.3:0/2207683856 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.116+0000 7f181dd9b640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1820105380 0x7f1820102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.116+0000 7f181ed9d640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f182010de60 0x7f18201031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.117+0000 7f181e59c640 1 -- 172.21.15.3:0/2207683856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1820105380 msgr2=0x7f1820102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.117+0000 7f181e59c640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1820105380 0x7f1820102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.117+0000 7f181e59c640 1 -- 172.21.15.3:0/2207683856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f182010de60 msgr2=0x7f18201031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.117+0000 7f181e59c640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f182010de60 0x7f18201031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.117+0000 7f181e59c640 1 -- 172.21.15.3:0/2207683856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18201a86e0 con 0x7f1820104a30 2024-09-21T06:25:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.117+0000 7f181dd9b640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1820105380 0x7f1820102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.117+0000 7f181ed9d640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f182010de60 0x7f18201031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.117+0000 7f181e59c640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820104a30 0x7f1820102770 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1810031a90 tx=0x7f1810004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.117+0000 7f180f7fe640 1 -- 172.21.15.3:0/2207683856 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181003f030 con 0x7f1820104a30 2024-09-21T06:25:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.117+0000 7f180f7fe640 1 -- 172.21.15.3:0/2207683856 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1810003cb0 con 0x7f1820104a30 2024-09-21T06:25:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.117+0000 7f1824b4d640 1 -- 172.21.15.3:0/2207683856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18201a8970 con 0x7f1820104a30 2024-09-21T06:25:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.117+0000 7f180f7fe640 1 -- 172.21.15.3:0/2207683856 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810035070 con 0x7f1820104a30 2024-09-21T06:25:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.117+0000 7f1824b4d640 1 -- 172.21.15.3:0/2207683856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18201a8e50 con 0x7f1820104a30 2024-09-21T06:25:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.120+0000 7f182b0c1740 1 Processor -- start 2024-09-21T06:25:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.120+0000 7f180f7fe640 1 -- 172.21.15.3:0/2207683856 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f1810042030 con 0x7f1820104a30 2024-09-21T06:25:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.120+0000 7f182b0c1740 1 -- start start 2024-09-21T06:25:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.120+0000 7f182b0c1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd0c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.120+0000 7f182b0c1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd0cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.120+0000 7f182b0c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d944390 0x55ad2dd0d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.120+0000 7f182b0c1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ad2dc82ea0 con 0x55ad2d965070 2024-09-21T06:25:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.120+0000 7f182b0c1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ad2dc722d0 con 0x55ad2d93c1b0 2024-09-21T06:25:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.120+0000 7f182b0c1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ad2dc760e0 con 0x55ad2d944390 2024-09-21T06:25:06.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.120+0000 7f181ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd0c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.120+0000 7f181ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd0c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44372/0 (socket says 172.21.15.3:44372) 2024-09-21T06:25:06.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.120+0000 7f181ed9d640 1 -- 172.21.15.3:0/1295376448 learned_addr learned my addr 172.21.15.3:0/1295376448 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:06.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.121+0000 7f181ed9d640 1 -- 172.21.15.3:0/1295376448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d944390 msgr2=0x55ad2dd0d120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:25:06.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.121+0000 7f181ed9d640 1 --2- 172.21.15.3:0/1295376448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d944390 0x55ad2dd0d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.121+0000 7f181ed9d640 1 -- 172.21.15.3:0/1295376448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d93c1b0 msgr2=0x55ad2dd0cbe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:25:06.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.121+0000 7f181ed9d640 1 --2- 172.21.15.3:0/1295376448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd0cbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.121+0000 7f181ed9d640 1 -- 172.21.15.3:0/1295376448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad2d9d0650 con 0x55ad2d965070 2024-09-21T06:25:06.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.121+0000 7f180f7fe640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18080b6680 0x7f18080b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.121+0000 7f180f7fe640 1 -- 172.21.15.3:0/2207683856 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f1810107dc0 con 0x7f1820104a30 2024-09-21T06:25:06.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.122+0000 7f181ed9d640 1 --2- 172.21.15.3:0/1295376448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd0c6a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f181800bae0 tx=0x7f181800d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db10f4ac001e2eea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:06.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.122+0000 7f181e59c640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18080b6680 0x7f18080b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.122+0000 7f180cff9640 1 -- 172.21.15.3:0/1295376448 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818017420 con 0x55ad2d965070 2024-09-21T06:25:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.122+0000 7f180cff9640 1 -- 172.21.15.3:0/1295376448 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1818017a70 con 0x55ad2d965070 2024-09-21T06:25:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.122+0000 7f180cff9640 1 -- 172.21.15.3:0/1295376448 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181800fa20 con 0x55ad2d965070 2024-09-21T06:25:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.123+0000 7f182b0c1740 1 -- 172.21.15.3:0/1295376448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 msgr2=0x55ad2dd0c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.123+0000 7f182b0c1740 1 --2- 172.21.15.3:0/1295376448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd0c6a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f181800bae0 tx=0x7f181800d340 comp rx=0 tx=0).stop 2024-09-21T06:25:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.123+0000 7f181e59c640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18080b6680 0x7f18080b8b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55ad2dd0b320 tx=0x7f181002a3f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.123+0000 7f182b0c1740 1 -- 172.21.15.3:0/1295376448 shutdown_connections 2024-09-21T06:25:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.123+0000 7f182b0c1740 1 --2- 172.21.15.3:0/1295376448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d944390 0x55ad2dd0d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.123+0000 7f182b0c1740 1 --2- 172.21.15.3:0/1295376448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd0cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.123+0000 7f182b0c1740 1 --2- 172.21.15.3:0/1295376448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd0c6a0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.123+0000 7f182b0c1740 1 -- 172.21.15.3:0/1295376448 >> 172.21.15.3:0/1295376448 conn(0x55ad2d959460 msgr2=0x55ad2dd0a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.123+0000 7f182b0c1740 1 -- 172.21.15.3:0/1295376448 shutdown_connections 2024-09-21T06:25:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.123+0000 7f182b0c1740 1 -- 172.21.15.3:0/1295376448 wait complete. 2024-09-21T06:25:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.124+0000 7f182b0c1740 1 -- start start 2024-09-21T06:25:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.124+0000 7f182b0c1740 1 Processor -- start 2024-09-21T06:25:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.124+0000 7f182b0c1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd17d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.124+0000 7f182b0c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d944390 0x55ad2dd182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.124+0000 7f182b0c1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd241a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.124+0000 7f182b0c1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ad2dad3700 con 0x55ad2d965070 2024-09-21T06:25:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.124+0000 7f182b0c1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ad2dc0d160 con 0x55ad2d93c1b0 2024-09-21T06:25:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.124+0000 7f182b0c1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ad2dd1ae80 con 0x55ad2d944390 2024-09-21T06:25:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.124+0000 7f182b0c1740 10 client.0 _ll_register_callbacks cb 0x55ad2dc269e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f181ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd241a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f181ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd241a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44378/0 (socket says 172.21.15.3:44378) 2024-09-21T06:25:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f181ed9d640 1 -- 172.21.15.3:0/409782020 learned_addr learned my addr 172.21.15.3:0/409782020 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f181dd9b640 1 --2- 172.21.15.3:0/409782020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d944390 0x55ad2dd182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f181ed9d640 1 --2- 172.21.15.3:0/409782020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd17d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f181ed9d640 1 -- 172.21.15.3:0/409782020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d944390 msgr2=0x55ad2dd182c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:06.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f181ed9d640 1 --2- 172.21.15.3:0/409782020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d944390 0x55ad2dd182c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f181ed9d640 1 -- 172.21.15.3:0/409782020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d93c1b0 msgr2=0x55ad2dd17d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:06.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f181ed9d640 1 --2- 172.21.15.3:0/409782020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd17d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f181ed9d640 1 -- 172.21.15.3:0/409782020 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad2dc3ac90 con 0x55ad2d965070 2024-09-21T06:25:06.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f181ed9d640 1 --2- 172.21.15.3:0/409782020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd241a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f181800f680 tx=0x7f181800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbb5502f4aa8fcc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:06.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f1805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:06.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f1805ffb640 1 -- 172.21.15.3:0/409782020 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181800f9c0 con 0x55ad2d965070 2024-09-21T06:25:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f182b0c1740 1 -- 172.21.15.3:0/409782020 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1818017a70 con 0x55ad2d965070 2024-09-21T06:25:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.125+0000 7f1805ffb640 1 -- 172.21.15.3:0/409782020 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f1818017420 con 0x55ad2d965070 2024-09-21T06:25:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.126+0000 7f1805ffb640 1 -- 172.21.15.3:0/409782020 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1818017620 con 0x55ad2d965070 2024-09-21T06:25:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.126+0000 7f182b0c1740 10 client.18294 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.126+0000 7f182b0c1740 1 -- 172.21.15.3:0/409782020 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ad2dd26b80 con 0x55ad2d965070 2024-09-21T06:25:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.126+0000 7f1805ffb640 1 -- 172.21.15.3:0/409782020 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1818017a70 con 0x55ad2d965070 2024-09-21T06:25:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.126+0000 7f1805ffb640 1 -- 172.21.15.3:0/409782020 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f18180233f0 con 0x55ad2d965070 2024-09-21T06:25:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.126+0000 7f182b0c1740 10 client.18294 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:06.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.126+0000 7f182b0c1740 10 client.18294 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:25:06.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.126+0000 7f182b0c1740 20 client.18294 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:06.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.126+0000 7f182b0c1740 1 --2- 172.21.15.3:0/409782020 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55ad2d9239f0 0x55ad2dd29f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.126+0000 7f182b0c1740 4 client.18294 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:06.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.126+0000 7f182b0c1740 1 -- 172.21.15.3:0/409782020 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad2dc84df0 con 0x55ad2d9239f0 2024-09-21T06:25:06.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.126+0000 7f181dd9b640 1 --2- 172.21.15.3:0/409782020 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55ad2d9239f0 0x55ad2dd29f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.127+0000 7f181dd9b640 1 --2- 172.21.15.3:0/409782020 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55ad2d9239f0 0x55ad2dd29f40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2c5d6e12a28a8ba server_cookie=b0d7b4877fbe4841 in_seq=0 out_seq=0 2024-09-21T06:25:06.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.127+0000 7f1805ffb640 10 client.18294 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:25:06.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f1805ffb640 1 -- 172.21.15.3:0/409782020 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ad2dc84df0 con 0x55ad2d9239f0 2024-09-21T06:25:06.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f1805ffb640 10 client.18294 handle_command_reply: tid=0 2024-09-21T06:25:06.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f182b0c1740 1 client.18294 shutdown 2024-09-21T06:25:06.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f182b0c1740 1 -- 172.21.15.3:0/409782020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 msgr2=0x55ad2dd241a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:06.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f182b0c1740 1 --2- 172.21.15.3:0/409782020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd241a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f181800f680 tx=0x7f181800f580 comp rx=0 tx=0).stop 2024-09-21T06:25:06.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f182b0c1740 1 -- 172.21.15.3:0/409782020 shutdown_connections 2024-09-21T06:25:06.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f182b0c1740 1 --2- 172.21.15.3:0/409782020 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55ad2d9239f0 0x55ad2dd29f40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f182b0c1740 1 --2- 172.21.15.3:0/409782020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d944390 0x55ad2dd182c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f182b0c1740 1 --2- 172.21.15.3:0/409782020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd241a0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f182b0c1740 1 --2- 172.21.15.3:0/409782020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd17d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f1805ffb640 0 client.18294 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:25:06.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f182b0c1740 1 -- 172.21.15.3:0/409782020 >> 172.21.15.3:0/409782020 conn(0x55ad2d959460 msgr2=0x55ad2dd0f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:06.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f182b0c1740 1 -- 172.21.15.3:0/409782020 shutdown_connections 2024-09-21T06:25:06.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f182b0c1740 1 -- 172.21.15.3:0/409782020 wait complete. 2024-09-21T06:25:06.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.128+0000 7f182b0c1740 20 client.18294 trim_cache size 0 max 16384 2024-09-21T06:25:06.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.141+0000 7f182b0c1740 1 Processor -- start 2024-09-21T06:25:06.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.141+0000 7f182b0c1740 1 -- start start 2024-09-21T06:25:06.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f182b0c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd199b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f182b0c1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d944390 0x55ad2dd09bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f182b0c1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd0a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f182b0c1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18201ad4b0 con 0x55ad2d965070 2024-09-21T06:25:06.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f182b0c1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f18201aeff0 con 0x55ad2d944390 2024-09-21T06:25:06.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f182b0c1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18201adb80 con 0x55ad2d93c1b0 2024-09-21T06:25:06.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f181ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd0a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f181ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd0a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44384/0 (socket says 172.21.15.3:44384) 2024-09-21T06:25:06.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f181ed9d640 1 -- 172.21.15.3:0/921189669 learned_addr learned my addr 172.21.15.3:0/921189669 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:06.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f181ed9d640 1 --2- 172.21.15.3:0/921189669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f181dd9b640 1 --2- 172.21.15.3:0/921189669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d944390 0x55ad2dd09bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f181ed9d640 1 -- 172.21.15.3:0/921189669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d93c1b0 msgr2=0x55ad2dd199b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:06.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f181ed9d640 1 --2- 172.21.15.3:0/921189669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd199b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f181ed9d640 1 -- 172.21.15.3:0/921189669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d944390 msgr2=0x55ad2dd09bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:06.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f181ed9d640 1 --2- 172.21.15.3:0/921189669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d944390 0x55ad2dd09bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f181ed9d640 1 -- 172.21.15.3:0/921189669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad2dd0b150 con 0x55ad2d965070 2024-09-21T06:25:06.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f181ed9d640 1 --2- 172.21.15.3:0/921189669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd0a100 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f181800f650 tx=0x7f181800d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dad9b596ae90fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:06.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.142+0000 7f1807fff640 1 -- 172.21.15.3:0/921189669 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181803f070 con 0x55ad2d965070 2024-09-21T06:25:06.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.143+0000 7f1807fff640 1 -- 172.21.15.3:0/921189669 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f181802b070 con 0x55ad2d965070 2024-09-21T06:25:06.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.143+0000 7f1807fff640 1 -- 172.21.15.3:0/921189669 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181800fa20 con 0x55ad2d965070 2024-09-21T06:25:06.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.143+0000 7f182b0c1740 1 -- 172.21.15.3:0/921189669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 msgr2=0x55ad2dd0a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:06.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.143+0000 7f182b0c1740 1 --2- 172.21.15.3:0/921189669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd0a100 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f181800f650 tx=0x7f181800d340 comp rx=0 tx=0).stop 2024-09-21T06:25:06.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.143+0000 7f182b0c1740 1 -- 172.21.15.3:0/921189669 shutdown_connections 2024-09-21T06:25:06.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.143+0000 7f182b0c1740 1 --2- 172.21.15.3:0/921189669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd0a100 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.143+0000 7f182b0c1740 1 --2- 172.21.15.3:0/921189669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d944390 0x55ad2dd09bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.143+0000 7f182b0c1740 1 --2- 172.21.15.3:0/921189669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.143+0000 7f182b0c1740 1 -- 172.21.15.3:0/921189669 >> 172.21.15.3:0/921189669 conn(0x55ad2d959460 msgr2=0x55ad2dd192a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:06.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.143+0000 7f182b0c1740 1 -- 172.21.15.3:0/921189669 shutdown_connections 2024-09-21T06:25:06.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.143+0000 7f182b0c1740 1 -- 172.21.15.3:0/921189669 wait complete. 2024-09-21T06:25:06.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f182b0c1740 1 -- start start 2024-09-21T06:25:06.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f182b0c1740 1 Processor -- start 2024-09-21T06:25:06.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f182b0c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd2af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f182b0c1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d944390 0x55ad2dd2b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f182b0c1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd2ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f182b0c1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18201ae920 con 0x55ad2d965070 2024-09-21T06:25:06.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f182b0c1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f18201ae250 con 0x55ad2d944390 2024-09-21T06:25:06.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f182b0c1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18201ab4d0 con 0x55ad2d93c1b0 2024-09-21T06:25:06.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f182b0c1740 10 client.0 _ll_register_callbacks cb 0x7f181800b930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:06.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f181ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd2ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f181ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd2ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44394/0 (socket says 172.21.15.3:44394) 2024-09-21T06:25:06.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f181ed9d640 1 -- 172.21.15.3:0/3918324068 learned_addr learned my addr 172.21.15.3:0/3918324068 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:06.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f181ed9d640 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd2af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.144+0000 7f181dd9b640 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d944390 0x55ad2dd2b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f181ed9d640 1 -- 172.21.15.3:0/3918324068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d93c1b0 msgr2=0x55ad2dd2af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:06.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f181ed9d640 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd2af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f181ed9d640 1 -- 172.21.15.3:0/3918324068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d944390 msgr2=0x55ad2dd2b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:06.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f181ed9d640 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d944390 0x55ad2dd2b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f181ed9d640 1 -- 172.21.15.3:0/3918324068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad2dd936f0 con 0x55ad2d965070 2024-09-21T06:25:06.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f181dd9b640 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d944390 0x55ad2dd2b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:06.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f181ed9d640 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd2ba00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1818000c00 tx=0x7f1818005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f9bdb0fe42bc41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:06.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f1805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:06.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f1805ffb640 1 -- 172.21.15.3:0/3918324068 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181803f040 con 0x55ad2d965070 2024-09-21T06:25:06.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f1805ffb640 1 -- 172.21.15.3:0/3918324068 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8507+0+0 (secure 0 0 0) 0x7f1818023a80 con 0x55ad2d965070 2024-09-21T06:25:06.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f182b0c1740 1 -- 172.21.15.3:0/3918324068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f181802b070 con 0x55ad2d965070 2024-09-21T06:25:06.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f1805ffb640 1 -- 172.21.15.3:0/3918324068 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1818023620 con 0x55ad2d965070 2024-09-21T06:25:06.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f182b0c1740 10 client.18300 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:06.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f182b0c1740 1 -- 172.21.15.3:0/3918324068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ad2dd94950 con 0x55ad2d965070 2024-09-21T06:25:06.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.145+0000 7f1805ffb640 1 -- 172.21.15.3:0/3918324068 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f181802b070 con 0x55ad2d965070 2024-09-21T06:25:06.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.146+0000 7f1805ffb640 1 -- 172.21.15.3:0/3918324068 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f1818026880 con 0x55ad2d965070 2024-09-21T06:25:06.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.146+0000 7f182b0c1740 10 client.18300 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:06.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.146+0000 7f182b0c1740 10 client.18300 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:25:06.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.146+0000 7f182b0c1740 20 client.18300 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:06.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.146+0000 7f182b0c1740 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55ad2d8f8da0 0x55ad2dd96ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:06.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.146+0000 7f182b0c1740 4 client.18300 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:25:06.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.146+0000 7f182b0c1740 1 -- 172.21.15.3:0/3918324068 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad2dc808a0 con 0x55ad2d8f8da0 2024-09-21T06:25:06.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.146+0000 7f181dd9b640 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55ad2d8f8da0 0x55ad2dd96ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:06.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.146+0000 7f181dd9b640 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55ad2d8f8da0 0x55ad2dd96ee0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=831a5036dca96546 server_cookie=797dced77f37d5a8 in_seq=0 out_seq=0 2024-09-21T06:25:06.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.146+0000 7f1805ffb640 10 client.18300 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:25:06.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f1805ffb640 1 -- 172.21.15.3:0/3918324068 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad2dc808a0 con 0x55ad2d8f8da0 2024-09-21T06:25:06.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f1805ffb640 10 client.18300 handle_command_reply: tid=0 2024-09-21T06:25:06.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f182b0c1740 1 client.18300 shutdown 2024-09-21T06:25:06.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f182b0c1740 1 -- 172.21.15.3:0/3918324068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 msgr2=0x55ad2dd2ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:06.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f182b0c1740 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd2ba00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1818000c00 tx=0x7f1818005c30 comp rx=0 tx=0).stop 2024-09-21T06:25:06.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f182b0c1740 1 -- 172.21.15.3:0/3918324068 shutdown_connections 2024-09-21T06:25:06.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f182b0c1740 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55ad2d8f8da0 0x55ad2dd96ee0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f182b0c1740 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad2d965070 0x55ad2dd2ba00 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f182b0c1740 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ad2d944390 0x55ad2dd2b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f1805ffb640 0 client.18300 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:25:06.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f182b0c1740 1 --2- 172.21.15.3:0/3918324068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad2d93c1b0 0x55ad2dd2af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f182b0c1740 1 -- 172.21.15.3:0/3918324068 >> 172.21.15.3:0/3918324068 conn(0x55ad2d959460 msgr2=0x55ad2dd0e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:06.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f182b0c1740 1 -- 172.21.15.3:0/3918324068 shutdown_connections 2024-09-21T06:25:06.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f182b0c1740 1 -- 172.21.15.3:0/3918324068 wait complete. 2024-09-21T06:25:06.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f182b0c1740 20 client.18300 trim_cache size 0 max 16384 2024-09-21T06:25:06.157 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:25:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f1824b4d640 1 -- 172.21.15.3:0/2207683856 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18080b6680 msgr2=0x7f18080b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f1824b4d640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18080b6680 0x7f18080b8b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55ad2dd0b320 tx=0x7f181002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f1824b4d640 1 -- 172.21.15.3:0/2207683856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820104a30 msgr2=0x7f1820102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.147+0000 7f1824b4d640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820104a30 0x7f1820102770 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1810031a90 tx=0x7f1810004270 comp rx=0 tx=0).stop 2024-09-21T06:25:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.148+0000 7f1824b4d640 1 -- 172.21.15.3:0/2207683856 shutdown_connections 2024-09-21T06:25:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.148+0000 7f1824b4d640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f18080b6680 0x7f18080b8b90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.148+0000 7f1824b4d640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1820105380 0x7f1820102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.148+0000 7f1824b4d640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f182010de60 0x7f18201031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.148+0000 7f1824b4d640 1 --2- 172.21.15.3:0/2207683856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820104a30 0x7f1820102770 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.148+0000 7f1824b4d640 1 -- 172.21.15.3:0/2207683856 >> 172.21.15.3:0/2207683856 conn(0x7f1820079760 msgr2=0x7f1820100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.148+0000 7f1824b4d640 1 -- 172.21.15.3:0/2207683856 shutdown_connections 2024-09-21T06:25:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:06.148+0000 7f1824b4d640 1 -- 172.21.15.3:0/2207683856 wait complete. 2024-09-21T06:25:06.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:06 smithi003 ceph-mon[31227]: pgmap v636: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-09-21T06:25:06.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:06 smithi137 ceph-mon[34464]: pgmap v636: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-09-21T06:25:06.897 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:25:06.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:06 smithi204 ceph-mon[34659]: pgmap v636: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-09-21T06:25:07.363 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:07.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:07 smithi003 ceph-mon[31227]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:25:07.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:07 smithi137 ceph-mon[34464]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:25:07.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:07 smithi204 ceph-mon[34659]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:25:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.958+0000 7f359230d640 1 -- 172.21.15.3:0/2602638845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f358c10b6d0 msgr2=0x7f358c10bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.958+0000 7f359230d640 1 --2- 172.21.15.3:0/2602638845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f358c10b6d0 0x7f358c10bae0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f357c009900 tx=0x7f357c02a400 comp rx=0 tx=0).stop 2024-09-21T06:25:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.959+0000 7f359230d640 1 -- 172.21.15.3:0/2602638845 shutdown_connections 2024-09-21T06:25:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.959+0000 7f359230d640 1 --2- 172.21.15.3:0/2602638845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f358c10b6d0 0x7f358c10bae0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.959+0000 7f359230d640 1 --2- 172.21.15.3:0/2602638845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f358c10c020 0x7f358c112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.959+0000 7f359230d640 1 --2- 172.21.15.3:0/2602638845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f358c105d90 0x7f358c1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.959+0000 7f359230d640 1 -- 172.21.15.3:0/2602638845 >> 172.21.15.3:0/2602638845 conn(0x7f358c1017c0 msgr2=0x7f358c103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.959+0000 7f359230d640 1 -- 172.21.15.3:0/2602638845 shutdown_connections 2024-09-21T06:25:07.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.960+0000 7f359230d640 1 -- 172.21.15.3:0/2602638845 wait complete. 2024-09-21T06:25:07.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.960+0000 7f359230d640 1 Processor -- start 2024-09-21T06:25:07.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.961+0000 7f359230d640 1 -- start start 2024-09-21T06:25:07.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.961+0000 7f359230d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f358c105d90 0x7f358c111710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.961+0000 7f359230d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f358c10b6d0 0x7f358c111c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.961+0000 7f359230d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f358c10c020 0x7f358c1a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.962+0000 7f359230d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f358c078870 con 0x7f358c105d90 2024-09-21T06:25:07.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.962+0000 7f359230d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f358c0786f0 con 0x7f358c10b6d0 2024-09-21T06:25:07.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.962+0000 7f359230d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f358c0789f0 con 0x7f358c10c020 2024-09-21T06:25:07.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.962+0000 7f358b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f358c10b6d0 0x7f358c111c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.962+0000 7f358b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f358c10b6d0 0x7f358c111c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.3:59922/0 (socket says 172.21.15.3:59922) 2024-09-21T06:25:07.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.962+0000 7f358b7fe640 1 -- 172.21.15.3:0/3124716768 learned_addr learned my addr 172.21.15.3:0/3124716768 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.963+0000 7f358b7fe640 1 -- 172.21.15.3:0/3124716768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f358c10c020 msgr2=0x7f358c1a5250 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:25:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.963+0000 7f358b7fe640 1 --2- 172.21.15.3:0/3124716768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f358c10c020 0x7f358c1a5250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.963+0000 7f358b7fe640 1 -- 172.21.15.3:0/3124716768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f358c105d90 msgr2=0x7f358c111710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:25:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.963+0000 7f358b7fe640 1 --2- 172.21.15.3:0/3124716768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f358c105d90 0x7f358c111710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.963+0000 7f358b7fe640 1 -- 172.21.15.3:0/3124716768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c1a5980 con 0x7f358c10b6d0 2024-09-21T06:25:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.963+0000 7f358b7fe640 1 --2- 172.21.15.3:0/3124716768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f358c10b6d0 0x7f358c111c90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f357c006010 tx=0x7f357c004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.964+0000 7f35897fa640 1 -- 172.21.15.3:0/3124716768 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c03f030 con 0x7f358c10b6d0 2024-09-21T06:25:07.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.964+0000 7f35897fa640 1 -- 172.21.15.3:0/3124716768 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f357c003aa0 con 0x7f358c10b6d0 2024-09-21T06:25:07.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.964+0000 7f359230d640 1 -- 172.21.15.3:0/3124716768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358c1a5c10 con 0x7f358c10b6d0 2024-09-21T06:25:07.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.965+0000 7f35897fa640 1 -- 172.21.15.3:0/3124716768 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c035070 con 0x7f358c10b6d0 2024-09-21T06:25:07.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.965+0000 7f359230d640 1 -- 172.21.15.3:0/3124716768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358c1ad500 con 0x7f358c10b6d0 2024-09-21T06:25:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.971+0000 7f35897fa640 1 -- 172.21.15.3:0/3124716768 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f357c042030 con 0x7f358c10b6d0 2024-09-21T06:25:07.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.972+0000 7f35897fa640 1 --2- 172.21.15.3:0/3124716768 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f35740b6560 0x7f35740b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.973+0000 7f35897fa640 1 -- 172.21.15.3:0/3124716768 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8551+0+0 (secure 0 0 0) 0x7f357c1078b0 con 0x7f358c10b6d0 2024-09-21T06:25:07.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.973+0000 7f3598881740 1 Processor -- start 2024-09-21T06:25:07.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.973+0000 7f3598881740 1 -- start start 2024-09-21T06:25:07.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.973+0000 7f3598881740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d48c070 0x558e9d8336a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.973+0000 7f3598881740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d4631b0 0x558e9d833be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.973+0000 7f3598881740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d46b390 0x558e9d834120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.973+0000 7f3598881740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558e9d7a9ea0 con 0x558e9d48c070 2024-09-21T06:25:07.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.973+0000 7f3598881740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558e9d7992d0 con 0x558e9d4631b0 2024-09-21T06:25:07.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.973+0000 7f3598881740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558e9d79d0e0 con 0x558e9d46b390 2024-09-21T06:25:07.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.973+0000 7f358bfff640 1 --2- 172.21.15.3:0/3124716768 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f35740b6560 0x7f35740b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.974+0000 7f358bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d48c070 0x558e9d8336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.974+0000 7f358bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d48c070 0x558e9d8336a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44414/0 (socket says 172.21.15.3:44414) 2024-09-21T06:25:07.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.974+0000 7f358bfff640 1 -- 172.21.15.3:0/4284295924 learned_addr learned my addr 172.21.15.3:0/4284295924 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:07.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.974+0000 7f358bfff640 1 --2- 172.21.15.3:0/3124716768 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f35740b6560 0x7f35740b8a70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3580009c70 tx=0x7f35800125e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:07.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.974+0000 7f35908aa640 1 --2- 172.21.15.3:0/4284295924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d46b390 0x558e9d834120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.975+0000 7f358bfff640 1 -- 172.21.15.3:0/4284295924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d46b390 msgr2=0x558e9d834120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:07.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.975+0000 7f358bfff640 1 --2- 172.21.15.3:0/4284295924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d46b390 0x558e9d834120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.975+0000 7f358bfff640 1 -- 172.21.15.3:0/4284295924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d4631b0 msgr2=0x558e9d833be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:07.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.975+0000 7f358bfff640 1 --2- 172.21.15.3:0/4284295924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d4631b0 0x558e9d833be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.975+0000 7f358bfff640 1 -- 172.21.15.3:0/4284295924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9d4f7650 con 0x558e9d48c070 2024-09-21T06:25:07.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.975+0000 7f358bfff640 1 --2- 172.21.15.3:0/4284295924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d48c070 0x558e9d8336a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f3580012ae0 tx=0x7f3580017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b0e2aba922a358 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:07.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.975+0000 7f35727fc640 1 -- 172.21.15.3:0/4284295924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580025070 con 0x558e9d48c070 2024-09-21T06:25:07.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.975+0000 7f35727fc640 1 -- 172.21.15.3:0/4284295924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f358001c040 con 0x558e9d48c070 2024-09-21T06:25:07.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.976+0000 7f35727fc640 1 -- 172.21.15.3:0/4284295924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358000f050 con 0x558e9d48c070 2024-09-21T06:25:07.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.977+0000 7f3598881740 1 -- 172.21.15.3:0/4284295924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d48c070 msgr2=0x558e9d8336a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:07.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.977+0000 7f3598881740 1 --2- 172.21.15.3:0/4284295924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d48c070 0x558e9d8336a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f3580012ae0 tx=0x7f3580017000 comp rx=0 tx=0).stop 2024-09-21T06:25:07.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.978+0000 7f3598881740 1 -- 172.21.15.3:0/4284295924 shutdown_connections 2024-09-21T06:25:07.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.978+0000 7f3598881740 1 --2- 172.21.15.3:0/4284295924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d46b390 0x558e9d834120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.978+0000 7f3598881740 1 --2- 172.21.15.3:0/4284295924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d4631b0 0x558e9d833be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.978+0000 7f3598881740 1 --2- 172.21.15.3:0/4284295924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d48c070 0x558e9d8336a0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.978+0000 7f3598881740 1 -- 172.21.15.3:0/4284295924 >> 172.21.15.3:0/4284295924 conn(0x558e9d480460 msgr2=0x558e9d831590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:07.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.978+0000 7f3598881740 1 -- 172.21.15.3:0/4284295924 shutdown_connections 2024-09-21T06:25:07.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.978+0000 7f3598881740 1 -- 172.21.15.3:0/4284295924 wait complete. 2024-09-21T06:25:07.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.979+0000 7f3598881740 1 -- start start 2024-09-21T06:25:07.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.979+0000 7f3598881740 1 Processor -- start 2024-09-21T06:25:07.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.980+0000 7f3598881740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d4631b0 0x558e9d83ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.980+0000 7f3598881740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d46b390 0x558e9d83f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.980+0000 7f3598881740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d48c070 0x558e9d84b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.980+0000 7f3598881740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558e9d5fa700 con 0x558e9d48c070 2024-09-21T06:25:07.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.980+0000 7f3598881740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558e9d734160 con 0x558e9d46b390 2024-09-21T06:25:07.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.980+0000 7f3598881740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558e9d841ec0 con 0x558e9d4631b0 2024-09-21T06:25:07.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.981+0000 7f358bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d4631b0 0x558e9d83ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.981+0000 7f358bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d4631b0 0x558e9d83ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:51326/0 (socket says 172.21.15.3:51326) 2024-09-21T06:25:07.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.981+0000 7f358bfff640 1 -- 172.21.15.3:0/4172995451 learned_addr learned my addr 172.21.15.3:0/4172995451 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:07.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.981+0000 7f3598881740 10 client.0 _ll_register_callbacks cb 0x558e9d74d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:07.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.981+0000 7f358b7fe640 1 --2- 172.21.15.3:0/4172995451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d46b390 0x558e9d83f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.981+0000 7f35908aa640 1 --2- 172.21.15.3:0/4172995451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d48c070 0x558e9d84b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.982+0000 7f358b7fe640 1 -- 172.21.15.3:0/4172995451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d4631b0 msgr2=0x558e9d83ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:07.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.982+0000 7f358b7fe640 1 --2- 172.21.15.3:0/4172995451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d4631b0 0x558e9d83ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.982+0000 7f358b7fe640 1 -- 172.21.15.3:0/4172995451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d48c070 msgr2=0x558e9d84b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:07.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.982+0000 7f358b7fe640 1 --2- 172.21.15.3:0/4172995451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d48c070 0x558e9d84b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.982+0000 7f358b7fe640 1 -- 172.21.15.3:0/4172995451 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9d761c90 con 0x558e9d46b390 2024-09-21T06:25:07.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.982+0000 7f358b7fe640 1 --2- 172.21.15.3:0/4172995451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d46b390 0x558e9d83f280 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f357c104fe0 tx=0x7f357c0cd4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2248b3f91d6c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:07.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.983+0000 7f356b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:25:07.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.983+0000 7f356b7fe640 1 -- 172.21.15.3:0/4172995451 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c049040 con 0x558e9d46b390 2024-09-21T06:25:07.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.983+0000 7f356b7fe640 1 -- 172.21.15.3:0/4172995451 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8551+0+0 (secure 0 0 0) 0x7f357c0037f0 con 0x558e9d46b390 2024-09-21T06:25:07.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.983+0000 7f3598881740 1 -- 172.21.15.3:0/4172995451 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f358001c040 con 0x558e9d46b390 2024-09-21T06:25:07.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.984+0000 7f356b7fe640 1 -- 172.21.15.3:0/4172995451 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f357c0cd1f0 con 0x558e9d46b390 2024-09-21T06:25:07.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.985+0000 7f3598881740 10 client.27809 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:07.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.985+0000 7f3598881740 1 -- 172.21.15.3:0/4172995451 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558e9d84dba0 con 0x558e9d46b390 2024-09-21T06:25:07.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.986+0000 7f356b7fe640 1 -- 172.21.15.3:0/4172995451 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f357c0f4020 con 0x558e9d46b390 2024-09-21T06:25:07.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.986+0000 7f356b7fe640 1 -- 172.21.15.3:0/4172995451 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f357c0c9110 con 0x558e9d46b390 2024-09-21T06:25:07.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.986+0000 7f3598881740 10 client.27809 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:07.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.986+0000 7f3598881740 10 client.27809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:07.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.986+0000 7f3598881740 20 client.27809 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:07.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.986+0000 7f3598881740 1 --2- 172.21.15.3:0/4172995451 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558e9d44a9f0 0x558e9d851080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.986+0000 7f3598881740 4 client.27809 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:07.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.986+0000 7f3598881740 1 -- 172.21.15.3:0/4172995451 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e9d7abdf0 con 0x558e9d44a9f0 2024-09-21T06:25:07.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.988+0000 7f358bfff640 1 --2- 172.21.15.3:0/4172995451 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558e9d44a9f0 0x558e9d851080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.988+0000 7f358bfff640 1 --2- 172.21.15.3:0/4172995451 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558e9d44a9f0 0x558e9d851080 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cca95773d0d234b server_cookie=20231e5542ace7d5 in_seq=0 out_seq=0 2024-09-21T06:25:07.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.989+0000 7f356b7fe640 10 client.27809 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:07.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.994+0000 7f356b7fe640 1 -- 172.21.15.3:0/4172995451 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558e9d7abdf0 con 0x558e9d44a9f0 2024-09-21T06:25:07.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.994+0000 7f356b7fe640 10 client.27809 handle_command_reply: tid=0 2024-09-21T06:25:07.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.994+0000 7f3598881740 1 client.27809 shutdown 2024-09-21T06:25:07.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.996+0000 7f3598881740 1 -- 172.21.15.3:0/4172995451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d46b390 msgr2=0x558e9d83f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:07.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.996+0000 7f3598881740 1 --2- 172.21.15.3:0/4172995451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d46b390 0x558e9d83f280 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f357c104fe0 tx=0x7f357c0cd4b0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.997+0000 7f3598881740 1 -- 172.21.15.3:0/4172995451 shutdown_connections 2024-09-21T06:25:07.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.997+0000 7f3598881740 1 --2- 172.21.15.3:0/4172995451 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558e9d44a9f0 0x558e9d851080 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.997+0000 7f3598881740 1 --2- 172.21.15.3:0/4172995451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d48c070 0x558e9d84b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.997+0000 7f3598881740 1 --2- 172.21.15.3:0/4172995451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d46b390 0x558e9d83f280 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.997+0000 7f3598881740 1 --2- 172.21.15.3:0/4172995451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d4631b0 0x558e9d83ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.997+0000 7f356b7fe640 0 client.27809 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:08.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.997+0000 7f3598881740 1 -- 172.21.15.3:0/4172995451 >> 172.21.15.3:0/4172995451 conn(0x558e9d480460 msgr2=0x558e9d8346d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:08.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.998+0000 7f3598881740 1 -- 172.21.15.3:0/4172995451 shutdown_connections 2024-09-21T06:25:08.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.999+0000 7f3598881740 1 -- 172.21.15.3:0/4172995451 wait complete. 2024-09-21T06:25:08.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:07.999+0000 7f3598881740 20 client.27809 trim_cache size 0 max 16384 2024-09-21T06:25:08.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.026+0000 7f3598881740 1 Processor -- start 2024-09-21T06:25:08.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.026+0000 7f3598881740 1 -- start start 2024-09-21T06:25:08.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.026+0000 7f3598881740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d44a9f0 0x558e9d840970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:08.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.026+0000 7f3598881740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d4631b0 0x558e9d83bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:08.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.026+0000 7f3598881740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d46b390 0x558e9d83c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:08.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.026+0000 7f3598881740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f358c11d1b0 con 0x558e9d44a9f0 2024-09-21T06:25:08.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.026+0000 7f3598881740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f358c1aef20 con 0x558e9d4631b0 2024-09-21T06:25:08.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.026+0000 7f3598881740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f358c1adab0 con 0x558e9d46b390 2024-09-21T06:25:08.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.027+0000 7f358b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d4631b0 0x558e9d83bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:08.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.027+0000 7f35908aa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d46b390 0x558e9d83c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:08.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.027+0000 7f35908aa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d46b390 0x558e9d83c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.3:51328/0 (socket says 172.21.15.3:51328) 2024-09-21T06:25:08.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.027+0000 7f35908aa640 1 -- 172.21.15.3:0/2157033568 learned_addr learned my addr 172.21.15.3:0/2157033568 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:08.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.027+0000 7f358b7fe640 1 -- 172.21.15.3:0/2157033568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d46b390 msgr2=0x558e9d83c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:08.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.027+0000 7f358b7fe640 1 --2- 172.21.15.3:0/2157033568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d46b390 0x558e9d83c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.027+0000 7f358b7fe640 1 -- 172.21.15.3:0/2157033568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d44a9f0 msgr2=0x558e9d840970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:08.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.027+0000 7f358b7fe640 1 --2- 172.21.15.3:0/2157033568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d44a9f0 0x558e9d840970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.027+0000 7f358b7fe640 1 -- 172.21.15.3:0/2157033568 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9d841480 con 0x558e9d4631b0 2024-09-21T06:25:08.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.029+0000 7f358b7fe640 1 --2- 172.21.15.3:0/2157033568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d4631b0 0x558e9d83bf30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f357c0cd670 tx=0x7f357c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11258cd8abedd352 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:08.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.029+0000 7f3571ffb640 1 -- 172.21.15.3:0/2157033568 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c049070 con 0x558e9d4631b0 2024-09-21T06:25:08.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.030+0000 7f3571ffb640 1 -- 172.21.15.3:0/2157033568 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f357c0f4070 con 0x558e9d4631b0 2024-09-21T06:25:08.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.031+0000 7f3598881740 1 -- 172.21.15.3:0/2157033568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d4631b0 msgr2=0x558e9d83bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:08.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.031+0000 7f3598881740 1 --2- 172.21.15.3:0/2157033568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d4631b0 0x558e9d83bf30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f357c0cd670 tx=0x7f357c000bc0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.032+0000 7f3598881740 1 -- 172.21.15.3:0/2157033568 shutdown_connections 2024-09-21T06:25:08.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.032+0000 7f3598881740 1 --2- 172.21.15.3:0/2157033568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d46b390 0x558e9d83c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.032+0000 7f3598881740 1 --2- 172.21.15.3:0/2157033568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d4631b0 0x558e9d83bf30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.032+0000 7f3598881740 1 --2- 172.21.15.3:0/2157033568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d44a9f0 0x558e9d840970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.032+0000 7f3598881740 1 -- 172.21.15.3:0/2157033568 >> 172.21.15.3:0/2157033568 conn(0x558e9d480460 msgr2=0x558e9d840260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:08.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.032+0000 7f3598881740 1 -- 172.21.15.3:0/2157033568 shutdown_connections 2024-09-21T06:25:08.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.032+0000 7f3598881740 1 -- 172.21.15.3:0/2157033568 wait complete. 2024-09-21T06:25:08.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.033+0000 7f3598881740 1 -- start start 2024-09-21T06:25:08.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.033+0000 7f3598881740 1 Processor -- start 2024-09-21T06:25:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.034+0000 7f3598881740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d44a9f0 0x558e9d847930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.034+0000 7f3598881740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d4631b0 0x558e9d844ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.034+0000 7f3598881740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d46b390 0x558e9d8453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.034+0000 7f3598881740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f358c1ae850 con 0x558e9d44a9f0 2024-09-21T06:25:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.034+0000 7f3598881740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f358c1ae180 con 0x558e9d46b390 2024-09-21T06:25:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.034+0000 7f3598881740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f358c11b1b0 con 0x558e9d4631b0 2024-09-21T06:25:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.034+0000 7f3598881740 10 client.0 _ll_register_callbacks cb 0x7f357c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.034+0000 7f358bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d44a9f0 0x558e9d847930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.034+0000 7f358bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d44a9f0 0x558e9d847930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44450/0 (socket says 172.21.15.3:44450) 2024-09-21T06:25:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.034+0000 7f358bfff640 1 -- 172.21.15.3:0/1814419378 learned_addr learned my addr 172.21.15.3:0/1814419378 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.034+0000 7f358b7fe640 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d4631b0 0x558e9d844ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.034+0000 7f35908aa640 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d46b390 0x558e9d8453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.035+0000 7f358bfff640 1 -- 172.21.15.3:0/1814419378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d4631b0 msgr2=0x558e9d844ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.035+0000 7f358bfff640 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d4631b0 0x558e9d844ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.035+0000 7f358bfff640 1 -- 172.21.15.3:0/1814419378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d46b390 msgr2=0x558e9d8453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.035+0000 7f358bfff640 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d46b390 0x558e9d8453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.035+0000 7f358bfff640 1 -- 172.21.15.3:0/1814419378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9d845b70 con 0x558e9d44a9f0 2024-09-21T06:25:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.035+0000 7f358b7fe640 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d4631b0 0x558e9d844ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.035+0000 7f358bfff640 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d44a9f0 0x558e9d847930 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3580012fd0 tx=0x7f3580009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60111c68067d18d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.035+0000 7f356b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.036+0000 7f356b7fe640 1 -- 172.21.15.3:0/1814419378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580031070 con 0x558e9d44a9f0 2024-09-21T06:25:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.036+0000 7f356b7fe640 1 -- 172.21.15.3:0/1814419378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8551+0+0 (secure 0 0 0) 0x7f3580018db0 con 0x558e9d44a9f0 2024-09-21T06:25:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.037+0000 7f3598881740 1 -- 172.21.15.3:0/1814419378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f357c0f4070 con 0x558e9d44a9f0 2024-09-21T06:25:08.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.039+0000 7f356b7fe640 1 -- 172.21.15.3:0/1814419378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f358000f070 con 0x558e9d44a9f0 2024-09-21T06:25:08.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.040+0000 7f3598881740 10 client.18312 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:08.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.040+0000 7f3598881740 1 -- 172.21.15.3:0/1814419378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558e9d8cca50 con 0x558e9d44a9f0 2024-09-21T06:25:08.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.041+0000 7f356b7fe640 1 -- 172.21.15.3:0/1814419378 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f357c0f4070 con 0x558e9d44a9f0 2024-09-21T06:25:08.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.041+0000 7f356b7fe640 1 -- 172.21.15.3:0/1814419378 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f358003c030 con 0x558e9d44a9f0 2024-09-21T06:25:08.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.041+0000 7f3598881740 10 client.18312 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:08.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.041+0000 7f3598881740 10 client.18312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:08.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.041+0000 7f3598881740 20 client.18312 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:08.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.041+0000 7f3598881740 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558e9d48c070 0x558e9d8cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:08.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.041+0000 7f3598881740 4 client.18312 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:25:08.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.041+0000 7f3598881740 1 -- 172.21.15.3:0/1814419378 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e9d7a78a0 con 0x558e9d48c070 2024-09-21T06:25:08.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.042+0000 7f358b7fe640 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558e9d48c070 0x558e9d8cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:08.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.044+0000 7f358b7fe640 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558e9d48c070 0x558e9d8cf290 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af6c40f51a8ae6f3 server_cookie=e41a0547403c97a2 in_seq=0 out_seq=0 2024-09-21T06:25:08.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.044+0000 7f356b7fe640 10 client.18312 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:08.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.045+0000 7f356b7fe640 1 -- 172.21.15.3:0/1814419378 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e9d7a78a0 con 0x558e9d48c070 2024-09-21T06:25:08.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.045+0000 7f356b7fe640 10 client.18312 handle_command_reply: tid=0 2024-09-21T06:25:08.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.045+0000 7f3598881740 1 client.18312 shutdown 2024-09-21T06:25:08.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.046+0000 7f3598881740 1 -- 172.21.15.3:0/1814419378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d44a9f0 msgr2=0x558e9d847930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:08.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.046+0000 7f3598881740 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d44a9f0 0x558e9d847930 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3580012fd0 tx=0x7f3580009920 comp rx=0 tx=0).stop 2024-09-21T06:25:08.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.046+0000 7f3598881740 1 -- 172.21.15.3:0/1814419378 shutdown_connections 2024-09-21T06:25:08.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.046+0000 7f3598881740 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x558e9d48c070 0x558e9d8cf290 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.046+0000 7f3598881740 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e9d4631b0 0x558e9d844ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.046+0000 7f3598881740 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e9d46b390 0x558e9d8453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.046+0000 7f3598881740 1 --2- 172.21.15.3:0/1814419378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e9d44a9f0 0x558e9d847930 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.046+0000 7f356b7fe640 0 client.18312 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:08.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.046+0000 7f3598881740 1 -- 172.21.15.3:0/1814419378 >> 172.21.15.3:0/1814419378 conn(0x558e9d480460 msgr2=0x558e9d8387b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:08.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.047+0000 7f3598881740 1 -- 172.21.15.3:0/1814419378 shutdown_connections 2024-09-21T06:25:08.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.047+0000 7f3598881740 1 -- 172.21.15.3:0/1814419378 wait complete. 2024-09-21T06:25:08.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.048+0000 7f3598881740 20 client.18312 trim_cache size 0 max 16384 2024-09-21T06:25:08.051 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:25:08.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.049+0000 7f359230d640 1 -- 172.21.15.3:0/3124716768 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f35740b6560 msgr2=0x7f35740b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:08.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.049+0000 7f359230d640 1 --2- 172.21.15.3:0/3124716768 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f35740b6560 0x7f35740b8a70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3580009c70 tx=0x7f35800125e0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.049+0000 7f359230d640 1 -- 172.21.15.3:0/3124716768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f358c10b6d0 msgr2=0x7f358c111c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:08.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.049+0000 7f359230d640 1 --2- 172.21.15.3:0/3124716768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f358c10b6d0 0x7f358c111c90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f357c006010 tx=0x7f357c004060 comp rx=0 tx=0).stop 2024-09-21T06:25:08.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.049+0000 7f359230d640 1 -- 172.21.15.3:0/3124716768 shutdown_connections 2024-09-21T06:25:08.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.049+0000 7f359230d640 1 --2- 172.21.15.3:0/3124716768 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f35740b6560 0x7f35740b8a70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.049+0000 7f359230d640 1 --2- 172.21.15.3:0/3124716768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f358c10c020 0x7f358c1a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.049+0000 7f359230d640 1 --2- 172.21.15.3:0/3124716768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f358c10b6d0 0x7f358c111c90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.049+0000 7f359230d640 1 --2- 172.21.15.3:0/3124716768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f358c105d90 0x7f358c111710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:08.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.049+0000 7f359230d640 1 -- 172.21.15.3:0/3124716768 >> 172.21.15.3:0/3124716768 conn(0x7f358c1017c0 msgr2=0x7f358c07b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:08.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.050+0000 7f359230d640 1 -- 172.21.15.3:0/3124716768 shutdown_connections 2024-09-21T06:25:08.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:08.050+0000 7f359230d640 1 -- 172.21.15.3:0/3124716768 wait complete. 2024-09-21T06:25:08.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:08 smithi003 ceph-mon[31227]: pgmap v638: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 59 KiB/s wr, 31 op/s 2024-09-21T06:25:08.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:08 smithi137 ceph-mon[34464]: pgmap v638: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 59 KiB/s wr, 31 op/s 2024-09-21T06:25:08.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:08 smithi204 ceph-mon[34659]: pgmap v638: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 59 KiB/s wr, 31 op/s 2024-09-21T06:25:10.185 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:25:10.185 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:25:10.458 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:10.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.955+0000 7fd49a7dc640 1 -- 172.21.15.3:0/1629691406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd494108cf0 msgr2=0x7fd49410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.955+0000 7fd49a7dc640 1 --2- 172.21.15.3:0/1629691406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd494108cf0 0x7fd49410f520 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009ca0 tx=0x7fd48c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.955+0000 7fd49a7dc640 1 -- 172.21.15.3:0/1629691406 shutdown_connections 2024-09-21T06:25:10.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.955+0000 7fd49a7dc640 1 --2- 172.21.15.3:0/1629691406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4941083a0 0x7fd4941087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.955+0000 7fd49a7dc640 1 --2- 172.21.15.3:0/1629691406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd494108cf0 0x7fd49410f520 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.955+0000 7fd49a7dc640 1 --2- 172.21.15.3:0/1629691406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd494102a60 0x7fd494102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.955+0000 7fd49a7dc640 1 -- 172.21.15.3:0/1629691406 >> 172.21.15.3:0/1629691406 conn(0x7fd494079730 msgr2=0x7fd494079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:10.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.955+0000 7fd49a7dc640 1 -- 172.21.15.3:0/1629691406 shutdown_connections 2024-09-21T06:25:10.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.955+0000 7fd49a7dc640 1 -- 172.21.15.3:0/1629691406 wait complete. 2024-09-21T06:25:10.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.956+0000 7fd49a7dc640 1 Processor -- start 2024-09-21T06:25:10.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.956+0000 7fd49a7dc640 1 -- start start 2024-09-21T06:25:10.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.956+0000 7fd49a7dc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd494102a60 0x7fd49419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.956+0000 7fd49a7dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4941083a0 0x7fd49419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.956+0000 7fd49a7dc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd494108cf0 0x7fd49419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.956+0000 7fd49a7dc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd494112960 con 0x7fd494108cf0 2024-09-21T06:25:10.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.956+0000 7fd49a7dc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941127e0 con 0x7fd4941083a0 2024-09-21T06:25:10.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.956+0000 7fd49a7dc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd494112ae0 con 0x7fd494102a60 2024-09-21T06:25:10.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.956+0000 7fd498d79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd494108cf0 0x7fd49419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.956+0000 7fd498d79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd494108cf0 0x7fd49419bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44468/0 (socket says 172.21.15.3:44468) 2024-09-21T06:25:10.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.956+0000 7fd498d79640 1 -- 172.21.15.3:0/2953809590 learned_addr learned my addr 172.21.15.3:0/2953809590 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:10.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd48bfff640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4941083a0 0x7fd49419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd493fff640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd494102a60 0x7fd49419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd498d79640 1 -- 172.21.15.3:0/2953809590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd494102a60 msgr2=0x7fd49419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd498d79640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd494102a60 0x7fd49419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd498d79640 1 -- 172.21.15.3:0/2953809590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4941083a0 msgr2=0x7fd49419b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd498d79640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4941083a0 0x7fd49419b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd498d79640 1 -- 172.21.15.3:0/2953809590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4941a6560 con 0x7fd494108cf0 2024-09-21T06:25:10.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd48bfff640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4941083a0 0x7fd49419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:10.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd498d79640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd494108cf0 0x7fd49419bd50 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd48c005cc0 tx=0x7fd48c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:10.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd491ffb640 1 -- 172.21.15.3:0/2953809590 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c003bf0 con 0x7fd494108cf0 2024-09-21T06:25:10.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd491ffb640 1 -- 172.21.15.3:0/2953809590 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd48c003d90 con 0x7fd494108cf0 2024-09-21T06:25:10.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd49a7dc640 1 -- 172.21.15.3:0/2953809590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4941a67f0 con 0x7fd494108cf0 2024-09-21T06:25:10.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd491ffb640 1 -- 172.21.15.3:0/2953809590 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c02fbc0 con 0x7fd494108cf0 2024-09-21T06:25:10.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.957+0000 7fd49a7dc640 1 -- 172.21.15.3:0/2953809590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4941a6d30 con 0x7fd494108cf0 2024-09-21T06:25:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.960+0000 7fd4a0d50740 1 Processor -- start 2024-09-21T06:25:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.961+0000 7fd491ffb640 1 -- 172.21.15.3:0/2953809590 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fd48c02fd60 con 0x7fd494108cf0 2024-09-21T06:25:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.961+0000 7fd4a0d50740 1 -- start start 2024-09-21T06:25:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.961+0000 7fd4a0d50740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b18c350 0x55890b533430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.961+0000 7fd4a0d50740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 0x55890b533970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.961+0000 7fd4a0d50740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 0x55890b533eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.961+0000 7fd4a0d50740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55890b4a9e60 con 0x55890b16b670 2024-09-21T06:25:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.961+0000 7fd4a0d50740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55890b499290 con 0x55890b163490 2024-09-21T06:25:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.961+0000 7fd4a0d50740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55890b49d0a0 con 0x55890b18c350 2024-09-21T06:25:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.961+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 0x55890b533eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.961+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 0x55890b533eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44482/0 (socket says 172.21.15.3:44482) 2024-09-21T06:25:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.961+0000 7fd48bfff640 1 -- 172.21.15.3:0/506806937 learned_addr learned my addr 172.21.15.3:0/506806937 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.961+0000 7fd493fff640 1 --2- 172.21.15.3:0/506806937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 0x55890b533970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.961+0000 7fd48bfff640 1 --2- 172.21.15.3:0/506806937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b18c350 0x55890b533430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.962+0000 7fd48bfff640 1 -- 172.21.15.3:0/506806937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b18c350 msgr2=0x55890b533430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.962+0000 7fd48bfff640 1 --2- 172.21.15.3:0/506806937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b18c350 0x55890b533430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.962+0000 7fd48bfff640 1 -- 172.21.15.3:0/506806937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 msgr2=0x55890b533970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.962+0000 7fd48bfff640 1 --2- 172.21.15.3:0/506806937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 0x55890b533970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.962+0000 7fd48bfff640 1 -- 172.21.15.3:0/506806937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890b1f7930 con 0x55890b16b670 2024-09-21T06:25:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.962+0000 7fd491ffb640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd4800b66b0 0x7fd4800b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.962+0000 7fd493fff640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd4800b66b0 0x7fd4800b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.962+0000 7fd48bfff640 1 --2- 172.21.15.3:0/506806937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 0x55890b533eb0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0103d0 tx=0x7fd47c00e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342e27d2b3b934d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.962+0000 7fd491ffb640 1 -- 172.21.15.3:0/2953809590 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8551+0+0 (secure 0 0 0) 0x7fd48c108b80 con 0x7fd494108cf0 2024-09-21T06:25:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.962+0000 7fd48a7fc640 1 -- 172.21.15.3:0/506806937 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c018070 con 0x55890b16b670 2024-09-21T06:25:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.962+0000 7fd48a7fc640 1 -- 172.21.15.3:0/506806937 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47c012b00 con 0x55890b16b670 2024-09-21T06:25:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.962+0000 7fd48a7fc640 1 -- 172.21.15.3:0/506806937 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c011b60 con 0x55890b16b670 2024-09-21T06:25:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.963+0000 7fd4a0d50740 1 -- 172.21.15.3:0/506806937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 msgr2=0x55890b533eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.963+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/506806937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 0x55890b533eb0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0103d0 tx=0x7fd47c00e5b0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.963+0000 7fd493fff640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd4800b66b0 0x7fd4800b8bc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd49406b9c0 tx=0x7fd48400b500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:10.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.963+0000 7fd4a0d50740 1 -- 172.21.15.3:0/506806937 shutdown_connections 2024-09-21T06:25:10.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.963+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/506806937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 0x55890b533eb0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.963+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/506806937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 0x55890b533970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.963+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/506806937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b18c350 0x55890b533430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.963+0000 7fd4a0d50740 1 -- 172.21.15.3:0/506806937 >> 172.21.15.3:0/506806937 conn(0x55890b180740 msgr2=0x55890b531320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:10.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.963+0000 7fd4a0d50740 1 -- 172.21.15.3:0/506806937 shutdown_connections 2024-09-21T06:25:10.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.963+0000 7fd4a0d50740 1 -- 172.21.15.3:0/506806937 wait complete. 2024-09-21T06:25:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.964+0000 7fd4a0d50740 1 -- start start 2024-09-21T06:25:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.964+0000 7fd4a0d50740 1 Processor -- start 2024-09-21T06:25:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.964+0000 7fd4a0d50740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 0x55890b5433d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.964+0000 7fd4a0d50740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b16b670 0x55890b543910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.964+0000 7fd4a0d50740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 0x55890b54f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.964+0000 7fd4a0d50740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55890b2fa890 con 0x55890b18c350 2024-09-21T06:25:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.964+0000 7fd4a0d50740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55890b433d40 con 0x55890b163490 2024-09-21T06:25:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.964+0000 7fd4a0d50740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55890b545f40 con 0x55890b16b670 2024-09-21T06:25:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.964+0000 7fd4a0d50740 10 client.0 _ll_register_callbacks cb 0x55890b497c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:10.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.964+0000 7fd498d79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 0x55890b54f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.964+0000 7fd493fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 0x55890b5433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.964+0000 7fd498d79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 0x55890b54f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44490/0 (socket says 172.21.15.3:44490) 2024-09-21T06:25:10.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.964+0000 7fd498d79640 1 -- 172.21.15.3:0/2457872992 learned_addr learned my addr 172.21.15.3:0/2457872992 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:10.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd48bfff640 1 --2- 172.21.15.3:0/2457872992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b16b670 0x55890b543910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd498d79640 1 -- 172.21.15.3:0/2457872992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b16b670 msgr2=0x55890b543910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd498d79640 1 --2- 172.21.15.3:0/2457872992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b16b670 0x55890b543910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd498d79640 1 -- 172.21.15.3:0/2457872992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 msgr2=0x55890b5433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd498d79640 1 --2- 172.21.15.3:0/2457872992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 0x55890b5433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd498d79640 1 -- 172.21.15.3:0/2457872992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890b54f800 con 0x55890b18c350 2024-09-21T06:25:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd48bfff640 1 --2- 172.21.15.3:0/2457872992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b16b670 0x55890b543910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd498d79640 1 --2- 172.21.15.3:0/2457872992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 0x55890b54f270 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0ce940 tx=0x7fd48c0ce430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa3d79b2ab4669f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd4737fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd4737fe640 1 -- 172.21.15.3:0/2457872992 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c0cf0b0 con 0x55890b18c350 2024-09-21T06:25:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd4737fe640 1 -- 172.21.15.3:0/2457872992 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8551+0+0 (secure 0 0 0) 0x7fd48c0d7f40 con 0x55890b18c350 2024-09-21T06:25:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2457872992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd47c012b00 con 0x55890b18c350 2024-09-21T06:25:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd4737fe640 1 -- 172.21.15.3:0/2457872992 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd48c0e9050 con 0x55890b18c350 2024-09-21T06:25:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd4a0d50740 10 client.18324 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:10.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.965+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2457872992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55890b552130 con 0x55890b18c350 2024-09-21T06:25:10.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.966+0000 7fd4737fe640 1 -- 172.21.15.3:0/2457872992 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47c012b00 con 0x55890b18c350 2024-09-21T06:25:10.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.966+0000 7fd4737fe640 1 -- 172.21.15.3:0/2457872992 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fd48c0cfce0 con 0x55890b18c350 2024-09-21T06:25:10.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.966+0000 7fd4a0d50740 10 client.18324 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:10.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.966+0000 7fd4a0d50740 10 client.18324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:10.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.966+0000 7fd4a0d50740 20 client.18324 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:10.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.966+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/2457872992 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55890b14acd0 0x55890b555640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.966+0000 7fd4a0d50740 4 client.18324 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:10.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.966+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2457872992 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55890b4abdb0 con 0x55890b14acd0 2024-09-21T06:25:10.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.966+0000 7fd493fff640 1 --2- 172.21.15.3:0/2457872992 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55890b14acd0 0x55890b555640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.967+0000 7fd4737fe640 10 client.18324 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:10.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.967+0000 7fd493fff640 1 --2- 172.21.15.3:0/2457872992 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55890b14acd0 0x55890b555640 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdfa8808be0b0fba server_cookie=be2b4c1fa07b3182 in_seq=0 out_seq=0 2024-09-21T06:25:10.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4737fe640 1 -- 172.21.15.3:0/2457872992 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55890b4abdb0 con 0x55890b14acd0 2024-09-21T06:25:10.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4737fe640 10 client.18324 handle_command_reply: tid=0 2024-09-21T06:25:10.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4a0d50740 1 client.18324 shutdown 2024-09-21T06:25:10.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2457872992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 msgr2=0x55890b54f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/2457872992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 0x55890b54f270 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0ce940 tx=0x7fd48c0ce430 comp rx=0 tx=0).stop 2024-09-21T06:25:10.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2457872992 shutdown_connections 2024-09-21T06:25:10.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/2457872992 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55890b14acd0 0x55890b555640 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/2457872992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b16b670 0x55890b543910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/2457872992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 0x55890b54f270 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/2457872992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 0x55890b5433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4737fe640 0 client.18324 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2457872992 >> 172.21.15.3:0/2457872992 conn(0x55890b180740 msgr2=0x55890b536580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2457872992 shutdown_connections 2024-09-21T06:25:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2457872992 wait complete. 2024-09-21T06:25:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.968+0000 7fd4a0d50740 20 client.18324 trim_cache size 0 max 16384 2024-09-21T06:25:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.986+0000 7fd4a0d50740 1 Processor -- start 2024-09-21T06:25:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.986+0000 7fd4a0d50740 1 -- start start 2024-09-21T06:25:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.986+0000 7fd4a0d50740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 0x55890b54f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.986+0000 7fd4a0d50740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b16b670 0x55890b5403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.986+0000 7fd4a0d50740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 0x55890b540900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.986+0000 7fd4a0d50740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941ad4c0 con 0x55890b18c350 2024-09-21T06:25:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.986+0000 7fd4a0d50740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941af000 con 0x55890b163490 2024-09-21T06:25:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.986+0000 7fd4a0d50740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941adb90 con 0x55890b16b670 2024-09-21T06:25:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.986+0000 7fd498d79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 0x55890b540900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.986+0000 7fd498d79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 0x55890b540900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44500/0 (socket says 172.21.15.3:44500) 2024-09-21T06:25:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.986+0000 7fd498d79640 1 -- 172.21.15.3:0/979587167 learned_addr learned my addr 172.21.15.3:0/979587167 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.986+0000 7fd48bfff640 1 --2- 172.21.15.3:0/979587167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b16b670 0x55890b5403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.986+0000 7fd493fff640 1 --2- 172.21.15.3:0/979587167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 0x55890b54f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.987+0000 7fd498d79640 1 -- 172.21.15.3:0/979587167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b16b670 msgr2=0x55890b5403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.987+0000 7fd498d79640 1 --2- 172.21.15.3:0/979587167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b16b670 0x55890b5403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.987+0000 7fd498d79640 1 -- 172.21.15.3:0/979587167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 msgr2=0x55890b54f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.987+0000 7fd498d79640 1 --2- 172.21.15.3:0/979587167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 0x55890b54f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.987+0000 7fd498d79640 1 -- 172.21.15.3:0/979587167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890b5374e0 con 0x55890b18c350 2024-09-21T06:25:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.987+0000 7fd493fff640 1 --2- 172.21.15.3:0/979587167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 0x55890b54f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.987+0000 7fd498d79640 1 --2- 172.21.15.3:0/979587167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 0x55890b540900 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0094e0 tx=0x7fd48c02f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c28a442d008071 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.987+0000 7fd489ffb640 1 -- 172.21.15.3:0/979587167 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c0ce780 con 0x55890b18c350 2024-09-21T06:25:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.987+0000 7fd489ffb640 1 -- 172.21.15.3:0/979587167 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd48c036070 con 0x55890b18c350 2024-09-21T06:25:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.987+0000 7fd489ffb640 1 -- 172.21.15.3:0/979587167 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c0098b0 con 0x55890b18c350 2024-09-21T06:25:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.988+0000 7fd4a0d50740 1 -- 172.21.15.3:0/979587167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 msgr2=0x55890b540900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.988+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/979587167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 0x55890b540900 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0094e0 tx=0x7fd48c02f660 comp rx=0 tx=0).stop 2024-09-21T06:25:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.988+0000 7fd4a0d50740 1 -- 172.21.15.3:0/979587167 shutdown_connections 2024-09-21T06:25:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.988+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/979587167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b16b670 0x55890b5403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.988+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/979587167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b18c350 0x55890b540900 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.988+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/979587167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b163490 0x55890b54f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.988+0000 7fd4a0d50740 1 -- 172.21.15.3:0/979587167 >> 172.21.15.3:0/979587167 conn(0x55890b180740 msgr2=0x55890b541a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.988+0000 7fd4a0d50740 1 -- 172.21.15.3:0/979587167 shutdown_connections 2024-09-21T06:25:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.988+0000 7fd4a0d50740 1 -- 172.21.15.3:0/979587167 wait complete. 2024-09-21T06:25:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.988+0000 7fd4a0d50740 1 -- start start 2024-09-21T06:25:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.988+0000 7fd4a0d50740 1 Processor -- start 2024-09-21T06:25:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd4a0d50740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b163490 0x55890b547c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd4a0d50740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 0x55890b548240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd4a0d50740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b18c350 0x55890b5b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd4a0d50740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941ae930 con 0x55890b16b670 2024-09-21T06:25:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd4a0d50740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941ae260 con 0x55890b18c350 2024-09-21T06:25:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd4a0d50740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941ab4c0 con 0x55890b163490 2024-09-21T06:25:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd4a0d50740 10 client.0 _ll_register_callbacks cb 0x7fd48c009430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 0x55890b548240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 0x55890b548240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44508/0 (socket says 172.21.15.3:44508) 2024-09-21T06:25:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd48bfff640 1 -- 172.21.15.3:0/2922038729 learned_addr learned my addr 172.21.15.3:0/2922038729 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd493fff640 1 --2- 172.21.15.3:0/2922038729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b163490 0x55890b547c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd498d79640 1 --2- 172.21.15.3:0/2922038729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b18c350 0x55890b5b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd48bfff640 1 -- 172.21.15.3:0/2922038729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b163490 msgr2=0x55890b547c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd48bfff640 1 --2- 172.21.15.3:0/2922038729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b163490 0x55890b547c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd48bfff640 1 -- 172.21.15.3:0/2922038729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b18c350 msgr2=0x55890b5b90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd48bfff640 1 --2- 172.21.15.3:0/2922038729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b18c350 0x55890b5b90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd48bfff640 1 -- 172.21.15.3:0/2922038729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890b5b9740 con 0x55890b16b670 2024-09-21T06:25:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd48bfff640 1 --2- 172.21.15.3:0/2922038729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 0x55890b548240 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd47c01a000 tx=0x7fd47c005560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b127e1e29e428de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.989+0000 7fd4737fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4737fe640 1 -- 172.21.15.3:0/2922038729 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c018050 con 0x55890b16b670 2024-09-21T06:25:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4737fe640 1 -- 172.21.15.3:0/2922038729 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8551+0+0 (secure 0 0 0) 0x7fd47c004a30 con 0x55890b16b670 2024-09-21T06:25:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2922038729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd48c036070 con 0x55890b16b670 2024-09-21T06:25:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4737fe640 1 -- 172.21.15.3:0/2922038729 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd47c004c30 con 0x55890b16b670 2024-09-21T06:25:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4a0d50740 10 client.18330 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2922038729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55890b5c0b70 con 0x55890b16b670 2024-09-21T06:25:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4737fe640 1 -- 172.21.15.3:0/2922038729 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd48c036070 con 0x55890b16b670 2024-09-21T06:25:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4737fe640 1 -- 172.21.15.3:0/2922038729 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fd47c00f040 con 0x55890b16b670 2024-09-21T06:25:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4a0d50740 10 client.18330 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4a0d50740 10 client.18330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4a0d50740 20 client.18330 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/2922038729 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55890b120080 0x55890b5c3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4a0d50740 4 client.18330 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:25:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.990+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2922038729 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55890b4a7860 con 0x55890b120080 2024-09-21T06:25:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.991+0000 7fd493fff640 1 --2- 172.21.15.3:0/2922038729 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55890b120080 0x55890b5c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.991+0000 7fd4737fe640 10 client.18330 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.991+0000 7fd493fff640 1 --2- 172.21.15.3:0/2922038729 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55890b120080 0x55890b5c3100 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3643dc9969810bd5 server_cookie=47491651018d3622 in_seq=0 out_seq=0 2024-09-21T06:25:11.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4737fe640 1 -- 172.21.15.3:0/2922038729 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55890b4a7860 con 0x55890b120080 2024-09-21T06:25:11.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4737fe640 10 client.18330 handle_command_reply: tid=0 2024-09-21T06:25:11.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4a0d50740 1 client.18330 shutdown 2024-09-21T06:25:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2922038729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 msgr2=0x55890b548240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/2922038729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 0x55890b548240 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd47c01a000 tx=0x7fd47c005560 comp rx=0 tx=0).stop 2024-09-21T06:25:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2922038729 shutdown_connections 2024-09-21T06:25:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/2922038729 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55890b120080 0x55890b5c3100 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/2922038729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55890b18c350 0x55890b5b90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4737fe640 0 client.18330 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/2922038729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55890b16b670 0x55890b548240 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4a0d50740 1 --2- 172.21.15.3:0/2922038729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890b163490 0x55890b547c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2922038729 >> 172.21.15.3:0/2922038729 conn(0x55890b180740 msgr2=0x55890b541650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:11.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2922038729 shutdown_connections 2024-09-21T06:25:11.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4a0d50740 1 -- 172.21.15.3:0/2922038729 wait complete. 2024-09-21T06:25:11.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.993+0000 7fd4a0d50740 20 client.18330 trim_cache size 0 max 16384 2024-09-21T06:25:11.002 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:25:11.002 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-21T06:25:11.002 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "9135cf69-938c-4d40-9a5c-95e31d48b371", 2024-09-21T06:25:11.003 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-21T06:25:11.003 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:25:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.994+0000 7fd49a7dc640 1 -- 172.21.15.3:0/2953809590 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd4800b66b0 msgr2=0x7fd4800b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.994+0000 7fd49a7dc640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd4800b66b0 0x7fd4800b8bc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd49406b9c0 tx=0x7fd48400b500 comp rx=0 tx=0).stop 2024-09-21T06:25:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.994+0000 7fd49a7dc640 1 -- 172.21.15.3:0/2953809590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd494108cf0 msgr2=0x7fd49419bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.994+0000 7fd49a7dc640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd494108cf0 0x7fd49419bd50 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd48c005cc0 tx=0x7fd48c02f380 comp rx=0 tx=0).stop 2024-09-21T06:25:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.994+0000 7fd49a7dc640 1 -- 172.21.15.3:0/2953809590 shutdown_connections 2024-09-21T06:25:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.994+0000 7fd49a7dc640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fd4800b66b0 0x7fd4800b8bc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.994+0000 7fd49a7dc640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd494108cf0 0x7fd49419bd50 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.994+0000 7fd49a7dc640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4941083a0 0x7fd49419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.994+0000 7fd49a7dc640 1 --2- 172.21.15.3:0/2953809590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd494102a60 0x7fd49419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.994+0000 7fd49a7dc640 1 -- 172.21.15.3:0/2953809590 >> 172.21.15.3:0/2953809590 conn(0x7fd494079730 msgr2=0x7fd494104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.994+0000 7fd49a7dc640 1 -- 172.21.15.3:0/2953809590 shutdown_connections 2024-09-21T06:25:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:10.994+0000 7fd49a7dc640 1 -- 172.21.15.3:0/2953809590 wait complete. 2024-09-21T06:25:11.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:10 smithi137 ceph-mon[34464]: pgmap v639: 97 pgs: 97 active+clean; 142 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 59 KiB/s wr, 31 op/s 2024-09-21T06:25:11.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:10 smithi204 ceph-mon[34659]: pgmap v639: 97 pgs: 97 active+clean; 142 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 59 KiB/s wr, 31 op/s 2024-09-21T06:25:11.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:10 smithi003 ceph-mon[31227]: pgmap v639: 97 pgs: 97 active+clean; 142 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 59 KiB/s wr, 31 op/s 2024-09-21T06:25:11.787 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:25:12.051 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:12.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:11 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:25:12.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:11 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:25:12.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:11 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:12.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:11 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:12.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:11 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:25:12.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:11 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:25:12.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:11 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:12.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:11 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:12.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:11 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:25:12.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:11 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:25:12.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:11 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:12.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:11 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:12.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.701+0000 7f35966dd640 1 -- 172.21.15.3:0/160953602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3590109e40 msgr2=0x7f3590110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:12.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.701+0000 7f35966dd640 1 --2- 172.21.15.3:0/160953602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3590109e40 0x7f3590110670 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f3588009ca0 tx=0x7f358802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.701+0000 7f35966dd640 1 -- 172.21.15.3:0/160953602 shutdown_connections 2024-09-21T06:25:12.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.701+0000 7f35966dd640 1 --2- 172.21.15.3:0/160953602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35901094f0 0x7f3590109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.701+0000 7f35966dd640 1 --2- 172.21.15.3:0/160953602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3590109e40 0x7f3590110670 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.701+0000 7f35966dd640 1 --2- 172.21.15.3:0/160953602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3590103bb0 0x7f3590103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.701+0000 7f35966dd640 1 -- 172.21.15.3:0/160953602 >> 172.21.15.3:0/160953602 conn(0x7f35900ff600 msgr2=0x7f3590101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:12.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.701+0000 7f35966dd640 1 -- 172.21.15.3:0/160953602 shutdown_connections 2024-09-21T06:25:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.701+0000 7f35966dd640 1 -- 172.21.15.3:0/160953602 wait complete. 2024-09-21T06:25:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.701+0000 7f35966dd640 1 Processor -- start 2024-09-21T06:25:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.701+0000 7f35966dd640 1 -- start start 2024-09-21T06:25:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f35966dd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3590103bb0 0x7f359019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f35966dd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35901094f0 0x7f359019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f35966dd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3590109e40 0x7f359019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f35966dd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3590113b00 con 0x7f3590109e40 2024-09-21T06:25:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f35966dd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3590113980 con 0x7f3590103bb0 2024-09-21T06:25:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f35966dd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3590113c80 con 0x7f35901094f0 2024-09-21T06:25:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f3594c7a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3590109e40 0x7f359019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f3594c7a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3590109e40 0x7f359019fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44532/0 (socket says 172.21.15.3:44532) 2024-09-21T06:25:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f3594c7a640 1 -- 172.21.15.3:0/3215240970 learned_addr learned my addr 172.21.15.3:0/3215240970 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f358ffff640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3590103bb0 0x7f359019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f3587fff640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35901094f0 0x7f359019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f3594c7a640 1 -- 172.21.15.3:0/3215240970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35901094f0 msgr2=0x7f359019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f3594c7a640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35901094f0 0x7f359019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f3594c7a640 1 -- 172.21.15.3:0/3215240970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3590103bb0 msgr2=0x7f359019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f3594c7a640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3590103bb0 0x7f359019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.702+0000 7f3594c7a640 1 -- 172.21.15.3:0/3215240970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35901a0320 con 0x7f3590109e40 2024-09-21T06:25:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.703+0000 7f3594c7a640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3590109e40 0x7f359019fcb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f3588005cc0 tx=0x7f358802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.703+0000 7f358dffb640 1 -- 172.21.15.3:0/3215240970 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588003bf0 con 0x7f3590109e40 2024-09-21T06:25:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.703+0000 7f358dffb640 1 -- 172.21.15.3:0/3215240970 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3588003d90 con 0x7f3590109e40 2024-09-21T06:25:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.703+0000 7f358dffb640 1 -- 172.21.15.3:0/3215240970 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358802fb20 con 0x7f3590109e40 2024-09-21T06:25:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.703+0000 7f35966dd640 1 -- 172.21.15.3:0/3215240970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35901a05b0 con 0x7f3590109e40 2024-09-21T06:25:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.703+0000 7f35966dd640 1 -- 172.21.15.3:0/3215240970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35901a0aa0 con 0x7f3590109e40 2024-09-21T06:25:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.706+0000 7f359cc51740 1 Processor -- start 2024-09-21T06:25:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.706+0000 7f358dffb640 1 -- 172.21.15.3:0/3215240970 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f3588004580 con 0x7f3590109e40 2024-09-21T06:25:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.706+0000 7f359cc51740 1 -- start start 2024-09-21T06:25:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.706+0000 7f359cc51740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 0x556d257f1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.706+0000 7f359cc51740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d254211b0 0x556d257f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.706+0000 7f359cc51740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d25429390 0x556d257f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.706+0000 7f359cc51740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556d25767e50 con 0x556d2544a070 2024-09-21T06:25:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.706+0000 7f359cc51740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556d25757280 con 0x556d25429390 2024-09-21T06:25:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.706+0000 7f359cc51740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556d2575b090 con 0x556d254211b0 2024-09-21T06:25:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.706+0000 7f3587fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 0x556d257f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.706+0000 7f3587fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 0x556d257f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44540/0 (socket says 172.21.15.3:44540) 2024-09-21T06:25:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.706+0000 7f3587fff640 1 -- 172.21.15.3:0/3857329422 learned_addr learned my addr 172.21.15.3:0/3857329422 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.706+0000 7f358ffff640 1 --2- 172.21.15.3:0/3857329422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d254211b0 0x556d257f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.707+0000 7f358dffb640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f357c0b66b0 0x7f357c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.707+0000 7f3587fff640 1 -- 172.21.15.3:0/3857329422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d254211b0 msgr2=0x556d257f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.707+0000 7f3587fff640 1 --2- 172.21.15.3:0/3857329422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d254211b0 0x556d257f1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.707+0000 7f3587fff640 1 -- 172.21.15.3:0/3857329422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d25429390 msgr2=0x556d257f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.707+0000 7f3587fff640 1 --2- 172.21.15.3:0/3857329422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d25429390 0x556d257f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.707+0000 7f3587fff640 1 -- 172.21.15.3:0/3857329422 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d254b5650 con 0x556d2544a070 2024-09-21T06:25:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.707+0000 7f358ffff640 1 --2- 172.21.15.3:0/3857329422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d254211b0 0x556d257f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.707+0000 7f3594c7a640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f357c0b66b0 0x7f357c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.707+0000 7f3587fff640 1 --2- 172.21.15.3:0/3857329422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 0x556d257f1650 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f357800bcd0 tx=0x7f35780127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551b26c8feb36eff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.707+0000 7f358dffb640 1 -- 172.21.15.3:0/3215240970 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8551+0+0 (secure 0 0 0) 0x7f35881089e0 con 0x7f3590109e40 2024-09-21T06:25:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.707+0000 7f35867fc640 1 -- 172.21.15.3:0/3857329422 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35780120d0 con 0x556d2544a070 2024-09-21T06:25:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.707+0000 7f35867fc640 1 -- 172.21.15.3:0/3857329422 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3578016070 con 0x556d2544a070 2024-09-21T06:25:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.707+0000 7f35867fc640 1 -- 172.21.15.3:0/3857329422 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35780047a0 con 0x556d2544a070 2024-09-21T06:25:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.708+0000 7f359cc51740 1 -- 172.21.15.3:0/3857329422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 msgr2=0x556d257f1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.708+0000 7f3594c7a640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f357c0b66b0 0x7f357c0b8bc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f35881061c0 tx=0x7f358809bc00 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.708+0000 7f359cc51740 1 --2- 172.21.15.3:0/3857329422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 0x556d257f1650 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f357800bcd0 tx=0x7f35780127c0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.708+0000 7f359cc51740 1 -- 172.21.15.3:0/3857329422 shutdown_connections 2024-09-21T06:25:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.708+0000 7f359cc51740 1 --2- 172.21.15.3:0/3857329422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d254211b0 0x556d257f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.708+0000 7f359cc51740 1 --2- 172.21.15.3:0/3857329422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d25429390 0x556d257f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.708+0000 7f359cc51740 1 --2- 172.21.15.3:0/3857329422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 0x556d257f1650 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.708+0000 7f359cc51740 1 -- 172.21.15.3:0/3857329422 >> 172.21.15.3:0/3857329422 conn(0x556d2543e460 msgr2=0x556d257ef540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.708+0000 7f359cc51740 1 -- 172.21.15.3:0/3857329422 shutdown_connections 2024-09-21T06:25:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.708+0000 7f359cc51740 1 -- 172.21.15.3:0/3857329422 wait complete. 2024-09-21T06:25:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.708+0000 7f359cc51740 1 -- start start 2024-09-21T06:25:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.709+0000 7f359cc51740 1 Processor -- start 2024-09-21T06:25:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.709+0000 7f359cc51740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d254211b0 0x556d258010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.709+0000 7f359cc51740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 0x556d25801610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.709+0000 7f359cc51740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d2544a070 0x556d25801b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.709+0000 7f359cc51740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556d255b8700 con 0x556d25429390 2024-09-21T06:25:12.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.709+0000 7f359cc51740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556d256f2110 con 0x556d2544a070 2024-09-21T06:25:12.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.709+0000 7f359cc51740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556d25804190 con 0x556d254211b0 2024-09-21T06:25:12.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.709+0000 7f359cc51740 10 client.0 _ll_register_callbacks cb 0x556d2570b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:12.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.709+0000 7f358ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 0x556d25801610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.709+0000 7f358ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 0x556d25801610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44556/0 (socket says 172.21.15.3:44556) 2024-09-21T06:25:12.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.709+0000 7f358ffff640 1 -- 172.21.15.3:0/1455669038 learned_addr learned my addr 172.21.15.3:0/1455669038 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.709+0000 7f3587fff640 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d254211b0 0x556d258010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.709+0000 7f3587fff640 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d2544a070 0x556d25801b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f358ffff640 1 -- 172.21.15.3:0/1455669038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d254211b0 msgr2=0x556d258010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f358ffff640 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d254211b0 0x556d258010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f358ffff640 1 -- 172.21.15.3:0/1455669038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d2544a070 msgr2=0x556d25801b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f358ffff640 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d2544a070 0x556d25801b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f358ffff640 1 -- 172.21.15.3:0/1455669038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2571fc40 con 0x556d25429390 2024-09-21T06:25:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f3587fff640 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d2544a070 0x556d25801b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f358ffff640 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 0x556d25801610 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f35800127d0 tx=0x7f3580012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b402135118a12b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:12.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f356f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:12.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f356f7fe640 1 -- 172.21.15.3:0/1455669038 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358001f060 con 0x556d25429390 2024-09-21T06:25:12.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f356f7fe640 1 -- 172.21.15.3:0/1455669038 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8551+0+0 (secure 0 0 0) 0x7f3580010dc0 con 0x556d25429390 2024-09-21T06:25:12.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f359cc51740 1 -- 172.21.15.3:0/1455669038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3578016070 con 0x556d25429390 2024-09-21T06:25:12.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f356f7fe640 1 -- 172.21.15.3:0/1455669038 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3580023430 con 0x556d25429390 2024-09-21T06:25:12.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f359cc51740 10 client.18342 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:12.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f359cc51740 1 -- 172.21.15.3:0/1455669038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556d2580fed0 con 0x556d25429390 2024-09-21T06:25:12.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.710+0000 7f356f7fe640 1 -- 172.21.15.3:0/1455669038 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3578016070 con 0x556d25429390 2024-09-21T06:25:12.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.711+0000 7f356f7fe640 1 -- 172.21.15.3:0/1455669038 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f358000e4f0 con 0x556d25429390 2024-09-21T06:25:12.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.711+0000 7f359cc51740 10 client.18342 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:12.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.711+0000 7f359cc51740 10 client.18342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:12.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.711+0000 7f359cc51740 20 client.18342 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:12.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.711+0000 7f359cc51740 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556d254089f0 0x556d25813420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.711+0000 7f359cc51740 4 client.18342 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:12.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.711+0000 7f359cc51740 1 -- 172.21.15.3:0/1455669038 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d25769da0 con 0x556d254089f0 2024-09-21T06:25:12.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.711+0000 7f3587fff640 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556d254089f0 0x556d25813420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.712+0000 7f3587fff640 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556d254089f0 0x556d25813420 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba3abcb1e3beb0d server_cookie=5325cbd7e03dd053 in_seq=0 out_seq=0 2024-09-21T06:25:12.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.712+0000 7f356f7fe640 10 client.18342 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:12.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.712+0000 7f356f7fe640 1 -- 172.21.15.3:0/1455669038 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556d25769da0 con 0x556d254089f0 2024-09-21T06:25:12.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.712+0000 7f356f7fe640 10 client.18342 handle_command_reply: tid=0 2024-09-21T06:25:12.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.712+0000 7f359cc51740 1 client.18342 shutdown 2024-09-21T06:25:12.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.713+0000 7f359cc51740 1 -- 172.21.15.3:0/1455669038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 msgr2=0x556d25801610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:12.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.713+0000 7f359cc51740 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 0x556d25801610 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f35800127d0 tx=0x7f3580012c90 comp rx=0 tx=0).stop 2024-09-21T06:25:12.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.713+0000 7f359cc51740 1 -- 172.21.15.3:0/1455669038 shutdown_connections 2024-09-21T06:25:12.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.713+0000 7f359cc51740 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556d254089f0 0x556d25813420 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.713+0000 7f359cc51740 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d2544a070 0x556d25801b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.713+0000 7f359cc51740 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 0x556d25801610 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.713+0000 7f359cc51740 1 --2- 172.21.15.3:0/1455669038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d254211b0 0x556d258010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.713+0000 7f359cc51740 1 -- 172.21.15.3:0/1455669038 >> 172.21.15.3:0/1455669038 conn(0x556d2543e460 msgr2=0x556d257edb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:12.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.713+0000 7f356f7fe640 0 client.18342 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:12.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.713+0000 7f359cc51740 1 -- 172.21.15.3:0/1455669038 shutdown_connections 2024-09-21T06:25:12.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.713+0000 7f359cc51740 1 -- 172.21.15.3:0/1455669038 wait complete. 2024-09-21T06:25:12.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.713+0000 7f359cc51740 20 client.18342 trim_cache size 0 max 16384 2024-09-21T06:25:12.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.730+0000 7f359cc51740 1 Processor -- start 2024-09-21T06:25:12.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.730+0000 7f359cc51740 1 -- start start 2024-09-21T06:25:12.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f359cc51740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d254211b0 0x556d25801590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f359cc51740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 0x556d257eee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f359cc51740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2544a070 0x556d257ef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f359cc51740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f35901ad4e0 con 0x556d25429390 2024-09-21T06:25:12.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f359cc51740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f35901af020 con 0x556d254211b0 2024-09-21T06:25:12.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f359cc51740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f35901adbb0 con 0x556d2544a070 2024-09-21T06:25:12.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f358ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 0x556d257eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f358ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 0x556d257eee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44572/0 (socket says 172.21.15.3:44572) 2024-09-21T06:25:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f358ffff640 1 -- 172.21.15.3:0/4056273544 learned_addr learned my addr 172.21.15.3:0/4056273544 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f3587fff640 1 --2- 172.21.15.3:0/4056273544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d254211b0 0x556d25801590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f358ffff640 1 -- 172.21.15.3:0/4056273544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2544a070 msgr2=0x556d257ef3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f3587fff640 1 --2- 172.21.15.3:0/4056273544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2544a070 0x556d257ef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f358ffff640 1 --2- 172.21.15.3:0/4056273544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2544a070 0x556d257ef3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f358ffff640 1 -- 172.21.15.3:0/4056273544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d254211b0 msgr2=0x556d25801590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f358ffff640 1 --2- 172.21.15.3:0/4056273544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d254211b0 0x556d25801590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f358ffff640 1 -- 172.21.15.3:0/4056273544 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2580d3c0 con 0x556d25429390 2024-09-21T06:25:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.731+0000 7f3587fff640 1 --2- 172.21.15.3:0/4056273544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2544a070 0x556d257ef3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:12.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.732+0000 7f358ffff640 1 --2- 172.21.15.3:0/4056273544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 0x556d257eee60 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f3580000c00 tx=0x7f358000af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdeb47844bb9ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:12.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.732+0000 7f3585ffb640 1 -- 172.21.15.3:0/4056273544 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580010bc0 con 0x556d25429390 2024-09-21T06:25:12.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.732+0000 7f3585ffb640 1 -- 172.21.15.3:0/4056273544 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3580030070 con 0x556d25429390 2024-09-21T06:25:12.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.732+0000 7f3585ffb640 1 -- 172.21.15.3:0/4056273544 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580023b50 con 0x556d25429390 2024-09-21T06:25:12.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.732+0000 7f359cc51740 1 -- 172.21.15.3:0/4056273544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 msgr2=0x556d257eee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:12.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.732+0000 7f359cc51740 1 --2- 172.21.15.3:0/4056273544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 0x556d257eee60 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f3580000c00 tx=0x7f358000af70 comp rx=0 tx=0).stop 2024-09-21T06:25:12.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.733+0000 7f359cc51740 1 -- 172.21.15.3:0/4056273544 shutdown_connections 2024-09-21T06:25:12.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.733+0000 7f359cc51740 1 --2- 172.21.15.3:0/4056273544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2544a070 0x556d257ef3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.733+0000 7f359cc51740 1 --2- 172.21.15.3:0/4056273544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d25429390 0x556d257eee60 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.733+0000 7f359cc51740 1 --2- 172.21.15.3:0/4056273544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d254211b0 0x556d25801590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.733+0000 7f359cc51740 1 -- 172.21.15.3:0/4056273544 >> 172.21.15.3:0/4056273544 conn(0x556d2543e460 msgr2=0x556d25800e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.733+0000 7f359cc51740 1 -- 172.21.15.3:0/4056273544 shutdown_connections 2024-09-21T06:25:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.733+0000 7f359cc51740 1 -- 172.21.15.3:0/4056273544 wait complete. 2024-09-21T06:25:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.733+0000 7f359cc51740 1 -- start start 2024-09-21T06:25:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.733+0000 7f359cc51740 1 Processor -- start 2024-09-21T06:25:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f359cc51740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d254211b0 0x556d25805850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f359cc51740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d25429390 0x556d25805e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f359cc51740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 0x556d2587cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f359cc51740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f35901ae950 con 0x556d2544a070 2024-09-21T06:25:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f359cc51740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f35901ae280 con 0x556d254211b0 2024-09-21T06:25:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f359cc51740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f35901ab4e0 con 0x556d25429390 2024-09-21T06:25:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f359cc51740 10 client.0 _ll_register_callbacks cb 0x7f358000a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f3587fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 0x556d2587cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f3587fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 0x556d2587cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44574/0 (socket says 172.21.15.3:44574) 2024-09-21T06:25:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f3587fff640 1 -- 172.21.15.3:0/2272135254 learned_addr learned my addr 172.21.15.3:0/2272135254 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f358ffff640 1 --2- 172.21.15.3:0/2272135254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d25429390 0x556d25805e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f3587fff640 1 --2- 172.21.15.3:0/2272135254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d254211b0 0x556d25805850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f3587fff640 1 -- 172.21.15.3:0/2272135254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d25429390 msgr2=0x556d25805e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f3587fff640 1 --2- 172.21.15.3:0/2272135254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d25429390 0x556d25805e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f3587fff640 1 -- 172.21.15.3:0/2272135254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d254211b0 msgr2=0x556d25805850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f3587fff640 1 --2- 172.21.15.3:0/2272135254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d254211b0 0x556d25805850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f3587fff640 1 -- 172.21.15.3:0/2272135254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2587d300 con 0x556d2544a070 2024-09-21T06:25:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f3587fff640 1 --2- 172.21.15.3:0/2272135254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 0x556d2587cdc0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x556d2570b9d0 tx=0x7f3578014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792a1e28939d35bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.734+0000 7f356f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f356f7fe640 1 -- 172.21.15.3:0/2272135254 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578037060 con 0x556d2544a070 2024-09-21T06:25:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f356f7fe640 1 -- 172.21.15.3:0/2272135254 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8551+0+0 (secure 0 0 0) 0x7f35780053d0 con 0x556d2544a070 2024-09-21T06:25:12.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f359cc51740 1 -- 172.21.15.3:0/2272135254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3580030070 con 0x556d2544a070 2024-09-21T06:25:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f356f7fe640 1 -- 172.21.15.3:0/2272135254 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f35780055d0 con 0x556d2544a070 2024-09-21T06:25:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f359cc51740 10 client.18348 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f359cc51740 1 -- 172.21.15.3:0/2272135254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556d2587e440 con 0x556d2544a070 2024-09-21T06:25:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f356f7fe640 1 -- 172.21.15.3:0/2272135254 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3580030070 con 0x556d2544a070 2024-09-21T06:25:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f356f7fe640 1 -- 172.21.15.3:0/2272135254 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f357802a4a0 con 0x556d2544a070 2024-09-21T06:25:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f359cc51740 10 client.18348 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f359cc51740 10 client.18348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f359cc51740 20 client.18348 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f359cc51740 1 --2- 172.21.15.3:0/2272135254 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556d253ddda0 0x556d25880980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:12.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f359cc51740 4 client.18348 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:25:12.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.735+0000 7f359cc51740 1 -- 172.21.15.3:0/2272135254 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556d25765850 con 0x556d253ddda0 2024-09-21T06:25:12.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.736+0000 7f358ffff640 1 --2- 172.21.15.3:0/2272135254 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556d253ddda0 0x556d25880980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:12.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.736+0000 7f358ffff640 1 --2- 172.21.15.3:0/2272135254 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556d253ddda0 0x556d25880980 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e28f67445f4f602f server_cookie=d7a9ecf84b65170a in_seq=0 out_seq=0 2024-09-21T06:25:12.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.736+0000 7f356f7fe640 10 client.18348 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:12.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.736+0000 7f356f7fe640 1 -- 172.21.15.3:0/2272135254 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x556d25765850 con 0x556d253ddda0 2024-09-21T06:25:12.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.736+0000 7f356f7fe640 10 client.18348 handle_command_reply: tid=0 2024-09-21T06:25:12.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.736+0000 7f359cc51740 1 client.18348 shutdown 2024-09-21T06:25:12.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f359cc51740 1 -- 172.21.15.3:0/2272135254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 msgr2=0x556d2587cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:12.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f359cc51740 1 --2- 172.21.15.3:0/2272135254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 0x556d2587cdc0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x556d2570b9d0 tx=0x7f3578014040 comp rx=0 tx=0).stop 2024-09-21T06:25:12.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f359cc51740 1 -- 172.21.15.3:0/2272135254 shutdown_connections 2024-09-21T06:25:12.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f359cc51740 1 --2- 172.21.15.3:0/2272135254 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x556d253ddda0 0x556d25880980 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f359cc51740 1 --2- 172.21.15.3:0/2272135254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d25429390 0x556d25805e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f359cc51740 1 --2- 172.21.15.3:0/2272135254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556d2544a070 0x556d2587cdc0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f359cc51740 1 --2- 172.21.15.3:0/2272135254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d254211b0 0x556d25805850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f356f7fe640 0 client.18348 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:12.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f359cc51740 1 -- 172.21.15.3:0/2272135254 >> 172.21.15.3:0/2272135254 conn(0x556d2543e460 msgr2=0x556d257eee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:12.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f359cc51740 1 -- 172.21.15.3:0/2272135254 shutdown_connections 2024-09-21T06:25:12.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f359cc51740 1 -- 172.21.15.3:0/2272135254 wait complete. 2024-09-21T06:25:12.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f359cc51740 20 client.18348 trim_cache size 0 max 16384 2024-09-21T06:25:12.746 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:25:12.746 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:25:12.746 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-21T06:25:12.746 INFO:teuthology.orchestra.run.smithi003.stdout: "9135cf69-938c-4d40-9a5c-95e31d48b371": { 2024-09-21T06:25:12.746 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.746 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "9135cf69-938c-4d40-9a5c-95e31d48b371", 2024-09-21T06:25:12.747 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-21T06:25:12.747 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:12.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.747 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:25:12.747 INFO:teuthology.orchestra.run.smithi003.stdout: "9135cf69-938c-4d40-9a5c-95e31d48b371": { 2024-09-21T06:25:12.747 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:25:10.991220+0000", 2024-09-21T06:25:12.747 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.747 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.747 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.748 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.748 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.748 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:25:12.748 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:25:12.748 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.748 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.748 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.748 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.748 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.749 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.749 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:25:12.749 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:25:12.749 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.749 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.749 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.749 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.749 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.749 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.749 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:25:12.750 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:25:12.750 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.750 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.750 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.750 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.750 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.750 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:25:12.750 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:25:12.751 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.751 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.751 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.751 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.751 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.751 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.751 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:25:12.751 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:25:12.751 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.751 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.752 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.752 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.752 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.752 INFO:teuthology.orchestra.run.smithi003.stdout: "cbb5d4a7-b96d-4c2c-b7ab-7a3716c17f5b": { 2024-09-21T06:25:12.752 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:23:38.372765+0000", 2024-09-21T06:25:12.752 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.752 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.752 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.753 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.753 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.753 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.753 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:25:12.753 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:25:12.753 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.753 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.753 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.753 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.753 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.754 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.754 INFO:teuthology.orchestra.run.smithi003.stdout: "0bf99782-6ab2-4000-ba98-9ddb00a60402": { 2024-09-21T06:25:12.754 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:24:24.782991+0000", 2024-09-21T06:25:12.754 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.754 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.754 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.754 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.754 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.754 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.754 INFO:teuthology.orchestra.run.smithi003.stdout: "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58": { 2024-09-21T06:25:12.755 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:21:17.396219+0000", 2024-09-21T06:25:12.755 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.755 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.755 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.755 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.755 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.755 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.755 INFO:teuthology.orchestra.run.smithi003.stdout: "714b9934-a2d3-4a78-b185-01254776061b": { 2024-09-21T06:25:12.755 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:04.685380+0000", 2024-09-21T06:25:12.755 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.756 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.756 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.756 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.756 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.756 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.756 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:25:12.756 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:25:12.756 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.756 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.756 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.756 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.757 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.757 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.757 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:25:12.757 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:25:12.757 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.757 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.757 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.757 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.757 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.757 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.758 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:25:12.758 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:25:12.758 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.758 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.758 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.758 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.758 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.758 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:12.758 INFO:teuthology.orchestra.run.smithi003.stdout: "535fb164-fd4f-4d88-b7e0-575559314385": { 2024-09-21T06:25:12.759 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:51.152572+0000", 2024-09-21T06:25:12.759 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:12.759 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:12.759 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.759 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.759 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.759 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:12.759 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:12.760 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:25:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f35966dd640 1 -- 172.21.15.3:0/3215240970 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f357c0b66b0 msgr2=0x7f357c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f35966dd640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f357c0b66b0 0x7f357c0b8bc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f35881061c0 tx=0x7f358809bc00 comp rx=0 tx=0).stop 2024-09-21T06:25:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f35966dd640 1 -- 172.21.15.3:0/3215240970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3590109e40 msgr2=0x7f359019fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.737+0000 7f35966dd640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3590109e40 0x7f359019fcb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f3588005cc0 tx=0x7f358802f380 comp rx=0 tx=0).stop 2024-09-21T06:25:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.738+0000 7f35966dd640 1 -- 172.21.15.3:0/3215240970 shutdown_connections 2024-09-21T06:25:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.738+0000 7f35966dd640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f357c0b66b0 0x7f357c0b8bc0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.738+0000 7f35966dd640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35901094f0 0x7f359019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.738+0000 7f35966dd640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3590109e40 0x7f359019fcb0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.738+0000 7f35966dd640 1 --2- 172.21.15.3:0/3215240970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3590103bb0 0x7f359019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.738+0000 7f35966dd640 1 -- 172.21.15.3:0/3215240970 >> 172.21.15.3:0/3215240970 conn(0x7f35900ff600 msgr2=0x7f3590101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.738+0000 7f35966dd640 1 -- 172.21.15.3:0/3215240970 shutdown_connections 2024-09-21T06:25:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:12.738+0000 7f35966dd640 1 -- 172.21.15.3:0/3215240970 wait complete. 2024-09-21T06:25:13.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:12 smithi137 ceph-mon[34464]: scrub queued for path: / 2024-09-21T06:25:13.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:12 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:13.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:12 smithi137 ceph-mon[34464]: scrub summary: active paths [/] 2024-09-21T06:25:13.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:12 smithi137 ceph-mon[34464]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:13.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:12 smithi137 ceph-mon[34464]: pgmap v640: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-09-21T06:25:13.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:12 smithi204 ceph-mon[34659]: scrub queued for path: / 2024-09-21T06:25:13.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:12 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:13.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:12 smithi204 ceph-mon[34659]: scrub summary: active paths [/] 2024-09-21T06:25:13.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:12 smithi204 ceph-mon[34659]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:13.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:12 smithi204 ceph-mon[34659]: pgmap v640: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-09-21T06:25:13.193 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:12 smithi003 ceph-mon[31227]: scrub queued for path: / 2024-09-21T06:25:13.194 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:12 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:13.194 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:12 smithi003 ceph-mon[31227]: scrub summary: active paths [/] 2024-09-21T06:25:13.194 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:12 smithi003 ceph-mon[31227]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:13.194 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:12 smithi003 ceph-mon[31227]: pgmap v640: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-09-21T06:25:13.533 INFO:tasks.cephfs.filesystem:scrub status for tag:9135cf69-938c-4d40-9a5c-95e31d48b371 - {'path': '/', 'tag': '9135cf69-938c-4d40-9a5c-95e31d48b371', 'options': 'recursive,force'} 2024-09-21T06:25:15.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:15 smithi204 ceph-mon[34659]: pgmap v641: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-09-21T06:25:15.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:15 smithi204 ceph-mon[34659]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:25:15.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:15 smithi003 ceph-mon[31227]: pgmap v641: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-09-21T06:25:15.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:15 smithi003 ceph-mon[31227]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:25:15.848 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:15 smithi137 ceph-mon[34464]: pgmap v641: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-09-21T06:25:15.848 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:15 smithi137 ceph-mon[34464]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:25:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:17 smithi204 ceph-mon[34659]: pgmap v643: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 221 B/s wr, 17 op/s 2024-09-21T06:25:17.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:17 smithi003 ceph-mon[31227]: pgmap v643: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 221 B/s wr, 17 op/s 2024-09-21T06:25:17.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:17 smithi137 ceph-mon[34464]: pgmap v643: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 221 B/s wr, 17 op/s 2024-09-21T06:25:18.841 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T06:25:18.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.969+0000 7f967a958640 1 -- 172.21.15.3:0/3127831999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9674108c60 msgr2=0x7f967410f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:18.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.969+0000 7f967a958640 1 --2- 172.21.15.3:0/3127831999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9674108c60 0x7f967410f490 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f966c009ca0 tx=0x7f966c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:25:18.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.969+0000 7f967a958640 1 -- 172.21.15.3:0/3127831999 shutdown_connections 2024-09-21T06:25:18.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.969+0000 7f967a958640 1 --2- 172.21.15.3:0/3127831999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9674108310 0x7f9674108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.969+0000 7f967a958640 1 --2- 172.21.15.3:0/3127831999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9674108c60 0x7f967410f490 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.969+0000 7f967a958640 1 --2- 172.21.15.3:0/3127831999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96741029e0 0x7f9674102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.969+0000 7f967a958640 1 -- 172.21.15.3:0/3127831999 >> 172.21.15.3:0/3127831999 conn(0x7f96740fe3c0 msgr2=0x7f9674100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.969+0000 7f967a958640 1 -- 172.21.15.3:0/3127831999 shutdown_connections 2024-09-21T06:25:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.969+0000 7f967a958640 1 -- 172.21.15.3:0/3127831999 wait complete. 2024-09-21T06:25:18.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.970+0000 7f967a958640 1 Processor -- start 2024-09-21T06:25:18.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.970+0000 7f967a958640 1 -- start start 2024-09-21T06:25:18.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.970+0000 7f967a958640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96741029e0 0x7f967419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:18.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.970+0000 7f967a958640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9674108310 0x7f967419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:18.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.970+0000 7f967a958640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9674108c60 0x7f967419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:18.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.970+0000 7f967a958640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9674112920 con 0x7f9674108310 2024-09-21T06:25:18.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.970+0000 7f967a958640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f96741127a0 con 0x7f9674108c60 2024-09-21T06:25:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.970+0000 7f967a958640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9674112aa0 con 0x7f96741029e0 2024-09-21T06:25:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f96737fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9674108310 0x7f967419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f96737fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9674108310 0x7f967419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38930/0 (socket says 172.21.15.3:38930) 2024-09-21T06:25:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f96737fe640 1 -- 172.21.15.3:0/3909014289 learned_addr learned my addr 172.21.15.3:0/3909014289 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f9678ef5640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9674108c60 0x7f967419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f9673fff640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96741029e0 0x7f967419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f96737fe640 1 -- 172.21.15.3:0/3909014289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96741029e0 msgr2=0x7f967419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f96737fe640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96741029e0 0x7f967419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f96737fe640 1 -- 172.21.15.3:0/3909014289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9674108c60 msgr2=0x7f967419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f96737fe640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9674108c60 0x7f967419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f96737fe640 1 -- 172.21.15.3:0/3909014289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96741a0320 con 0x7f9674108310 2024-09-21T06:25:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f96737fe640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9674108310 0x7f967419a990 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f966800e9f0 tx=0x7f966800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f96717fa640 1 -- 172.21.15.3:0/3909014289 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966800ccc0 con 0x7f9674108310 2024-09-21T06:25:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f96717fa640 1 -- 172.21.15.3:0/3909014289 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96680108e0 con 0x7f9674108310 2024-09-21T06:25:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f967a958640 1 -- 172.21.15.3:0/3909014289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96741a0610 con 0x7f9674108310 2024-09-21T06:25:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f96717fa640 1 -- 172.21.15.3:0/3909014289 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668018860 con 0x7f9674108310 2024-09-21T06:25:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.971+0000 7f967a958640 1 -- 172.21.15.3:0/3909014289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96741a6da0 con 0x7f9674108310 2024-09-21T06:25:18.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.972+0000 7f967a958640 1 -- 172.21.15.3:0/3909014289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9674102df0 con 0x7f9674108310 2024-09-21T06:25:18.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.974+0000 7f96717fa640 1 -- 172.21.15.3:0/3909014289 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f9668010430 con 0x7f9674108310 2024-09-21T06:25:18.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.975+0000 7f96717fa640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f965c0b6680 0x7f965c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:18.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.975+0000 7f96717fa640 1 -- 172.21.15.3:0/3909014289 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f96680e7ea0 con 0x7f9674108310 2024-09-21T06:25:18.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.975+0000 7f9673fff640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f965c0b6680 0x7f965c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:18.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.976+0000 7f9673fff640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f965c0b6680 0x7f965c0b8b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9664005d00 tx=0x7f9664005c50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:18.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:18.978+0000 7f96717fa640 1 -- 172.21.15.3:0/3909014289 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f96680ae400 con 0x7f9674108310 2024-09-21T06:25:19.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.189+0000 7f967a958640 1 -- 172.21.15.3:0/3909014289 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9674063800 con 0x7f965c0b6680 2024-09-21T06:25:19.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.198+0000 7f96717fa640 1 -- 172.21.15.3:0/3909014289 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f9674063800 con 0x7f965c0b6680 2024-09-21T06:25:19.200 INFO:teuthology.orchestra.run.smithi003.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T06:15:17", "first": "2024-09-21T06:16:00", "last": "2024-09-21T06:25:00", "last_pruned": "2024-09-21T06:25:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-09-21T06:25:19.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.201+0000 7f967a958640 1 -- 172.21.15.3:0/3909014289 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f965c0b6680 msgr2=0x7f965c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:19.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.201+0000 7f967a958640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f965c0b6680 0x7f965c0b8b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9664005d00 tx=0x7f9664005c50 comp rx=0 tx=0).stop 2024-09-21T06:25:19.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.201+0000 7f967a958640 1 -- 172.21.15.3:0/3909014289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9674108310 msgr2=0x7f967419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:19.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.201+0000 7f967a958640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9674108310 0x7f967419a990 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f966800e9f0 tx=0x7f966800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.201+0000 7f967a958640 1 -- 172.21.15.3:0/3909014289 shutdown_connections 2024-09-21T06:25:19.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.201+0000 7f967a958640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f965c0b6680 0x7f965c0b8b90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.201+0000 7f967a958640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9674108c60 0x7f967419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.201+0000 7f967a958640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9674108310 0x7f967419a990 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.201+0000 7f967a958640 1 --2- 172.21.15.3:0/3909014289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96741029e0 0x7f967419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.201+0000 7f967a958640 1 -- 172.21.15.3:0/3909014289 >> 172.21.15.3:0/3909014289 conn(0x7f96740fe3c0 msgr2=0x7f9674101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.201+0000 7f967a958640 1 -- 172.21.15.3:0/3909014289 shutdown_connections 2024-09-21T06:25:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.201+0000 7f967a958640 1 -- 172.21.15.3:0/3909014289 wait complete. 2024-09-21T06:25:19.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T06:25:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.327+0000 7f2a93921640 1 -- 172.21.15.3:0/2222019748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a8c105280 msgr2=0x7f2a8c10d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.327+0000 7f2a93921640 1 --2- 172.21.15.3:0/2222019748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a8c105280 0x7f2a8c10d7c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009900 tx=0x7f2a8802a410 comp rx=0 tx=0).stop 2024-09-21T06:25:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.327+0000 7f2a93921640 1 -- 172.21.15.3:0/2222019748 shutdown_connections 2024-09-21T06:25:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.327+0000 7f2a93921640 1 --2- 172.21.15.3:0/2222019748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a8c10dd90 0x7f2a8c110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.327+0000 7f2a93921640 1 --2- 172.21.15.3:0/2222019748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a8c105280 0x7f2a8c10d7c0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.327+0000 7f2a93921640 1 --2- 172.21.15.3:0/2222019748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a8c104930 0x7f2a8c104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.327+0000 7f2a93921640 1 -- 172.21.15.3:0/2222019748 >> 172.21.15.3:0/2222019748 conn(0x7f2a8c0fe3e0 msgr2=0x7f2a8c100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:19.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.327+0000 7f2a93921640 1 -- 172.21.15.3:0/2222019748 shutdown_connections 2024-09-21T06:25:19.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.327+0000 7f2a93921640 1 -- 172.21.15.3:0/2222019748 wait complete. 2024-09-21T06:25:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.327+0000 7f2a93921640 1 Processor -- start 2024-09-21T06:25:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.327+0000 7f2a93921640 1 -- start start 2024-09-21T06:25:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a93921640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a8c104930 0x7f2a8c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a93921640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a8c105280 0x7f2a8c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a93921640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a8c10dd90 0x7f2a8c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a93921640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1139f0 con 0x7f2a8c104930 2024-09-21T06:25:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a93921640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c113870 con 0x7f2a8c105280 2024-09-21T06:25:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a93921640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c113b70 con 0x7f2a8c10dd90 2024-09-21T06:25:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a916bd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a8c104930 0x7f2a8c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a916bd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a8c104930 0x7f2a8c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38940/0 (socket says 172.21.15.3:38940) 2024-09-21T06:25:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a916bd640 1 -- 172.21.15.3:0/739701983 learned_addr learned my addr 172.21.15.3:0/739701983 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a90ebc640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a8c105280 0x7f2a8c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a91ebe640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a8c10dd90 0x7f2a8c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a916bd640 1 -- 172.21.15.3:0/739701983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a8c10dd90 msgr2=0x7f2a8c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a916bd640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a8c10dd90 0x7f2a8c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a916bd640 1 -- 172.21.15.3:0/739701983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a8c105280 msgr2=0x7f2a8c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a916bd640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a8c105280 0x7f2a8c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a916bd640 1 -- 172.21.15.3:0/739701983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c1a03a0 con 0x7f2a8c104930 2024-09-21T06:25:19.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.328+0000 7f2a916bd640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a8c104930 0x7f2a8c19a470 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000e9f0 tx=0x7f2a8000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:19.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.329+0000 7f2a7e7fc640 1 -- 172.21.15.3:0/739701983 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8000cd80 con 0x7f2a8c104930 2024-09-21T06:25:19.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.329+0000 7f2a93921640 1 -- 172.21.15.3:0/739701983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c1a0690 con 0x7f2a8c104930 2024-09-21T06:25:19.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.329+0000 7f2a7e7fc640 1 -- 172.21.15.3:0/739701983 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a800054d0 con 0x7f2a8c104930 2024-09-21T06:25:19.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.329+0000 7f2a7e7fc640 1 -- 172.21.15.3:0/739701983 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80010690 con 0x7f2a8c104930 2024-09-21T06:25:19.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.329+0000 7f2a93921640 1 -- 172.21.15.3:0/739701983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c1a6e20 con 0x7f2a8c104930 2024-09-21T06:25:19.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.330+0000 7f2a93921640 1 -- 172.21.15.3:0/739701983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a8c104d40 con 0x7f2a8c104930 2024-09-21T06:25:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.331+0000 7f2a7e7fc640 1 -- 172.21.15.3:0/739701983 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f2a80004e30 con 0x7f2a8c104930 2024-09-21T06:25:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.332+0000 7f2a7e7fc640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a780b6680 0x7f2a780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.332+0000 7f2a90ebc640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a780b6680 0x7f2a780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.333+0000 7f2a7e7fc640 1 -- 172.21.15.3:0/739701983 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f2a800e7cf0 con 0x7f2a8c104930 2024-09-21T06:25:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.333+0000 7f2a90ebc640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a780b6680 0x7f2a780b8b90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009230 tx=0x7f2a88031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.336+0000 7f2a7e7fc640 1 -- 172.21.15.3:0/739701983 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2a800ae250 con 0x7f2a8c104930 2024-09-21T06:25:19.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.545+0000 7f2a93921640 1 -- 172.21.15.3:0/739701983 --> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2a8c107fd0 con 0x7f2a780b6680 2024-09-21T06:25:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.554+0000 7f2a7e7fc640 1 -- 172.21.15.3:0/739701983 <== mgr.14223 v2:172.21.15.3:6800/2227168690 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2a8c107fd0 con 0x7f2a780b6680 2024-09-21T06:25:19.555 INFO:teuthology.orchestra.run.smithi003.stdout:/ 1m 6m3h 2024-09-21T06:25:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.556+0000 7f2a93921640 1 -- 172.21.15.3:0/739701983 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a780b6680 msgr2=0x7f2a780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.556+0000 7f2a93921640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a780b6680 0x7f2a780b8b90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009230 tx=0x7f2a88031400 comp rx=0 tx=0).stop 2024-09-21T06:25:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.556+0000 7f2a93921640 1 -- 172.21.15.3:0/739701983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a8c104930 msgr2=0x7f2a8c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.556+0000 7f2a93921640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a8c104930 0x7f2a8c19a470 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000e9f0 tx=0x7f2a8000edf0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.556+0000 7f2a93921640 1 -- 172.21.15.3:0/739701983 shutdown_connections 2024-09-21T06:25:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.556+0000 7f2a93921640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2a780b6680 0x7f2a780b8b90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.556+0000 7f2a93921640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a8c10dd90 0x7f2a8c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.556+0000 7f2a93921640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a8c105280 0x7f2a8c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.556+0000 7f2a93921640 1 --2- 172.21.15.3:0/739701983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a8c104930 0x7f2a8c19a470 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:19.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.556+0000 7f2a93921640 1 -- 172.21.15.3:0/739701983 >> 172.21.15.3:0/739701983 conn(0x7f2a8c0fe3e0 msgr2=0x7f2a8c1078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:19.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.556+0000 7f2a93921640 1 -- 172.21.15.3:0/739701983 shutdown_connections 2024-09-21T06:25:19.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:19.556+0000 7f2a93921640 1 -- 172.21.15.3:0/739701983 wait complete. 2024-09-21T06:25:19.568 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:19 smithi003 ceph-mon[31227]: pgmap v644: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 KiB/s wr, 18 op/s 2024-09-21T06:25:19.569 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-21T06:25:19.578 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-21T06:25:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:19 smithi204 ceph-mon[34659]: pgmap v644: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 KiB/s wr, 18 op/s 2024-09-21T06:25:19.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:19 smithi137 ceph-mon[34464]: pgmap v644: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 KiB/s wr, 18 op/s 2024-09-21T06:25:20.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:20 smithi003 ceph-mon[31227]: from='client.18354 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:25:20.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:20 smithi003 ceph-mon[31227]: from='client.18360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:25:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:20 smithi137 ceph-mon[34464]: from='client.18354 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:25:20.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:20 smithi137 ceph-mon[34464]: from='client.18360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:25:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:20 smithi204 ceph-mon[34659]: from='client.18354 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:25:20.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:20 smithi204 ceph-mon[34659]: from='client.18360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:25:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:21 smithi003 ceph-mon[31227]: pgmap v645: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 KiB/s wr, 18 op/s 2024-09-21T06:25:21.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:21 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:25:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:21 smithi137 ceph-mon[34464]: pgmap v645: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 KiB/s wr, 18 op/s 2024-09-21T06:25:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:21 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:25:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:21 smithi204 ceph-mon[34659]: pgmap v645: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 KiB/s wr, 18 op/s 2024-09-21T06:25:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:21 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:25:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:22 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:25:22.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:22 smithi003 ceph-mon[31227]: scrub summary: idle 2024-09-21T06:25:22.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:22 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:25:22.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:22 smithi137 ceph-mon[34464]: scrub summary: idle 2024-09-21T06:25:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:22 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:25:22.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:22 smithi204 ceph-mon[34659]: scrub summary: idle 2024-09-21T06:25:23.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:23 smithi003 ceph-mon[31227]: pgmap v646: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.6 KiB/s wr, 18 op/s 2024-09-21T06:25:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:23 smithi137 ceph-mon[34464]: pgmap v646: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.6 KiB/s wr, 18 op/s 2024-09-21T06:25:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:23 smithi204 ceph-mon[34659]: pgmap v646: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.6 KiB/s wr, 18 op/s 2024-09-21T06:25:25.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:25 smithi003 ceph-mon[31227]: pgmap v647: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.6 KiB/s wr, 17 op/s 2024-09-21T06:25:25.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:25 smithi137 ceph-mon[34464]: pgmap v647: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.6 KiB/s wr, 17 op/s 2024-09-21T06:25:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:25 smithi204 ceph-mon[34659]: pgmap v647: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.6 KiB/s wr, 17 op/s 2024-09-21T06:25:27.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:27 smithi003 ceph-mon[31227]: pgmap v648: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-09-21T06:25:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:27 smithi137 ceph-mon[34464]: pgmap v648: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-09-21T06:25:27.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:27 smithi204 ceph-mon[34659]: pgmap v648: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-09-21T06:25:28.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:28 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:25:28.851 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:28 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:25:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:28 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:25:29.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:29 smithi003 ceph-mon[31227]: pgmap v649: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-09-21T06:25:29.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:29.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:29 smithi137 ceph-mon[34464]: pgmap v649: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-09-21T06:25:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:29 smithi204 ceph-mon[34659]: pgmap v649: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-09-21T06:25:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:30 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:30 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:30 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:30 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:30 smithi003 ceph-mon[31227]: pgmap v650: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-21T06:25:31.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:30 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:30 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:30 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:25:31.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:30 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:25:31.092 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:30 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:30 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:30 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:30 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:30 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:30 smithi137 ceph-mon[34464]: pgmap v650: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-21T06:25:31.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:30 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:30 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:30 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:25:31.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:30 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:25:31.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:30 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:30 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:30 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:30 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:30 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:30 smithi204 ceph-mon[34659]: pgmap v650: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-21T06:25:31.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:30 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:30 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:31.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:30 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:25:31.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:30 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:25:31.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:30 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:25:33.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:32 smithi137 ceph-mon[34464]: pgmap v651: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2024-09-21T06:25:33.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:32 smithi204 ceph-mon[34659]: pgmap v651: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2024-09-21T06:25:33.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:32 smithi003 ceph-mon[31227]: pgmap v651: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2024-09-21T06:25:35.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:34 smithi137 ceph-mon[34464]: pgmap v652: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-21T06:25:35.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:34 smithi204 ceph-mon[34659]: pgmap v652: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-21T06:25:35.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:34 smithi003 ceph-mon[31227]: pgmap v652: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-21T06:25:37.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:36 smithi137 ceph-mon[34464]: pgmap v653: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-21T06:25:37.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:36 smithi204 ceph-mon[34659]: pgmap v653: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-21T06:25:37.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:36 smithi003 ceph-mon[31227]: pgmap v653: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-21T06:25:39.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:38 smithi204 ceph-mon[34659]: pgmap v654: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.9 KiB/s wr, 11 op/s 2024-09-21T06:25:39.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:38 smithi003 ceph-mon[31227]: pgmap v654: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.9 KiB/s wr, 11 op/s 2024-09-21T06:25:39.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:38 smithi137 ceph-mon[34464]: pgmap v654: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.9 KiB/s wr, 11 op/s 2024-09-21T06:25:41.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:40 smithi204 ceph-mon[34659]: pgmap v655: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T06:25:41.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:40 smithi003 ceph-mon[31227]: pgmap v655: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T06:25:41.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:40 smithi137 ceph-mon[34464]: pgmap v655: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T06:25:43.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:42 smithi204 ceph-mon[34659]: pgmap v656: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T06:25:43.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:42 smithi003 ceph-mon[31227]: pgmap v656: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T06:25:43.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:42 smithi137 ceph-mon[34464]: pgmap v656: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T06:25:43.534 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:25:43.807 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:44.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.285+0000 7f8cf215e640 1 -- 172.21.15.3:0/3629942542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cec10de90 msgr2=0x7f8cec110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.285+0000 7f8cf215e640 1 --2- 172.21.15.3:0/3629942542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cec10de90 0x7f8cec110380 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009ca0 tx=0x7f8ce402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.285+0000 7f8cf215e640 1 -- 172.21.15.3:0/3629942542 shutdown_connections 2024-09-21T06:25:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.285+0000 7f8cf215e640 1 --2- 172.21.15.3:0/3629942542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cec105380 0x7f8cec10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.285+0000 7f8cf215e640 1 --2- 172.21.15.3:0/3629942542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cec10de90 0x7f8cec110380 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.285+0000 7f8cf215e640 1 --2- 172.21.15.3:0/3629942542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cec104a30 0x7f8cec104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.285+0000 7f8cf215e640 1 -- 172.21.15.3:0/3629942542 >> 172.21.15.3:0/3629942542 conn(0x7f8cec079770 msgr2=0x7f8cec079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.285+0000 7f8cf215e640 1 -- 172.21.15.3:0/3629942542 shutdown_connections 2024-09-21T06:25:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.285+0000 7f8cf215e640 1 -- 172.21.15.3:0/3629942542 wait complete. 2024-09-21T06:25:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.286+0000 7f8cf215e640 1 Processor -- start 2024-09-21T06:25:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.286+0000 7f8cf215e640 1 -- start start 2024-09-21T06:25:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.286+0000 7f8cf215e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cec104a30 0x7f8cec19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.286+0000 7f8cf215e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cec105380 0x7f8cec19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.286+0000 7f8cf215e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cec10de90 0x7f8cec19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.286+0000 7f8cf215e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec113af0 con 0x7f8cec104a30 2024-09-21T06:25:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.286+0000 7f8cf215e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec113970 con 0x7f8cec105380 2024-09-21T06:25:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.286+0000 7f8cf215e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec113c70 con 0x7f8cec10de90 2024-09-21T06:25:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.286+0000 7f8ceb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cec104a30 0x7f8cec19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8ceb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cec104a30 0x7f8cec19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37568/0 (socket says 172.21.15.3:37568) 2024-09-21T06:25:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/1472549703 learned_addr learned my addr 172.21.15.3:0/1472549703 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8ceaffd640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cec105380 0x7f8cec19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8cebfff640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cec10de90 0x7f8cec19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/1472549703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cec10de90 msgr2=0x7f8cec19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8ceb7fe640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cec10de90 0x7f8cec19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/1472549703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cec105380 msgr2=0x7f8cec19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8ceb7fe640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cec105380 0x7f8cec19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/1472549703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cec1a03f0 con 0x7f8cec104a30 2024-09-21T06:25:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8cebfff640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cec10de90 0x7f8cec19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8ceb7fe640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cec104a30 0x7f8cec19a480 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00c890 tx=0x7f8cdc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8ce8ff9640 1 -- 172.21.15.3:0/1472549703 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc004220 con 0x7f8cec104a30 2024-09-21T06:25:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8cf215e640 1 -- 172.21.15.3:0/1472549703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cec1a06e0 con 0x7f8cec104a30 2024-09-21T06:25:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8ce8ff9640 1 -- 172.21.15.3:0/1472549703 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cdc0043c0 con 0x7f8cec104a30 2024-09-21T06:25:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8ce8ff9640 1 -- 172.21.15.3:0/1472549703 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc004a00 con 0x7f8cec104a30 2024-09-21T06:25:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.287+0000 7f8cf215e640 1 -- 172.21.15.3:0/1472549703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cec1a6e40 con 0x7f8cec104a30 2024-09-21T06:25:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.290+0000 7f8ce8ff9640 1 -- 172.21.15.3:0/1472549703 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f8cdc0079b0 con 0x7f8cec104a30 2024-09-21T06:25:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.290+0000 7f8cf86d2740 1 Processor -- start 2024-09-21T06:25:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.290+0000 7f8cf86d2740 1 -- start start 2024-09-21T06:25:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.290+0000 7f8cf86d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fbd1070 0x55706ff78270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.290+0000 7f8cf86d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 0x55706ff787b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.290+0000 7f8cf86d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 0x55706ff78cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.290+0000 7f8cf86d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55706feeee50 con 0x55706fba81b0 2024-09-21T06:25:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.290+0000 7f8cf86d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55706fede280 con 0x55706fbd1070 2024-09-21T06:25:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.290+0000 7f8cf86d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55706fee2090 con 0x55706fbb0390 2024-09-21T06:25:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.291+0000 7f8ceb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 0x55706ff787b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.291+0000 7f8ceb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 0x55706ff787b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37582/0 (socket says 172.21.15.3:37582) 2024-09-21T06:25:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.291+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/2063192199 learned_addr learned my addr 172.21.15.3:0/2063192199 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.291+0000 7f8cebfff640 1 --2- 172.21.15.3:0/2063192199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fbd1070 0x55706ff78270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.291+0000 7f8ceaffd640 1 --2- 172.21.15.3:0/2063192199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 0x55706ff78cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.291+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/2063192199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 msgr2=0x55706ff78cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.291+0000 7f8ceb7fe640 1 --2- 172.21.15.3:0/2063192199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 0x55706ff78cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.291+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/2063192199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fbd1070 msgr2=0x55706ff78270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.291+0000 7f8ceb7fe640 1 --2- 172.21.15.3:0/2063192199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fbd1070 0x55706ff78270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.291+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/2063192199 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706fc3c650 con 0x55706fba81b0 2024-09-21T06:25:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.291+0000 7f8ce8ff9640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8cd40b6680 0x7f8cd40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.291+0000 7f8cebfff640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8cd40b6680 0x7f8cd40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.292+0000 7f8ce8ff9640 1 -- 172.21.15.3:0/1472549703 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f8cdc0e7490 con 0x7f8cec104a30 2024-09-21T06:25:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.292+0000 7f8ceb7fe640 1 --2- 172.21.15.3:0/2063192199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 0x55706ff787b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0e4b60 tx=0x7f8cdc0edb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b9c22b360f6f8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.292+0000 7f8cd1ffb640 1 -- 172.21.15.3:0/2063192199 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc01d070 con 0x55706fba81b0 2024-09-21T06:25:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.292+0000 7f8cebfff640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8cd40b6680 0x7f8cd40b8b90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4005cc0 tx=0x7f8ce4005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.292+0000 7f8cd1ffb640 1 -- 172.21.15.3:0/2063192199 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cdc005290 con 0x55706fba81b0 2024-09-21T06:25:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.292+0000 7f8cd1ffb640 1 -- 172.21.15.3:0/2063192199 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc0a9190 con 0x55706fba81b0 2024-09-21T06:25:44.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.293+0000 7f8cf86d2740 1 -- 172.21.15.3:0/2063192199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 msgr2=0x55706ff787b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.293+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/2063192199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 0x55706ff787b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0e4b60 tx=0x7f8cdc0edb00 comp rx=0 tx=0).stop 2024-09-21T06:25:44.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.293+0000 7f8cf86d2740 1 -- 172.21.15.3:0/2063192199 shutdown_connections 2024-09-21T06:25:44.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.293+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/2063192199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 0x55706ff78cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.293+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/2063192199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 0x55706ff787b0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.293+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/2063192199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fbd1070 0x55706ff78270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.293+0000 7f8cf86d2740 1 -- 172.21.15.3:0/2063192199 >> 172.21.15.3:0/2063192199 conn(0x55706fbc5460 msgr2=0x55706ff76160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:44.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.293+0000 7f8cf86d2740 1 -- 172.21.15.3:0/2063192199 shutdown_connections 2024-09-21T06:25:44.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.293+0000 7f8cf86d2740 1 -- 172.21.15.3:0/2063192199 wait complete. 2024-09-21T06:25:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.293+0000 7f8cf86d2740 1 -- start start 2024-09-21T06:25:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cf86d2740 1 Processor -- start 2024-09-21T06:25:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cf86d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fba81b0 0x55706ff87d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cf86d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 0x55706ff88290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cf86d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 0x55706ff94110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cf86d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55706fd3f700 con 0x55706fbd1070 2024-09-21T06:25:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cf86d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55706fe79110 con 0x55706fba81b0 2024-09-21T06:25:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cf86d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55706ff8adb0 con 0x55706fbb0390 2024-09-21T06:25:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cf86d2740 10 client.0 _ll_register_callbacks cb 0x55706fe92990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 0x55706ff94110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 0x55706ff94110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37598/0 (socket says 172.21.15.3:37598) 2024-09-21T06:25:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cebfff640 1 -- 172.21.15.3:0/1401935656 learned_addr learned my addr 172.21.15.3:0/1401935656 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cebfff640 1 -- 172.21.15.3:0/1401935656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 msgr2=0x55706ff88290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cebfff640 1 --2- 172.21.15.3:0/1401935656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 0x55706ff88290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cebfff640 1 -- 172.21.15.3:0/1401935656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fba81b0 msgr2=0x55706ff87d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cebfff640 1 --2- 172.21.15.3:0/1401935656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fba81b0 0x55706ff87d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.294+0000 7f8cebfff640 1 -- 172.21.15.3:0/1401935656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706fea6c40 con 0x55706fbd1070 2024-09-21T06:25:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.295+0000 7f8cebfff640 1 --2- 172.21.15.3:0/1401935656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 0x55706ff94110 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4031680 tx=0x7f8ce402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbe78017078cb47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.295+0000 7f8ccaffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.295+0000 7f8ccaffd640 1 -- 172.21.15.3:0/1401935656 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4004040 con 0x55706fbd1070 2024-09-21T06:25:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.295+0000 7f8ccaffd640 1 -- 172.21.15.3:0/1401935656 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f8ce40041e0 con 0x55706fbd1070 2024-09-21T06:25:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.295+0000 7f8cf86d2740 1 -- 172.21.15.3:0/1401935656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55706fef0da0 con 0x55706fbd1070 2024-09-21T06:25:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.295+0000 7f8ccaffd640 1 -- 172.21.15.3:0/1401935656 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8ce40043e0 con 0x55706fbd1070 2024-09-21T06:25:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.295+0000 7f8cf86d2740 10 client.18372 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.295+0000 7f8cf86d2740 1 -- 172.21.15.3:0/1401935656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55706ff96b10 con 0x55706fbd1070 2024-09-21T06:25:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.295+0000 7f8ccaffd640 1 -- 172.21.15.3:0/1401935656 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55706fef0da0 con 0x55706fbd1070 2024-09-21T06:25:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.295+0000 7f8ccaffd640 1 -- 172.21.15.3:0/1401935656 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f8ce4051020 con 0x55706fbd1070 2024-09-21T06:25:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.296+0000 7f8cf86d2740 10 client.18372 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:44.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.296+0000 7f8cf86d2740 10 client.18372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:44.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.296+0000 7f8cf86d2740 20 client.18372 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:44.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.296+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/1401935656 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55706fb8f9f0 0x55706ff9a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.296+0000 7f8cf86d2740 4 client.18372 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:44.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.296+0000 7f8cf86d2740 1 -- 172.21.15.3:0/1401935656 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55706feec850 con 0x55706fb8f9f0 2024-09-21T06:25:44.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.296+0000 7f8ceb7fe640 1 --2- 172.21.15.3:0/1401935656 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55706fb8f9f0 0x55706ff9a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.296+0000 7f8ccaffd640 10 client.18372 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:44.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.296+0000 7f8ceb7fe640 1 --2- 172.21.15.3:0/1401935656 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55706fb8f9f0 0x55706ff9a0c0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bc6337dd3a6b757 server_cookie=566f9ec91a86205c in_seq=0 out_seq=0 2024-09-21T06:25:44.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.297+0000 7f8ccaffd640 1 -- 172.21.15.3:0/1401935656 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55706feec850 con 0x55706fb8f9f0 2024-09-21T06:25:44.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.297+0000 7f8ccaffd640 10 client.18372 handle_command_reply: tid=0 2024-09-21T06:25:44.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.297+0000 7f8cf86d2740 1 client.18372 shutdown 2024-09-21T06:25:44.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.297+0000 7f8cf86d2740 1 -- 172.21.15.3:0/1401935656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 msgr2=0x55706ff94110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.297+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/1401935656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 0x55706ff94110 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4031680 tx=0x7f8ce402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.297+0000 7f8cf86d2740 1 -- 172.21.15.3:0/1401935656 shutdown_connections 2024-09-21T06:25:44.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.297+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/1401935656 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55706fb8f9f0 0x55706ff9a0c0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.298+0000 7f8ccaffd640 0 client.18372 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:44.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.298+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/1401935656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 0x55706ff88290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.298+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/1401935656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 0x55706ff94110 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.298+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/1401935656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fba81b0 0x55706ff87d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.298+0000 7f8cf86d2740 1 -- 172.21.15.3:0/1401935656 >> 172.21.15.3:0/1401935656 conn(0x55706fbc5460 msgr2=0x55706ff74700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.298+0000 7f8cf86d2740 1 -- 172.21.15.3:0/1401935656 shutdown_connections 2024-09-21T06:25:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.298+0000 7f8cf86d2740 1 -- 172.21.15.3:0/1401935656 wait complete. 2024-09-21T06:25:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.298+0000 7f8cf86d2740 20 client.18372 trim_cache size 0 max 16384 2024-09-21T06:25:44.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.315+0000 7f8cf86d2740 1 Processor -- start 2024-09-21T06:25:44.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.315+0000 7f8cf86d2740 1 -- start start 2024-09-21T06:25:44.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.315+0000 7f8cf86d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fba81b0 0x55706ff88210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.315+0000 7f8cf86d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 0x55706ff74700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cf86d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 0x55706ff75aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cf86d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec1ad4c0 con 0x55706fbd1070 2024-09-21T06:25:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cf86d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec1af000 con 0x55706fba81b0 2024-09-21T06:25:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cf86d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec1adb90 con 0x55706fbb0390 2024-09-21T06:25:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 0x55706ff75aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 0x55706ff75aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37612/0 (socket says 172.21.15.3:37612) 2024-09-21T06:25:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cebfff640 1 -- 172.21.15.3:0/3253337522 learned_addr learned my addr 172.21.15.3:0/3253337522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8ceb7fe640 1 --2- 172.21.15.3:0/3253337522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fba81b0 0x55706ff88210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8ceaffd640 1 --2- 172.21.15.3:0/3253337522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 0x55706ff74700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cebfff640 1 -- 172.21.15.3:0/3253337522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 msgr2=0x55706ff74700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cebfff640 1 --2- 172.21.15.3:0/3253337522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 0x55706ff74700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cebfff640 1 -- 172.21.15.3:0/3253337522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fba81b0 msgr2=0x55706ff88210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cebfff640 1 --2- 172.21.15.3:0/3253337522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fba81b0 0x55706ff88210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cebfff640 1 -- 172.21.15.3:0/3253337522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706ff86cb0 con 0x55706fbd1070 2024-09-21T06:25:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8ceaffd640 1 --2- 172.21.15.3:0/3253337522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 0x55706ff74700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cebfff640 1 --2- 172.21.15.3:0/3253337522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 0x55706ff75aa0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f8ce402ff90 tx=0x7f8ce4003a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21b91c51793bc6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:44.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.316+0000 7f8cd17fa640 1 -- 172.21.15.3:0/3253337522 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4060070 con 0x55706fbd1070 2024-09-21T06:25:44.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.317+0000 7f8cd17fa640 1 -- 172.21.15.3:0/3253337522 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ce4036040 con 0x55706fbd1070 2024-09-21T06:25:44.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.317+0000 7f8cd17fa640 1 -- 172.21.15.3:0/3253337522 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce40306e0 con 0x55706fbd1070 2024-09-21T06:25:44.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.317+0000 7f8cf86d2740 1 -- 172.21.15.3:0/3253337522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 msgr2=0x55706ff75aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.317+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/3253337522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 0x55706ff75aa0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f8ce402ff90 tx=0x7f8ce4003a40 comp rx=0 tx=0).stop 2024-09-21T06:25:44.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.317+0000 7f8cf86d2740 1 -- 172.21.15.3:0/3253337522 shutdown_connections 2024-09-21T06:25:44.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.317+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/3253337522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbb0390 0x55706ff74700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.317+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/3253337522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fbd1070 0x55706ff75aa0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.317+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/3253337522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fba81b0 0x55706ff88210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.317+0000 7f8cf86d2740 1 -- 172.21.15.3:0/3253337522 >> 172.21.15.3:0/3253337522 conn(0x55706fbc5460 msgr2=0x55706ff87b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:44.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.317+0000 7f8cf86d2740 1 -- 172.21.15.3:0/3253337522 shutdown_connections 2024-09-21T06:25:44.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.318+0000 7f8cf86d2740 1 -- 172.21.15.3:0/3253337522 wait complete. 2024-09-21T06:25:44.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.318+0000 7f8cf86d2740 1 -- start start 2024-09-21T06:25:44.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.318+0000 7f8cf86d2740 1 Processor -- start 2024-09-21T06:25:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.318+0000 7f8cf86d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 0x55706ff90080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.318+0000 7f8cf86d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fbb0390 0x55706ff905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.318+0000 7f8cf86d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbd1070 0x55706ff8b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.318+0000 7f8cf86d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec1ae930 con 0x55706fba81b0 2024-09-21T06:25:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.318+0000 7f8cf86d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec1ae260 con 0x55706fbb0390 2024-09-21T06:25:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.318+0000 7f8cf86d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec1ab4c0 con 0x55706fbd1070 2024-09-21T06:25:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8cf86d2740 10 client.0 _ll_register_callbacks cb 0x7f8ce402f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8ceb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 0x55706ff90080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8ceb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 0x55706ff90080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37624/0 (socket says 172.21.15.3:37624) 2024-09-21T06:25:44.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/2594895674 learned_addr learned my addr 172.21.15.3:0/2594895674 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:44.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8ceaffd640 1 --2- 172.21.15.3:0/2594895674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fbb0390 0x55706ff905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8cebfff640 1 --2- 172.21.15.3:0/2594895674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbd1070 0x55706ff8b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/2594895674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbd1070 msgr2=0x55706ff8b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8ceb7fe640 1 --2- 172.21.15.3:0/2594895674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbd1070 0x55706ff8b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/2594895674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fbb0390 msgr2=0x55706ff905c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8ceb7fe640 1 --2- 172.21.15.3:0/2594895674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fbb0390 0x55706ff905c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/2594895674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706ff90b00 con 0x55706fba81b0 2024-09-21T06:25:44.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8ceb7fe640 1 --2- 172.21.15.3:0/2594895674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 0x55706ff90080 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0ed5c0 tx=0x7f8cdc0e7870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af1a9ea33cedd0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:44.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8ccaffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:44.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8ccaffd640 1 -- 172.21.15.3:0/2594895674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc0d4070 con 0x55706fba81b0 2024-09-21T06:25:44.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8ccaffd640 1 -- 172.21.15.3:0/2594895674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f8cdc0b6880 con 0x55706fba81b0 2024-09-21T06:25:44.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.319+0000 7f8cf86d2740 1 -- 172.21.15.3:0/2594895674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ce4036040 con 0x55706fba81b0 2024-09-21T06:25:44.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.320+0000 7f8ccaffd640 1 -- 172.21.15.3:0/2594895674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8cdc01d070 con 0x55706fba81b0 2024-09-21T06:25:44.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.320+0000 7f8cf86d2740 10 client.18378 fetch_fsmap learned FSMap version 38 2024-09-21T06:25:44.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.320+0000 7f8cf86d2740 1 -- 172.21.15.3:0/2594895674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557070004d50 con 0x55706fba81b0 2024-09-21T06:25:44.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.320+0000 7f8ccaffd640 1 -- 172.21.15.3:0/2594895674 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ce4036040 con 0x55706fba81b0 2024-09-21T06:25:44.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.320+0000 7f8ccaffd640 1 -- 172.21.15.3:0/2594895674 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f8ce4031d20 con 0x55706fba81b0 2024-09-21T06:25:44.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.320+0000 7f8cf86d2740 10 client.18378 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:25:44.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.320+0000 7f8cf86d2740 10 client.18378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:44.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.320+0000 7f8cf86d2740 20 client.18378 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:44.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.320+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/2594895674 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55706fb64da0 0x557070007040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.320+0000 7f8cf86d2740 4 client.18378 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:25:44.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.320+0000 7f8cf86d2740 1 -- 172.21.15.3:0/2594895674 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55706fee9c80 con 0x55706fb64da0 2024-09-21T06:25:44.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.320+0000 7f8ceaffd640 1 --2- 172.21.15.3:0/2594895674 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55706fb64da0 0x557070007040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.321+0000 7f8ccaffd640 10 client.18378 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.321+0000 7f8ceaffd640 1 --2- 172.21.15.3:0/2594895674 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55706fb64da0 0x557070007040 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d102963fbda59562 server_cookie=a7babd740aefd8c5 in_seq=0 out_seq=0 2024-09-21T06:25:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.321+0000 7f8ccaffd640 1 -- 172.21.15.3:0/2594895674 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x55706fee9c80 con 0x55706fb64da0 2024-09-21T06:25:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.321+0000 7f8ccaffd640 10 client.18378 handle_command_reply: tid=0 2024-09-21T06:25:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.321+0000 7f8cf86d2740 1 client.18378 shutdown 2024-09-21T06:25:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.321+0000 7f8cf86d2740 1 -- 172.21.15.3:0/2594895674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 msgr2=0x55706ff90080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.321+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/2594895674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 0x55706ff90080 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0ed5c0 tx=0x7f8cdc0e7870 comp rx=0 tx=0).stop 2024-09-21T06:25:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.321+0000 7f8cf86d2740 1 -- 172.21.15.3:0/2594895674 shutdown_connections 2024-09-21T06:25:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.321+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/2594895674 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x55706fb64da0 0x557070007040 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.321+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/2594895674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55706fbd1070 0x55706ff8b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/2594895674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706fbb0390 0x55706ff905c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf86d2740 1 --2- 172.21.15.3:0/2594895674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55706fba81b0 0x55706ff90080 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf86d2740 1 -- 172.21.15.3:0/2594895674 >> 172.21.15.3:0/2594895674 conn(0x55706fbc5460 msgr2=0x55706ff75890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:44.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8ccaffd640 0 client.18378 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:44.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf86d2740 1 -- 172.21.15.3:0/2594895674 shutdown_connections 2024-09-21T06:25:44.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf86d2740 1 -- 172.21.15.3:0/2594895674 wait complete. 2024-09-21T06:25:44.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf86d2740 20 client.18378 trim_cache size 0 max 16384 2024-09-21T06:25:44.332 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:25:44.332 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-21T06:25:44.332 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {}, 2024-09-21T06:25:44.332 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_stats": { 2024-09-21T06:25:44.332 INFO:teuthology.orchestra.run.smithi003.stdout: "9135cf69-938c-4d40-9a5c-95e31d48b371": { 2024-09-21T06:25:44.332 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:25:10.991220+0000", 2024-09-21T06:25:44.332 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.332 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.333 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.333 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.333 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.333 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.333 INFO:teuthology.orchestra.run.smithi003.stdout: "4957cde1-7a9e-430b-a809-39863daca10a": { 2024-09-21T06:25:44.333 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:15:18.019645+0000", 2024-09-21T06:25:44.333 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.333 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.333 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.333 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.334 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.334 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.334 INFO:teuthology.orchestra.run.smithi003.stdout: "c367f31f-a684-41c6-88ee-4b4e42f0d4e2": { 2024-09-21T06:25:44.334 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:04.443813+0000", 2024-09-21T06:25:44.334 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.334 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.334 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.334 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.335 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.335 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.335 INFO:teuthology.orchestra.run.smithi003.stdout: "adba1d94-e33f-4efd-a020-63ebe07f6992": { 2024-09-21T06:25:44.335 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:08.425600+0000", 2024-09-21T06:25:44.335 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.335 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.335 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.335 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.335 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.336 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.336 INFO:teuthology.orchestra.run.smithi003.stdout: "640ef6d2-da94-4f3b-b441-02e50117a2f6": { 2024-09-21T06:25:44.336 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:20.233531+0000", 2024-09-21T06:25:44.336 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.336 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.336 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.336 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.336 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.337 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.337 INFO:teuthology.orchestra.run.smithi003.stdout: "a95403a2-bc2a-4a17-b9c3-b4ab47fef5c9": { 2024-09-21T06:25:44.337 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:16:35.220263+0000", 2024-09-21T06:25:44.337 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.337 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.337 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.337 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.337 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.337 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.337 INFO:teuthology.orchestra.run.smithi003.stdout: "cbb5d4a7-b96d-4c2c-b7ab-7a3716c17f5b": { 2024-09-21T06:25:44.338 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:23:38.372765+0000", 2024-09-21T06:25:44.338 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.338 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.338 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.338 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.338 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.338 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.338 INFO:teuthology.orchestra.run.smithi003.stdout: "a2e8b9b7-c5fb-4d8f-b3c8-8258179296e9": { 2024-09-21T06:25:44.338 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:17:21.174442+0000", 2024-09-21T06:25:44.339 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.339 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.339 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.339 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.339 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.339 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.339 INFO:teuthology.orchestra.run.smithi003.stdout: "0bf99782-6ab2-4000-ba98-9ddb00a60402": { 2024-09-21T06:25:44.339 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:24:24.782991+0000", 2024-09-21T06:25:44.339 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.340 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.340 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.340 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.340 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.340 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.340 INFO:teuthology.orchestra.run.smithi003.stdout: "ef20c0c4-ebd9-4d7b-8b8b-bc06c2f43a58": { 2024-09-21T06:25:44.340 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:21:17.396219+0000", 2024-09-21T06:25:44.340 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.340 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.341 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.341 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.341 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.341 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.341 INFO:teuthology.orchestra.run.smithi003.stdout: "714b9934-a2d3-4a78-b185-01254776061b": { 2024-09-21T06:25:44.341 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:04.685380+0000", 2024-09-21T06:25:44.341 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.341 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.341 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.342 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.342 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.342 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.342 INFO:teuthology.orchestra.run.smithi003.stdout: "1296f5f3-c779-4eac-ba35-9ebe6d75c09a": { 2024-09-21T06:25:44.342 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:18:55.529762+0000", 2024-09-21T06:25:44.342 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.343 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.343 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.343 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.343 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.343 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.343 INFO:teuthology.orchestra.run.smithi003.stdout: "a4c8375d-619d-46de-94b8-b655e2a13724": { 2024-09-21T06:25:44.343 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:20:30.080068+0000", 2024-09-21T06:25:44.343 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.343 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.344 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.344 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.344 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.344 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.344 INFO:teuthology.orchestra.run.smithi003.stdout: "fab678ea-0ea8-42c2-b8b6-dd890880c82f": { 2024-09-21T06:25:44.344 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:19:43.103910+0000", 2024-09-21T06:25:44.344 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.344 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.344 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.344 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.345 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.345 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:44.345 INFO:teuthology.orchestra.run.smithi003.stdout: "535fb164-fd4f-4d88-b7e0-575559314385": { 2024-09-21T06:25:44.345 INFO:teuthology.orchestra.run.smithi003.stdout: "start_time": "2024-09-21T06:22:51.152572+0000", 2024-09-21T06:25:44.345 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-21T06:25:44.345 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:44.345 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_passed": 0, 2024-09-21T06:25:44.345 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_failed": 0, 2024-09-21T06:25:44.345 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_skipped": 0 2024-09-21T06:25:44.346 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:44.346 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:44.346 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:25:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf215e640 1 -- 172.21.15.3:0/1472549703 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8cd40b6680 msgr2=0x7f8cd40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf215e640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8cd40b6680 0x7f8cd40b8b90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4005cc0 tx=0x7f8ce4005c30 comp rx=0 tx=0).stop 2024-09-21T06:25:44.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf215e640 1 -- 172.21.15.3:0/1472549703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cec104a30 msgr2=0x7f8cec19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf215e640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cec104a30 0x7f8cec19a480 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00c890 tx=0x7f8cdc00cd50 comp rx=0 tx=0).stop 2024-09-21T06:25:44.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf215e640 1 -- 172.21.15.3:0/1472549703 shutdown_connections 2024-09-21T06:25:44.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf215e640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f8cd40b6680 0x7f8cd40b8b90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf215e640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cec10de90 0x7f8cec19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf215e640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cec105380 0x7f8cec19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf215e640 1 --2- 172.21.15.3:0/1472549703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cec104a30 0x7f8cec19a480 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf215e640 1 -- 172.21.15.3:0/1472549703 >> 172.21.15.3:0/1472549703 conn(0x7f8cec079770 msgr2=0x7f8cec107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:44.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.322+0000 7f8cf215e640 1 -- 172.21.15.3:0/1472549703 shutdown_connections 2024-09-21T06:25:44.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:44.323+0000 7f8cf215e640 1 -- 172.21.15.3:0/1472549703 wait complete. 2024-09-21T06:25:45.197 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:44 smithi137 ceph-mon[34464]: pgmap v657: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-21T06:25:45.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:44 smithi003 ceph-mon[31227]: pgmap v657: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-21T06:25:45.287 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:25:45.287 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:25:45.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:44 smithi204 ceph-mon[34659]: pgmap v657: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-21T06:25:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:46.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.052+0000 7f2eba6a4640 1 -- 172.21.15.3:0/795079598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2eb4103b60 msgr2=0x7f2eb4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:46.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.052+0000 7f2eba6a4640 1 --2- 172.21.15.3:0/795079598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2eb4103b60 0x7f2eb4103f70 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4009900 tx=0x7f2ea402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.052+0000 7f2eba6a4640 1 -- 172.21.15.3:0/795079598 shutdown_connections 2024-09-21T06:25:46.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.052+0000 7f2eba6a4640 1 --2- 172.21.15.3:0/795079598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eb4109df0 0x7f2eb4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.052+0000 7f2eba6a4640 1 --2- 172.21.15.3:0/795079598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2eb41094a0 0x7f2eb41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.052+0000 7f2eba6a4640 1 --2- 172.21.15.3:0/795079598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2eb4103b60 0x7f2eb4103f70 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.052+0000 7f2eba6a4640 1 -- 172.21.15.3:0/795079598 >> 172.21.15.3:0/795079598 conn(0x7f2eb407b850 msgr2=0x7f2eb407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:46.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.052+0000 7f2eba6a4640 1 -- 172.21.15.3:0/795079598 shutdown_connections 2024-09-21T06:25:46.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.052+0000 7f2eba6a4640 1 -- 172.21.15.3:0/795079598 wait complete. 2024-09-21T06:25:46.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.052+0000 7f2eba6a4640 1 Processor -- start 2024-09-21T06:25:46.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eba6a4640 1 -- start start 2024-09-21T06:25:46.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eba6a4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eb4103b60 0x7f2eb4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:46.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eba6a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2eb41094a0 0x7f2eb4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:46.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eba6a4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2eb4109df0 0x7f2eb4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:46.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eba6a4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4113ab0 con 0x7f2eb4109df0 2024-09-21T06:25:46.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eba6a4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4113930 con 0x7f2eb41094a0 2024-09-21T06:25:46.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eba6a4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4113c30 con 0x7f2eb4103b60 2024-09-21T06:25:46.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eb8c41640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2eb4109df0 0x7f2eb4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:46.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eb8c41640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2eb4109df0 0x7f2eb4076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40832/0 (socket says 172.21.15.3:40832) 2024-09-21T06:25:46.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eb8c41640 1 -- 172.21.15.3:0/513276486 learned_addr learned my addr 172.21.15.3:0/513276486 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:46.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eb37fe640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2eb41094a0 0x7f2eb4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:46.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eb3fff640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eb4103b60 0x7f2eb4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:46.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eb8c41640 1 -- 172.21.15.3:0/513276486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eb4103b60 msgr2=0x7f2eb4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:46.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eb8c41640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eb4103b60 0x7f2eb4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eb8c41640 1 -- 172.21.15.3:0/513276486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2eb41094a0 msgr2=0x7f2eb4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:46.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eb8c41640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2eb41094a0 0x7f2eb4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.053+0000 7f2eb8c41640 1 -- 172.21.15.3:0/513276486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb41a8790 con 0x7f2eb4109df0 2024-09-21T06:25:46.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.054+0000 7f2eb8c41640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2eb4109df0 0x7f2eb4076190 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00cce0 tx=0x7f2eac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:46.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.054+0000 7f2eb17fa640 1 -- 172.21.15.3:0/513276486 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac00f450 con 0x7f2eb4109df0 2024-09-21T06:25:46.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.054+0000 7f2eba6a4640 1 -- 172.21.15.3:0/513276486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb41a8a80 con 0x7f2eb4109df0 2024-09-21T06:25:46.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.054+0000 7f2eb17fa640 1 -- 172.21.15.3:0/513276486 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2eac00faa0 con 0x7f2eb4109df0 2024-09-21T06:25:46.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.054+0000 7f2eba6a4640 1 -- 172.21.15.3:0/513276486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb41a8fc0 con 0x7f2eb4109df0 2024-09-21T06:25:46.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.054+0000 7f2eb17fa640 1 -- 172.21.15.3:0/513276486 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac018b70 con 0x7f2eb4109df0 2024-09-21T06:25:46.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.055+0000 7f2eba6a4640 1 -- 172.21.15.3:0/513276486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb4103f70 con 0x7f2eb4109df0 2024-09-21T06:25:46.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.056+0000 7f2eb17fa640 1 -- 172.21.15.3:0/513276486 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f2eac018d10 con 0x7f2eb4109df0 2024-09-21T06:25:46.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.057+0000 7f2eb17fa640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e9c0b6680 0x7f2e9c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:46.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.057+0000 7f2eb3fff640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e9c0b6680 0x7f2e9c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:46.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.057+0000 7f2eb17fa640 1 -- 172.21.15.3:0/513276486 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f2eac0e7af0 con 0x7f2eb4109df0 2024-09-21T06:25:46.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.058+0000 7f2eb3fff640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e9c0b6680 0x7f2e9c0b8b90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4000c00 tx=0x7f2ea4031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:46.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.062+0000 7f2eb17fa640 1 -- 172.21.15.3:0/513276486 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2eac014030 con 0x7f2eb4109df0 2024-09-21T06:25:46.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.288+0000 7f2eba6a4640 1 -- 172.21.15.3:0/513276486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2eb402e640 con 0x7f2eb4109df0 2024-09-21T06:25:46.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.289+0000 7f2eb17fa640 1 -- 172.21.15.3:0/513276486 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11965 (secure 0 0 0) 0x7f2eac0ae050 con 0x7f2eb4109df0 2024-09-21T06:25:46.292 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:25:46.292 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":39,"btime":"2024-09-21T06:25:45:903545+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:25:45.739295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27344":{"gid":27344,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3457686047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3457686047},{"type":"v1","addr":"172.21.15.137:6835","nonce":3457686047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:25:46.292 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 39 2024-09-21T06:25:46.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.291+0000 7f2eba6a4640 1 -- 172.21.15.3:0/513276486 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e9c0b6680 msgr2=0x7f2e9c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:46.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.291+0000 7f2eba6a4640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e9c0b6680 0x7f2e9c0b8b90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4000c00 tx=0x7f2ea4031400 comp rx=0 tx=0).stop 2024-09-21T06:25:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.291+0000 7f2eba6a4640 1 -- 172.21.15.3:0/513276486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2eb4109df0 msgr2=0x7f2eb4076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.291+0000 7f2eba6a4640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2eb4109df0 0x7f2eb4076190 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00cce0 tx=0x7f2eac00a620 comp rx=0 tx=0).stop 2024-09-21T06:25:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.292+0000 7f2eba6a4640 1 -- 172.21.15.3:0/513276486 shutdown_connections 2024-09-21T06:25:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.292+0000 7f2eba6a4640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f2e9c0b6680 0x7f2e9c0b8b90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.292+0000 7f2eba6a4640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2eb4109df0 0x7f2eb4076190 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.292+0000 7f2eba6a4640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2eb41094a0 0x7f2eb4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.292+0000 7f2eba6a4640 1 --2- 172.21.15.3:0/513276486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eb4103b60 0x7f2eb4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.292+0000 7f2eba6a4640 1 -- 172.21.15.3:0/513276486 >> 172.21.15.3:0/513276486 conn(0x7f2eb407b850 msgr2=0x7f2eb41075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:46.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.292+0000 7f2eba6a4640 1 -- 172.21.15.3:0/513276486 shutdown_connections 2024-09-21T06:25:46.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:46.292+0000 7f2eba6a4640 1 -- 172.21.15.3:0/513276486 wait complete. 2024-09-21T06:25:47.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:46 smithi003 ceph-mon[31227]: pgmap v658: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-21T06:25:47.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:46 smithi003 ceph-mon[31227]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:25:47.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:46 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/513276486' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:25:47.037 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:25:47.206 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:46 smithi204 ceph-mon[34659]: pgmap v658: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-21T06:25:47.207 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:46 smithi204 ceph-mon[34659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:25:47.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:46 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/513276486' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:25:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:47.344 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:46 smithi137 ceph-mon[34464]: pgmap v658: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-21T06:25:47.344 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:46 smithi137 ceph-mon[34464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:25:47.344 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:46 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/513276486' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:25:47.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.772+0000 7fb1f5cbc640 1 -- 172.21.15.3:0/3597258679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f0102a60 msgr2=0x7fb1f0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:47.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.772+0000 7fb1f5cbc640 1 --2- 172.21.15.3:0/3597258679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f0102a60 0x7fb1f0102e70 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009900 tx=0x7fb1e002a410 comp rx=0 tx=0).stop 2024-09-21T06:25:47.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.772+0000 7fb1f5cbc640 1 -- 172.21.15.3:0/3597258679 shutdown_connections 2024-09-21T06:25:47.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.772+0000 7fb1f5cbc640 1 --2- 172.21.15.3:0/3597258679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f01083a0 0x7fb1f01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.772+0000 7fb1f5cbc640 1 --2- 172.21.15.3:0/3597258679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1f0108cf0 0x7fb1f010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.772+0000 7fb1f5cbc640 1 --2- 172.21.15.3:0/3597258679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f0102a60 0x7fb1f0102e70 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.772+0000 7fb1f5cbc640 1 -- 172.21.15.3:0/3597258679 >> 172.21.15.3:0/3597258679 conn(0x7fb1f0079730 msgr2=0x7fb1f0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:47.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.772+0000 7fb1f5cbc640 1 -- 172.21.15.3:0/3597258679 shutdown_connections 2024-09-21T06:25:47.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.772+0000 7fb1f5cbc640 1 -- 172.21.15.3:0/3597258679 wait complete. 2024-09-21T06:25:47.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.773+0000 7fb1f5cbc640 1 Processor -- start 2024-09-21T06:25:47.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.773+0000 7fb1f5cbc640 1 -- start start 2024-09-21T06:25:47.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.773+0000 7fb1f5cbc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f0102a60 0x7fb1f019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.773+0000 7fb1f5cbc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1f01083a0 0x7fb1f019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.773+0000 7fb1f5cbc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f0108cf0 0x7fb1f019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.773+0000 7fb1f5cbc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0112960 con 0x7fb1f0102a60 2024-09-21T06:25:47.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.773+0000 7fb1f5cbc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01127e0 con 0x7fb1f01083a0 2024-09-21T06:25:47.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.773+0000 7fb1f5cbc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0112ae0 con 0x7fb1f0108cf0 2024-09-21T06:25:47.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.773+0000 7fb1ef7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f0102a60 0x7fb1f019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.773+0000 7fb1ef7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f0102a60 0x7fb1f019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40850/0 (socket says 172.21.15.3:40850) 2024-09-21T06:25:47.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.773+0000 7fb1ef7fe640 1 -- 172.21.15.3:0/1879987408 learned_addr learned my addr 172.21.15.3:0/1879987408 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:47.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.774+0000 7fb1effff640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f0108cf0 0x7fb1f019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.774+0000 7fb1eeffd640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1f01083a0 0x7fb1f019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.774+0000 7fb1ef7fe640 1 -- 172.21.15.3:0/1879987408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f0108cf0 msgr2=0x7fb1f019bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:47.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.774+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f0108cf0 0x7fb1f019bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.774+0000 7fb1ef7fe640 1 -- 172.21.15.3:0/1879987408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1f01083a0 msgr2=0x7fb1f019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:47.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.774+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1f01083a0 0x7fb1f019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.774+0000 7fb1ef7fe640 1 -- 172.21.15.3:0/1879987408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f01a64d0 con 0x7fb1f0102a60 2024-09-21T06:25:47.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.774+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f0102a60 0x7fb1f019b2d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0005940 tx=0x7fb1e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:47.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.774+0000 7fb1ecff9640 1 -- 172.21.15.3:0/1879987408 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e003f030 con 0x7fb1f0102a60 2024-09-21T06:25:47.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.774+0000 7fb1ecff9640 1 -- 172.21.15.3:0/1879987408 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e0003cb0 con 0x7fb1f0102a60 2024-09-21T06:25:47.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.774+0000 7fb1f5cbc640 1 -- 172.21.15.3:0/1879987408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f01a6760 con 0x7fb1f0102a60 2024-09-21T06:25:47.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.774+0000 7fb1ecff9640 1 -- 172.21.15.3:0/1879987408 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0035070 con 0x7fb1f0102a60 2024-09-21T06:25:47.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.774+0000 7fb1f5cbc640 1 -- 172.21.15.3:0/1879987408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f01a6c40 con 0x7fb1f0102a60 2024-09-21T06:25:47.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.777+0000 7fb1ecff9640 1 -- 172.21.15.3:0/1879987408 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fb1e0042030 con 0x7fb1f0102a60 2024-09-21T06:25:47.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.777+0000 7fb1fc230740 1 Processor -- start 2024-09-21T06:25:47.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.777+0000 7fb1fc230740 1 -- start start 2024-09-21T06:25:47.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.777+0000 7fb1fc230740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bf8070 0x562b81f9f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.777+0000 7fb1fc230740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bcf1b0 0x562b81f9fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.777+0000 7fb1fc230740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bd7390 0x562b81fa0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.777+0000 7fb1fc230740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562b81f15ea0 con 0x562b81bd7390 2024-09-21T06:25:47.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.777+0000 7fb1fc230740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562b81f052d0 con 0x562b81bf8070 2024-09-21T06:25:47.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.777+0000 7fb1fc230740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562b81f090e0 con 0x562b81bcf1b0 2024-09-21T06:25:47.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.777+0000 7fb1eeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bd7390 0x562b81fa0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1eeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bd7390 0x562b81fa0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40866/0 (socket says 172.21.15.3:40866) 2024-09-21T06:25:47.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1eeffd640 1 -- 172.21.15.3:0/3732554260 learned_addr learned my addr 172.21.15.3:0/3732554260 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:47.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1effff640 1 --2- 172.21.15.3:0/3732554260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bf8070 0x562b81f9f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/3732554260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bcf1b0 0x562b81f9fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1ecff9640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb1dc0b66b0 0x7fb1dc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1eeffd640 1 -- 172.21.15.3:0/3732554260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bcf1b0 msgr2=0x562b81f9fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:47.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1eeffd640 1 --2- 172.21.15.3:0/3732554260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bcf1b0 0x562b81f9fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1eeffd640 1 -- 172.21.15.3:0/3732554260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bf8070 msgr2=0x562b81f9f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1eeffd640 1 --2- 172.21.15.3:0/3732554260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bf8070 0x562b81f9f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1eeffd640 1 -- 172.21.15.3:0/3732554260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b81c63650 con 0x562b81bd7390 2024-09-21T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1effff640 1 --2- 172.21.15.3:0/3732554260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bf8070 0x562b81f9f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1effff640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb1dc0b66b0 0x7fb1dc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1ecff9640 1 -- 172.21.15.3:0/1879987408 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7fb1e0107ce0 con 0x7fb1f0102a60 2024-09-21T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1eeffd640 1 --2- 172.21.15.3:0/3732554260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bd7390 0x562b81fa0120 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800bc50 tx=0x7fb1d800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414ee5c7c064cbc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.778+0000 7fb1e5ffb640 1 -- 172.21.15.3:0/3732554260 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8015070 con 0x562b81bd7390 2024-09-21T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.779+0000 7fb1e5ffb640 1 -- 172.21.15.3:0/3732554260 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1d801a910 con 0x562b81bd7390 2024-09-21T06:25:47.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.779+0000 7fb1e5ffb640 1 -- 172.21.15.3:0/3732554260 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d80118c0 con 0x562b81bd7390 2024-09-21T06:25:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.779+0000 7fb1fc230740 1 -- 172.21.15.3:0/3732554260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bd7390 msgr2=0x562b81fa0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.779+0000 7fb1effff640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb1dc0b66b0 0x7fb1dc0b8bc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8011580 tx=0x7fb1e8004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.779+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3732554260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bd7390 0x562b81fa0120 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800bc50 tx=0x7fb1d800f730 comp rx=0 tx=0).stop 2024-09-21T06:25:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.779+0000 7fb1fc230740 1 -- 172.21.15.3:0/3732554260 shutdown_connections 2024-09-21T06:25:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.779+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3732554260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bcf1b0 0x562b81f9fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.779+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3732554260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bd7390 0x562b81fa0120 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.779+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3732554260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bf8070 0x562b81f9f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.779+0000 7fb1fc230740 1 -- 172.21.15.3:0/3732554260 >> 172.21.15.3:0/3732554260 conn(0x562b81bec460 msgr2=0x562b81f9d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.780+0000 7fb1fc230740 1 -- 172.21.15.3:0/3732554260 shutdown_connections 2024-09-21T06:25:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.780+0000 7fb1fc230740 1 -- 172.21.15.3:0/3732554260 wait complete. 2024-09-21T06:25:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.780+0000 7fb1fc230740 1 -- start start 2024-09-21T06:25:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.780+0000 7fb1fc230740 1 Processor -- start 2024-09-21T06:25:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.780+0000 7fb1fc230740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 0x562b81faf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.780+0000 7fb1fc230740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bd7390 0x562b81faf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1fc230740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bf8070 0x562b81fbb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1fc230740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562b81d66700 con 0x562b81bcf1b0 2024-09-21T06:25:47.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1fc230740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562b81ea0160 con 0x562b81bd7390 2024-09-21T06:25:47.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1fc230740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562b81fb21b0 con 0x562b81bf8070 2024-09-21T06:25:47.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1fc230740 10 client.0 _ll_register_callbacks cb 0x562b81eb99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:47.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1ef7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 0x562b81faf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1eeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bd7390 0x562b81faf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1ef7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 0x562b81faf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40874/0 (socket says 172.21.15.3:40874) 2024-09-21T06:25:47.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1ef7fe640 1 -- 172.21.15.3:0/3716830316 learned_addr learned my addr 172.21.15.3:0/3716830316 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:47.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1effff640 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bf8070 0x562b81fbb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1eeffd640 1 -- 172.21.15.3:0/3716830316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bf8070 msgr2=0x562b81fbb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:47.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1eeffd640 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bf8070 0x562b81fbb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1eeffd640 1 -- 172.21.15.3:0/3716830316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 msgr2=0x562b81faf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:47.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1eeffd640 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 0x562b81faf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1eeffd640 1 -- 172.21.15.3:0/3716830316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b81ecdc90 con 0x562b81bd7390 2024-09-21T06:25:47.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 0x562b81faf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1effff640 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bf8070 0x562b81fbb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1eeffd640 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bd7390 0x562b81faf6a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800bc50 tx=0x7fb1d800bd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cf1720d2b29a8cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.781+0000 7fb1ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-21T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.782+0000 7fb1ceffd640 1 -- 172.21.15.3:0/3716830316 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8028060 con 0x562b81bd7390 2024-09-21T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.782+0000 7fb1fc230740 1 -- 172.21.15.3:0/3716830316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d801a910 con 0x562b81bd7390 2024-09-21T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.782+0000 7fb1ceffd640 1 -- 172.21.15.3:0/3716830316 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7fb1d801a670 con 0x562b81bd7390 2024-09-21T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.782+0000 7fb1ceffd640 1 -- 172.21.15.3:0/3716830316 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb1d8015070 con 0x562b81bd7390 2024-09-21T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.782+0000 7fb1fc230740 10 client.27878 fetch_fsmap learned FSMap version 39 2024-09-21T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.782+0000 7fb1fc230740 1 -- 172.21.15.3:0/3716830316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562b81fbdf00 con 0x562b81bd7390 2024-09-21T06:25:47.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.782+0000 7fb1ceffd640 1 -- 172.21.15.3:0/3716830316 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1d801a910 con 0x562b81bd7390 2024-09-21T06:25:47.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.783+0000 7fb1ceffd640 1 -- 172.21.15.3:0/3716830316 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb1d80046c0 con 0x562b81bd7390 2024-09-21T06:25:47.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.783+0000 7fb1fc230740 10 client.27878 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:25:47.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.783+0000 7fb1fc230740 10 client.27878 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:25:47.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.783+0000 7fb1fc230740 20 client.27878 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:47.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.783+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562b81bb69f0 0x562b81fc12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.783+0000 7fb1fc230740 4 client.27878 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:47.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.783+0000 7fb1fc230740 1 -- 172.21.15.3:0/3716830316 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b81f17df0 con 0x562b81bb69f0 2024-09-21T06:25:47.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.783+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562b81bb69f0 0x562b81fc12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.783+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562b81bb69f0 0x562b81fc12c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=94899a0ad99c8c16 server_cookie=fbb802afec94ab36 in_seq=0 out_seq=0 2024-09-21T06:25:47.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.783+0000 7fb1ceffd640 10 client.27878 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:25:47.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.784+0000 7fb1ceffd640 1 -- 172.21.15.3:0/3716830316 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562b81f17df0 con 0x562b81bb69f0 2024-09-21T06:25:47.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.784+0000 7fb1ceffd640 10 client.27878 handle_command_reply: tid=0 2024-09-21T06:25:47.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.784+0000 7fb1fc230740 1 client.27878 shutdown 2024-09-21T06:25:47.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.784+0000 7fb1fc230740 1 -- 172.21.15.3:0/3716830316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bd7390 msgr2=0x562b81faf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:47.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.784+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bd7390 0x562b81faf6a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800bc50 tx=0x7fb1d800bd40 comp rx=0 tx=0).stop 2024-09-21T06:25:47.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.784+0000 7fb1fc230740 1 -- 172.21.15.3:0/3716830316 shutdown_connections 2024-09-21T06:25:47.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.784+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562b81bb69f0 0x562b81fc12c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.784+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bf8070 0x562b81fbb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.784+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bd7390 0x562b81faf6a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.784+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3716830316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 0x562b81faf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.784+0000 7fb1fc230740 1 -- 172.21.15.3:0/3716830316 >> 172.21.15.3:0/3716830316 conn(0x562b81bec460 msgr2=0x562b81f9bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:47.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.784+0000 7fb1ceffd640 0 client.27878 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:25:47.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.785+0000 7fb1fc230740 1 -- 172.21.15.3:0/3716830316 shutdown_connections 2024-09-21T06:25:47.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.785+0000 7fb1fc230740 1 -- 172.21.15.3:0/3716830316 wait complete. 2024-09-21T06:25:47.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.785+0000 7fb1fc230740 20 client.27878 trim_cache size 0 max 16384 2024-09-21T06:25:47.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1fc230740 1 Processor -- start 2024-09-21T06:25:47.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1fc230740 1 -- start start 2024-09-21T06:25:47.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1fc230740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 0x562b81fb1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1fc230740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bd7390 0x562b81fa0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1fc230740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bf8070 0x562b81f9cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1fc230740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01ad3e0 con 0x562b81bcf1b0 2024-09-21T06:25:47.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1fc230740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01aef20 con 0x562b81bf8070 2024-09-21T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1fc230740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01adab0 con 0x562b81bd7390 2024-09-21T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1ef7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 0x562b81fb1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1ef7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 0x562b81fb1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40880/0 (socket says 172.21.15.3:40880) 2024-09-21T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1ef7fe640 1 -- 172.21.15.3:0/3098874483 learned_addr learned my addr 172.21.15.3:0/3098874483 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1effff640 1 --2- 172.21.15.3:0/3098874483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bf8070 0x562b81f9cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1eeffd640 1 --2- 172.21.15.3:0/3098874483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bd7390 0x562b81fa0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1ef7fe640 1 -- 172.21.15.3:0/3098874483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bd7390 msgr2=0x562b81fa0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/3098874483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bd7390 0x562b81fa0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1ef7fe640 1 -- 172.21.15.3:0/3098874483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bf8070 msgr2=0x562b81f9cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.798+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/3098874483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bf8070 0x562b81f9cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.799+0000 7fb1ef7fe640 1 -- 172.21.15.3:0/3098874483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b81fa0480 con 0x562b81bcf1b0 2024-09-21T06:25:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.799+0000 7fb1effff640 1 --2- 172.21.15.3:0/3098874483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bf8070 0x562b81f9cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.799+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/3098874483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 0x562b81fb1040 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00045e0 tx=0x7fb1e00f1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8dc256616fd1ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.799+0000 7fb1e57fa640 1 -- 172.21.15.3:0/3098874483 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0049040 con 0x562b81bcf1b0 2024-09-21T06:25:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.799+0000 7fb1e57fa640 1 -- 172.21.15.3:0/3098874483 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e00cd650 con 0x562b81bcf1b0 2024-09-21T06:25:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.799+0000 7fb1e57fa640 1 -- 172.21.15.3:0/3098874483 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e00f10b0 con 0x562b81bcf1b0 2024-09-21T06:25:47.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.800+0000 7fb1fc230740 1 -- 172.21.15.3:0/3098874483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 msgr2=0x562b81fb1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:47.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.800+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3098874483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 0x562b81fb1040 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00045e0 tx=0x7fb1e00f1040 comp rx=0 tx=0).stop 2024-09-21T06:25:47.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.800+0000 7fb1fc230740 1 -- 172.21.15.3:0/3098874483 shutdown_connections 2024-09-21T06:25:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.800+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3098874483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bd7390 0x562b81fa0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.800+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3098874483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bf8070 0x562b81f9cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.800+0000 7fb1fc230740 1 --2- 172.21.15.3:0/3098874483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bcf1b0 0x562b81fb1040 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.800+0000 7fb1fc230740 1 -- 172.21.15.3:0/3098874483 >> 172.21.15.3:0/3098874483 conn(0x562b81bec460 msgr2=0x562b81fa06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.800+0000 7fb1fc230740 1 -- 172.21.15.3:0/3098874483 shutdown_connections 2024-09-21T06:25:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.800+0000 7fb1fc230740 1 -- 172.21.15.3:0/3098874483 wait complete. 2024-09-21T06:25:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.800+0000 7fb1fc230740 1 -- start start 2024-09-21T06:25:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.800+0000 7fb1fc230740 1 Processor -- start 2024-09-21T06:25:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1fc230740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bcf1b0 0x562b81fc1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1fc230740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bd7390 0x562b81fc2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1fc230740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bf8070 0x562b82023a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1fc230740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01ae850 con 0x562b81bf8070 2024-09-21T06:25:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1fc230740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01ae180 con 0x562b81bcf1b0 2024-09-21T06:25:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1fc230740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01ab3e0 con 0x562b81bd7390 2024-09-21T06:25:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1fc230740 10 client.0 _ll_register_callbacks cb 0x7fb1e8007520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:47.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1effff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bf8070 0x562b82023a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1eeffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bd7390 0x562b81fc2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1effff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bf8070 0x562b82023a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40886/0 (socket says 172.21.15.3:40886) 2024-09-21T06:25:47.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1effff640 1 -- 172.21.15.3:0/1793368291 learned_addr learned my addr 172.21.15.3:0/1793368291 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:47.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/1793368291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bcf1b0 0x562b81fc1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1effff640 1 -- 172.21.15.3:0/1793368291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bd7390 msgr2=0x562b81fc2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:47.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1effff640 1 --2- 172.21.15.3:0/1793368291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bd7390 0x562b81fc2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1effff640 1 -- 172.21.15.3:0/1793368291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bcf1b0 msgr2=0x562b81fc1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:47.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1effff640 1 --2- 172.21.15.3:0/1793368291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bcf1b0 0x562b81fc1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1effff640 1 -- 172.21.15.3:0/1793368291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b820240e0 con 0x562b81bf8070 2024-09-21T06:25:47.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/1793368291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bcf1b0 0x562b81fc1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:47.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1effff640 1 --2- 172.21.15.3:0/1793368291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bf8070 0x562b82023a70 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb1e801f490 tx=0x7fb1e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84783e055280cfde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:47.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.801+0000 7fb1ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:47.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1ceffd640 1 -- 172.21.15.3:0/1793368291 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e801d660 con 0x562b81bf8070 2024-09-21T06:25:47.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1ceffd640 1 -- 172.21.15.3:0/1793368291 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7fb1e801d800 con 0x562b81bf8070 2024-09-21T06:25:47.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1fc230740 1 -- 172.21.15.3:0/1793368291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e00cd650 con 0x562b81bf8070 2024-09-21T06:25:47.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1ceffd640 1 -- 172.21.15.3:0/1793368291 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb1e801da00 con 0x562b81bf8070 2024-09-21T06:25:47.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1fc230740 10 client.18402 fetch_fsmap learned FSMap version 39 2024-09-21T06:25:47.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1fc230740 1 -- 172.21.15.3:0/1793368291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562b8202b480 con 0x562b81bf8070 2024-09-21T06:25:47.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1ceffd640 1 -- 172.21.15.3:0/1793368291 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e00cd650 con 0x562b81bf8070 2024-09-21T06:25:47.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1ceffd640 1 -- 172.21.15.3:0/1793368291 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb1e801b420 con 0x562b81bf8070 2024-09-21T06:25:47.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1fc230740 10 client.18402 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:25:47.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1fc230740 10 client.18402 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T06:25:47.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1fc230740 20 client.18402 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:47.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1fc230740 1 --2- 172.21.15.3:0/1793368291 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562b81b8bda0 0x562b8202dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:47.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1fc230740 4 client.18402 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:25:47.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.802+0000 7fb1fc230740 1 -- 172.21.15.3:0/1793368291 --> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b81f138a0 con 0x562b81b8bda0 2024-09-21T06:25:47.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.803+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/1793368291 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562b81b8bda0 0x562b8202dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:47.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.803+0000 7fb1ef7fe640 1 --2- 172.21.15.3:0/1793368291 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562b81b8bda0 0x562b8202dc90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=22f622df8eff1a44 server_cookie=bebdf4db5737eacb in_seq=0 out_seq=0 2024-09-21T06:25:47.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.803+0000 7fb1ceffd640 10 client.18402 ms_handle_connect on v2:172.21.15.204:6834/879721642 2024-09-21T06:25:47.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.803+0000 7fb1ceffd640 1 -- 172.21.15.3:0/1793368291 <== mds.4 v2:172.21.15.204:6834/879721642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b81f138a0 con 0x562b81b8bda0 2024-09-21T06:25:47.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.803+0000 7fb1ceffd640 10 client.18402 handle_command_reply: tid=0 2024-09-21T06:25:47.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.803+0000 7fb1fc230740 1 client.18402 shutdown 2024-09-21T06:25:47.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1fc230740 1 -- 172.21.15.3:0/1793368291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bf8070 msgr2=0x562b82023a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:47.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1fc230740 1 --2- 172.21.15.3:0/1793368291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bf8070 0x562b82023a70 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb1e801f490 tx=0x7fb1e8004ab0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1fc230740 1 -- 172.21.15.3:0/1793368291 shutdown_connections 2024-09-21T06:25:47.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1fc230740 1 --2- 172.21.15.3:0/1793368291 >> [v2:172.21.15.204:6834/879721642,v1:172.21.15.204:6835/879721642] conn(0x562b81b8bda0 0x562b8202dc90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1fc230740 1 --2- 172.21.15.3:0/1793368291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b81bd7390 0x562b81fc2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1fc230740 1 --2- 172.21.15.3:0/1793368291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b81bf8070 0x562b82023a70 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1ceffd640 0 client.18402 ms_handle_reset on v2:172.21.15.204:6834/879721642 2024-09-21T06:25:47.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1fc230740 1 --2- 172.21.15.3:0/1793368291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b81bcf1b0 0x562b81fc1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1fc230740 1 -- 172.21.15.3:0/1793368291 >> 172.21.15.3:0/1793368291 conn(0x562b81bec460 msgr2=0x562b81fafc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:47.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1fc230740 1 -- 172.21.15.3:0/1793368291 shutdown_connections 2024-09-21T06:25:47.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1fc230740 1 -- 172.21.15.3:0/1793368291 wait complete. 2024-09-21T06:25:47.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1fc230740 20 client.18402 trim_cache size 0 max 16384 2024-09-21T06:25:47.815 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:25:47.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1f5cbc640 1 -- 172.21.15.3:0/1879987408 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb1dc0b66b0 msgr2=0x7fb1dc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:47.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1f5cbc640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb1dc0b66b0 0x7fb1dc0b8bc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8011580 tx=0x7fb1e8004a40 comp rx=0 tx=0).stop 2024-09-21T06:25:47.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1f5cbc640 1 -- 172.21.15.3:0/1879987408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f0102a60 msgr2=0x7fb1f019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:47.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1f5cbc640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f0102a60 0x7fb1f019b2d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0005940 tx=0x7fb1e00042f0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1f5cbc640 1 -- 172.21.15.3:0/1879987408 shutdown_connections 2024-09-21T06:25:47.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.804+0000 7fb1f5cbc640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fb1dc0b66b0 0x7fb1dc0b8bc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.805+0000 7fb1f5cbc640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f0108cf0 0x7fb1f019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.805+0000 7fb1f5cbc640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1f01083a0 0x7fb1f019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.805+0000 7fb1f5cbc640 1 --2- 172.21.15.3:0/1879987408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f0102a60 0x7fb1f019b2d0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:47.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.805+0000 7fb1f5cbc640 1 -- 172.21.15.3:0/1879987408 >> 172.21.15.3:0/1879987408 conn(0x7fb1f0079730 msgr2=0x7fb1f0101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:47.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.805+0000 7fb1f5cbc640 1 -- 172.21.15.3:0/1879987408 shutdown_connections 2024-09-21T06:25:47.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:47.805+0000 7fb1f5cbc640 1 -- 172.21.15.3:0/1879987408 wait complete. 2024-09-21T06:25:48.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:25:48.854 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:49.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:48 smithi003 ceph-mon[31227]: pgmap v659: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-21T06:25:49.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:48 smithi137 ceph-mon[34464]: pgmap v659: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-21T06:25:49.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:48 smithi204 ceph-mon[34659]: pgmap v659: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-21T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.477+0000 7f1988011640 1 -- 172.21.15.3:0/2385158746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19801083c0 msgr2=0x7f19801087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.477+0000 7f1988011640 1 --2- 172.21.15.3:0/2385158746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19801083c0 0x7f19801087d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1970009900 tx=0x7f197002a410 comp rx=0 tx=0).stop 2024-09-21T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.477+0000 7f1988011640 1 -- 172.21.15.3:0/2385158746 shutdown_connections 2024-09-21T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.477+0000 7f1988011640 1 --2- 172.21.15.3:0/2385158746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1980108d10 0x7f198010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.477+0000 7f1988011640 1 --2- 172.21.15.3:0/2385158746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19801083c0 0x7f19801087d0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.477+0000 7f1988011640 1 --2- 172.21.15.3:0/2385158746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1980102a80 0x7f1980102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.477+0000 7f1988011640 1 -- 172.21.15.3:0/2385158746 >> 172.21.15.3:0/2385158746 conn(0x7f1980079740 msgr2=0x7f1980079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.477+0000 7f1988011640 1 -- 172.21.15.3:0/2385158746 shutdown_connections 2024-09-21T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.477+0000 7f1988011640 1 -- 172.21.15.3:0/2385158746 wait complete. 2024-09-21T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.478+0000 7f1988011640 1 Processor -- start 2024-09-21T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.478+0000 7f1988011640 1 -- start start 2024-09-21T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.478+0000 7f1988011640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1980102a80 0x7f198010fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.478+0000 7f1988011640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19801083c0 0x7f1980110310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.478+0000 7f1988011640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1980108d10 0x7f1980110850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.478+0000 7f1988011640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1980113a90 con 0x7f19801083c0 2024-09-21T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.478+0000 7f1988011640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1980113910 con 0x7f1980108d10 2024-09-21T06:25:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.478+0000 7f1988011640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1980113c10 con 0x7f1980102a80 2024-09-21T06:25:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.478+0000 7f19855ac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19801083c0 0x7f1980110310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.478+0000 7f19855ac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19801083c0 0x7f1980110310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40902/0 (socket says 172.21.15.3:40902) 2024-09-21T06:25:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.478+0000 7f19855ac640 1 -- 172.21.15.3:0/957719467 learned_addr learned my addr 172.21.15.3:0/957719467 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.478+0000 7f1985dad640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1980102a80 0x7f198010fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.479+0000 7f19855ac640 1 -- 172.21.15.3:0/957719467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1980102a80 msgr2=0x7f198010fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.479+0000 7f19855ac640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1980102a80 0x7f198010fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.479+0000 7f19855ac640 1 -- 172.21.15.3:0/957719467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1980108d10 msgr2=0x7f1980110850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.479+0000 7f19865ae640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1980108d10 0x7f1980110850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.479+0000 7f19855ac640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1980108d10 0x7f1980110850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.479+0000 7f19855ac640 1 -- 172.21.15.3:0/957719467 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19801a8800 con 0x7f19801083c0 2024-09-21T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.479+0000 7f19865ae640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1980108d10 0x7f1980110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.479+0000 7f19855ac640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19801083c0 0x7f1980110310 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1970032fd0 tx=0x7f1970004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.479+0000 7f1976ffd640 1 -- 172.21.15.3:0/957719467 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197003f030 con 0x7f19801083c0 2024-09-21T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.479+0000 7f1976ffd640 1 -- 172.21.15.3:0/957719467 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1970004490 con 0x7f19801083c0 2024-09-21T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.479+0000 7f1988011640 1 -- 172.21.15.3:0/957719467 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19801a8a90 con 0x7f19801083c0 2024-09-21T06:25:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.479+0000 7f1976ffd640 1 -- 172.21.15.3:0/957719467 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970035070 con 0x7f19801083c0 2024-09-21T06:25:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.479+0000 7f1988011640 1 -- 172.21.15.3:0/957719467 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19801a8f70 con 0x7f19801083c0 2024-09-21T06:25:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.482+0000 7f198e585740 1 Processor -- start 2024-09-21T06:25:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.482+0000 7f198e585740 1 -- start start 2024-09-21T06:25:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.482+0000 7f1976ffd640 1 -- 172.21.15.3:0/957719467 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f1970042030 con 0x7f19801083c0 2024-09-21T06:25:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.482+0000 7f198e585740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 0x55762a4066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.482+0000 7f198e585740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a0361b0 0x55762a406be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.482+0000 7f198e585740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 0x55762a407120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.482+0000 7f198e585740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55762a37cea0 con 0x55762a05f070 2024-09-21T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.482+0000 7f198e585740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55762a36c2d0 con 0x55762a03e390 2024-09-21T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.482+0000 7f198e585740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55762a3700e0 con 0x55762a0361b0 2024-09-21T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.482+0000 7f19865ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 0x55762a4066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.482+0000 7f19865ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 0x55762a4066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40908/0 (socket says 172.21.15.3:40908) 2024-09-21T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.482+0000 7f19865ae640 1 -- 172.21.15.3:0/2164341716 learned_addr learned my addr 172.21.15.3:0/2164341716 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.483+0000 7f19855ac640 1 --2- 172.21.15.3:0/2164341716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 0x55762a407120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.483+0000 7f1985dad640 1 --2- 172.21.15.3:0/2164341716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a0361b0 0x55762a406be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.483+0000 7f19865ae640 1 -- 172.21.15.3:0/2164341716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a0361b0 msgr2=0x55762a406be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.483+0000 7f19865ae640 1 --2- 172.21.15.3:0/2164341716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a0361b0 0x55762a406be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.483+0000 7f19865ae640 1 -- 172.21.15.3:0/2164341716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 msgr2=0x55762a407120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.483+0000 7f19865ae640 1 --2- 172.21.15.3:0/2164341716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 0x55762a407120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.483+0000 7f19865ae640 1 -- 172.21.15.3:0/2164341716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55762a0ca650 con 0x55762a05f070 2024-09-21T06:25:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.483+0000 7f1985dad640 1 --2- 172.21.15.3:0/2164341716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a0361b0 0x55762a406be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.483+0000 7f19865ae640 1 --2- 172.21.15.3:0/2164341716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 0x55762a4066a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f1978009ea0 tx=0x7f1978009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71f3d2c8b58d11a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.483+0000 7f196bfff640 1 -- 172.21.15.3:0/2164341716 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978016070 con 0x55762a05f070 2024-09-21T06:25:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.483+0000 7f196bfff640 1 -- 172.21.15.3:0/2164341716 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f197800ee80 con 0x55762a05f070 2024-09-21T06:25:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.483+0000 7f196bfff640 1 -- 172.21.15.3:0/2164341716 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19780116d0 con 0x55762a05f070 2024-09-21T06:25:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.483+0000 7f1976ffd640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f196c0b6680 0x7f196c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.484+0000 7f1985dad640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f196c0b6680 0x7f196c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.484+0000 7f1976ffd640 1 -- 172.21.15.3:0/957719467 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f1970107d40 con 0x7f19801083c0 2024-09-21T06:25:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.484+0000 7f198e585740 1 -- 172.21.15.3:0/2164341716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 msgr2=0x55762a4066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.484+0000 7f1985dad640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f196c0b6680 0x7f196c0b8b90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f197c005d00 tx=0x7f197c00b670 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.484+0000 7f198e585740 1 --2- 172.21.15.3:0/2164341716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 0x55762a4066a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f1978009ea0 tx=0x7f1978009f90 comp rx=0 tx=0).stop 2024-09-21T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.484+0000 7f198e585740 1 -- 172.21.15.3:0/2164341716 shutdown_connections 2024-09-21T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.484+0000 7f198e585740 1 --2- 172.21.15.3:0/2164341716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a0361b0 0x55762a406be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.484+0000 7f198e585740 1 --2- 172.21.15.3:0/2164341716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 0x55762a407120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.484+0000 7f198e585740 1 --2- 172.21.15.3:0/2164341716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 0x55762a4066a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.484+0000 7f198e585740 1 -- 172.21.15.3:0/2164341716 >> 172.21.15.3:0/2164341716 conn(0x55762a053460 msgr2=0x55762a404590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.484+0000 7f198e585740 1 -- 172.21.15.3:0/2164341716 shutdown_connections 2024-09-21T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.484+0000 7f198e585740 1 -- 172.21.15.3:0/2164341716 wait complete. 2024-09-21T06:25:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.485+0000 7f198e585740 1 -- start start 2024-09-21T06:25:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.485+0000 7f198e585740 1 Processor -- start 2024-09-21T06:25:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.485+0000 7f198e585740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a0361b0 0x55762a411d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.485+0000 7f198e585740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 0x55762a412280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.485+0000 7f198e585740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 0x55762a412800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.485+0000 7f198e585740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55762a1cd700 con 0x55762a05f070 2024-09-21T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.485+0000 7f198e585740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55762a307160 con 0x55762a03e390 2024-09-21T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.485+0000 7f198e585740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55762a414e80 con 0x55762a0361b0 2024-09-21T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f198e585740 10 client.0 _ll_register_callbacks cb 0x55762a3209e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f19865ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 0x55762a412800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f19865ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 0x55762a412800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40916/0 (socket says 172.21.15.3:40916) 2024-09-21T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f19865ae640 1 -- 172.21.15.3:0/792623008 learned_addr learned my addr 172.21.15.3:0/792623008 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f1985dad640 1 --2- 172.21.15.3:0/792623008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a0361b0 0x55762a411d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f19855ac640 1 --2- 172.21.15.3:0/792623008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 0x55762a412280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f19865ae640 1 -- 172.21.15.3:0/792623008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a0361b0 msgr2=0x55762a411d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f19865ae640 1 --2- 172.21.15.3:0/792623008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a0361b0 0x55762a411d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f19865ae640 1 -- 172.21.15.3:0/792623008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 msgr2=0x55762a412280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f19865ae640 1 --2- 172.21.15.3:0/792623008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 0x55762a412280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f19865ae640 1 -- 172.21.15.3:0/792623008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55762a334c90 con 0x55762a05f070 2024-09-21T06:25:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f19865ae640 1 --2- 172.21.15.3:0/792623008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 0x55762a412800 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1978005a00 tx=0x7f1978011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ae5f7d57462d29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f19697fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f19697fa640 1 -- 172.21.15.3:0/792623008 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978028060 con 0x55762a05f070 2024-09-21T06:25:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f19697fa640 1 -- 172.21.15.3:0/792623008 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f1978023410 con 0x55762a05f070 2024-09-21T06:25:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.486+0000 7f198e585740 1 -- 172.21.15.3:0/792623008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55762a37edf0 con 0x55762a05f070 2024-09-21T06:25:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.487+0000 7f19697fa640 1 -- 172.21.15.3:0/792623008 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1978016040 con 0x55762a05f070 2024-09-21T06:25:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.487+0000 7f198e585740 10 client.18414 fetch_fsmap learned FSMap version 39 2024-09-21T06:25:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.487+0000 7f198e585740 1 -- 172.21.15.3:0/792623008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55762a420ac0 con 0x55762a05f070 2024-09-21T06:25:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.487+0000 7f19697fa640 1 -- 172.21.15.3:0/792623008 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55762a37edf0 con 0x55762a05f070 2024-09-21T06:25:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.487+0000 7f19697fa640 1 -- 172.21.15.3:0/792623008 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f197800f040 con 0x55762a05f070 2024-09-21T06:25:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.487+0000 7f198e585740 10 client.18414 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:25:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.487+0000 7f198e585740 10 client.18414 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:25:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.487+0000 7f198e585740 20 client.18414 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.487+0000 7f198e585740 1 --2- 172.21.15.3:0/792623008 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55762a01d9f0 0x55762a423ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.487+0000 7f198e585740 4 client.18414 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.487+0000 7f198e585740 1 -- 172.21.15.3:0/792623008 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55762a37a8a0 con 0x55762a01d9f0 2024-09-21T06:25:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.487+0000 7f1985dad640 1 --2- 172.21.15.3:0/792623008 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55762a01d9f0 0x55762a423ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.488+0000 7f19697fa640 10 client.18414 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:25:49.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.488+0000 7f1985dad640 1 --2- 172.21.15.3:0/792623008 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55762a01d9f0 0x55762a423ed0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6dbf59b524394623 server_cookie=c329739920260de7 in_seq=0 out_seq=0 2024-09-21T06:25:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f19697fa640 1 -- 172.21.15.3:0/792623008 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55762a37a8a0 con 0x55762a01d9f0 2024-09-21T06:25:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f19697fa640 10 client.18414 handle_command_reply: tid=0 2024-09-21T06:25:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f198e585740 1 client.18414 shutdown 2024-09-21T06:25:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f198e585740 1 -- 172.21.15.3:0/792623008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 msgr2=0x55762a412800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f198e585740 1 --2- 172.21.15.3:0/792623008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 0x55762a412800 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1978005a00 tx=0x7f1978011ce0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f198e585740 1 -- 172.21.15.3:0/792623008 shutdown_connections 2024-09-21T06:25:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f198e585740 1 --2- 172.21.15.3:0/792623008 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55762a01d9f0 0x55762a423ed0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f198e585740 1 --2- 172.21.15.3:0/792623008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a05f070 0x55762a412800 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f198e585740 1 --2- 172.21.15.3:0/792623008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 0x55762a412280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f19697fa640 0 client.18414 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:25:49.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f198e585740 1 --2- 172.21.15.3:0/792623008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a0361b0 0x55762a411d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f198e585740 1 -- 172.21.15.3:0/792623008 >> 172.21.15.3:0/792623008 conn(0x55762a053460 msgr2=0x55762a409880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:49.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f198e585740 1 -- 172.21.15.3:0/792623008 shutdown_connections 2024-09-21T06:25:49.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f198e585740 1 -- 172.21.15.3:0/792623008 wait complete. 2024-09-21T06:25:49.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.489+0000 7f198e585740 20 client.18414 trim_cache size 0 max 16384 2024-09-21T06:25:49.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.502+0000 7f198e585740 1 Processor -- start 2024-09-21T06:25:49.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f198e585740 1 -- start start 2024-09-21T06:25:49.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f198e585740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a01d9f0 0x55762a413970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f198e585740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a0361b0 0x55762a40ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f198e585740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a03e390 0x55762a40f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f198e585740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f19801b19f0 con 0x55762a03e390 2024-09-21T06:25:49.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f198e585740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f19801b3530 con 0x55762a0361b0 2024-09-21T06:25:49.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f198e585740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19801b20c0 con 0x55762a01d9f0 2024-09-21T06:25:49.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f19865ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a03e390 0x55762a40f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f19865ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a03e390 0x55762a40f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40924/0 (socket says 172.21.15.3:40924) 2024-09-21T06:25:49.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f19865ae640 1 -- 172.21.15.3:0/1156610002 learned_addr learned my addr 172.21.15.3:0/1156610002 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:49.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f1985dad640 1 --2- 172.21.15.3:0/1156610002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a01d9f0 0x55762a413970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f19855ac640 1 --2- 172.21.15.3:0/1156610002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a0361b0 0x55762a40ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f19865ae640 1 -- 172.21.15.3:0/1156610002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a01d9f0 msgr2=0x55762a413970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:49.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f19865ae640 1 --2- 172.21.15.3:0/1156610002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a01d9f0 0x55762a413970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f19865ae640 1 -- 172.21.15.3:0/1156610002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a0361b0 msgr2=0x55762a40ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:49.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f19865ae640 1 --2- 172.21.15.3:0/1156610002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a0361b0 0x55762a40ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f19865ae640 1 -- 172.21.15.3:0/1156610002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55762a410240 con 0x55762a03e390 2024-09-21T06:25:49.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f19855ac640 1 --2- 172.21.15.3:0/1156610002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a0361b0 0x55762a40ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:49.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.503+0000 7f19865ae640 1 --2- 172.21.15.3:0/1156610002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a03e390 0x55762a40f470 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1978009ea0 tx=0x7f19780021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25df0c8115e04a6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:49.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.504+0000 7f196b7fe640 1 -- 172.21.15.3:0/1156610002 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978028050 con 0x55762a03e390 2024-09-21T06:25:49.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.504+0000 7f196b7fe640 1 -- 172.21.15.3:0/1156610002 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1978011e50 con 0x55762a03e390 2024-09-21T06:25:49.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.504+0000 7f196b7fe640 1 -- 172.21.15.3:0/1156610002 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197802c6b0 con 0x55762a03e390 2024-09-21T06:25:49.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.504+0000 7f198e585740 1 -- 172.21.15.3:0/1156610002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a03e390 msgr2=0x55762a40f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:49.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.504+0000 7f198e585740 1 --2- 172.21.15.3:0/1156610002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a03e390 0x55762a40f470 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1978009ea0 tx=0x7f19780021d0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.504+0000 7f198e585740 1 -- 172.21.15.3:0/1156610002 shutdown_connections 2024-09-21T06:25:49.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.504+0000 7f198e585740 1 --2- 172.21.15.3:0/1156610002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a03e390 0x55762a40f470 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.504+0000 7f198e585740 1 --2- 172.21.15.3:0/1156610002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a0361b0 0x55762a40ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.504+0000 7f198e585740 1 --2- 172.21.15.3:0/1156610002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a01d9f0 0x55762a413970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.504+0000 7f198e585740 1 -- 172.21.15.3:0/1156610002 >> 172.21.15.3:0/1156610002 conn(0x55762a053460 msgr2=0x55762a413260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:49.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.505+0000 7f198e585740 1 -- 172.21.15.3:0/1156610002 shutdown_connections 2024-09-21T06:25:49.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.505+0000 7f198e585740 1 -- 172.21.15.3:0/1156610002 wait complete. 2024-09-21T06:25:49.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.505+0000 7f198e585740 1 -- start start 2024-09-21T06:25:49.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.505+0000 7f198e585740 1 Processor -- start 2024-09-21T06:25:49.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.505+0000 7f198e585740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a01d9f0 0x55762a424e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.505+0000 7f198e585740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a0361b0 0x55762a425380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.505+0000 7f198e585740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 0x55762a4258c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f198e585740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f19801b2e60 con 0x55762a0361b0 2024-09-21T06:25:49.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f198e585740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f19801b2790 con 0x55762a03e390 2024-09-21T06:25:49.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f198e585740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19801af9f0 con 0x55762a01d9f0 2024-09-21T06:25:49.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f198e585740 10 client.0 _ll_register_callbacks cb 0x7f1978009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:49.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f19855ac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a0361b0 0x55762a425380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f19855ac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a0361b0 0x55762a425380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40936/0 (socket says 172.21.15.3:40936) 2024-09-21T06:25:49.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f19855ac640 1 -- 172.21.15.3:0/4135990745 learned_addr learned my addr 172.21.15.3:0/4135990745 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:49.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f1985dad640 1 --2- 172.21.15.3:0/4135990745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a01d9f0 0x55762a424e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f19865ae640 1 --2- 172.21.15.3:0/4135990745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 0x55762a4258c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f19855ac640 1 -- 172.21.15.3:0/4135990745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a01d9f0 msgr2=0x55762a424e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f19855ac640 1 --2- 172.21.15.3:0/4135990745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a01d9f0 0x55762a424e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f19855ac640 1 -- 172.21.15.3:0/4135990745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 msgr2=0x55762a4258c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f19855ac640 1 --2- 172.21.15.3:0/4135990745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 0x55762a4258c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f19855ac640 1 -- 172.21.15.3:0/4135990745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55762a48d650 con 0x55762a0361b0 2024-09-21T06:25:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f19855ac640 1 --2- 172.21.15.3:0/4135990745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a0361b0 0x55762a425380 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f1970031440 tx=0x7f1970031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f513967cddaf44a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f19697fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f19697fa640 1 -- 172.21.15.3:0/4135990745 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970049040 con 0x55762a0361b0 2024-09-21T06:25:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f19697fa640 1 -- 172.21.15.3:0/4135990745 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f19700f10b0 con 0x55762a0361b0 2024-09-21T06:25:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.506+0000 7f198e585740 1 -- 172.21.15.3:0/4135990745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1978011e50 con 0x55762a0361b0 2024-09-21T06:25:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.507+0000 7f19697fa640 1 -- 172.21.15.3:0/4135990745 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f19700cd4f0 con 0x55762a0361b0 2024-09-21T06:25:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.507+0000 7f198e585740 10 client.18420 fetch_fsmap learned FSMap version 39 2024-09-21T06:25:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.507+0000 7f198e585740 1 -- 172.21.15.3:0/4135990745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55762a48e780 con 0x55762a0361b0 2024-09-21T06:25:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.507+0000 7f19697fa640 1 -- 172.21.15.3:0/4135990745 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1978011e50 con 0x55762a0361b0 2024-09-21T06:25:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.507+0000 7f19697fa640 1 -- 172.21.15.3:0/4135990745 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f19700d7130 con 0x55762a0361b0 2024-09-21T06:25:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.507+0000 7f198e585740 10 client.18420 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:25:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.507+0000 7f198e585740 10 client.18420 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-21T06:25:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.507+0000 7f198e585740 20 client.18420 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.507+0000 7f198e585740 1 --2- 172.21.15.3:0/4135990745 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55762a05f070 0x55762a490f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.507+0000 7f198e585740 4 client.18420 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:25:49.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.507+0000 7f198e585740 1 -- 172.21.15.3:0/4135990745 --> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55762a377cd0 con 0x55762a05f070 2024-09-21T06:25:49.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.507+0000 7f1985dad640 1 --2- 172.21.15.3:0/4135990745 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55762a05f070 0x55762a490f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:49.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f19697fa640 10 client.18420 ms_handle_connect on v2:172.21.15.137:6836/44063020 2024-09-21T06:25:49.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f1985dad640 1 --2- 172.21.15.3:0/4135990745 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55762a05f070 0x55762a490f40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7348bd950c0bca91 server_cookie=bdea7b4ce2e2db41 in_seq=0 out_seq=0 2024-09-21T06:25:49.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f19697fa640 1 -- 172.21.15.3:0/4135990745 <== mds.3 v2:172.21.15.137:6836/44063020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55762a377cd0 con 0x55762a05f070 2024-09-21T06:25:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f19697fa640 10 client.18420 handle_command_reply: tid=0 2024-09-21T06:25:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f198e585740 1 client.18420 shutdown 2024-09-21T06:25:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f198e585740 1 -- 172.21.15.3:0/4135990745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a0361b0 msgr2=0x55762a425380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f198e585740 1 --2- 172.21.15.3:0/4135990745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a0361b0 0x55762a425380 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f1970031440 tx=0x7f1970031a10 comp rx=0 tx=0).stop 2024-09-21T06:25:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f198e585740 1 -- 172.21.15.3:0/4135990745 shutdown_connections 2024-09-21T06:25:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f198e585740 1 --2- 172.21.15.3:0/4135990745 >> [v2:172.21.15.137:6836/44063020,v1:172.21.15.137:6837/44063020] conn(0x55762a05f070 0x55762a490f40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f198e585740 1 --2- 172.21.15.3:0/4135990745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55762a03e390 0x55762a4258c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f198e585740 1 --2- 172.21.15.3:0/4135990745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55762a0361b0 0x55762a425380 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f19697fa640 0 client.18420 ms_handle_reset on v2:172.21.15.137:6836/44063020 2024-09-21T06:25:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f198e585740 1 --2- 172.21.15.3:0/4135990745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55762a01d9f0 0x55762a424e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f198e585740 1 -- 172.21.15.3:0/4135990745 >> 172.21.15.3:0/4135990745 conn(0x55762a053460 msgr2=0x55762a4097a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f198e585740 1 -- 172.21.15.3:0/4135990745 shutdown_connections 2024-09-21T06:25:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f198e585740 1 -- 172.21.15.3:0/4135990745 wait complete. 2024-09-21T06:25:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.508+0000 7f198e585740 20 client.18420 trim_cache size 0 max 16384 2024-09-21T06:25:49.521 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:25:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.509+0000 7f1988011640 1 -- 172.21.15.3:0/957719467 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f196c0b6680 msgr2=0x7f196c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.509+0000 7f1988011640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f196c0b6680 0x7f196c0b8b90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f197c005d00 tx=0x7f197c00b670 comp rx=0 tx=0).stop 2024-09-21T06:25:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.509+0000 7f1988011640 1 -- 172.21.15.3:0/957719467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19801083c0 msgr2=0x7f1980110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.509+0000 7f1988011640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19801083c0 0x7f1980110310 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1970032fd0 tx=0x7f1970004270 comp rx=0 tx=0).stop 2024-09-21T06:25:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.509+0000 7f1988011640 1 -- 172.21.15.3:0/957719467 shutdown_connections 2024-09-21T06:25:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.509+0000 7f1988011640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f196c0b6680 0x7f196c0b8b90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.509+0000 7f1988011640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1980108d10 0x7f1980110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.509+0000 7f1988011640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19801083c0 0x7f1980110310 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.509+0000 7f1988011640 1 --2- 172.21.15.3:0/957719467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1980102a80 0x7f198010fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.509+0000 7f1988011640 1 -- 172.21.15.3:0/957719467 >> 172.21.15.3:0/957719467 conn(0x7f1980079740 msgr2=0x7f19801044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.509+0000 7f1988011640 1 -- 172.21.15.3:0/957719467 shutdown_connections 2024-09-21T06:25:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:49.509+0000 7f1988011640 1 -- 172.21.15.3:0/957719467 wait complete. 2024-09-21T06:25:50.307 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:25:50.573 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:50.969 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:50 smithi003 ceph-mon[31227]: pgmap v660: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T06:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.061+0000 7fad6dc9d640 1 -- 172.21.15.3:0/3538210323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad68104a30 msgr2=0x7fad68104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.061+0000 7fad6dc9d640 1 --2- 172.21.15.3:0/3538210323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad68104a30 0x7fad68104e40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fad58009900 tx=0x7fad5802a400 comp rx=0 tx=0).stop 2024-09-21T06:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.061+0000 7fad6dc9d640 1 -- 172.21.15.3:0/3538210323 shutdown_connections 2024-09-21T06:25:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.061+0000 7fad6dc9d640 1 --2- 172.21.15.3:0/3538210323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6810de90 0x7fad68110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.061+0000 7fad6dc9d640 1 --2- 172.21.15.3:0/3538210323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad68105380 0x7fad6810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.061+0000 7fad6dc9d640 1 --2- 172.21.15.3:0/3538210323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad68104a30 0x7fad68104e40 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.061+0000 7fad6dc9d640 1 -- 172.21.15.3:0/3538210323 >> 172.21.15.3:0/3538210323 conn(0x7fad68079770 msgr2=0x7fad68079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.061+0000 7fad6dc9d640 1 -- 172.21.15.3:0/3538210323 shutdown_connections 2024-09-21T06:25:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.061+0000 7fad6dc9d640 1 -- 172.21.15.3:0/3538210323 wait complete. 2024-09-21T06:25:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.062+0000 7fad6dc9d640 1 Processor -- start 2024-09-21T06:25:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.062+0000 7fad6dc9d640 1 -- start start 2024-09-21T06:25:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad6dc9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad68104a30 0x7fad6819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad6dc9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad68105380 0x7fad6819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad6dc9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad6810de90 0x7fad6819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad6dc9d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fad68113b70 con 0x7fad6810de90 2024-09-21T06:25:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad6dc9d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fad681139f0 con 0x7fad68105380 2024-09-21T06:25:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad6dc9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad68113cf0 con 0x7fad68104a30 2024-09-21T06:25:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad67fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad6810de90 0x7fad6819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad67fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad6810de90 0x7fad6819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40960/0 (socket says 172.21.15.3:40960) 2024-09-21T06:25:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad677fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad68104a30 0x7fad6819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad67fff640 1 -- 172.21.15.3:0/127485386 learned_addr learned my addr 172.21.15.3:0/127485386 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad66ffd640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad68105380 0x7fad6819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad67fff640 1 -- 172.21.15.3:0/127485386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad68104a30 msgr2=0x7fad6819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad67fff640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad68104a30 0x7fad6819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad67fff640 1 -- 172.21.15.3:0/127485386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad68105380 msgr2=0x7fad6819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad67fff640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad68105380 0x7fad6819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad67fff640 1 -- 172.21.15.3:0/127485386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad681a0510 con 0x7fad6810de90 2024-09-21T06:25:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad66ffd640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad68105380 0x7fad6819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.063+0000 7fad67fff640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad6810de90 0x7fad6819fde0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fad6000b970 tx=0x7fad6000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.064+0000 7fad64ff9640 1 -- 172.21.15.3:0/127485386 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60019070 con 0x7fad6810de90 2024-09-21T06:25:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.064+0000 7fad64ff9640 1 -- 172.21.15.3:0/127485386 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad60014440 con 0x7fad6810de90 2024-09-21T06:25:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.064+0000 7fad6dc9d640 1 -- 172.21.15.3:0/127485386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad681a0800 con 0x7fad6810de90 2024-09-21T06:25:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.064+0000 7fad64ff9640 1 -- 172.21.15.3:0/127485386 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad600075d0 con 0x7fad6810de90 2024-09-21T06:25:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.064+0000 7fad6dc9d640 1 -- 172.21.15.3:0/127485386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad681a6f60 con 0x7fad6810de90 2024-09-21T06:25:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad74211740 1 Processor -- start 2024-09-21T06:25:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad64ff9640 1 -- 172.21.15.3:0/127485386 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fad6001c070 con 0x7fad6810de90 2024-09-21T06:25:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad74211740 1 -- start start 2024-09-21T06:25:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad74211740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc9746a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad74211740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc974be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad74211740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc975120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad74211740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e4fc8eaea0 con 0x55e4fc5a41b0 2024-09-21T06:25:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad74211740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e4fc8da2d0 con 0x55e4fc5ac390 2024-09-21T06:25:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad74211740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e4fc8de0e0 con 0x55e4fc5cd070 2024-09-21T06:25:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad677fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc974be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad677fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc974be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40962/0 (socket says 172.21.15.3:40962) 2024-09-21T06:25:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad677fe640 1 -- 172.21.15.3:0/1881693260 learned_addr learned my addr 172.21.15.3:0/1881693260 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad66ffd640 1 --2- 172.21.15.3:0/1881693260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc9746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.067+0000 7fad66ffd640 1 --2- 172.21.15.3:0/1881693260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc975120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.068+0000 7fad64ff9640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fad500b6680 0x7fad500b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.068+0000 7fad677fe640 1 -- 172.21.15.3:0/1881693260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 msgr2=0x55e4fc9746a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.068+0000 7fad677fe640 1 --2- 172.21.15.3:0/1881693260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc9746a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.068+0000 7fad677fe640 1 -- 172.21.15.3:0/1881693260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 msgr2=0x55e4fc975120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.068+0000 7fad677fe640 1 --2- 172.21.15.3:0/1881693260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc975120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.068+0000 7fad677fe640 1 -- 172.21.15.3:0/1881693260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4fc638650 con 0x55e4fc5a41b0 2024-09-21T06:25:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.068+0000 7fad66ffd640 1 --2- 172.21.15.3:0/1881693260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc975120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.068+0000 7fad67fff640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fad500b6680 0x7fad500b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.068+0000 7fad64ff9640 1 -- 172.21.15.3:0/127485386 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7fad600e6c10 con 0x7fad6810de90 2024-09-21T06:25:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.068+0000 7fad677fe640 1 --2- 172.21.15.3:0/1881693260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc974be0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fad58009900 tx=0x7fad58032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c025e4daf0b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.068+0000 7fad4dffb640 1 -- 172.21.15.3:0/1881693260 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58040070 con 0x55e4fc5a41b0 2024-09-21T06:25:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.068+0000 7fad4dffb640 1 -- 172.21.15.3:0/1881693260 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad580316a0 con 0x55e4fc5a41b0 2024-09-21T06:25:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.068+0000 7fad4dffb640 1 -- 172.21.15.3:0/1881693260 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58035040 con 0x55e4fc5a41b0 2024-09-21T06:25:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.069+0000 7fad74211740 1 -- 172.21.15.3:0/1881693260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 msgr2=0x55e4fc974be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.069+0000 7fad67fff640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fad500b6680 0x7fad500b8b90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fad600e4420 tx=0x7fad600d0040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.069+0000 7fad74211740 1 --2- 172.21.15.3:0/1881693260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc974be0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fad58009900 tx=0x7fad58032e60 comp rx=0 tx=0).stop 2024-09-21T06:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.069+0000 7fad74211740 1 -- 172.21.15.3:0/1881693260 shutdown_connections 2024-09-21T06:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.069+0000 7fad74211740 1 --2- 172.21.15.3:0/1881693260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc975120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.069+0000 7fad74211740 1 --2- 172.21.15.3:0/1881693260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc974be0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.069+0000 7fad74211740 1 --2- 172.21.15.3:0/1881693260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc9746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.069+0000 7fad74211740 1 -- 172.21.15.3:0/1881693260 >> 172.21.15.3:0/1881693260 conn(0x55e4fc5c1460 msgr2=0x55e4fc972590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.069+0000 7fad74211740 1 -- 172.21.15.3:0/1881693260 shutdown_connections 2024-09-21T06:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.069+0000 7fad74211740 1 -- 172.21.15.3:0/1881693260 wait complete. 2024-09-21T06:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.070+0000 7fad74211740 1 -- start start 2024-09-21T06:25:51.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.070+0000 7fad74211740 1 Processor -- start 2024-09-21T06:25:51.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.070+0000 7fad74211740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc97fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.070+0000 7fad74211740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc9802d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.070+0000 7fad74211740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc990550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.070+0000 7fad74211740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e4fc73b700 con 0x55e4fc5a41b0 2024-09-21T06:25:51.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.070+0000 7fad74211740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e4fc875160 con 0x55e4fc5ac390 2024-09-21T06:25:51.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.070+0000 7fad74211740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e4fc982e20 con 0x55e4fc5cd070 2024-09-21T06:25:51.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.070+0000 7fad66ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc97fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.070+0000 7fad74211740 10 client.0 _ll_register_callbacks cb 0x55e4fc88e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:51.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.070+0000 7fad66ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc97fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40978/0 (socket says 172.21.15.3:40978) 2024-09-21T06:25:51.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.070+0000 7fad66ffd640 1 -- 172.21.15.3:0/1037904168 learned_addr learned my addr 172.21.15.3:0/1037904168 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad677fe640 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc9802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad66ffd640 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc990550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad66ffd640 1 -- 172.21.15.3:0/1037904168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 msgr2=0x55e4fc990550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad66ffd640 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc990550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad66ffd640 1 -- 172.21.15.3:0/1037904168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 msgr2=0x55e4fc9802d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad66ffd640 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc9802d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad66ffd640 1 -- 172.21.15.3:0/1037904168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4fc8a2c90 con 0x55e4fc5a41b0 2024-09-21T06:25:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad677fe640 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc9802d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad66ffd640 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc97fd90 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00ce60 tx=0x7fad5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1f30b020f340f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad46ffd640 1 -- 172.21.15.3:0/1037904168 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c011e90 con 0x55e4fc5a41b0 2024-09-21T06:25:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad46ffd640 1 -- 172.21.15.3:0/1037904168 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7fad5c004020 con 0x55e4fc5a41b0 2024-09-21T06:25:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad74211740 1 -- 172.21.15.3:0/1037904168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad580316a0 con 0x55e4fc5a41b0 2024-09-21T06:25:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad46ffd640 1 -- 172.21.15.3:0/1037904168 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fad5c025050 con 0x55e4fc5a41b0 2024-09-21T06:25:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad74211740 10 client.18432 fetch_fsmap learned FSMap version 39 2024-09-21T06:25:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.071+0000 7fad74211740 1 -- 172.21.15.3:0/1037904168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e4fc992fa0 con 0x55e4fc5a41b0 2024-09-21T06:25:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.072+0000 7fad46ffd640 1 -- 172.21.15.3:0/1037904168 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad580316a0 con 0x55e4fc5a41b0 2024-09-21T06:25:51.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.072+0000 7fad46ffd640 1 -- 172.21.15.3:0/1037904168 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fad5c0035f0 con 0x55e4fc5a41b0 2024-09-21T06:25:51.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.072+0000 7fad74211740 10 client.18432 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:25:51.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.072+0000 7fad74211740 10 client.18432 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:25:51.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.072+0000 7fad74211740 20 client.18432 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:51.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.072+0000 7fad74211740 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55e4fc58b9f0 0x55e4fc996450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.072+0000 7fad74211740 4 client.18432 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:51.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.072+0000 7fad74211740 1 -- 172.21.15.3:0/1037904168 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4fc8ecdf0 con 0x55e4fc58b9f0 2024-09-21T06:25:51.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.072+0000 7fad677fe640 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55e4fc58b9f0 0x55e4fc996450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.073+0000 7fad677fe640 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55e4fc58b9f0 0x55e4fc996450 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa3ccac4d157e61c server_cookie=9e963298335117cc in_seq=0 out_seq=0 2024-09-21T06:25:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.073+0000 7fad46ffd640 10 client.18432 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:25:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad46ffd640 1 -- 172.21.15.3:0/1037904168 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e4fc8ecdf0 con 0x55e4fc58b9f0 2024-09-21T06:25:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad46ffd640 10 client.18432 handle_command_reply: tid=0 2024-09-21T06:25:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad74211740 1 client.18432 shutdown 2024-09-21T06:25:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad74211740 1 -- 172.21.15.3:0/1037904168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 msgr2=0x55e4fc97fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad74211740 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc97fd90 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00ce60 tx=0x7fad5c0046c0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad74211740 1 -- 172.21.15.3:0/1037904168 shutdown_connections 2024-09-21T06:25:51.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad74211740 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55e4fc58b9f0 0x55e4fc996450 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad74211740 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc990550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad74211740 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc9802d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad74211740 1 --2- 172.21.15.3:0/1037904168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc97fd90 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad74211740 1 -- 172.21.15.3:0/1037904168 >> 172.21.15.3:0/1037904168 conn(0x55e4fc5c1460 msgr2=0x55e4fc977830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:51.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad46ffd640 0 client.18432 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:25:51.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad74211740 1 -- 172.21.15.3:0/1037904168 shutdown_connections 2024-09-21T06:25:51.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad74211740 1 -- 172.21.15.3:0/1037904168 wait complete. 2024-09-21T06:25:51.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.074+0000 7fad74211740 20 client.18432 trim_cache size 0 max 16384 2024-09-21T06:25:51.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.087+0000 7fad74211740 1 Processor -- start 2024-09-21T06:25:51.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad74211740 1 -- start start 2024-09-21T06:25:51.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad74211740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc97ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad74211740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc9819c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad74211740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc971bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad74211740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fad681ad550 con 0x55e4fc5a41b0 2024-09-21T06:25:51.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad74211740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fad681af090 con 0x55e4fc5ac390 2024-09-21T06:25:51.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad74211740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad681adc20 con 0x55e4fc5cd070 2024-09-21T06:25:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad66ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc97ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad66ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc97ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40986/0 (socket says 172.21.15.3:40986) 2024-09-21T06:25:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad66ffd640 1 -- 172.21.15.3:0/3630822306 learned_addr learned my addr 172.21.15.3:0/3630822306 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad677fe640 1 --2- 172.21.15.3:0/3630822306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc9819c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad66ffd640 1 -- 172.21.15.3:0/3630822306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 msgr2=0x55e4fc971bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad66ffd640 1 --2- 172.21.15.3:0/3630822306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc971bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad66ffd640 1 -- 172.21.15.3:0/3630822306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 msgr2=0x55e4fc9819c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad66ffd640 1 --2- 172.21.15.3:0/3630822306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc9819c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad66ffd640 1 -- 172.21.15.3:0/3630822306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4fc97e290 con 0x55e4fc5a41b0 2024-09-21T06:25:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.088+0000 7fad66ffd640 1 --2- 172.21.15.3:0/3630822306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc97ec80 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0148a0 tx=0x7fad5c011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e16b0a82f10042 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.089+0000 7fad4d7fa640 1 -- 172.21.15.3:0/3630822306 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0035f0 con 0x55e4fc5a41b0 2024-09-21T06:25:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.089+0000 7fad4d7fa640 1 -- 172.21.15.3:0/3630822306 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad5c0044b0 con 0x55e4fc5a41b0 2024-09-21T06:25:51.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.089+0000 7fad4d7fa640 1 -- 172.21.15.3:0/3630822306 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c00f900 con 0x55e4fc5a41b0 2024-09-21T06:25:51.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.089+0000 7fad74211740 1 -- 172.21.15.3:0/3630822306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 msgr2=0x55e4fc97ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.089+0000 7fad74211740 1 --2- 172.21.15.3:0/3630822306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc97ec80 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0148a0 tx=0x7fad5c011d50 comp rx=0 tx=0).stop 2024-09-21T06:25:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.089+0000 7fad74211740 1 -- 172.21.15.3:0/3630822306 shutdown_connections 2024-09-21T06:25:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.089+0000 7fad74211740 1 --2- 172.21.15.3:0/3630822306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc971bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.090+0000 7fad74211740 1 --2- 172.21.15.3:0/3630822306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc9819c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.090+0000 7fad74211740 1 --2- 172.21.15.3:0/3630822306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc97ec80 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.090+0000 7fad74211740 1 -- 172.21.15.3:0/3630822306 >> 172.21.15.3:0/3630822306 conn(0x55e4fc5c1460 msgr2=0x55e4fc9812b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.090+0000 7fad74211740 1 -- 172.21.15.3:0/3630822306 shutdown_connections 2024-09-21T06:25:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.090+0000 7fad74211740 1 -- 172.21.15.3:0/3630822306 wait complete. 2024-09-21T06:25:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.090+0000 7fad74211740 1 -- start start 2024-09-21T06:25:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.090+0000 7fad74211740 1 Processor -- start 2024-09-21T06:25:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.090+0000 7fad74211740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc98cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.090+0000 7fad74211740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc988040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad74211740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc988580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad74211740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fad681ae9c0 con 0x55e4fc5a41b0 2024-09-21T06:25:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad74211740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fad681ae2f0 con 0x55e4fc5ac390 2024-09-21T06:25:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad74211740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad681ab550 con 0x55e4fc5cd070 2024-09-21T06:25:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad74211740 10 client.0 _ll_register_callbacks cb 0x7fad5c00c8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad66ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc98cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad66ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc98cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40992/0 (socket says 172.21.15.3:40992) 2024-09-21T06:25:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad66ffd640 1 -- 172.21.15.3:0/3030689416 learned_addr learned my addr 172.21.15.3:0/3030689416 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad677fe640 1 --2- 172.21.15.3:0/3030689416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc988040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad66ffd640 1 --2- 172.21.15.3:0/3030689416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc988580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad66ffd640 1 -- 172.21.15.3:0/3030689416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 msgr2=0x55e4fc988580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad66ffd640 1 --2- 172.21.15.3:0/3030689416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc988580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad66ffd640 1 -- 172.21.15.3:0/3030689416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 msgr2=0x55e4fc988040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad66ffd640 1 --2- 172.21.15.3:0/3030689416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc988040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad66ffd640 1 -- 172.21.15.3:0/3030689416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4fc988bc0 con 0x55e4fc5a41b0 2024-09-21T06:25:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad66ffd640 1 --2- 172.21.15.3:0/3030689416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc98cb90 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00ca00 tx=0x7fad5c003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a96a90fc814c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad46ffd640 1 -- 172.21.15.3:0/3030689416 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c003800 con 0x55e4fc5a41b0 2024-09-21T06:25:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad46ffd640 1 -- 172.21.15.3:0/3030689416 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7fad5c004200 con 0x55e4fc5a41b0 2024-09-21T06:25:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.091+0000 7fad74211740 1 -- 172.21.15.3:0/3030689416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad5c0044b0 con 0x55e4fc5a41b0 2024-09-21T06:25:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.092+0000 7fad46ffd640 1 -- 172.21.15.3:0/3030689416 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fad5c00f6a0 con 0x55e4fc5a41b0 2024-09-21T06:25:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.092+0000 7fad74211740 10 client.18438 fetch_fsmap learned FSMap version 39 2024-09-21T06:25:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.092+0000 7fad74211740 1 -- 172.21.15.3:0/3030689416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e4fca00c00 con 0x55e4fc5a41b0 2024-09-21T06:25:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.092+0000 7fad46ffd640 1 -- 172.21.15.3:0/3030689416 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad5c0044b0 con 0x55e4fc5a41b0 2024-09-21T06:25:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.092+0000 7fad46ffd640 1 -- 172.21.15.3:0/3030689416 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fad5c012860 con 0x55e4fc5a41b0 2024-09-21T06:25:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.092+0000 7fad74211740 10 client.18438 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:25:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.092+0000 7fad74211740 10 client.18438 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:25:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.092+0000 7fad74211740 20 client.18438 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.092+0000 7fad74211740 1 --2- 172.21.15.3:0/3030689416 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55e4fc560da0 0x55e4fca032d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.092+0000 7fad74211740 4 client.18438 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:25:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.092+0000 7fad74211740 1 -- 172.21.15.3:0/3030689416 --> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4fc8e88a0 con 0x55e4fc560da0 2024-09-21T06:25:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.092+0000 7fad677fe640 1 --2- 172.21.15.3:0/3030689416 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55e4fc560da0 0x55e4fca032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.093+0000 7fad677fe640 1 --2- 172.21.15.3:0/3030689416 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55e4fc560da0 0x55e4fca032d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=582693d038cbccca server_cookie=cb4f28d146c82072 in_seq=0 out_seq=0 2024-09-21T06:25:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.093+0000 7fad46ffd640 10 client.18438 ms_handle_connect on v2:172.21.15.204:6836/3211759973 2024-09-21T06:25:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.093+0000 7fad46ffd640 1 -- 172.21.15.3:0/3030689416 <== mds.2 v2:172.21.15.204:6836/3211759973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4fc8e88a0 con 0x55e4fc560da0 2024-09-21T06:25:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.093+0000 7fad46ffd640 10 client.18438 handle_command_reply: tid=0 2024-09-21T06:25:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.093+0000 7fad74211740 1 client.18438 shutdown 2024-09-21T06:25:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.094+0000 7fad74211740 1 -- 172.21.15.3:0/3030689416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 msgr2=0x55e4fc98cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.094+0000 7fad74211740 1 --2- 172.21.15.3:0/3030689416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc98cb90 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00ca00 tx=0x7fad5c003e60 comp rx=0 tx=0).stop 2024-09-21T06:25:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.094+0000 7fad74211740 1 -- 172.21.15.3:0/3030689416 shutdown_connections 2024-09-21T06:25:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.094+0000 7fad74211740 1 --2- 172.21.15.3:0/3030689416 >> [v2:172.21.15.204:6836/3211759973,v1:172.21.15.204:6837/3211759973] conn(0x55e4fc560da0 0x55e4fca032d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.094+0000 7fad74211740 1 --2- 172.21.15.3:0/3030689416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4fc5cd070 0x55e4fc988580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.094+0000 7fad74211740 1 --2- 172.21.15.3:0/3030689416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4fc5ac390 0x55e4fc988040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.094+0000 7fad74211740 1 --2- 172.21.15.3:0/3030689416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4fc5a41b0 0x55e4fc98cb90 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.094+0000 7fad74211740 1 -- 172.21.15.3:0/3030689416 >> 172.21.15.3:0/3030689416 conn(0x55e4fc5c1460 msgr2=0x55e4fc981b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.094+0000 7fad46ffd640 0 client.18438 ms_handle_reset on v2:172.21.15.204:6836/3211759973 2024-09-21T06:25:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.094+0000 7fad74211740 1 -- 172.21.15.3:0/3030689416 shutdown_connections 2024-09-21T06:25:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.094+0000 7fad74211740 1 -- 172.21.15.3:0/3030689416 wait complete. 2024-09-21T06:25:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.094+0000 7fad74211740 20 client.18438 trim_cache size 0 max 16384 2024-09-21T06:25:51.105 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:25:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.094+0000 7fad6dc9d640 1 -- 172.21.15.3:0/127485386 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fad500b6680 msgr2=0x7fad500b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.095+0000 7fad6dc9d640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fad500b6680 0x7fad500b8b90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fad600e4420 tx=0x7fad600d0040 comp rx=0 tx=0).stop 2024-09-21T06:25:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.095+0000 7fad6dc9d640 1 -- 172.21.15.3:0/127485386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad6810de90 msgr2=0x7fad6819fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.095+0000 7fad6dc9d640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad6810de90 0x7fad6819fde0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fad6000b970 tx=0x7fad6000be30 comp rx=0 tx=0).stop 2024-09-21T06:25:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.095+0000 7fad6dc9d640 1 -- 172.21.15.3:0/127485386 shutdown_connections 2024-09-21T06:25:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.095+0000 7fad6dc9d640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fad500b6680 0x7fad500b8b90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.095+0000 7fad6dc9d640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad6810de90 0x7fad6819fde0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.095+0000 7fad6dc9d640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad68105380 0x7fad6819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.095+0000 7fad6dc9d640 1 --2- 172.21.15.3:0/127485386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad68104a30 0x7fad6819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.095+0000 7fad6dc9d640 1 -- 172.21.15.3:0/127485386 >> 172.21.15.3:0/127485386 conn(0x7fad68079770 msgr2=0x7fad681079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.095+0000 7fad6dc9d640 1 -- 172.21.15.3:0/127485386 shutdown_connections 2024-09-21T06:25:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:51.095+0000 7fad6dc9d640 1 -- 172.21.15.3:0/127485386 wait complete. 2024-09-21T06:25:51.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:50 smithi137 ceph-mon[34464]: pgmap v660: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T06:25:51.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:50 smithi204 ceph-mon[34659]: pgmap v660: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T06:25:51.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:25:52.117 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.595+0000 7f0541f5a640 1 -- 172.21.15.3:0/2530654200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f053c102a80 msgr2=0x7f053c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.595+0000 7f0541f5a640 1 --2- 172.21.15.3:0/2530654200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f053c102a80 0x7f053c102e90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f052c009900 tx=0x7f052c02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.595+0000 7f0541f5a640 1 -- 172.21.15.3:0/2530654200 shutdown_connections 2024-09-21T06:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.595+0000 7f0541f5a640 1 --2- 172.21.15.3:0/2530654200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f053c108d10 0x7f053c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.595+0000 7f0541f5a640 1 --2- 172.21.15.3:0/2530654200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f053c1083c0 0x7f053c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.595+0000 7f0541f5a640 1 --2- 172.21.15.3:0/2530654200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f053c102a80 0x7f053c102e90 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.595+0000 7f0541f5a640 1 -- 172.21.15.3:0/2530654200 >> 172.21.15.3:0/2530654200 conn(0x7f053c079740 msgr2=0x7f053c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.595+0000 7f0541f5a640 1 -- 172.21.15.3:0/2530654200 shutdown_connections 2024-09-21T06:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.595+0000 7f0541f5a640 1 -- 172.21.15.3:0/2530654200 wait complete. 2024-09-21T06:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.596+0000 7f0541f5a640 1 Processor -- start 2024-09-21T06:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.596+0000 7f0541f5a640 1 -- start start 2024-09-21T06:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.596+0000 7f0541f5a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f053c102a80 0x7f053c19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.596+0000 7f0541f5a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f053c1083c0 0x7f053c19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.596+0000 7f0541f5a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f053c108d10 0x7f053c19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.596+0000 7f0541f5a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f053c112980 con 0x7f053c102a80 2024-09-21T06:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.596+0000 7f0541f5a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f053c112800 con 0x7f053c1083c0 2024-09-21T06:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.596+0000 7f0541f5a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f053c112b00 con 0x7f053c108d10 2024-09-21T06:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f053b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f053c102a80 0x7f053c19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f053b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f053c102a80 0x7f053c19b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41004/0 (socket says 172.21.15.3:41004) 2024-09-21T06:25:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f053b7fe640 1 -- 172.21.15.3:0/1811521669 learned_addr learned my addr 172.21.15.3:0/1811521669 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f0532ffd640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f053c1083c0 0x7f053c19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f053bfff640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f053c108d10 0x7f053c19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f053b7fe640 1 -- 172.21.15.3:0/1811521669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f053c108d10 msgr2=0x7f053c19bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f053b7fe640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f053c108d10 0x7f053c19bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f053b7fe640 1 -- 172.21.15.3:0/1811521669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f053c1083c0 msgr2=0x7f053c19b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:52.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f053b7fe640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f053c1083c0 0x7f053c19b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f053b7fe640 1 -- 172.21.15.3:0/1811521669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c1a6560 con 0x7f053c102a80 2024-09-21T06:25:52.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f053bfff640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f053c108d10 0x7f053c19bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:52.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f053b7fe640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f053c102a80 0x7f053c19b2d0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f052c005940 tx=0x7f052c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:52.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f05397fa640 1 -- 172.21.15.3:0/1811521669 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c03f030 con 0x7f053c102a80 2024-09-21T06:25:52.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f0541f5a640 1 -- 172.21.15.3:0/1811521669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053c1a67f0 con 0x7f053c102a80 2024-09-21T06:25:52.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f05397fa640 1 -- 172.21.15.3:0/1811521669 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f052c003aa0 con 0x7f053c102a80 2024-09-21T06:25:52.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.597+0000 7f05397fa640 1 -- 172.21.15.3:0/1811521669 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c035070 con 0x7f053c102a80 2024-09-21T06:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.598+0000 7f0541f5a640 1 -- 172.21.15.3:0/1811521669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053c1a6d20 con 0x7f053c102a80 2024-09-21T06:25:52.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.600+0000 7f05397fa640 1 -- 172.21.15.3:0/1811521669 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f052c042030 con 0x7f053c102a80 2024-09-21T06:25:52.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.600+0000 7f05484ce740 1 Processor -- start 2024-09-21T06:25:52.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.600+0000 7f05484ce740 1 -- start start 2024-09-21T06:25:52.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.601+0000 7f05484ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374921070 0x55a374cc86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.601+0000 7f05484ce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3748f81b0 0x55a374cc8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.601+0000 7f05484ce740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374900390 0x55a374cc9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.601+0000 7f05484ce740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a374c3eea0 con 0x55a374900390 2024-09-21T06:25:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.601+0000 7f05484ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a374c2e2d0 con 0x55a374921070 2024-09-21T06:25:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.601+0000 7f05484ce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a374c320e0 con 0x55a3748f81b0 2024-09-21T06:25:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.601+0000 7f0532ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374900390 0x55a374cc9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.601+0000 7f0532ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374900390 0x55a374cc9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41018/0 (socket says 172.21.15.3:41018) 2024-09-21T06:25:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.601+0000 7f0532ffd640 1 -- 172.21.15.3:0/3921316114 learned_addr learned my addr 172.21.15.3:0/3921316114 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.601+0000 7f05397fa640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f05280b66b0 0x7f05280b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.601+0000 7f053b7fe640 1 --2- 172.21.15.3:0/3921316114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3748f81b0 0x55a374cc8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.601+0000 7f053bfff640 1 --2- 172.21.15.3:0/3921316114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374921070 0x55a374cc86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.601+0000 7f053bfff640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f05280b66b0 0x7f05280b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.602+0000 7f0532ffd640 1 -- 172.21.15.3:0/3921316114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3748f81b0 msgr2=0x55a374cc8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.602+0000 7f0532ffd640 1 --2- 172.21.15.3:0/3921316114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3748f81b0 0x55a374cc8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.602+0000 7f0532ffd640 1 -- 172.21.15.3:0/3921316114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374921070 msgr2=0x55a374cc86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.602+0000 7f0532ffd640 1 --2- 172.21.15.3:0/3921316114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374921070 0x55a374cc86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.602+0000 7f0532ffd640 1 -- 172.21.15.3:0/3921316114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a37498c650 con 0x55a374900390 2024-09-21T06:25:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.602+0000 7f05397fa640 1 -- 172.21.15.3:0/1811521669 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f052c107dd0 con 0x7f053c102a80 2024-09-21T06:25:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.602+0000 7f0532ffd640 1 --2- 172.21.15.3:0/3921316114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374900390 0x55a374cc9120 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0524009c60 tx=0x7f0524011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b684659c824c1c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.602+0000 7f053bfff640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f05280b66b0 0x7f05280b8bc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0534005c40 tx=0x7f0534009280 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.602+0000 7f0531ffb640 1 -- 172.21.15.3:0/3921316114 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524004230 con 0x55a374900390 2024-09-21T06:25:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.602+0000 7f0531ffb640 1 -- 172.21.15.3:0/3921316114 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05240043d0 con 0x55a374900390 2024-09-21T06:25:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.602+0000 7f0531ffb640 1 -- 172.21.15.3:0/3921316114 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524004cd0 con 0x55a374900390 2024-09-21T06:25:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.603+0000 7f05484ce740 1 -- 172.21.15.3:0/3921316114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374900390 msgr2=0x55a374cc9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.603+0000 7f05484ce740 1 --2- 172.21.15.3:0/3921316114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374900390 0x55a374cc9120 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0524009c60 tx=0x7f0524011810 comp rx=0 tx=0).stop 2024-09-21T06:25:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.603+0000 7f05484ce740 1 -- 172.21.15.3:0/3921316114 shutdown_connections 2024-09-21T06:25:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.603+0000 7f05484ce740 1 --2- 172.21.15.3:0/3921316114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3748f81b0 0x55a374cc8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.603+0000 7f05484ce740 1 --2- 172.21.15.3:0/3921316114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374900390 0x55a374cc9120 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.603+0000 7f05484ce740 1 --2- 172.21.15.3:0/3921316114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374921070 0x55a374cc86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.603+0000 7f05484ce740 1 -- 172.21.15.3:0/3921316114 >> 172.21.15.3:0/3921316114 conn(0x55a374915460 msgr2=0x55a374cc6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.603+0000 7f05484ce740 1 -- 172.21.15.3:0/3921316114 shutdown_connections 2024-09-21T06:25:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.603+0000 7f05484ce740 1 -- 172.21.15.3:0/3921316114 wait complete. 2024-09-21T06:25:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.604+0000 7f05484ce740 1 -- start start 2024-09-21T06:25:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.604+0000 7f05484ce740 1 Processor -- start 2024-09-21T06:25:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.604+0000 7f05484ce740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 0x55a374cd3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.604+0000 7f05484ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374900390 0x55a374cd41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.604+0000 7f05484ce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374921070 0x55a374cd4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.604+0000 7f05484ce740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a374a8f700 con 0x55a3748f81b0 2024-09-21T06:25:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.604+0000 7f05484ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a374bc9160 con 0x55a374900390 2024-09-21T06:25:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.604+0000 7f05484ce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a374cd6e80 con 0x55a374921070 2024-09-21T06:25:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.604+0000 7f05484ce740 10 client.0 _ll_register_callbacks cb 0x55a374be29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.604+0000 7f053b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 0x55a374cd3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.604+0000 7f053b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 0x55a374cd3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41024/0 (socket says 172.21.15.3:41024) 2024-09-21T06:25:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.604+0000 7f053b7fe640 1 -- 172.21.15.3:0/887911076 learned_addr learned my addr 172.21.15.3:0/887911076 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f0532ffd640 1 --2- 172.21.15.3:0/887911076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374900390 0x55a374cd41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f053bfff640 1 --2- 172.21.15.3:0/887911076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374921070 0x55a374cd4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f053b7fe640 1 -- 172.21.15.3:0/887911076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374921070 msgr2=0x55a374cd4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f053b7fe640 1 --2- 172.21.15.3:0/887911076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374921070 0x55a374cd4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f053b7fe640 1 -- 172.21.15.3:0/887911076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374900390 msgr2=0x55a374cd41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f053b7fe640 1 --2- 172.21.15.3:0/887911076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374900390 0x55a374cd41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f053b7fe640 1 -- 172.21.15.3:0/887911076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a374bf6c90 con 0x55a3748f81b0 2024-09-21T06:25:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f053b7fe640 1 --2- 172.21.15.3:0/887911076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 0x55a374cd3c80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f052c105590 tx=0x7f052c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4856db36db49920d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f051affd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f051affd640 1 -- 172.21.15.3:0/887911076 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c049070 con 0x55a3748f81b0 2024-09-21T06:25:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f051affd640 1 -- 172.21.15.3:0/887911076 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f052c0f1110 con 0x55a3748f81b0 2024-09-21T06:25:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f05484ce740 1 -- 172.21.15.3:0/887911076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05240043d0 con 0x55a3748f81b0 2024-09-21T06:25:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f051affd640 1 -- 172.21.15.3:0/887911076 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f052c0cd580 con 0x55a3748f81b0 2024-09-21T06:25:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f05484ce740 10 client.18450 fetch_fsmap learned FSMap version 39 2024-09-21T06:25:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.605+0000 7f05484ce740 1 -- 172.21.15.3:0/887911076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a374ce2ab0 con 0x55a3748f81b0 2024-09-21T06:25:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.606+0000 7f051affd640 1 -- 172.21.15.3:0/887911076 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05240043d0 con 0x55a3748f81b0 2024-09-21T06:25:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.606+0000 7f051affd640 1 -- 172.21.15.3:0/887911076 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f052c0d21f0 con 0x55a3748f81b0 2024-09-21T06:25:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.606+0000 7f05484ce740 10 client.18450 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:25:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.606+0000 7f05484ce740 10 client.18450 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:25:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.606+0000 7f05484ce740 20 client.18450 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.606+0000 7f05484ce740 1 --2- 172.21.15.3:0/887911076 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55a3748df9f0 0x55a374ce6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.606+0000 7f05484ce740 4 client.18450 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.606+0000 7f05484ce740 1 -- 172.21.15.3:0/887911076 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a374c40df0 con 0x55a3748df9f0 2024-09-21T06:25:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.606+0000 7f0532ffd640 1 --2- 172.21.15.3:0/887911076 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55a3748df9f0 0x55a374ce6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.607+0000 7f051affd640 10 client.18450 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:25:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.607+0000 7f0532ffd640 1 --2- 172.21.15.3:0/887911076 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55a3748df9f0 0x55a374ce6040 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da0f798af3aeb358 server_cookie=e134e6e7ab76ee8f in_seq=0 out_seq=0 2024-09-21T06:25:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f051affd640 1 -- 172.21.15.3:0/887911076 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a374c40df0 con 0x55a3748df9f0 2024-09-21T06:25:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f051affd640 10 client.18450 handle_command_reply: tid=0 2024-09-21T06:25:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f05484ce740 1 client.18450 shutdown 2024-09-21T06:25:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f05484ce740 1 -- 172.21.15.3:0/887911076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 msgr2=0x55a374cd3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f05484ce740 1 --2- 172.21.15.3:0/887911076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 0x55a374cd3c80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f052c105590 tx=0x7f052c03d000 comp rx=0 tx=0).stop 2024-09-21T06:25:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f05484ce740 1 -- 172.21.15.3:0/887911076 shutdown_connections 2024-09-21T06:25:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f05484ce740 1 --2- 172.21.15.3:0/887911076 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55a3748df9f0 0x55a374ce6040 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f05484ce740 1 --2- 172.21.15.3:0/887911076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374921070 0x55a374cd4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f05484ce740 1 --2- 172.21.15.3:0/887911076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374900390 0x55a374cd41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f051affd640 0 client.18450 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:25:52.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f05484ce740 1 --2- 172.21.15.3:0/887911076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 0x55a374cd3c80 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f05484ce740 1 -- 172.21.15.3:0/887911076 >> 172.21.15.3:0/887911076 conn(0x55a374915460 msgr2=0x55a374ccb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:52.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f05484ce740 1 -- 172.21.15.3:0/887911076 shutdown_connections 2024-09-21T06:25:52.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f05484ce740 1 -- 172.21.15.3:0/887911076 wait complete. 2024-09-21T06:25:52.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.608+0000 7f05484ce740 20 client.18450 trim_cache size 0 max 16384 2024-09-21T06:25:52.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f05484ce740 1 Processor -- start 2024-09-21T06:25:52.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f05484ce740 1 -- start start 2024-09-21T06:25:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f05484ce740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 0x55a374cd58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f05484ce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374900390 0x55a374cd0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f05484ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374921070 0x55a374cd13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f05484ce740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f053c1af6c0 con 0x55a3748f81b0 2024-09-21T06:25:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f05484ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f053c1b1200 con 0x55a374921070 2024-09-21T06:25:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f05484ce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f053c1afd90 con 0x55a374900390 2024-09-21T06:25:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f053b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 0x55a374cd58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f053b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 0x55a374cd58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41034/0 (socket says 172.21.15.3:41034) 2024-09-21T06:25:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f053b7fe640 1 -- 172.21.15.3:0/2946951828 learned_addr learned my addr 172.21.15.3:0/2946951828 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f0532ffd640 1 --2- 172.21.15.3:0/2946951828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374900390 0x55a374cd0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f053b7fe640 1 -- 172.21.15.3:0/2946951828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374900390 msgr2=0x55a374cd0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f053bfff640 1 --2- 172.21.15.3:0/2946951828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374921070 0x55a374cd13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f053b7fe640 1 --2- 172.21.15.3:0/2946951828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374900390 0x55a374cd0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f053b7fe640 1 -- 172.21.15.3:0/2946951828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374921070 msgr2=0x55a374cd13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f053b7fe640 1 --2- 172.21.15.3:0/2946951828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374921070 0x55a374cd13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.622+0000 7f053b7fe640 1 -- 172.21.15.3:0/2946951828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a374cd2180 con 0x55a3748f81b0 2024-09-21T06:25:52.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.623+0000 7f053bfff640 1 --2- 172.21.15.3:0/2946951828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374921070 0x55a374cd13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:52.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.623+0000 7f053b7fe640 1 --2- 172.21.15.3:0/2946951828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 0x55a374cd58b0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f052c006010 tx=0x7f052c0ca280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb340c8bbfb06f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:52.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.623+0000 7f05317fa640 1 -- 172.21.15.3:0/2946951828 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c049070 con 0x55a3748f81b0 2024-09-21T06:25:52.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.623+0000 7f05317fa640 1 -- 172.21.15.3:0/2946951828 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f052c0cd740 con 0x55a3748f81b0 2024-09-21T06:25:52.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.623+0000 7f05317fa640 1 -- 172.21.15.3:0/2946951828 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c0b50b0 con 0x55a3748f81b0 2024-09-21T06:25:52.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.624+0000 7f05484ce740 1 -- 172.21.15.3:0/2946951828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 msgr2=0x55a374cd58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:52.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.624+0000 7f05484ce740 1 --2- 172.21.15.3:0/2946951828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 0x55a374cd58b0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f052c006010 tx=0x7f052c0ca280 comp rx=0 tx=0).stop 2024-09-21T06:25:52.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.624+0000 7f05484ce740 1 -- 172.21.15.3:0/2946951828 shutdown_connections 2024-09-21T06:25:52.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.624+0000 7f05484ce740 1 --2- 172.21.15.3:0/2946951828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374900390 0x55a374cd0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.624+0000 7f05484ce740 1 --2- 172.21.15.3:0/2946951828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a374921070 0x55a374cd13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.624+0000 7f05484ce740 1 --2- 172.21.15.3:0/2946951828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a3748f81b0 0x55a374cd58b0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.624+0000 7f05484ce740 1 -- 172.21.15.3:0/2946951828 >> 172.21.15.3:0/2946951828 conn(0x55a374915460 msgr2=0x55a374cd51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:52.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.624+0000 7f05484ce740 1 -- 172.21.15.3:0/2946951828 shutdown_connections 2024-09-21T06:25:52.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.624+0000 7f05484ce740 1 -- 172.21.15.3:0/2946951828 wait complete. 2024-09-21T06:25:52.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.624+0000 7f05484ce740 1 -- start start 2024-09-21T06:25:52.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.624+0000 7f05484ce740 1 Processor -- start 2024-09-21T06:25:52.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f05484ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3748f81b0 0x55a374cd82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f05484ce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374900390 0x55a374cd87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f05484ce740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374921070 0x55a374cd8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f05484ce740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f053c1b0b30 con 0x55a374921070 2024-09-21T06:25:52.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f05484ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f053c1b0460 con 0x55a3748f81b0 2024-09-21T06:25:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f05484ce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f053c1ad6c0 con 0x55a374900390 2024-09-21T06:25:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f05484ce740 10 client.0 _ll_register_callbacks cb 0x7f052c1059e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f053bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374921070 0x55a374cd8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f053bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374921070 0x55a374cd8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41036/0 (socket says 172.21.15.3:41036) 2024-09-21T06:25:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f053bfff640 1 -- 172.21.15.3:0/989842680 learned_addr learned my addr 172.21.15.3:0/989842680 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f0532ffd640 1 --2- 172.21.15.3:0/989842680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374900390 0x55a374cd87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f053b7fe640 1 --2- 172.21.15.3:0/989842680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3748f81b0 0x55a374cd82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f053bfff640 1 -- 172.21.15.3:0/989842680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374900390 msgr2=0x55a374cd87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:52.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f053bfff640 1 --2- 172.21.15.3:0/989842680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374900390 0x55a374cd87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f053bfff640 1 -- 172.21.15.3:0/989842680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3748f81b0 msgr2=0x55a374cd82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:52.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f053bfff640 1 --2- 172.21.15.3:0/989842680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3748f81b0 0x55a374cd82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f053bfff640 1 -- 172.21.15.3:0/989842680 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a374d4fc00 con 0x55a374921070 2024-09-21T06:25:52.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f053bfff640 1 --2- 172.21.15.3:0/989842680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374921070 0x55a374cd8d20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f05340129c0 tx=0x7f0534004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa65a4b8c4aec2ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:52.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.625+0000 7f051affd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:52.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f051affd640 1 -- 172.21.15.3:0/989842680 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053402f060 con 0x55a374921070 2024-09-21T06:25:52.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f051affd640 1 -- 172.21.15.3:0/989842680 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f0534020470 con 0x55a374921070 2024-09-21T06:25:52.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f05484ce740 1 -- 172.21.15.3:0/989842680 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f052c0cd740 con 0x55a374921070 2024-09-21T06:25:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f051affd640 1 -- 172.21.15.3:0/989842680 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0534022e30 con 0x55a374921070 2024-09-21T06:25:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f05484ce740 10 client.18456 fetch_fsmap learned FSMap version 39 2024-09-21T06:25:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f05484ce740 1 -- 172.21.15.3:0/989842680 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a374d50e50 con 0x55a374921070 2024-09-21T06:25:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f051affd640 1 -- 172.21.15.3:0/989842680 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f052c0cd740 con 0x55a374921070 2024-09-21T06:25:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f051affd640 1 -- 172.21.15.3:0/989842680 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f0534029030 con 0x55a374921070 2024-09-21T06:25:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f05484ce740 10 client.18456 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:25:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f05484ce740 10 client.18456 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:25:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f05484ce740 20 client.18456 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f05484ce740 1 --2- 172.21.15.3:0/989842680 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55a3748b4da0 0x55a374d535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:52.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f05484ce740 4 client.18456 mds_command: new command op to 24691 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:25:52.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.626+0000 7f05484ce740 1 -- 172.21.15.3:0/989842680 --> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a374c3c8a0 con 0x55a3748b4da0 2024-09-21T06:25:52.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.627+0000 7f053b7fe640 1 --2- 172.21.15.3:0/989842680 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55a3748b4da0 0x55a374d535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:52.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.627+0000 7f053b7fe640 1 --2- 172.21.15.3:0/989842680 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55a3748b4da0 0x55a374d535e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6290708c675a9ed server_cookie=d494326d48aad8e in_seq=0 out_seq=0 2024-09-21T06:25:52.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.627+0000 7f051affd640 10 client.18456 ms_handle_connect on v2:172.21.15.204:6838/2228257050 2024-09-21T06:25:52.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.627+0000 7f051affd640 1 -- 172.21.15.3:0/989842680 <== mds.1 v2:172.21.15.204:6838/2228257050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a374c3c8a0 con 0x55a3748b4da0 2024-09-21T06:25:52.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.627+0000 7f051affd640 10 client.18456 handle_command_reply: tid=0 2024-09-21T06:25:52.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.628+0000 7f05484ce740 1 client.18456 shutdown 2024-09-21T06:25:52.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.628+0000 7f05484ce740 1 -- 172.21.15.3:0/989842680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374921070 msgr2=0x55a374cd8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:52.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.628+0000 7f05484ce740 1 --2- 172.21.15.3:0/989842680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374921070 0x55a374cd8d20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f05340129c0 tx=0x7f0534004a40 comp rx=0 tx=0).stop 2024-09-21T06:25:52.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.628+0000 7f05484ce740 1 -- 172.21.15.3:0/989842680 shutdown_connections 2024-09-21T06:25:52.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.628+0000 7f05484ce740 1 --2- 172.21.15.3:0/989842680 >> [v2:172.21.15.204:6838/2228257050,v1:172.21.15.204:6839/2228257050] conn(0x55a3748b4da0 0x55a374d535e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.628+0000 7f05484ce740 1 --2- 172.21.15.3:0/989842680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a374900390 0x55a374cd87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.628+0000 7f05484ce740 1 --2- 172.21.15.3:0/989842680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a374921070 0x55a374cd8d20 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.628+0000 7f05484ce740 1 --2- 172.21.15.3:0/989842680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3748f81b0 0x55a374cd82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.628+0000 7f051affd640 0 client.18456 ms_handle_reset on v2:172.21.15.204:6838/2228257050 2024-09-21T06:25:52.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.628+0000 7f05484ce740 1 -- 172.21.15.3:0/989842680 >> 172.21.15.3:0/989842680 conn(0x55a374915460 msgr2=0x55a374cc9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:52.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.628+0000 7f05484ce740 1 -- 172.21.15.3:0/989842680 shutdown_connections 2024-09-21T06:25:52.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.628+0000 7f05484ce740 1 -- 172.21.15.3:0/989842680 wait complete. 2024-09-21T06:25:52.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.628+0000 7f05484ce740 20 client.18456 trim_cache size 0 max 16384 2024-09-21T06:25:52.638 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:25:52.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.629+0000 7f0541f5a640 1 -- 172.21.15.3:0/1811521669 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f05280b66b0 msgr2=0x7f05280b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:52.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.629+0000 7f0541f5a640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f05280b66b0 0x7f05280b8bc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0534005c40 tx=0x7f0534009280 comp rx=0 tx=0).stop 2024-09-21T06:25:52.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.629+0000 7f0541f5a640 1 -- 172.21.15.3:0/1811521669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f053c102a80 msgr2=0x7f053c19b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:52.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.629+0000 7f0541f5a640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f053c102a80 0x7f053c19b2d0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f052c005940 tx=0x7f052c004270 comp rx=0 tx=0).stop 2024-09-21T06:25:52.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.629+0000 7f0541f5a640 1 -- 172.21.15.3:0/1811521669 shutdown_connections 2024-09-21T06:25:52.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.629+0000 7f0541f5a640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f05280b66b0 0x7f05280b8bc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.629+0000 7f0541f5a640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f053c108d10 0x7f053c19bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.629+0000 7f0541f5a640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f053c1083c0 0x7f053c19b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.629+0000 7f0541f5a640 1 --2- 172.21.15.3:0/1811521669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f053c102a80 0x7f053c19b2d0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.629+0000 7f0541f5a640 1 -- 172.21.15.3:0/1811521669 >> 172.21.15.3:0/1811521669 conn(0x7f053c079740 msgr2=0x7f053c1014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.629+0000 7f0541f5a640 1 -- 172.21.15.3:0/1811521669 shutdown_connections 2024-09-21T06:25:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:52.629+0000 7f0541f5a640 1 -- 172.21.15.3:0/1811521669 wait complete. 2024-09-21T06:25:53.283 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:52 smithi003 ceph-mon[31227]: pgmap v661: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T06:25:53.313 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:25:53.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:52 smithi137 ceph-mon[34464]: pgmap v661: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T06:25:53.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:52 smithi204 ceph-mon[34659]: pgmap v661: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T06:25:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.063+0000 7fcd03de8640 1 -- 172.21.15.3:0/2533075756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccfc109df0 msgr2=0x7fccfc110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.063+0000 7fcd03de8640 1 --2- 172.21.15.3:0/2533075756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccfc109df0 0x7fccfc110620 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009900 tx=0x7fccf402a410 comp rx=0 tx=0).stop 2024-09-21T06:25:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.063+0000 7fcd03de8640 1 -- 172.21.15.3:0/2533075756 shutdown_connections 2024-09-21T06:25:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.063+0000 7fcd03de8640 1 --2- 172.21.15.3:0/2533075756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccfc109df0 0x7fccfc110620 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.063+0000 7fcd03de8640 1 --2- 172.21.15.3:0/2533075756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccfc1094a0 0x7fccfc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.063+0000 7fcd03de8640 1 --2- 172.21.15.3:0/2533075756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccfc103b60 0x7fccfc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.063+0000 7fcd03de8640 1 -- 172.21.15.3:0/2533075756 >> 172.21.15.3:0/2533075756 conn(0x7fccfc07b850 msgr2=0x7fccfc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.064+0000 7fcd03de8640 1 -- 172.21.15.3:0/2533075756 shutdown_connections 2024-09-21T06:25:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.064+0000 7fcd03de8640 1 -- 172.21.15.3:0/2533075756 wait complete. 2024-09-21T06:25:54.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.064+0000 7fcd03de8640 1 Processor -- start 2024-09-21T06:25:54.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd03de8640 1 -- start start 2024-09-21T06:25:54.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd03de8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccfc103b60 0x7fccfc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd03de8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccfc1094a0 0x7fccfc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd03de8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccfc109df0 0x7fccfc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd03de8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc113ab0 con 0x7fccfc109df0 2024-09-21T06:25:54.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd03de8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc113930 con 0x7fccfc103b60 2024-09-21T06:25:54.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd03de8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc113c30 con 0x7fccfc1094a0 2024-09-21T06:25:54.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd02385640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccfc109df0 0x7fccfc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd02385640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccfc109df0 0x7fccfc076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41064/0 (socket says 172.21.15.3:41064) 2024-09-21T06:25:54.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd02385640 1 -- 172.21.15.3:0/2626749254 learned_addr learned my addr 172.21.15.3:0/2626749254 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:54.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd01b84640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccfc103b60 0x7fccfc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd01383640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccfc1094a0 0x7fccfc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd02385640 1 -- 172.21.15.3:0/2626749254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccfc1094a0 msgr2=0x7fccfc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:54.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd02385640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccfc1094a0 0x7fccfc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd02385640 1 -- 172.21.15.3:0/2626749254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccfc103b60 msgr2=0x7fccfc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd02385640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccfc103b60 0x7fccfc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd02385640 1 -- 172.21.15.3:0/2626749254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc1a8740 con 0x7fccfc109df0 2024-09-21T06:25:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.065+0000 7fcd01383640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccfc1094a0 0x7fccfc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.066+0000 7fcd02385640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccfc109df0 0x7fccfc076190 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fccf4004770 tx=0x7fccf4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.066+0000 7fccf2ffd640 1 -- 172.21.15.3:0/2626749254 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf403f030 con 0x7fccfc109df0 2024-09-21T06:25:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.066+0000 7fccf2ffd640 1 -- 172.21.15.3:0/2626749254 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccf4003aa0 con 0x7fccfc109df0 2024-09-21T06:25:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.066+0000 7fcd03de8640 1 -- 172.21.15.3:0/2626749254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccfc1a89d0 con 0x7fccfc109df0 2024-09-21T06:25:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.066+0000 7fccf2ffd640 1 -- 172.21.15.3:0/2626749254 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4035070 con 0x7fccfc109df0 2024-09-21T06:25:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.066+0000 7fcd03de8640 1 -- 172.21.15.3:0/2626749254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccfc1a8eb0 con 0x7fccfc109df0 2024-09-21T06:25:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.070+0000 7fccf2ffd640 1 -- 172.21.15.3:0/2626749254 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fccf4042030 con 0x7fccfc109df0 2024-09-21T06:25:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.070+0000 7fcd0a35c740 1 Processor -- start 2024-09-21T06:25:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.070+0000 7fcd0a35c740 1 -- start start 2024-09-21T06:25:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd0a35c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e2e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd0a35c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 0x562ac9e2ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd0a35c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e2f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd0a35c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562ac9da4ea0 con 0x562ac9a87070 2024-09-21T06:25:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd0a35c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562ac9d942d0 con 0x562ac9a5e1b0 2024-09-21T06:25:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd0a35c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562ac9d980e0 con 0x562ac9a66390 2024-09-21T06:25:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd01383640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e2e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd01383640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e2e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41068/0 (socket says 172.21.15.3:41068) 2024-09-21T06:25:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd01383640 1 -- 172.21.15.3:0/55480281 learned_addr learned my addr 172.21.15.3:0/55480281 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd01b84640 1 --2- 172.21.15.3:0/55480281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 0x562ac9e2ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fccf2ffd640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcce80b6680 0x7fcce80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd01383640 1 --2- 172.21.15.3:0/55480281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e2f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd02385640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcce80b6680 0x7fcce80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd01383640 1 -- 172.21.15.3:0/55480281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 msgr2=0x562ac9e2f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd01383640 1 --2- 172.21.15.3:0/55480281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e2f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd01383640 1 -- 172.21.15.3:0/55480281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 msgr2=0x562ac9e2ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd01383640 1 --2- 172.21.15.3:0/55480281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 0x562ac9e2ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fcd01383640 1 -- 172.21.15.3:0/55480281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ac9af2650 con 0x562ac9a87070 2024-09-21T06:25:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.071+0000 7fccf2ffd640 1 -- 172.21.15.3:0/2626749254 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7fccf4107e70 con 0x7fccfc109df0 2024-09-21T06:25:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.072+0000 7fcd01383640 1 --2- 172.21.15.3:0/55480281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e2e6a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fccec00bfd0 tx=0x7fccec00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f753ccff60462d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.072+0000 7fcce7fff640 1 -- 172.21.15.3:0/55480281 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec019030 con 0x562ac9a87070 2024-09-21T06:25:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.072+0000 7fcce7fff640 1 -- 172.21.15.3:0/55480281 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccec0130d0 con 0x562ac9a87070 2024-09-21T06:25:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.072+0000 7fcce7fff640 1 -- 172.21.15.3:0/55480281 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec0047a0 con 0x562ac9a87070 2024-09-21T06:25:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.073+0000 7fcd0a35c740 1 -- 172.21.15.3:0/55480281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 msgr2=0x562ac9e2e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.073+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/55480281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e2e6a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fccec00bfd0 tx=0x7fccec00b8e0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.073+0000 7fcd02385640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcce80b6680 0x7fcce80b8b90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fccf40caf40 tx=0x7fccf40c9bf0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.073+0000 7fcd0a35c740 1 -- 172.21.15.3:0/55480281 shutdown_connections 2024-09-21T06:25:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.073+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/55480281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e2f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.073+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/55480281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 0x562ac9e2ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.073+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/55480281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e2e6a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.073+0000 7fcd0a35c740 1 -- 172.21.15.3:0/55480281 >> 172.21.15.3:0/55480281 conn(0x562ac9a7b460 msgr2=0x562ac9e2c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.073+0000 7fcd0a35c740 1 -- 172.21.15.3:0/55480281 shutdown_connections 2024-09-21T06:25:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.073+0000 7fcd0a35c740 1 -- 172.21.15.3:0/55480281 wait complete. 2024-09-21T06:25:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.073+0000 7fcd0a35c740 1 -- start start 2024-09-21T06:25:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.073+0000 7fcd0a35c740 1 Processor -- start 2024-09-21T06:25:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd0a35c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 0x562ac9e39d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd0a35c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e3a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd0a35c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e46170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd0a35c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562ac9bf5700 con 0x562ac9a87070 2024-09-21T06:25:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd0a35c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562ac9d2f160 con 0x562ac9a5e1b0 2024-09-21T06:25:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd0a35c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562ac9e3ce20 con 0x562ac9a66390 2024-09-21T06:25:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd0a35c740 10 client.0 _ll_register_callbacks cb 0x562ac9d489e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd01383640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e46170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd01383640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e46170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41078/0 (socket says 172.21.15.3:41078) 2024-09-21T06:25:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd01383640 1 -- 172.21.15.3:0/1809507169 learned_addr learned my addr 172.21.15.3:0/1809507169 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd01b84640 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e3a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd01383640 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 0x562ac9e39d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd01383640 1 -- 172.21.15.3:0/1809507169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 msgr2=0x562ac9e3a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd01383640 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e3a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd01383640 1 -- 172.21.15.3:0/1809507169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 msgr2=0x562ac9e39d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd01383640 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 0x562ac9e39d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd01383640 1 -- 172.21.15.3:0/1809507169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ac9d5cc90 con 0x562ac9a87070 2024-09-21T06:25:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.074+0000 7fcd01b84640 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e3a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcd01383640 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e46170 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x562ac9e1e7b0 tx=0x7fccec005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c1f43c9e0be902 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcce57fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcce57fa640 1 -- 172.21.15.3:0/1809507169 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec026020 con 0x562ac9a87070 2024-09-21T06:25:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcd0a35c740 1 -- 172.21.15.3:0/1809507169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccec0130d0 con 0x562ac9a87070 2024-09-21T06:25:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcce57fa640 1 -- 172.21.15.3:0/1809507169 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7fccec004ad0 con 0x562ac9a87070 2024-09-21T06:25:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcce57fa640 1 -- 172.21.15.3:0/1809507169 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fccec004cd0 con 0x562ac9a87070 2024-09-21T06:25:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcd0a35c740 10 client.18468 fetch_fsmap learned FSMap version 39 2024-09-21T06:25:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcd0a35c740 1 -- 172.21.15.3:0/1809507169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562ac9e48b60 con 0x562ac9a87070 2024-09-21T06:25:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcce57fa640 1 -- 172.21.15.3:0/1809507169 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccec0130d0 con 0x562ac9a87070 2024-09-21T06:25:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcce57fa640 1 -- 172.21.15.3:0/1809507169 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fccec011bd0 con 0x562ac9a87070 2024-09-21T06:25:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcd0a35c740 10 client.18468 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:25:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcd0a35c740 10 client.18468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcd0a35c740 20 client.18468 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.075+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ac9a459f0 0x562ac9e4bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.076+0000 7fcd0a35c740 4 client.18468 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.076+0000 7fcd0a35c740 1 -- 172.21.15.3:0/1809507169 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ac9da6df0 con 0x562ac9a459f0 2024-09-21T06:25:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.076+0000 7fcd01b84640 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ac9a459f0 0x562ac9e4bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.076+0000 7fcd01b84640 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ac9a459f0 0x562ac9e4bf10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c222c419e869137e server_cookie=d41bf846a02541f0 in_seq=0 out_seq=0 2024-09-21T06:25:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.076+0000 7fcce57fa640 10 client.18468 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.077+0000 7fcce57fa640 1 -- 172.21.15.3:0/1809507169 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562ac9da6df0 con 0x562ac9a459f0 2024-09-21T06:25:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.077+0000 7fcce57fa640 10 client.18468 handle_command_reply: tid=0 2024-09-21T06:25:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.077+0000 7fcd0a35c740 1 client.18468 shutdown 2024-09-21T06:25:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.078+0000 7fcd0a35c740 1 -- 172.21.15.3:0/1809507169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 msgr2=0x562ac9e46170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.078+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e46170 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x562ac9e1e7b0 tx=0x7fccec005460 comp rx=0 tx=0).stop 2024-09-21T06:25:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.078+0000 7fcd0a35c740 1 -- 172.21.15.3:0/1809507169 shutdown_connections 2024-09-21T06:25:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.078+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ac9a459f0 0x562ac9e4bf10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.078+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e3a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.078+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e46170 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.078+0000 7fcce57fa640 0 client.18468 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:54.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.078+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/1809507169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 0x562ac9e39d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.078+0000 7fcd0a35c740 1 -- 172.21.15.3:0/1809507169 >> 172.21.15.3:0/1809507169 conn(0x562ac9a7b460 msgr2=0x562ac9e31880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:54.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.078+0000 7fcd0a35c740 1 -- 172.21.15.3:0/1809507169 shutdown_connections 2024-09-21T06:25:54.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.078+0000 7fcd0a35c740 1 -- 172.21.15.3:0/1809507169 wait complete. 2024-09-21T06:25:54.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.078+0000 7fcd0a35c740 20 client.18468 trim_cache size 0 max 16384 2024-09-21T06:25:54.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.091+0000 7fcd0a35c740 1 Processor -- start 2024-09-21T06:25:54.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.091+0000 7fcd0a35c740 1 -- start start 2024-09-21T06:25:54.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.091+0000 7fcd0a35c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a5e1b0 0x562ac9e392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.091+0000 7fcd0a35c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e31eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.091+0000 7fcd0a35c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a87070 0x562ac9e460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.091+0000 7fcd0a35c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc1b1910 con 0x562ac9a5e1b0 2024-09-21T06:25:54.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.091+0000 7fcd0a35c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc1b3450 con 0x562ac9a87070 2024-09-21T06:25:54.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.091+0000 7fcd0a35c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc1b1fe0 con 0x562ac9a66390 2024-09-21T06:25:54.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.092+0000 7fcd01383640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a5e1b0 0x562ac9e392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.092+0000 7fcd01383640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a5e1b0 0x562ac9e392a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41088/0 (socket says 172.21.15.3:41088) 2024-09-21T06:25:54.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.092+0000 7fcd01383640 1 -- 172.21.15.3:0/385798612 learned_addr learned my addr 172.21.15.3:0/385798612 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:54.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.092+0000 7fcd01b84640 1 --2- 172.21.15.3:0/385798612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e31eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.092+0000 7fcd01383640 1 -- 172.21.15.3:0/385798612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 msgr2=0x562ac9e31eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:54.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.092+0000 7fcd01383640 1 --2- 172.21.15.3:0/385798612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e31eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.092+0000 7fcd01383640 1 -- 172.21.15.3:0/385798612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a87070 msgr2=0x562ac9e460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:54.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.092+0000 7fcd01383640 1 --2- 172.21.15.3:0/385798612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a87070 0x562ac9e460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.092+0000 7fcd01383640 1 -- 172.21.15.3:0/385798612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ac9e323f0 con 0x562ac9a5e1b0 2024-09-21T06:25:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.092+0000 7fcd01383640 1 --2- 172.21.15.3:0/385798612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a5e1b0 0x562ac9e392a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fccec00b920 tx=0x7fccec00b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb84c75bc67f23a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.092+0000 7fcce77fe640 1 -- 172.21.15.3:0/385798612 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec019070 con 0x562ac9a5e1b0 2024-09-21T06:25:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.092+0000 7fcce77fe640 1 -- 172.21.15.3:0/385798612 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccec00b520 con 0x562ac9a5e1b0 2024-09-21T06:25:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.092+0000 7fcce77fe640 1 -- 172.21.15.3:0/385798612 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec026040 con 0x562ac9a5e1b0 2024-09-21T06:25:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.093+0000 7fcd0a35c740 1 -- 172.21.15.3:0/385798612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a5e1b0 msgr2=0x562ac9e392a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.093+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/385798612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a5e1b0 0x562ac9e392a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fccec00b920 tx=0x7fccec00b3c0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.093+0000 7fcd0a35c740 1 -- 172.21.15.3:0/385798612 shutdown_connections 2024-09-21T06:25:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.093+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/385798612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e31eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.093+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/385798612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a87070 0x562ac9e460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.093+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/385798612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a5e1b0 0x562ac9e392a0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.093+0000 7fcd0a35c740 1 -- 172.21.15.3:0/385798612 >> 172.21.15.3:0/385798612 conn(0x562ac9a7b460 msgr2=0x562ac9e317a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.093+0000 7fcd0a35c740 1 -- 172.21.15.3:0/385798612 shutdown_connections 2024-09-21T06:25:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.093+0000 7fcd0a35c740 1 -- 172.21.15.3:0/385798612 wait complete. 2024-09-21T06:25:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd0a35c740 1 -- start start 2024-09-21T06:25:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd0a35c740 1 Processor -- start 2024-09-21T06:25:54.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd0a35c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 0x562ac9e4f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd0a35c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e4cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd0a35c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e4d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd0a35c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc1b2d80 con 0x562ac9a87070 2024-09-21T06:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd0a35c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc1b26b0 con 0x562ac9a5e1b0 2024-09-21T06:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd0a35c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc1af910 con 0x562ac9a66390 2024-09-21T06:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd0a35c740 10 client.0 _ll_register_callbacks cb 0x7fccf80099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd01383640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e4d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd01383640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e4d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41100/0 (socket says 172.21.15.3:41100) 2024-09-21T06:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd01383640 1 -- 172.21.15.3:0/3698415049 learned_addr learned my addr 172.21.15.3:0/3698415049 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:54.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd01383640 1 --2- 172.21.15.3:0/3698415049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 0x562ac9e4f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.094+0000 7fcd01b84640 1 --2- 172.21.15.3:0/3698415049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e4cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.095+0000 7fcd01383640 1 -- 172.21.15.3:0/3698415049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 msgr2=0x562ac9e4cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:54.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.095+0000 7fcd01383640 1 --2- 172.21.15.3:0/3698415049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e4cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.095+0000 7fcd01383640 1 -- 172.21.15.3:0/3698415049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 msgr2=0x562ac9e4f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:54.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.095+0000 7fcd01383640 1 --2- 172.21.15.3:0/3698415049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 0x562ac9e4f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.095+0000 7fcd01383640 1 -- 172.21.15.3:0/3698415049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ac9e4d950 con 0x562ac9a87070 2024-09-21T06:25:54.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.095+0000 7fcd01383640 1 --2- 172.21.15.3:0/3698415049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e4d1c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fccec005510 tx=0x7fccec0130d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba086cc9961d5ac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:54.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.095+0000 7fcce57fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-21T06:25:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.095+0000 7fcce57fa640 1 -- 172.21.15.3:0/3698415049 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec026030 con 0x562ac9a87070 2024-09-21T06:25:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.095+0000 7fcce57fa640 1 -- 172.21.15.3:0/3698415049 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7fccec00fa70 con 0x562ac9a87070 2024-09-21T06:25:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.095+0000 7fcd0a35c740 1 -- 172.21.15.3:0/3698415049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccec00b520 con 0x562ac9a87070 2024-09-21T06:25:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.095+0000 7fcce57fa640 1 -- 172.21.15.3:0/3698415049 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fccec019040 con 0x562ac9a87070 2024-09-21T06:25:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.095+0000 7fcd0a35c740 10 client.18474 fetch_fsmap learned FSMap version 39 2024-09-21T06:25:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.095+0000 7fcd0a35c740 1 -- 172.21.15.3:0/3698415049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562ac9eb6610 con 0x562ac9a87070 2024-09-21T06:25:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.096+0000 7fcce57fa640 1 -- 172.21.15.3:0/3698415049 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccec00b520 con 0x562ac9a87070 2024-09-21T06:25:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.096+0000 7fcce57fa640 1 -- 172.21.15.3:0/3698415049 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fccec011ad0 con 0x562ac9a87070 2024-09-21T06:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.096+0000 7fcd0a35c740 10 client.18474 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.096+0000 7fcd0a35c740 10 client.18474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.096+0000 7fcd0a35c740 20 client.18474 populate_metadata read hostname 'smithi003' 2024-09-21T06:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.096+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/3698415049 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ac9a1ada0 0x562ac9eb8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.096+0000 7fcd0a35c740 4 client.18474 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.096+0000 7fcd0a35c740 1 -- 172.21.15.3:0/3698415049 --> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ac9da28a0 con 0x562ac9a1ada0 2024-09-21T06:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.096+0000 7fcd01b84640 1 --2- 172.21.15.3:0/3698415049 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ac9a1ada0 0x562ac9eb8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcd01b84640 1 --2- 172.21.15.3:0/3698415049 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ac9a1ada0 0x562ac9eb8d40 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e33d977434770dd server_cookie=c4c60a9d38b0c2b6 in_seq=0 out_seq=0 2024-09-21T06:25:54.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcce57fa640 10 client.18474 ms_handle_connect on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcce57fa640 1 -- 172.21.15.3:0/3698415049 <== mds.0 v2:172.21.15.137:6838/2663570031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ac9da28a0 con 0x562ac9a1ada0 2024-09-21T06:25:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcce57fa640 10 client.18474 handle_command_reply: tid=0 2024-09-21T06:25:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcd0a35c740 1 client.18474 shutdown 2024-09-21T06:25:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcd0a35c740 1 -- 172.21.15.3:0/3698415049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 msgr2=0x562ac9e4d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/3698415049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e4d1c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fccec005510 tx=0x7fccec0130d0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcd0a35c740 1 -- 172.21.15.3:0/3698415049 shutdown_connections 2024-09-21T06:25:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/3698415049 >> [v2:172.21.15.137:6838/2663570031,v1:172.21.15.137:6839/2663570031] conn(0x562ac9a1ada0 0x562ac9eb8d40 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/3698415049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ac9a66390 0x562ac9e4cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcce57fa640 0 client.18474 ms_handle_reset on v2:172.21.15.137:6838/2663570031 2024-09-21T06:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/3698415049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ac9a87070 0x562ac9e4d1c0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcd0a35c740 1 --2- 172.21.15.3:0/3698415049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ac9a5e1b0 0x562ac9e4f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcd0a35c740 1 -- 172.21.15.3:0/3698415049 >> 172.21.15.3:0/3698415049 conn(0x562ac9a7b460 msgr2=0x562ac9e2f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcd0a35c740 1 -- 172.21.15.3:0/3698415049 shutdown_connections 2024-09-21T06:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcd0a35c740 1 -- 172.21.15.3:0/3698415049 wait complete. 2024-09-21T06:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.097+0000 7fcd0a35c740 20 client.18474 trim_cache size 0 max 16384 2024-09-21T06:25:54.108 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-21T06:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.098+0000 7fcd03de8640 1 -- 172.21.15.3:0/2626749254 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcce80b6680 msgr2=0x7fcce80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.098+0000 7fcd03de8640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcce80b6680 0x7fcce80b8b90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fccf40caf40 tx=0x7fccf40c9bf0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.098+0000 7fcd03de8640 1 -- 172.21.15.3:0/2626749254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccfc109df0 msgr2=0x7fccfc076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:54.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.098+0000 7fcd03de8640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccfc109df0 0x7fccfc076190 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fccf4004770 tx=0x7fccf4004270 comp rx=0 tx=0).stop 2024-09-21T06:25:54.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.098+0000 7fcd03de8640 1 -- 172.21.15.3:0/2626749254 shutdown_connections 2024-09-21T06:25:54.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.098+0000 7fcd03de8640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcce80b6680 0x7fcce80b8b90 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.098+0000 7fcd03de8640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccfc1094a0 0x7fccfc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.098+0000 7fcd03de8640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccfc109df0 0x7fccfc076190 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.098+0000 7fcd03de8640 1 --2- 172.21.15.3:0/2626749254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccfc103b60 0x7fccfc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.098+0000 7fcd03de8640 1 -- 172.21.15.3:0/2626749254 >> 172.21.15.3:0/2626749254 conn(0x7fccfc07b850 msgr2=0x7fccfc105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:54.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.098+0000 7fcd03de8640 1 -- 172.21.15.3:0/2626749254 shutdown_connections 2024-09-21T06:25:54.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.098+0000 7fcd03de8640 1 -- 172.21.15.3:0/2626749254 wait complete. 2024-09-21T06:25:54.806 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-21T06:25:54.811 INFO:tasks.fwd_scrub:done joining 2024-09-21T06:25:54.811 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T06:25:54.829 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:25:54.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.947+0000 7f713c269640 1 -- 172.21.15.3:0/995825163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7134102140 msgr2=0x7f7134102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:54.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.947+0000 7f713c269640 1 --2- 172.21.15.3:0/995825163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7134102140 0x7f7134102550 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f7130009900 tx=0x7f713002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.948+0000 7f713c269640 1 -- 172.21.15.3:0/995825163 shutdown_connections 2024-09-21T06:25:54.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.948+0000 7f713c269640 1 --2- 172.21.15.3:0/995825163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71341083c0 0x7f713410ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.948+0000 7f713c269640 1 --2- 172.21.15.3:0/995825163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7134107a70 0x7f7134107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.948+0000 7f713c269640 1 --2- 172.21.15.3:0/995825163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7134102140 0x7f7134102550 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.948+0000 7f713c269640 1 -- 172.21.15.3:0/995825163 >> 172.21.15.3:0/995825163 conn(0x7f7134079e50 msgr2=0x7f713407a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:54.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.948+0000 7f713c269640 1 -- 172.21.15.3:0/995825163 shutdown_connections 2024-09-21T06:25:54.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.948+0000 7f713c269640 1 -- 172.21.15.3:0/995825163 wait complete. 2024-09-21T06:25:54.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.948+0000 7f713c269640 1 Processor -- start 2024-09-21T06:25:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.948+0000 7f713c269640 1 -- start start 2024-09-21T06:25:54.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713c269640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7134102140 0x7f713419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713c269640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7134107a70 0x7f713419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713c269640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71341083c0 0x7f713419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713c269640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f71341139f0 con 0x7f7134102140 2024-09-21T06:25:54.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713c269640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7134113870 con 0x7f7134107a70 2024-09-21T06:25:54.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713c269640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7134113b70 con 0x7f71341083c0 2024-09-21T06:25:54.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713a005640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7134102140 0x7f713419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713a005640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7134102140 0x7f713419a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41112/0 (socket says 172.21.15.3:41112) 2024-09-21T06:25:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713a005640 1 -- 172.21.15.3:0/3494655631 learned_addr learned my addr 172.21.15.3:0/3494655631 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f7139804640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7134107a70 0x7f713419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713a806640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71341083c0 0x7f713419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713a005640 1 -- 172.21.15.3:0/3494655631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71341083c0 msgr2=0x7f713419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713a005640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71341083c0 0x7f713419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713a005640 1 -- 172.21.15.3:0/3494655631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7134107a70 msgr2=0x7f713419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713a005640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7134107a70 0x7f713419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713a005640 1 -- 172.21.15.3:0/3494655631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71341a0270 con 0x7f7134102140 2024-09-21T06:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.949+0000 7f713a005640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7134102140 0x7f713419a3d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f7130006010 tx=0x7f71300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.950+0000 7f712b7fe640 1 -- 172.21.15.3:0/3494655631 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713003f030 con 0x7f7134102140 2024-09-21T06:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.950+0000 7f713c269640 1 -- 172.21.15.3:0/3494655631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71341a0500 con 0x7f7134102140 2024-09-21T06:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.950+0000 7f712b7fe640 1 -- 172.21.15.3:0/3494655631 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7130003800 con 0x7f7134102140 2024-09-21T06:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.950+0000 7f712b7fe640 1 -- 172.21.15.3:0/3494655631 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130035070 con 0x7f7134102140 2024-09-21T06:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.950+0000 7f713c269640 1 -- 172.21.15.3:0/3494655631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71341a0a40 con 0x7f7134102140 2024-09-21T06:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.951+0000 7f713c269640 1 -- 172.21.15.3:0/3494655631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71341033b0 con 0x7f7134102140 2024-09-21T06:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.952+0000 7f712b7fe640 1 -- 172.21.15.3:0/3494655631 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f7130003cb0 con 0x7f7134102140 2024-09-21T06:25:54.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.953+0000 7f712b7fe640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f71200b6680 0x7f71200b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:54.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.953+0000 7f7139804640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f71200b6680 0x7f71200b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:54.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.953+0000 7f712b7fe640 1 -- 172.21.15.3:0/3494655631 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f71301080a0 con 0x7f7134102140 2024-09-21T06:25:54.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.953+0000 7f7139804640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f71200b6680 0x7f71200b8b90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7134107720 tx=0x7f712400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:54.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:54.957+0000 7f712b7fe640 1 -- 172.21.15.3:0/3494655631 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f71300ce600 con 0x7f7134102140 2024-09-21T06:25:55.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:54 smithi003 ceph-mon[31227]: pgmap v662: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T06:25:55.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.147+0000 7f713c269640 1 -- 172.21.15.3:0/3494655631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f713419b4f0 con 0x7f7134102140 2024-09-21T06:25:55.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.148+0000 7f712b7fe640 1 -- 172.21.15.3:0/3494655631 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23138 (secure 0 0 0) 0x7f71300d29e0 con 0x7f7134102140 2024-09-21T06:25:55.151 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:25:55.151 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":89,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","created":"2024-09-21T06:04:24.293786+0000","modified":"2024-09-21T06:25:14.392296+0000","last_up_change":"2024-09-21T06:12:17.781243+0000","last_in_change":"2024-09-21T06:11:54.727435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:08:26.015803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:13:07.084406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:13:09.149891+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":88,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51736f81-f410-48ee-a8f3-121289607309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6803","nonce":615312328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6805","nonce":615312328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6809","nonce":615312328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6807","nonce":615312328}]},"public_addr":"172.21.15.3:6803/615312328","cluster_addr":"172.21.15.3:6805/615312328","heartbeat_back_addr":"172.21.15.3:6809/615312328","heartbeat_front_addr":"172.21.15.3:6807/615312328","state":["exists","up"]},{"osd":1,"uuid":"55696a37-34c1-482a-8561-7d08f0aa29fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6801","nonce":4150747115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6803","nonce":4150747115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6807","nonce":4150747115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6805","nonce":4150747115}]},"public_addr":"172.21.15.137:6801/4150747115","cluster_addr":"172.21.15.137:6803/4150747115","heartbeat_back_addr":"172.21.15.137:6807/4150747115","heartbeat_front_addr":"172.21.15.137:6805/4150747115","state":["exists","up"]},{"osd":2,"uuid":"8a81d34b-464b-46a2-8ef2-5223b5947004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6801","nonce":3392577077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6803","nonce":3392577077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6807","nonce":3392577077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6805","nonce":3392577077}]},"public_addr":"172.21.15.204:6801/3392577077","cluster_addr":"172.21.15.204:6803/3392577077","heartbeat_back_addr":"172.21.15.204:6807/3392577077","heartbeat_front_addr":"172.21.15.204:6805/3392577077","state":["exists","up"]},{"osd":3,"uuid":"94ab3938-7a70-44cb-99bf-9d2dedc4eb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6811","nonce":4293741635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6813","nonce":4293741635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6817","nonce":4293741635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6815","nonce":4293741635}]},"public_addr":"172.21.15.3:6811/4293741635","cluster_addr":"172.21.15.3:6813/4293741635","heartbeat_back_addr":"172.21.15.3:6817/4293741635","heartbeat_front_addr":"172.21.15.3:6815/4293741635","state":["exists","up"]},{"osd":4,"uuid":"38551535-d692-401c-aad0-bee38f3026d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6809","nonce":2608988785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6811","nonce":2608988785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6815","nonce":2608988785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6813","nonce":2608988785}]},"public_addr":"172.21.15.137:6809/2608988785","cluster_addr":"172.21.15.137:6811/2608988785","heartbeat_back_addr":"172.21.15.137:6815/2608988785","heartbeat_front_addr":"172.21.15.137:6813/2608988785","state":["exists","up"]},{"osd":5,"uuid":"3d4b785b-28d0-4654-a9b1-4c5c905f4bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6809","nonce":3709074044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6811","nonce":3709074044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6815","nonce":3709074044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6813","nonce":3709074044}]},"public_addr":"172.21.15.204:6809/3709074044","cluster_addr":"172.21.15.204:6811/3709074044","heartbeat_back_addr":"172.21.15.204:6815/3709074044","heartbeat_front_addr":"172.21.15.204:6813/3709074044","state":["exists","up"]},{"osd":6,"uuid":"ba21dbe9-ab43-4b7e-9446-2929ee874c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6819","nonce":3862646594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6821","nonce":3862646594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6825","nonce":3862646594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6823","nonce":3862646594}]},"public_addr":"172.21.15.3:6819/3862646594","cluster_addr":"172.21.15.3:6821/3862646594","heartbeat_back_addr":"172.21.15.3:6825/3862646594","heartbeat_front_addr":"172.21.15.3:6823/3862646594","state":["exists","up"]},{"osd":7,"uuid":"9610fe5c-93df-49f9-b8bd-5cb480e0871b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6817","nonce":2086822496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6819","nonce":2086822496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6823","nonce":2086822496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6821","nonce":2086822496}]},"public_addr":"172.21.15.137:6817/2086822496","cluster_addr":"172.21.15.137:6819/2086822496","heartbeat_back_addr":"172.21.15.137:6823/2086822496","heartbeat_front_addr":"172.21.15.137:6821/2086822496","state":["exists","up"]},{"osd":8,"uuid":"49f6e9b4-6d14-434e-8b42-8b7ffeb2a072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6817","nonce":153476798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6819","nonce":153476798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6823","nonce":153476798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6821","nonce":153476798}]},"public_addr":"172.21.15.204:6817/153476798","cluster_addr":"172.21.15.204:6819/153476798","heartbeat_back_addr":"172.21.15.204:6823/153476798","heartbeat_front_addr":"172.21.15.204:6821/153476798","state":["exists","up"]},{"osd":9,"uuid":"633f67a7-b759-4ea6-82ed-3c42b3fb70ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6827","nonce":1238491549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6829","nonce":1238491549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6833","nonce":1238491549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6831","nonce":1238491549}]},"public_addr":"172.21.15.3:6827/1238491549","cluster_addr":"172.21.15.3:6829/1238491549","heartbeat_back_addr":"172.21.15.3:6833/1238491549","heartbeat_front_addr":"172.21.15.3:6831/1238491549","state":["exists","up"]},{"osd":10,"uuid":"94b65a48-ee87-4e59-9296-a72a22787f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6825","nonce":3486552683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6827","nonce":3486552683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6831","nonce":3486552683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6829","nonce":3486552683}]},"public_addr":"172.21.15.137:6825/3486552683","cluster_addr":"172.21.15.137:6827/3486552683","heartbeat_back_addr":"172.21.15.137:6831/3486552683","heartbeat_front_addr":"172.21.15.137:6829/3486552683","state":["exists","up"]},{"osd":11,"uuid":"c5cc2850-9062-4b8e-beb3-bef0db30a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6825","nonce":1084308610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6827","nonce":1084308610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6831","nonce":1084308610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6829","nonce":1084308610}]},"public_addr":"172.21.15.204:6825/1084308610","cluster_addr":"172.21.15.204:6827/1084308610","heartbeat_back_addr":"172.21.15.204:6831/1084308610","heartbeat_front_addr":"172.21.15.204:6829/1084308610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:31.898745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.089965+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:22.367464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:48.543248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:14.687331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:41.243579+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:07.290227+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:32.930046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:59.197638+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:24.315219+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:50.333639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:12:16.453640+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1573374138":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1595702358":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/547897695":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/3673347975":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/141972779":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6800/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:6801/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/2363985440":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/2138752581":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1820734047":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/2929116087":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/45453677":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/2929116087":"2024-09-22T06:05:21.689191+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:25:55.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.150+0000 7f713c269640 1 -- 172.21.15.3:0/3494655631 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f71200b6680 msgr2=0x7f71200b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.150+0000 7f713c269640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f71200b6680 0x7f71200b8b90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7134107720 tx=0x7f712400a2d0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.150+0000 7f713c269640 1 -- 172.21.15.3:0/3494655631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7134102140 msgr2=0x7f713419a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.150+0000 7f713c269640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7134102140 0x7f713419a3d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f7130006010 tx=0x7f71300042f0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.150+0000 7f713c269640 1 -- 172.21.15.3:0/3494655631 shutdown_connections 2024-09-21T06:25:55.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.150+0000 7f713c269640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f71200b6680 0x7f71200b8b90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.150+0000 7f713c269640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71341083c0 0x7f713419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.150+0000 7f713c269640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7134107a70 0x7f713419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.150+0000 7f713c269640 1 --2- 172.21.15.3:0/3494655631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7134102140 0x7f713419a3d0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.150+0000 7f713c269640 1 -- 172.21.15.3:0/3494655631 >> 172.21.15.3:0/3494655631 conn(0x7f7134079e50 msgr2=0x7f7134105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:55.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.151+0000 7f713c269640 1 -- 172.21.15.3:0/3494655631 shutdown_connections 2024-09-21T06:25:55.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.151+0000 7f713c269640 1 -- 172.21.15.3:0/3494655631 wait complete. 2024-09-21T06:25:55.164 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:08:26.015803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T06:13:07.084406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T06:13:09.149891+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 88, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:25:55.165 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:25:55.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.279+0000 7f0c14dad640 1 -- 172.21.15.3:0/807225296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c10108c50 msgr2=0x7f0c1010f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.279+0000 7f0c14dad640 1 --2- 172.21.15.3:0/807225296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c10108c50 0x7f0c1010f480 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009900 tx=0x7f0c0802a410 comp rx=0 tx=0).stop 2024-09-21T06:25:55.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.280+0000 7f0c14dad640 1 -- 172.21.15.3:0/807225296 shutdown_connections 2024-09-21T06:25:55.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.280+0000 7f0c14dad640 1 --2- 172.21.15.3:0/807225296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10108300 0x7f0c10108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.280+0000 7f0c14dad640 1 --2- 172.21.15.3:0/807225296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c10108c50 0x7f0c1010f480 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.280+0000 7f0c14dad640 1 --2- 172.21.15.3:0/807225296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c10102a20 0x7f0c10102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.280+0000 7f0c14dad640 1 -- 172.21.15.3:0/807225296 >> 172.21.15.3:0/807225296 conn(0x7f0c10079630 msgr2=0x7f0c10079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:55.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.280+0000 7f0c14dad640 1 -- 172.21.15.3:0/807225296 shutdown_connections 2024-09-21T06:25:55.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.280+0000 7f0c14dad640 1 -- 172.21.15.3:0/807225296 wait complete. 2024-09-21T06:25:55.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.280+0000 7f0c14dad640 1 Processor -- start 2024-09-21T06:25:55.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.280+0000 7f0c14dad640 1 -- start start 2024-09-21T06:25:55.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c14dad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c10102a20 0x7f0c1019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:55.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c14dad640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c10108300 0x7f0c1019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:55.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c14dad640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10108c50 0x7f0c1019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c14dad640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10113a30 con 0x7f0c10102a20 2024-09-21T06:25:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c14dad640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c101138b0 con 0x7f0c10108300 2024-09-21T06:25:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c14dad640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10113bb0 con 0x7f0c10108c50 2024-09-21T06:25:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c0e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c10102a20 0x7f0c1019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c0e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c10102a20 0x7f0c1019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57300/0 (socket says 172.21.15.3:57300) 2024-09-21T06:25:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c0e59c640 1 -- 172.21.15.3:0/3342348981 learned_addr learned my addr 172.21.15.3:0/3342348981 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c0dd9b640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c10108300 0x7f0c1019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c0ed9d640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10108c50 0x7f0c1019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c0e59c640 1 -- 172.21.15.3:0/3342348981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10108c50 msgr2=0x7f0c1019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c0e59c640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10108c50 0x7f0c1019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c0e59c640 1 -- 172.21.15.3:0/3342348981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c10108300 msgr2=0x7f0c1019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c0e59c640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c10108300 0x7f0c1019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c0e59c640 1 -- 172.21.15.3:0/3342348981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c101a02c0 con 0x7f0c10102a20 2024-09-21T06:25:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c0dd9b640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c10108300 0x7f0c1019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.281+0000 7f0c0e59c640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c10102a20 0x7f0c1019a390 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000eab0 tx=0x7f0c0000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.282+0000 7f0c077fe640 1 -- 172.21.15.3:0/3342348981 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0000ce40 con 0x7f0c10102a20 2024-09-21T06:25:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.282+0000 7f0c077fe640 1 -- 172.21.15.3:0/3342348981 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c00011920 con 0x7f0c10102a20 2024-09-21T06:25:55.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.282+0000 7f0c14dad640 1 -- 172.21.15.3:0/3342348981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c101a05b0 con 0x7f0c10102a20 2024-09-21T06:25:55.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.282+0000 7f0c077fe640 1 -- 172.21.15.3:0/3342348981 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00010a30 con 0x7f0c10102a20 2024-09-21T06:25:55.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.282+0000 7f0c14dad640 1 -- 172.21.15.3:0/3342348981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c101a6d40 con 0x7f0c10102a20 2024-09-21T06:25:55.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.283+0000 7f0c14dad640 1 -- 172.21.15.3:0/3342348981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c10102e30 con 0x7f0c10102a20 2024-09-21T06:25:55.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.284+0000 7f0c077fe640 1 -- 172.21.15.3:0/3342348981 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f0c00010bd0 con 0x7f0c10102a20 2024-09-21T06:25:55.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.285+0000 7f0c077fe640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0bfc0b6680 0x7f0bfc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:55.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.285+0000 7f0c077fe640 1 -- 172.21.15.3:0/3342348981 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f0c000e7de0 con 0x7f0c10102a20 2024-09-21T06:25:55.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.285+0000 7f0c0dd9b640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0bfc0b6680 0x7f0bfc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:55.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.286+0000 7f0c0dd9b640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0bfc0b6680 0x7f0bfc0b8b90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80058c0 tx=0x7f0bf8005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:55.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.288+0000 7f0c077fe640 1 -- 172.21.15.3:0/3342348981 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0c00016030 con 0x7f0c10102a20 2024-09-21T06:25:55.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:54 smithi137 ceph-mon[34464]: pgmap v662: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T06:25:55.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:54 smithi204 ceph-mon[34659]: pgmap v662: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T06:25:55.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.478+0000 7f0c14dad640 1 -- 172.21.15.3:0/3342348981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0c1002e640 con 0x7f0c10102a20 2024-09-21T06:25:55.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.478+0000 7f0c077fe640 1 -- 172.21.15.3:0/3342348981 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f0c000ae340 con 0x7f0c10102a20 2024-09-21T06:25:55.480 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-21T06:25:55.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.480+0000 7f0c14dad640 1 -- 172.21.15.3:0/3342348981 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0bfc0b6680 msgr2=0x7f0bfc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.480+0000 7f0c14dad640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0bfc0b6680 0x7f0bfc0b8b90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80058c0 tx=0x7f0bf8005850 comp rx=0 tx=0).stop 2024-09-21T06:25:55.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.480+0000 7f0c14dad640 1 -- 172.21.15.3:0/3342348981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c10102a20 msgr2=0x7f0c1019a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.480+0000 7f0c14dad640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c10102a20 0x7f0c1019a390 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000eab0 tx=0x7f0c0000ef70 comp rx=0 tx=0).stop 2024-09-21T06:25:55.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.480+0000 7f0c14dad640 1 -- 172.21.15.3:0/3342348981 shutdown_connections 2024-09-21T06:25:55.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.480+0000 7f0c14dad640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f0bfc0b6680 0x7f0bfc0b8b90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.480+0000 7f0c14dad640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10108c50 0x7f0c1019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.480+0000 7f0c14dad640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c10108300 0x7f0c1019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.480+0000 7f0c14dad640 1 --2- 172.21.15.3:0/3342348981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c10102a20 0x7f0c1019a390 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.480+0000 7f0c14dad640 1 -- 172.21.15.3:0/3342348981 >> 172.21.15.3:0/3342348981 conn(0x7f0c10079630 msgr2=0x7f0c10106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:55.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.480+0000 7f0c14dad640 1 -- 172.21.15.3:0/3342348981 shutdown_connections 2024-09-21T06:25:55.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.481+0000 7f0c14dad640 1 -- 172.21.15.3:0/3342348981 wait complete. 2024-09-21T06:25:55.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T06:25:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.609+0000 7fcb9db94640 1 -- 172.21.15.3:0/1629333454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb9810dd30 msgr2=0x7fcb98110240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.609+0000 7fcb9db94640 1 --2- 172.21.15.3:0/1629333454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb9810dd30 0x7fcb98110240 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009900 tx=0x7fcb9002a410 comp rx=0 tx=0).stop 2024-09-21T06:25:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.609+0000 7fcb9db94640 1 -- 172.21.15.3:0/1629333454 shutdown_connections 2024-09-21T06:25:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.609+0000 7fcb9db94640 1 --2- 172.21.15.3:0/1629333454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb981051b0 0x7fcb9810d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.609+0000 7fcb9db94640 1 --2- 172.21.15.3:0/1629333454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb9810dd30 0x7fcb98110240 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.609+0000 7fcb9db94640 1 --2- 172.21.15.3:0/1629333454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb98104780 0x7fcb98104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.609+0000 7fcb9db94640 1 -- 172.21.15.3:0/1629333454 >> 172.21.15.3:0/1629333454 conn(0x7fcb980fe3b0 msgr2=0x7fcb98100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.609+0000 7fcb9db94640 1 -- 172.21.15.3:0/1629333454 shutdown_connections 2024-09-21T06:25:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.609+0000 7fcb9db94640 1 -- 172.21.15.3:0/1629333454 wait complete. 2024-09-21T06:25:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.610+0000 7fcb9db94640 1 Processor -- start 2024-09-21T06:25:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.610+0000 7fcb9db94640 1 -- start start 2024-09-21T06:25:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.610+0000 7fcb9db94640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb98104780 0x7fcb9819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.610+0000 7fcb9db94640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb981051b0 0x7fcb9819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.610+0000 7fcb9db94640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9810dd30 0x7fcb9819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.610+0000 7fcb9db94640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcb981139d0 con 0x7fcb98104780 2024-09-21T06:25:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.610+0000 7fcb9db94640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcb98113850 con 0x7fcb981051b0 2024-09-21T06:25:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.610+0000 7fcb9db94640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcb98113b50 con 0x7fcb9810dd30 2024-09-21T06:25:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.610+0000 7fcb977fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb98104780 0x7fcb9819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.610+0000 7fcb977fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb98104780 0x7fcb9819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57320/0 (socket says 172.21.15.3:57320) 2024-09-21T06:25:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.610+0000 7fcb977fe640 1 -- 172.21.15.3:0/394233114 learned_addr learned my addr 172.21.15.3:0/394233114 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.610+0000 7fcb96ffd640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb981051b0 0x7fcb9819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.610+0000 7fcb97fff640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9810dd30 0x7fcb9819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.611+0000 7fcb977fe640 1 -- 172.21.15.3:0/394233114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9810dd30 msgr2=0x7fcb9819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.611+0000 7fcb977fe640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9810dd30 0x7fcb9819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.611+0000 7fcb977fe640 1 -- 172.21.15.3:0/394233114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb981051b0 msgr2=0x7fcb9819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.611+0000 7fcb977fe640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb981051b0 0x7fcb9819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.611+0000 7fcb977fe640 1 -- 172.21.15.3:0/394233114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb981a0360 con 0x7fcb98104780 2024-09-21T06:25:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.611+0000 7fcb97fff640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9810dd30 0x7fcb9819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.611+0000 7fcb96ffd640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb981051b0 0x7fcb9819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.611+0000 7fcb977fe640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb98104780 0x7fcb9819a430 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800ebc0 tx=0x7fcb8800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.611+0000 7fcb94ff9640 1 -- 172.21.15.3:0/394233114 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88010630 con 0x7fcb98104780 2024-09-21T06:25:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.611+0000 7fcb94ff9640 1 -- 172.21.15.3:0/394233114 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb88010c80 con 0x7fcb98104780 2024-09-21T06:25:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.611+0000 7fcb9db94640 1 -- 172.21.15.3:0/394233114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb981a0650 con 0x7fcb98104780 2024-09-21T06:25:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.611+0000 7fcb94ff9640 1 -- 172.21.15.3:0/394233114 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88018b60 con 0x7fcb98104780 2024-09-21T06:25:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.611+0000 7fcb9db94640 1 -- 172.21.15.3:0/394233114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb981a6de0 con 0x7fcb98104780 2024-09-21T06:25:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.612+0000 7fcb9db94640 1 -- 172.21.15.3:0/394233114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb98104b90 con 0x7fcb98104780 2024-09-21T06:25:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.614+0000 7fcb94ff9640 1 -- 172.21.15.3:0/394233114 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fcb88018d00 con 0x7fcb98104780 2024-09-21T06:25:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.614+0000 7fcb94ff9640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcb800b6680 0x7fcb800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.614+0000 7fcb94ff9640 1 -- 172.21.15.3:0/394233114 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7fcb880e8370 con 0x7fcb98104780 2024-09-21T06:25:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.614+0000 7fcb96ffd640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcb800b6680 0x7fcb800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.615+0000 7fcb96ffd640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcb800b6680 0x7fcb800b8b90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcb980ff5a0 tx=0x7fcb8c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:55.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.618+0000 7fcb94ff9640 1 -- 172.21.15.3:0/394233114 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb880ae870 con 0x7fcb98104780 2024-09-21T06:25:55.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.805+0000 7fcb9db94640 1 -- 172.21.15.3:0/394233114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcb981a7110 con 0x7fcb98104780 2024-09-21T06:25:55.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.806+0000 7fcb94ff9640 1 -- 172.21.15.3:0/394233114 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fcb880b2c50 con 0x7fcb98104780 2024-09-21T06:25:55.808 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-21T06:25:55.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.808+0000 7fcb9db94640 1 -- 172.21.15.3:0/394233114 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcb800b6680 msgr2=0x7fcb800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.808+0000 7fcb9db94640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcb800b6680 0x7fcb800b8b90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcb980ff5a0 tx=0x7fcb8c005f90 comp rx=0 tx=0).stop 2024-09-21T06:25:55.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.808+0000 7fcb9db94640 1 -- 172.21.15.3:0/394233114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb98104780 msgr2=0x7fcb9819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.808+0000 7fcb9db94640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb98104780 0x7fcb9819a430 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800ebc0 tx=0x7fcb8800c6a0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.808+0000 7fcb9db94640 1 -- 172.21.15.3:0/394233114 shutdown_connections 2024-09-21T06:25:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.808+0000 7fcb9db94640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fcb800b6680 0x7fcb800b8b90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.808+0000 7fcb9db94640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9810dd30 0x7fcb9819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.808+0000 7fcb9db94640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb981051b0 0x7fcb9819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.808+0000 7fcb9db94640 1 --2- 172.21.15.3:0/394233114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb98104780 0x7fcb9819a430 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.808+0000 7fcb9db94640 1 -- 172.21.15.3:0/394233114 >> 172.21.15.3:0/394233114 conn(0x7fcb980fe3b0 msgr2=0x7fcb981077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.808+0000 7fcb9db94640 1 -- 172.21.15.3:0/394233114 shutdown_connections 2024-09-21T06:25:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.808+0000 7fcb9db94640 1 -- 172.21.15.3:0/394233114 wait complete. 2024-09-21T06:25:55.821 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T06:25:55.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.938+0000 7f26b663f640 1 -- 172.21.15.3:0/3061784244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26b0104900 msgr2=0x7f26b0104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.938+0000 7f26b663f640 1 --2- 172.21.15.3:0/3061784244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26b0104900 0x7f26b0104d10 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009900 tx=0x7f26a002a410 comp rx=0 tx=0).stop 2024-09-21T06:25:55.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.938+0000 7f26b663f640 1 -- 172.21.15.3:0/3061784244 shutdown_connections 2024-09-21T06:25:55.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.938+0000 7f26b663f640 1 --2- 172.21.15.3:0/3061784244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26b0105250 0x7f26b010d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.938+0000 7f26b663f640 1 --2- 172.21.15.3:0/3061784244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26b010dd60 0x7f26b0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.938+0000 7f26b663f640 1 --2- 172.21.15.3:0/3061784244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26b0104900 0x7f26b0104d10 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.938+0000 7f26b663f640 1 -- 172.21.15.3:0/3061784244 >> 172.21.15.3:0/3061784244 conn(0x7f26b00fe3b0 msgr2=0x7f26b0100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:55.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.938+0000 7f26b663f640 1 -- 172.21.15.3:0/3061784244 shutdown_connections 2024-09-21T06:25:55.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.938+0000 7f26b663f640 1 -- 172.21.15.3:0/3061784244 wait complete. 2024-09-21T06:25:55.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.938+0000 7f26b663f640 1 Processor -- start 2024-09-21T06:25:55.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.938+0000 7f26b663f640 1 -- start start 2024-09-21T06:25:55.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26b663f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26b0104900 0x7f26b019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:55.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26b663f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26b0105250 0x7f26b019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:55.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26b663f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26b010dd60 0x7f26b019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:55.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26b663f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01139e0 con 0x7f26b0104900 2024-09-21T06:25:55.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26b663f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0113860 con 0x7f26b010dd60 2024-09-21T06:25:55.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26b663f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0113b60 con 0x7f26b0105250 2024-09-21T06:25:55.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26affff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26b0104900 0x7f26b019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:55.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26affff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26b0104900 0x7f26b019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57324/0 (socket says 172.21.15.3:57324) 2024-09-21T06:25:55.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26affff640 1 -- 172.21.15.3:0/3499722635 learned_addr learned my addr 172.21.15.3:0/3499722635 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:55.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26a7fff640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26b0105250 0x7f26b019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:55.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26b4bdc640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26b010dd60 0x7f26b019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:55.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26affff640 1 -- 172.21.15.3:0/3499722635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26b0105250 msgr2=0x7f26b019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26affff640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26b0105250 0x7f26b019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26affff640 1 -- 172.21.15.3:0/3499722635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26b010dd60 msgr2=0x7f26b019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:55.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26affff640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26b010dd60 0x7f26b019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:55.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26affff640 1 -- 172.21.15.3:0/3499722635 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b01a0310 con 0x7f26b0104900 2024-09-21T06:25:55.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.939+0000 7f26affff640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26b0104900 0x7f26b019a3e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f26a0005940 tx=0x7f26a00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:55.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.940+0000 7f26adffb640 1 -- 172.21.15.3:0/3499722635 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a003f030 con 0x7f26b0104900 2024-09-21T06:25:55.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.940+0000 7f26b663f640 1 -- 172.21.15.3:0/3499722635 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b01a05a0 con 0x7f26b0104900 2024-09-21T06:25:55.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.940+0000 7f26adffb640 1 -- 172.21.15.3:0/3499722635 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a00044c0 con 0x7f26b0104900 2024-09-21T06:25:55.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.940+0000 7f26adffb640 1 -- 172.21.15.3:0/3499722635 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0035070 con 0x7f26b0104900 2024-09-21T06:25:55.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.940+0000 7f26b663f640 1 -- 172.21.15.3:0/3499722635 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b01a6d90 con 0x7f26b0104900 2024-09-21T06:25:55.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.941+0000 7f26b663f640 1 -- 172.21.15.3:0/3499722635 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b0104d10 con 0x7f26b0104900 2024-09-21T06:25:55.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.942+0000 7f26adffb640 1 -- 172.21.15.3:0/3499722635 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f26a0042030 con 0x7f26b0104900 2024-09-21T06:25:55.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.943+0000 7f26adffb640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f269c0b6680 0x7f269c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:55.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.943+0000 7f26adffb640 1 -- 172.21.15.3:0/3499722635 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f26a0107d10 con 0x7f26b0104900 2024-09-21T06:25:55.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.943+0000 7f26a7fff640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f269c0b6680 0x7f269c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:55.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.944+0000 7f26a7fff640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f269c0b6680 0x7f269c0b8b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f26b00ff4c0 tx=0x7f2698005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:55.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:55.946+0000 7f26adffb640 1 -- 172.21.15.3:0/3499722635 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f26a00d2650 con 0x7f26b0104900 2024-09-21T06:25:56.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:55 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3494655631' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:25:56.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:55 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3342348981' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:25:56.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:55 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/394233114' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:25:56.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.134+0000 7f26b663f640 1 -- 172.21.15.3:0/3499722635 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f26b01a6fc0 con 0x7f26b0104900 2024-09-21T06:25:56.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.135+0000 7f26adffb640 1 -- 172.21.15.3:0/3499722635 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f26a00d7590 con 0x7f26b0104900 2024-09-21T06:25:56.137 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-21T06:25:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.137+0000 7f26b663f640 1 -- 172.21.15.3:0/3499722635 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f269c0b6680 msgr2=0x7f269c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.137+0000 7f26b663f640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f269c0b6680 0x7f269c0b8b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f26b00ff4c0 tx=0x7f2698005f90 comp rx=0 tx=0).stop 2024-09-21T06:25:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.137+0000 7f26b663f640 1 -- 172.21.15.3:0/3499722635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26b0104900 msgr2=0x7f26b019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.137+0000 7f26b663f640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26b0104900 0x7f26b019a3e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f26a0005940 tx=0x7f26a00040c0 comp rx=0 tx=0).stop 2024-09-21T06:25:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.137+0000 7f26b663f640 1 -- 172.21.15.3:0/3499722635 shutdown_connections 2024-09-21T06:25:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.137+0000 7f26b663f640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f269c0b6680 0x7f269c0b8b90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:56.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.137+0000 7f26b663f640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26b0105250 0x7f26b019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.137+0000 7f26b663f640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26b010dd60 0x7f26b019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.137+0000 7f26b663f640 1 --2- 172.21.15.3:0/3499722635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26b0104900 0x7f26b019a3e0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.137+0000 7f26b663f640 1 -- 172.21.15.3:0/3499722635 >> 172.21.15.3:0/3499722635 conn(0x7f26b00fe3b0 msgr2=0x7f26b00fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.137+0000 7f26b663f640 1 -- 172.21.15.3:0/3499722635 shutdown_connections 2024-09-21T06:25:56.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.137+0000 7f26b663f640 1 -- 172.21.15.3:0/3499722635 wait complete. 2024-09-21T06:25:56.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T06:25:56.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.261+0000 7f56a7f39640 1 -- 172.21.15.3:0/3414778089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a0108c00 msgr2=0x7f56a010f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.261+0000 7f56a7f39640 1 --2- 172.21.15.3:0/3414778089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a0108c00 0x7f56a010f430 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f5698009ca0 tx=0x7f569802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:25:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.261+0000 7f56a7f39640 1 -- 172.21.15.3:0/3414778089 shutdown_connections 2024-09-21T06:25:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.261+0000 7f56a7f39640 1 --2- 172.21.15.3:0/3414778089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56a01082b0 0x7f56a01086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.261+0000 7f56a7f39640 1 --2- 172.21.15.3:0/3414778089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a0108c00 0x7f56a010f430 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.261+0000 7f56a7f39640 1 --2- 172.21.15.3:0/3414778089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a0102980 0x7f56a0102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.261+0000 7f56a7f39640 1 -- 172.21.15.3:0/3414778089 >> 172.21.15.3:0/3414778089 conn(0x7f56a00fe380 msgr2=0x7f56a01007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.261+0000 7f56a7f39640 1 -- 172.21.15.3:0/3414778089 shutdown_connections 2024-09-21T06:25:56.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.261+0000 7f56a7f39640 1 -- 172.21.15.3:0/3414778089 wait complete. 2024-09-21T06:25:56.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.262+0000 7f56a7f39640 1 Processor -- start 2024-09-21T06:25:56.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.262+0000 7f56a7f39640 1 -- start start 2024-09-21T06:25:56.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.262+0000 7f56a7f39640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a0102980 0x7f56a019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:56.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.262+0000 7f56a7f39640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a01082b0 0x7f56a019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:56.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.262+0000 7f56a7f39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56a0108c00 0x7f56a019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:56.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.262+0000 7f56a7f39640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01128c0 con 0x7f56a01082b0 2024-09-21T06:25:56.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.262+0000 7f56a7f39640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0112740 con 0x7f56a0102980 2024-09-21T06:25:56.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.262+0000 7f56a7f39640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0112a40 con 0x7f56a0108c00 2024-09-21T06:25:56.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f56a54d4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a01082b0 0x7f56a019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:56.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f56a54d4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a01082b0 0x7f56a019a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57336/0 (socket says 172.21.15.3:57336) 2024-09-21T06:25:56.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f56a54d4640 1 -- 172.21.15.3:0/2305752537 learned_addr learned my addr 172.21.15.3:0/2305752537 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:25:56.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f56a5cd5640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a0102980 0x7f56a019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:56.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f56a64d6640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56a0108c00 0x7f56a019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:56.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f56a54d4640 1 -- 172.21.15.3:0/2305752537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56a0108c00 msgr2=0x7f56a019fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:56.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f56a54d4640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56a0108c00 0x7f56a019fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:56.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f56a54d4640 1 -- 172.21.15.3:0/2305752537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a0102980 msgr2=0x7f56a019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:56.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f56a54d4640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a0102980 0x7f56a019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:56.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f56a54d4640 1 -- 172.21.15.3:0/2305752537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a01a0170 con 0x7f56a01082b0 2024-09-21T06:25:56.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f56a54d4640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a01082b0 0x7f56a019a870 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f569000c8c0 tx=0x7f569000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:56.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f5696ffd640 1 -- 172.21.15.3:0/2305752537 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690007bb0 con 0x7f56a01082b0 2024-09-21T06:25:56.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f5696ffd640 1 -- 172.21.15.3:0/2305752537 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5690007d50 con 0x7f56a01082b0 2024-09-21T06:25:56.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f56a7f39640 1 -- 172.21.15.3:0/2305752537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a01a0460 con 0x7f56a01082b0 2024-09-21T06:25:56.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f5696ffd640 1 -- 172.21.15.3:0/2305752537 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56900186e0 con 0x7f56a01082b0 2024-09-21T06:25:56.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.263+0000 7f56a7f39640 1 -- 172.21.15.3:0/2305752537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a01a6c80 con 0x7f56a01082b0 2024-09-21T06:25:56.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.264+0000 7f56a7f39640 1 -- 172.21.15.3:0/2305752537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a0103bf0 con 0x7f56a01082b0 2024-09-21T06:25:56.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.266+0000 7f5696ffd640 1 -- 172.21.15.3:0/2305752537 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f569000f450 con 0x7f56a01082b0 2024-09-21T06:25:56.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.267+0000 7f5696ffd640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f568c0b6680 0x7f568c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:56.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.267+0000 7f56a5cd5640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f568c0b6680 0x7f568c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:56.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.267+0000 7f5696ffd640 1 -- 172.21.15.3:0/2305752537 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8507+0+0 (secure 0 0 0) 0x7f56900e7b30 con 0x7f56a01082b0 2024-09-21T06:25:56.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.267+0000 7f56a5cd5640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f568c0b6680 0x7f568c0b8b90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f569c00abe0 tx=0x7f569c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:56.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.270+0000 7f5696ffd640 1 -- 172.21.15.3:0/2305752537 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5690014030 con 0x7f56a01082b0 2024-09-21T06:25:56.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:55 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3494655631' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:25:56.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:55 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3342348981' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:25:56.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:55 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/394233114' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:25:56.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:55 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3494655631' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:25:56.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:55 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3342348981' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:25:56.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:55 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/394233114' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:25:56.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.508+0000 7f56a7f39640 1 -- 172.21.15.3:0/2305752537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f56a019b3f0 con 0x7f56a01082b0 2024-09-21T06:25:56.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.516+0000 7f5696ffd640 1 -- 172.21.15.3:0/2305752537 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+834043 (secure 0 0 0) 0x7f56900ae120 con 0x7f56a01082b0 2024-09-21T06:25:56.528 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:25:56.528 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:25:56.528 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 27, 2024-09-21T06:25:56.528 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.528 INFO:teuthology.orchestra.run.smithi003.stdout: "active_gid": 14223, 2024-09-21T06:25:56.528 INFO:teuthology.orchestra.run.smithi003.stdout: "active_name": "x", 2024-09-21T06:25:56.529 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addrs": { 2024-09-21T06:25:56.529 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-21T06:25:56.529 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.529 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-21T06:25:56.529 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6800", 2024-09-21T06:25:56.529 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2227168690 2024-09-21T06:25:56.529 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.529 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.530 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-21T06:25:56.530 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6801", 2024-09-21T06:25:56.530 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2227168690 2024-09-21T06:25:56.530 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.530 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-21T06:25:56.530 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.530 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addr": "172.21.15.3:6801/2227168690", 2024-09-21T06:25:56.530 INFO:teuthology.orchestra.run.smithi003.stdout: "active_change": "2024-09-21T06:06:23.529345+0000", 2024-09-21T06:25:56.531 INFO:teuthology.orchestra.run.smithi003.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T06:25:56.531 INFO:teuthology.orchestra.run.smithi003.stdout: "available": true, 2024-09-21T06:25:56.531 INFO:teuthology.orchestra.run.smithi003.stdout: "standbys": [ 2024-09-21T06:25:56.531 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.531 INFO:teuthology.orchestra.run.smithi003.stdout: "gid": 24142, 2024-09-21T06:25:56.531 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "z", 2024-09-21T06:25:56.531 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_features": 4540719139924082687, 2024-09-21T06:25:56.531 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-09-21T06:25:56.532 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.532 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-09-21T06:25:56.532 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.532 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.532 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.532 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:56.532 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:56.532 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.532 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.533 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.533 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:56.533 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.533 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.533 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.533 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T06:25:56.533 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.533 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.534 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.534 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.534 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.534 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.534 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.534 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.534 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.534 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.535 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.535 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.535 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.535 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.535 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.535 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.535 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.535 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.535 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.536 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.536 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.536 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.536 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.536 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.536 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.536 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.536 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.537 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.537 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.537 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.537 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.537 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.537 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.537 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.537 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.538 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.538 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.538 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.538 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.538 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.538 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.538 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.538 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.538 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.539 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.539 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.539 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.539 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.539 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.539 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.539 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.539 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.539 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.540 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.540 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.540 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.540 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.540 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.540 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.540 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.540 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.541 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.541 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.541 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.541 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.541 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.541 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.541 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.541 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.542 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.542 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.542 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.542 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.542 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.542 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-09-21T06:25:56.542 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-09-21T06:25:56.542 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.542 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.543 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.543 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.543 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.543 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.543 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.543 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T06:25:56.543 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.543 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.543 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.544 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.544 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-09-21T06:25:56.544 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-09-21T06:25:56.544 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.544 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.544 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.544 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-09-21T06:25:56.544 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.544 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.545 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.545 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-09-21T06:25:56.545 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.545 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.545 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.545 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.545 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-09-21T06:25:56.545 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-09-21T06:25:56.545 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.546 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.546 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.546 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.546 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.546 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.546 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.546 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-09-21T06:25:56.546 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.547 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.547 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.547 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.547 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-09-21T06:25:56.547 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-09-21T06:25:56.547 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.547 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.547 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.548 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.548 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.548 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.548 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.548 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-09-21T06:25:56.548 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.548 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.548 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.548 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.549 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-09-21T06:25:56.549 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-09-21T06:25:56.549 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.549 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.549 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.549 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-09-21T06:25:56.549 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.549 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.550 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.550 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-09-21T06:25:56.550 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.550 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.550 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.550 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-09-21T06:25:56.551 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-09-21T06:25:56.551 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.551 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.551 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.551 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.551 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.551 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.551 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.551 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-09-21T06:25:56.552 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.552 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.552 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.552 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-09-21T06:25:56.552 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-09-21T06:25:56.552 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.553 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.553 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.553 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.553 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.553 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.553 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.553 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T06:25:56.553 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.553 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.554 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.554 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.554 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-09-21T06:25:56.554 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-09-21T06:25:56.554 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.554 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.554 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.554 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.555 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.555 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.555 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.555 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-09-21T06:25:56.555 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.555 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.555 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.555 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.556 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.556 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.556 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.556 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.556 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.556 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.556 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.556 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.556 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.556 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.557 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.557 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.557 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.557 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.557 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.557 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.557 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.557 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-09-21T06:25:56.557 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.558 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.558 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.558 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-09-21T06:25:56.558 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-09-21T06:25:56.558 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.558 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.558 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.558 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.559 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.559 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.559 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.559 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T06:25:56.559 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.559 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.559 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.559 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.559 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-09-21T06:25:56.560 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-09-21T06:25:56.560 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.560 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.560 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.560 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-09-21T06:25:56.560 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.560 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.560 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.560 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T06:25:56.561 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:25:56.561 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.561 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.561 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.561 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-09-21T06:25:56.561 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-09-21T06:25:56.561 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:56.561 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.561 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.562 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.562 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-09-21T06:25:56.562 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-09-21T06:25:56.562 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.562 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T06:25:56.562 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:25:56.562 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.562 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.562 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.563 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-09-21T06:25:56.563 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-09-21T06:25:56.563 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:56.563 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.563 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.563 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-09-21T06:25:56.563 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-21T06:25:56.563 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-09-21T06:25:56.563 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.563 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T06:25:56.564 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.564 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.564 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.564 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.564 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-09-21T06:25:56.564 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-09-21T06:25:56.564 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.564 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.564 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.565 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T06:25:56.565 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.565 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.565 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.565 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T06:25:56.565 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T06:25:56.565 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.565 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.565 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.565 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-09-21T06:25:56.566 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-09-21T06:25:56.566 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:56.566 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.566 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.566 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-09-21T06:25:56.566 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-09-21T06:25:56.566 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-09-21T06:25:56.566 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.566 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-09-21T06:25:56.567 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T06:25:56.567 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.567 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.567 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.567 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-09-21T06:25:56.567 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-09-21T06:25:56.567 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.568 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.568 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.568 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-09-21T06:25:56.568 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.568 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.568 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.568 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T06:25:56.568 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:25:56.568 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.569 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.569 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.569 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-09-21T06:25:56.569 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-09-21T06:25:56.569 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:56.569 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.569 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.569 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.570 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-09-21T06:25:56.570 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-09-21T06:25:56.570 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.570 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T06:25:56.570 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:25:56.570 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.570 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.570 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.570 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.571 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.571 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.571 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.571 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.571 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.571 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.571 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.571 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.571 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.572 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.572 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.572 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.572 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.572 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.572 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.572 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.572 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.572 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.573 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.573 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.573 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.573 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.573 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.573 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.573 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.573 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.573 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.574 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.574 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.574 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.574 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.574 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.574 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.574 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.574 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.574 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.575 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.575 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.575 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.575 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.575 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.575 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.575 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.575 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.575 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.576 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.576 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.576 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.576 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.576 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.576 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.576 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.576 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.577 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.577 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.577 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.577 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.577 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.577 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.577 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.577 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.578 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.578 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.578 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.578 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.578 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.578 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.578 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.578 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.578 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-09-21T06:25:56.579 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-09-21T06:25:56.579 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:56.579 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.579 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.579 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.579 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.579 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.579 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.580 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T06:25:56.580 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.580 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.580 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.580 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.580 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-09-21T06:25:56.580 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-09-21T06:25:56.580 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.580 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.580 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.581 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-09-21T06:25:56.581 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.581 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.581 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.581 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-09-21T06:25:56.581 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-09-21T06:25:56.581 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-09-21T06:25:56.581 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-09-21T06:25:56.582 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-09-21T06:25:56.582 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.582 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-09-21T06:25:56.582 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.582 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.582 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.582 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.582 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-09-21T06:25:56.583 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-09-21T06:25:56.583 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.583 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.583 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.583 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.583 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.583 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.583 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.583 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T06:25:56.584 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.584 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.584 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.584 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.584 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:56.584 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:56.584 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.584 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.585 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.585 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:56.585 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.585 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.585 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.585 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T06:25:56.585 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.585 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.585 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.586 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.586 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.586 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.586 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.586 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.586 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.586 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.586 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.586 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.586 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.587 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.587 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.587 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.587 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.587 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.587 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-09-21T06:25:56.587 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-09-21T06:25:56.587 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.587 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.588 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.588 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:56.588 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-21T06:25:56.588 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.588 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.588 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T06:25:56.588 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T06:25:56.588 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.588 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.589 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.589 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-09-21T06:25:56.589 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-09-21T06:25:56.589 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:56.589 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.589 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.589 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:56.589 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.589 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.590 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.590 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T06:25:56.590 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.590 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.590 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.590 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.590 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.590 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.591 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-09-21T06:25:56.591 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.591 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.591 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.591 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-09-21T06:25:56.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-09-21T06:25:56.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:56.591 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.592 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.592 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-09-21T06:25:56.592 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.592 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.592 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.592 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T06:25:56.592 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.592 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.592 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.593 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.593 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-09-21T06:25:56.593 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-09-21T06:25:56.593 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.593 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.593 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.593 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-09-21T06:25:56.593 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.593 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.594 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.594 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T06:25:56.594 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.594 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.594 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.594 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.594 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-09-21T06:25:56.594 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-09-21T06:25:56.595 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.595 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.595 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.595 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-09-21T06:25:56.595 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.595 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.595 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.595 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T06:25:56.595 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.596 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.596 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.596 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.596 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-09-21T06:25:56.596 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-09-21T06:25:56.596 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.596 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.596 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.596 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.596 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.597 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.597 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.597 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T06:25:56.597 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T06:25:56.597 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.597 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.597 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.597 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-09-21T06:25:56.597 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-09-21T06:25:56.597 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.598 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.598 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.598 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:56.598 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.598 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.598 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.598 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T06:25:56.598 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.599 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.599 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.599 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.599 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-09-21T06:25:56.599 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T06:25:56.599 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:56.599 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.599 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.599 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-09-21T06:25:56.600 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.600 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.600 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.600 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T06:25:56.600 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.600 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.600 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.600 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.601 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-09-21T06:25:56.601 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-09-21T06:25:56.601 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.601 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.601 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.601 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.601 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.601 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.602 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.602 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-09-21T06:25:56.602 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-09-21T06:25:56.602 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-09-21T06:25:56.602 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.602 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T06:25:56.602 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.602 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.603 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.603 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.603 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-09-21T06:25:56.603 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-09-21T06:25:56.603 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.603 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.603 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.603 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.603 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.604 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.604 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.604 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T06:25:56.604 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.604 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.604 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.604 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.604 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-09-21T06:25:56.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-09-21T06:25:56.605 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.605 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.605 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.605 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.605 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.605 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.605 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.605 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T06:25:56.605 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.605 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.606 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.606 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.606 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-09-21T06:25:56.606 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-09-21T06:25:56.606 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.606 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.606 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.606 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.607 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.607 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.607 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.607 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T06:25:56.607 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.607 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.607 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.607 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.608 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-09-21T06:25:56.608 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-09-21T06:25:56.608 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.608 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.608 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.608 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T06:25:56.608 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.608 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.609 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.609 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-21T06:25:56.609 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.609 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.609 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.609 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.609 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-09-21T06:25:56.609 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-09-21T06:25:56.609 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.610 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.610 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.610 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T06:25:56.610 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.610 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.610 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.610 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-09-21T06:25:56.610 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.611 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.611 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.611 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.611 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-09-21T06:25:56.611 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-09-21T06:25:56.611 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.611 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.611 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.611 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T06:25:56.612 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.612 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.612 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.612 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-09-21T06:25:56.612 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.612 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.612 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.613 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.613 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-09-21T06:25:56.613 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-09-21T06:25:56.613 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.613 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.613 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.613 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T06:25:56.613 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.613 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.614 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.614 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-21T06:25:56.614 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.614 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.614 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.614 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.614 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-09-21T06:25:56.614 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-09-21T06:25:56.615 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.615 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.615 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.615 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T06:25:56.615 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.615 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.615 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.615 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-09-21T06:25:56.615 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.616 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.616 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.616 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.616 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-09-21T06:25:56.616 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-09-21T06:25:56.616 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.616 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.616 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.617 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T06:25:56.617 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.617 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.617 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.617 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-09-21T06:25:56.617 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.617 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.617 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.618 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.618 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-09-21T06:25:56.618 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-09-21T06:25:56.618 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.618 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.618 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.618 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T06:25:56.618 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.618 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.619 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.619 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-09-21T06:25:56.619 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.619 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.619 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.619 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.619 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-09-21T06:25:56.619 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-09-21T06:25:56.620 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.620 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.620 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.620 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T06:25:56.620 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.620 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.620 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.620 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-09-21T06:25:56.620 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.621 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.621 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.621 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.621 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-09-21T06:25:56.621 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-09-21T06:25:56.621 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.621 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.621 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.622 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T06:25:56.622 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.622 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.622 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.622 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-09-21T06:25:56.622 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.622 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.622 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.623 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.623 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-09-21T06:25:56.623 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-09-21T06:25:56.623 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.623 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.623 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.623 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T06:25:56.623 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.624 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.624 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.624 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-09-21T06:25:56.624 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.624 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.624 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.624 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.624 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nginx": { 2024-09-21T06:25:56.625 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nginx", 2024-09-21T06:25:56.625 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.625 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.625 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.625 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T06:25:56.625 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.625 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.625 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.625 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nginx container image", 2024-09-21T06:25:56.626 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.626 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.626 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.626 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.626 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-09-21T06:25:56.626 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-09-21T06:25:56.626 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.626 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.627 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.627 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T06:25:56.627 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.627 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.627 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.627 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-21T06:25:56.627 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.627 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.627 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.628 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.628 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-09-21T06:25:56.628 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-09-21T06:25:56.628 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.628 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.628 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.628 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T06:25:56.628 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.628 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.629 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.629 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-09-21T06:25:56.629 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.629 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.629 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.629 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.629 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_oauth2_proxy": { 2024-09-21T06:25:56.629 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T06:25:56.629 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.630 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.630 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.630 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T06:25:56.630 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.630 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.630 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.630 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "oauth2-proxy container image", 2024-09-21T06:25:56.630 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.630 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.630 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.631 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.631 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-09-21T06:25:56.631 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-09-21T06:25:56.631 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.631 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.631 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.631 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T06:25:56.631 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.631 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.632 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.632 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-21T06:25:56.632 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.632 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.632 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.632 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.632 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-09-21T06:25:56.632 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-09-21T06:25:56.633 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.633 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.633 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.633 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T06:25:56.633 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.633 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.633 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.633 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-09-21T06:25:56.634 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.634 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.634 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.634 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.634 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-09-21T06:25:56.634 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-09-21T06:25:56.634 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.634 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.634 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.635 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T06:25:56.635 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.635 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.635 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.635 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-09-21T06:25:56.635 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.635 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.635 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.636 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.636 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba_metrics": { 2024-09-21T06:25:56.636 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba_metrics", 2024-09-21T06:25:56.636 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.636 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.636 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.636 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T06:25:56.636 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.636 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.637 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.637 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T06:25:56.637 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.637 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.637 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.637 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.637 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-09-21T06:25:56.637 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-09-21T06:25:56.638 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.638 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.638 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.638 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T06:25:56.638 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.638 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.638 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.638 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-09-21T06:25:56.638 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.639 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.639 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.639 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.639 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-09-21T06:25:56.639 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-09-21T06:25:56.639 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.639 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.639 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.639 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.640 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.640 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.640 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.640 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T06:25:56.640 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.640 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.640 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.640 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.640 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-09-21T06:25:56.641 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-09-21T06:25:56.641 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.641 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.641 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.641 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:56.641 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.641 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.641 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.642 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T06:25:56.642 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.642 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.642 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.642 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.642 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-09-21T06:25:56.642 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T06:25:56.642 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.642 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.643 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.643 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-09-21T06:25:56.643 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.643 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.643 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.643 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T06:25:56.643 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.643 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.643 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.643 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.644 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-09-21T06:25:56.644 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-09-21T06:25:56.644 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.644 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.644 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.644 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-09-21T06:25:56.644 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.644 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.644 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.645 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T06:25:56.645 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.645 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.645 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.645 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.645 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-09-21T06:25:56.645 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-09-21T06:25:56.645 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.645 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.646 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.646 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-09-21T06:25:56.646 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.646 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.646 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.646 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-09-21T06:25:56.646 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.646 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.646 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.647 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.647 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-09-21T06:25:56.647 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-09-21T06:25:56.647 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.647 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.647 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.647 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.647 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.647 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.647 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.648 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T06:25:56.648 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.648 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.648 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.648 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.648 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-09-21T06:25:56.648 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-09-21T06:25:56.648 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.648 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.649 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.649 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:56.649 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.649 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.649 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.649 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-09-21T06:25:56.649 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.649 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.649 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.650 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.650 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-09-21T06:25:56.650 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-09-21T06:25:56.650 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.650 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.650 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.650 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T06:25:56.650 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.651 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.651 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.651 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T06:25:56.651 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.651 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.651 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.651 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.652 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-09-21T06:25:56.652 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-09-21T06:25:56.652 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.652 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.652 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.652 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:56.652 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.652 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.652 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.653 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-09-21T06:25:56.653 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.653 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.653 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.653 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.653 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-09-21T06:25:56.653 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-09-21T06:25:56.653 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.654 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.654 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.654 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.654 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.654 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.654 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.654 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T06:25:56.654 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.655 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.655 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.655 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.655 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-09-21T06:25:56.655 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-09-21T06:25:56.655 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.655 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.655 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.656 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.656 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.656 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.656 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.656 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T06:25:56.656 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.656 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.656 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.656 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.657 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.657 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.657 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.657 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.657 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.657 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.657 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.657 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.658 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.658 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.658 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.658 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.658 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.658 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.658 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.658 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.658 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.659 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.659 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.659 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.659 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.659 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-09-21T06:25:56.659 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-09-21T06:25:56.659 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.659 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.660 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.660 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.660 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.660 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.660 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.660 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T06:25:56.660 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.660 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.661 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.661 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.661 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.661 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.661 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.661 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.661 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.661 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.661 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.662 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.662 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.662 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T06:25:56.662 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.662 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.662 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.662 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.662 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.663 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.663 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.663 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.663 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.663 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.663 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.663 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.663 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.663 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.664 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.664 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.664 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.664 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.664 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.664 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.664 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.664 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.665 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.665 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.665 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.665 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.665 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.665 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.665 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.665 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.665 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.666 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.666 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.666 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.666 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.666 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.666 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.666 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.666 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.666 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T06:25:56.667 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T06:25:56.667 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.667 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.667 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.667 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.667 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.667 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.667 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.668 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T06:25:56.668 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.668 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.668 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.668 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.668 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T06:25:56.668 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T06:25:56.668 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.668 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.669 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.669 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-09-21T06:25:56.669 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.669 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.669 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.669 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T06:25:56.669 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.669 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.670 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.670 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.670 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-09-21T06:25:56.670 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-09-21T06:25:56.670 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.670 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.670 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.670 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:56.670 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.671 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.671 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.671 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-09-21T06:25:56.671 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.671 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.671 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.671 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.671 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-09-21T06:25:56.672 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-09-21T06:25:56.672 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.672 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.672 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.672 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:56.672 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.672 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.672 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.672 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T06:25:56.673 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.673 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.673 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.673 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.673 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-09-21T06:25:56.673 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-09-21T06:25:56.673 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.673 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.673 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.674 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.674 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.674 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.674 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.674 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-09-21T06:25:56.674 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.674 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.674 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.675 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.675 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-09-21T06:25:56.675 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-09-21T06:25:56.675 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.675 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.675 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.675 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-09-21T06:25:56.675 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.675 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.676 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.676 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-09-21T06:25:56.676 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-09-21T06:25:56.676 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.676 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T06:25:56.676 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.676 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.676 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.677 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.677 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-09-21T06:25:56.677 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-09-21T06:25:56.677 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.677 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.677 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.677 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-09-21T06:25:56.677 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.677 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.678 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.678 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T06:25:56.678 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.678 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.678 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.678 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.678 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-09-21T06:25:56.678 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-09-21T06:25:56.679 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.679 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.679 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.679 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T06:25:56.679 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.679 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.679 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.679 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T06:25:56.680 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.680 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.680 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.680 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.680 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-09-21T06:25:56.680 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-09-21T06:25:56.680 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.680 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.680 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.681 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.681 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.681 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.681 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.681 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T06:25:56.681 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.681 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.681 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.681 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.682 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-09-21T06:25:56.682 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-09-21T06:25:56.682 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.682 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.682 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.682 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.682 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.682 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.682 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.683 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T06:25:56.683 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.683 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.683 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.683 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.683 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-09-21T06:25:56.683 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-09-21T06:25:56.683 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.684 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.684 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.684 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.684 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.684 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.684 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.684 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T06:25:56.684 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.684 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.685 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.685 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.685 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-09-21T06:25:56.685 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-09-21T06:25:56.685 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.685 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.685 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.685 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.685 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.686 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.686 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.686 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T06:25:56.686 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.686 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.686 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.686 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.686 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-09-21T06:25:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-09-21T06:25:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T06:25:56.688 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.688 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.688 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.688 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.688 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-09-21T06:25:56.688 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-09-21T06:25:56.688 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.688 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.689 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.689 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-09-21T06:25:56.689 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.689 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.689 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.689 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-09-21T06:25:56.689 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.689 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.690 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.690 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.690 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.690 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.690 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.690 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.690 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.690 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.690 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.691 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.691 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.691 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.691 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.691 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.691 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.691 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.691 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-09-21T06:25:56.692 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-09-21T06:25:56.692 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.692 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.692 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.692 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.692 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.692 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.692 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.693 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T06:25:56.693 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.693 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.693 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.693 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.693 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_keepalive_count_max": { 2024-09-21T06:25:56.693 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T06:25:56.693 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.694 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.694 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.694 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-09-21T06:25:56.694 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.694 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.694 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.694 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T06:25:56.694 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.694 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.695 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.695 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.695 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_keepalive_interval": { 2024-09-21T06:25:56.695 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_keepalive_interval", 2024-09-21T06:25:56.695 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.695 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.695 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.695 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-09-21T06:25:56.696 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.696 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.696 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.696 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T06:25:56.696 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.696 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.696 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.696 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.696 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-09-21T06:25:56.697 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-09-21T06:25:56.697 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.697 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.697 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.697 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.697 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.697 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.697 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.698 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T06:25:56.698 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.698 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.698 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.698 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.698 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-09-21T06:25:56.698 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-09-21T06:25:56.698 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.698 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.699 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.699 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.699 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.699 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.699 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.699 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T06:25:56.699 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.699 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.700 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.700 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.700 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-09-21T06:25:56.700 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-09-21T06:25:56.700 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.700 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.700 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.700 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.700 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.701 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.701 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.701 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T06:25:56.701 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.701 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.701 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.701 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.701 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-09-21T06:25:56.701 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-09-21T06:25:56.702 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.702 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.702 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.702 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.702 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.702 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.702 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.702 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T06:25:56.703 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.703 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.703 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.703 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.703 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-09-21T06:25:56.703 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-09-21T06:25:56.703 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.703 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.703 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.704 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.704 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.704 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.704 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.704 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T06:25:56.704 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.704 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.704 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.704 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.705 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.705 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.705 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.705 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-09-21T06:25:56.705 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.705 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.705 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.705 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.706 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.706 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.706 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.706 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.706 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.706 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.706 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.706 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.706 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.707 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.707 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.707 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.707 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.707 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.707 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.707 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.707 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.707 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.708 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.708 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.708 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.708 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.708 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.708 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.708 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.708 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.709 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.709 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.709 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.709 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.709 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.709 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.709 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.709 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.709 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.710 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.710 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.710 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.710 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.710 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.710 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.710 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.710 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.711 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.711 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.711 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.711 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.711 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.711 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.711 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.711 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.711 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.712 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.712 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.712 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.712 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.712 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.712 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.712 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.712 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.712 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.713 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.713 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.713 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.713 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.713 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.713 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.713 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.714 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.714 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-09-21T06:25:56.714 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-09-21T06:25:56.714 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.714 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.714 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.714 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-09-21T06:25:56.714 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.714 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.715 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.715 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T06:25:56.715 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.715 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.715 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.715 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.715 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.715 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.715 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.716 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.716 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.716 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.716 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.716 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.716 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.716 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.716 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.716 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.717 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.717 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.717 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-09-21T06:25:56.717 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-09-21T06:25:56.717 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.717 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.717 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.717 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-09-21T06:25:56.717 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.718 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.718 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.718 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T06:25:56.718 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.718 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.718 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.718 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.718 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.719 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.719 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.719 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-09-21T06:25:56.719 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.719 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.719 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.719 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T06:25:56.719 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T06:25:56.720 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.720 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.720 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.720 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:56.720 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.720 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.720 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.720 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.720 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.721 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.721 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.721 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.721 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T06:25:56.721 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T06:25:56.721 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.721 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.721 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.722 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.722 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.722 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.722 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.722 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.722 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.722 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.722 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.723 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.723 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T06:25:56.723 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T06:25:56.723 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.723 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.723 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.723 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.723 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.723 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.724 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.724 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.724 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.724 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.724 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.724 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.724 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-09-21T06:25:56.724 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T06:25:56.725 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.725 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.725 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.725 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.725 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.725 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.725 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.725 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.726 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.726 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.726 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.726 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.726 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T06:25:56.726 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T06:25:56.726 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.726 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.726 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.727 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.727 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.727 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.727 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.727 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.727 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.727 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.727 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.727 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.728 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T06:25:56.728 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T06:25:56.728 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.728 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.728 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.728 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.728 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.728 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.728 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.728 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.729 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.729 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.729 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.729 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.729 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T06:25:56.729 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T06:25:56.729 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.729 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.729 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.730 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.730 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.730 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.730 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.730 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.730 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.730 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.730 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.731 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T06:25:56.731 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T06:25:56.731 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.731 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.731 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.731 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.731 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.731 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.731 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.732 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.732 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.732 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.732 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.732 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T06:25:56.732 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T06:25:56.732 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.732 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.732 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.733 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.733 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.733 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.733 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.733 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.733 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.733 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.733 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.734 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.734 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T06:25:56.734 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T06:25:56.734 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.734 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.734 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.734 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.734 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.735 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.735 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.735 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.735 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.735 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.735 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.735 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.735 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T06:25:56.736 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T06:25:56.736 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.736 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.736 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.736 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.736 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.736 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.736 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.737 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.737 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.737 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.737 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.737 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.737 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T06:25:56.737 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T06:25:56.737 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.737 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.738 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.738 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.738 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.738 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.738 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.738 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.738 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.738 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.739 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.739 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.739 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T06:25:56.739 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T06:25:56.739 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.739 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.739 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.739 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.739 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.740 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.740 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.740 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.740 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.740 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.740 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.740 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.740 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T06:25:56.740 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T06:25:56.741 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.741 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.741 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.741 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.741 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.741 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.741 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.741 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.742 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.742 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.742 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.742 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.742 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T06:25:56.742 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T06:25:56.742 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.742 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.742 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.743 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-21T06:25:56.743 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.743 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.743 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.743 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.743 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.743 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.743 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.744 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T06:25:56.744 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T06:25:56.744 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.744 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.744 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.744 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.744 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.744 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.745 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.745 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.745 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.745 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.745 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.745 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.745 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-09-21T06:25:56.745 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-09-21T06:25:56.745 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.746 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.746 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.746 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.746 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.746 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.746 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.746 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.746 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.747 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.747 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.747 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T06:25:56.747 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T06:25:56.747 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.747 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.747 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.747 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-21T06:25:56.748 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.748 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.748 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.748 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.748 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.748 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.748 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.748 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T06:25:56.749 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T06:25:56.749 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.749 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.749 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.749 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.749 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.749 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.749 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.749 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.750 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.750 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.750 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.750 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T06:25:56.750 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T06:25:56.750 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.750 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.751 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.751 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.751 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.751 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.751 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.751 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.751 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.751 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.751 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.752 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T06:25:56.752 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T06:25:56.752 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.752 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.752 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.752 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.752 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.752 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.753 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.753 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.753 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.753 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.753 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.753 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.753 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T06:25:56.753 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T06:25:56.753 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.754 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.754 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.754 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.754 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.754 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.754 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.754 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.754 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.755 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.755 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.755 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.755 INFO:teuthology.orchestra.run.smithi003.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T06:25:56.755 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T06:25:56.755 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.755 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.755 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.755 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "[]", 2024-09-21T06:25:56.756 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.756 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.756 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.756 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.756 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.756 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.756 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.757 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.757 INFO:teuthology.orchestra.run.smithi003.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T06:25:56.757 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T06:25:56.757 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.757 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.757 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.757 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "{}", 2024-09-21T06:25:56.757 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.758 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.758 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.758 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.758 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.758 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.758 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.758 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.758 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T06:25:56.758 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T06:25:56.759 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.759 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.759 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.759 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.759 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.759 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.759 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.759 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.760 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.760 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.760 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.760 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.760 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T06:25:56.760 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T06:25:56.760 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.760 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.761 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.761 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.761 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.761 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.761 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.761 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.761 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.761 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.762 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.762 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.762 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T06:25:56.762 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T06:25:56.762 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.762 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.762 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.762 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.762 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.763 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.763 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.763 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.763 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.763 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.763 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.763 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.763 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T06:25:56.764 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T06:25:56.764 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.764 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.764 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.764 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.764 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.764 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.764 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.765 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.765 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.765 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.765 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.765 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.765 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T06:25:56.765 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T06:25:56.765 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.765 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.766 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.766 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.766 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.766 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.766 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.766 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.766 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.766 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.766 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.767 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.767 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T06:25:56.767 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T06:25:56.767 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.767 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.767 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.767 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.767 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.768 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.768 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.768 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.768 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.768 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.768 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.768 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.768 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T06:25:56.768 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T06:25:56.769 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.769 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.769 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.769 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.769 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.769 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.769 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.769 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.769 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.770 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.770 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.770 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.770 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T06:25:56.770 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T06:25:56.770 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.770 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.770 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.770 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.771 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.771 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.771 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.771 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.771 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.771 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.771 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.771 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.771 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T06:25:56.772 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T06:25:56.772 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.772 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.772 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.772 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.772 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.772 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.772 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.772 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.773 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.773 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.773 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.773 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.773 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T06:25:56.773 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T06:25:56.773 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.773 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.773 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.774 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.774 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.774 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.774 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.774 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.774 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.774 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.774 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.774 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.775 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T06:25:56.775 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T06:25:56.775 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.775 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.775 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.775 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T06:25:56.775 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.775 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.776 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.776 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.776 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.776 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.776 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.776 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.776 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T06:25:56.776 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T06:25:56.776 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.777 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.777 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.777 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:56.777 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.777 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.777 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.777 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.777 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.778 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.778 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.778 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.778 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T06:25:56.778 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T06:25:56.778 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.778 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.778 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.778 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-09-21T06:25:56.779 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.779 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.779 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.779 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.779 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.779 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.779 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.779 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.780 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T06:25:56.780 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T06:25:56.780 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.780 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.780 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.780 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-09-21T06:25:56.780 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.780 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.780 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.781 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.781 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.781 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.781 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.781 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.781 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T06:25:56.781 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T06:25:56.781 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.782 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.782 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.782 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.782 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.782 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.782 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.782 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.782 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.783 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.783 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.783 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.783 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T06:25:56.783 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T06:25:56.783 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.783 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.783 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.784 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-21T06:25:56.784 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.784 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.784 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.784 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.784 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.784 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.784 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.784 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.785 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T06:25:56.785 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T06:25:56.785 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.785 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.785 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.785 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.785 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.785 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.785 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.786 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.786 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.786 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.786 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.786 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.786 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T06:25:56.786 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T06:25:56.786 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.787 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.787 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.787 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.787 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.787 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.787 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.787 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.787 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.788 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.788 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.788 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.788 INFO:teuthology.orchestra.run.smithi003.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T06:25:56.788 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T06:25:56.788 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.788 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.788 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.788 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.789 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.789 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.789 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.789 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.789 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.789 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.789 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.789 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.789 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T06:25:56.790 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T06:25:56.790 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.790 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.790 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.790 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.790 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.790 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.790 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.791 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.791 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.791 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.791 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.791 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T06:25:56.791 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T06:25:56.791 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.791 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.792 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.792 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:56.792 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.792 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.792 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.792 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.792 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.792 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.792 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.793 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T06:25:56.793 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T06:25:56.793 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.793 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.793 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.793 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:56.793 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.793 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.793 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.794 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.794 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.794 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.794 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.794 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.794 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-09-21T06:25:56.794 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-09-21T06:25:56.794 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.794 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.794 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.795 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.795 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.795 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.795 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.795 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.795 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.795 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.795 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.796 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-09-21T06:25:56.796 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-09-21T06:25:56.796 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.796 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.796 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.796 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.796 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.796 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.796 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.796 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.797 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.797 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.797 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.797 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.797 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-09-21T06:25:56.797 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-09-21T06:25:56.797 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.797 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.797 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.798 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.798 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.798 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.798 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.798 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-09-21T06:25:56.798 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.798 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.798 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.799 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.799 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-09-21T06:25:56.799 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-09-21T06:25:56.799 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.799 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.799 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.799 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-09-21T06:25:56.799 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.799 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.800 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.800 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.800 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.800 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.800 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.800 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.800 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-09-21T06:25:56.800 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-09-21T06:25:56.801 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.801 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.801 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.801 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.801 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.801 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.801 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.801 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.801 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.802 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.802 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.802 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.802 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.802 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.802 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.802 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.802 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.803 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.803 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.803 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.803 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.803 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.803 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.803 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.803 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.804 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.804 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.804 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.804 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.804 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.804 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.804 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.804 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.804 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.805 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.805 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.805 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.805 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.805 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.805 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.805 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.805 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.806 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.806 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.806 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.806 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.806 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.806 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.806 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.806 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.806 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.807 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.807 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.807 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.807 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.807 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.807 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.807 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.807 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.808 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.808 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.808 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.808 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.808 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.808 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.808 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.808 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.808 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.809 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.809 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.809 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.809 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.809 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.809 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.809 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.809 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.809 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.810 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.810 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.810 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.810 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.810 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-09-21T06:25:56.810 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-09-21T06:25:56.810 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.810 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.811 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.811 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.811 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.811 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.811 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.811 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-09-21T06:25:56.811 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.811 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.812 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.812 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.812 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-09-21T06:25:56.812 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T06:25:56.812 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.812 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.812 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.812 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.812 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.813 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.813 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.813 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.813 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.813 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.813 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.813 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.813 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-21T06:25:56.813 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-21T06:25:56.814 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.814 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.814 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.814 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-09-21T06:25:56.814 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.814 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.814 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.814 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.815 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.815 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.815 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.815 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.815 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-21T06:25:56.815 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-21T06:25:56.815 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.815 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.816 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.816 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-09-21T06:25:56.816 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.816 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.816 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.816 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.816 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.816 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.816 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.817 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.817 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.817 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.817 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.817 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.817 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.817 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.817 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.817 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.818 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.818 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.818 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.818 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.818 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.818 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.818 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-09-21T06:25:56.818 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-09-21T06:25:56.819 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.819 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.819 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.819 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.819 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.819 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.819 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.819 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.820 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.820 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.820 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.820 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.820 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-09-21T06:25:56.820 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-09-21T06:25:56.820 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.820 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.820 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.821 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-09-21T06:25:56.821 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.821 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.821 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.821 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.821 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.821 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.821 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.821 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.822 INFO:teuthology.orchestra.run.smithi003.stdout: "sso_oauth2": { 2024-09-21T06:25:56.822 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sso_oauth2", 2024-09-21T06:25:56.822 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.822 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.822 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.822 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.822 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.822 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.823 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.823 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.823 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.823 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.823 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.823 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.823 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-09-21T06:25:56.823 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-09-21T06:25:56.824 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.824 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.824 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.824 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-09-21T06:25:56.825 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.825 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.826 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.826 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.826 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-09-21T06:25:56.826 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.826 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.826 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.826 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.826 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.827 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.827 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-09-21T06:25:56.827 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-09-21T06:25:56.827 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.827 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.827 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.827 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-21T06:25:56.827 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-09-21T06:25:56.827 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-09-21T06:25:56.827 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.828 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.828 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.828 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.828 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.828 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.828 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-09-21T06:25:56.828 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-09-21T06:25:56.828 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.828 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.829 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.829 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.829 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.829 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.829 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.829 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.829 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.829 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.830 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.830 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.830 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.830 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.830 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.830 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-09-21T06:25:56.830 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.830 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.831 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.831 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-09-21T06:25:56.831 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-09-21T06:25:56.831 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.831 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.831 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.831 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.831 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.831 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.832 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.832 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-09-21T06:25:56.832 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.832 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.832 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.832 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.832 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.832 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.832 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.833 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.833 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.833 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.833 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.833 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.833 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.833 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.833 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.834 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.834 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.834 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.834 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.834 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.834 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.834 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.834 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.834 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.835 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.835 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.835 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.835 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.835 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.835 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.835 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.835 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.836 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.836 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.836 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.836 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.836 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.836 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.836 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.836 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.837 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.837 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.837 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.837 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.837 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.837 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.837 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.837 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.837 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.838 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.838 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.838 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.838 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.838 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.838 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.838 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.838 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.838 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.838 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.839 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.839 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.839 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.839 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.839 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.839 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.839 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.839 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.839 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.840 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.840 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.840 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.840 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.840 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.840 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-09-21T06:25:56.840 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-09-21T06:25:56.840 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.841 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.841 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.841 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-09-21T06:25:56.841 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.841 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.841 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.841 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T06:25:56.841 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.842 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.842 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.842 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.842 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-09-21T06:25:56.842 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-09-21T06:25:56.842 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.842 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.842 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.842 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-09-21T06:25:56.843 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.843 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.843 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.843 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T06:25:56.843 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.843 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.843 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.843 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.844 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-09-21T06:25:56.844 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-09-21T06:25:56.844 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.844 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.844 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.844 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-09-21T06:25:56.844 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.844 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.845 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.845 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-09-21T06:25:56.845 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.845 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.845 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.845 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-09-21T06:25:56.845 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-09-21T06:25:56.846 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.846 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.846 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.846 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-09-21T06:25:56.846 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.846 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.846 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.846 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T06:25:56.847 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.847 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.847 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.847 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-09-21T06:25:56.847 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-09-21T06:25:56.847 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.847 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.847 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.848 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:56.848 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.848 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.848 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.848 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T06:25:56.848 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.848 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.848 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.849 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.849 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:56.849 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:56.849 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.849 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.849 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.849 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:56.849 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.849 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.850 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.850 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T06:25:56.850 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.850 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.850 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.850 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.850 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.850 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.851 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.851 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.851 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.851 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.851 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.851 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.851 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.851 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.851 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.852 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.852 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.852 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.852 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-09-21T06:25:56.852 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-09-21T06:25:56.852 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.852 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.852 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.853 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-09-21T06:25:56.853 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.853 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.853 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.853 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T06:25:56.853 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.853 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.853 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.853 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.854 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.854 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.854 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.854 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-09-21T06:25:56.854 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.854 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.854 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.854 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.855 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.855 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.855 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.855 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.855 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.855 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.855 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.855 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.855 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.856 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.856 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.856 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.856 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.856 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.856 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.856 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.856 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.856 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.857 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.857 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.857 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.857 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.857 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.857 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.857 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.858 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.858 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.858 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.858 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.858 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.858 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.858 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.858 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.859 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.859 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.859 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.859 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.859 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.859 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.859 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.859 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.859 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.860 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.860 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.860 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.860 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.860 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.860 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.860 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.860 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.861 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.861 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.861 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.861 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.861 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.861 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.861 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.861 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.862 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.862 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.862 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.862 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.862 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.862 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.862 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.862 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.862 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.863 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-09-21T06:25:56.863 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-09-21T06:25:56.863 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.863 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.863 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.863 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-09-21T06:25:56.863 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.863 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.863 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.864 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.864 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.864 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.864 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.864 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.864 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-09-21T06:25:56.864 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-09-21T06:25:56.864 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.864 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.864 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.865 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-09-21T06:25:56.865 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.865 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.865 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.865 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.865 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.865 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.865 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.865 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.866 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:56.866 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:56.866 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.866 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.866 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.866 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:56.866 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.866 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.867 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.867 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.867 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.867 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.867 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.867 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.867 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.867 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.868 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.868 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.868 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.868 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.868 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.868 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.868 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.868 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.868 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.869 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.869 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.869 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.869 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.869 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.869 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.869 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-09-21T06:25:56.869 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-09-21T06:25:56.869 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-09-21T06:25:56.869 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.870 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-09-21T06:25:56.870 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-09-21T06:25:56.870 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.870 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.870 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.870 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-09-21T06:25:56.870 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.870 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.870 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.871 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T06:25:56.871 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.871 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.871 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.871 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.871 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-09-21T06:25:56.871 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-09-21T06:25:56.871 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.871 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.872 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.872 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-09-21T06:25:56.872 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.872 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.872 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.872 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T06:25:56.872 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.872 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.872 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.873 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.873 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-09-21T06:25:56.873 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-09-21T06:25:56.873 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.873 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.873 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.873 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.873 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.873 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.874 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.874 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-09-21T06:25:56.874 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.874 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.874 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.874 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.874 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:56.874 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:56.875 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:56.875 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.875 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.875 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-09-21T06:25:56.875 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-09-21T06:25:56.875 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.875 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.875 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T06:25:56.875 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.876 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.876 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.876 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.876 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.876 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.876 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.876 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.876 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.876 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.877 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.877 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.877 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.877 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.877 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.877 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.877 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.877 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.878 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.878 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.878 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.878 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.878 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.878 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.878 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.878 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.879 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.879 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.879 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.879 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.879 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.879 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.879 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.879 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.879 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.880 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.880 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.880 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.880 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.880 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.880 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.880 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.880 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.880 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.881 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.881 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.881 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.881 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.881 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.881 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.881 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.881 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.881 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.882 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.882 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.882 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.882 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.882 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.882 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.882 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.882 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.882 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.883 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.883 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.883 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.883 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.883 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.883 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.883 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.883 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.884 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.884 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.884 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.884 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.884 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-09-21T06:25:56.884 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-09-21T06:25:56.884 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.884 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.884 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.885 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.885 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.885 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.885 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.885 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-09-21T06:25:56.885 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.885 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.885 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.885 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.886 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-09-21T06:25:56.886 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-09-21T06:25:56.886 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.886 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.886 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.886 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-09-21T06:25:56.886 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.886 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.887 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.887 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-09-21T06:25:56.887 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.887 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.887 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.887 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.887 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.887 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.888 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.888 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.888 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.888 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.888 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.888 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.888 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.888 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.888 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.889 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.889 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.889 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.889 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-09-21T06:25:56.889 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-09-21T06:25:56.889 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.889 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.889 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.889 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-09-21T06:25:56.890 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.890 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.890 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.890 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:25:56.890 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.890 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.890 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.890 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.891 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-09-21T06:25:56.891 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-09-21T06:25:56.891 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.891 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.891 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.891 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:56.891 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-21T06:25:56.891 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-09-21T06:25:56.891 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.892 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T06:25:56.892 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.892 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.892 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.892 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.892 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-09-21T06:25:56.892 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-09-21T06:25:56.892 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.893 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.893 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.893 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.893 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.893 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.893 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.893 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-09-21T06:25:56.893 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.893 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.894 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.894 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.894 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-09-21T06:25:56.894 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-09-21T06:25:56.894 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.894 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.894 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.894 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-09-21T06:25:56.895 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.895 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.895 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.895 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:25:56.895 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.895 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.895 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.895 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.895 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.896 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.896 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.896 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-09-21T06:25:56.896 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.896 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.896 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.896 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.896 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.896 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.897 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.897 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.897 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.897 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.897 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.897 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.897 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.897 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.897 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.898 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.898 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.898 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.898 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.898 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.898 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.898 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.898 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.898 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.899 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.899 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.899 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.899 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.899 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.899 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.899 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.899 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.900 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.900 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.900 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.900 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.900 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.900 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.900 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.900 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.900 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.901 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.901 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.901 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.901 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.901 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.901 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.901 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.901 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.902 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.902 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.902 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.902 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.902 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.902 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.902 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.902 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.903 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.903 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.903 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.903 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.903 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.903 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.903 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.903 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.903 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.904 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.904 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.904 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.904 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.904 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.904 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.904 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.904 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.904 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.905 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.905 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.905 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.905 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.905 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.905 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.905 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.905 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.906 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.906 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.906 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.906 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.906 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.906 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.906 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.906 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-09-21T06:25:56.906 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.907 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.907 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.907 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.907 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.907 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.907 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.907 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.907 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.908 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.908 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.908 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.908 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.908 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.908 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.908 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.908 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.908 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.909 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.909 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.909 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.909 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.909 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.909 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.909 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.909 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.910 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.910 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.910 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.910 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.910 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.910 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.910 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.910 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.911 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.911 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.911 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.911 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.911 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.911 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.911 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.911 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.912 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.912 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.912 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.912 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.912 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.912 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.912 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.912 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.913 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.913 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.913 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.913 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.913 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.913 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.913 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.913 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.913 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.914 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.914 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.914 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.914 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.914 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.914 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.914 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.914 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.915 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.915 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.915 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.915 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.915 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.915 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.915 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.915 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.916 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.916 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.916 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.916 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.916 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.916 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.916 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.916 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.916 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.917 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.917 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.917 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.917 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.917 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.917 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.917 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-09-21T06:25:56.917 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.917 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.918 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.918 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-09-21T06:25:56.918 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-09-21T06:25:56.918 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.918 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.918 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.918 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-09-21T06:25:56.918 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.919 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.919 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.919 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T06:25:56.919 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.919 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.919 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.919 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.919 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-09-21T06:25:56.919 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-09-21T06:25:56.920 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.920 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.920 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.920 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:56.920 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-09-21T06:25:56.920 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.920 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.920 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T06:25:56.921 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.921 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.921 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.921 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.921 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.921 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.921 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.921 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.922 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.922 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.922 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.922 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.922 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.922 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.922 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.922 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.922 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.923 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.923 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.923 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.923 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.923 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.923 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.923 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.923 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.924 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.924 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.924 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.924 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.924 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.924 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.924 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.924 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.925 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.925 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.925 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.925 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.925 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.925 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.925 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.925 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.925 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.926 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.926 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.926 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.926 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.926 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.926 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.926 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.926 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.927 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.927 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.927 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.927 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.927 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.927 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.927 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.927 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.927 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.928 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.928 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.928 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.928 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.928 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.928 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.928 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.928 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.929 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.929 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.929 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.929 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.929 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.929 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.929 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.929 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.929 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.929 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.930 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.930 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.930 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.930 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.930 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.930 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.930 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.930 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.930 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.931 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.931 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.931 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.931 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.931 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-09-21T06:25:56.931 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.931 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.931 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.931 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-09-21T06:25:56.932 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-09-21T06:25:56.932 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.932 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.932 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.932 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-09-21T06:25:56.932 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.932 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.932 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.932 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-09-21T06:25:56.933 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T06:25:56.933 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.933 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.933 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.933 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.933 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.933 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.933 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.933 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.934 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.934 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.934 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.934 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.934 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.934 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.934 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.934 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.935 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.935 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.935 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.935 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.935 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.935 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.935 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.935 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.936 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.936 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.936 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.936 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.936 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.936 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.936 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.936 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.937 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.937 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.937 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.937 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.937 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.937 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.937 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.937 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.937 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.938 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.938 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.938 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.938 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.938 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.938 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.938 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.938 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.939 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.939 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.939 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.939 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.939 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.939 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.939 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.939 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.939 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.940 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.940 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.940 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.940 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.940 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.940 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.940 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.940 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.940 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.941 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.941 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.941 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.941 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.941 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.941 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.941 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-09-21T06:25:56.941 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-09-21T06:25:56.941 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.942 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.942 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.942 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.942 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.942 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.942 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.942 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-09-21T06:25:56.942 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T06:25:56.943 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.943 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.943 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.943 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-09-21T06:25:56.943 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-09-21T06:25:56.943 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.943 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.943 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.944 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-09-21T06:25:56.944 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.944 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.944 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.944 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-09-21T06:25:56.944 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.944 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.944 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.944 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.944 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-09-21T06:25:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-09-21T06:25:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-09-21T06:25:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.946 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T06:25:56.946 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.946 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.946 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.946 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.946 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-09-21T06:25:56.946 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-09-21T06:25:56.946 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.946 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.947 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.947 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.947 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.947 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.947 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.947 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-09-21T06:25:56.947 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.947 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.947 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.948 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.948 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.948 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.948 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.948 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.948 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.948 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.948 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.948 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.949 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.949 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.949 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.949 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.949 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.949 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.949 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-09-21T06:25:56.949 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-09-21T06:25:56.949 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.950 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.950 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.950 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-09-21T06:25:56.950 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.950 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.950 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.950 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T06:25:56.950 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T06:25:56.950 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.951 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.951 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.951 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.951 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.951 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-09-21T06:25:56.951 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.951 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.951 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.952 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.952 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.952 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.952 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.952 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.952 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.952 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.952 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.953 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.953 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.953 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.953 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.953 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.953 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.953 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.953 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.953 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.954 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.954 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.954 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.954 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.954 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.954 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.954 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.954 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.955 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.955 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.955 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.955 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.955 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.955 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.955 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.955 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.955 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.956 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.956 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.956 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.956 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.956 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.956 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.956 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.956 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.956 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.957 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.957 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.957 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.957 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.957 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.957 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.957 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.957 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.958 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.958 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.958 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.958 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.958 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.958 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.958 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.958 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.958 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.959 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.959 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.959 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.959 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.959 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.959 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.959 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.959 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.960 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.960 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.960 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.960 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.960 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.960 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.960 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.960 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.960 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.961 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.961 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.961 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.961 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.961 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.961 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.961 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.961 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.962 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.962 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.962 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-09-21T06:25:56.962 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.962 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.962 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.962 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.962 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.962 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.963 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.963 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.963 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.963 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.963 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.963 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.963 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.963 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.964 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.964 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.964 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.964 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.964 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.964 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.964 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.964 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.964 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.965 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.965 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.965 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.965 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.965 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.965 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.965 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.965 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.965 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.965 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.966 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.966 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.966 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.966 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.966 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.966 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.966 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.966 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.966 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.967 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.967 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.967 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.967 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.967 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.967 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.967 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.967 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.967 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.968 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.968 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.968 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.968 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.968 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.968 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.968 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.968 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.968 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.968 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.969 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.969 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.969 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.969 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.969 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.969 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.969 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.969 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.969 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.970 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.970 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.970 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.970 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.970 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.970 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.970 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.970 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.970 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.971 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.971 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.971 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.971 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.971 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.971 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.971 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.971 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.971 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.972 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.972 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.972 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-09-21T06:25:56.972 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.972 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.972 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.972 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.972 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.972 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.973 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.973 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.973 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.973 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.973 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.973 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.973 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.973 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.973 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.973 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.974 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.974 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.974 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.974 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.974 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.974 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.974 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.974 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.975 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.975 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.975 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.975 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.975 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.975 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.975 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.975 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.975 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.975 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.976 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.976 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.976 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.976 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.976 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.976 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.976 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.976 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.977 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.977 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.977 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.977 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.977 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.977 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.977 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.977 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.977 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.978 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.978 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.978 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.978 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.978 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.978 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.978 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.978 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.978 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.979 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.979 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.979 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.979 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.979 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.979 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.979 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.979 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.979 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.980 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.980 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.980 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.980 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.980 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.980 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.980 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.980 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.980 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.981 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.981 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.981 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.981 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.981 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.981 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.981 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.981 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.982 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.982 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.982 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.982 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.982 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-09-21T06:25:56.982 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.982 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.982 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.982 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-09-21T06:25:56.983 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-09-21T06:25:56.983 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.983 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.983 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:56.983 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.983 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.983 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.983 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.983 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T06:25:56.984 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.984 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.984 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.984 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.984 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.984 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.984 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.984 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-09-21T06:25:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-09-21T06:25:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-09-21T06:25:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-09-21T06:25:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-09-21T06:25:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-09-21T06:25:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-09-21T06:25:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-09-21T06:25:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.007 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.007 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.007 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.007 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.007 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.007 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.007 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.007 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.008 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.008 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.008 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.008 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.008 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.008 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.008 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.008 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.008 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.008 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.009 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.009 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.009 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.009 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.009 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.009 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.009 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.009 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.010 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.010 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.010 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.010 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.010 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.010 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.010 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.010 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.010 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.011 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.011 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.011 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.011 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.011 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.011 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.011 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.011 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.011 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.012 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.012 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.012 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.012 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.012 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.012 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.012 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.012 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.012 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.012 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.013 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.013 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.013 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.013 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.013 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.013 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.013 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.014 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.014 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.014 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.014 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.014 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.014 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.014 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.014 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.014 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.015 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.015 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.015 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.015 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.015 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.015 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.015 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.015 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.016 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.016 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.016 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.016 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.016 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.016 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.016 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.016 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.017 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-09-21T06:25:57.017 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.017 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.017 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.017 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.017 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.017 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.017 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.017 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.017 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.018 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.018 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.018 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.018 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.018 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.018 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.018 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.018 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.018 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.019 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.019 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.019 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.019 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.019 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.019 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.019 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.019 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.020 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.020 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.020 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.020 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.020 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.020 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.020 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.020 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.021 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.021 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.021 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.021 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.021 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.021 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.021 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.021 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.021 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.022 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.022 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.022 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.022 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.022 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.022 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.022 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.022 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.022 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.023 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.023 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.023 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.023 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.023 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.023 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.023 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.023 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.023 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.024 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.024 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.024 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.024 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.024 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.024 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.024 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.024 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.024 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.025 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.025 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.025 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:57.025 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:57.025 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.025 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.025 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.025 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:57.025 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.025 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.026 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.026 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.026 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.026 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.026 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.026 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.026 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.026 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.026 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.027 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.027 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.027 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.027 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.027 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.027 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.027 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.027 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.027 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.028 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.028 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.028 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-09-21T06:25:57.028 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-09-21T06:25:57.028 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:57.028 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.028 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.028 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-09-21T06:25:57.028 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-09-21T06:25:57.028 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.029 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.029 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-09-21T06:25:57.029 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T06:25:57.029 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.029 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.029 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.029 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.029 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.029 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.030 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-09-21T06:25:57.030 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.030 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.030 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.030 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-09-21T06:25:57.030 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-09-21T06:25:57.030 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.030 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.030 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.031 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.031 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.031 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.031 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.031 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T06:25:57.031 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.031 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.031 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.031 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.032 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-09-21T06:25:57.032 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-09-21T06:25:57.032 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.032 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.032 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.032 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.032 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.032 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.033 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.033 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.033 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.033 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.033 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.033 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.033 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.033 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.034 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.034 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.034 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.034 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.034 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.034 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.034 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.034 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.035 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.035 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.035 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.035 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.035 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.035 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.035 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.035 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.035 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.036 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.036 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.036 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.036 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.036 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.036 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.036 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.036 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.037 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.037 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.037 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.037 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.037 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.037 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.037 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.037 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.037 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.038 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.038 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.038 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.038 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.038 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.038 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.038 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.038 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.039 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.039 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.039 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.039 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.039 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.039 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.039 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.039 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.040 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.040 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.040 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.040 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.040 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.040 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.040 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.040 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.040 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.041 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.041 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.041 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.041 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.041 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.041 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.041 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.041 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.041 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.041 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-09-21T06:25:57.042 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-09-21T06:25:57.042 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.042 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.042 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.042 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-09-21T06:25:57.042 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.042 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.043 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.043 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-09-21T06:25:57.043 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.043 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.043 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.043 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:57.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-09-21T06:25:57.044 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.045 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.046 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-09-21T06:25:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-09-21T06:25:57.047 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-09-21T06:25:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.048 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-09-21T06:25:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-09-21T06:25:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.049 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T06:25:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T06:25:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.050 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.051 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.052 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.053 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.054 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.056 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.056 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.056 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.056 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.056 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.056 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.056 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.056 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.056 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.057 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.057 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.057 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.057 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.057 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.057 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.057 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.057 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.057 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.057 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.058 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.058 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.058 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.058 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.058 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.058 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.058 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.059 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.059 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-09-21T06:25:57.059 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-09-21T06:25:57.059 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.059 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.059 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.059 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.059 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.059 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.060 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.060 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.060 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.060 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.060 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.060 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.060 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T06:25:57.060 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T06:25:57.060 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.061 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.061 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.061 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-09-21T06:25:57.061 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.061 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.061 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.061 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.061 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.061 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.062 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.062 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.062 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-09-21T06:25:57.062 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-09-21T06:25:57.062 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:57.062 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.062 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.062 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-09-21T06:25:57.062 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.063 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.063 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.063 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.063 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.063 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.063 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.063 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.063 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-21T06:25:57.064 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-21T06:25:57.064 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.064 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.064 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.064 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-09-21T06:25:57.064 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.064 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.064 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.065 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T06:25:57.065 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.065 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.065 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.065 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.065 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-21T06:25:57.065 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-21T06:25:57.065 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.065 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.066 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.066 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-09-21T06:25:57.066 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.066 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.066 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.066 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T06:25:57.066 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.066 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.067 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.067 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.067 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.067 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.067 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.067 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.067 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.067 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.068 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.068 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.068 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.068 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.068 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.068 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.068 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.068 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.068 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-09-21T06:25:57.069 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-09-21T06:25:57.069 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.069 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.069 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.069 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-09-21T06:25:57.069 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.069 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.069 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.069 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.070 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.070 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.070 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.070 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.070 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-09-21T06:25:57.070 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-09-21T06:25:57.070 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.070 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.071 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.071 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-09-21T06:25:57.071 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.071 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.071 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.071 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-09-21T06:25:57.071 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-09-21T06:25:57.071 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.072 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.072 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.072 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.072 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.072 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.072 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-09-21T06:25:57.072 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-09-21T06:25:57.072 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.072 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.073 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.073 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-21T06:25:57.073 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-09-21T06:25:57.073 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-09-21T06:25:57.073 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.073 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.073 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.073 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.074 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.074 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.074 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.074 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.074 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.074 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-09-21T06:25:57.074 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.074 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.075 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.075 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.075 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.075 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.075 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.075 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.075 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.075 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.075 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.076 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.076 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.076 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.076 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.076 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.076 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.076 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.076 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.076 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.077 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.077 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.077 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.077 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.077 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.077 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.077 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.077 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.078 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.078 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.078 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.078 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.078 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.078 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.078 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.078 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.079 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.079 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.079 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.079 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.079 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.079 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.079 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.079 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.080 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.080 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.080 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.080 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.080 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.080 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.080 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.080 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.080 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.081 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.081 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.081 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.081 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.081 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.081 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.081 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.082 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.082 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.082 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.082 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.082 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.082 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.082 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.082 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.083 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.083 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.083 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-09-21T06:25:57.083 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-09-21T06:25:57.083 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.083 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.083 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.083 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:57.083 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.084 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.084 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.084 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.084 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.084 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.084 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.084 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.084 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-09-21T06:25:57.084 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T06:25:57.085 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.085 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.085 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.085 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.085 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.085 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.085 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.085 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.085 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.086 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.086 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.086 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.086 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.086 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.086 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.086 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.086 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.087 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.087 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.087 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.087 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.087 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.087 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.087 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.087 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.087 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.088 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-09-21T06:25:57.088 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-09-21T06:25:57.088 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.088 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.088 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.088 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.088 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.088 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.089 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.089 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.089 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.089 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.089 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.089 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.089 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.089 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.089 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.090 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-09-21T06:25:57.090 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.090 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.090 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.090 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-09-21T06:25:57.090 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-09-21T06:25:57.090 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.090 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.090 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.091 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.091 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.091 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.091 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.091 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.091 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.091 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.091 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.092 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.092 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-09-21T06:25:57.092 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-09-21T06:25:57.092 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.092 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.092 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.092 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.092 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.092 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.093 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.093 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.093 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.093 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.093 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.093 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.093 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.093 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.094 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.094 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.094 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.094 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.094 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.094 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.094 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.094 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.094 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.095 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.095 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.095 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.095 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.095 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.095 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.095 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.095 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.095 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.096 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.096 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.096 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.096 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.096 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.096 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.096 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.096 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.096 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.097 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.097 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.097 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.097 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.097 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.097 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.097 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.097 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.097 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.098 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.098 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.098 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.098 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.098 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.098 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.098 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.098 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.099 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.099 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.099 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.099 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.099 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.099 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.099 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.099 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.100 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.100 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.100 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.100 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.100 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.100 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.100 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.100 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.100 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.100 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.101 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.101 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.101 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.101 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.101 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.101 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.101 INFO:teuthology.orchestra.run.smithi003.stdout: "max_requests": { 2024-09-21T06:25:57.101 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_requests", 2024-09-21T06:25:57.102 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.102 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.102 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.102 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-21T06:25:57.102 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.102 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.102 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.102 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T06:25:57.102 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.103 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.103 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.103 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.103 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-21T06:25:57.103 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-21T06:25:57.103 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.103 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.103 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.104 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.104 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.104 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.104 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.104 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.104 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.104 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.105 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-21T06:25:57.105 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-21T06:25:57.105 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.105 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.105 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.105 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.105 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.105 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.105 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.106 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.106 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.106 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.106 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.106 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.106 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.106 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.106 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.107 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.107 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.107 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.107 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.107 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.107 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.107 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.107 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.108 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.108 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.108 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.108 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.108 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.108 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.108 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-09-21T06:25:57.108 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.108 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.109 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.109 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.109 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.109 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.109 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.109 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.109 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.109 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.109 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.110 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.110 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.110 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.110 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.110 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.110 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.110 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.110 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.110 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.111 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.111 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.111 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.111 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.111 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.111 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.111 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.111 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.112 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.112 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.112 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.112 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.112 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.112 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.112 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.112 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.113 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.113 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.113 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.113 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.113 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.113 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.113 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.113 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.114 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.114 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.114 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.114 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.114 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.114 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.114 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.114 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.114 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.115 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.115 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.115 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.115 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.115 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.115 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.115 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.115 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.116 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.116 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.116 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.116 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.116 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.116 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.116 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.116 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.116 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.117 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.117 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.117 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.117 INFO:teuthology.orchestra.run.smithi003.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T06:25:57.117 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T06:25:57.117 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.117 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.117 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.117 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:57.118 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.118 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.118 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.118 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T06:25:57.118 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.118 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.118 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.118 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.118 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.119 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.119 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.119 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.119 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.119 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.119 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.119 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.119 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.120 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.120 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.120 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.120 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.120 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.120 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.120 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.120 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.121 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-09-21T06:25:57.121 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.121 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.121 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.121 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.121 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.121 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.121 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.121 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.122 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.122 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.122 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.122 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.122 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.122 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.122 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.122 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.123 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.123 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.123 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.123 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.123 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.123 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.123 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.124 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.124 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.124 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.124 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.124 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.124 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.124 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.124 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.125 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.125 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.125 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.125 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.125 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.125 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.125 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.125 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.125 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.126 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.126 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.126 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.126 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.126 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.126 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.126 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.126 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.127 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.127 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.127 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.127 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.127 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.127 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.127 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.127 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.127 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.128 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.128 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.128 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.128 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.128 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.128 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.128 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.129 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.129 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.129 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.129 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.129 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.129 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.129 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.129 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.129 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.130 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.130 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.130 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.130 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.130 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.130 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.130 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.131 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.131 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.131 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.131 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.131 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-09-21T06:25:57.131 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-09-21T06:25:57.131 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.131 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.131 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.132 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-09-21T06:25:57.132 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.132 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.132 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.132 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T06:25:57.132 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.132 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.132 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.133 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.133 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.133 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.133 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-09-21T06:25:57.133 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.133 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.133 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.133 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.133 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.134 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.134 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.134 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.134 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.134 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.134 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.134 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.134 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.134 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.135 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.135 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.135 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.135 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.135 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.135 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.135 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.135 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.135 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.136 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.136 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.136 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.136 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.136 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.136 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.136 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.136 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.137 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.137 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.137 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.137 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.137 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.137 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.137 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.137 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.138 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.138 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.138 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.138 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.138 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.138 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.138 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.138 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.138 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.139 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.139 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.139 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.139 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.139 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.139 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.139 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.139 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.139 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.140 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.140 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.140 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.140 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.140 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.140 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.140 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.140 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.140 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.141 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.141 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.141 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.141 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.141 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.141 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.141 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.141 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-09-21T06:25:57.142 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-09-21T06:25:57.142 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.142 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.142 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.142 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.142 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.142 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.142 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.142 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.143 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.143 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.143 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.143 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.143 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-09-21T06:25:57.143 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-09-21T06:25:57.143 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.143 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.143 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.144 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-09-21T06:25:57.144 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.144 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.144 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.144 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.144 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.144 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.144 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.145 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.145 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-09-21T06:25:57.145 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-09-21T06:25:57.145 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.145 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.145 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.145 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.145 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.145 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.146 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.146 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.146 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.146 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.146 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.146 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.146 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-09-21T06:25:57.146 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-09-21T06:25:57.146 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.147 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.147 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.147 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.147 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.147 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.147 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.147 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.147 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.148 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.148 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.148 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.148 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-09-21T06:25:57.148 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-09-21T06:25:57.148 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:57.148 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.148 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.148 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.149 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.149 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.149 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.149 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.149 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.149 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.149 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.149 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.150 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-09-21T06:25:57.150 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-09-21T06:25:57.150 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.150 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.150 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.150 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.150 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.150 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.150 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.151 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.151 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.151 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.151 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.151 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.151 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-09-21T06:25:57.151 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-09-21T06:25:57.151 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.152 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.152 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.152 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.152 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.152 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.152 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.152 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.152 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.152 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.153 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.153 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.153 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-09-21T06:25:57.153 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-09-21T06:25:57.153 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.153 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.153 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.153 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.154 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.154 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.154 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.154 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.154 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.154 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.154 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.154 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.154 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-09-21T06:25:57.155 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-09-21T06:25:57.155 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.155 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.155 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.155 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.155 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-21T06:25:57.155 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-09-21T06:25:57.155 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.156 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.156 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.156 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.156 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.156 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.156 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.156 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.156 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.156 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.157 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.157 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.157 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.157 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.157 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.157 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.157 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.157 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.157 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.158 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.158 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-09-21T06:25:57.158 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-09-21T06:25:57.158 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.158 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.158 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.158 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.158 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.158 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.159 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.159 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.159 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.159 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.159 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.159 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.159 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-09-21T06:25:57.159 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-09-21T06:25:57.159 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.160 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.160 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.160 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.160 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.160 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.160 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.160 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.160 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.161 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.161 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.161 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.161 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-09-21T06:25:57.161 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-09-21T06:25:57.161 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.161 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.161 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.161 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.162 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.162 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.162 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.162 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.162 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.162 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.162 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.162 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.163 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.163 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.163 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.163 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smb", 2024-09-21T06:25:57.163 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.163 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.163 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.163 INFO:teuthology.orchestra.run.smithi003.stdout: "internal_store_backend": { 2024-09-21T06:25:57.163 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "internal_store_backend", 2024-09-21T06:25:57.164 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.164 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.164 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.164 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.164 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.164 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.164 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.164 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T06:25:57.165 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.165 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.165 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.165 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.165 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.165 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.165 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.165 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.165 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.166 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.166 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.166 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.166 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.166 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.166 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.166 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.166 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.166 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.167 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.167 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.167 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.167 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.167 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.167 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.167 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.167 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.167 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.168 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.168 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.168 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.168 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.168 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.168 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.168 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.168 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.168 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.169 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.169 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.169 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.169 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.169 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.169 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.169 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.169 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.169 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.170 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.170 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.170 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.170 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.170 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.170 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.170 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.170 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.171 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.171 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.171 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.171 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.171 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.171 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.171 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.171 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.171 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.172 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.172 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.172 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.172 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.172 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.172 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.172 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.172 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.173 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.173 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.173 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.173 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.173 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.173 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.173 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.173 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.173 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.173 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.174 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.174 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.174 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.174 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.174 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.174 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.174 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.174 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.175 INFO:teuthology.orchestra.run.smithi003.stdout: "update_orchestration": { 2024-09-21T06:25:57.175 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "update_orchestration", 2024-09-21T06:25:57.175 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.175 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.175 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.175 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.175 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.175 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.176 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.176 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T06:25:57.176 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.176 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.176 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.176 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.176 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.176 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.176 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.176 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-09-21T06:25:57.177 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.177 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.177 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.177 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-09-21T06:25:57.177 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-09-21T06:25:57.177 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.177 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.177 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.177 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.178 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.178 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.178 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.178 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T06:25:57.178 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.178 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.178 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.178 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.178 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-09-21T06:25:57.179 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-09-21T06:25:57.179 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.179 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.179 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.179 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.179 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.179 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.179 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.179 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-09-21T06:25:57.179 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.180 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.180 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.180 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.180 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.180 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.180 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.180 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.180 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.180 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.181 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.181 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.181 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.181 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.181 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.181 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.181 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.181 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.181 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.182 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.182 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.182 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.182 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.182 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.182 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.182 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.182 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.182 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.183 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.183 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.183 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.183 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.183 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.183 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.183 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.184 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.184 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.184 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.184 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.184 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.185 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.185 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.185 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.185 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.185 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.185 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.185 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.185 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.185 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.186 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.186 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.186 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.186 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.186 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.186 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.186 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.186 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.186 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.187 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.187 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.187 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.187 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.187 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.187 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.187 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.187 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.187 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.188 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.188 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.188 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.188 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.188 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.188 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.188 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.188 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.189 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.189 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.189 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.189 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.189 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.189 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.189 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.189 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.189 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.190 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.190 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-09-21T06:25:57.190 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.190 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.190 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.190 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.190 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.190 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.191 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.191 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.191 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.191 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.191 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.191 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.191 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.191 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.192 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.192 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.192 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.192 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.192 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.192 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.192 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.192 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.192 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.193 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.193 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.193 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.193 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.193 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.193 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.193 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.193 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.194 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.194 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.194 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.194 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.194 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.194 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.194 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.194 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.194 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.195 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.195 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.195 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.195 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.195 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.195 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.195 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.195 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.196 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.196 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.196 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.196 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.196 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.196 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.196 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.196 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.196 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.196 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.197 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.197 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.197 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.197 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.197 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.197 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.197 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.197 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.198 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.198 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.198 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.198 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.198 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.198 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.198 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.198 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.198 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.199 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.199 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.199 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.199 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.199 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.199 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.199 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.199 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.199 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.200 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.200 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.200 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.200 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.200 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.200 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.200 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-09-21T06:25:57.200 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.200 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.201 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.201 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.201 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.201 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.201 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.201 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.201 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.201 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.201 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.201 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.202 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.202 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.202 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.202 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.202 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.202 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.202 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.202 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.202 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.203 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.203 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.203 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.203 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.203 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.203 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.203 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.203 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.204 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.204 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.204 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.204 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.204 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.204 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.204 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.204 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.204 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.204 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.205 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.205 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.205 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.205 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.205 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.205 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.205 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.205 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.205 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.206 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.206 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.206 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.206 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.206 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.206 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.206 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.206 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.206 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.206 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.208 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.208 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.208 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.208 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.208 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.208 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.208 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.208 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.208 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.210 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.210 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.210 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.210 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.210 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.210 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-09-21T06:25:57.210 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.210 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.211 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.211 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-09-21T06:25:57.211 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-09-21T06:25:57.211 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.211 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.211 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.211 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T06:25:57.211 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.211 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.213 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.213 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-09-21T06:25:57.213 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.213 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.213 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.213 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.213 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.213 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.213 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.214 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.214 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.214 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.214 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.214 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.214 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.214 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.214 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.214 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.215 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.215 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.215 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.215 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.215 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.215 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.215 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.215 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.215 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.216 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.216 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.216 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.216 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.216 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.216 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.216 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.216 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.216 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.218 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.218 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.218 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.218 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.218 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.218 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.218 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.218 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.218 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.220 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.220 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.220 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.220 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.220 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.220 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.220 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.220 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.222 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.222 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.222 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.222 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.222 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.222 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.222 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.222 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.222 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.223 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.223 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.223 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.223 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.223 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.223 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.223 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-09-21T06:25:57.223 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.224 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.224 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.224 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-09-21T06:25:57.224 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-09-21T06:25:57.224 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.224 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.224 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.224 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.225 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.225 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.225 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.225 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T06:25:57.225 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.225 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.225 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.225 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.225 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-09-21T06:25:57.226 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-09-21T06:25:57.226 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.226 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.226 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.226 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.226 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.226 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.226 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.227 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T06:25:57.227 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.227 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.227 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.227 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.227 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-09-21T06:25:57.227 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-09-21T06:25:57.227 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.228 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.228 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.228 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.228 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.228 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.228 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.228 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T06:25:57.228 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.228 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.229 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.229 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.229 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-09-21T06:25:57.229 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-09-21T06:25:57.229 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.229 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.229 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.229 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.229 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.230 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.230 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.230 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T06:25:57.230 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.230 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.230 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.230 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.230 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-09-21T06:25:57.231 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-09-21T06:25:57.231 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.231 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.231 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.231 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.231 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.231 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.231 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T06:25:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-09-21T06:25:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-09-21T06:25:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.232 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.233 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-09-21T06:25:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-09-21T06:25:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.234 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-09-21T06:25:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-09-21T06:25:57.235 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.236 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.236 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.236 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T06:25:57.236 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.236 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.236 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.236 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.236 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.237 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.237 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.237 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.237 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-09-21T06:25:57.237 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-09-21T06:25:57.237 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.237 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.237 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.237 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.238 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.238 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.238 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.238 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.238 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.238 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.238 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.238 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.238 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:57.239 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:57.239 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.239 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.239 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.239 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-09-21T06:25:57.239 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-09-21T06:25:57.239 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.239 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.239 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.240 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.240 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.240 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.240 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.240 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-09-21T06:25:57.240 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-09-21T06:25:57.240 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.240 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.241 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.241 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-09-21T06:25:57.241 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.241 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.241 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.241 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.241 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.241 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.241 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.242 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.242 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-09-21T06:25:57.242 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-09-21T06:25:57.242 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.242 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.242 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.242 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.242 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.242 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.243 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.243 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.243 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.243 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.243 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.243 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.243 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-09-21T06:25:57.243 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-09-21T06:25:57.244 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.244 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.244 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.244 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.244 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.244 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.244 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.244 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.245 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.245 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.245 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.245 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.245 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.245 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.245 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.245 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.245 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.246 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.246 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.246 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.246 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.246 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.246 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.246 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.246 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.246 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.247 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.247 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.247 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.247 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.247 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.247 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.247 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.247 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.247 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.248 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.248 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.248 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.248 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.248 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.248 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.248 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.248 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.248 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.249 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.249 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.249 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.249 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.249 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.249 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.249 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.249 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.249 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.250 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.250 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.250 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.250 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.250 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.250 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.250 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.250 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.250 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.251 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.251 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.251 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.251 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.251 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.251 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.251 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.251 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.251 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.252 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.252 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.252 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.252 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.252 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.252 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.252 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.252 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.252 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.253 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.253 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.253 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-09-21T06:25:57.253 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-09-21T06:25:57.253 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.253 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.253 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.253 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.253 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.253 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.254 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.254 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.254 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.254 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.254 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.254 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.254 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-09-21T06:25:57.254 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-09-21T06:25:57.254 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.255 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.255 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.255 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.255 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.255 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.255 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.255 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.255 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.255 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.256 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.256 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.256 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.256 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.256 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.256 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.256 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.256 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.256 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.257 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.257 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.257 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.257 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.257 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.257 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.257 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.257 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-09-21T06:25:57.257 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-09-21T06:25:57.257 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.258 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.258 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.258 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T06:25:57.258 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.258 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.258 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.258 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.258 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.259 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.259 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.259 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.259 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.259 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.259 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.259 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-09-21T06:25:57.259 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.259 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.260 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.260 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.260 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.260 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.260 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.260 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.260 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.260 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.260 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.260 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.261 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.261 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.261 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.261 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.261 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.261 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.261 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.261 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.261 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.262 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.262 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.262 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.262 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.262 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.262 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.262 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.262 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.263 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.263 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.263 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.263 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.263 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.263 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.263 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.263 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.264 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.264 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.264 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.264 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.264 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.264 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.264 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.264 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.264 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.265 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.265 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.265 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.265 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.265 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.265 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.265 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.265 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.265 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.266 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.266 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.266 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.266 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.266 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.266 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.266 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.266 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.267 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.267 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.267 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.267 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.267 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.267 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.267 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.267 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.267 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.268 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.268 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.268 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.268 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.268 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.268 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.268 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.268 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.268 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.269 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.269 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.269 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.269 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.269 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.269 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.269 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.269 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.270 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.270 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-09-21T06:25:57.270 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.270 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.270 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.270 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.270 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.270 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.270 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.271 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.271 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.271 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.271 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.271 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.271 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.271 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.271 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.272 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.272 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.272 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.272 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.272 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.272 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.272 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.272 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.272 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.273 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.273 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.273 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.273 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.273 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.273 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.273 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.273 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.274 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.274 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.274 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.274 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.274 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.274 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.274 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.274 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.275 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.275 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.275 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.275 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.275 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.275 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.275 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.275 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.275 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.276 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.276 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.276 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.276 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.276 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.276 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.276 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.276 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.276 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.277 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.277 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.277 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.277 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.277 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.277 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.277 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.277 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.278 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.278 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.278 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.278 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.278 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.278 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.278 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.278 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-09-21T06:25:57.279 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-09-21T06:25:57.279 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.279 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.279 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.279 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-09-21T06:25:57.279 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.279 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.279 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.279 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T06:25:57.280 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.280 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.280 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.280 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.280 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-09-21T06:25:57.280 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-09-21T06:25:57.280 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.280 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.281 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.281 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.281 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.281 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.281 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.281 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-09-21T06:25:57.281 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.281 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.281 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.282 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.282 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-09-21T06:25:57.282 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-09-21T06:25:57.282 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.282 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.282 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.282 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.282 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.283 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.283 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.283 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T06:25:57.283 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.283 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.283 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.283 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.283 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-09-21T06:25:57.283 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T06:25:57.284 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.284 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.284 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.284 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.284 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.284 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.284 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.284 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T06:25:57.284 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.285 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.285 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.285 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.285 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.285 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.285 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.285 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.285 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.285 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.286 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.286 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.286 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.286 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.286 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.286 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.286 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.286 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.286 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.287 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.287 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.287 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-09-21T06:25:57.287 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.287 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.287 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.287 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-09-21T06:25:57.287 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-09-21T06:25:57.287 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:57.288 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.288 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.288 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-09-21T06:25:57.288 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.288 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.288 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.288 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.288 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.289 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.289 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.289 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.289 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-09-21T06:25:57.289 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-09-21T06:25:57.289 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.289 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.289 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.289 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.290 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.290 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.290 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.290 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.290 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.290 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.290 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.290 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.291 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:57.291 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:57.291 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.291 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.291 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.291 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:57.291 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.291 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.291 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.292 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.292 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.292 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.292 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.292 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.292 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.292 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.292 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.292 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.293 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.293 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.293 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.293 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.293 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.293 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.293 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.293 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.294 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.294 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.294 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.294 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.294 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.294 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.294 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.294 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.294 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.295 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.295 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.295 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.295 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.295 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.295 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.295 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.295 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.295 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.296 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.296 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.296 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.296 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.296 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.296 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.296 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.296 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.296 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.297 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.297 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.297 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.297 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.297 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.297 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.297 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.297 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.298 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.298 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.298 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.298 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.298 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.298 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.298 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.298 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.298 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.299 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.299 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.299 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.299 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.299 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.299 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.299 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.300 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.300 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.300 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.300 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.300 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.300 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.300 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.300 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.300 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.300 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.301 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.301 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.301 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.301 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.301 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.301 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.301 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.301 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.301 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.302 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.302 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.302 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-09-21T06:25:57.302 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-09-21T06:25:57.302 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.302 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.302 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.302 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.302 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.303 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.303 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.303 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.303 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.303 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.303 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.303 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.303 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-09-21T06:25:57.303 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-09-21T06:25:57.303 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.304 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.304 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.304 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-09-21T06:25:57.304 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.304 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.304 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.304 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.304 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.304 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.305 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.305 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.305 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-09-21T06:25:57.305 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-09-21T06:25:57.305 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.305 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.305 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.305 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T06:25:57.305 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.306 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.306 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.306 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.306 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.306 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.306 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.306 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.306 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.306 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.307 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-21T06:25:57.307 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.307 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.307 INFO:teuthology.orchestra.run.smithi003.stdout: "gid": 24164, 2024-09-21T06:25:57.307 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "y", 2024-09-21T06:25:57.307 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_features": 4540719139924082687, 2024-09-21T06:25:57.307 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-09-21T06:25:57.307 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.307 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-09-21T06:25:57.308 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.308 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.308 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.308 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:57.308 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:57.308 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.308 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.308 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.308 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:57.309 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.309 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.309 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.309 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T06:25:57.309 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.309 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.309 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.309 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.310 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.310 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.310 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.310 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.310 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.310 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.310 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.310 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.311 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.311 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.311 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.311 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.311 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.311 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.311 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.311 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.311 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.311 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.312 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.312 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.312 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.312 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.312 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.312 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.312 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.312 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.312 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.313 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.313 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.313 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.313 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.313 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.313 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.313 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.313 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.313 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.314 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.314 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.314 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.314 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.314 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.314 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.314 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.314 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.315 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.315 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.315 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.315 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.315 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.315 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.315 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.315 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.315 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.316 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.316 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.316 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.316 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.316 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.316 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.316 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.316 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.316 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.316 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.317 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.317 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.317 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.317 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.317 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.317 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.317 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.317 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-09-21T06:25:57.317 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-09-21T06:25:57.318 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.318 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.318 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.318 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.318 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.318 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.318 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.318 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T06:25:57.318 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.319 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.319 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.319 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.319 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-09-21T06:25:57.319 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-09-21T06:25:57.319 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.319 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.319 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.320 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-09-21T06:25:57.320 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.320 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.320 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.320 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-09-21T06:25:57.320 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.320 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.320 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.320 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.321 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-09-21T06:25:57.321 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-09-21T06:25:57.321 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.321 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.321 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.321 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.321 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.321 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.321 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.322 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-09-21T06:25:57.322 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.322 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.322 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.322 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.322 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-09-21T06:25:57.322 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-09-21T06:25:57.322 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.323 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.323 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.323 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.323 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.323 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.323 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.323 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-09-21T06:25:57.323 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.323 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.324 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.324 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.324 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-09-21T06:25:57.324 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-09-21T06:25:57.324 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.324 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.324 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.324 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-09-21T06:25:57.324 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.325 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.325 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.325 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-09-21T06:25:57.325 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.325 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.325 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.325 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.325 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-09-21T06:25:57.325 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-09-21T06:25:57.325 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.326 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.326 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.326 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.326 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.326 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.326 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.326 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-09-21T06:25:57.326 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.326 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.327 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.327 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.327 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-09-21T06:25:57.327 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-09-21T06:25:57.327 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.327 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.327 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.327 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.328 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.328 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.328 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.328 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T06:25:57.328 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.328 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.328 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.328 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.328 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-09-21T06:25:57.329 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-09-21T06:25:57.329 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.329 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.329 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.329 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.329 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.329 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.329 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.329 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-09-21T06:25:57.329 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.330 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.330 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.330 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.330 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.330 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.330 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.330 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.330 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.330 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.331 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.331 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.331 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.331 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.331 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.331 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.331 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.331 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.331 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.332 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.332 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.332 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-09-21T06:25:57.332 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.332 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.332 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.332 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-09-21T06:25:57.332 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-09-21T06:25:57.332 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.333 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.333 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.333 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.333 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.333 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.333 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.333 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T06:25:57.333 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.333 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.333 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.334 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.334 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-09-21T06:25:57.334 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-09-21T06:25:57.334 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.334 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.334 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.334 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-09-21T06:25:57.334 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.334 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.335 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.335 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T06:25:57.335 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:25:57.335 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.335 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.335 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.335 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-09-21T06:25:57.335 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-09-21T06:25:57.336 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:57.336 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.336 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.336 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.336 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-09-21T06:25:57.336 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-09-21T06:25:57.336 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.336 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T06:25:57.336 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:25:57.337 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.337 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.337 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.337 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-09-21T06:25:57.337 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-09-21T06:25:57.337 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:57.337 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.337 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.338 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-09-21T06:25:57.338 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-21T06:25:57.338 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-09-21T06:25:57.338 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.338 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T06:25:57.338 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.338 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.338 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.338 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.339 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-09-21T06:25:57.339 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-09-21T06:25:57.339 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.339 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.339 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.339 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T06:25:57.339 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.339 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.339 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.340 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T06:25:57.340 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T06:25:57.340 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.340 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.340 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.340 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-09-21T06:25:57.340 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-09-21T06:25:57.340 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:57.341 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.341 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.341 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-09-21T06:25:57.341 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-09-21T06:25:57.341 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-09-21T06:25:57.341 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.341 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-09-21T06:25:57.341 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T06:25:57.341 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.342 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.342 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.342 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-09-21T06:25:57.342 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-09-21T06:25:57.342 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.342 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.342 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.342 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-09-21T06:25:57.342 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.343 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.343 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.343 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T06:25:57.343 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:25:57.343 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.343 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.343 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.343 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-09-21T06:25:57.344 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-09-21T06:25:57.344 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:57.344 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.344 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.344 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.344 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-09-21T06:25:57.344 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-09-21T06:25:57.344 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.344 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T06:25:57.345 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:25:57.345 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.345 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.345 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.345 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.345 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.345 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.345 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.345 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.346 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.346 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.346 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.346 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.346 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.346 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.346 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.346 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.347 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.347 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.347 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.347 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.347 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.347 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.347 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.347 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.347 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.348 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.348 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.348 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.348 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.348 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.348 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.348 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.348 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.349 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.349 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.349 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.349 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.349 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.349 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.349 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.349 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.349 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.350 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.350 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.350 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.350 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.350 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.350 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.350 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.350 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.350 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.351 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.351 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.351 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.351 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.351 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.351 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.351 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.351 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.351 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.352 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.352 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.352 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.352 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.352 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.352 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.352 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.352 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.352 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.353 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.353 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.353 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.353 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.353 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-09-21T06:25:57.353 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-09-21T06:25:57.353 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:57.353 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.354 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.354 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.354 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.354 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.354 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.354 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T06:25:57.354 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.354 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.355 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.355 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.355 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-09-21T06:25:57.355 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-09-21T06:25:57.355 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.355 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.355 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.355 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-09-21T06:25:57.355 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.356 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.356 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.356 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-09-21T06:25:57.356 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-09-21T06:25:57.356 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-09-21T06:25:57.356 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-09-21T06:25:57.356 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-09-21T06:25:57.356 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.356 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-09-21T06:25:57.357 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.357 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.357 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.357 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.357 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-09-21T06:25:57.357 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-09-21T06:25:57.357 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.357 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.358 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.358 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.358 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.358 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.358 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.358 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T06:25:57.358 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.358 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.359 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.359 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.359 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:57.359 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:57.359 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.359 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.359 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.359 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:57.359 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.360 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.360 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.360 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T06:25:57.360 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.360 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.360 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.360 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.360 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.360 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.361 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.361 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.361 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.361 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.361 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.361 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.361 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.361 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.362 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.362 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.362 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.362 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.362 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-09-21T06:25:57.362 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-09-21T06:25:57.362 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.362 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.362 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.363 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:57.363 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-21T06:25:57.363 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.363 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.363 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T06:25:57.363 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T06:25:57.363 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.363 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.364 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.364 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-09-21T06:25:57.364 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-09-21T06:25:57.364 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:57.364 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.364 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.364 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:57.364 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.364 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.365 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.365 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T06:25:57.365 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.365 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.365 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.365 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.365 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.365 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.366 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-09-21T06:25:57.366 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.366 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.366 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.366 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-09-21T06:25:57.366 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-09-21T06:25:57.366 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:57.366 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.367 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.367 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-09-21T06:25:57.367 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.367 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.367 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.367 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T06:25:57.367 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.367 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.367 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.368 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.368 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-09-21T06:25:57.368 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-09-21T06:25:57.368 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.368 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.368 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.368 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-09-21T06:25:57.368 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.369 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.369 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.369 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T06:25:57.369 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.369 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.369 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.369 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.369 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-09-21T06:25:57.370 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-09-21T06:25:57.370 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.370 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.370 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.370 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-09-21T06:25:57.370 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.370 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.370 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.371 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T06:25:57.371 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.371 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.371 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.371 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.371 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-09-21T06:25:57.371 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-09-21T06:25:57.371 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.371 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.372 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.372 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.372 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.372 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.372 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.372 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T06:25:57.372 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T06:25:57.372 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.373 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.373 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.373 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-09-21T06:25:57.373 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-09-21T06:25:57.373 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.373 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.373 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.373 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:57.374 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.374 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.374 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.374 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T06:25:57.374 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.374 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.374 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.374 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.374 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-09-21T06:25:57.375 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T06:25:57.375 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:57.375 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.375 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.375 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-09-21T06:25:57.375 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.375 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.375 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.376 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T06:25:57.376 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.376 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.376 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.376 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.376 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-09-21T06:25:57.376 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-09-21T06:25:57.376 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.377 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.377 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.377 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.377 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.377 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.377 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.377 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-09-21T06:25:57.377 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-09-21T06:25:57.377 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-09-21T06:25:57.378 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.378 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T06:25:57.378 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.378 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.378 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.378 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.378 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-09-21T06:25:57.378 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-09-21T06:25:57.378 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.379 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.379 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.379 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.379 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.379 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.379 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.379 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T06:25:57.379 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.379 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.379 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.380 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.380 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-09-21T06:25:57.380 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-09-21T06:25:57.380 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.380 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.380 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.380 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.380 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.380 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.381 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.381 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T06:25:57.381 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.381 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.381 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.381 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.381 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-09-21T06:25:57.381 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-09-21T06:25:57.381 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.382 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.382 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.382 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.382 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.382 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.382 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.382 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T06:25:57.382 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.383 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.383 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.383 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.383 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-09-21T06:25:57.383 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-09-21T06:25:57.383 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.383 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.383 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.383 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T06:25:57.383 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.384 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.384 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.384 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-21T06:25:57.384 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.384 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.384 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.384 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.384 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-09-21T06:25:57.385 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-09-21T06:25:57.385 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.385 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.385 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.385 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T06:25:57.385 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.385 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.385 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.386 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-09-21T06:25:57.386 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.386 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.386 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.386 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.386 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-09-21T06:25:57.386 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-09-21T06:25:57.386 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.386 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.387 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.387 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T06:25:57.387 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.387 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.387 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.387 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-09-21T06:25:57.387 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.387 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.387 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.388 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.388 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-09-21T06:25:57.388 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-09-21T06:25:57.388 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.388 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.388 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.388 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T06:25:57.388 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.388 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.389 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.389 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-21T06:25:57.389 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.389 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.389 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.389 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.389 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-09-21T06:25:57.389 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-09-21T06:25:57.390 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.390 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.390 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.390 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T06:25:57.390 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.390 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.390 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.390 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-09-21T06:25:57.390 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.391 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.391 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.391 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.391 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-09-21T06:25:57.391 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-09-21T06:25:57.391 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.391 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.391 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.392 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T06:25:57.392 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.392 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.392 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.392 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-09-21T06:25:57.392 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.392 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.392 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.392 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.393 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-09-21T06:25:57.393 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-09-21T06:25:57.393 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.393 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.393 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.393 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T06:25:57.393 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.393 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.393 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.394 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-09-21T06:25:57.394 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.394 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.394 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.394 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.394 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-09-21T06:25:57.394 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-09-21T06:25:57.394 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.394 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.395 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.395 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T06:25:57.395 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.395 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.395 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.395 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-09-21T06:25:57.395 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.395 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.395 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.396 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.396 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-09-21T06:25:57.396 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-09-21T06:25:57.396 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.396 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.396 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.396 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T06:25:57.396 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.396 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.396 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.397 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-09-21T06:25:57.397 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.397 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.397 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.397 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.397 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-09-21T06:25:57.397 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-09-21T06:25:57.397 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.398 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.398 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.398 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T06:25:57.398 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.398 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.398 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.398 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-09-21T06:25:57.398 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.398 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.399 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.399 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.399 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nginx": { 2024-09-21T06:25:57.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nginx", 2024-09-21T06:25:57.399 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.399 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.399 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.399 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T06:25:57.400 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.400 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.400 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.400 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nginx container image", 2024-09-21T06:25:57.400 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.400 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.400 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.400 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.400 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-09-21T06:25:57.401 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-09-21T06:25:57.401 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.401 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.401 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.401 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T06:25:57.401 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.401 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.401 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.401 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-21T06:25:57.402 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.402 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.402 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.402 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.402 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-09-21T06:25:57.402 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-09-21T06:25:57.402 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.402 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.402 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.403 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T06:25:57.403 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.403 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.403 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.403 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-09-21T06:25:57.403 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.403 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.403 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.404 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.404 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_oauth2_proxy": { 2024-09-21T06:25:57.404 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T06:25:57.404 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.404 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.404 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.404 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T06:25:57.404 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.404 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.405 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.405 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "oauth2-proxy container image", 2024-09-21T06:25:57.405 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.405 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.405 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.405 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.405 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-09-21T06:25:57.405 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-09-21T06:25:57.405 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.406 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.406 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.406 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T06:25:57.406 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.406 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.406 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.406 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-21T06:25:57.406 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.406 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-09-21T06:25:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-09-21T06:25:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T06:25:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.408 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.408 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.408 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-09-21T06:25:57.408 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.408 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.408 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.408 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.408 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-09-21T06:25:57.408 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-09-21T06:25:57.408 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.409 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.409 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.409 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T06:25:57.409 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.409 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.409 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.409 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-09-21T06:25:57.409 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.409 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.410 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.410 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.410 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba_metrics": { 2024-09-21T06:25:57.410 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba_metrics", 2024-09-21T06:25:57.410 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.410 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.410 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.410 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T06:25:57.410 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.411 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.411 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.411 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T06:25:57.411 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.411 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.411 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.411 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.411 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-09-21T06:25:57.411 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-09-21T06:25:57.412 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.412 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.412 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.412 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T06:25:57.412 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.412 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.412 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.412 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-09-21T06:25:57.413 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.413 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.413 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.413 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.413 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-09-21T06:25:57.413 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-09-21T06:25:57.413 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.413 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.413 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.414 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.414 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.414 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.414 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.414 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T06:25:57.414 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.414 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.414 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-09-21T06:25:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-09-21T06:25:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T06:25:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-09-21T06:25:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T06:25:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-09-21T06:25:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T06:25:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-09-21T06:25:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-09-21T06:25:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-09-21T06:25:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T06:25:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-09-21T06:25:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-09-21T06:25:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-09-21T06:25:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-09-21T06:25:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-09-21T06:25:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-09-21T06:25:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T06:25:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-09-21T06:25:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-09-21T06:25:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-09-21T06:25:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-09-21T06:25:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-09-21T06:25:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T06:25:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T06:25:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-09-21T06:25:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-09-21T06:25:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-09-21T06:25:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-09-21T06:25:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-09-21T06:25:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T06:25:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-09-21T06:25:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-09-21T06:25:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T06:25:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-09-21T06:25:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-09-21T06:25:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T06:25:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T06:25:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.441 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.441 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T06:25:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T06:25:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T06:25:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T06:25:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T06:25:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-09-21T06:25:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T06:25:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-09-21T06:25:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-09-21T06:25:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-09-21T06:25:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-09-21T06:25:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-09-21T06:25:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T06:25:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-09-21T06:25:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-09-21T06:25:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-09-21T06:25:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-09-21T06:25:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-09-21T06:25:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-09-21T06:25:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-09-21T06:25:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-09-21T06:25:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T06:25:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-09-21T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-09-21T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-09-21T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T06:25:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-09-21T06:25:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-09-21T06:25:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T06:25:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T06:25:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-09-21T06:25:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-09-21T06:25:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T06:25:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-09-21T06:25:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-09-21T06:25:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T06:25:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-09-21T06:25:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-09-21T06:25:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T06:25:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-09-21T06:25:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-09-21T06:25:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T06:25:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-09-21T06:25:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-09-21T06:25:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T06:25:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-09-21T06:25:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-09-21T06:25:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-09-21T06:25:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-09-21T06:25:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-09-21T06:25:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-09-21T06:25:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T06:25:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_keepalive_count_max": { 2024-09-21T06:25:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T06:25:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-09-21T06:25:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T06:25:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_keepalive_interval": { 2024-09-21T06:25:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_keepalive_interval", 2024-09-21T06:25:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-09-21T06:25:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T06:25:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-09-21T06:25:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-09-21T06:25:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T06:25:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-09-21T06:25:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-09-21T06:25:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T06:25:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-09-21T06:25:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-09-21T06:25:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T06:25:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-09-21T06:25:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-09-21T06:25:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T06:25:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-09-21T06:25:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-09-21T06:25:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T06:25:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-09-21T06:25:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-09-21T06:25:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-09-21T06:25:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-09-21T06:25:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T06:25:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-09-21T06:25:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-09-21T06:25:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-09-21T06:25:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T06:25:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-09-21T06:25:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T06:25:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T06:25:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T06:25:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T06:25:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T06:25:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T06:25:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-09-21T06:25:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T06:25:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T06:25:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T06:25:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T06:25:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T06:25:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T06:25:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T06:25:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T06:25:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T06:25:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T06:25:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T06:25:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T06:25:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T06:25:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T06:25:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T06:25:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T06:25:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T06:25:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T06:25:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T06:25:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T06:25:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T06:25:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-21T06:25:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T06:25:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T06:25:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-09-21T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-09-21T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T06:25:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T06:25:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-21T06:25:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T06:25:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T06:25:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T06:25:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T06:25:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T06:25:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T06:25:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T06:25:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T06:25:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T06:25:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T06:25:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "[]", 2024-09-21T06:25:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T06:25:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T06:25:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "{}", 2024-09-21T06:25:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T06:25:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T06:25:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T06:25:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T06:25:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T06:25:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T06:25:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T06:25:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T06:25:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T06:25:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T06:25:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T06:25:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T06:25:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T06:25:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T06:25:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T06:25:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T06:25:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T06:25:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T06:25:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T06:25:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T06:25:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T06:25:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T06:25:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T06:25:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T06:25:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T06:25:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T06:25:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T06:25:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-09-21T06:25:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T06:25:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T06:25:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-09-21T06:25:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T06:25:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T06:25:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T06:25:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T06:25:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-21T06:25:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T06:25:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T06:25:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T06:25:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T06:25:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T06:25:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T06:25:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T06:25:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T06:25:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T06:25:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T06:25:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T06:25:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T06:25:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-09-21T06:25:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-09-21T06:25:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-09-21T06:25:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-09-21T06:25:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-09-21T06:25:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-09-21T06:25:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-09-21T06:25:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-09-21T06:25:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-09-21T06:25:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-09-21T06:25:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-09-21T06:25:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-09-21T06:25:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-09-21T06:25:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-09-21T06:25:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-09-21T06:25:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-09-21T06:25:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T06:25:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-21T06:25:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-21T06:25:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-09-21T06:25:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-21T06:25:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-21T06:25:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-09-21T06:25:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-09-21T06:25:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-09-21T06:25:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-09-21T06:25:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-09-21T06:25:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-09-21T06:25:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "sso_oauth2": { 2024-09-21T06:25:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sso_oauth2", 2024-09-21T06:25:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-09-21T06:25:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-09-21T06:25:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-09-21T06:25:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-09-21T06:25:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-09-21T06:25:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-09-21T06:25:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-21T06:25:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-09-21T06:25:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-09-21T06:25:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-09-21T06:25:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-09-21T06:25:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-09-21T06:25:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-09-21T06:25:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-09-21T06:25:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-09-21T06:25:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-09-21T06:25:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-09-21T06:25:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-09-21T06:25:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T06:25:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-09-21T06:25:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-09-21T06:25:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-09-21T06:25:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T06:25:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-09-21T06:25:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-09-21T06:25:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-09-21T06:25:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-09-21T06:25:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-09-21T06:25:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-09-21T06:25:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-09-21T06:25:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T06:25:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-09-21T06:25:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-09-21T06:25:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T06:25:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T06:25:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-09-21T06:25:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-09-21T06:25:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-09-21T06:25:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T06:25:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-09-21T06:25:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-09-21T06:25:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-09-21T06:25:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-09-21T06:25:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-09-21T06:25:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-09-21T06:25:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-09-21T06:25:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:57.640 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:57.640 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.640 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.641 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.641 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:57.641 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-09-21T06:25:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-09-21T06:25:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-09-21T06:25:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-09-21T06:25:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-09-21T06:25:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-09-21T06:25:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T06:25:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-09-21T06:25:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-09-21T06:25:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-09-21T06:25:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T06:25:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-09-21T06:25:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-09-21T06:25:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-09-21T06:25:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-09-21T06:25:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-09-21T06:25:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T06:25:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-09-21T06:25:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-09-21T06:25:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-09-21T06:25:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-09-21T06:25:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-09-21T06:25:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-09-21T06:25:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-09-21T06:25:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-09-21T06:25:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-09-21T06:25:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-09-21T06:25:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:25:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-09-21T06:25:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-09-21T06:25:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-21T06:25:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-09-21T06:25:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T06:25:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-09-21T06:25:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-09-21T06:25:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-09-21T06:25:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-09-21T06:25:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-09-21T06:25:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-09-21T06:25:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:25:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.672 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.672 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-09-21T06:25:57.672 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.672 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.672 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.672 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.672 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.672 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.672 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.673 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.673 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.673 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.673 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.673 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.673 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.673 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.673 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.673 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.682 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.682 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.682 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.682 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.682 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-09-21T06:25:57.682 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.682 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.682 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.683 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.683 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.683 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.683 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.683 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.683 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.683 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.683 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.683 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.684 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.684 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.684 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.684 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.684 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.684 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.684 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.684 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.685 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.685 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.685 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.685 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.685 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.685 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.685 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.685 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.685 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.686 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.686 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.686 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.686 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.686 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.686 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.686 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-09-21T06:25:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-09-21T06:25:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-09-21T06:25:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-09-21T06:25:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T06:25:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-09-21T06:25:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-09-21T06:25:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-09-21T06:25:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T06:25:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-09-21T06:25:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-09-21T06:25:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-09-21T06:25:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-09-21T06:25:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-09-21T06:25:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T06:25:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-09-21T06:25:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-09-21T06:25:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-09-21T06:25:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T06:25:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-09-21T06:25:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-09-21T06:25:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-09-21T06:25:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-09-21T06:25:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-09-21T06:25:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-09-21T06:25:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-09-21T06:25:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T06:25:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.722 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-09-21T06:25:57.722 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-09-21T06:25:57.722 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.722 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.722 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.722 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.722 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.722 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-09-21T06:25:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-09-21T06:25:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-09-21T06:25:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-09-21T06:25:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T06:25:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T06:25:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-09-21T06:25:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-09-21T06:25:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-09-21T06:25:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-09-21T06:25:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-09-21T06:25:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-09-21T06:25:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T06:25:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-09-21T06:25:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-09-21T06:25:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-09-21T06:25:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-09-21T06:25:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-09-21T06:25:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-09-21T06:25:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-09-21T06:25:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-09-21T06:25:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-09-21T06:25:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-09-21T06:25:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-09-21T06:25:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-09-21T06:25:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-09-21T06:25:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-09-21T06:25:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T06:25:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-09-21T06:25:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-09-21T06:25:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-09-21T06:25:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T06:25:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-09-21T06:25:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-09-21T06:25:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-09-21T06:25:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-09-21T06:25:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-09-21T06:25:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-09-21T06:25:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-09-21T06:25:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-09-21T06:25:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-09-21T06:25:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-09-21T06:25:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-09-21T06:25:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-09-21T06:25:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T06:25:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T06:25:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-09-21T06:25:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-09-21T06:25:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T06:25:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T06:25:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-09-21T06:25:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-09-21T06:25:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-09-21T06:25:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-09-21T06:25:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-21T06:25:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-21T06:25:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-09-21T06:25:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T06:25:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-21T06:25:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-21T06:25:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-09-21T06:25:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T06:25:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-09-21T06:25:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-09-21T06:25:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-09-21T06:25:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-09-21T06:25:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-09-21T06:25:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-09-21T06:25:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-09-21T06:25:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-09-21T06:25:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-09-21T06:25:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-09-21T06:25:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-21T06:25:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-09-21T06:25:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-09-21T06:25:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-09-21T06:25:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-09-21T06:25:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-09-21T06:25:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-09-21T06:25:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T06:25:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-09-21T06:25:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-09-21T06:25:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-09-21T06:25:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-09-21T06:25:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-09-21T06:25:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-09-21T06:25:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-09-21T06:25:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "max_requests": { 2024-09-21T06:25:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_requests", 2024-09-21T06:25:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-21T06:25:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T06:25:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-21T06:25:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-21T06:25:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-21T06:25:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-21T06:25:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-09-21T06:25:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T06:25:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T06:25:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T06:25:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-09-21T06:25:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-09-21T06:25:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-09-21T06:25:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-09-21T06:25:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T06:25:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-09-21T06:25:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-09-21T06:25:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-09-21T06:25:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-09-21T06:25:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-09-21T06:25:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-09-21T06:25:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-09-21T06:25:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-09-21T06:25:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-09-21T06:25:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-09-21T06:25:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-09-21T06:25:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-09-21T06:25:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-09-21T06:25:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-09-21T06:25:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-09-21T06:25:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-09-21T06:25:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-09-21T06:25:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-09-21T06:25:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-09-21T06:25:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-09-21T06:25:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-21T06:25:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-09-21T06:25:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-09-21T06:25:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-09-21T06:25:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-09-21T06:25:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-09-21T06:25:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-09-21T06:25:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-09-21T06:25:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smb", 2024-09-21T06:25:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "internal_store_backend": { 2024-09-21T06:25:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "internal_store_backend", 2024-09-21T06:25:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T06:25:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "update_orchestration": { 2024-09-21T06:25:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "update_orchestration", 2024-09-21T06:25:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T06:25:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-09-21T06:25:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-09-21T06:25:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-09-21T06:25:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T06:25:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-09-21T06:25:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-09-21T06:25:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-09-21T06:25:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-09-21T06:25:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-09-21T06:25:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-09-21T06:25:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-09-21T06:25:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-09-21T06:25:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T06:25:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-09-21T06:25:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.000 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.000 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.001 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.001 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.001 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.001 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.002 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.002 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.002 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-09-21T06:25:58.002 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-09-21T06:25:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-09-21T06:25:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T06:25:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-09-21T06:25:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-09-21T06:25:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T06:25:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-09-21T06:25:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-09-21T06:25:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T06:25:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-09-21T06:25:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-09-21T06:25:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T06:25:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-09-21T06:25:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-09-21T06:25:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T06:25:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-09-21T06:25:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-09-21T06:25:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-09-21T06:25:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-09-21T06:25:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-09-21T06:25:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-09-21T06:25:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T06:25:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-09-21T06:25:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-09-21T06:25:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-09-21T06:25:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-09-21T06:25:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-09-21T06:25:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-09-21T06:25:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-09-21T06:25:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-09-21T06:25:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-09-21T06:25:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-09-21T06:25:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-09-21T06:25:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-09-21T06:25:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-09-21T06:25:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-09-21T06:25:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-09-21T06:25:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-09-21T06:25:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-09-21T06:25:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T06:25:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-09-21T06:25:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-09-21T06:25:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-09-21T06:25:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-09-21T06:25:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-09-21T06:25:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T06:25:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-09-21T06:25:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-09-21T06:25:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-09-21T06:25:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-09-21T06:25:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-09-21T06:25:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T06:25:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-09-21T06:25:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T06:25:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T06:25:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-09-21T06:25:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-09-21T06:25:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-09-21T06:25:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-09-21T06:25:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-09-21T06:25:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-09-21T06:25:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-09-21T06:25:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-09-21T06:25:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-09-21T06:25:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-09-21T06:25:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-09-21T06:25:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-09-21T06:25:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-09-21T06:25:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T06:25:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-21T06:25:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "modules": [ 2024-09-21T06:25:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-09-21T06:25:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "dashboard", 2024-09-21T06:25:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "iostat", 2024-09-21T06:25:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "nfs", 2024-09-21T06:25:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "restful", 2024-09-21T06:25:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_schedule" 2024-09-21T06:25:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-09-21T06:25:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-09-21T06:25:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T06:25:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-09-21T06:25:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-09-21T06:25:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T06:25:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-09-21T06:25:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-09-21T06:25:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-09-21T06:25:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-09-21T06:25:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-09-21T06:25:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-09-21T06:25:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-09-21T06:25:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-09-21T06:25:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-09-21T06:25:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-09-21T06:25:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-09-21T06:25:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-09-21T06:25:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-09-21T06:25:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-09-21T06:25:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-09-21T06:25:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-09-21T06:25:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-09-21T06:25:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-09-21T06:25:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-09-21T06:25:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T06:25:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-09-21T06:25:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-09-21T06:25:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-09-21T06:25:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-09-21T06:25:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-09-21T06:25:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-09-21T06:25:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T06:25:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-09-21T06:25:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-09-21T06:25:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-09-21T06:25:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T06:25:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:25:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-09-21T06:25:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-09-21T06:25:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-09-21T06:25:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-09-21T06:25:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T06:25:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:25:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-09-21T06:25:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-09-21T06:25:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-09-21T06:25:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-21T06:25:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-09-21T06:25:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T06:25:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-09-21T06:25:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-09-21T06:25:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T06:25:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T06:25:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T06:25:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-09-21T06:25:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-09-21T06:25:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-09-21T06:25:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-09-21T06:25:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-09-21T06:25:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-09-21T06:25:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T06:25:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-09-21T06:25:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-09-21T06:25:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-09-21T06:25:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T06:25:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:25:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-09-21T06:25:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-09-21T06:25:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-09-21T06:25:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-09-21T06:25:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T06:25:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:25:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-09-21T06:25:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-09-21T06:25:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T06:25:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-09-21T06:25:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-09-21T06:25:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-09-21T06:25:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-09-21T06:25:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-09-21T06:25:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-09-21T06:25:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-09-21T06:25:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-09-21T06:25:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-09-21T06:25:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-09-21T06:25:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-09-21T06:25:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T06:25:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T06:25:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-09-21T06:25:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-09-21T06:25:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-21T06:25:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T06:25:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T06:25:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-09-21T06:25:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-09-21T06:25:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T06:25:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-09-21T06:25:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-09-21T06:25:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-09-21T06:25:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-09-21T06:25:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T06:25:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-09-21T06:25:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-09-21T06:25:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-09-21T06:25:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T06:25:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-09-21T06:25:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-09-21T06:25:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-09-21T06:25:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T06:25:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-09-21T06:25:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-09-21T06:25:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T06:25:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T06:25:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-09-21T06:25:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-09-21T06:25:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T06:25:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-09-21T06:25:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T06:25:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-09-21T06:25:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T06:25:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-09-21T06:25:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-09-21T06:25:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-09-21T06:25:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-09-21T06:25:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-09-21T06:25:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T06:25:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-09-21T06:25:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-09-21T06:25:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T06:25:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-09-21T06:25:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-09-21T06:25:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T06:25:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-09-21T06:25:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-09-21T06:25:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T06:25:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-09-21T06:25:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-09-21T06:25:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T06:25:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-21T06:25:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-09-21T06:25:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-09-21T06:25:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T06:25:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-09-21T06:25:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-09-21T06:25:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-09-21T06:25:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T06:25:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-09-21T06:25:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-09-21T06:25:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-09-21T06:25:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T06:25:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-21T06:25:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-09-21T06:25:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-09-21T06:25:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T06:25:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-09-21T06:25:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-09-21T06:25:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-09-21T06:25:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T06:25:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-09-21T06:25:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-09-21T06:25:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-09-21T06:25:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T06:25:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-09-21T06:25:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-09-21T06:25:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-09-21T06:25:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T06:25:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-09-21T06:25:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-09-21T06:25:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-09-21T06:25:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T06:25:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-09-21T06:25:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-09-21T06:25:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-09-21T06:25:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T06:25:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-09-21T06:25:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nginx": { 2024-09-21T06:25:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nginx", 2024-09-21T06:25:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T06:25:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nginx container image", 2024-09-21T06:25:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-09-21T06:25:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-09-21T06:25:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T06:25:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-21T06:25:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-09-21T06:25:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-09-21T06:25:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T06:25:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-09-21T06:25:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_oauth2_proxy": { 2024-09-21T06:25:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T06:25:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T06:25:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "oauth2-proxy container image", 2024-09-21T06:25:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-09-21T06:25:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-09-21T06:25:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T06:25:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-21T06:25:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-09-21T06:25:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-09-21T06:25:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T06:25:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-09-21T06:25:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-09-21T06:25:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-09-21T06:25:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T06:25:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-09-21T06:25:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba_metrics": { 2024-09-21T06:25:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba_metrics", 2024-09-21T06:25:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T06:25:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T06:25:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-09-21T06:25:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-09-21T06:25:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T06:25:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-09-21T06:25:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-09-21T06:25:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-09-21T06:25:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T06:25:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-09-21T06:25:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-09-21T06:25:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T06:25:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-09-21T06:25:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T06:25:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-09-21T06:25:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T06:25:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-09-21T06:25:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-09-21T06:25:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-09-21T06:25:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T06:25:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-09-21T06:25:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-09-21T06:25:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-09-21T06:25:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-09-21T06:25:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-09-21T06:25:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-09-21T06:25:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T06:25:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-09-21T06:25:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-09-21T06:25:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-09-21T06:25:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-09-21T06:25:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-09-21T06:25:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T06:25:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T06:25:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-09-21T06:25:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-09-21T06:25:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-09-21T06:25:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-09-21T06:25:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-09-21T06:25:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T06:25:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-09-21T06:25:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-09-21T06:25:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T06:25:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-09-21T06:25:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-09-21T06:25:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T06:25:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T06:25:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T06:25:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T06:25:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T06:25:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T06:25:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T06:25:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-09-21T06:25:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T06:25:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-09-21T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-09-21T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-09-21T06:25:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-09-21T06:25:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-09-21T06:25:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T06:25:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-09-21T06:25:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-09-21T06:25:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-09-21T06:25:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-09-21T06:25:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-09-21T06:25:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-09-21T06:25:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-09-21T06:25:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-09-21T06:25:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T06:25:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-09-21T06:25:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-09-21T06:25:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-09-21T06:25:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T06:25:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-09-21T06:25:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-09-21T06:25:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T06:25:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T06:25:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-09-21T06:25:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-09-21T06:25:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T06:25:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-09-21T06:25:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-09-21T06:25:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T06:25:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-09-21T06:25:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-09-21T06:25:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T06:25:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-09-21T06:25:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-09-21T06:25:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T06:25:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-09-21T06:25:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-09-21T06:25:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T06:25:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-09-21T06:25:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-09-21T06:25:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-09-21T06:25:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-09-21T06:25:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-09-21T06:25:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-09-21T06:25:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T06:25:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_keepalive_count_max": { 2024-09-21T06:25:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T06:25:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-09-21T06:25:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T06:25:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_keepalive_interval": { 2024-09-21T06:25:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_keepalive_interval", 2024-09-21T06:25:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-09-21T06:25:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T06:25:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-09-21T06:25:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-09-21T06:25:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T06:25:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-09-21T06:25:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-09-21T06:25:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T06:25:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-09-21T06:25:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-09-21T06:25:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T06:25:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-09-21T06:25:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-09-21T06:25:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T06:25:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-09-21T06:25:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-09-21T06:25:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T06:25:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-09-21T06:25:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-09-21T06:25:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-09-21T06:25:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-09-21T06:25:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T06:25:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-09-21T06:25:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-09-21T06:25:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-09-21T06:25:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T06:25:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-09-21T06:25:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T06:25:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T06:25:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T06:25:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T06:25:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T06:25:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T06:25:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-09-21T06:25:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T06:25:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T06:25:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T06:25:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T06:25:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T06:25:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T06:25:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T06:25:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T06:25:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T06:25:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T06:25:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T06:25:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T06:25:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T06:25:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.288 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.288 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.288 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.288 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.288 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.288 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.288 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.288 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.288 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T06:25:58.289 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T06:25:58.289 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.289 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.289 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.289 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.289 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.289 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.289 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T06:25:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T06:25:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T06:25:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T06:25:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T06:25:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T06:25:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T06:25:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T06:25:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-21T06:25:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T06:25:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T06:25:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-09-21T06:25:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-09-21T06:25:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T06:25:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T06:25:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-21T06:25:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T06:25:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T06:25:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T06:25:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T06:25:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T06:25:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T06:25:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T06:25:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T06:25:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T06:25:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T06:25:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "[]", 2024-09-21T06:25:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T06:25:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T06:25:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "{}", 2024-09-21T06:25:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T06:25:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T06:25:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T06:25:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T06:25:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T06:25:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T06:25:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T06:25:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T06:25:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T06:25:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T06:25:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T06:25:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T06:25:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T06:25:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T06:25:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T06:25:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T06:25:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T06:25:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T06:25:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T06:25:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T06:25:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T06:25:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T06:25:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T06:25:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T06:25:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T06:25:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T06:25:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T06:25:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-09-21T06:25:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T06:25:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T06:25:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-09-21T06:25:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T06:25:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T06:25:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T06:25:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T06:25:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-21T06:25:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T06:25:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T06:25:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T06:25:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T06:25:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T06:25:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T06:25:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T06:25:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T06:25:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T06:25:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T06:25:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T06:25:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T06:25:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-09-21T06:25:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-09-21T06:25:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-09-21T06:25:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-09-21T06:25:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-09-21T06:25:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-09-21T06:25:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-09-21T06:25:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-09-21T06:25:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-09-21T06:25:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-09-21T06:25:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-09-21T06:25:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-09-21T06:25:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-09-21T06:25:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-09-21T06:25:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-09-21T06:25:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-09-21T06:25:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T06:25:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-21T06:25:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-21T06:25:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-09-21T06:25:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-21T06:25:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-21T06:25:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-09-21T06:25:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-09-21T06:25:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-09-21T06:25:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-09-21T06:25:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-09-21T06:25:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-09-21T06:25:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "sso_oauth2": { 2024-09-21T06:25:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sso_oauth2", 2024-09-21T06:25:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-09-21T06:25:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-09-21T06:25:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-09-21T06:25:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-09-21T06:25:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-09-21T06:25:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-09-21T06:25:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-21T06:25:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-09-21T06:25:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-09-21T06:25:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-09-21T06:25:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-09-21T06:25:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-09-21T06:25:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-09-21T06:25:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-09-21T06:25:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-09-21T06:25:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-09-21T06:25:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-09-21T06:25:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-09-21T06:25:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T06:25:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-09-21T06:25:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-09-21T06:25:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-09-21T06:25:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T06:25:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-09-21T06:25:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-09-21T06:25:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-09-21T06:25:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-09-21T06:25:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-09-21T06:25:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-09-21T06:25:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-09-21T06:25:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T06:25:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-09-21T06:25:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-09-21T06:25:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T06:25:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T06:25:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-09-21T06:25:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-09-21T06:25:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-09-21T06:25:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T06:25:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-09-21T06:25:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-09-21T06:25:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-09-21T06:25:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-09-21T06:25:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-09-21T06:25:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-09-21T06:25:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-09-21T06:25:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-21T06:25:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-09-21T06:25:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-09-21T06:25:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-09-21T06:25:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-09-21T06:25:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-09-21T06:25:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-09-21T06:25:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T06:25:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-09-21T06:25:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-09-21T06:25:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-09-21T06:25:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T06:25:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-09-21T06:25:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-09-21T06:25:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-09-21T06:25:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-09-21T06:25:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-09-21T06:25:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T06:25:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-09-21T06:25:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-09-21T06:25:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-09-21T06:25:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-09-21T06:25:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-09-21T06:25:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-09-21T06:25:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-09-21T06:25:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-09-21T06:25:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-09-21T06:25:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-09-21T06:25:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:25:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-09-21T06:25:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-09-21T06:25:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-21T06:25:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-09-21T06:25:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T06:25:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-09-21T06:25:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-09-21T06:25:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-09-21T06:25:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-09-21T06:25:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-09-21T06:25:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-09-21T06:25:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:25:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-09-21T06:25:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-09-21T06:25:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.460 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.460 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.460 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.461 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.461 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-09-21T06:25:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-09-21T06:25:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-09-21T06:25:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-09-21T06:25:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T06:25:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-09-21T06:25:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-09-21T06:25:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-09-21T06:25:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T06:25:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-09-21T06:25:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-09-21T06:25:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-09-21T06:25:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-09-21T06:25:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-09-21T06:25:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T06:25:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.490 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.490 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.490 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.490 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.490 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.490 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.490 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.490 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.491 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.491 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.491 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.491 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.491 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.491 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.491 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.491 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.491 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.492 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.492 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-09-21T06:25:58.492 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-09-21T06:25:58.492 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.492 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.492 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.492 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.492 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.492 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.493 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.493 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-09-21T06:25:58.493 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T06:25:58.493 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.493 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.493 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.493 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-09-21T06:25:58.493 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-09-21T06:25:58.493 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-09-21T06:25:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-09-21T06:25:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-09-21T06:25:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-09-21T06:25:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-09-21T06:25:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T06:25:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-09-21T06:25:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-09-21T06:25:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-09-21T06:25:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-09-21T06:25:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-09-21T06:25:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-09-21T06:25:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T06:25:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T06:25:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-09-21T06:25:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-09-21T06:25:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-09-21T06:25:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.530 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.530 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.530 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.530 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.530 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.530 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.530 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.530 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.531 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.531 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.531 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.531 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.531 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.531 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.531 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.531 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.531 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.532 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.532 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.532 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.532 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.532 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.532 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.532 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.532 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.533 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.533 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.533 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.533 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.533 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.533 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.533 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.533 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.534 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.534 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.534 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.534 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.534 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.534 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.534 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.534 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.534 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.535 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.535 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.535 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.535 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.535 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-09-21T06:25:58.535 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.535 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.535 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.535 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-09-21T06:25:58.536 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-09-21T06:25:58.536 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.536 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.536 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.536 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.536 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.536 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.536 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T06:25:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-09-21T06:25:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-09-21T06:25:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-09-21T06:25:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-09-21T06:25:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-09-21T06:25:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-09-21T06:25:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-09-21T06:25:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-09-21T06:25:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-09-21T06:25:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-09-21T06:25:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-09-21T06:25:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-09-21T06:25:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-09-21T06:25:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-09-21T06:25:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T06:25:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-09-21T06:25:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-09-21T06:25:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-09-21T06:25:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T06:25:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-09-21T06:25:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-09-21T06:25:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-09-21T06:25:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-09-21T06:25:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-09-21T06:25:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-09-21T06:25:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-21T06:25:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-21T06:25:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-09-21T06:25:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-09-21T06:25:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-09-21T06:25:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-09-21T06:25:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-09-21T06:25:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-09-21T06:25:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T06:25:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T06:25:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-09-21T06:25:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-09-21T06:25:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T06:25:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T06:25:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-09-21T06:25:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-09-21T06:25:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-09-21T06:25:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-09-21T06:25:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-21T06:25:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-21T06:25:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-09-21T06:25:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T06:25:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-21T06:25:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-21T06:25:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-09-21T06:25:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T06:25:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-09-21T06:25:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-09-21T06:25:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-09-21T06:25:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-09-21T06:25:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-09-21T06:25:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-09-21T06:25:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-09-21T06:25:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-09-21T06:25:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-09-21T06:25:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-09-21T06:25:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-21T06:25:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-09-21T06:25:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-09-21T06:25:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-09-21T06:25:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-09-21T06:25:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-09-21T06:25:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-21T06:25:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-09-21T06:25:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T06:25:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-09-21T06:25:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-09-21T06:25:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-09-21T06:25:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-09-21T06:25:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-09-21T06:25:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-09-21T06:25:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-09-21T06:25:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "max_requests": { 2024-09-21T06:25:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_requests", 2024-09-21T06:25:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-21T06:25:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T06:25:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-21T06:25:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-21T06:25:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-21T06:25:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-21T06:25:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-09-21T06:25:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T06:25:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T06:25:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-21T06:25:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T06:25:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-09-21T06:25:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-09-21T06:25:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-09-21T06:25:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-09-21T06:25:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T06:25:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-09-21T06:25:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-09-21T06:25:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-09-21T06:25:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-09-21T06:25:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-09-21T06:25:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-09-21T06:25:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-09-21T06:25:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-09-21T06:25:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-09-21T06:25:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-09-21T06:25:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-09-21T06:25:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-09-21T06:25:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-21T06:25:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-09-21T06:25:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-09-21T06:25:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-09-21T06:25:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-09-21T06:25:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-09-21T06:25:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-09-21T06:25:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-09-21T06:25:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-09-21T06:25:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-21T06:25:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-09-21T06:25:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-09-21T06:25:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-09-21T06:25:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-09-21T06:25:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-09-21T06:25:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-09-21T06:25:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-09-21T06:25:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smb", 2024-09-21T06:25:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "internal_store_backend": { 2024-09-21T06:25:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "internal_store_backend", 2024-09-21T06:25:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T06:25:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "update_orchestration": { 2024-09-21T06:25:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "update_orchestration", 2024-09-21T06:25:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T06:25:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-09-21T06:25:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-09-21T06:25:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-09-21T06:25:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T06:25:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-09-21T06:25:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-09-21T06:25:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-09-21T06:25:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-09-21T06:25:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-09-21T06:25:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-09-21T06:25:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-09-21T06:25:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-09-21T06:25:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T06:25:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-09-21T06:25:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-09-21T06:25:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-09-21T06:25:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-09-21T06:25:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T06:25:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-09-21T06:25:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-09-21T06:25:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T06:25:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-09-21T06:25:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-09-21T06:25:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T06:25:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-09-21T06:25:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-09-21T06:25:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T06:25:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-09-21T06:25:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-09-21T06:25:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T06:25:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-09-21T06:25:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-09-21T06:25:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-09-21T06:25:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-09-21T06:25:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-09-21T06:25:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-09-21T06:25:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T06:25:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-09-21T06:25:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-09-21T06:25:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-09-21T06:25:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-09-21T06:25:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-09-21T06:25:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-09-21T06:25:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-09-21T06:25:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-09-21T06:25:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-09-21T06:25:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-09-21T06:25:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-09-21T06:25:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-09-21T06:25:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-09-21T06:25:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-09-21T06:25:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-09-21T06:25:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-09-21T06:25:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-09-21T06:25:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T06:25:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-09-21T06:25:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-09-21T06:25:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-09-21T06:25:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-09-21T06:25:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-09-21T06:25:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T06:25:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-09-21T06:25:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-09-21T06:25:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-09-21T06:25:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-09-21T06:25:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-09-21T06:25:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T06:25:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-09-21T06:25:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T06:25:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-21T06:25:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T06:25:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-09-21T06:25:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-21T06:25:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-21T06:25:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-21T06:25:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-09-21T06:25:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-09-21T06:25:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-21T06:25:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-09-21T06:25:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-09-21T06:25:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-09-21T06:25:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-21T06:25:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-21T06:25:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-21T06:25:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-21T06:25:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-21T06:25:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-21T06:25:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-21T06:25:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-21T06:25:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-21T06:25:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-21T06:25:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-21T06:25:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-21T06:25:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-21T06:25:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-21T06:25:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-21T06:25:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-21T06:25:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-21T06:25:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-21T06:25:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-21T06:25:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-21T06:25:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-21T06:25:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-21T06:25:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-09-21T06:25:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:25:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-09-21T06:25:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-21T06:25:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-21T06:25:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-09-21T06:25:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-09-21T06:25:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-21T06:25:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-09-21T06:25:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-09-21T06:25:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-21T06:25:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-09-21T06:25:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-09-21T06:25:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-09-21T06:25:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-21T06:25:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-21T06:25:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-21T06:25:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T06:25:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-21T06:25:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-21T06:25:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-21T06:25:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-21T06:25:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-21T06:25:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-21T06:25:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-21T06:25:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: "services": { 2024-09-21T06:25:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: "dashboard": "https://172.21.15.3:8443/" 2024-09-21T06:25:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: "always_on_modules": { 2024-09-21T06:25:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: "octopus": [ 2024-09-21T06:25:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-09-21T06:25:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-09-21T06:25:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-09-21T06:25:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-09-21T06:25:58.865 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-09-21T06:25:58.865 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-09-21T06:25:58.865 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-09-21T06:25:58.865 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-09-21T06:25:58.865 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-09-21T06:25:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-09-21T06:25:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: "pacific": [ 2024-09-21T06:25:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-09-21T06:25:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-09-21T06:25:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-09-21T06:25:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-09-21T06:25:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-09-21T06:25:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-09-21T06:25:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-09-21T06:25:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-09-21T06:25:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-09-21T06:25:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-09-21T06:25:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "quincy": [ 2024-09-21T06:25:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-09-21T06:25:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-09-21T06:25:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-09-21T06:25:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-09-21T06:25:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-09-21T06:25:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-09-21T06:25:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-09-21T06:25:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-09-21T06:25:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-09-21T06:25:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-09-21T06:25:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "reef": [ 2024-09-21T06:25:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-09-21T06:25:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-09-21T06:25:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-09-21T06:25:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-09-21T06:25:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-09-21T06:25:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-09-21T06:25:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-09-21T06:25:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-09-21T06:25:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-09-21T06:25:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-09-21T06:25:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-21T06:25:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "squid": [ 2024-09-21T06:25:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-09-21T06:25:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-09-21T06:25:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-09-21T06:25:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-09-21T06:25:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-09-21T06:25:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-09-21T06:25:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-09-21T06:25:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-09-21T06:25:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-09-21T06:25:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-09-21T06:25:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-21T06:25:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: "last_failure_osd_epoch": 5, 2024-09-21T06:25:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: "active_clients": [ 2024-09-21T06:25:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-09-21T06:25:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-21T06:25:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-21T06:25:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-09-21T06:25:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2266698128 2024-09-21T06:25:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-21T06:25:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "libcephsqlite", 2024-09-21T06:25:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-21T06:25:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-21T06:25:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-09-21T06:25:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 4066069382 2024-09-21T06:25:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-21T06:25:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-09-21T06:25:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-21T06:25:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-21T06:25:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-09-21T06:25:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2473672193 2024-09-21T06:25:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-21T06:25:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-09-21T06:25:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-21T06:25:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-21T06:25:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-09-21T06:25:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 4178832632 2024-09-21T06:25:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-21T06:25:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-09-21T06:25:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-21T06:25:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-21T06:25:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-09-21T06:25:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1519833507 2024-09-21T06:25:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-21T06:25:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-09-21T06:25:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-21T06:25:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-21T06:25:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-21T06:25:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-09-21T06:25:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 394674212 2024-09-21T06:25:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-21T06:25:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-21T06:25:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:25:58.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.522+0000 7f56a7f39640 1 -- 172.21.15.3:0/2305752537 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f568c0b6680 msgr2=0x7f568c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:58.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.522+0000 7f56a7f39640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f568c0b6680 0x7f568c0b8b90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f569c00abe0 tx=0x7f569c00a2d0 comp rx=0 tx=0).stop 2024-09-21T06:25:58.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.522+0000 7f56a7f39640 1 -- 172.21.15.3:0/2305752537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a01082b0 msgr2=0x7f56a019a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:58.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.522+0000 7f56a7f39640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a01082b0 0x7f56a019a870 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f569000c8c0 tx=0x7f569000ccc0 comp rx=0 tx=0).stop 2024-09-21T06:25:58.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.522+0000 7f56a7f39640 1 -- 172.21.15.3:0/2305752537 shutdown_connections 2024-09-21T06:25:58.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.522+0000 7f56a7f39640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f568c0b6680 0x7f568c0b8b90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:58.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.522+0000 7f56a7f39640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56a0108c00 0x7f56a019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:58.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.522+0000 7f56a7f39640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a01082b0 0x7f56a019a870 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:58.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.522+0000 7f56a7f39640 1 --2- 172.21.15.3:0/2305752537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a0102980 0x7f56a019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:58.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.522+0000 7f56a7f39640 1 -- 172.21.15.3:0/2305752537 >> 172.21.15.3:0/2305752537 conn(0x7f56a00fe380 msgr2=0x7f56a0107810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:58.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.522+0000 7f56a7f39640 1 -- 172.21.15.3:0/2305752537 shutdown_connections 2024-09-21T06:25:58.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:25:56.522+0000 7f56a7f39640 1 -- 172.21.15.3:0/2305752537 wait complete. 2024-09-21T06:25:58.889 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T06:25:58.889 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T06:25:58.889 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:56 smithi003 ceph-mon[31227]: pgmap v663: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T06:25:58.889 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:56 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3499722635' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:25:58.890 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:56 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2305752537' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T06:25:58.890 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:56 smithi204 ceph-mon[34659]: pgmap v663: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T06:25:58.891 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:56 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3499722635' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:25:58.891 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:56 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2305752537' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T06:25:58.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:56 smithi137 ceph-mon[34464]: pgmap v663: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T06:25:58.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:56 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3499722635' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:25:58.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:56 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2305752537' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T06:25:59.161 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:59.282 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:25:58 smithi003 ceph-mon[31227]: pgmap v664: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T06:25:59.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:25:58 smithi137 ceph-mon[34464]: pgmap v664: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T06:25:59.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:25:58 smithi204 ceph-mon[34659]: pgmap v664: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T06:25:59.704 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:25:59.704 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:25:59.704 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 1173, 2024-09-21T06:25:59.704 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 1175, 2024-09-21T06:25:59.704 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 12943693, 2024-09-21T06:25:59.704 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 373015, 2024-09-21T06:25:59.704 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 7, 2024-09-21T06:25:59.705 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-09-21T06:25:59.705 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.134949225, 2024-09-21T06:25:59.705 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.055024694, 2024-09-21T06:25:59.705 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.035884684, 2024-09-21T06:25:59.705 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.033296943, 2024-09-21T06:25:59.705 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:25:59.705 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 1173, 2024-09-21T06:25:59.705 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.042342555, 2024-09-21T06:25:59.705 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000036097 2024-09-21T06:25:59.706 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.706 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:25:59.706 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.706 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.706 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.706 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.706 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:25:59.706 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:25:59.707 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.707 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:25:59.707 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 655, 2024-09-21T06:25:59.707 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 811, 2024-09-21T06:25:59.707 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 19483261, 2024-09-21T06:25:59.707 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 573794, 2024-09-21T06:25:59.707 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 6, 2024-09-21T06:25:59.708 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 5, 2024-09-21T06:25:59.708 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.085105052, 2024-09-21T06:25:59.708 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.033335268, 2024-09-21T06:25:59.708 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.030109548, 2024-09-21T06:25:59.708 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.016410953, 2024-09-21T06:25:59.708 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:25:59.708 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 810, 2024-09-21T06:25:59.708 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.038641125, 2024-09-21T06:25:59.708 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000047705 2024-09-21T06:25:59.709 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.709 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:25:59.709 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.709 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.709 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.709 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.709 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-21T06:25:59.709 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 367344 2024-09-21T06:25:59.709 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.710 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:25:59.710 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 1008, 2024-09-21T06:25:59.710 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 989, 2024-09-21T06:25:59.710 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 34994664, 2024-09-21T06:25:59.710 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 374586, 2024-09-21T06:25:59.710 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-09-21T06:25:59.710 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 4, 2024-09-21T06:25:59.710 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.128555304, 2024-09-21T06:25:59.710 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.044846519, 2024-09-21T06:25:59.711 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.071085349, 2024-09-21T06:25:59.711 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.018061045, 2024-09-21T06:25:59.711 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:25:59.711 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 988, 2024-09-21T06:25:59.711 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.034863463, 2024-09-21T06:25:59.711 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000035286 2024-09-21T06:25:59.711 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.711 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:25:59.711 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.712 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.712 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.712 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.712 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1247792, 2024-09-21T06:25:59.712 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 131952 2024-09-21T06:25:59.712 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.712 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-09-21T06:25:59.712 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 1, 2024-09-21T06:25:59.712 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-09-21T06:25:59.713 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.713 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-MDSRank": { 2024-09-21T06:25:59.713 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2024-09-21T06:25:59.713 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2024-09-21T06:25:59.713 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 4424, 2024-09-21T06:25:59.713 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.113403726, 2024-09-21T06:25:59.713 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000025633 2024-09-21T06:25:59.713 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:59.714 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.714 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-PurgeQueue": { 2024-09-21T06:25:59.714 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2024-09-21T06:25:59.714 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2024-09-21T06:25:59.714 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.714 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.714 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.715 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:59.715 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.715 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": { 2024-09-21T06:25:59.715 INFO:teuthology.orchestra.run.smithi003.stdout: "request": 0, 2024-09-21T06:25:59.715 INFO:teuthology.orchestra.run.smithi003.stdout: "reply": 0, 2024-09-21T06:25:59.715 INFO:teuthology.orchestra.run.smithi003.stdout: "reply_latency": { 2024-09-21T06:25:59.715 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.715 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.716 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.716 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.716 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_reply": 0, 2024-09-21T06:25:59.716 INFO:teuthology.orchestra.run.smithi003.stdout: "forward": 0, 2024-09-21T06:25:59.716 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_complete": 0, 2024-09-21T06:25:59.716 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_keys": 0, 2024-09-21T06:25:59.716 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_commit": 0, 2024-09-21T06:25:59.716 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_split": 0, 2024-09-21T06:25:59.716 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_merge": 0, 2024-09-21T06:25:59.717 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes": 9569, 2024-09-21T06:25:59.717 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_top": 21, 2024-09-21T06:25:59.717 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_bottom": 9548, 2024-09-21T06:25:59.717 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pin_tail": 0, 2024-09-21T06:25:59.717 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pinned": 9539, 2024-09-21T06:25:59.717 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_expired": 0, 2024-09-21T06:25:59.717 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_with_caps": 0, 2024-09-21T06:25:59.717 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": 0, 2024-09-21T06:25:59.717 INFO:teuthology.orchestra.run.smithi003.stdout: "subtrees": 3, 2024-09-21T06:25:59.718 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse": 0, 2024-09-21T06:25:59.718 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_hit": 0, 2024-09-21T06:25:59.718 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_forward": 0, 2024-09-21T06:25:59.718 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_discover": 0, 2024-09-21T06:25:59.718 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:25:59.718 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_remote_ino": 0, 2024-09-21T06:25:59.718 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_lock": 0, 2024-09-21T06:25:59.718 INFO:teuthology.orchestra.run.smithi003.stdout: "load_cent": 0, 2024-09-21T06:25:59.718 INFO:teuthology.orchestra.run.smithi003.stdout: "q": 0, 2024-09-21T06:25:59.719 INFO:teuthology.orchestra.run.smithi003.stdout: "exported": 0, 2024-09-21T06:25:59.719 INFO:teuthology.orchestra.run.smithi003.stdout: "exported_inodes": 0, 2024-09-21T06:25:59.719 INFO:teuthology.orchestra.run.smithi003.stdout: "imported": 0, 2024-09-21T06:25:59.719 INFO:teuthology.orchestra.run.smithi003.stdout: "imported_inodes": 0, 2024-09-21T06:25:59.719 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_dir_fetch": 0, 2024-09-21T06:25:59.719 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:25:59.719 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_peer_discover": 0, 2024-09-21T06:25:59.720 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rfiles": 0, 2024-09-21T06:25:59.720 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rbytes": 0, 2024-09-21T06:25:59.720 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rsnaps": 0, 2024-09-21T06:25:59.720 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:25:59.720 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_set_tag": 0, 2024-09-21T06:25:59.720 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:25:59.720 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:25:59.720 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:25:59.721 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:25:59.721 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:25:59.721 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_file_inodes": 0, 2024-09-21T06:25:59.721 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:25:59.721 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:25:59.721 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:25:59.721 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:25:59.721 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:25:59.721 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:25:59.721 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps": 0, 2024-09-21T06:25:59.722 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:25:59.722 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_cap_release": 0, 2024-09-21T06:25:59.722 INFO:teuthology.orchestra.run.smithi003.stdout: "process_request_cap_release": 0 2024-09-21T06:25:59.722 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.722 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_cache": { 2024-09-21T06:25:59.722 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update": 0, 2024-09-21T06:25:59.722 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update_receipt": 0, 2024-09-21T06:25:59.722 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_try_discover": 0, 2024-09-21T06:25:59.722 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_send_discover": 0, 2024-09-21T06:25:59.723 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_handle_discover": 0, 2024-09-21T06:25:59.723 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays": 0, 2024-09-21T06:25:59.723 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_delayed": 0, 2024-09-21T06:25:59.723 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:25:59.723 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_created": 0, 2024-09-21T06:25:59.723 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_enqueued": 0, 2024-09-21T06:25:59.723 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_reintegrated": 0, 2024-09-21T06:25:59.723 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_migrated": 0, 2024-09-21T06:25:59.724 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_processing": 0, 2024-09-21T06:25:59.724 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:25:59.724 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:25:59.724 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_started": 0, 2024-09-21T06:25:59.724 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_completed": 0, 2024-09-21T06:25:59.724 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:25:59.724 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:25:59.724 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:25:59.725 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_exportdir": 0, 2024-09-21T06:25:59.725 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_flush": 0, 2024-09-21T06:25:59.725 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:25:59.725 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragstats": 0, 2024-09-21T06:25:59.725 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_inodestats": 0, 2024-09-21T06:25:59.725 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:25:59.725 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_succeeded": 0, 2024-09-21T06:25:59.725 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_write_failed": 0 2024-09-21T06:25:59.725 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.726 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_log": { 2024-09-21T06:25:59.726 INFO:teuthology.orchestra.run.smithi003.stdout: "evlrg": 0, 2024-09-21T06:25:59.726 INFO:teuthology.orchestra.run.smithi003.stdout: "evadd": 0, 2024-09-21T06:25:59.726 INFO:teuthology.orchestra.run.smithi003.stdout: "evex": 0, 2024-09-21T06:25:59.726 INFO:teuthology.orchestra.run.smithi003.stdout: "evtrm": 0, 2024-09-21T06:25:59.726 INFO:teuthology.orchestra.run.smithi003.stdout: "ev": 39100, 2024-09-21T06:25:59.726 INFO:teuthology.orchestra.run.smithi003.stdout: "evexg": 0, 2024-09-21T06:25:59.726 INFO:teuthology.orchestra.run.smithi003.stdout: "evexd": 0, 2024-09-21T06:25:59.727 INFO:teuthology.orchestra.run.smithi003.stdout: "segadd": 0, 2024-09-21T06:25:59.727 INFO:teuthology.orchestra.run.smithi003.stdout: "segex": 0, 2024-09-21T06:25:59.727 INFO:teuthology.orchestra.run.smithi003.stdout: "segtrm": 0, 2024-09-21T06:25:59.727 INFO:teuthology.orchestra.run.smithi003.stdout: "seg": 49, 2024-09-21T06:25:59.727 INFO:teuthology.orchestra.run.smithi003.stdout: "segmjr": 5, 2024-09-21T06:25:59.727 INFO:teuthology.orchestra.run.smithi003.stdout: "segexg": 0, 2024-09-21T06:25:59.727 INFO:teuthology.orchestra.run.smithi003.stdout: "segexd": 0, 2024-09-21T06:25:59.727 INFO:teuthology.orchestra.run.smithi003.stdout: "expos": 4194304, 2024-09-21T06:25:59.727 INFO:teuthology.orchestra.run.smithi003.stdout: "wrpos": 69646489, 2024-09-21T06:25:59.728 INFO:teuthology.orchestra.run.smithi003.stdout: "rdpos": 69645890, 2024-09-21T06:25:59.728 INFO:teuthology.orchestra.run.smithi003.stdout: "jlat": { 2024-09-21T06:25:59.728 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.728 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.728 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.728 INFO:teuthology.orchestra.run.smithi003.stdout: "replayed": 39100 2024-09-21T06:25:59.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.729 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_mem": { 2024-09-21T06:25:59.729 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 47, 2024-09-21T06:25:59.729 INFO:teuthology.orchestra.run.smithi003.stdout: "ino+": 9525, 2024-09-21T06:25:59.729 INFO:teuthology.orchestra.run.smithi003.stdout: "ino-": 9478, 2024-09-21T06:25:59.729 INFO:teuthology.orchestra.run.smithi003.stdout: "dir": 22, 2024-09-21T06:25:59.729 INFO:teuthology.orchestra.run.smithi003.stdout: "dir+": 22, 2024-09-21T06:25:59.729 INFO:teuthology.orchestra.run.smithi003.stdout: "dir-": 0, 2024-09-21T06:25:59.729 INFO:teuthology.orchestra.run.smithi003.stdout: "dn": 9569, 2024-09-21T06:25:59.729 INFO:teuthology.orchestra.run.smithi003.stdout: "dn+": 9569, 2024-09-21T06:25:59.730 INFO:teuthology.orchestra.run.smithi003.stdout: "dn-": 0, 2024-09-21T06:25:59.730 INFO:teuthology.orchestra.run.smithi003.stdout: "cap": 0, 2024-09-21T06:25:59.730 INFO:teuthology.orchestra.run.smithi003.stdout: "cap+": 0, 2024-09-21T06:25:59.730 INFO:teuthology.orchestra.run.smithi003.stdout: "cap-": 0, 2024-09-21T06:25:59.730 INFO:teuthology.orchestra.run.smithi003.stdout: "rss": 127880, 2024-09-21T06:25:59.730 INFO:teuthology.orchestra.run.smithi003.stdout: "heap": 328085504 2024-09-21T06:25:59.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.730 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_server": { 2024-09-21T06:25:59.731 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_client_request": 0, 2024-09-21T06:25:59.731 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_server_request": 0, 2024-09-21T06:25:59.731 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_request": 0, 2024-09-21T06:25:59.731 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_session": 0, 2024-09-21T06:25:59.731 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_peer_request": 0, 2024-09-21T06:25:59.731 INFO:teuthology.orchestra.run.smithi003.stdout: "req_create_latency": { 2024-09-21T06:25:59.731 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.731 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.731 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.732 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getattr_latency": { 2024-09-21T06:25:59.732 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.732 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.732 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.732 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getfilelock_latency": { 2024-09-21T06:25:59.732 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.732 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.733 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.733 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.733 INFO:teuthology.orchestra.run.smithi003.stdout: "req_link_latency": { 2024-09-21T06:25:59.733 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.733 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.733 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.733 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.733 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookup_latency": { 2024-09-21T06:25:59.734 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.734 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.734 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.734 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.734 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookuphash_latency": { 2024-09-21T06:25:59.734 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.734 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.734 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.735 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.735 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupino_latency": { 2024-09-21T06:25:59.735 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.735 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.735 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.735 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.735 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupname_latency": { 2024-09-21T06:25:59.735 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.735 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.736 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.736 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.736 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupparent_latency": { 2024-09-21T06:25:59.736 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.736 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.736 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.736 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.736 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupsnap_latency": { 2024-09-21T06:25:59.736 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.737 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.737 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.737 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.737 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lssnap_latency": { 2024-09-21T06:25:59.737 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.738 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.738 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.738 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.738 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mkdir_latency": { 2024-09-21T06:25:59.738 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.738 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.738 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.739 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.739 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mknod_latency": { 2024-09-21T06:25:59.739 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.739 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.739 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.739 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.739 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mksnap_latency": { 2024-09-21T06:25:59.739 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.740 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.740 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.740 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.740 INFO:teuthology.orchestra.run.smithi003.stdout: "req_open_latency": { 2024-09-21T06:25:59.740 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.740 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.740 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.740 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.741 INFO:teuthology.orchestra.run.smithi003.stdout: "req_readdir_latency": { 2024-09-21T06:25:59.741 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.741 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.741 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.741 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.741 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rename_latency": { 2024-09-21T06:25:59.741 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.741 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.741 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.742 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.742 INFO:teuthology.orchestra.run.smithi003.stdout: "req_renamesnap_latency": { 2024-09-21T06:25:59.742 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.742 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.742 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.742 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.742 INFO:teuthology.orchestra.run.smithi003.stdout: "req_snapdiff_latency": { 2024-09-21T06:25:59.742 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.742 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.743 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.743 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.743 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmdir_latency": { 2024-09-21T06:25:59.743 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.743 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.743 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.743 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.743 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmsnap_latency": { 2024-09-21T06:25:59.743 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.744 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.744 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.744 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmxattr_latency": { 2024-09-21T06:25:59.744 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.744 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.744 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.744 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setattr_latency": { 2024-09-21T06:25:59.745 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.745 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.745 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.745 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.745 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setdirlayout_latency": { 2024-09-21T06:25:59.745 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.745 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.745 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.745 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.746 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setfilelock_latency": { 2024-09-21T06:25:59.746 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.746 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.746 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.746 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.746 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setlayout_latency": { 2024-09-21T06:25:59.746 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.746 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.747 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.747 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setxattr_latency": { 2024-09-21T06:25:59.747 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.747 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.747 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.747 INFO:teuthology.orchestra.run.smithi003.stdout: "req_symlink_latency": { 2024-09-21T06:25:59.747 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.748 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.748 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.748 INFO:teuthology.orchestra.run.smithi003.stdout: "req_unlink_latency": { 2024-09-21T06:25:59.748 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.748 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.748 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.748 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:25:59.749 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:25:59.749 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getvxattr_latency": { 2024-09-21T06:25:59.749 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.749 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.749 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.749 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:59.749 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.749 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_sessions": { 2024-09-21T06:25:59.749 INFO:teuthology.orchestra.run.smithi003.stdout: "session_count": 2, 2024-09-21T06:25:59.750 INFO:teuthology.orchestra.run.smithi003.stdout: "session_add": 4, 2024-09-21T06:25:59.750 INFO:teuthology.orchestra.run.smithi003.stdout: "session_remove": 2, 2024-09-21T06:25:59.750 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_open": 2, 2024-09-21T06:25:59.750 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_stale": 0, 2024-09-21T06:25:59.750 INFO:teuthology.orchestra.run.smithi003.stdout: "total_load": 0, 2024-09-21T06:25:59.750 INFO:teuthology.orchestra.run.smithi003.stdout: "average_load": 0, 2024-09-21T06:25:59.750 INFO:teuthology.orchestra.run.smithi003.stdout: "avg_session_uptime": 2882, 2024-09-21T06:25:59.750 INFO:teuthology.orchestra.run.smithi003.stdout: "mdthresh_evicted": 0 2024-09-21T06:25:59.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.751 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-09-21T06:25:59.751 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:25:59.751 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-09-21T06:25:59.751 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:25:59.751 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:25:59.751 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:25:59.751 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:25:59.751 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:25:59.751 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:25:59.751 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:25:59.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:25:59.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:25:59.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:25:59.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:25:59.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:25:59.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:25:59.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-09-21T06:25:59.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:25:59.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-09-21T06:25:59.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:25:59.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:25:59.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:25:59.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:25:59.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:25:59.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:25:59.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:25:59.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-09-21T06:25:59.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:25:59.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:25:59.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:25:59.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-09-21T06:25:59.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-09-21T06:25:59.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-09-21T06:25:59.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:25:59.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:25:59.755 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:25:59.755 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:25:59.755 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 224418, 2024-09-21T06:25:59.755 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 101, 2024-09-21T06:25:59.755 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:25:59.755 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-09-21T06:25:59.755 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-09-21T06:25:59.755 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-09-21T06:25:59.755 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:25:59.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-09-21T06:25:59.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:25:59.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-09-21T06:25:59.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 8168, 2024-09-21T06:25:59.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 181, 2024-09-21T06:25:59.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:25:59.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:25:59.756 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-09-21T06:25:59.756 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-09-21T06:25:59.757 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 6715100, 2024-09-21T06:25:59.757 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 39294, 2024-09-21T06:25:59.757 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-09-21T06:25:59.757 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-09-21T06:25:59.757 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-09-21T06:25:59.757 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-09-21T06:25:59.757 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.757 INFO:teuthology.orchestra.run.smithi003.stdout: "objecter": { 2024-09-21T06:25:59.758 INFO:teuthology.orchestra.run.smithi003.stdout: "op_active": 0, 2024-09-21T06:25:59.758 INFO:teuthology.orchestra.run.smithi003.stdout: "op_laggy": 0, 2024-09-21T06:25:59.758 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send": 2585, 2024-09-21T06:25:59.758 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send_bytes": 21, 2024-09-21T06:25:59.758 INFO:teuthology.orchestra.run.smithi003.stdout: "op_resend": 3, 2024-09-21T06:25:59.758 INFO:teuthology.orchestra.run.smithi003.stdout: "op_reply": 2585, 2024-09-21T06:25:59.758 INFO:teuthology.orchestra.run.smithi003.stdout: "op_latency": { 2024-09-21T06:25:59.758 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 2585, 2024-09-21T06:25:59.758 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 3.364954441, 2024-09-21T06:25:59.759 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.001301723 2024-09-21T06:25:59.759 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.759 INFO:teuthology.orchestra.run.smithi003.stdout: "op_inflight": 0, 2024-09-21T06:25:59.759 INFO:teuthology.orchestra.run.smithi003.stdout: "oplen_avg": { 2024-09-21T06:25:59.759 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 2585, 2024-09-21T06:25:59.759 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 2586 2024-09-21T06:25:59.759 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.759 INFO:teuthology.orchestra.run.smithi003.stdout: "op": 2585, 2024-09-21T06:25:59.759 INFO:teuthology.orchestra.run.smithi003.stdout: "op_r": 2585, 2024-09-21T06:25:59.760 INFO:teuthology.orchestra.run.smithi003.stdout: "op_w": 0, 2024-09-21T06:25:59.760 INFO:teuthology.orchestra.run.smithi003.stdout: "op_rmw": 0, 2024-09-21T06:25:59.760 INFO:teuthology.orchestra.run.smithi003.stdout: "op_pg": 0, 2024-09-21T06:25:59.760 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_stat": 1410, 2024-09-21T06:25:59.760 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_create": 0, 2024-09-21T06:25:59.760 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_read": 1173, 2024-09-21T06:25:59.760 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_write": 0, 2024-09-21T06:25:59.760 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writefull": 0, 2024-09-21T06:25:59.761 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writesame": 0, 2024-09-21T06:25:59.761 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_append": 0, 2024-09-21T06:25:59.761 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_zero": 0, 2024-09-21T06:25:59.761 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_truncate": 0, 2024-09-21T06:25:59.761 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_delete": 0, 2024-09-21T06:25:59.761 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_mapext": 0, 2024-09-21T06:25:59.761 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_sparse_read": 0, 2024-09-21T06:25:59.762 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_clonerange": 0, 2024-09-21T06:25:59.762 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_getxattr": 1, 2024-09-21T06:25:59.762 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_setxattr": 0, 2024-09-21T06:25:59.762 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:25:59.762 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_rmxattr": 0, 2024-09-21T06:25:59.762 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:25:59.762 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_call": 0, 2024-09-21T06:25:59.762 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_watch": 0, 2024-09-21T06:25:59.763 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_notify": 0, 2024-09-21T06:25:59.763 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:25:59.763 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls": 0, 2024-09-21T06:25:59.763 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:25:59.763 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_other": 0, 2024-09-21T06:25:59.763 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_active": 0, 2024-09-21T06:25:59.763 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_send": 0, 2024-09-21T06:25:59.763 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_resend": 0, 2024-09-21T06:25:59.764 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_ping": 0, 2024-09-21T06:25:59.764 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_active": 0, 2024-09-21T06:25:59.764 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_send": 0, 2024-09-21T06:25:59.764 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_resend": 0, 2024-09-21T06:25:59.764 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_active": 0, 2024-09-21T06:25:59.764 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_send": 0, 2024-09-21T06:25:59.764 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_resend": 0, 2024-09-21T06:25:59.764 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_active": 0, 2024-09-21T06:25:59.764 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_send": 0, 2024-09-21T06:25:59.765 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_resend": 0, 2024-09-21T06:25:59.765 INFO:teuthology.orchestra.run.smithi003.stdout: "command_active": 0, 2024-09-21T06:25:59.765 INFO:teuthology.orchestra.run.smithi003.stdout: "command_send": 0, 2024-09-21T06:25:59.765 INFO:teuthology.orchestra.run.smithi003.stdout: "command_resend": 0, 2024-09-21T06:25:59.765 INFO:teuthology.orchestra.run.smithi003.stdout: "map_epoch": 89, 2024-09-21T06:25:59.765 INFO:teuthology.orchestra.run.smithi003.stdout: "map_full": 0, 2024-09-21T06:25:59.765 INFO:teuthology.orchestra.run.smithi003.stdout: "map_inc": 11, 2024-09-21T06:25:59.765 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_sessions": 9, 2024-09-21T06:25:59.765 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_open": 9, 2024-09-21T06:25:59.766 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_close": 0, 2024-09-21T06:25:59.766 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_laggy": 0, 2024-09-21T06:25:59.766 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_wr": 0, 2024-09-21T06:25:59.766 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_rd": 2, 2024-09-21T06:25:59.766 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_del": 0 2024-09-21T06:25:59.766 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.766 INFO:teuthology.orchestra.run.smithi003.stdout: "oft": { 2024-09-21T06:25:59.766 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_objs": 0, 2024-09-21T06:25:59.767 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:25:59.767 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_updates": 0, 2024-09-21T06:25:59.767 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_removes": 0 2024-09-21T06:25:59.767 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.767 INFO:teuthology.orchestra.run.smithi003.stdout: "purge_queue": { 2024-09-21T06:25:59.767 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops": 0, 2024-09-21T06:25:59.767 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:25:59.767 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing": 0, 2024-09-21T06:25:59.767 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_high_water": 0, 2024-09-21T06:25:59.768 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executed_ops": 0, 2024-09-21T06:25:59.768 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executed": 0, 2024-09-21T06:25:59.768 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_item_in_journal": 0 2024-09-21T06:25:59.768 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.768 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:25:59.768 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-21T06:25:59.768 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-09-21T06:25:59.768 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-21T06:25:59.769 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 2832, 2024-09-21T06:25:59.769 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 67151299, 2024-09-21T06:25:59.769 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-21T06:25:59.769 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 2832, 2024-09-21T06:25:59.769 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-21T06:25:59.769 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-21T06:25:59.769 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 2832, 2024-09-21T06:25:59.769 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 67151299, 2024-09-21T06:25:59.770 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-21T06:25:59.770 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.770 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.770 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.770 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:59.770 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.770 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_bytes": { 2024-09-21T06:25:59.770 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-21T06:25:59.771 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-09-21T06:25:59.771 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-21T06:25:59.771 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-21T06:25:59.771 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-21T06:25:59.771 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-21T06:25:59.771 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-21T06:25:59.771 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 2585, 2024-09-21T06:25:59.771 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 65452190, 2024-09-21T06:25:59.771 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 2585, 2024-09-21T06:25:59.772 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 65452190, 2024-09-21T06:25:59.772 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-21T06:25:59.772 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.772 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.772 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.772 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:59.772 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.772 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_ops": { 2024-09-21T06:25:59.773 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-21T06:25:59.773 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 1024, 2024-09-21T06:25:59.773 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-21T06:25:59.773 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-21T06:25:59.773 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-21T06:25:59.773 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-21T06:25:59.773 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-21T06:25:59.773 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 2585, 2024-09-21T06:25:59.774 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 2585, 2024-09-21T06:25:59.774 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 2585, 2024-09-21T06:25:59.774 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 2585, 2024-09-21T06:25:59.774 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-21T06:25:59.774 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.774 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.774 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.774 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:59.774 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.775 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:25:59.775 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-21T06:25:59.775 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2024-09-21T06:25:59.775 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-21T06:25:59.775 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-21T06:25:59.775 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-21T06:25:59.775 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-21T06:25:59.775 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-21T06:25:59.776 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-21T06:25:59.776 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-21T06:25:59.776 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 0, 2024-09-21T06:25:59.776 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 0, 2024-09-21T06:25:59.776 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-21T06:25:59.776 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.776 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.776 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.777 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:59.777 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.777 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle-0x55a35d1b95f0": { 2024-09-21T06:25:59.777 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-21T06:25:59.777 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2024-09-21T06:25:59.777 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-21T06:25:59.777 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-21T06:25:59.777 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-21T06:25:59.777 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-21T06:25:59.778 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-21T06:25:59.778 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-21T06:25:59.778 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-21T06:25:59.778 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 0, 2024-09-21T06:25:59.778 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 0, 2024-09-21T06:25:59.778 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-21T06:25:59.778 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:25:59.778 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:25:59.779 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:25:59.779 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:59.779 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:25:59.779 INFO:teuthology.orchestra.run.smithi003.stdout: "trackedop": { 2024-09-21T06:25:59.779 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_ops_count": 0 2024-09-21T06:25:59.779 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:25:59.779 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:26:00.433 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-21T06:26:00.433 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-21T06:26:00.433 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-21T06:26:00.433 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:26:00.433 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:26:00.433 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T06:26:00.434 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T06:26:00.691 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:01.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:00 smithi003 ceph-mon[31227]: pgmap v665: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-21T06:26:01.223 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:26:01.224 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:26:01.224 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 0, 2024-09-21T06:26:01.224 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 2, 2024-09-21T06:26:01.224 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 0, 2024-09-21T06:26:01.224 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 0, 2024-09-21T06:26:01.224 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-09-21T06:26:01.224 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-09-21T06:26:01.224 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.001258304, 2024-09-21T06:26:01.225 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T06:26:01.225 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.000202580, 2024-09-21T06:26:01.225 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:26:01.225 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:01.225 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:01.225 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:01.225 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:01.225 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:01.226 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:01.226 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:01.226 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:01.226 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:01.226 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:01.226 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:26:01.226 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:26:01.226 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:01.226 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:26:01.227 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 226, 2024-09-21T06:26:01.227 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 217, 2024-09-21T06:26:01.227 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1150080, 2024-09-21T06:26:01.227 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 132832, 2024-09-21T06:26:01.227 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-09-21T06:26:01.227 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-09-21T06:26:01.227 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.040127801, 2024-09-21T06:26:01.227 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.016634391, 2024-09-21T06:26:01.228 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.033530876, 2024-09-21T06:26:01.228 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:26:01.228 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:01.228 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 217, 2024-09-21T06:26:01.228 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.013615293, 2024-09-21T06:26:01.228 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000062743 2024-09-21T06:26:01.228 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:01.228 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:01.228 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:01.229 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:01.229 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:01.229 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:01.229 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1150080, 2024-09-21T06:26:01.229 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 132832 2024-09-21T06:26:01.229 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:01.229 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:26:01.229 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 1, 2024-09-21T06:26:01.229 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 164, 2024-09-21T06:26:01.230 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 160, 2024-09-21T06:26:01.230 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 109104, 2024-09-21T06:26:01.230 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 3, 2024-09-21T06:26:01.230 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 2, 2024-09-21T06:26:01.230 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.015276179, 2024-09-21T06:26:01.230 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.012178818, 2024-09-21T06:26:01.230 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.001408046, 2024-09-21T06:26:01.230 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:26:01.231 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:01.231 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 162, 2024-09-21T06:26:01.231 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.007074787, 2024-09-21T06:26:01.231 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000043671 2024-09-21T06:26:01.231 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:01.231 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:01.231 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:01.231 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:01.231 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:01.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:01.232 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:26:01.232 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 109104 2024-09-21T06:26:01.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:01.232 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-09-21T06:26:01.232 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-09-21T06:26:01.232 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-09-21T06:26:01.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:01.233 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-09-21T06:26:01.233 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:26:01.233 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-09-21T06:26:01.233 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:26:01.233 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:26:01.233 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:26:01.233 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:26:01.233 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:26:01.233 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:26:01.234 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:26:01.234 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:26:01.234 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:26:01.234 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:26:01.234 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:26:01.234 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:26:01.234 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:26:01.234 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-09-21T06:26:01.235 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:26:01.235 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-09-21T06:26:01.235 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:26:01.235 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:26:01.235 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:26:01.235 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:26:01.235 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:26:01.235 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:26:01.235 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:26:01.236 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-09-21T06:26:01.236 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:26:01.236 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:26:01.236 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:26:01.236 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-09-21T06:26:01.236 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-09-21T06:26:01.236 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-09-21T06:26:01.236 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:26:01.236 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:26:01.237 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:26:01.237 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:26:01.237 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 149007, 2024-09-21T06:26:01.237 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 30, 2024-09-21T06:26:01.237 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:26:01.237 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-09-21T06:26:01.237 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-09-21T06:26:01.237 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-09-21T06:26:01.237 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:26:01.238 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-09-21T06:26:01.238 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:26:01.238 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-09-21T06:26:01.238 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-09-21T06:26:01.238 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-09-21T06:26:01.238 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:26:01.239 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:26:01.239 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-09-21T06:26:01.239 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-09-21T06:26:01.239 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 968, 2024-09-21T06:26:01.239 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-09-21T06:26:01.239 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-09-21T06:26:01.239 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-09-21T06:26:01.239 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-09-21T06:26:01.239 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-09-21T06:26:01.240 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:01.240 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:26:01.240 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-21T06:26:01.240 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-09-21T06:26:01.240 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-21T06:26:01.240 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 223, 2024-09-21T06:26:01.240 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1115737, 2024-09-21T06:26:01.240 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:01.240 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 223, 2024-09-21T06:26:01.241 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-21T06:26:01.241 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-21T06:26:01.241 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 223, 2024-09-21T06:26:01.241 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1115737, 2024-09-21T06:26:01.241 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-21T06:26:01.241 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:01.241 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:01.241 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:01.241 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:26:01.242 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:26:01.242 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:26:01.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:00 smithi137 ceph-mon[34464]: pgmap v665: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-21T06:26:01.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:00 smithi204 ceph-mon[34659]: pgmap v665: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-21T06:26:01.973 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T06:26:01.973 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T06:26:01.973 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-21T06:26:01.973 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-21T06:26:01.973 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-21T06:26:01.974 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T06:26:01.974 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T06:26:02.236 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:02.740 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:26:02.741 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:26:02.741 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 759, 2024-09-21T06:26:02.741 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 761, 2024-09-21T06:26:02.741 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 254969, 2024-09-21T06:26:02.741 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 241363, 2024-09-21T06:26:02.741 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 6, 2024-09-21T06:26:02.741 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 2, 2024-09-21T06:26:02.742 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.092556863, 2024-09-21T06:26:02.742 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.046088589, 2024-09-21T06:26:02.742 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.017770096, 2024-09-21T06:26:02.742 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.020818573, 2024-09-21T06:26:02.742 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:02.742 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 759, 2024-09-21T06:26:02.742 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.034242820, 2024-09-21T06:26:02.743 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000045115 2024-09-21T06:26:02.743 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.743 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:02.743 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.743 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.743 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.743 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.743 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:26:02.744 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:26:02.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.744 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:26:02.744 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 1420, 2024-09-21T06:26:02.744 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 1578, 2024-09-21T06:26:02.744 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 364862, 2024-09-21T06:26:02.744 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 819179, 2024-09-21T06:26:02.744 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-09-21T06:26:02.744 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 4, 2024-09-21T06:26:02.745 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.117384710, 2024-09-21T06:26:02.745 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.051389753, 2024-09-21T06:26:02.745 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.025851208, 2024-09-21T06:26:02.745 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.032369418, 2024-09-21T06:26:02.745 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:02.745 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 1577, 2024-09-21T06:26:02.745 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.057926324, 2024-09-21T06:26:02.745 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000036731 2024-09-21T06:26:02.746 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.746 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:02.746 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.746 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.746 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.746 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.746 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-21T06:26:02.746 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 369152 2024-09-21T06:26:02.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.747 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:26:02.747 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 250, 2024-09-21T06:26:02.747 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 231, 2024-09-21T06:26:02.747 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1249684, 2024-09-21T06:26:02.747 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 134197, 2024-09-21T06:26:02.747 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-09-21T06:26:02.747 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-09-21T06:26:02.748 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.042101283, 2024-09-21T06:26:02.748 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.017130794, 2024-09-21T06:26:02.748 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.037217783, 2024-09-21T06:26:02.748 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000057648, 2024-09-21T06:26:02.748 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:02.748 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 230, 2024-09-21T06:26:02.748 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.011513544, 2024-09-21T06:26:02.748 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000050058 2024-09-21T06:26:02.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.749 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:02.749 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.749 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.749 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.749 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.749 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1248336, 2024-09-21T06:26:02.749 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 133232 2024-09-21T06:26:02.749 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.749 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-09-21T06:26:02.750 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 1, 2024-09-21T06:26:02.750 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-09-21T06:26:02.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.750 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-MDSRank": { 2024-09-21T06:26:02.750 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2024-09-21T06:26:02.750 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2024-09-21T06:26:02.750 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 3680, 2024-09-21T06:26:02.750 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.090578949, 2024-09-21T06:26:02.750 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000024613 2024-09-21T06:26:02.751 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:26:02.751 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.751 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-PurgeQueue": { 2024-09-21T06:26:02.751 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2024-09-21T06:26:02.751 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2024-09-21T06:26:02.751 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.751 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.751 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.751 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:26:02.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.752 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": { 2024-09-21T06:26:02.752 INFO:teuthology.orchestra.run.smithi003.stdout: "request": 0, 2024-09-21T06:26:02.752 INFO:teuthology.orchestra.run.smithi003.stdout: "reply": 0, 2024-09-21T06:26:02.752 INFO:teuthology.orchestra.run.smithi003.stdout: "reply_latency": { 2024-09-21T06:26:02.752 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.752 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.752 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.753 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_reply": 0, 2024-09-21T06:26:02.753 INFO:teuthology.orchestra.run.smithi003.stdout: "forward": 0, 2024-09-21T06:26:02.753 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_complete": 0, 2024-09-21T06:26:02.753 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_keys": 0, 2024-09-21T06:26:02.753 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_commit": 0, 2024-09-21T06:26:02.753 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_split": 0, 2024-09-21T06:26:02.753 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_merge": 0, 2024-09-21T06:26:02.753 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes": 0, 2024-09-21T06:26:02.753 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_top": 0, 2024-09-21T06:26:02.754 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_bottom": 0, 2024-09-21T06:26:02.754 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pin_tail": 0, 2024-09-21T06:26:02.754 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pinned": 0, 2024-09-21T06:26:02.754 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_expired": 0, 2024-09-21T06:26:02.754 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_with_caps": 0, 2024-09-21T06:26:02.754 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": 0, 2024-09-21T06:26:02.754 INFO:teuthology.orchestra.run.smithi003.stdout: "subtrees": 1, 2024-09-21T06:26:02.754 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse": 0, 2024-09-21T06:26:02.755 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_hit": 0, 2024-09-21T06:26:02.755 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_forward": 0, 2024-09-21T06:26:02.755 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_discover": 0, 2024-09-21T06:26:02.755 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:26:02.755 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_remote_ino": 0, 2024-09-21T06:26:02.755 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_lock": 0, 2024-09-21T06:26:02.755 INFO:teuthology.orchestra.run.smithi003.stdout: "load_cent": 0, 2024-09-21T06:26:02.755 INFO:teuthology.orchestra.run.smithi003.stdout: "q": 0, 2024-09-21T06:26:02.756 INFO:teuthology.orchestra.run.smithi003.stdout: "exported": 0, 2024-09-21T06:26:02.756 INFO:teuthology.orchestra.run.smithi003.stdout: "exported_inodes": 0, 2024-09-21T06:26:02.756 INFO:teuthology.orchestra.run.smithi003.stdout: "imported": 0, 2024-09-21T06:26:02.756 INFO:teuthology.orchestra.run.smithi003.stdout: "imported_inodes": 0, 2024-09-21T06:26:02.756 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_dir_fetch": 0, 2024-09-21T06:26:02.756 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:26:02.756 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_peer_discover": 0, 2024-09-21T06:26:02.756 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rfiles": 0, 2024-09-21T06:26:02.757 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rbytes": 0, 2024-09-21T06:26:02.757 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rsnaps": 0, 2024-09-21T06:26:02.757 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:26:02.757 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_set_tag": 0, 2024-09-21T06:26:02.757 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:26:02.757 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:26:02.757 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:26:02.757 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:26:02.757 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:26:02.758 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_file_inodes": 0, 2024-09-21T06:26:02.758 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:26:02.758 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:26:02.758 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:26:02.758 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:26:02.758 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:26:02.758 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:26:02.759 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps": 0, 2024-09-21T06:26:02.759 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:26:02.759 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_cap_release": 0, 2024-09-21T06:26:02.759 INFO:teuthology.orchestra.run.smithi003.stdout: "process_request_cap_release": 0 2024-09-21T06:26:02.759 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.760 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_cache": { 2024-09-21T06:26:02.760 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update": 0, 2024-09-21T06:26:02.760 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update_receipt": 0, 2024-09-21T06:26:02.760 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_try_discover": 0, 2024-09-21T06:26:02.760 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_send_discover": 0, 2024-09-21T06:26:02.760 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_handle_discover": 0, 2024-09-21T06:26:02.761 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays": 0, 2024-09-21T06:26:02.761 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_delayed": 0, 2024-09-21T06:26:02.761 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:26:02.761 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_created": 0, 2024-09-21T06:26:02.761 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_enqueued": 0, 2024-09-21T06:26:02.762 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_reintegrated": 0, 2024-09-21T06:26:02.762 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_migrated": 0, 2024-09-21T06:26:02.762 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_processing": 0, 2024-09-21T06:26:02.762 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:26:02.762 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:26:02.763 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_started": 0, 2024-09-21T06:26:02.763 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_completed": 0, 2024-09-21T06:26:02.763 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:26:02.763 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:26:02.763 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:26:02.763 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_exportdir": 0, 2024-09-21T06:26:02.764 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_flush": 0, 2024-09-21T06:26:02.764 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:26:02.764 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragstats": 0, 2024-09-21T06:26:02.764 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_inodestats": 0, 2024-09-21T06:26:02.764 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_started": 0, 2024-09-21T06:26:02.765 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_succeeded": 0, 2024-09-21T06:26:02.765 INFO:teuthology.orchestra.run.smithi003.stdout: "uninline_write_failed": 0 2024-09-21T06:26:02.765 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.765 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_log": { 2024-09-21T06:26:02.765 INFO:teuthology.orchestra.run.smithi003.stdout: "evlrg": 0, 2024-09-21T06:26:02.765 INFO:teuthology.orchestra.run.smithi003.stdout: "evadd": 0, 2024-09-21T06:26:02.766 INFO:teuthology.orchestra.run.smithi003.stdout: "evex": 0, 2024-09-21T06:26:02.766 INFO:teuthology.orchestra.run.smithi003.stdout: "evtrm": 0, 2024-09-21T06:26:02.766 INFO:teuthology.orchestra.run.smithi003.stdout: "ev": 5, 2024-09-21T06:26:02.766 INFO:teuthology.orchestra.run.smithi003.stdout: "evexg": 0, 2024-09-21T06:26:02.766 INFO:teuthology.orchestra.run.smithi003.stdout: "evexd": 0, 2024-09-21T06:26:02.767 INFO:teuthology.orchestra.run.smithi003.stdout: "segadd": 0, 2024-09-21T06:26:02.767 INFO:teuthology.orchestra.run.smithi003.stdout: "segex": 0, 2024-09-21T06:26:02.767 INFO:teuthology.orchestra.run.smithi003.stdout: "segtrm": 0, 2024-09-21T06:26:02.767 INFO:teuthology.orchestra.run.smithi003.stdout: "seg": 2, 2024-09-21T06:26:02.767 INFO:teuthology.orchestra.run.smithi003.stdout: "segmjr": 2, 2024-09-21T06:26:02.767 INFO:teuthology.orchestra.run.smithi003.stdout: "segexg": 0, 2024-09-21T06:26:02.768 INFO:teuthology.orchestra.run.smithi003.stdout: "segexd": 0, 2024-09-21T06:26:02.768 INFO:teuthology.orchestra.run.smithi003.stdout: "expos": 4194304, 2024-09-21T06:26:02.768 INFO:teuthology.orchestra.run.smithi003.stdout: "wrpos": 4195732, 2024-09-21T06:26:02.768 INFO:teuthology.orchestra.run.smithi003.stdout: "rdpos": 4195424, 2024-09-21T06:26:02.769 INFO:teuthology.orchestra.run.smithi003.stdout: "jlat": { 2024-09-21T06:26:02.769 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.769 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.769 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.769 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.770 INFO:teuthology.orchestra.run.smithi003.stdout: "replayed": 5 2024-09-21T06:26:02.770 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.770 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_mem": { 2024-09-21T06:26:02.770 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 3, 2024-09-21T06:26:02.770 INFO:teuthology.orchestra.run.smithi003.stdout: "ino+": 3, 2024-09-21T06:26:02.771 INFO:teuthology.orchestra.run.smithi003.stdout: "ino-": 0, 2024-09-21T06:26:02.771 INFO:teuthology.orchestra.run.smithi003.stdout: "dir": 1, 2024-09-21T06:26:02.771 INFO:teuthology.orchestra.run.smithi003.stdout: "dir+": 1, 2024-09-21T06:26:02.771 INFO:teuthology.orchestra.run.smithi003.stdout: "dir-": 0, 2024-09-21T06:26:02.771 INFO:teuthology.orchestra.run.smithi003.stdout: "dn": 0, 2024-09-21T06:26:02.771 INFO:teuthology.orchestra.run.smithi003.stdout: "dn+": 0, 2024-09-21T06:26:02.772 INFO:teuthology.orchestra.run.smithi003.stdout: "dn-": 0, 2024-09-21T06:26:02.772 INFO:teuthology.orchestra.run.smithi003.stdout: "cap": 0, 2024-09-21T06:26:02.772 INFO:teuthology.orchestra.run.smithi003.stdout: "cap+": 0, 2024-09-21T06:26:02.772 INFO:teuthology.orchestra.run.smithi003.stdout: "cap-": 0, 2024-09-21T06:26:02.772 INFO:teuthology.orchestra.run.smithi003.stdout: "rss": 119364, 2024-09-21T06:26:02.773 INFO:teuthology.orchestra.run.smithi003.stdout: "heap": 316424192 2024-09-21T06:26:02.773 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.773 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_server": { 2024-09-21T06:26:02.773 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_client_request": 0, 2024-09-21T06:26:02.773 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_server_request": 0, 2024-09-21T06:26:02.774 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_request": 0, 2024-09-21T06:26:02.774 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_session": 0, 2024-09-21T06:26:02.774 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_peer_request": 0, 2024-09-21T06:26:02.774 INFO:teuthology.orchestra.run.smithi003.stdout: "req_create_latency": { 2024-09-21T06:26:02.774 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.774 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.774 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.775 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.775 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getattr_latency": { 2024-09-21T06:26:02.775 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.775 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.775 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.775 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.775 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getfilelock_latency": { 2024-09-21T06:26:02.775 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.776 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.776 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.776 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.776 INFO:teuthology.orchestra.run.smithi003.stdout: "req_link_latency": { 2024-09-21T06:26:02.776 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.776 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.776 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.776 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.777 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookup_latency": { 2024-09-21T06:26:02.777 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.777 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.777 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.777 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.777 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookuphash_latency": { 2024-09-21T06:26:02.777 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.777 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.778 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.778 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.778 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupino_latency": { 2024-09-21T06:26:02.778 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.778 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.778 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.778 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.779 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupname_latency": { 2024-09-21T06:26:02.779 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.779 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.779 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.779 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.779 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupparent_latency": { 2024-09-21T06:26:02.779 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.779 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.779 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.780 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.780 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupsnap_latency": { 2024-09-21T06:26:02.780 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.780 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.780 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.780 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.780 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lssnap_latency": { 2024-09-21T06:26:02.780 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.780 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.781 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.781 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.781 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mkdir_latency": { 2024-09-21T06:26:02.781 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.782 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.782 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.782 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.782 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mknod_latency": { 2024-09-21T06:26:02.782 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.782 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.782 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.783 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.783 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mksnap_latency": { 2024-09-21T06:26:02.783 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.783 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.783 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.783 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.783 INFO:teuthology.orchestra.run.smithi003.stdout: "req_open_latency": { 2024-09-21T06:26:02.783 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.783 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.784 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.784 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.784 INFO:teuthology.orchestra.run.smithi003.stdout: "req_readdir_latency": { 2024-09-21T06:26:02.784 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.784 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.784 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.784 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.784 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rename_latency": { 2024-09-21T06:26:02.784 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.785 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.785 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.785 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.785 INFO:teuthology.orchestra.run.smithi003.stdout: "req_renamesnap_latency": { 2024-09-21T06:26:02.785 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.785 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.785 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.785 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.786 INFO:teuthology.orchestra.run.smithi003.stdout: "req_snapdiff_latency": { 2024-09-21T06:26:02.786 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.786 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.786 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.786 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.786 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmdir_latency": { 2024-09-21T06:26:02.786 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.786 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.786 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmsnap_latency": { 2024-09-21T06:26:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmxattr_latency": { 2024-09-21T06:26:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.788 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.788 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.788 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setattr_latency": { 2024-09-21T06:26:02.788 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.788 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.788 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.788 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.788 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setdirlayout_latency": { 2024-09-21T06:26:02.788 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.789 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.789 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.789 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.789 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setfilelock_latency": { 2024-09-21T06:26:02.789 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.789 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.789 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.789 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.790 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setlayout_latency": { 2024-09-21T06:26:02.790 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.790 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.790 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.790 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setxattr_latency": { 2024-09-21T06:26:02.790 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.790 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.790 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.791 INFO:teuthology.orchestra.run.smithi003.stdout: "req_symlink_latency": { 2024-09-21T06:26:02.791 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.791 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.791 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.791 INFO:teuthology.orchestra.run.smithi003.stdout: "req_unlink_latency": { 2024-09-21T06:26:02.791 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.792 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.792 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.792 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:26:02.792 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:26:02.792 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getvxattr_latency": { 2024-09-21T06:26:02.792 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.792 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.792 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:26:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_sessions": { 2024-09-21T06:26:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: "session_count": 1, 2024-09-21T06:26:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: "session_add": 2, 2024-09-21T06:26:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: "session_remove": 1, 2024-09-21T06:26:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_open": 1, 2024-09-21T06:26:02.794 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_stale": 0, 2024-09-21T06:26:02.794 INFO:teuthology.orchestra.run.smithi003.stdout: "total_load": 0, 2024-09-21T06:26:02.794 INFO:teuthology.orchestra.run.smithi003.stdout: "average_load": 0, 2024-09-21T06:26:02.794 INFO:teuthology.orchestra.run.smithi003.stdout: "avg_session_uptime": 1766, 2024-09-21T06:26:02.794 INFO:teuthology.orchestra.run.smithi003.stdout: "mdthresh_evicted": 0 2024-09-21T06:26:02.794 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.794 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-09-21T06:26:02.794 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:26:02.794 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-09-21T06:26:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:26:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:26:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:26:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:26:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:26:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:26:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:26:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:26:02.795 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:26:02.796 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:26:02.796 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:26:02.796 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:26:02.796 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:26:02.796 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-09-21T06:26:02.796 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:26:02.796 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-09-21T06:26:02.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:26:02.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:26:02.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:26:02.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:26:02.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:26:02.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:26:02.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:26:02.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-09-21T06:26:02.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:26:02.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:26:02.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:26:02.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-09-21T06:26:02.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-09-21T06:26:02.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-09-21T06:26:02.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:26:02.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:26:02.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:26:02.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:26:02.799 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 197463, 2024-09-21T06:26:02.799 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 54, 2024-09-21T06:26:02.799 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:26:02.799 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-09-21T06:26:02.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-09-21T06:26:02.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-09-21T06:26:02.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:26:02.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-09-21T06:26:02.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:26:02.800 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-09-21T06:26:02.800 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 8168, 2024-09-21T06:26:02.800 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 181, 2024-09-21T06:26:02.800 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:26:02.800 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:26:02.800 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-09-21T06:26:02.800 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-09-21T06:26:02.800 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 8672, 2024-09-21T06:26:02.800 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 13, 2024-09-21T06:26:02.801 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-09-21T06:26:02.801 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-09-21T06:26:02.801 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-09-21T06:26:02.801 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-09-21T06:26:02.801 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.801 INFO:teuthology.orchestra.run.smithi003.stdout: "objecter": { 2024-09-21T06:26:02.801 INFO:teuthology.orchestra.run.smithi003.stdout: "op_active": 0, 2024-09-21T06:26:02.801 INFO:teuthology.orchestra.run.smithi003.stdout: "op_laggy": 0, 2024-09-21T06:26:02.802 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send": 2177, 2024-09-21T06:26:02.802 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send_bytes": 21, 2024-09-21T06:26:02.802 INFO:teuthology.orchestra.run.smithi003.stdout: "op_resend": 3, 2024-09-21T06:26:02.802 INFO:teuthology.orchestra.run.smithi003.stdout: "op_reply": 2177, 2024-09-21T06:26:02.802 INFO:teuthology.orchestra.run.smithi003.stdout: "op_latency": { 2024-09-21T06:26:02.802 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 2177, 2024-09-21T06:26:02.802 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 1.571978709, 2024-09-21T06:26:02.802 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000722084 2024-09-21T06:26:02.802 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.803 INFO:teuthology.orchestra.run.smithi003.stdout: "op_inflight": 0, 2024-09-21T06:26:02.803 INFO:teuthology.orchestra.run.smithi003.stdout: "oplen_avg": { 2024-09-21T06:26:02.803 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 2177, 2024-09-21T06:26:02.803 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 2178 2024-09-21T06:26:02.803 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.803 INFO:teuthology.orchestra.run.smithi003.stdout: "op": 2177, 2024-09-21T06:26:02.803 INFO:teuthology.orchestra.run.smithi003.stdout: "op_r": 2177, 2024-09-21T06:26:02.803 INFO:teuthology.orchestra.run.smithi003.stdout: "op_w": 0, 2024-09-21T06:26:02.803 INFO:teuthology.orchestra.run.smithi003.stdout: "op_rmw": 0, 2024-09-21T06:26:02.804 INFO:teuthology.orchestra.run.smithi003.stdout: "op_pg": 0, 2024-09-21T06:26:02.804 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_stat": 1410, 2024-09-21T06:26:02.804 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_create": 0, 2024-09-21T06:26:02.804 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_read": 765, 2024-09-21T06:26:02.804 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_write": 0, 2024-09-21T06:26:02.804 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writefull": 0, 2024-09-21T06:26:02.804 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writesame": 0, 2024-09-21T06:26:02.804 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_append": 0, 2024-09-21T06:26:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_zero": 0, 2024-09-21T06:26:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_truncate": 0, 2024-09-21T06:26:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_delete": 0, 2024-09-21T06:26:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_mapext": 0, 2024-09-21T06:26:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_sparse_read": 0, 2024-09-21T06:26:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_clonerange": 0, 2024-09-21T06:26:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_getxattr": 1, 2024-09-21T06:26:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_setxattr": 0, 2024-09-21T06:26:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:26:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_rmxattr": 0, 2024-09-21T06:26:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:26:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_call": 0, 2024-09-21T06:26:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_watch": 0, 2024-09-21T06:26:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_notify": 0, 2024-09-21T06:26:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:26:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls": 0, 2024-09-21T06:26:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:26:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_other": 0, 2024-09-21T06:26:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_active": 0, 2024-09-21T06:26:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_send": 0, 2024-09-21T06:26:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_resend": 0, 2024-09-21T06:26:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_ping": 0, 2024-09-21T06:26:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_active": 0, 2024-09-21T06:26:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_send": 0, 2024-09-21T06:26:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_resend": 0, 2024-09-21T06:26:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_active": 0, 2024-09-21T06:26:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_send": 0, 2024-09-21T06:26:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_resend": 0, 2024-09-21T06:26:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_active": 0, 2024-09-21T06:26:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_send": 0, 2024-09-21T06:26:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_resend": 0, 2024-09-21T06:26:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: "command_active": 0, 2024-09-21T06:26:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: "command_send": 0, 2024-09-21T06:26:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: "command_resend": 0, 2024-09-21T06:26:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: "map_epoch": 89, 2024-09-21T06:26:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: "map_full": 0, 2024-09-21T06:26:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: "map_inc": 11, 2024-09-21T06:26:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_sessions": 6, 2024-09-21T06:26:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_open": 6, 2024-09-21T06:26:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_close": 0, 2024-09-21T06:26:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_laggy": 0, 2024-09-21T06:26:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_wr": 0, 2024-09-21T06:26:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_rd": 2, 2024-09-21T06:26:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_del": 0 2024-09-21T06:26:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: "oft": { 2024-09-21T06:26:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_objs": 0, 2024-09-21T06:26:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:26:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_updates": 0, 2024-09-21T06:26:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_removes": 0 2024-09-21T06:26:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: "purge_queue": { 2024-09-21T06:26:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops": 0, 2024-09-21T06:26:02.812 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:26:02.812 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing": 0, 2024-09-21T06:26:02.812 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_high_water": 0, 2024-09-21T06:26:02.812 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executed_ops": 0, 2024-09-21T06:26:02.813 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executed": 0, 2024-09-21T06:26:02.813 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_item_in_journal": 0 2024-09-21T06:26:02.813 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.813 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:26:02.813 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-21T06:26:02.814 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-09-21T06:26:02.814 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-21T06:26:02.814 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 2425, 2024-09-21T06:26:02.814 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1635722, 2024-09-21T06:26:02.815 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:02.815 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 2425, 2024-09-21T06:26:02.815 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-21T06:26:02.815 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-21T06:26:02.816 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 2425, 2024-09-21T06:26:02.816 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1635722, 2024-09-21T06:26:02.816 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-21T06:26:02.816 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.817 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.817 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.817 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:26:02.817 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.817 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_bytes": { 2024-09-21T06:26:02.818 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-21T06:26:02.818 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-09-21T06:26:02.818 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-21T06:26:02.818 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-21T06:26:02.819 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-21T06:26:02.819 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:02.819 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:02.819 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 2177, 2024-09-21T06:26:02.819 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 1433, 2024-09-21T06:26:02.819 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 2177, 2024-09-21T06:26:02.820 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1433, 2024-09-21T06:26:02.820 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-21T06:26:02.820 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.820 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.820 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.820 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:26:02.820 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.820 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_ops": { 2024-09-21T06:26:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-21T06:26:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 1024, 2024-09-21T06:26:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-21T06:26:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-21T06:26:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-21T06:26:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 2177, 2024-09-21T06:26:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 2177, 2024-09-21T06:26:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 2177, 2024-09-21T06:26:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 2177, 2024-09-21T06:26:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-21T06:26:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:26:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:26:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-21T06:26:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2024-09-21T06:26:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-21T06:26:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-21T06:26:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-21T06:26:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:02.824 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-21T06:26:02.824 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-21T06:26:02.824 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 0, 2024-09-21T06:26:02.824 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 0, 2024-09-21T06:26:02.824 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-21T06:26:02.825 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.825 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.825 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.825 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:26:02.825 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.826 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle-0x5635b6b335f0": { 2024-09-21T06:26:02.826 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-21T06:26:02.826 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2024-09-21T06:26:02.826 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-21T06:26:02.826 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-21T06:26:02.826 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-21T06:26:02.827 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:02.827 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:02.827 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-21T06:26:02.827 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-21T06:26:02.828 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 0, 2024-09-21T06:26:02.828 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 0, 2024-09-21T06:26:02.828 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-21T06:26:02.829 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:02.829 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:02.829 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:02.830 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:26:02.830 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:02.830 INFO:teuthology.orchestra.run.smithi003.stdout: "trackedop": { 2024-09-21T06:26:02.830 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_ops_count": 0 2024-09-21T06:26:02.830 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:26:02.830 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:26:03.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:02 smithi003 ceph-mon[31227]: pgmap v666: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T06:26:03.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:02 smithi137 ceph-mon[34464]: pgmap v666: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T06:26:03.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:02 smithi204 ceph-mon[34659]: pgmap v666: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T06:26:03.736 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-21T06:26:03.737 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-21T06:26:03.737 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-21T06:26:03.737 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:26:03.737 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:26:03.737 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T06:26:03.737 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T06:26:04.001 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:04.505 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-21T06:26:04.506 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:26:04.506 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 0, 2024-09-21T06:26:04.506 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 2, 2024-09-21T06:26:04.506 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 0, 2024-09-21T06:26:04.506 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 0, 2024-09-21T06:26:04.506 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-09-21T06:26:04.506 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-09-21T06:26:04.507 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.001294429, 2024-09-21T06:26:04.507 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T06:26:04.507 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.000270779, 2024-09-21T06:26:04.507 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:26:04.507 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:04.507 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:04.507 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:04.507 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:04.508 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:04.508 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:04.508 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:04.508 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:04.508 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:04.508 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:04.508 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:26:04.508 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:26:04.509 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:04.509 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:26:04.509 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 1, 2024-09-21T06:26:04.509 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 165, 2024-09-21T06:26:04.509 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 160, 2024-09-21T06:26:04.509 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 109280, 2024-09-21T06:26:04.509 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 3, 2024-09-21T06:26:04.509 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 2, 2024-09-21T06:26:04.510 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.016350932, 2024-09-21T06:26:04.510 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.012638162, 2024-09-21T06:26:04.510 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.002070348, 2024-09-21T06:26:04.510 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:26:04.510 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:04.510 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 163, 2024-09-21T06:26:04.510 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.007646585, 2024-09-21T06:26:04.510 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000046911 2024-09-21T06:26:04.510 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:04.511 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:04.511 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:04.511 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:04.511 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:04.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:04.511 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:26:04.511 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 109280 2024-09-21T06:26:04.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:04.511 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:26:04.512 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 228, 2024-09-21T06:26:04.512 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 219, 2024-09-21T06:26:04.512 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1150624, 2024-09-21T06:26:04.512 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 134096, 2024-09-21T06:26:04.512 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-09-21T06:26:04.512 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-09-21T06:26:04.512 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.045436243, 2024-09-21T06:26:04.512 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.017404818, 2024-09-21T06:26:04.513 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.039321244, 2024-09-21T06:26:04.513 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:26:04.513 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:04.513 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 219, 2024-09-21T06:26:04.513 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.013740512, 2024-09-21T06:26:04.513 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000062742 2024-09-21T06:26:04.513 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:04.513 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:04.514 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:04.514 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:04.514 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:04.514 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:04.514 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1150624, 2024-09-21T06:26:04.514 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 134096 2024-09-21T06:26:04.514 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:04.514 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-09-21T06:26:04.514 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-09-21T06:26:04.515 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-09-21T06:26:04.515 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:04.515 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-09-21T06:26:04.515 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:26:04.515 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-09-21T06:26:04.515 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:26:04.515 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:26:04.515 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:26:04.515 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:26:04.515 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:26:04.516 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:26:04.516 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:26:04.516 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:26:04.516 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:26:04.516 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:26:04.516 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:26:04.516 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:26:04.516 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:26:04.516 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-09-21T06:26:04.516 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:26:04.517 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-09-21T06:26:04.517 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:26:04.517 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:26:04.517 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:26:04.517 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:26:04.517 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:26:04.517 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:26:04.517 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:26:04.518 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-09-21T06:26:04.518 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:26:04.518 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:26:04.518 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:26:04.518 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-09-21T06:26:04.518 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-09-21T06:26:04.518 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-09-21T06:26:04.518 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:26:04.518 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:26:04.519 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:26:04.519 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:26:04.519 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 149998, 2024-09-21T06:26:04.519 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 30, 2024-09-21T06:26:04.519 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:26:04.519 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-09-21T06:26:04.519 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-09-21T06:26:04.519 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-09-21T06:26:04.519 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:26:04.520 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-09-21T06:26:04.520 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:26:04.520 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-09-21T06:26:04.520 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-09-21T06:26:04.520 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-09-21T06:26:04.520 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:26:04.520 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:26:04.520 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-09-21T06:26:04.520 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-09-21T06:26:04.520 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 968, 2024-09-21T06:26:04.521 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-09-21T06:26:04.521 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-09-21T06:26:04.521 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-09-21T06:26:04.521 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-09-21T06:26:04.521 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-09-21T06:26:04.521 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-21T06:26:04.521 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:26:04.521 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-21T06:26:04.521 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-09-21T06:26:04.521 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-21T06:26:04.522 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 225, 2024-09-21T06:26:04.522 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1115997, 2024-09-21T06:26:04.522 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:04.522 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 225, 2024-09-21T06:26:04.522 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-21T06:26:04.522 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-21T06:26:04.522 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 225, 2024-09-21T06:26:04.522 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1115997, 2024-09-21T06:26:04.522 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-21T06:26:04.523 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-21T06:26:04.523 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-21T06:26:04.523 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-21T06:26:04.523 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:26:04.523 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-21T06:26:04.523 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-21T06:26:05.320 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:04 smithi003 ceph-mon[31227]: pgmap v667: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:26:05.354 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T06:26:05.354 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T06:26:05.354 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-21T06:26:05.354 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-21T06:26:05.354 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-21T06:26:05.354 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T06:26:05.355 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T06:26:05.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:04 smithi137 ceph-mon[34464]: pgmap v667: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:26:05.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:04 smithi204 ceph-mon[34659]: pgmap v667: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:26:05.693 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:26:06.453 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-21T06:26:06.453 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:26:06.454 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 385436, 2024-09-21T06:26:06.454 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 566920, 2024-09-21T06:26:06.454 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 109896620, 2024-09-21T06:26:06.454 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 610254574, 2024-09-21T06:26:06.454 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 26, 2024-09-21T06:26:06.454 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 6, 2024-09-21T06:26:06.454 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 26.793466599, 2024-09-21T06:26:06.455 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 14.850124687, 2024-09-21T06:26:06.455 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 12.845243154, 2024-09-21T06:26:06.455 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 1.171870570, 2024-09-21T06:26:06.455 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:06.455 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 566918, 2024-09-21T06:26:06.455 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 22.476895196, 2024-09-21T06:26:06.455 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000039647 2024-09-21T06:26:06.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.456 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:06.456 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 843487, 2024-09-21T06:26:06.456 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.133231500, 2024-09-21T06:26:06.456 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000157 2024-09-21T06:26:06.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.456 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 380832, 2024-09-21T06:26:06.456 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 131440 2024-09-21T06:26:06.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.457 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:26:06.457 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 114492, 2024-09-21T06:26:06.457 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 185941, 2024-09-21T06:26:06.457 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 26199682, 2024-09-21T06:26:06.457 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 286744111, 2024-09-21T06:26:06.457 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 26, 2024-09-21T06:26:06.457 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 8, 2024-09-21T06:26:06.458 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 10.167051303, 2024-09-21T06:26:06.458 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 5.303387982, 2024-09-21T06:26:06.458 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 3.557205318, 2024-09-21T06:26:06.458 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 1.212961546, 2024-09-21T06:26:06.458 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:06.458 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 185936, 2024-09-21T06:26:06.458 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 23.224198689, 2024-09-21T06:26:06.458 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000124904 2024-09-21T06:26:06.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.459 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:06.459 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 213314, 2024-09-21T06:26:06.459 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.037860433, 2024-09-21T06:26:06.459 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000177 2024-09-21T06:26:06.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.459 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:26:06.459 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 377440 2024-09-21T06:26:06.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.460 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:26:06.460 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 194532, 2024-09-21T06:26:06.460 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 318638, 2024-09-21T06:26:06.460 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 39363705, 2024-09-21T06:26:06.460 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 246855505, 2024-09-21T06:26:06.460 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 176, 2024-09-21T06:26:06.460 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 8, 2024-09-21T06:26:06.460 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 15.485593689, 2024-09-21T06:26:06.460 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 8.577293269, 2024-09-21T06:26:06.461 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 7.082656198, 2024-09-21T06:26:06.461 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.601542123, 2024-09-21T06:26:06.461 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:06.461 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 318641, 2024-09-21T06:26:06.461 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 13.110094988, 2024-09-21T06:26:06.461 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000041143 2024-09-21T06:26:06.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.462 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:06.462 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 469442, 2024-09-21T06:26:06.462 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.072297125, 2024-09-21T06:26:06.462 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000154 2024-09-21T06:26:06.462 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.462 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1327952, 2024-09-21T06:26:06.462 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 160464 2024-09-21T06:26:06.462 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.462 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-09-21T06:26:06.463 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-09-21T06:26:06.463 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-09-21T06:26:06.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.463 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-09-21T06:26:06.463 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-21T06:26:06.463 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-21T06:26:06.463 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 104782, 2024-09-21T06:26:06.464 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 84.069930250, 2024-09-21T06:26:06.464 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000802331 2024-09-21T06:26:06.464 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:06.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.464 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-09-21T06:26:06.464 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-21T06:26:06.464 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-21T06:26:06.465 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 42857, 2024-09-21T06:26:06.465 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 4.298775874, 2024-09-21T06:26:06.465 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000100305 2024-09-21T06:26:06.465 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:06.465 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.465 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-09-21T06:26:06.465 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 196912, 2024-09-21T06:26:06.465 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 192193, 2024-09-21T06:26:06.465 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-09-21T06:26:06.466 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 192193, 2024-09-21T06:26:06.466 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 298.836519637, 2024-09-21T06:26:06.466 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001554877 2024-09-21T06:26:06.466 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.466 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-09-21T06:26:06.466 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 4719, 2024-09-21T06:26:06.466 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 5, 2024-09-21T06:26:06.466 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 2, 2024-09-21T06:26:06.466 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 2262, 2024-09-21T06:26:06.467 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 15, 2024-09-21T06:26:06.467 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 15, 2024-09-21T06:26:06.467 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 1916, 2024-09-21T06:26:06.467 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 654, 2024-09-21T06:26:06.467 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 1235, 2024-09-21T06:26:06.467 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 27, 2024-09-21T06:26:06.467 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 981, 2024-09-21T06:26:06.467 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 64, 2024-09-21T06:26:06.468 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 39, 2024-09-21T06:26:06.468 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 39, 2024-09-21T06:26:06.468 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 5, 2024-09-21T06:26:06.468 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 288137, 2024-09-21T06:26:06.468 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 189277, 2024-09-21T06:26:06.468 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 4719, 2024-09-21T06:26:06.468 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-09-21T06:26:06.469 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:26:06.469 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-09-21T06:26:06.469 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 2424, 2024-09-21T06:26:06.469 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-09-21T06:26:06.469 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-09-21T06:26:06.469 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 2, 2024-09-21T06:26:06.469 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 53, 2024-09-21T06:26:06.469 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-09-21T06:26:06.470 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-09-21T06:26:06.470 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-09-21T06:26:06.470 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 1160, 2024-09-21T06:26:06.470 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-09-21T06:26:06.470 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 87, 2024-09-21T06:26:06.470 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 23676942, 2024-09-21T06:26:06.470 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 6, 2024-09-21T06:26:06.470 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 1160, 2024-09-21T06:26:06.470 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-09-21T06:26:06.471 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:26:06.471 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:26:06.471 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 275, 2024-09-21T06:26:06.471 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 15, 2024-09-21T06:26:06.471 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 260, 2024-09-21T06:26:06.471 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 885, 2024-09-21T06:26:06.471 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 3, 2024-09-21T06:26:06.471 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 17559, 2024-09-21T06:26:06.471 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 96322, 2024-09-21T06:26:06.472 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:26:06.472 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:26:06.472 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:26:06.472 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 64483, 2024-09-21T06:26:06.472 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 25554, 2024-09-21T06:26:06.472 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 12504, 2024-09-21T06:26:06.472 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 126177 2024-09-21T06:26:06.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.473 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-09-21T06:26:06.473 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 12, 2024-09-21T06:26:06.473 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-09-21T06:26:06.473 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-09-21T06:26:06.473 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-09-21T06:26:06.473 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 166, 2024-09-21T06:26:06.473 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-09-21T06:26:06.473 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-09-21T06:26:06.473 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:26:06.474 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 23309, 2024-09-21T06:26:06.474 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 23309, 2024-09-21T06:26:06.474 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-09-21T06:26:06.474 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-09-21T06:26:06.474 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-09-21T06:26:06.474 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:26:06.474 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:26:06.474 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-09-21T06:26:06.475 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-09-21T06:26:06.475 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:26:06.475 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:26:06.475 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 15, 2024-09-21T06:26:06.475 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 3, 2024-09-21T06:26:06.475 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-09-21T06:26:06.475 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 30, 2024-09-21T06:26:06.475 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-09-21T06:26:06.475 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0, 2024-09-21T06:26:06.476 INFO:teuthology.orchestra.run.smithi137.stdout: "uninline_started": 0, 2024-09-21T06:26:06.476 INFO:teuthology.orchestra.run.smithi137.stdout: "uninline_succeeded": 0, 2024-09-21T06:26:06.476 INFO:teuthology.orchestra.run.smithi137.stdout: "uninline_write_failed": 0 2024-09-21T06:26:06.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.476 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-09-21T06:26:06.476 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-09-21T06:26:06.476 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 133799, 2024-09-21T06:26:06.476 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 130579, 2024-09-21T06:26:06.476 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 130579, 2024-09-21T06:26:06.477 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 3220, 2024-09-21T06:26:06.477 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-09-21T06:26:06.477 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 2995, 2024-09-21T06:26:06.477 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 211, 2024-09-21T06:26:06.477 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 207, 2024-09-21T06:26:06.477 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 207, 2024-09-21T06:26:06.477 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 4, 2024-09-21T06:26:06.477 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 1, 2024-09-21T06:26:06.477 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-09-21T06:26:06.478 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 5, 2024-09-21T06:26:06.478 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 474585867, 2024-09-21T06:26:06.478 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 486049915, 2024-09-21T06:26:06.478 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-09-21T06:26:06.478 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-09-21T06:26:06.478 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 53509, 2024-09-21T06:26:06.478 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 146.097842543, 2024-09-21T06:26:06.478 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002730341 2024-09-21T06:26:06.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.479 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-09-21T06:26:06.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.479 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-09-21T06:26:06.479 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 612, 2024-09-21T06:26:06.479 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 23948, 2024-09-21T06:26:06.479 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 23336, 2024-09-21T06:26:06.479 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 59, 2024-09-21T06:26:06.479 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 169, 2024-09-21T06:26:06.480 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 110, 2024-09-21T06:26:06.480 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 1916, 2024-09-21T06:26:06.480 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 49683, 2024-09-21T06:26:06.480 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 47767, 2024-09-21T06:26:06.480 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 39, 2024-09-21T06:26:06.480 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 26788, 2024-09-21T06:26:06.480 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 26749, 2024-09-21T06:26:06.480 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 154748, 2024-09-21T06:26:06.481 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 377368576 2024-09-21T06:26:06.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.481 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-09-21T06:26:06.481 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 281071, 2024-09-21T06:26:06.481 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 9759, 2024-09-21T06:26:06.481 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 196912, 2024-09-21T06:26:06.481 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 4686, 2024-09-21T06:26:06.481 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 19518, 2024-09-21T06:26:06.481 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-09-21T06:26:06.482 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 23396, 2024-09-21T06:26:06.482 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 28.115091676, 2024-09-21T06:26:06.482 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001201705 2024-09-21T06:26:06.482 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.482 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-09-21T06:26:06.482 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 9938, 2024-09-21T06:26:06.482 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 62.478052225, 2024-09-21T06:26:06.482 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.006286783 2024-09-21T06:26:06.483 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.483 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-09-21T06:26:06.483 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.483 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.483 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.483 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.483 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-09-21T06:26:06.483 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.483 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.484 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.484 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.484 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-09-21T06:26:06.484 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 57869, 2024-09-21T06:26:06.484 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 29.968469093, 2024-09-21T06:26:06.484 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000517867 2024-09-21T06:26:06.484 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.484 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-09-21T06:26:06.484 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.485 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.485 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.485 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.485 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-09-21T06:26:06.485 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.485 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.485 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.485 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.485 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-09-21T06:26:06.486 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.486 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.486 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.486 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.486 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-09-21T06:26:06.486 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.486 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.487 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.487 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.487 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-09-21T06:26:06.487 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.488 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.488 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.488 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.488 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-09-21T06:26:06.488 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 11, 2024-09-21T06:26:06.488 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.002581448, 2024-09-21T06:26:06.488 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000234677 2024-09-21T06:26:06.488 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.489 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-09-21T06:26:06.489 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 20, 2024-09-21T06:26:06.489 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.022698703, 2024-09-21T06:26:06.489 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001134935 2024-09-21T06:26:06.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.489 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-09-21T06:26:06.489 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.489 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.489 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.490 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.490 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-09-21T06:26:06.490 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 11, 2024-09-21T06:26:06.490 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2.796649148, 2024-09-21T06:26:06.490 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.254240831 2024-09-21T06:26:06.490 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.490 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-09-21T06:26:06.490 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1, 2024-09-21T06:26:06.491 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000432262, 2024-09-21T06:26:06.491 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000432262 2024-09-21T06:26:06.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.491 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-09-21T06:26:06.491 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 69698, 2024-09-21T06:26:06.491 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 133.262923703, 2024-09-21T06:26:06.491 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001912004 2024-09-21T06:26:06.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.491 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-09-21T06:26:06.492 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 6868, 2024-09-21T06:26:06.492 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 10.647494351, 2024-09-21T06:26:06.492 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001550304 2024-09-21T06:26:06.492 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.492 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-09-21T06:26:06.492 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.492 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.492 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.492 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.493 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-09-21T06:26:06.493 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.493 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.493 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.493 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.493 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-09-21T06:26:06.493 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.493 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.494 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.494 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.494 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-09-21T06:26:06.494 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 5, 2024-09-21T06:26:06.494 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.035009820, 2024-09-21T06:26:06.494 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.007001964 2024-09-21T06:26:06.494 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.494 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-09-21T06:26:06.495 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.495 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.495 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.495 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-09-21T06:26:06.495 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1, 2024-09-21T06:26:06.495 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001233721, 2024-09-21T06:26:06.495 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001233721 2024-09-21T06:26:06.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.496 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-09-21T06:26:06.496 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.496 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.496 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.496 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-09-21T06:26:06.496 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1056, 2024-09-21T06:26:06.496 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.725315043, 2024-09-21T06:26:06.496 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000686851 2024-09-21T06:26:06.497 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.497 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-09-21T06:26:06.497 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.497 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.497 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.497 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.497 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-09-21T06:26:06.497 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 10, 2024-09-21T06:26:06.498 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.022666623, 2024-09-21T06:26:06.498 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002266662 2024-09-21T06:26:06.498 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.498 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-09-21T06:26:06.498 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.498 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.498 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.499 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.499 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-09-21T06:26:06.499 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 23309, 2024-09-21T06:26:06.499 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 30.757901821, 2024-09-21T06:26:06.499 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001319571 2024-09-21T06:26:06.499 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.499 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:26:06.499 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:26:06.499 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-09-21T06:26:06.500 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.500 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.500 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.500 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:06.500 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.500 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-09-21T06:26:06.500 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 2, 2024-09-21T06:26:06.500 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 4, 2024-09-21T06:26:06.501 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 2, 2024-09-21T06:26:06.501 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 2, 2024-09-21T06:26:06.501 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-09-21T06:26:06.501 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 9446, 2024-09-21T06:26:06.501 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 4723, 2024-09-21T06:26:06.501 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 631, 2024-09-21T06:26:06.501 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-09-21T06:26:06.501 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.501 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-09-21T06:26:06.502 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:26:06.502 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-09-21T06:26:06.502 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:26:06.502 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:26:06.502 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:26:06.502 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:26:06.502 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:26:06.502 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:26:06.503 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:26:06.503 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:26:06.503 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:26:06.503 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:26:06.503 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:26:06.503 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:26:06.503 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:26:06.503 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-09-21T06:26:06.503 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:26:06.504 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-09-21T06:26:06.504 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:26:06.504 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:26:06.504 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:26:06.504 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:26:06.504 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:26:06.504 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:26:06.504 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:26:06.505 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-09-21T06:26:06.505 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:26:06.505 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:26:06.505 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:26:06.505 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-09-21T06:26:06.505 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-09-21T06:26:06.505 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-09-21T06:26:06.505 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:26:06.505 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:26:06.506 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:26:06.506 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:26:06.506 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 348233, 2024-09-21T06:26:06.506 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 454, 2024-09-21T06:26:06.506 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 88, 2024-09-21T06:26:06.506 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 1, 2024-09-21T06:26:06.507 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-09-21T06:26:06.507 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-09-21T06:26:06.507 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:26:06.507 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-09-21T06:26:06.507 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:26:06.507 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-09-21T06:26:06.507 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8168, 2024-09-21T06:26:06.507 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 181, 2024-09-21T06:26:06.508 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:26:06.508 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:26:06.508 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-09-21T06:26:06.508 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-09-21T06:26:06.508 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 3002681, 2024-09-21T06:26:06.508 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 29571, 2024-09-21T06:26:06.508 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-09-21T06:26:06.508 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-09-21T06:26:06.509 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-09-21T06:26:06.509 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-09-21T06:26:06.509 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.509 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-09-21T06:26:06.509 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-09-21T06:26:06.509 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-09-21T06:26:06.509 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 97843, 2024-09-21T06:26:06.509 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 504586010, 2024-09-21T06:26:06.509 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 37, 2024-09-21T06:26:06.510 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 97843, 2024-09-21T06:26:06.510 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-09-21T06:26:06.510 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 97843, 2024-09-21T06:26:06.510 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1092.981712037, 2024-09-21T06:26:06.510 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.011170770 2024-09-21T06:26:06.510 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.510 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-09-21T06:26:06.510 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-09-21T06:26:06.511 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 97843, 2024-09-21T06:26:06.511 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 132185 2024-09-21T06:26:06.511 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.511 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 97843, 2024-09-21T06:26:06.511 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 1968, 2024-09-21T06:26:06.511 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 95875, 2024-09-21T06:26:06.511 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-09-21T06:26:06.511 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-09-21T06:26:06.512 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 2097, 2024-09-21T06:26:06.512 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 14110, 2024-09-21T06:26:06.512 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 786, 2024-09-21T06:26:06.512 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 54280, 2024-09-21T06:26:06.512 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 513, 2024-09-21T06:26:06.512 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-09-21T06:26:06.512 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-09-21T06:26:06.512 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-09-21T06:26:06.512 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 15, 2024-09-21T06:26:06.513 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 24140, 2024-09-21T06:26:06.513 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-09-21T06:26:06.513 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-09-21T06:26:06.513 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-09-21T06:26:06.513 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 1182, 2024-09-21T06:26:06.513 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 28220, 2024-09-21T06:26:06.513 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:26:06.513 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-09-21T06:26:06.514 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:26:06.514 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-09-21T06:26:06.514 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-09-21T06:26:06.514 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-09-21T06:26:06.514 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:26:06.514 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-09-21T06:26:06.514 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:26:06.514 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-09-21T06:26:06.515 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-09-21T06:26:06.515 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-09-21T06:26:06.515 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-09-21T06:26:06.515 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-09-21T06:26:06.515 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-09-21T06:26:06.515 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-09-21T06:26:06.515 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-09-21T06:26:06.515 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-09-21T06:26:06.515 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-09-21T06:26:06.516 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-09-21T06:26:06.516 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-09-21T06:26:06.516 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-09-21T06:26:06.516 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-09-21T06:26:06.516 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-09-21T06:26:06.516 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-09-21T06:26:06.516 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-09-21T06:26:06.516 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 90, 2024-09-21T06:26:06.517 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-09-21T06:26:06.517 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 13, 2024-09-21T06:26:06.517 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 12, 2024-09-21T06:26:06.517 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 12, 2024-09-21T06:26:06.517 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-09-21T06:26:06.517 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-09-21T06:26:06.517 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 5052, 2024-09-21T06:26:06.517 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 14, 2024-09-21T06:26:06.518 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 1776 2024-09-21T06:26:06.518 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.518 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-09-21T06:26:06.518 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-09-21T06:26:06.518 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:26:06.518 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-09-21T06:26:06.518 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-09-21T06:26:06.518 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.519 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-09-21T06:26:06.519 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-09-21T06:26:06.519 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 6, 2024-09-21T06:26:06.519 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-09-21T06:26:06.519 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 6, 2024-09-21T06:26:06.519 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 23309, 2024-09-21T06:26:06.519 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 23309, 2024-09-21T06:26:06.519 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-09-21T06:26:06.520 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.520 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:26:06.520 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:06.520 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-21T06:26:06.520 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:06.520 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 694456, 2024-09-21T06:26:06.520 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 112895775, 2024-09-21T06:26:06.520 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:06.521 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 694456, 2024-09-21T06:26:06.521 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-21T06:26:06.521 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-21T06:26:06.521 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 694456, 2024-09-21T06:26:06.521 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 112895775, 2024-09-21T06:26:06.521 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:06.521 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.521 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.522 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.522 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:06.522 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.522 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-09-21T06:26:06.522 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:06.522 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-21T06:26:06.522 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:06.522 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-21T06:26:06.523 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-21T06:26:06.523 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:06.523 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:06.523 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 97843, 2024-09-21T06:26:06.523 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 506944043, 2024-09-21T06:26:06.523 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 97843, 2024-09-21T06:26:06.523 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 506944043, 2024-09-21T06:26:06.523 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:06.524 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.524 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.524 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.524 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:06.524 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.524 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-09-21T06:26:06.524 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:06.524 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-09-21T06:26:06.525 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:06.525 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-21T06:26:06.525 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-21T06:26:06.525 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:06.525 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:06.525 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 97843, 2024-09-21T06:26:06.525 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 97843, 2024-09-21T06:26:06.525 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 97843, 2024-09-21T06:26:06.526 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 97843, 2024-09-21T06:26:06.526 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:06.526 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.526 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.526 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.526 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:06.526 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.526 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:26:06.527 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:06.527 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-21T06:26:06.527 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:06.527 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 23309, 2024-09-21T06:26:06.527 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 2358121, 2024-09-21T06:26:06.527 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:06.527 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 23309, 2024-09-21T06:26:06.527 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-21T06:26:06.527 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-21T06:26:06.528 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 771, 2024-09-21T06:26:06.528 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 2358121, 2024-09-21T06:26:06.528 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:06.528 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.528 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.528 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.528 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:06.528 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.528 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55c8e275d2b0": { 2024-09-21T06:26:06.529 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 112, 2024-09-21T06:26:06.529 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-21T06:26:06.529 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:06.529 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 133799, 2024-09-21T06:26:06.529 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 481855611, 2024-09-21T06:26:06.529 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:06.529 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 133799, 2024-09-21T06:26:06.529 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-21T06:26:06.530 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-21T06:26:06.530 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 53509, 2024-09-21T06:26:06.530 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 481855499, 2024-09-21T06:26:06.530 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:06.530 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:06.530 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:06.530 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:06.530 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:06.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:06.531 INFO:teuthology.orchestra.run.smithi137.stdout: "trackedop": { 2024-09-21T06:26:06.531 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-09-21T06:26:06.531 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:06.531 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-21T06:26:07.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:06 smithi137 ceph-mon[34464]: pgmap v668: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:26:07.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:06 smithi137 ceph-mon[34464]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T06:26:07.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:06 smithi003 ceph-mon[31227]: pgmap v668: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:26:07.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:06 smithi003 ceph-mon[31227]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T06:26:07.403 INFO:tasks.check_counter:Daemon mds.b mds.exported=2 2024-09-21T06:26:07.404 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-21T06:26:07.404 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-21T06:26:07.404 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-09-21T06:26:07.404 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-09-21T06:26:07.404 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T06:26:07.404 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T06:26:07.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:06 smithi204 ceph-mon[34659]: pgmap v668: 97 pgs: 97 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:26:07.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:06 smithi204 ceph-mon[34659]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T06:26:07.691 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:26:08.231 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-21T06:26:08.231 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:26:08.232 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 352, 2024-09-21T06:26:08.232 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 349, 2024-09-21T06:26:08.232 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 17080458, 2024-09-21T06:26:08.232 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 124744, 2024-09-21T06:26:08.232 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 7, 2024-09-21T06:26:08.232 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-09-21T06:26:08.232 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.050234559, 2024-09-21T06:26:08.233 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.020938510, 2024-09-21T06:26:08.233 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.023241512, 2024-09-21T06:26:08.233 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.007288235, 2024-09-21T06:26:08.233 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:08.233 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 348, 2024-09-21T06:26:08.233 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.015692501, 2024-09-21T06:26:08.233 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000045093 2024-09-21T06:26:08.233 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.233 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:08.234 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.234 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.234 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.234 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.234 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 208224, 2024-09-21T06:26:08.234 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 36336 2024-09-21T06:26:08.234 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.235 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:26:08.235 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 224, 2024-09-21T06:26:08.235 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 266, 2024-09-21T06:26:08.235 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 24234164, 2024-09-21T06:26:08.235 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 234347, 2024-09-21T06:26:08.235 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2024-09-21T06:26:08.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-09-21T06:26:08.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.038712503, 2024-09-21T06:26:08.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.010661916, 2024-09-21T06:26:08.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.020544125, 2024-09-21T06:26:08.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.005507635, 2024-09-21T06:26:08.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:08.236 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 264, 2024-09-21T06:26:08.237 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.010320422, 2024-09-21T06:26:08.237 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000039092 2024-09-21T06:26:08.237 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.237 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:08.237 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.237 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.237 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.238 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:26:08.238 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 163376 2024-09-21T06:26:08.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.238 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:26:08.238 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 262, 2024-09-21T06:26:08.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 262, 2024-09-21T06:26:08.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 37817938, 2024-09-21T06:26:08.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 82508, 2024-09-21T06:26:08.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2024-09-21T06:26:08.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-09-21T06:26:08.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.045235474, 2024-09-21T06:26:08.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.009249453, 2024-09-21T06:26:08.240 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.026725660, 2024-09-21T06:26:08.240 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.007446080, 2024-09-21T06:26:08.240 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:08.240 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 261, 2024-09-21T06:26:08.240 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.014360846, 2024-09-21T06:26:08.240 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000055022 2024-09-21T06:26:08.241 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.241 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:08.242 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.242 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.243 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.243 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.243 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T06:26:08.243 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T06:26:08.243 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.244 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-09-21T06:26:08.244 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-09-21T06:26:08.244 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-09-21T06:26:08.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.244 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-09-21T06:26:08.245 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-21T06:26:08.245 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-21T06:26:08.245 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1217, 2024-09-21T06:26:08.245 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.054295201, 2024-09-21T06:26:08.245 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000044613 2024-09-21T06:26:08.245 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:08.245 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.246 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-09-21T06:26:08.246 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-21T06:26:08.246 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-21T06:26:08.246 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.246 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.246 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.246 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:08.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.247 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-09-21T06:26:08.247 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2024-09-21T06:26:08.247 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2024-09-21T06:26:08.247 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-09-21T06:26:08.247 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.247 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.247 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.248 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.248 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-09-21T06:26:08.248 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-09-21T06:26:08.248 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-09-21T06:26:08.248 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-09-21T06:26:08.248 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2024-09-21T06:26:08.248 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-09-21T06:26:08.249 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-09-21T06:26:08.249 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 3811, 2024-09-21T06:26:08.249 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 1831, 2024-09-21T06:26:08.249 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 1980, 2024-09-21T06:26:08.249 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-09-21T06:26:08.249 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 1243, 2024-09-21T06:26:08.249 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-09-21T06:26:08.249 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-09-21T06:26:08.249 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-09-21T06:26:08.250 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 4, 2024-09-21T06:26:08.250 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2024-09-21T06:26:08.250 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2024-09-21T06:26:08.250 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-09-21T06:26:08.250 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-09-21T06:26:08.250 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:26:08.250 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-09-21T06:26:08.251 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-09-21T06:26:08.251 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-09-21T06:26:08.251 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-09-21T06:26:08.251 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-09-21T06:26:08.251 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-09-21T06:26:08.251 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-09-21T06:26:08.251 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-09-21T06:26:08.251 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-09-21T06:26:08.252 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:26:08.252 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-09-21T06:26:08.252 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 87, 2024-09-21T06:26:08.252 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 23676942, 2024-09-21T06:26:08.252 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 6, 2024-09-21T06:26:08.252 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:26:08.252 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-09-21T06:26:08.253 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:26:08.253 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:26:08.253 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:26:08.253 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:26:08.253 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:26:08.253 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-09-21T06:26:08.253 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:26:08.253 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:26:08.254 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:26:08.254 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:26:08.254 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:26:08.254 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:26:08.254 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-09-21T06:26:08.254 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:26:08.254 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-09-21T06:26:08.254 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2024-09-21T06:26:08.255 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.255 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-09-21T06:26:08.255 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-09-21T06:26:08.255 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-09-21T06:26:08.255 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-09-21T06:26:08.255 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-09-21T06:26:08.255 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-09-21T06:26:08.256 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-09-21T06:26:08.256 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-09-21T06:26:08.256 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:26:08.256 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-09-21T06:26:08.256 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-09-21T06:26:08.256 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-09-21T06:26:08.257 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-09-21T06:26:08.257 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-09-21T06:26:08.257 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:26:08.257 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:26:08.257 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-09-21T06:26:08.257 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-09-21T06:26:08.258 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:26:08.258 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:26:08.258 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:26:08.258 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-09-21T06:26:08.258 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-09-21T06:26:08.258 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:26:08.259 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-09-21T06:26:08.259 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0, 2024-09-21T06:26:08.259 INFO:teuthology.orchestra.run.smithi137.stdout: "uninline_started": 0, 2024-09-21T06:26:08.259 INFO:teuthology.orchestra.run.smithi137.stdout: "uninline_succeeded": 0, 2024-09-21T06:26:08.259 INFO:teuthology.orchestra.run.smithi137.stdout: "uninline_write_failed": 0 2024-09-21T06:26:08.259 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.259 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-09-21T06:26:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-09-21T06:26:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2024-09-21T06:26:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-09-21T06:26:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-09-21T06:26:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 17260, 2024-09-21T06:26:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-09-21T06:26:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-09-21T06:26:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2024-09-21T06:26:08.260 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-09-21T06:26:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-09-21T06:26:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 4, 2024-09-21T06:26:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 6, 2024-09-21T06:26:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-09-21T06:26:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-09-21T06:26:08.261 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 474585867, 2024-09-21T06:26:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 486049803, 2024-09-21T06:26:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 486049743, 2024-09-21T06:26:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-09-21T06:26:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.262 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 17260 2024-09-21T06:26:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-09-21T06:26:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 261, 2024-09-21T06:26:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 3769, 2024-09-21T06:26:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 3508, 2024-09-21T06:26:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 57, 2024-09-21T06:26:08.263 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 68, 2024-09-21T06:26:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 11, 2024-09-21T06:26:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 3811, 2024-09-21T06:26:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 3811, 2024-09-21T06:26:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 0, 2024-09-21T06:26:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-09-21T06:26:08.264 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2024-09-21T06:26:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2024-09-21T06:26:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 122764, 2024-09-21T06:26:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 328376320 2024-09-21T06:26:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-09-21T06:26:08.265 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2024-09-21T06:26:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-09-21T06:26:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2024-09-21T06:26:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2024-09-21T06:26:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-09-21T06:26:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-09-21T06:26:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.266 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-09-21T06:26:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.267 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-09-21T06:26:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-09-21T06:26:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.268 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-09-21T06:26:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.269 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-09-21T06:26:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-09-21T06:26:08.270 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-09-21T06:26:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.271 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.272 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.272 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.272 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-09-21T06:26:08.272 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.272 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.272 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.272 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.273 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-09-21T06:26:08.273 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.273 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.273 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.273 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.273 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-09-21T06:26:08.274 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.274 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.275 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.275 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.275 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-09-21T06:26:08.275 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.275 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.276 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.276 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.276 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-09-21T06:26:08.276 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.276 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.276 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.276 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.276 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-09-21T06:26:08.277 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.277 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.277 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.277 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.277 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-09-21T06:26:08.277 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.277 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.277 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.278 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.278 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-09-21T06:26:08.278 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.278 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.278 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.278 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.278 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-09-21T06:26:08.278 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.279 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.279 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.279 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-09-21T06:26:08.279 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.279 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.279 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.279 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-09-21T06:26:08.280 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.280 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.280 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.280 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-09-21T06:26:08.280 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.280 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.281 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.281 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-09-21T06:26:08.281 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.281 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.281 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.281 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-09-21T06:26:08.282 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.282 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.282 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.282 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.282 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-09-21T06:26:08.282 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.282 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.282 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.283 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-09-21T06:26:08.283 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.283 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.283 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.283 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-09-21T06:26:08.283 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.284 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.284 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.284 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-09-21T06:26:08.284 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.284 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.285 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.285 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-09-21T06:26:08.285 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.285 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.286 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.286 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.286 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-09-21T06:26:08.286 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.286 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.286 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.286 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.287 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-09-21T06:26:08.287 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.287 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.287 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.287 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.287 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:26:08.287 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:26:08.287 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-09-21T06:26:08.287 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.288 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.288 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.288 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:08.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.288 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-09-21T06:26:08.288 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 2, 2024-09-21T06:26:08.288 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 2, 2024-09-21T06:26:08.288 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 0, 2024-09-21T06:26:08.289 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 0, 2024-09-21T06:26:08.289 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-09-21T06:26:08.289 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2024-09-21T06:26:08.289 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2024-09-21T06:26:08.289 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 2009, 2024-09-21T06:26:08.289 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-09-21T06:26:08.290 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.290 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-09-21T06:26:08.290 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:26:08.290 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-09-21T06:26:08.290 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:26:08.291 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:26:08.291 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:26:08.291 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:26:08.291 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:26:08.291 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:26:08.292 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:26:08.292 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:26:08.292 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:26:08.292 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:26:08.292 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:26:08.293 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:26:08.293 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:26:08.293 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-09-21T06:26:08.293 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:26:08.293 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-09-21T06:26:08.294 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:26:08.294 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:26:08.294 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:26:08.294 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:26:08.294 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:26:08.294 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:26:08.295 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:26:08.295 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-09-21T06:26:08.295 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:26:08.295 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:26:08.295 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:26:08.296 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-09-21T06:26:08.296 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-09-21T06:26:08.296 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-09-21T06:26:08.296 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:26:08.296 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:26:08.297 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:26:08.297 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:26:08.297 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 243435, 2024-09-21T06:26:08.297 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 414, 2024-09-21T06:26:08.297 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:26:08.298 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-09-21T06:26:08.298 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-09-21T06:26:08.298 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-09-21T06:26:08.298 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:26:08.298 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-09-21T06:26:08.299 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:26:08.299 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-09-21T06:26:08.299 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8168, 2024-09-21T06:26:08.299 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 181, 2024-09-21T06:26:08.299 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:26:08.300 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:26:08.300 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-09-21T06:26:08.300 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-09-21T06:26:08.300 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 3415666, 2024-09-21T06:26:08.300 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 23515, 2024-09-21T06:26:08.301 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-09-21T06:26:08.301 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-09-21T06:26:08.301 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-09-21T06:26:08.301 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-09-21T06:26:08.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.301 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-09-21T06:26:08.302 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-09-21T06:26:08.302 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-09-21T06:26:08.302 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 759, 2024-09-21T06:26:08.302 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 26, 2024-09-21T06:26:08.303 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 4, 2024-09-21T06:26:08.303 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 759, 2024-09-21T06:26:08.303 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-09-21T06:26:08.303 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 759, 2024-09-21T06:26:08.303 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.872985269, 2024-09-21T06:26:08.303 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001150178 2024-09-21T06:26:08.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.304 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-09-21T06:26:08.304 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-09-21T06:26:08.304 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 759, 2024-09-21T06:26:08.304 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 760 2024-09-21T06:26:08.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.305 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 759, 2024-09-21T06:26:08.305 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 759, 2024-09-21T06:26:08.305 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2024-09-21T06:26:08.305 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-09-21T06:26:08.305 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-09-21T06:26:08.305 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 450, 2024-09-21T06:26:08.305 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2024-09-21T06:26:08.305 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 306, 2024-09-21T06:26:08.305 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2024-09-21T06:26:08.306 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2024-09-21T06:26:08.306 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-09-21T06:26:08.306 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-09-21T06:26:08.306 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-09-21T06:26:08.306 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-09-21T06:26:08.306 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2024-09-21T06:26:08.306 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-09-21T06:26:08.306 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-09-21T06:26:08.307 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-09-21T06:26:08.307 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 2, 2024-09-21T06:26:08.307 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2024-09-21T06:26:08.307 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:26:08.307 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-09-21T06:26:08.307 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:26:08.307 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-09-21T06:26:08.307 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-09-21T06:26:08.307 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-09-21T06:26:08.308 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:26:08.308 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-09-21T06:26:08.308 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:26:08.308 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-09-21T06:26:08.308 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-09-21T06:26:08.308 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-09-21T06:26:08.308 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-09-21T06:26:08.309 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-09-21T06:26:08.309 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-09-21T06:26:08.309 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-09-21T06:26:08.309 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-09-21T06:26:08.309 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-09-21T06:26:08.309 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-09-21T06:26:08.310 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-09-21T06:26:08.310 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-09-21T06:26:08.310 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-09-21T06:26:08.310 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-09-21T06:26:08.310 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-09-21T06:26:08.310 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-09-21T06:26:08.310 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-09-21T06:26:08.311 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 90, 2024-09-21T06:26:08.311 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-09-21T06:26:08.311 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 7, 2024-09-21T06:26:08.311 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 10, 2024-09-21T06:26:08.311 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 10, 2024-09-21T06:26:08.312 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-09-21T06:26:08.312 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-09-21T06:26:08.312 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2024-09-21T06:26:08.312 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2024-09-21T06:26:08.312 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2024-09-21T06:26:08.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.312 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-09-21T06:26:08.313 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2024-09-21T06:26:08.313 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:26:08.313 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-09-21T06:26:08.313 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-09-21T06:26:08.313 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.314 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-09-21T06:26:08.314 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-09-21T06:26:08.314 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:26:08.314 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-09-21T06:26:08.314 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-09-21T06:26:08.314 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 0, 2024-09-21T06:26:08.314 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-09-21T06:26:08.315 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-09-21T06:26:08.315 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.315 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:26:08.315 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:08.315 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-21T06:26:08.315 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:08.315 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 834, 2024-09-21T06:26:08.315 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 79050758, 2024-09-21T06:26:08.315 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:08.316 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 834, 2024-09-21T06:26:08.316 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-21T06:26:08.316 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-21T06:26:08.316 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 834, 2024-09-21T06:26:08.316 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 79050758, 2024-09-21T06:26:08.316 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:08.316 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.317 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.317 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.317 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:08.317 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.317 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-09-21T06:26:08.317 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:08.317 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-21T06:26:08.317 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:08.318 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-21T06:26:08.318 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-21T06:26:08.318 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:08.318 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:08.318 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 759, 2024-09-21T06:26:08.318 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 78704781, 2024-09-21T06:26:08.318 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 759, 2024-09-21T06:26:08.318 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 78704781, 2024-09-21T06:26:08.319 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:08.319 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.319 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.319 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.319 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:08.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.319 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-09-21T06:26:08.319 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:08.320 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-09-21T06:26:08.320 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:08.320 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-21T06:26:08.320 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-21T06:26:08.320 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:08.320 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:08.320 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 759, 2024-09-21T06:26:08.320 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 759, 2024-09-21T06:26:08.320 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 759, 2024-09-21T06:26:08.321 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 759, 2024-09-21T06:26:08.321 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:08.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.321 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.321 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:08.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.322 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:26:08.322 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:08.322 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-21T06:26:08.322 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:08.322 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-21T06:26:08.323 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-21T06:26:08.323 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:08.323 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:08.323 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-21T06:26:08.323 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-21T06:26:08.323 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-09-21T06:26:08.323 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-09-21T06:26:08.323 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:08.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.324 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.324 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:08.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.324 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x56514da3f930": { 2024-09-21T06:26:08.324 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:08.324 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-21T06:26:08.325 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:08.325 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-21T06:26:08.325 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-21T06:26:08.325 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:08.325 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:08.325 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-21T06:26:08.325 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-21T06:26:08.325 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-09-21T06:26:08.325 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-09-21T06:26:08.326 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:08.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:08.326 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:08.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:08.326 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:08.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:08.326 INFO:teuthology.orchestra.run.smithi137.stdout: "trackedop": { 2024-09-21T06:26:08.327 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-09-21T06:26:08.327 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:08.327 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-21T06:26:09.040 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:08 smithi137 ceph-mon[34464]: pgmap v670: 97 pgs: 4 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 88 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-21T06:26:09.066 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T06:26:09.066 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-21T06:26:09.067 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-21T06:26:09.067 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:26:09.067 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:26:09.067 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T06:26:09.067 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T06:26:09.313 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:26:09.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:08 smithi003 ceph-mon[31227]: pgmap v670: 97 pgs: 4 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 88 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-21T06:26:09.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:08 smithi204 ceph-mon[34659]: pgmap v670: 97 pgs: 4 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 88 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-21T06:26:09.871 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-21T06:26:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:26:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 66964, 2024-09-21T06:26:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 114173, 2024-09-21T06:26:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 21733725, 2024-09-21T06:26:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 86147219, 2024-09-21T06:26:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 41, 2024-09-21T06:26:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 9, 2024-09-21T06:26:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 5.302911390, 2024-09-21T06:26:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 3.044784853, 2024-09-21T06:26:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 2.249813685, 2024-09-21T06:26:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.180287144, 2024-09-21T06:26:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 114172, 2024-09-21T06:26:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.854705065, 2024-09-21T06:26:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000016244 2024-09-21T06:26:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 153714, 2024-09-21T06:26:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.022164536, 2024-09-21T06:26:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000144 2024-09-21T06:26:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 4848, 2024-09-21T06:26:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 2160 2024-09-21T06:26:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:26:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 8497, 2024-09-21T06:26:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 8478, 2024-09-21T06:26:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 3325489, 2024-09-21T06:26:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 36410481, 2024-09-21T06:26:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 12, 2024-09-21T06:26:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 7, 2024-09-21T06:26:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.750092647, 2024-09-21T06:26:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.336960584, 2024-09-21T06:26:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.180175185, 2024-09-21T06:26:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.211762828, 2024-09-21T06:26:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 8477, 2024-09-21T06:26:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.210133374, 2024-09-21T06:26:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000024788 2024-09-21T06:26:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 226, 2024-09-21T06:26:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000037117, 2024-09-21T06:26:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000164 2024-09-21T06:26:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1274032, 2024-09-21T06:26:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 155168 2024-09-21T06:26:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:26:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 161962, 2024-09-21T06:26:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 109219, 2024-09-21T06:26:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 66642334, 2024-09-21T06:26:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 43008431, 2024-09-21T06:26:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 11, 2024-09-21T06:26:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 7, 2024-09-21T06:26:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 9.041092793, 2024-09-21T06:26:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 4.451008084, 2024-09-21T06:26:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 5.723378060, 2024-09-21T06:26:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.264582808, 2024-09-21T06:26:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 109217, 2024-09-21T06:26:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.654177091, 2024-09-21T06:26:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000015145 2024-09-21T06:26:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 256181, 2024-09-21T06:26:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.032830904, 2024-09-21T06:26:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000128 2024-09-21T06:26:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:26:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 373056 2024-09-21T06:26:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-09-21T06:26:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-09-21T06:26:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-09-21T06:26:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-09-21T06:26:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-21T06:26:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-21T06:26:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 20922, 2024-09-21T06:26:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 11.917989498, 2024-09-21T06:26:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000569639 2024-09-21T06:26:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-09-21T06:26:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-21T06:26:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-21T06:26:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 18147, 2024-09-21T06:26:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.006035676, 2024-09-21T06:26:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000055438 2024-09-21T06:26:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-09-21T06:26:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 52680, 2024-09-21T06:26:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 46785, 2024-09-21T06:26:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-09-21T06:26:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 46785, 2024-09-21T06:26:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 47.474435895, 2024-09-21T06:26:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001014736 2024-09-21T06:26:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-09-21T06:26:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 5895, 2024-09-21T06:26:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-09-21T06:26:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-09-21T06:26:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 11, 2024-09-21T06:26:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-09-21T06:26:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-09-21T06:26:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 98, 2024-09-21T06:26:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 25, 2024-09-21T06:26:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 73, 2024-09-21T06:26:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-09-21T06:26:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 61, 2024-09-21T06:26:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-09-21T06:26:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 1, 2024-09-21T06:26:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 1, 2024-09-21T06:26:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 3, 2024-09-21T06:26:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 98489, 2024-09-21T06:26:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 36817, 2024-09-21T06:26:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 5895, 2024-09-21T06:26:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 4, 2024-09-21T06:26:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:26:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-09-21T06:26:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-09-21T06:26:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-09-21T06:26:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-09-21T06:26:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-09-21T06:26:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-09-21T06:26:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 1, 2024-09-21T06:26:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 50, 2024-09-21T06:26:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-09-21T06:26:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 40, 2024-09-21T06:26:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-09-21T06:26:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 87, 2024-09-21T06:26:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 1062181, 2024-09-21T06:26:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 6, 2024-09-21T06:26:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 40, 2024-09-21T06:26:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-09-21T06:26:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:26:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:26:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:26:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:26:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:26:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 40, 2024-09-21T06:26:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:26:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 256, 2024-09-21T06:26:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 29321, 2024-09-21T06:26:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:26:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:26:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:26:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 1225, 2024-09-21T06:26:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 269, 2024-09-21T06:26:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 395, 2024-09-21T06:26:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 46010 2024-09-21T06:26:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-09-21T06:26:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 111, 2024-09-21T06:26:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 9, 2024-09-21T06:26:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 6, 2024-09-21T06:26:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 8, 2024-09-21T06:26:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 131, 2024-09-21T06:26:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-09-21T06:26:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-09-21T06:26:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:26:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 9478, 2024-09-21T06:26:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 9478, 2024-09-21T06:26:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-09-21T06:26:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-09-21T06:26:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-09-21T06:26:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:26:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:26:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-09-21T06:26:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-09-21T06:26:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:26:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:26:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:26:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-09-21T06:26:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-09-21T06:26:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:26:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-09-21T06:26:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0, 2024-09-21T06:26:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "uninline_started": 0, 2024-09-21T06:26:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "uninline_succeeded": 0, 2024-09-21T06:26:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "uninline_write_failed": 0 2024-09-21T06:26:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-09-21T06:26:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-09-21T06:26:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 39100, 2024-09-21T06:26:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-09-21T06:26:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-09-21T06:26:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 39100, 2024-09-21T06:26:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-09-21T06:26:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-09-21T06:26:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 49, 2024-09-21T06:26:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-09-21T06:26:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-09-21T06:26:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 49, 2024-09-21T06:26:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 5, 2024-09-21T06:26:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-09-21T06:26:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-09-21T06:26:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-09-21T06:26:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 69646489, 2024-09-21T06:26:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-09-21T06:26:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-09-21T06:26:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 10211, 2024-09-21T06:26:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 21.643298976, 2024-09-21T06:26:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002119606 2024-09-21T06:26:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-09-21T06:26:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-09-21T06:26:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 50, 2024-09-21T06:26:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 9528, 2024-09-21T06:26:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 9478, 2024-09-21T06:26:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 39, 2024-09-21T06:26:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 42, 2024-09-21T06:26:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 3, 2024-09-21T06:26:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 98, 2024-09-21T06:26:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 14563, 2024-09-21T06:26:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 14465, 2024-09-21T06:26:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 1, 2024-09-21T06:26:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 9596, 2024-09-21T06:26:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 9595, 2024-09-21T06:26:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 132168, 2024-09-21T06:26:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 336478208 2024-09-21T06:26:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-09-21T06:26:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 88193, 2024-09-21T06:26:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-09-21T06:26:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 52680, 2024-09-21T06:26:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 25, 2024-09-21T06:26:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 9747, 2024-09-21T06:26:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-09-21T06:26:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 9478, 2024-09-21T06:26:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 20.969887987, 2024-09-21T06:26:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002212480 2024-09-21T06:26:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-09-21T06:26:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 114, 2024-09-21T06:26:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.071539944, 2024-09-21T06:26:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000627543 2024-09-21T06:26:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-09-21T06:26:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-09-21T06:26:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-09-21T06:26:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 10139, 2024-09-21T06:26:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.818836168, 2024-09-21T06:26:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000179390 2024-09-21T06:26:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-09-21T06:26:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-09-21T06:26:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-09-21T06:26:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-09-21T06:26:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-09-21T06:26:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-09-21T06:26:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-09-21T06:26:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-09-21T06:26:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-09-21T06:26:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-09-21T06:26:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-09-21T06:26:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 17576, 2024-09-21T06:26:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 6.200895849, 2024-09-21T06:26:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000352804 2024-09-21T06:26:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-09-21T06:26:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-09-21T06:26:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-09-21T06:26:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-09-21T06:26:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-09-21T06:26:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-09-21T06:26:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-09-21T06:26:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-09-21T06:26:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-09-21T06:26:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-09-21T06:26:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-09-21T06:26:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-09-21T06:26:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-09-21T06:26:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 9478, 2024-09-21T06:26:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 18.413275947, 2024-09-21T06:26:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001942738 2024-09-21T06:26:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:26:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:26:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-09-21T06:26:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-09-21T06:26:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 2, 2024-09-21T06:26:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 4, 2024-09-21T06:26:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 2, 2024-09-21T06:26:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 2, 2024-09-21T06:26:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-09-21T06:26:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 6117, 2024-09-21T06:26:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 3058, 2024-09-21T06:26:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 592, 2024-09-21T06:26:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-09-21T06:26:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-09-21T06:26:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:26:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-09-21T06:26:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:26:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:26:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:26:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:26:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:26:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:26:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:26:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:26:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:26:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:26:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:26:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:26:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:26:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-09-21T06:26:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:26:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-09-21T06:26:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:26:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:26:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:26:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:26:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:26:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:26:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:26:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-09-21T06:26:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:26:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:26:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:26:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-09-21T06:26:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-09-21T06:26:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-09-21T06:26:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:26:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:26:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:26:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:26:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 326714, 2024-09-21T06:26:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 167, 2024-09-21T06:26:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 264, 2024-09-21T06:26:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 3, 2024-09-21T06:26:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-09-21T06:26:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-09-21T06:26:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:26:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-09-21T06:26:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:26:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-09-21T06:26:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8288, 2024-09-21T06:26:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 184, 2024-09-21T06:26:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:26:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:26:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-09-21T06:26:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-09-21T06:26:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 307437, 2024-09-21T06:26:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 1745, 2024-09-21T06:26:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-09-21T06:26:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-09-21T06:26:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-09-21T06:26:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-09-21T06:26:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-09-21T06:26:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-09-21T06:26:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-09-21T06:26:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 21437, 2024-09-21T06:26:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 67239407, 2024-09-21T06:26:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 28, 2024-09-21T06:26:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 21437, 2024-09-21T06:26:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-09-21T06:26:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 21437, 2024-09-21T06:26:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 48.839138411, 2024-09-21T06:26:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002278263 2024-09-21T06:26:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-09-21T06:26:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-09-21T06:26:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 21437, 2024-09-21T06:26:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 22089 2024-09-21T06:26:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 21437, 2024-09-21T06:26:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 530, 2024-09-21T06:26:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 20907, 2024-09-21T06:26:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-09-21T06:26:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-09-21T06:26:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 0, 2024-09-21T06:26:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 10, 2024-09-21T06:26:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 490, 2024-09-21T06:26:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 10701, 2024-09-21T06:26:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 269, 2024-09-21T06:26:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-09-21T06:26:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-09-21T06:26:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-09-21T06:26:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-09-21T06:26:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 9505, 2024-09-21T06:26:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-09-21T06:26:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-09-21T06:26:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-09-21T06:26:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 40, 2024-09-21T06:26:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 20, 2024-09-21T06:26:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:26:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-09-21T06:26:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:26:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-09-21T06:26:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-09-21T06:26:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-09-21T06:26:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:26:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-09-21T06:26:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:26:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-09-21T06:26:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-09-21T06:26:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-09-21T06:26:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-09-21T06:26:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-09-21T06:26:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-09-21T06:26:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-09-21T06:26:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-09-21T06:26:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-09-21T06:26:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-09-21T06:26:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-09-21T06:26:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-09-21T06:26:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-09-21T06:26:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-09-21T06:26:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-09-21T06:26:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-09-21T06:26:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-09-21T06:26:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 90, 2024-09-21T06:26:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-09-21T06:26:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 13, 2024-09-21T06:26:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 12, 2024-09-21T06:26:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 12, 2024-09-21T06:26:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-09-21T06:26:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-09-21T06:26:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 810, 2024-09-21T06:26:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-09-21T06:26:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 244 2024-09-21T06:26:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-09-21T06:26:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-09-21T06:26:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:26:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-09-21T06:26:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-09-21T06:26:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-09-21T06:26:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-09-21T06:26:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 6, 2024-09-21T06:26:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-09-21T06:26:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 6, 2024-09-21T06:26:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 9478, 2024-09-21T06:26:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 9478, 2024-09-21T06:26:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-09-21T06:26:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:26:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-21T06:26:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 237419, 2024-09-21T06:26:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 70315004, 2024-09-21T06:26:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 237419, 2024-09-21T06:26:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-21T06:26:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-21T06:26:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 237419, 2024-09-21T06:26:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 70315004, 2024-09-21T06:26:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-09-21T06:26:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-21T06:26:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-21T06:26:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-21T06:26:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 21437, 2024-09-21T06:26:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 68196893, 2024-09-21T06:26:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 21437, 2024-09-21T06:26:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 68196893, 2024-09-21T06:26:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-09-21T06:26:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-09-21T06:26:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-21T06:26:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-21T06:26:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 21437, 2024-09-21T06:26:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 21437, 2024-09-21T06:26:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 21437, 2024-09-21T06:26:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 21437, 2024-09-21T06:26:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:26:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-21T06:26:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 9478, 2024-09-21T06:26:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 957486, 2024-09-21T06:26:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 9478, 2024-09-21T06:26:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-21T06:26:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-21T06:26:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 490, 2024-09-21T06:26:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 957486, 2024-09-21T06:26:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x56266dd5d520": { 2024-09-21T06:26:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-21T06:26:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 39100, 2024-09-21T06:26:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 65452185, 2024-09-21T06:26:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 39100, 2024-09-21T06:26:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-21T06:26:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-21T06:26:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 10211, 2024-09-21T06:26:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 65452185, 2024-09-21T06:26:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "trackedop": { 2024-09-21T06:26:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-09-21T06:26:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:09.949 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-21T06:26:11.195 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-21T06:26:11.195 INFO:tasks.check_counter:Daemon mds.h mds.imported=1 2024-09-21T06:26:11.195 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-21T06:26:11.195 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:26:11.195 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:26:11.196 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T06:26:11.196 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T06:26:11.219 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:10 smithi137 ceph-mon[34464]: pgmap v671: 97 pgs: 4 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 88 active+clean; 134 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-21T06:26:11.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:10 smithi003 ceph-mon[31227]: pgmap v671: 97 pgs: 4 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 88 active+clean; 134 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-21T06:26:11.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:10 smithi204 ceph-mon[34659]: pgmap v671: 97 pgs: 4 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 88 active+clean; 134 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-21T06:26:11.447 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:26:12.006 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-21T06:26:12.007 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:26:12.007 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 257, 2024-09-21T06:26:12.007 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 238, 2024-09-21T06:26:12.007 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1251622, 2024-09-21T06:26:12.007 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 138851, 2024-09-21T06:26:12.007 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-09-21T06:26:12.007 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-09-21T06:26:12.007 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.043021419, 2024-09-21T06:26:12.008 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.016554622, 2024-09-21T06:26:12.008 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.038587172, 2024-09-21T06:26:12.008 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000039781, 2024-09-21T06:26:12.008 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:12.008 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 237, 2024-09-21T06:26:12.008 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.011562770, 2024-09-21T06:26:12.008 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000048788 2024-09-21T06:26:12.008 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.008 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:12.009 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.009 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.009 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.009 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.009 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1251376, 2024-09-21T06:26:12.009 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 138528 2024-09-21T06:26:12.009 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.009 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:26:12.009 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 8, 2024-09-21T06:26:12.010 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 171, 2024-09-21T06:26:12.010 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 4672, 2024-09-21T06:26:12.010 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 372514, 2024-09-21T06:26:12.010 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-09-21T06:26:12.010 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-09-21T06:26:12.010 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.013970044, 2024-09-21T06:26:12.010 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.008704532, 2024-09-21T06:26:12.010 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.003896966, 2024-09-21T06:26:12.010 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000061233, 2024-09-21T06:26:12.011 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:12.011 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 170, 2024-09-21T06:26:12.011 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.012788954, 2024-09-21T06:26:12.011 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000075229 2024-09-21T06:26:12.011 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.011 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:12.011 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.011 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.011 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.012 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.012 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-21T06:26:12.012 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 371872 2024-09-21T06:26:12.012 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.012 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:26:12.012 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 2280, 2024-09-21T06:26:12.012 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2282, 2024-09-21T06:26:12.012 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 912212, 2024-09-21T06:26:12.012 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 725098, 2024-09-21T06:26:12.013 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-09-21T06:26:12.013 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-09-21T06:26:12.013 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.202870115, 2024-09-21T06:26:12.013 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.087876177, 2024-09-21T06:26:12.013 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.042592168, 2024-09-21T06:26:12.013 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.059003208, 2024-09-21T06:26:12.013 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:12.013 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2280, 2024-09-21T06:26:12.013 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.079389105, 2024-09-21T06:26:12.014 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000034819 2024-09-21T06:26:12.014 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.014 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:12.014 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.014 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.014 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.014 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.014 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:26:12.015 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:26:12.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.015 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-09-21T06:26:12.015 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-09-21T06:26:12.015 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-09-21T06:26:12.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.015 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-09-21T06:26:12.015 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-21T06:26:12.016 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-21T06:26:12.016 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3873, 2024-09-21T06:26:12.016 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.100358821, 2024-09-21T06:26:12.016 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000025912 2024-09-21T06:26:12.016 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:12.016 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.016 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-09-21T06:26:12.016 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-21T06:26:12.016 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-21T06:26:12.017 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.017 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.017 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.017 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:12.017 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.017 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-09-21T06:26:12.017 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2024-09-21T06:26:12.017 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2024-09-21T06:26:12.018 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-09-21T06:26:12.018 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.018 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.018 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.018 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.018 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-09-21T06:26:12.019 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-09-21T06:26:12.019 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-09-21T06:26:12.019 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-09-21T06:26:12.019 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2024-09-21T06:26:12.019 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-09-21T06:26:12.019 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-09-21T06:26:12.019 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 44, 2024-09-21T06:26:12.019 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2024-09-21T06:26:12.020 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 44, 2024-09-21T06:26:12.020 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-09-21T06:26:12.020 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 44, 2024-09-21T06:26:12.020 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-09-21T06:26:12.020 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-09-21T06:26:12.020 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-09-21T06:26:12.020 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 3, 2024-09-21T06:26:12.020 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2024-09-21T06:26:12.021 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2024-09-21T06:26:12.021 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-09-21T06:26:12.021 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-09-21T06:26:12.021 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:26:12.021 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-09-21T06:26:12.021 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-09-21T06:26:12.022 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-09-21T06:26:12.022 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-09-21T06:26:12.022 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-09-21T06:26:12.022 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-09-21T06:26:12.022 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-09-21T06:26:12.022 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-09-21T06:26:12.022 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-09-21T06:26:12.022 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:26:12.023 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-09-21T06:26:12.023 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 0, 2024-09-21T06:26:12.023 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 0, 2024-09-21T06:26:12.023 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-09-21T06:26:12.023 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:26:12.023 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-09-21T06:26:12.023 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:26:12.023 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:26:12.024 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:26:12.024 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:26:12.024 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:26:12.024 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-09-21T06:26:12.024 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:26:12.024 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:26:12.024 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:26:12.024 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:26:12.024 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:26:12.025 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:26:12.025 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-09-21T06:26:12.025 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:26:12.025 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-09-21T06:26:12.025 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2024-09-21T06:26:12.025 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.025 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-09-21T06:26:12.025 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-09-21T06:26:12.025 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-09-21T06:26:12.026 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-09-21T06:26:12.026 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-09-21T06:26:12.026 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-09-21T06:26:12.026 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-09-21T06:26:12.026 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-09-21T06:26:12.026 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:26:12.026 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-09-21T06:26:12.026 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-09-21T06:26:12.027 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-09-21T06:26:12.027 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-09-21T06:26:12.027 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-09-21T06:26:12.027 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:26:12.027 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:26:12.027 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-09-21T06:26:12.027 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-09-21T06:26:12.027 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:26:12.028 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:26:12.028 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:26:12.028 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-09-21T06:26:12.028 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-09-21T06:26:12.028 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:26:12.028 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-09-21T06:26:12.028 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0, 2024-09-21T06:26:12.028 INFO:teuthology.orchestra.run.smithi137.stdout: "uninline_started": 0, 2024-09-21T06:26:12.029 INFO:teuthology.orchestra.run.smithi137.stdout: "uninline_succeeded": 0, 2024-09-21T06:26:12.029 INFO:teuthology.orchestra.run.smithi137.stdout: "uninline_write_failed": 0 2024-09-21T06:26:12.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.029 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-09-21T06:26:12.029 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-09-21T06:26:12.029 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2024-09-21T06:26:12.029 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-09-21T06:26:12.029 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-09-21T06:26:12.029 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 195, 2024-09-21T06:26:12.030 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-09-21T06:26:12.030 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-09-21T06:26:12.030 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2024-09-21T06:26:12.030 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-09-21T06:26:12.030 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-09-21T06:26:12.030 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 2, 2024-09-21T06:26:12.030 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-09-21T06:26:12.030 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-09-21T06:26:12.030 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-09-21T06:26:12.031 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-09-21T06:26:12.031 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 4464235, 2024-09-21T06:26:12.031 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 4461201, 2024-09-21T06:26:12.031 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-09-21T06:26:12.031 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.031 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.031 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.031 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.031 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 195 2024-09-21T06:26:12.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.032 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-09-21T06:26:12.032 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 41, 2024-09-21T06:26:12.032 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 47, 2024-09-21T06:26:12.032 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 6, 2024-09-21T06:26:12.032 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 13, 2024-09-21T06:26:12.032 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 13, 2024-09-21T06:26:12.032 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 0, 2024-09-21T06:26:12.032 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 44, 2024-09-21T06:26:12.032 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 44, 2024-09-21T06:26:12.033 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 0, 2024-09-21T06:26:12.033 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-09-21T06:26:12.033 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2024-09-21T06:26:12.033 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2024-09-21T06:26:12.033 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 117668, 2024-09-21T06:26:12.033 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 316428288 2024-09-21T06:26:12.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.033 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-09-21T06:26:12.033 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2024-09-21T06:26:12.033 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-09-21T06:26:12.034 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2024-09-21T06:26:12.034 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2024-09-21T06:26:12.034 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-09-21T06:26:12.034 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-09-21T06:26:12.034 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.034 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.034 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.034 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-09-21T06:26:12.035 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.035 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.035 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.035 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-09-21T06:26:12.035 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.035 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.035 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.036 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-09-21T06:26:12.036 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.036 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.036 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.036 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-09-21T06:26:12.036 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.036 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.036 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.037 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-09-21T06:26:12.037 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.037 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.037 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.037 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-09-21T06:26:12.037 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.037 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.038 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.038 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-09-21T06:26:12.038 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.038 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.038 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.038 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-09-21T06:26:12.038 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.039 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.039 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.039 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-09-21T06:26:12.039 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.039 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.039 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.039 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-09-21T06:26:12.039 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.040 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.040 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.040 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.040 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-09-21T06:26:12.041 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.041 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.041 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.041 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-09-21T06:26:12.041 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.041 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.041 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.042 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-09-21T06:26:12.042 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.042 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.042 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.042 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.042 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-09-21T06:26:12.042 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.042 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.042 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.043 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.043 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-09-21T06:26:12.043 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.043 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.043 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.043 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.043 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-09-21T06:26:12.043 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.044 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.044 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.044 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-09-21T06:26:12.044 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.044 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.044 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.044 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-09-21T06:26:12.044 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.045 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.045 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.045 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-09-21T06:26:12.045 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.045 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.045 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.045 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-09-21T06:26:12.046 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.046 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.046 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.046 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-09-21T06:26:12.046 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.046 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.046 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.047 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-09-21T06:26:12.047 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.047 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.047 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.047 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.047 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-09-21T06:26:12.047 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.047 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.048 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.048 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-09-21T06:26:12.048 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.048 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.048 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.048 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-09-21T06:26:12.048 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.049 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.049 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.049 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-09-21T06:26:12.049 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.049 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.049 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.049 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-09-21T06:26:12.050 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.050 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.050 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.050 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-09-21T06:26:12.050 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.050 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.050 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.051 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:26:12.051 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:26:12.051 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-09-21T06:26:12.051 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.051 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.051 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.051 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:12.051 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.052 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-09-21T06:26:12.052 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 2, 2024-09-21T06:26:12.052 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 4, 2024-09-21T06:26:12.052 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 2, 2024-09-21T06:26:12.052 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 2, 2024-09-21T06:26:12.052 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-09-21T06:26:12.052 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2024-09-21T06:26:12.052 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2024-09-21T06:26:12.052 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 3368, 2024-09-21T06:26:12.053 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-09-21T06:26:12.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.053 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-09-21T06:26:12.053 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:26:12.053 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-09-21T06:26:12.053 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:26:12.053 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:26:12.053 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:26:12.053 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:26:12.053 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:26:12.054 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:26:12.054 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:26:12.054 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:26:12.054 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:26:12.054 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:26:12.054 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:26:12.054 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:26:12.054 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:26:12.055 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-09-21T06:26:12.055 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:26:12.055 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-09-21T06:26:12.055 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:26:12.055 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:26:12.055 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:26:12.055 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:26:12.055 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:26:12.055 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:26:12.056 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:26:12.056 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-09-21T06:26:12.056 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:26:12.056 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:26:12.056 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:26:12.056 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-09-21T06:26:12.056 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-09-21T06:26:12.056 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-09-21T06:26:12.057 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:26:12.057 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:26:12.057 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:26:12.057 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:26:12.057 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 199752, 2024-09-21T06:26:12.057 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 89, 2024-09-21T06:26:12.057 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:26:12.057 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-09-21T06:26:12.057 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-09-21T06:26:12.058 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-09-21T06:26:12.058 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:26:12.058 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-09-21T06:26:12.058 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:26:12.058 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-09-21T06:26:12.058 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8288, 2024-09-21T06:26:12.058 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 184, 2024-09-21T06:26:12.058 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:26:12.059 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:26:12.059 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-09-21T06:26:12.059 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-09-21T06:26:12.059 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 153535, 2024-09-21T06:26:12.059 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 1126, 2024-09-21T06:26:12.059 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-09-21T06:26:12.059 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-09-21T06:26:12.059 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-09-21T06:26:12.059 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-09-21T06:26:12.060 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.060 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-09-21T06:26:12.060 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-09-21T06:26:12.060 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-09-21T06:26:12.060 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 2283, 2024-09-21T06:26:12.060 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 21, 2024-09-21T06:26:12.060 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 3, 2024-09-21T06:26:12.060 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 2283, 2024-09-21T06:26:12.060 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-09-21T06:26:12.061 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2283, 2024-09-21T06:26:12.061 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2.395957793, 2024-09-21T06:26:12.061 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001049477 2024-09-21T06:26:12.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.061 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-09-21T06:26:12.061 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-09-21T06:26:12.061 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2283, 2024-09-21T06:26:12.061 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2284 2024-09-21T06:26:12.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.062 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 2283, 2024-09-21T06:26:12.062 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 2283, 2024-09-21T06:26:12.062 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2024-09-21T06:26:12.062 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-09-21T06:26:12.062 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-09-21T06:26:12.062 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 1444, 2024-09-21T06:26:12.062 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2024-09-21T06:26:12.062 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 837, 2024-09-21T06:26:12.062 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2024-09-21T06:26:12.063 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2024-09-21T06:26:12.063 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-09-21T06:26:12.063 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-09-21T06:26:12.063 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-09-21T06:26:12.063 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-09-21T06:26:12.063 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2024-09-21T06:26:12.063 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-09-21T06:26:12.063 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-09-21T06:26:12.064 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-09-21T06:26:12.064 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 1, 2024-09-21T06:26:12.064 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2024-09-21T06:26:12.064 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:26:12.064 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-09-21T06:26:12.064 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:26:12.064 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-09-21T06:26:12.064 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-09-21T06:26:12.064 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-09-21T06:26:12.065 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:26:12.065 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-09-21T06:26:12.065 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:26:12.065 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-09-21T06:26:12.065 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-09-21T06:26:12.065 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-09-21T06:26:12.065 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-09-21T06:26:12.065 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-09-21T06:26:12.065 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-09-21T06:26:12.066 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-09-21T06:26:12.066 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-09-21T06:26:12.066 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-09-21T06:26:12.066 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-09-21T06:26:12.066 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-09-21T06:26:12.066 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-09-21T06:26:12.066 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-09-21T06:26:12.066 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-09-21T06:26:12.067 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-09-21T06:26:12.067 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-09-21T06:26:12.067 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-09-21T06:26:12.067 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 90, 2024-09-21T06:26:12.067 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-09-21T06:26:12.067 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 12, 2024-09-21T06:26:12.067 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 6, 2024-09-21T06:26:12.067 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 6, 2024-09-21T06:26:12.067 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-09-21T06:26:12.068 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-09-21T06:26:12.068 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2024-09-21T06:26:12.068 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2024-09-21T06:26:12.068 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2024-09-21T06:26:12.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.068 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-09-21T06:26:12.068 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2024-09-21T06:26:12.068 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:26:12.068 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-09-21T06:26:12.069 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-09-21T06:26:12.069 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.069 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-09-21T06:26:12.069 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-09-21T06:26:12.069 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:26:12.069 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-09-21T06:26:12.069 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-09-21T06:26:12.069 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 0, 2024-09-21T06:26:12.069 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-09-21T06:26:12.070 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-09-21T06:26:12.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.070 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:26:12.070 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:12.070 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-21T06:26:12.070 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:12.070 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 2541, 2024-09-21T06:26:12.070 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1923806, 2024-09-21T06:26:12.071 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:12.071 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 2541, 2024-09-21T06:26:12.071 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-21T06:26:12.071 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-21T06:26:12.071 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 2541, 2024-09-21T06:26:12.071 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1923806, 2024-09-21T06:26:12.071 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:12.071 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.071 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.072 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.072 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:12.072 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.072 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-09-21T06:26:12.072 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:12.072 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-21T06:26:12.072 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:12.072 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-21T06:26:12.073 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-21T06:26:12.073 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:12.073 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:12.073 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 2283, 2024-09-21T06:26:12.073 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 269936, 2024-09-21T06:26:12.073 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 2283, 2024-09-21T06:26:12.073 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 269936, 2024-09-21T06:26:12.073 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:12.073 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.074 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.074 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.074 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:12.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.074 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-09-21T06:26:12.074 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:12.074 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-09-21T06:26:12.074 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:12.074 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-21T06:26:12.075 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-21T06:26:12.075 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:12.075 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:12.075 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 2283, 2024-09-21T06:26:12.075 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 2283, 2024-09-21T06:26:12.075 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 2283, 2024-09-21T06:26:12.075 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 2283, 2024-09-21T06:26:12.075 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:12.075 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.076 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.076 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.076 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:12.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.076 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:26:12.076 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:12.076 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-21T06:26:12.076 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:12.077 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-21T06:26:12.077 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-21T06:26:12.077 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:12.077 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:12.077 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-21T06:26:12.077 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-21T06:26:12.077 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-09-21T06:26:12.077 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-09-21T06:26:12.077 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:12.078 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.078 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.078 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.078 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:12.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.078 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55e51b6776c0": { 2024-09-21T06:26:12.078 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-21T06:26:12.078 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-21T06:26:12.079 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-21T06:26:12.079 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-21T06:26:12.079 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-21T06:26:12.079 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:12.079 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:12.079 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-21T06:26:12.079 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-21T06:26:12.079 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-09-21T06:26:12.080 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-09-21T06:26:12.080 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-21T06:26:12.080 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-21T06:26:12.080 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-21T06:26:12.080 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-21T06:26:12.080 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:12.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-21T06:26:12.080 INFO:teuthology.orchestra.run.smithi137.stdout: "trackedop": { 2024-09-21T06:26:12.080 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-09-21T06:26:12.081 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-21T06:26:12.081 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-21T06:26:12.820 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-21T06:26:12.821 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-21T06:26:12.821 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-21T06:26:12.821 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:26:12.821 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:26:12.821 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T06:26:12.821 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T06:26:13.086 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:26:13.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:12 smithi137 ceph-mon[34464]: pgmap v672: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-21T06:26:13.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:12 smithi204 ceph-mon[34659]: pgmap v672: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-21T06:26:13.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:12 smithi003 ceph-mon[31227]: pgmap v672: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-21T06:26:13.845 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-21T06:26:13.846 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:26:13.846 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 6594, 2024-09-21T06:26:13.846 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 6743, 2024-09-21T06:26:13.846 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1887868, 2024-09-21T06:26:13.846 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 1152133, 2024-09-21T06:26:13.846 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 15, 2024-09-21T06:26:13.846 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-09-21T06:26:13.847 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.507001725, 2024-09-21T06:26:13.847 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.227308074, 2024-09-21T06:26:13.847 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.359331044, 2024-09-21T06:26:13.847 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.003957007, 2024-09-21T06:26:13.847 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:13.847 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 6741, 2024-09-21T06:26:13.847 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.117402693, 2024-09-21T06:26:13.847 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000017416 2024-09-21T06:26:13.848 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.848 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:13.848 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 12912, 2024-09-21T06:26:13.848 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.002046050, 2024-09-21T06:26:13.848 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000158 2024-09-21T06:26:13.848 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.848 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1856, 2024-09-21T06:26:13.848 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 1712 2024-09-21T06:26:13.848 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.849 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:26:13.849 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 69, 2024-09-21T06:26:13.849 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 249, 2024-09-21T06:26:13.849 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 16085, 2024-09-21T06:26:13.849 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 555366, 2024-09-21T06:26:13.849 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 20, 2024-09-21T06:26:13.850 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 6, 2024-09-21T06:26:13.850 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.023686528, 2024-09-21T06:26:13.850 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.011575908, 2024-09-21T06:26:13.850 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.008352617, 2024-09-21T06:26:13.850 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.001963016, 2024-09-21T06:26:13.850 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:13.850 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 247, 2024-09-21T06:26:13.850 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.025265211, 2024-09-21T06:26:13.850 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000102288 2024-09-21T06:26:13.851 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.851 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:13.851 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 46, 2024-09-21T06:26:13.851 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000009085, 2024-09-21T06:26:13.851 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000197 2024-09-21T06:26:13.851 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.851 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1888, 2024-09-21T06:26:13.851 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 380832 2024-09-21T06:26:13.851 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.852 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:26:13.852 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 143125, 2024-09-21T06:26:13.852 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 71776, 2024-09-21T06:26:13.852 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 62863182, 2024-09-21T06:26:13.852 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 12837087, 2024-09-21T06:26:13.852 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 24, 2024-09-21T06:26:13.852 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 8, 2024-09-21T06:26:13.852 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 7.529958282, 2024-09-21T06:26:13.853 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 3.406800309, 2024-09-21T06:26:13.853 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 5.528417458, 2024-09-21T06:26:13.853 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.004734264, 2024-09-21T06:26:13.853 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:13.853 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 71776, 2024-09-21T06:26:13.853 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.341532675, 2024-09-21T06:26:13.853 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000018690 2024-09-21T06:26:13.853 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.853 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:13.854 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 214022, 2024-09-21T06:26:13.854 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.033686217, 2024-09-21T06:26:13.854 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000157 2024-09-21T06:26:13.854 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.854 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1281888, 2024-09-21T06:26:13.854 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 152688 2024-09-21T06:26:13.854 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.854 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-21T06:26:13.855 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-21T06:26:13.855 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-21T06:26:13.855 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.855 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-21T06:26:13.855 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-21T06:26:13.855 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-21T06:26:13.855 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 306, 2024-09-21T06:26:13.855 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.062106101, 2024-09-21T06:26:13.855 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000202961 2024-09-21T06:26:13.855 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:13.856 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.856 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-21T06:26:13.856 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-21T06:26:13.856 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-21T06:26:13.856 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 33, 2024-09-21T06:26:13.856 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.001066344, 2024-09-21T06:26:13.856 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000032313 2024-09-21T06:26:13.856 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:13.857 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.857 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-21T06:26:13.857 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 6086, 2024-09-21T06:26:13.857 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 28, 2024-09-21T06:26:13.857 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-21T06:26:13.857 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 28, 2024-09-21T06:26:13.857 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.027692761, 2024-09-21T06:26:13.857 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000989027 2024-09-21T06:26:13.858 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.858 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-21T06:26:13.858 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 6058, 2024-09-21T06:26:13.858 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-21T06:26:13.858 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-21T06:26:13.858 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 11, 2024-09-21T06:26:13.858 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-21T06:26:13.858 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-21T06:26:13.858 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 52, 2024-09-21T06:26:13.859 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 2, 2024-09-21T06:26:13.859 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 50, 2024-09-21T06:26:13.859 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-09-21T06:26:13.859 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 47, 2024-09-21T06:26:13.859 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-09-21T06:26:13.859 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 4, 2024-09-21T06:26:13.859 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 4, 2024-09-21T06:26:13.859 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 3, 2024-09-21T06:26:13.859 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 6213, 2024-09-21T06:26:13.860 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 102, 2024-09-21T06:26:13.860 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 6058, 2024-09-21T06:26:13.860 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 17, 2024-09-21T06:26:13.860 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:26:13.860 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-21T06:26:13.860 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-21T06:26:13.860 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-21T06:26:13.860 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-21T06:26:13.860 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-09-21T06:26:13.861 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-09-21T06:26:13.861 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 1, 2024-09-21T06:26:13.861 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 3, 2024-09-21T06:26:13.861 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-21T06:26:13.861 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 36, 2024-09-21T06:26:13.861 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-21T06:26:13.861 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 87, 2024-09-21T06:26:13.861 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 1062181, 2024-09-21T06:26:13.861 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2024-09-21T06:26:13.862 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 36, 2024-09-21T06:26:13.862 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-21T06:26:13.862 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:26:13.862 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:26:13.862 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:26:13.862 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:26:13.862 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:26:13.862 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 36, 2024-09-21T06:26:13.862 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:26:13.863 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 77, 2024-09-21T06:26:13.863 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 63, 2024-09-21T06:26:13.863 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:26:13.863 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:26:13.863 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:26:13.863 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 156, 2024-09-21T06:26:13.863 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 111, 2024-09-21T06:26:13.863 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 6, 2024-09-21T06:26:13.864 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 26 2024-09-21T06:26:13.864 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.864 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-21T06:26:13.864 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-21T06:26:13.864 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 84, 2024-09-21T06:26:13.864 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 44, 2024-09-21T06:26:13.864 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 107, 2024-09-21T06:26:13.864 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-21T06:26:13.864 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-09-21T06:26:13.864 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-21T06:26:13.865 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:26:13.865 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 6, 2024-09-21T06:26:13.865 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 6, 2024-09-21T06:26:13.865 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-21T06:26:13.865 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-21T06:26:13.865 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-21T06:26:13.865 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:26:13.865 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:26:13.866 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-21T06:26:13.866 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-21T06:26:13.866 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:26:13.866 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:26:13.866 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:26:13.866 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 2, 2024-09-21T06:26:13.866 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-21T06:26:13.866 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:26:13.866 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-21T06:26:13.867 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0, 2024-09-21T06:26:13.867 INFO:teuthology.orchestra.run.smithi204.stdout: "uninline_started": 0, 2024-09-21T06:26:13.867 INFO:teuthology.orchestra.run.smithi204.stdout: "uninline_succeeded": 0, 2024-09-21T06:26:13.867 INFO:teuthology.orchestra.run.smithi204.stdout: "uninline_write_failed": 0 2024-09-21T06:26:13.867 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.867 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-21T06:26:13.867 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-09-21T06:26:13.867 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 195, 2024-09-21T06:26:13.867 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-21T06:26:13.869 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-21T06:26:13.869 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 195, 2024-09-21T06:26:13.869 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-21T06:26:13.870 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-21T06:26:13.870 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 2, 2024-09-21T06:26:13.870 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-21T06:26:13.870 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-21T06:26:13.870 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-09-21T06:26:13.870 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-09-21T06:26:13.870 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-21T06:26:13.870 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-21T06:26:13.871 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-09-21T06:26:13.871 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4464235, 2024-09-21T06:26:13.871 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2024-09-21T06:26:13.871 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-21T06:26:13.871 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 107, 2024-09-21T06:26:13.871 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.314678940, 2024-09-21T06:26:13.871 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002940924 2024-09-21T06:26:13.871 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.872 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2024-09-21T06:26:13.872 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.872 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-21T06:26:13.872 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 54, 2024-09-21T06:26:13.872 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 60, 2024-09-21T06:26:13.872 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 6, 2024-09-21T06:26:13.872 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 39, 2024-09-21T06:26:13.872 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 91, 2024-09-21T06:26:13.872 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 52, 2024-09-21T06:26:13.873 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 52, 2024-09-21T06:26:13.873 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 58, 2024-09-21T06:26:13.873 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 6, 2024-09-21T06:26:13.873 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 4, 2024-09-21T06:26:13.873 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 10, 2024-09-21T06:26:13.873 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 6, 2024-09-21T06:26:13.873 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 124568, 2024-09-21T06:26:13.873 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 329072640 2024-09-21T06:26:13.873 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.874 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-21T06:26:13.874 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 6127, 2024-09-21T06:26:13.874 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-09-21T06:26:13.874 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 6086, 2024-09-21T06:26:13.874 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 25, 2024-09-21T06:26:13.874 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 11, 2024-09-21T06:26:13.874 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-21T06:26:13.874 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 6, 2024-09-21T06:26:13.875 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.012864735, 2024-09-21T06:26:13.875 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002144122 2024-09-21T06:26:13.875 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.875 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-21T06:26:13.875 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1, 2024-09-21T06:26:13.875 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.001032156, 2024-09-21T06:26:13.875 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001032156 2024-09-21T06:26:13.875 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.875 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-21T06:26:13.875 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.876 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.876 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.876 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.876 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-21T06:26:13.876 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.876 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.876 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.876 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.876 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-21T06:26:13.877 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 7, 2024-09-21T06:26:13.877 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.001666833, 2024-09-21T06:26:13.877 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000238119 2024-09-21T06:26:13.877 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.877 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-21T06:26:13.877 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.877 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.877 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.877 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.878 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-21T06:26:13.878 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.878 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.878 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.878 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.878 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-21T06:26:13.878 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.878 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.879 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.879 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.879 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-21T06:26:13.879 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.879 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.879 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.879 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.879 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-21T06:26:13.879 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.880 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.880 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.880 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.880 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-21T06:26:13.881 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.881 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.881 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.881 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.881 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-21T06:26:13.881 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.881 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.881 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.881 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.882 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-21T06:26:13.882 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.882 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.882 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.882 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.882 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-21T06:26:13.882 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.882 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.882 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.883 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.883 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-21T06:26:13.883 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.883 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.883 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.883 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.883 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-21T06:26:13.883 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 8, 2024-09-21T06:26:13.883 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.003647468, 2024-09-21T06:26:13.884 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000455933 2024-09-21T06:26:13.884 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.884 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-21T06:26:13.884 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.884 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.884 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.884 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.884 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-21T06:26:13.884 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.885 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.885 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.885 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.885 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-21T06:26:13.885 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.885 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.885 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.885 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.885 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-21T06:26:13.886 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.886 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.886 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.886 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.886 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-21T06:26:13.886 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.886 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.887 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.887 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.887 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-21T06:26:13.887 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.887 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.887 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.887 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.887 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-21T06:26:13.888 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.888 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.888 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.888 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.888 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-21T06:26:13.888 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.888 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.888 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.888 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.889 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-21T06:26:13.889 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.889 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.889 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.889 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.889 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-21T06:26:13.889 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.889 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.889 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.890 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.890 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-21T06:26:13.890 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.890 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.890 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.890 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.890 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-21T06:26:13.890 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.891 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.891 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.891 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.891 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-21T06:26:13.891 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 6, 2024-09-21T06:26:13.891 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.008481569, 2024-09-21T06:26:13.891 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001413594 2024-09-21T06:26:13.891 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.891 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:26:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:26:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-21T06:26:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-21T06:26:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2024-09-21T06:26:13.893 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 4, 2024-09-21T06:26:13.893 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 2, 2024-09-21T06:26:13.893 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 2, 2024-09-21T06:26:13.893 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-21T06:26:13.893 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 11, 2024-09-21T06:26:13.893 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 5, 2024-09-21T06:26:13.893 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 610, 2024-09-21T06:26:13.893 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-21T06:26:13.894 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.894 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-21T06:26:13.894 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:26:13.894 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-21T06:26:13.894 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:26:13.894 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:26:13.894 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:26:13.894 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:26:13.894 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:26:13.895 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:26:13.895 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:26:13.895 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:26:13.895 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:26:13.895 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:26:13.895 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:26:13.895 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:26:13.895 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:26:13.895 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-21T06:26:13.896 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:26:13.896 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-21T06:26:13.896 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:26:13.896 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:26:13.896 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:26:13.896 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:26:13.896 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:26:13.896 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:26:13.897 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:26:13.897 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-21T06:26:13.897 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:26:13.897 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:26:13.897 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:26:13.897 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-21T06:26:13.897 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-21T06:26:13.897 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-21T06:26:13.897 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:26:13.898 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:26:13.898 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:26:13.898 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:26:13.898 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 274420, 2024-09-21T06:26:13.898 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 126, 2024-09-21T06:26:13.898 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 176, 2024-09-21T06:26:13.898 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 2, 2024-09-21T06:26:13.898 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-21T06:26:13.898 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-21T06:26:13.899 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:26:13.899 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-21T06:26:13.899 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:26:13.899 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-21T06:26:13.899 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8288, 2024-09-21T06:26:13.899 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 184, 2024-09-21T06:26:13.899 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:26:13.899 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:26:13.899 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-21T06:26:13.900 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-21T06:26:13.900 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 223401, 2024-09-21T06:26:13.900 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 1281, 2024-09-21T06:26:13.900 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-21T06:26:13.900 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-21T06:26:13.900 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-21T06:26:13.900 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-21T06:26:13.900 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.900 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-21T06:26:13.901 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-21T06:26:13.901 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-21T06:26:13.901 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 235, 2024-09-21T06:26:13.901 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 286540, 2024-09-21T06:26:13.901 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 33, 2024-09-21T06:26:13.901 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 235, 2024-09-21T06:26:13.901 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-21T06:26:13.901 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 235, 2024-09-21T06:26:13.902 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.718990500, 2024-09-21T06:26:13.902 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003059534 2024-09-21T06:26:13.902 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.902 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-21T06:26:13.902 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-21T06:26:13.902 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 235, 2024-09-21T06:26:13.902 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 266 2024-09-21T06:26:13.902 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.903 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 235, 2024-09-21T06:26:13.903 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 39, 2024-09-21T06:26:13.903 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 196, 2024-09-21T06:26:13.903 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-21T06:26:13.903 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-21T06:26:13.903 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 0, 2024-09-21T06:26:13.903 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 10, 2024-09-21T06:26:13.903 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 3, 2024-09-21T06:26:13.904 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 110, 2024-09-21T06:26:13.904 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 36, 2024-09-21T06:26:13.904 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-21T06:26:13.904 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-21T06:26:13.904 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-21T06:26:13.904 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-21T06:26:13.904 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 18, 2024-09-21T06:26:13.904 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-21T06:26:13.905 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-21T06:26:13.905 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-21T06:26:13.905 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 36, 2024-09-21T06:26:13.905 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 20, 2024-09-21T06:26:13.905 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:26:13.905 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-21T06:26:13.905 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:26:13.905 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-21T06:26:13.906 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-21T06:26:13.906 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-21T06:26:13.906 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:26:13.906 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-21T06:26:13.906 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:26:13.906 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-21T06:26:13.906 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-21T06:26:13.906 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-21T06:26:13.907 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-21T06:26:13.907 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-21T06:26:13.907 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-21T06:26:13.907 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-21T06:26:13.907 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-21T06:26:13.907 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-21T06:26:13.907 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-21T06:26:13.907 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-21T06:26:13.907 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-21T06:26:13.908 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-21T06:26:13.908 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-21T06:26:13.908 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-21T06:26:13.908 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-21T06:26:13.908 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-21T06:26:13.908 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 90, 2024-09-21T06:26:13.908 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-21T06:26:13.908 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 13, 2024-09-21T06:26:13.908 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 10, 2024-09-21T06:26:13.909 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 10, 2024-09-21T06:26:13.909 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-21T06:26:13.909 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-21T06:26:13.909 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 29, 2024-09-21T06:26:13.909 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 0, 2024-09-21T06:26:13.909 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 4 2024-09-21T06:26:13.909 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.909 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-21T06:26:13.909 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2024-09-21T06:26:13.910 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:26:13.910 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-21T06:26:13.910 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-21T06:26:13.910 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.910 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-21T06:26:13.910 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-21T06:26:13.910 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 3, 2024-09-21T06:26:13.910 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-21T06:26:13.911 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 3, 2024-09-21T06:26:13.911 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 6, 2024-09-21T06:26:13.911 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 6, 2024-09-21T06:26:13.911 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-21T06:26:13.911 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.911 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:26:13.911 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:13.911 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-21T06:26:13.911 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:13.912 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 149784, 2024-09-21T06:26:13.912 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 51267279, 2024-09-21T06:26:13.912 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:13.912 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 149784, 2024-09-21T06:26:13.912 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-21T06:26:13.912 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-21T06:26:13.912 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 149784, 2024-09-21T06:26:13.912 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 51267279, 2024-09-21T06:26:13.912 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:13.913 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.913 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.913 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.913 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:13.913 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.913 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-21T06:26:13.913 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:13.913 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-21T06:26:13.914 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:13.914 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-21T06:26:13.914 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-21T06:26:13.914 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:13.914 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:13.914 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 235, 2024-09-21T06:26:13.914 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 287314, 2024-09-21T06:26:13.914 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 235, 2024-09-21T06:26:13.915 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 287314, 2024-09-21T06:26:13.915 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:13.915 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.915 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.915 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.915 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:13.915 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.915 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-21T06:26:13.916 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:13.916 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-21T06:26:13.916 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:13.916 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-21T06:26:13.916 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-21T06:26:13.916 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:13.916 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:13.916 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 235, 2024-09-21T06:26:13.916 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 235, 2024-09-21T06:26:13.917 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 235, 2024-09-21T06:26:13.917 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 235, 2024-09-21T06:26:13.917 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:13.917 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.917 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.917 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.917 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:13.917 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.918 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:26:13.918 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:13.918 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-21T06:26:13.918 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:13.918 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 6, 2024-09-21T06:26:13.918 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 774, 2024-09-21T06:26:13.918 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:13.918 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 6, 2024-09-21T06:26:13.918 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-21T06:26:13.919 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-21T06:26:13.919 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 3, 2024-09-21T06:26:13.919 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 774, 2024-09-21T06:26:13.919 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:13.919 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.919 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.919 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.919 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:13.919 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.920 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x5620a6d37380": { 2024-09-21T06:26:13.920 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:13.920 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-21T06:26:13.920 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:13.920 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 195, 2024-09-21T06:26:13.920 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 269931, 2024-09-21T06:26:13.920 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:13.920 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 195, 2024-09-21T06:26:13.920 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-21T06:26:13.921 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-21T06:26:13.921 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 107, 2024-09-21T06:26:13.921 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 269931, 2024-09-21T06:26:13.921 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:13.921 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:13.921 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:13.921 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:13.921 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:13.922 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:13.922 INFO:teuthology.orchestra.run.smithi204.stdout: "trackedop": { 2024-09-21T06:26:13.922 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-09-21T06:26:13.922 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:13.922 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-21T06:26:14.851 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-21T06:26:14.851 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-09-21T06:26:14.851 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-21T06:26:14.851 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:26:14.851 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:26:14.851 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T06:26:14.852 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T06:26:15.109 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:26:15.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:15 smithi204 ceph-mon[34659]: pgmap v673: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-09-21T06:26:15.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:15 smithi204 ceph-mon[34659]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T06:26:15.648 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-21T06:26:15.648 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:26:15.649 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 142896, 2024-09-21T06:26:15.649 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 71606, 2024-09-21T06:26:15.649 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 62793890, 2024-09-21T06:26:15.649 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 12498425, 2024-09-21T06:26:15.649 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 12, 2024-09-21T06:26:15.649 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-09-21T06:26:15.649 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 7.390927691, 2024-09-21T06:26:15.649 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 3.325280260, 2024-09-21T06:26:15.650 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 5.468790307, 2024-09-21T06:26:15.650 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.003938780, 2024-09-21T06:26:15.650 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:15.650 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 71605, 2024-09-21T06:26:15.650 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.305879951, 2024-09-21T06:26:15.650 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000018237 2024-09-21T06:26:15.650 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.650 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:15.651 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 213737, 2024-09-21T06:26:15.651 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.031708601, 2024-09-21T06:26:15.651 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000148 2024-09-21T06:26:15.651 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.651 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1264176, 2024-09-21T06:26:15.651 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 142400 2024-09-21T06:26:15.651 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.651 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:26:15.652 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 40, 2024-09-21T06:26:15.652 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 202, 2024-09-21T06:26:15.652 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 9839, 2024-09-21T06:26:15.652 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 680881, 2024-09-21T06:26:15.652 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 31, 2024-09-21T06:26:15.652 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 7, 2024-09-21T06:26:15.652 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.024168850, 2024-09-21T06:26:15.652 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.009633563, 2024-09-21T06:26:15.652 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.012051501, 2024-09-21T06:26:15.653 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000715504, 2024-09-21T06:26:15.653 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:15.653 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 201, 2024-09-21T06:26:15.653 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.023698505, 2024-09-21T06:26:15.653 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000117903 2024-09-21T06:26:15.653 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.653 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:15.653 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 24, 2024-09-21T06:26:15.653 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000002640, 2024-09-21T06:26:15.654 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000110 2024-09-21T06:26:15.654 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.654 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-21T06:26:15.654 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 374976 2024-09-21T06:26:15.654 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.654 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:26:15.654 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 109, 2024-09-21T06:26:15.654 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 114, 2024-09-21T06:26:15.655 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 59180, 2024-09-21T06:26:15.655 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 121497, 2024-09-21T06:26:15.655 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 14, 2024-09-21T06:26:15.655 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-09-21T06:26:15.655 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.012945796, 2024-09-21T06:26:15.655 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.003869369, 2024-09-21T06:26:15.655 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.008990632, 2024-09-21T06:26:15.655 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000478613, 2024-09-21T06:26:15.655 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:15.656 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 112, 2024-09-21T06:26:15.656 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.023653994, 2024-09-21T06:26:15.656 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000211196 2024-09-21T06:26:15.656 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.656 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:15.656 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 172, 2024-09-21T06:26:15.656 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000028960, 2024-09-21T06:26:15.656 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000168 2024-09-21T06:26:15.657 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.657 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-21T06:26:15.657 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 976 2024-09-21T06:26:15.657 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.657 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-21T06:26:15.657 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-21T06:26:15.657 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-21T06:26:15.657 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.658 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-21T06:26:15.658 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-21T06:26:15.658 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-21T06:26:15.658 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 43, 2024-09-21T06:26:15.658 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.001526018, 2024-09-21T06:26:15.658 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000035488 2024-09-21T06:26:15.658 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:15.658 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.658 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-21T06:26:15.659 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-21T06:26:15.659 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-21T06:26:15.659 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-09-21T06:26:15.659 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000025244, 2024-09-21T06:26:15.659 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000012622 2024-09-21T06:26:15.659 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:15.659 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.659 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-21T06:26:15.660 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 13, 2024-09-21T06:26:15.660 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-09-21T06:26:15.660 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-21T06:26:15.660 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.660 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.660 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.660 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.660 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-21T06:26:15.660 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 13, 2024-09-21T06:26:15.661 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-21T06:26:15.661 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-21T06:26:15.661 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 11, 2024-09-21T06:26:15.661 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-21T06:26:15.661 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-21T06:26:15.661 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 23, 2024-09-21T06:26:15.661 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 2, 2024-09-21T06:26:15.661 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 21, 2024-09-21T06:26:15.661 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-09-21T06:26:15.661 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 19, 2024-09-21T06:26:15.662 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-09-21T06:26:15.662 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-09-21T06:26:15.662 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-09-21T06:26:15.662 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 2, 2024-09-21T06:26:15.662 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 59, 2024-09-21T06:26:15.662 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 40, 2024-09-21T06:26:15.662 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 13, 2024-09-21T06:26:15.662 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 6, 2024-09-21T06:26:15.663 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:26:15.663 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-21T06:26:15.663 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-21T06:26:15.663 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-21T06:26:15.663 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-21T06:26:15.663 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-09-21T06:26:15.663 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-09-21T06:26:15.663 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-09-21T06:26:15.663 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-09-21T06:26:15.664 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-21T06:26:15.664 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:26:15.664 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-21T06:26:15.664 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 87, 2024-09-21T06:26:15.664 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 1062181, 2024-09-21T06:26:15.664 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2024-09-21T06:26:15.664 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:26:15.664 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-21T06:26:15.664 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:26:15.665 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:26:15.665 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:26:15.665 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:26:15.665 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:26:15.665 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-21T06:26:15.665 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:26:15.665 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:26:15.665 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:26:15.666 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:26:15.666 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:26:15.666 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:26:15.666 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-09-21T06:26:15.666 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:26:15.666 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-09-21T06:26:15.666 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-09-21T06:26:15.666 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.666 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-21T06:26:15.667 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-21T06:26:15.667 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 84, 2024-09-21T06:26:15.667 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 44, 2024-09-21T06:26:15.667 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 86, 2024-09-21T06:26:15.667 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-21T06:26:15.667 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-09-21T06:26:15.667 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-21T06:26:15.667 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:26:15.667 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-09-21T06:26:15.668 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-09-21T06:26:15.668 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-21T06:26:15.668 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-21T06:26:15.668 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-21T06:26:15.668 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:26:15.668 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:26:15.668 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-21T06:26:15.668 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-21T06:26:15.668 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:26:15.669 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:26:15.669 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:26:15.669 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-09-21T06:26:15.669 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-21T06:26:15.669 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:26:15.669 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-21T06:26:15.669 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0, 2024-09-21T06:26:15.669 INFO:teuthology.orchestra.run.smithi204.stdout: "uninline_started": 0, 2024-09-21T06:26:15.670 INFO:teuthology.orchestra.run.smithi204.stdout: "uninline_succeeded": 0, 2024-09-21T06:26:15.670 INFO:teuthology.orchestra.run.smithi204.stdout: "uninline_write_failed": 0 2024-09-21T06:26:15.670 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.670 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-21T06:26:15.670 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-09-21T06:26:15.670 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 5, 2024-09-21T06:26:15.670 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-21T06:26:15.670 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-21T06:26:15.670 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 5, 2024-09-21T06:26:15.671 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-21T06:26:15.671 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-21T06:26:15.671 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 2, 2024-09-21T06:26:15.671 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-21T06:26:15.671 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-21T06:26:15.671 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-09-21T06:26:15.671 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-09-21T06:26:15.671 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-21T06:26:15.672 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-21T06:26:15.672 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-09-21T06:26:15.673 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4195732, 2024-09-21T06:26:15.673 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2024-09-21T06:26:15.673 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-21T06:26:15.674 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4, 2024-09-21T06:26:15.674 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.008235825, 2024-09-21T06:26:15.674 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002058956 2024-09-21T06:26:15.674 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.674 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2024-09-21T06:26:15.674 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.674 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-21T06:26:15.674 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 26, 2024-09-21T06:26:15.675 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 26, 2024-09-21T06:26:15.675 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 0, 2024-09-21T06:26:15.675 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 38, 2024-09-21T06:26:15.675 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 80, 2024-09-21T06:26:15.675 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 42, 2024-09-21T06:26:15.675 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 23, 2024-09-21T06:26:15.675 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 23, 2024-09-21T06:26:15.675 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-09-21T06:26:15.675 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-09-21T06:26:15.676 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-09-21T06:26:15.676 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-09-21T06:26:15.676 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 120828, 2024-09-21T06:26:15.676 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 328024064 2024-09-21T06:26:15.676 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.676 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-21T06:26:15.676 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 15, 2024-09-21T06:26:15.676 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-09-21T06:26:15.676 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 13, 2024-09-21T06:26:15.677 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 12, 2024-09-21T06:26:15.677 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-09-21T06:26:15.677 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-21T06:26:15.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.677 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.677 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.677 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-21T06:26:15.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.678 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.678 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.678 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-21T06:26:15.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.678 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.678 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.679 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-21T06:26:15.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.679 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.679 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.679 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-21T06:26:15.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.679 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.679 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.680 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-21T06:26:15.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.680 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.680 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.680 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-21T06:26:15.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.680 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.681 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.681 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-21T06:26:15.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.681 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.681 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.681 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-21T06:26:15.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.682 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.682 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.682 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-21T06:26:15.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.682 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.682 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.682 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-21T06:26:15.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.683 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.683 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.684 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-21T06:26:15.684 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.684 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.684 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.684 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.684 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-21T06:26:15.684 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.684 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.685 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.685 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.685 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-21T06:26:15.685 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.685 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.685 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.685 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.685 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-21T06:26:15.685 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.686 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.686 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.686 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.686 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-21T06:26:15.686 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.686 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.686 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.686 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.686 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-21T06:26:15.687 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.687 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.687 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.687 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.687 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-21T06:26:15.687 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.687 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.687 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.687 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.688 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-21T06:26:15.688 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.688 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.688 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.688 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.688 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-21T06:26:15.688 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.688 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.688 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.689 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.689 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-21T06:26:15.689 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.689 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.689 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.689 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.689 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-21T06:26:15.689 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.689 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.689 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.690 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.690 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-21T06:26:15.690 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.690 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.690 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.690 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.690 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-21T06:26:15.690 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.690 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.691 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.691 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.691 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-21T06:26:15.691 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.691 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.691 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.691 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.691 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-21T06:26:15.692 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.692 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.692 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.692 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.692 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-21T06:26:15.692 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.692 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.692 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.692 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.693 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-21T06:26:15.693 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.693 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.693 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.693 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.693 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-21T06:26:15.693 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.693 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.693 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.694 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.694 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:26:15.694 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:26:15.694 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-21T06:26:15.694 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.694 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.694 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.694 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:15.694 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.695 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-21T06:26:15.695 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-09-21T06:26:15.695 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 2, 2024-09-21T06:26:15.695 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 1, 2024-09-21T06:26:15.695 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2024-09-21T06:26:15.695 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-21T06:26:15.695 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-09-21T06:26:15.695 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-09-21T06:26:15.695 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 554, 2024-09-21T06:26:15.696 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-21T06:26:15.696 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.696 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-21T06:26:15.696 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:26:15.696 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-21T06:26:15.696 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:26:15.696 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:26:15.696 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:26:15.697 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:26:15.697 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:26:15.697 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:26:15.697 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:26:15.697 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:26:15.697 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:26:15.697 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:26:15.697 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:26:15.697 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:26:15.697 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:26:15.698 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-21T06:26:15.698 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:26:15.698 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-21T06:26:15.698 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:26:15.698 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:26:15.698 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:26:15.698 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:26:15.699 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:26:15.699 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:26:15.699 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:26:15.699 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-21T06:26:15.699 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:26:15.699 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:26:15.699 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:26:15.699 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-21T06:26:15.699 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-21T06:26:15.700 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-21T06:26:15.700 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:26:15.700 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:26:15.700 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:26:15.700 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:26:15.700 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 249326, 2024-09-21T06:26:15.700 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 108, 2024-09-21T06:26:15.700 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:26:15.700 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-21T06:26:15.701 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-21T06:26:15.701 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-21T06:26:15.701 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:26:15.701 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-21T06:26:15.701 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:26:15.701 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-21T06:26:15.701 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8288, 2024-09-21T06:26:15.701 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 184, 2024-09-21T06:26:15.701 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:26:15.702 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:26:15.702 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-21T06:26:15.702 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-21T06:26:15.702 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 132063, 2024-09-21T06:26:15.702 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 678, 2024-09-21T06:26:15.702 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-21T06:26:15.702 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-21T06:26:15.702 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-21T06:26:15.702 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-21T06:26:15.703 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.703 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-21T06:26:15.703 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-21T06:26:15.703 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-21T06:26:15.703 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 41, 2024-09-21T06:26:15.703 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 11820, 2024-09-21T06:26:15.703 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 33, 2024-09-21T06:26:15.703 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 41, 2024-09-21T06:26:15.704 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-21T06:26:15.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 41, 2024-09-21T06:26:15.704 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.267996419, 2024-09-21T06:26:15.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.006536498 2024-09-21T06:26:15.704 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.704 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-21T06:26:15.704 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-21T06:26:15.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 41, 2024-09-21T06:26:15.704 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 63 2024-09-21T06:26:15.704 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.705 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 41, 2024-09-21T06:26:15.705 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 0, 2024-09-21T06:26:15.705 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 41, 2024-09-21T06:26:15.705 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-21T06:26:15.705 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-21T06:26:15.705 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 0, 2024-09-21T06:26:15.705 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 10, 2024-09-21T06:26:15.705 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 0, 2024-09-21T06:26:15.705 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 4, 2024-09-21T06:26:15.706 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 8, 2024-09-21T06:26:15.706 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-21T06:26:15.706 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-21T06:26:15.706 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-21T06:26:15.706 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-21T06:26:15.706 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 6, 2024-09-21T06:26:15.706 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-21T06:26:15.706 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-21T06:26:15.706 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-21T06:26:15.706 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 0, 2024-09-21T06:26:15.707 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 20, 2024-09-21T06:26:15.707 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:26:15.707 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-21T06:26:15.707 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:26:15.707 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-21T06:26:15.707 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-21T06:26:15.707 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-21T06:26:15.707 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:26:15.708 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-21T06:26:15.708 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:26:15.708 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-21T06:26:15.708 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-21T06:26:15.708 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-21T06:26:15.708 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-21T06:26:15.708 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-21T06:26:15.708 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-21T06:26:15.708 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-21T06:26:15.709 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-21T06:26:15.709 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-21T06:26:15.709 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-21T06:26:15.709 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-21T06:26:15.709 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-21T06:26:15.709 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-21T06:26:15.709 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-21T06:26:15.709 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-21T06:26:15.709 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-21T06:26:15.709 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-21T06:26:15.710 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 91, 2024-09-21T06:26:15.710 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-21T06:26:15.710 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 14, 2024-09-21T06:26:15.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 10, 2024-09-21T06:26:15.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 10, 2024-09-21T06:26:15.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-21T06:26:15.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-21T06:26:15.710 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 14, 2024-09-21T06:26:15.710 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 0, 2024-09-21T06:26:15.711 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 1 2024-09-21T06:26:15.711 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.711 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-21T06:26:15.711 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2024-09-21T06:26:15.711 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:26:15.711 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-21T06:26:15.711 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-21T06:26:15.711 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.711 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-21T06:26:15.712 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-21T06:26:15.712 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:26:15.712 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-21T06:26:15.712 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-09-21T06:26:15.712 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-09-21T06:26:15.712 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-09-21T06:26:15.712 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-21T06:26:15.712 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.713 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:26:15.713 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:15.713 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-21T06:26:15.713 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:15.713 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 143041, 2024-09-21T06:26:15.713 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 49971573, 2024-09-21T06:26:15.713 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:15.713 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 143041, 2024-09-21T06:26:15.714 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-21T06:26:15.714 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-21T06:26:15.714 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 143041, 2024-09-21T06:26:15.714 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 49971573, 2024-09-21T06:26:15.714 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:15.714 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.714 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.714 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.714 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:15.715 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.715 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-21T06:26:15.715 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:15.715 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-21T06:26:15.715 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:15.715 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-21T06:26:15.715 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-21T06:26:15.715 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:15.716 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:15.716 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 41, 2024-09-21T06:26:15.716 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 11820, 2024-09-21T06:26:15.716 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 41, 2024-09-21T06:26:15.716 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 11820, 2024-09-21T06:26:15.716 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:15.716 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.716 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.716 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.717 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:15.717 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.717 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-21T06:26:15.717 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:15.717 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-21T06:26:15.717 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:15.717 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-21T06:26:15.717 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-21T06:26:15.718 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:15.718 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:15.718 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 41, 2024-09-21T06:26:15.718 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 41, 2024-09-21T06:26:15.718 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 41, 2024-09-21T06:26:15.718 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 41, 2024-09-21T06:26:15.718 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:15.718 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.718 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.719 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.719 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:15.719 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.719 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:26:15.719 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:15.719 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-21T06:26:15.719 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:15.719 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-21T06:26:15.720 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-21T06:26:15.720 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:15.720 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:15.720 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-21T06:26:15.720 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-21T06:26:15.720 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-21T06:26:15.720 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-21T06:26:15.720 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:15.721 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.721 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.721 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.721 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:15.721 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.721 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x557ecc7916c0": { 2024-09-21T06:26:15.721 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:15.721 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-21T06:26:15.721 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:15.722 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 5, 2024-09-21T06:26:15.722 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1428, 2024-09-21T06:26:15.722 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:15.722 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 5, 2024-09-21T06:26:15.722 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-21T06:26:15.722 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-21T06:26:15.722 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 4, 2024-09-21T06:26:15.722 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1428, 2024-09-21T06:26:15.722 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:15.723 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:15.723 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:15.723 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:15.723 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:15.723 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:15.723 INFO:teuthology.orchestra.run.smithi204.stdout: "trackedop": { 2024-09-21T06:26:15.723 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-09-21T06:26:15.723 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:15.723 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-21T06:26:15.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:15 smithi003 ceph-mon[31227]: pgmap v673: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-09-21T06:26:15.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:15 smithi003 ceph-mon[31227]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T06:26:15.848 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:15 smithi137 ceph-mon[34464]: pgmap v673: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-09-21T06:26:15.848 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:15 smithi137 ceph-mon[34464]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T06:26:16.438 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-21T06:26:16.438 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-21T06:26:16.438 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-21T06:26:16.439 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:26:16.439 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:26:16.439 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T06:26:16.439 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T06:26:16.691 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:26:17.282 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-21T06:26:17.282 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:26:17.282 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 142729, 2024-09-21T06:26:17.282 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 71441, 2024-09-21T06:26:17.282 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 61597405, 2024-09-21T06:26:17.283 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 12414249, 2024-09-21T06:26:17.283 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 12, 2024-09-21T06:26:17.283 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-09-21T06:26:17.283 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 7.574197192, 2024-09-21T06:26:17.283 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 3.415448392, 2024-09-21T06:26:17.283 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 5.571570274, 2024-09-21T06:26:17.283 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.005085427, 2024-09-21T06:26:17.283 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:17.284 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 71440, 2024-09-21T06:26:17.284 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.342413887, 2024-09-21T06:26:17.284 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000018790 2024-09-21T06:26:17.284 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.284 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:17.284 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 213917, 2024-09-21T06:26:17.284 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.033741319, 2024-09-21T06:26:17.284 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000157 2024-09-21T06:26:17.284 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.285 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 5168, 2024-09-21T06:26:17.285 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 4256 2024-09-21T06:26:17.285 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.285 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:26:17.285 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 6331, 2024-09-21T06:26:17.285 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 6529, 2024-09-21T06:26:17.285 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1764328, 2024-09-21T06:26:17.285 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 1383652, 2024-09-21T06:26:17.286 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 38, 2024-09-21T06:26:17.286 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 8, 2024-09-21T06:26:17.286 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.500671114, 2024-09-21T06:26:17.286 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.222673438, 2024-09-21T06:26:17.286 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.362085127, 2024-09-21T06:26:17.286 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000844331, 2024-09-21T06:26:17.286 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:17.286 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 6527, 2024-09-21T06:26:17.286 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.109729975, 2024-09-21T06:26:17.287 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000016811 2024-09-21T06:26:17.287 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.287 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:17.287 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 12633, 2024-09-21T06:26:17.287 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.002028337, 2024-09-21T06:26:17.287 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000160 2024-09-21T06:26:17.287 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.287 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1744, 2024-09-21T06:26:17.287 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 377456 2024-09-21T06:26:17.288 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.288 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:26:17.288 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 305, 2024-09-21T06:26:17.288 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 278, 2024-09-21T06:26:17.288 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1282849, 2024-09-21T06:26:17.288 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 177686, 2024-09-21T06:26:17.288 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 8, 2024-09-21T06:26:17.288 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-09-21T06:26:17.288 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.047129766, 2024-09-21T06:26:17.289 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.018389127, 2024-09-21T06:26:17.289 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.040084469, 2024-09-21T06:26:17.289 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.001211626, 2024-09-21T06:26:17.289 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:17.289 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 277, 2024-09-21T06:26:17.289 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.022074841, 2024-09-21T06:26:17.289 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000079692 2024-09-21T06:26:17.289 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.290 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:17.290 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-09-21T06:26:17.290 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000181, 2024-09-21T06:26:17.290 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000090 2024-09-21T06:26:17.290 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.290 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1279168, 2024-09-21T06:26:17.290 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 141440 2024-09-21T06:26:17.290 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.290 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-21T06:26:17.291 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-21T06:26:17.291 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-21T06:26:17.291 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.291 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-21T06:26:17.291 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-21T06:26:17.291 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-21T06:26:17.291 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 45, 2024-09-21T06:26:17.291 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.001802036, 2024-09-21T06:26:17.291 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000040045 2024-09-21T06:26:17.291 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:17.292 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.292 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-21T06:26:17.292 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-21T06:26:17.292 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-21T06:26:17.292 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-09-21T06:26:17.292 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000027362, 2024-09-21T06:26:17.292 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000013681 2024-09-21T06:26:17.293 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:17.293 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.293 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-21T06:26:17.293 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 6280, 2024-09-21T06:26:17.293 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 7, 2024-09-21T06:26:17.293 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-21T06:26:17.293 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 7, 2024-09-21T06:26:17.294 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.003519153, 2024-09-21T06:26:17.294 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000502736 2024-09-21T06:26:17.294 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.294 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-21T06:26:17.294 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 6273, 2024-09-21T06:26:17.294 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-21T06:26:17.294 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-21T06:26:17.295 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 11, 2024-09-21T06:26:17.295 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-21T06:26:17.295 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-21T06:26:17.295 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 23, 2024-09-21T06:26:17.295 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 2, 2024-09-21T06:26:17.295 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 21, 2024-09-21T06:26:17.295 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-09-21T06:26:17.295 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 19, 2024-09-21T06:26:17.295 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-09-21T06:26:17.296 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 3, 2024-09-21T06:26:17.296 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 3, 2024-09-21T06:26:17.296 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 2, 2024-09-21T06:26:17.296 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 6333, 2024-09-21T06:26:17.296 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 49, 2024-09-21T06:26:17.296 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 6270, 2024-09-21T06:26:17.296 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 12, 2024-09-21T06:26:17.296 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:26:17.296 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-21T06:26:17.297 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-21T06:26:17.297 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-21T06:26:17.297 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-21T06:26:17.297 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-09-21T06:26:17.297 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-09-21T06:26:17.297 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-09-21T06:26:17.297 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-09-21T06:26:17.297 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-21T06:26:17.297 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:26:17.298 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-21T06:26:17.298 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 87, 2024-09-21T06:26:17.298 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 1062181, 2024-09-21T06:26:17.298 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2024-09-21T06:26:17.298 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:26:17.298 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-21T06:26:17.298 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:26:17.299 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:26:17.299 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:26:17.299 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:26:17.299 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:26:17.299 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-21T06:26:17.299 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:26:17.299 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 1, 2024-09-21T06:26:17.299 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 2, 2024-09-21T06:26:17.299 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:26:17.300 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:26:17.300 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:26:17.300 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 1, 2024-09-21T06:26:17.300 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:26:17.300 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-09-21T06:26:17.300 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 2 2024-09-21T06:26:17.300 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.300 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-21T06:26:17.301 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-21T06:26:17.301 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 84, 2024-09-21T06:26:17.301 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 44, 2024-09-21T06:26:17.301 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 96, 2024-09-21T06:26:17.301 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-21T06:26:17.301 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-09-21T06:26:17.301 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-21T06:26:17.301 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:26:17.301 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-09-21T06:26:17.302 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-09-21T06:26:17.302 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-21T06:26:17.302 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-21T06:26:17.302 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-21T06:26:17.302 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:26:17.302 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:26:17.302 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-21T06:26:17.302 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-21T06:26:17.303 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:26:17.303 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:26:17.303 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:26:17.303 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-09-21T06:26:17.303 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-21T06:26:17.303 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:26:17.303 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-21T06:26:17.304 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0, 2024-09-21T06:26:17.304 INFO:teuthology.orchestra.run.smithi204.stdout: "uninline_started": 0, 2024-09-21T06:26:17.304 INFO:teuthology.orchestra.run.smithi204.stdout: "uninline_succeeded": 0, 2024-09-21T06:26:17.304 INFO:teuthology.orchestra.run.smithi204.stdout: "uninline_write_failed": 0 2024-09-21T06:26:17.304 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.304 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-21T06:26:17.304 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-09-21T06:26:17.304 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 7, 2024-09-21T06:26:17.305 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-21T06:26:17.305 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-21T06:26:17.305 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 7, 2024-09-21T06:26:17.305 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-21T06:26:17.305 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-21T06:26:17.305 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 2, 2024-09-21T06:26:17.305 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-21T06:26:17.305 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-21T06:26:17.306 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-09-21T06:26:17.306 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-09-21T06:26:17.306 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-21T06:26:17.306 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-21T06:26:17.306 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-09-21T06:26:17.306 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4196264, 2024-09-21T06:26:17.306 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2024-09-21T06:26:17.306 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-21T06:26:17.306 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 6, 2024-09-21T06:26:17.307 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.012527055, 2024-09-21T06:26:17.307 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002087842 2024-09-21T06:26:17.307 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.307 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2024-09-21T06:26:17.307 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.307 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-21T06:26:17.307 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 26, 2024-09-21T06:26:17.307 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 26, 2024-09-21T06:26:17.308 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 0, 2024-09-21T06:26:17.308 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 38, 2024-09-21T06:26:17.308 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 84, 2024-09-21T06:26:17.308 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 46, 2024-09-21T06:26:17.308 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 23, 2024-09-21T06:26:17.308 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 23, 2024-09-21T06:26:17.308 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-09-21T06:26:17.308 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 3, 2024-09-21T06:26:17.308 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 7, 2024-09-21T06:26:17.309 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 4, 2024-09-21T06:26:17.309 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 120768, 2024-09-21T06:26:17.309 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 329068544 2024-09-21T06:26:17.309 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.309 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-21T06:26:17.309 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 6291, 2024-09-21T06:26:17.309 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-09-21T06:26:17.309 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 6280, 2024-09-21T06:26:17.310 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 17, 2024-09-21T06:26:17.310 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 1, 2024-09-21T06:26:17.310 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-21T06:26:17.310 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.310 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.310 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.310 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.310 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-21T06:26:17.310 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3, 2024-09-21T06:26:17.311 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.002533701, 2024-09-21T06:26:17.311 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000844567 2024-09-21T06:26:17.311 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.311 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-21T06:26:17.311 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.311 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.311 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.311 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.311 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-21T06:26:17.312 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.312 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.312 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.312 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.312 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-21T06:26:17.312 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-09-21T06:26:17.312 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000511956, 2024-09-21T06:26:17.312 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000255978 2024-09-21T06:26:17.313 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.313 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-21T06:26:17.313 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.313 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.313 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.313 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.313 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-21T06:26:17.313 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-09-21T06:26:17.314 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000473496, 2024-09-21T06:26:17.314 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000236748 2024-09-21T06:26:17.314 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.314 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-21T06:26:17.314 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.314 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.314 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.314 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.315 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-21T06:26:17.315 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.315 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.315 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.315 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.317 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-21T06:26:17.317 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.317 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.317 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.318 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.318 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-21T06:26:17.318 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.319 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.319 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.319 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.319 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-21T06:26:17.319 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.319 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.319 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.319 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.320 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-21T06:26:17.320 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.320 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.320 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.320 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.320 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-21T06:26:17.320 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.320 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.320 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.320 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.321 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-21T06:26:17.321 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.321 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.321 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.321 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.321 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-21T06:26:17.321 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.322 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.322 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.322 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.322 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-21T06:26:17.322 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.322 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.322 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.322 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.323 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-21T06:26:17.323 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.323 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.323 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.323 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.323 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-21T06:26:17.323 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.323 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.324 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.324 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.324 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-21T06:26:17.324 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.324 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.324 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.324 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.324 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-21T06:26:17.325 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.325 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.325 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.325 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.325 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-21T06:26:17.325 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.325 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.325 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.325 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.326 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-21T06:26:17.326 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.326 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.326 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.326 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.326 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-21T06:26:17.326 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.326 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.327 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.327 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.327 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-21T06:26:17.327 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.327 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.327 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.327 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.327 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-21T06:26:17.328 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.328 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.328 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.328 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.328 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-21T06:26:17.328 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.328 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.328 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.328 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.329 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-21T06:26:17.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.329 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.329 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.329 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-21T06:26:17.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.329 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.330 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.330 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:26:17.330 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:26:17.330 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-21T06:26:17.330 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.330 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.330 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.330 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:17.331 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.331 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-21T06:26:17.331 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-09-21T06:26:17.331 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-09-21T06:26:17.331 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 2, 2024-09-21T06:26:17.331 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2024-09-21T06:26:17.331 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-21T06:26:17.331 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 5, 2024-09-21T06:26:17.331 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 2, 2024-09-21T06:26:17.332 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 668, 2024-09-21T06:26:17.332 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-21T06:26:17.332 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.332 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-21T06:26:17.332 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:26:17.332 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-21T06:26:17.332 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:26:17.332 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:26:17.333 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:26:17.333 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:26:17.333 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:26:17.333 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:26:17.333 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:26:17.333 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:26:17.333 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:26:17.333 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:26:17.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:26:17.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:26:17.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:26:17.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-21T06:26:17.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:26:17.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-21T06:26:17.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:26:17.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:26:17.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:26:17.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:26:17.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:26:17.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:26:17.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:26:17.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-21T06:26:17.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:26:17.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:26:17.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:26:17.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-21T06:26:17.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-21T06:26:17.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-21T06:26:17.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:26:17.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:26:17.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:26:17.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:26:17.336 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 275051, 2024-09-21T06:26:17.336 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 125, 2024-09-21T06:26:17.337 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:26:17.337 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-21T06:26:17.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-21T06:26:17.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-21T06:26:17.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:26:17.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-21T06:26:17.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:26:17.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-21T06:26:17.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8168, 2024-09-21T06:26:17.338 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 181, 2024-09-21T06:26:17.338 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:26:17.338 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:26:17.338 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-21T06:26:17.338 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-21T06:26:17.338 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 129555, 2024-09-21T06:26:17.338 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 669, 2024-09-21T06:26:17.338 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-21T06:26:17.338 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-21T06:26:17.339 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-21T06:26:17.339 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-21T06:26:17.339 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.339 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-21T06:26:17.339 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-21T06:26:17.339 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-21T06:26:17.339 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 47, 2024-09-21T06:26:17.339 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 13127, 2024-09-21T06:26:17.340 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 31, 2024-09-21T06:26:17.340 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 47, 2024-09-21T06:26:17.340 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-21T06:26:17.340 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 47, 2024-09-21T06:26:17.340 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.261996525, 2024-09-21T06:26:17.340 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.005574394 2024-09-21T06:26:17.340 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.340 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-21T06:26:17.340 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-21T06:26:17.341 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 47, 2024-09-21T06:26:17.341 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 73 2024-09-21T06:26:17.341 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.341 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 47, 2024-09-21T06:26:17.341 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 0, 2024-09-21T06:26:17.341 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 47, 2024-09-21T06:26:17.341 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-21T06:26:17.341 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-21T06:26:17.341 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 0, 2024-09-21T06:26:17.342 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 10, 2024-09-21T06:26:17.342 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 0, 2024-09-21T06:26:17.342 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 6, 2024-09-21T06:26:17.342 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 8, 2024-09-21T06:26:17.342 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-21T06:26:17.342 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-21T06:26:17.342 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-21T06:26:17.342 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-21T06:26:17.343 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 6, 2024-09-21T06:26:17.343 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-21T06:26:17.343 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-21T06:26:17.343 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-21T06:26:17.343 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 0, 2024-09-21T06:26:17.343 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 20, 2024-09-21T06:26:17.343 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:26:17.343 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-21T06:26:17.343 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:26:17.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-21T06:26:17.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-21T06:26:17.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-21T06:26:17.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:26:17.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-21T06:26:17.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:26:17.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-21T06:26:17.344 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-21T06:26:17.345 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-21T06:26:17.345 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-21T06:26:17.345 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-21T06:26:17.345 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-21T06:26:17.345 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-21T06:26:17.345 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-21T06:26:17.345 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-21T06:26:17.345 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-21T06:26:17.345 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-21T06:26:17.346 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-21T06:26:17.346 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-21T06:26:17.346 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-21T06:26:17.346 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-21T06:26:17.346 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-21T06:26:17.346 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-21T06:26:17.346 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 91, 2024-09-21T06:26:17.347 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-21T06:26:17.347 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 14, 2024-09-21T06:26:17.347 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 10, 2024-09-21T06:26:17.347 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 10, 2024-09-21T06:26:17.347 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-21T06:26:17.347 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-21T06:26:17.347 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 22, 2024-09-21T06:26:17.347 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 0, 2024-09-21T06:26:17.348 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 1 2024-09-21T06:26:17.348 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.348 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-21T06:26:17.348 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2024-09-21T06:26:17.348 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:26:17.348 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-21T06:26:17.348 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-21T06:26:17.349 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.349 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-21T06:26:17.349 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-21T06:26:17.349 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:26:17.349 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-21T06:26:17.349 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-09-21T06:26:17.349 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-09-21T06:26:17.349 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-09-21T06:26:17.350 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-21T06:26:17.350 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.350 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:26:17.350 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:17.350 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-21T06:26:17.350 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:17.350 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 149361, 2024-09-21T06:26:17.350 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 51183872, 2024-09-21T06:26:17.351 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:17.351 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 149361, 2024-09-21T06:26:17.351 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-21T06:26:17.351 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-21T06:26:17.351 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 149361, 2024-09-21T06:26:17.351 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 51183872, 2024-09-21T06:26:17.351 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:17.351 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.352 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.352 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.352 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:17.352 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.352 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-21T06:26:17.352 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:17.352 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-21T06:26:17.352 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:17.353 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-21T06:26:17.353 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-21T06:26:17.353 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:17.353 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:17.353 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 47, 2024-09-21T06:26:17.353 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 13127, 2024-09-21T06:26:17.353 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 47, 2024-09-21T06:26:17.353 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 13127, 2024-09-21T06:26:17.353 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:17.354 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.354 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.354 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.354 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:17.354 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.354 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-21T06:26:17.354 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:17.354 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-21T06:26:17.354 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:17.355 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-21T06:26:17.355 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-21T06:26:17.355 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:17.355 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:17.355 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 47, 2024-09-21T06:26:17.355 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 47, 2024-09-21T06:26:17.355 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 47, 2024-09-21T06:26:17.355 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 47, 2024-09-21T06:26:17.356 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:17.356 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.356 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.356 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.356 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:17.356 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.356 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:26:17.356 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:17.356 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-21T06:26:17.357 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:17.357 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-21T06:26:17.357 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-21T06:26:17.357 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:17.357 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:17.357 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-21T06:26:17.357 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-21T06:26:17.357 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-21T06:26:17.357 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-21T06:26:17.357 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:17.358 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.358 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.358 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.358 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:17.358 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.358 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x5562409e1450": { 2024-09-21T06:26:17.358 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:17.358 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-21T06:26:17.358 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:17.359 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 7, 2024-09-21T06:26:17.359 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1960, 2024-09-21T06:26:17.359 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:17.359 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 7, 2024-09-21T06:26:17.359 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-21T06:26:17.359 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-21T06:26:17.359 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 6, 2024-09-21T06:26:17.359 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1960, 2024-09-21T06:26:17.359 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:17.359 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:17.360 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:17.360 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:17.360 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:17.360 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:17.360 INFO:teuthology.orchestra.run.smithi204.stdout: "trackedop": { 2024-09-21T06:26:17.360 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-09-21T06:26:17.360 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:17.360 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-21T06:26:17.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:17 smithi204 ceph-mon[34659]: pgmap v675: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 310 B/s wr, 10 op/s 2024-09-21T06:26:17.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:17 smithi003 ceph-mon[31227]: pgmap v675: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 310 B/s wr, 10 op/s 2024-09-21T06:26:17.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:17 smithi137 ceph-mon[34464]: pgmap v675: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 310 B/s wr, 10 op/s 2024-09-21T06:26:17.972 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-21T06:26:17.972 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-21T06:26:17.973 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-21T06:26:17.973 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:26:17.973 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:26:17.973 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T06:26:17.973 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T06:26:18.222 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:26:18.761 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-21T06:26:18.761 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:26:18.761 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2239, 2024-09-21T06:26:18.762 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 2241, 2024-09-21T06:26:18.762 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 635421, 2024-09-21T06:26:18.762 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 712009, 2024-09-21T06:26:18.762 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-09-21T06:26:18.762 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-09-21T06:26:18.762 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.204018682, 2024-09-21T06:26:18.762 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.088264957, 2024-09-21T06:26:18.762 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.043521593, 2024-09-21T06:26:18.763 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.059726881, 2024-09-21T06:26:18.763 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:18.763 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2239, 2024-09-21T06:26:18.763 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.082970978, 2024-09-21T06:26:18.763 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000037057 2024-09-21T06:26:18.763 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.763 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:18.763 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.763 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.764 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.764 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.764 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:26:18.764 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:26:18.764 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.764 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:26:18.764 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 3, 2024-09-21T06:26:18.764 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 171, 2024-09-21T06:26:18.764 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 716, 2024-09-21T06:26:18.765 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 375754, 2024-09-21T06:26:18.765 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-09-21T06:26:18.765 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-09-21T06:26:18.765 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.010366866, 2024-09-21T06:26:18.765 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.007909711, 2024-09-21T06:26:18.765 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.001137066, 2024-09-21T06:26:18.765 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000071621, 2024-09-21T06:26:18.765 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:18.765 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 169, 2024-09-21T06:26:18.766 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.007450782, 2024-09-21T06:26:18.766 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000044087 2024-09-21T06:26:18.766 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.766 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:18.766 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.766 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.766 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.766 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.766 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:26:18.767 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 375056 2024-09-21T06:26:18.767 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.767 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:26:18.767 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 264, 2024-09-21T06:26:18.767 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 243, 2024-09-21T06:26:18.767 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1255644, 2024-09-21T06:26:18.767 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 140750, 2024-09-21T06:26:18.767 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 3, 2024-09-21T06:26:18.767 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-09-21T06:26:18.768 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.046530555, 2024-09-21T06:26:18.768 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.017925966, 2024-09-21T06:26:18.768 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.043146681, 2024-09-21T06:26:18.768 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000035142, 2024-09-21T06:26:18.768 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:26:18.768 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 243, 2024-09-21T06:26:18.768 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.015797173, 2024-09-21T06:26:18.768 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000065008 2024-09-21T06:26:18.768 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.769 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:26:18.769 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.769 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.769 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.769 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.769 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1255376, 2024-09-21T06:26:18.769 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 140432 2024-09-21T06:26:18.769 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.769 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-21T06:26:18.769 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-21T06:26:18.770 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-21T06:26:18.770 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.770 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-21T06:26:18.770 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-21T06:26:18.770 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-21T06:26:18.770 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3789, 2024-09-21T06:26:18.770 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.099862528, 2024-09-21T06:26:18.770 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000026355 2024-09-21T06:26:18.770 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:18.771 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.771 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-21T06:26:18.771 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-21T06:26:18.771 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-21T06:26:18.771 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.771 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.771 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.771 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:18.771 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.772 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-21T06:26:18.772 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-09-21T06:26:18.772 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-09-21T06:26:18.772 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-21T06:26:18.772 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.772 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.772 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.772 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.772 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-21T06:26:18.772 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-09-21T06:26:18.773 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-21T06:26:18.773 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-21T06:26:18.773 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-09-21T06:26:18.773 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-21T06:26:18.773 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-21T06:26:18.773 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 0, 2024-09-21T06:26:18.773 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-09-21T06:26:18.773 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 0, 2024-09-21T06:26:18.773 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-09-21T06:26:18.773 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 0, 2024-09-21T06:26:18.774 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-09-21T06:26:18.774 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-09-21T06:26:18.774 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-09-21T06:26:18.774 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 1, 2024-09-21T06:26:18.774 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-09-21T06:26:18.774 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-09-21T06:26:18.774 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-09-21T06:26:18.774 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-09-21T06:26:18.775 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:26:18.775 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-21T06:26:18.775 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-21T06:26:18.775 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-21T06:26:18.775 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-21T06:26:18.775 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-09-21T06:26:18.775 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-09-21T06:26:18.775 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-09-21T06:26:18.775 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-09-21T06:26:18.775 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-21T06:26:18.776 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:26:18.776 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-21T06:26:18.776 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-09-21T06:26:18.776 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-09-21T06:26:18.776 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-09-21T06:26:18.776 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:26:18.776 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-21T06:26:18.776 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:26:18.776 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:26:18.777 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:26:18.777 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:26:18.777 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:26:18.777 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-21T06:26:18.777 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:26:18.777 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:26:18.777 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:26:18.777 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:26:18.777 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:26:18.778 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:26:18.778 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-09-21T06:26:18.778 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:26:18.778 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-09-21T06:26:18.778 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-09-21T06:26:18.778 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.778 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-21T06:26:18.778 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-21T06:26:18.778 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-09-21T06:26:18.778 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-09-21T06:26:18.779 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-09-21T06:26:18.779 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-21T06:26:18.779 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-09-21T06:26:18.779 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-21T06:26:18.779 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:26:18.779 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-09-21T06:26:18.779 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-09-21T06:26:18.779 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-21T06:26:18.779 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-21T06:26:18.780 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-21T06:26:18.780 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:26:18.780 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:26:18.780 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-21T06:26:18.780 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-21T06:26:18.780 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:26:18.780 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:26:18.780 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:26:18.780 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-09-21T06:26:18.781 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-21T06:26:18.781 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:26:18.781 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-21T06:26:18.781 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0, 2024-09-21T06:26:18.781 INFO:teuthology.orchestra.run.smithi204.stdout: "uninline_started": 0, 2024-09-21T06:26:18.781 INFO:teuthology.orchestra.run.smithi204.stdout: "uninline_succeeded": 0, 2024-09-21T06:26:18.781 INFO:teuthology.orchestra.run.smithi204.stdout: "uninline_write_failed": 0 2024-09-21T06:26:18.781 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.781 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-21T06:26:18.782 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-09-21T06:26:18.782 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-09-21T06:26:18.782 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-21T06:26:18.782 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-21T06:26:18.782 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 7, 2024-09-21T06:26:18.782 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-21T06:26:18.782 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-21T06:26:18.782 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-09-21T06:26:18.783 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-21T06:26:18.783 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-21T06:26:18.783 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-09-21T06:26:18.783 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-09-21T06:26:18.783 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-21T06:26:18.783 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-21T06:26:18.783 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-09-21T06:26:18.783 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4196264, 2024-09-21T06:26:18.783 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 4196123, 2024-09-21T06:26:18.784 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-21T06:26:18.784 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.784 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.784 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.784 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.784 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 7 2024-09-21T06:26:18.784 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.784 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-21T06:26:18.785 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 3, 2024-09-21T06:26:18.785 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 3, 2024-09-21T06:26:18.785 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 0, 2024-09-21T06:26:18.785 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 1, 2024-09-21T06:26:18.785 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 1, 2024-09-21T06:26:18.785 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 0, 2024-09-21T06:26:18.785 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 0, 2024-09-21T06:26:18.785 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 0, 2024-09-21T06:26:18.785 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-09-21T06:26:18.786 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-09-21T06:26:18.786 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-09-21T06:26:18.786 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-09-21T06:26:18.786 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 117720, 2024-09-21T06:26:18.786 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 316420096 2024-09-21T06:26:18.786 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.786 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-21T06:26:18.786 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-09-21T06:26:18.786 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-09-21T06:26:18.787 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-09-21T06:26:18.787 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-09-21T06:26:18.787 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-09-21T06:26:18.787 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-21T06:26:18.787 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.787 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.787 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.787 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.788 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-21T06:26:18.788 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.788 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.788 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.788 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.788 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-21T06:26:18.788 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.788 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.788 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.789 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.789 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-21T06:26:18.789 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.789 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.789 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.789 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.789 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-21T06:26:18.789 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.789 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.789 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.790 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.790 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-21T06:26:18.790 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.790 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.790 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.790 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.790 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-21T06:26:18.790 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.790 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.791 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.791 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.791 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-21T06:26:18.791 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.791 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.791 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.791 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.791 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-21T06:26:18.791 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.791 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.792 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.792 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.792 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-21T06:26:18.792 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.792 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.792 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.792 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.792 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-21T06:26:18.792 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.793 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.793 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.793 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.793 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-21T06:26:18.794 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.794 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.794 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.794 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.794 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-21T06:26:18.794 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.794 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.794 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.794 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.795 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-21T06:26:18.795 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.795 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.795 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.795 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.795 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-21T06:26:18.795 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.795 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.795 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.796 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.796 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-21T06:26:18.796 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.796 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.796 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.796 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.796 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-21T06:26:18.796 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.796 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.797 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.797 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.797 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-21T06:26:18.797 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.797 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.797 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.797 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.797 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-21T06:26:18.797 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.798 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.798 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.798 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.798 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-21T06:26:18.798 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.798 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.798 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.798 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.799 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-21T06:26:18.799 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.799 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.799 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.799 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.799 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-21T06:26:18.799 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.799 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.800 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.800 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.800 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-21T06:26:18.800 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.800 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.800 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.800 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.800 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-21T06:26:18.801 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.801 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.801 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.801 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.801 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-21T06:26:18.801 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.801 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.801 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.801 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.802 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-21T06:26:18.802 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.802 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.802 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.802 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.802 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-21T06:26:18.802 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.802 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.803 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.803 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.803 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-21T06:26:18.803 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.803 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.803 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.803 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.803 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-21T06:26:18.804 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.804 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.804 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.804 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.804 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:26:18.804 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:26:18.804 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-21T06:26:18.804 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.804 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.805 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.805 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:18.805 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.805 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-21T06:26:18.805 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-09-21T06:26:18.805 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-09-21T06:26:18.805 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 2, 2024-09-21T06:26:18.805 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 2, 2024-09-21T06:26:18.806 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-21T06:26:18.806 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-09-21T06:26:18.806 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-09-21T06:26:18.806 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 3340, 2024-09-21T06:26:18.806 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-21T06:26:18.806 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.806 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-21T06:26:18.806 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:26:18.807 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-21T06:26:18.807 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:26:18.807 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:26:18.807 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:26:18.807 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:26:18.807 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:26:18.807 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:26:18.807 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:26:18.807 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:26:18.808 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:26:18.808 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:26:18.808 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:26:18.808 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:26:18.808 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:26:18.808 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-21T06:26:18.808 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:26:18.808 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-21T06:26:18.809 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:26:18.809 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:26:18.809 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:26:18.809 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:26:18.809 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:26:18.809 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:26:18.809 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:26:18.809 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-21T06:26:18.809 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:26:18.810 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:26:18.810 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:26:18.810 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-21T06:26:18.810 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-21T06:26:18.810 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-21T06:26:18.810 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:26:18.810 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:26:18.810 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:26:18.810 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:26:18.811 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 181303, 2024-09-21T06:26:18.811 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 46, 2024-09-21T06:26:18.811 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:26:18.811 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-21T06:26:18.811 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-21T06:26:18.811 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-21T06:26:18.811 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:26:18.811 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-21T06:26:18.812 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:26:18.812 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-21T06:26:18.812 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8288, 2024-09-21T06:26:18.812 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 184, 2024-09-21T06:26:18.812 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:26:18.812 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:26:18.812 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-21T06:26:18.812 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-21T06:26:18.812 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 8672, 2024-09-21T06:26:18.813 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 13, 2024-09-21T06:26:18.813 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-21T06:26:18.813 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-21T06:26:18.813 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-21T06:26:18.813 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-21T06:26:18.813 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.813 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-21T06:26:18.813 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-21T06:26:18.813 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-21T06:26:18.814 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2242, 2024-09-21T06:26:18.814 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-09-21T06:26:18.814 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-09-21T06:26:18.814 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2242, 2024-09-21T06:26:18.814 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-21T06:26:18.814 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2242, 2024-09-21T06:26:18.814 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.151984933, 2024-09-21T06:26:18.814 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000513820 2024-09-21T06:26:18.815 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.815 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-21T06:26:18.815 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-21T06:26:18.815 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2242, 2024-09-21T06:26:18.815 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2243 2024-09-21T06:26:18.815 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.815 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2242, 2024-09-21T06:26:18.815 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2242, 2024-09-21T06:26:18.815 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-09-21T06:26:18.815 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-21T06:26:18.816 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-21T06:26:18.816 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1457, 2024-09-21T06:26:18.816 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-09-21T06:26:18.816 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 783, 2024-09-21T06:26:18.816 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-09-21T06:26:18.816 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-09-21T06:26:18.816 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-21T06:26:18.816 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-21T06:26:18.816 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-21T06:26:18.817 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-21T06:26:18.817 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-09-21T06:26:18.817 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-21T06:26:18.817 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-21T06:26:18.817 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-21T06:26:18.817 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-09-21T06:26:18.817 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-09-21T06:26:18.817 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:26:18.817 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-21T06:26:18.818 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:26:18.818 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-21T06:26:18.818 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-21T06:26:18.818 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-21T06:26:18.818 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:26:18.818 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-21T06:26:18.818 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:26:18.818 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-21T06:26:18.818 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-21T06:26:18.819 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-21T06:26:18.819 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-21T06:26:18.819 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-21T06:26:18.819 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-21T06:26:18.819 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-21T06:26:18.819 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-21T06:26:18.819 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-21T06:26:18.819 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-21T06:26:18.819 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-21T06:26:18.820 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-21T06:26:18.820 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-21T06:26:18.820 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-21T06:26:18.820 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-21T06:26:18.820 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-21T06:26:18.820 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-21T06:26:18.820 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 91, 2024-09-21T06:26:18.820 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-21T06:26:18.820 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 13, 2024-09-21T06:26:18.821 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 4, 2024-09-21T06:26:18.821 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 4, 2024-09-21T06:26:18.821 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-21T06:26:18.821 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-21T06:26:18.821 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-09-21T06:26:18.821 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-09-21T06:26:18.821 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-09-21T06:26:18.821 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.821 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-21T06:26:18.822 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-09-21T06:26:18.822 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:26:18.822 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-21T06:26:18.822 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-21T06:26:18.822 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.822 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-21T06:26:18.822 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-21T06:26:18.822 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:26:18.822 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-21T06:26:18.823 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-09-21T06:26:18.823 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-09-21T06:26:18.823 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-09-21T06:26:18.823 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-21T06:26:18.823 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.823 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:26:18.823 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:18.823 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-21T06:26:18.824 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:18.824 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2502, 2024-09-21T06:26:18.824 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1650445, 2024-09-21T06:26:18.824 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:18.824 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2502, 2024-09-21T06:26:18.824 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-21T06:26:18.824 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-21T06:26:18.824 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2502, 2024-09-21T06:26:18.824 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1650445, 2024-09-21T06:26:18.824 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:18.825 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.825 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.825 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.825 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:18.825 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.825 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-21T06:26:18.825 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:18.825 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-21T06:26:18.826 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:18.826 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-21T06:26:18.826 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-21T06:26:18.826 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:18.826 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:18.826 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2242, 2024-09-21T06:26:18.826 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 1965, 2024-09-21T06:26:18.826 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2242, 2024-09-21T06:26:18.826 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1965, 2024-09-21T06:26:18.827 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:18.827 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.827 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.827 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.827 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:18.827 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.827 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-21T06:26:18.827 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:18.828 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-21T06:26:18.828 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:18.828 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-21T06:26:18.828 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-21T06:26:18.828 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:18.828 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:18.828 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2242, 2024-09-21T06:26:18.828 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2242, 2024-09-21T06:26:18.828 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2242, 2024-09-21T06:26:18.829 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2242, 2024-09-21T06:26:18.829 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:18.829 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.829 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.829 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.829 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:18.829 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.829 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:26:18.830 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:18.830 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-21T06:26:18.830 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:18.830 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-21T06:26:18.830 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-21T06:26:18.830 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:18.830 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:18.830 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-21T06:26:18.830 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-21T06:26:18.830 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-21T06:26:18.831 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-21T06:26:18.831 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:18.831 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.831 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.831 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.831 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:18.831 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.831 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55bd1a2e7450": { 2024-09-21T06:26:18.832 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-21T06:26:18.832 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-21T06:26:18.832 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-21T06:26:18.832 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-21T06:26:18.832 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-21T06:26:18.832 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-21T06:26:18.832 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-21T06:26:18.832 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-21T06:26:18.832 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-21T06:26:18.833 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-21T06:26:18.833 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-21T06:26:18.833 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-21T06:26:18.833 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-21T06:26:18.833 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-21T06:26:18.833 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-21T06:26:18.833 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:18.833 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-21T06:26:18.833 INFO:teuthology.orchestra.run.smithi204.stdout: "trackedop": { 2024-09-21T06:26:18.834 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-09-21T06:26:18.834 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-21T06:26:18.834 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-21T06:26:19.498 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-21T06:26:19.498 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-21T06:26:19.498 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-21T06:26:19.498 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:26:19.498 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:26:19.499 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T06:26:19.511 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T06:26:19.511 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:26:19.512 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:26:19.533 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-21T06:26:19.534 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:26:19.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T06:26:19.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:19 smithi003 ceph-mon[31227]: pgmap v676: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T06:26:19.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:19 smithi137 ceph-mon[34464]: pgmap v676: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T06:26:19.903 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:19.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:19 smithi204 ceph-mon[34659]: pgmap v676: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T06:26:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.397+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/1204628930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc4110200 msgr2=0x7f1dc4112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.397+0000 7f1dcbfd9640 1 --2- 172.21.15.3:0/1204628930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc4110200 0x7f1dc4112620 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009ca0 tx=0x7f1dbc02a950 comp rx=0 tx=0).stop 2024-09-21T06:26:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.397+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/1204628930 shutdown_connections 2024-09-21T06:26:20.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.397+0000 7f1dcbfd9640 1 --2- 172.21.15.3:0/1204628930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc4110200 0x7f1dc4112620 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:20.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.397+0000 7f1dcbfd9640 1 --2- 172.21.15.3:0/1204628930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dc4102fa0 0x7f1dc410fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:20.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.397+0000 7f1dcbfd9640 1 --2- 172.21.15.3:0/1204628930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dc4102650 0x7f1dc4102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:20.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.397+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/1204628930 >> 172.21.15.3:0/1204628930 conn(0x7f1dc4079760 msgr2=0x7f1dc4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:20.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.397+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/1204628930 shutdown_connections 2024-09-21T06:26:20.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.397+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/1204628930 wait complete. 2024-09-21T06:26:20.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dcbfd9640 1 Processor -- start 2024-09-21T06:26:20.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dcbfd9640 1 -- start start 2024-09-21T06:26:20.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dcbfd9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dc4102650 0x7f1dc419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:20.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dcbfd9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc4102fa0 0x7f1dc419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:20.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dcbfd9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dc4110200 0x7f1dc41a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:20.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dcbfd9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4115ca0 con 0x7f1dc4102fa0 2024-09-21T06:26:20.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dcbfd9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4115b20 con 0x7f1dc4102650 2024-09-21T06:26:20.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dcbfd9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4115e20 con 0x7f1dc4110200 2024-09-21T06:26:20.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dc9574640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc4102fa0 0x7f1dc419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:20.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dc9574640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc4102fa0 0x7f1dc419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34718/0 (socket says 172.21.15.3:34718) 2024-09-21T06:26:20.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dc9574640 1 -- 172.21.15.3:0/3035482564 learned_addr learned my addr 172.21.15.3:0/3035482564 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:26:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dc9d75640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dc4102650 0x7f1dc419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dca576640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dc4110200 0x7f1dc41a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dc9574640 1 -- 172.21.15.3:0/3035482564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dc4110200 msgr2=0x7f1dc41a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dc9574640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dc4110200 0x7f1dc41a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dc9574640 1 -- 172.21.15.3:0/3035482564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dc4102650 msgr2=0x7f1dc419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dc9574640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dc4102650 0x7f1dc419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dc9574640 1 -- 172.21.15.3:0/3035482564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc41a2580 con 0x7f1dc4102fa0 2024-09-21T06:26:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.398+0000 7f1dca576640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dc4110200 0x7f1dc41a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:26:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.399+0000 7f1dc9574640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc4102fa0 0x7f1dc419cbb0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f1db400e9f0 tx=0x7f1db400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.399+0000 7f1dbaffd640 1 -- 172.21.15.3:0/3035482564 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db400cd80 con 0x7f1dc4102fa0 2024-09-21T06:26:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.399+0000 7f1dbaffd640 1 -- 172.21.15.3:0/3035482564 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db40108e0 con 0x7f1dc4102fa0 2024-09-21T06:26:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.399+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/3035482564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc41a2870 con 0x7f1dc4102fa0 2024-09-21T06:26:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.399+0000 7f1dbaffd640 1 -- 172.21.15.3:0/3035482564 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4018980 con 0x7f1dc4102fa0 2024-09-21T06:26:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.399+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/3035482564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc41a8fd0 con 0x7f1dc4102fa0 2024-09-21T06:26:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.400+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/3035482564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc406c1f0 con 0x7f1dc4102fa0 2024-09-21T06:26:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.401+0000 7f1dbaffd640 1 -- 172.21.15.3:0/3035482564 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f1db4010430 con 0x7f1dc4102fa0 2024-09-21T06:26:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.402+0000 7f1dbaffd640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1db00b6700 0x7f1db00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.402+0000 7f1dbaffd640 1 -- 172.21.15.3:0/3035482564 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8507+0+0 (secure 0 0 0) 0x7f1db40e7fa0 con 0x7f1dc4102fa0 2024-09-21T06:26:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.402+0000 7f1dc9d75640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1db00b6700 0x7f1db00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.403+0000 7f1dc9d75640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1db00b6700 0x7f1db00b8c10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009b60 tx=0x7f1dc0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.405+0000 7f1dbaffd640 1 -- 172.21.15.3:0/3035482564 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1db40ae500 con 0x7f1dc4102fa0 2024-09-21T06:26:20.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.613+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/3035482564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1dc419d740 con 0x7f1dc4102fa0 2024-09-21T06:26:20.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.613+0000 7f1dbaffd640 1 -- 172.21.15.3:0/3035482564 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v91) ==== 81+0+867 (secure 0 0 0) 0x7f1db40b28e0 con 0x7f1dc4102fa0 2024-09-21T06:26:20.616 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1573374138 2024-09-22T06:06:23.529208+0000 2024-09-21T06:26:20.616 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/986982328 2024-09-22T06:06:23.529208+0000 2024-09-21T06:26:20.616 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1595702358 2024-09-22T06:04:55.356595+0000 2024-09-21T06:26:20.616 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/547897695 2024-09-22T06:04:55.356595+0000 2024-09-21T06:26:20.616 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3673347975 2024-09-22T06:04:55.356595+0000 2024-09-21T06:26:20.616 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/141972779 2024-09-22T06:06:23.529208+0000 2024-09-21T06:26:20.616 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/1707771276 2024-09-22T06:04:55.356595+0000 2024-09-21T06:26:20.617 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/1707771276 2024-09-22T06:04:55.356595+0000 2024-09-21T06:26:20.617 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2363985440 2024-09-22T06:05:21.689191+0000 2024-09-21T06:26:20.617 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2138752581 2024-09-22T06:05:21.689191+0000 2024-09-21T06:26:20.617 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/986982328 2024-09-22T06:06:23.529208+0000 2024-09-21T06:26:20.617 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1820734047 2024-09-22T06:05:21.689191+0000 2024-09-21T06:26:20.617 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/2929116087 2024-09-22T06:05:21.689191+0000 2024-09-21T06:26:20.617 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/45453677 2024-09-22T06:06:23.529208+0000 2024-09-21T06:26:20.618 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/2929116087 2024-09-22T06:05:21.689191+0000 2024-09-21T06:26:20.618 INFO:teuthology.orchestra.run.smithi003.stderr:listed 15 entries 2024-09-21T06:26:20.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.616+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/3035482564 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1db00b6700 msgr2=0x7f1db00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:20.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.616+0000 7f1dcbfd9640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1db00b6700 0x7f1db00b8c10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009b60 tx=0x7f1dc0005f90 comp rx=0 tx=0).stop 2024-09-21T06:26:20.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.616+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/3035482564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc4102fa0 msgr2=0x7f1dc419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:20.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.616+0000 7f1dcbfd9640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc4102fa0 0x7f1dc419cbb0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f1db400e9f0 tx=0x7f1db400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:26:20.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.616+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/3035482564 shutdown_connections 2024-09-21T06:26:20.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.616+0000 7f1dcbfd9640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f1db00b6700 0x7f1db00b8c10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:20.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.616+0000 7f1dcbfd9640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dc4110200 0x7f1dc41a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:20.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.616+0000 7f1dcbfd9640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc4102fa0 0x7f1dc419cbb0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:20.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.616+0000 7f1dcbfd9640 1 --2- 172.21.15.3:0/3035482564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dc4102650 0x7f1dc419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:20.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.616+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/3035482564 >> 172.21.15.3:0/3035482564 conn(0x7f1dc4079760 msgr2=0x7f1dc4109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:20.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.616+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/3035482564 shutdown_connections 2024-09-21T06:26:20.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:20.616+0000 7f1dcbfd9640 1 -- 172.21.15.3:0/3035482564 wait complete. 2024-09-21T06:26:21.340 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:26:21.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:26:21.341 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:26:21.400 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-21T06:26:21.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:26:21.400 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:26:21.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:26:21.453 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-21T06:26:21.501 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:26:21.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:26:21.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:26:21.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:26:21.559 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-21T06:26:21.565 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:21 smithi003 ceph-mon[31227]: pgmap v677: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T06:26:21.566 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:21 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3035482564' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:26:21.651 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T06:26:21.662 INFO:tasks.cephadm:Teardown begin 2024-09-21T06:26:21.663 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:26:21.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:26:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:21 smithi137 ceph-mon[34464]: pgmap v677: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T06:26:21.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:21 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3035482564' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:26:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:21 smithi204 ceph-mon[34659]: pgmap v677: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T06:26:21.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:21 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3035482564' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:26:21.970 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:22.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.482+0000 7fb012610640 1 -- 172.21.15.3:0/2034113965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c1094c0 msgr2=0x7fb00c1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:22.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.482+0000 7fb012610640 1 --2- 172.21.15.3:0/2034113965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c1094c0 0x7fb00c1098d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb000009900 tx=0x7fb00002a410 comp rx=0 tx=0).stop 2024-09-21T06:26:22.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.483+0000 7fb012610640 1 -- 172.21.15.3:0/2034113965 shutdown_connections 2024-09-21T06:26:22.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.483+0000 7fb012610640 1 --2- 172.21.15.3:0/2034113965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb00c109e10 0x7fb00c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:22.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.483+0000 7fb012610640 1 --2- 172.21.15.3:0/2034113965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c1094c0 0x7fb00c1098d0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:22.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.483+0000 7fb012610640 1 --2- 172.21.15.3:0/2034113965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb00c101fe0 0x7fb00c1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:22.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.483+0000 7fb012610640 1 -- 172.21.15.3:0/2034113965 >> 172.21.15.3:0/2034113965 conn(0x7fb00c077bc0 msgr2=0x7fb00c078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:22.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.483+0000 7fb012610640 1 -- 172.21.15.3:0/2034113965 shutdown_connections 2024-09-21T06:26:22.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.483+0000 7fb012610640 1 -- 172.21.15.3:0/2034113965 wait complete. 2024-09-21T06:26:22.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.483+0000 7fb012610640 1 Processor -- start 2024-09-21T06:26:22.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.483+0000 7fb012610640 1 -- start start 2024-09-21T06:26:22.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb012610640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb00c101fe0 0x7fb00c102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:22.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb012610640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c1094c0 0x7fb00c1034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:22.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb012610640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb00c109e10 0x7fb00c103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:22.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb012610640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c113a80 con 0x7fb00c1094c0 2024-09-21T06:26:22.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb012610640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c113900 con 0x7fb00c101fe0 2024-09-21T06:26:22.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb012610640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c113c00 con 0x7fb00c109e10 2024-09-21T06:26:22.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c1094c0 0x7fb00c1034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:22.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c1094c0 0x7fb00c1034c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34750/0 (socket says 172.21.15.3:34750) 2024-09-21T06:26:22.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb00b7fe640 1 -- 172.21.15.3:0/3422193434 learned_addr learned my addr 172.21.15.3:0/3422193434 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:26:22.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb00bfff640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb00c101fe0 0x7fb00c102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:22.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb010bad640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb00c109e10 0x7fb00c103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:22.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb00b7fe640 1 -- 172.21.15.3:0/3422193434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb00c109e10 msgr2=0x7fb00c103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:22.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb00b7fe640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb00c109e10 0x7fb00c103a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:22.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb00b7fe640 1 -- 172.21.15.3:0/3422193434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb00c101fe0 msgr2=0x7fb00c102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:22.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb00b7fe640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb00c101fe0 0x7fb00c102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:22.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.484+0000 7fb00b7fe640 1 -- 172.21.15.3:0/3422193434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c1a8750 con 0x7fb00c1094c0 2024-09-21T06:26:22.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.485+0000 7fb00b7fe640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c1094c0 0x7fb00c1034c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb000006010 tx=0x7fb000004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:22.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.485+0000 7fb0097fa640 1 -- 172.21.15.3:0/3422193434 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00003f030 con 0x7fb00c1094c0 2024-09-21T06:26:22.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.485+0000 7fb0097fa640 1 -- 172.21.15.3:0/3422193434 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb000003aa0 con 0x7fb00c1094c0 2024-09-21T06:26:22.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.485+0000 7fb012610640 1 -- 172.21.15.3:0/3422193434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c1a89e0 con 0x7fb00c1094c0 2024-09-21T06:26:22.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.485+0000 7fb0097fa640 1 -- 172.21.15.3:0/3422193434 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000035070 con 0x7fb00c1094c0 2024-09-21T06:26:22.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.485+0000 7fb012610640 1 -- 172.21.15.3:0/3422193434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c1a8ec0 con 0x7fb00c1094c0 2024-09-21T06:26:22.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.486+0000 7fb012610640 1 -- 172.21.15.3:0/3422193434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00c1023f0 con 0x7fb00c1094c0 2024-09-21T06:26:22.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.487+0000 7fb0097fa640 1 -- 172.21.15.3:0/3422193434 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fb000042030 con 0x7fb00c1094c0 2024-09-21T06:26:22.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.488+0000 7fb0097fa640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faff40b6680 0x7faff40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:22.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.488+0000 7fb00bfff640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faff40b6680 0x7faff40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:22.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.488+0000 7fb0097fa640 1 -- 172.21.15.3:0/3422193434 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8507+0+0 (secure 0 0 0) 0x7fb000107ff0 con 0x7fb00c1094c0 2024-09-21T06:26:22.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.489+0000 7fb00bfff640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faff40b6680 0x7faff40b8b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb00c1049d0 tx=0x7faffc00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:22.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.492+0000 7fb0097fa640 1 -- 172.21.15.3:0/3422193434 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb0000ce550 con 0x7fb00c1094c0 2024-09-21T06:26:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.721+0000 7fb012610640 1 -- 172.21.15.3:0/3422193434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb00c02e640 con 0x7fb00c1094c0 2024-09-21T06:26:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.721+0000 7fb0097fa640 1 -- 172.21.15.3:0/3422193434 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11965 (secure 0 0 0) 0x7fb0000d2930 con 0x7fb00c1094c0 2024-09-21T06:26:22.724 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:26:22.724 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":39,"btime":"2024-09-21T06:25:45:903545+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:25:45.739295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27344":{"gid":27344,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3457686047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3457686047},{"type":"v1","addr":"172.21.15.137:6835","nonce":3457686047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:26:22.724 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 39 2024-09-21T06:26:22.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.724+0000 7fb012610640 1 -- 172.21.15.3:0/3422193434 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faff40b6680 msgr2=0x7faff40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:22.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.724+0000 7fb012610640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faff40b6680 0x7faff40b8b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb00c1049d0 tx=0x7faffc00b410 comp rx=0 tx=0).stop 2024-09-21T06:26:22.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.724+0000 7fb012610640 1 -- 172.21.15.3:0/3422193434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c1094c0 msgr2=0x7fb00c1034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:22.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.724+0000 7fb012610640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c1094c0 0x7fb00c1034c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb000006010 tx=0x7fb000004060 comp rx=0 tx=0).stop 2024-09-21T06:26:22.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.724+0000 7fb012610640 1 -- 172.21.15.3:0/3422193434 shutdown_connections 2024-09-21T06:26:22.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.724+0000 7fb012610640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7faff40b6680 0x7faff40b8b90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:22.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.724+0000 7fb012610640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb00c109e10 0x7fb00c103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.724+0000 7fb012610640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c1094c0 0x7fb00c1034c0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.724+0000 7fb012610640 1 --2- 172.21.15.3:0/3422193434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb00c101fe0 0x7fb00c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.724+0000 7fb012610640 1 -- 172.21.15.3:0/3422193434 >> 172.21.15.3:0/3422193434 conn(0x7fb00c077bc0 msgr2=0x7fb00c1074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.724+0000 7fb012610640 1 -- 172.21.15.3:0/3422193434 shutdown_connections 2024-09-21T06:26:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:22.724+0000 7fb012610640 1 -- 172.21.15.3:0/3422193434 wait complete. 2024-09-21T06:26:23.427 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:26:23.687 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:23.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:23 smithi003 ceph-mon[31227]: pgmap v678: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T06:26:23.809 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:23 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3422193434' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:26:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:23 smithi137 ceph-mon[34464]: pgmap v678: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T06:26:23.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:23 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3422193434' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:26:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:23 smithi204 ceph-mon[34659]: pgmap v678: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T06:26:23.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:23 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3422193434' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:26:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.182+0000 7f4afa17a640 1 -- 172.21.15.3:0/734170291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af41094d0 msgr2=0x7f4af41098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.182+0000 7f4afa17a640 1 --2- 172.21.15.3:0/734170291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af41094d0 0x7f4af41098e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009900 tx=0x7f4ae802a410 comp rx=0 tx=0).stop 2024-09-21T06:26:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.182+0000 7f4afa17a640 1 -- 172.21.15.3:0/734170291 shutdown_connections 2024-09-21T06:26:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.182+0000 7f4afa17a640 1 --2- 172.21.15.3:0/734170291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af4109e20 0x7f4af4110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:24.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.182+0000 7f4afa17a640 1 --2- 172.21.15.3:0/734170291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af41094d0 0x7f4af41098e0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:24.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.182+0000 7f4afa17a640 1 --2- 172.21.15.3:0/734170291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af4103b90 0x7f4af4103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:24.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.182+0000 7f4afa17a640 1 -- 172.21.15.3:0/734170291 >> 172.21.15.3:0/734170291 conn(0x7f4af40ff620 msgr2=0x7f4af4101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:24.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.182+0000 7f4afa17a640 1 -- 172.21.15.3:0/734170291 shutdown_connections 2024-09-21T06:26:24.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.182+0000 7f4afa17a640 1 -- 172.21.15.3:0/734170291 wait complete. 2024-09-21T06:26:24.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.183+0000 7f4afa17a640 1 Processor -- start 2024-09-21T06:26:24.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.183+0000 7f4afa17a640 1 -- start start 2024-09-21T06:26:24.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.183+0000 7f4afa17a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af4103b90 0x7f4af419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:24.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.183+0000 7f4afa17a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af41094d0 0x7f4af419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.183+0000 7f4afa17a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af4109e20 0x7f4af419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.183+0000 7f4afa17a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4af4113ae0 con 0x7f4af4109e20 2024-09-21T06:26:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.183+0000 7f4afa17a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4af4113960 con 0x7f4af41094d0 2024-09-21T06:26:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.183+0000 7f4afa17a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4af4113c60 con 0x7f4af4103b90 2024-09-21T06:26:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.183+0000 7f4af3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af4109e20 0x7f4af419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.183+0000 7f4af3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af4109e20 0x7f4af419fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34766/0 (socket says 172.21.15.3:34766) 2024-09-21T06:26:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.183+0000 7f4af3fff640 1 -- 172.21.15.3:0/3171035356 learned_addr learned my addr 172.21.15.3:0/3171035356 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:26:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.183+0000 7f4af37fe640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af4103b90 0x7f4af419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.184+0000 7f4af2ffd640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af41094d0 0x7f4af419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.184+0000 7f4af3fff640 1 -- 172.21.15.3:0/3171035356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af4103b90 msgr2=0x7f4af419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.184+0000 7f4af3fff640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af4103b90 0x7f4af419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.184+0000 7f4af3fff640 1 -- 172.21.15.3:0/3171035356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af41094d0 msgr2=0x7f4af419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.184+0000 7f4af3fff640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af41094d0 0x7f4af419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.184+0000 7f4af3fff640 1 -- 172.21.15.3:0/3171035356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af41a0360 con 0x7f4af4109e20 2024-09-21T06:26:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.184+0000 7f4af3fff640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af4109e20 0x7f4af419fcf0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00eaa0 tx=0x7f4aec00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.184+0000 7f4af0ff9640 1 -- 172.21.15.3:0/3171035356 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec004d20 con 0x7f4af4109e20 2024-09-21T06:26:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.184+0000 7f4af0ff9640 1 -- 172.21.15.3:0/3171035356 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aec005020 con 0x7f4af4109e20 2024-09-21T06:26:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.184+0000 7f4afa17a640 1 -- 172.21.15.3:0/3171035356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af41a0650 con 0x7f4af4109e20 2024-09-21T06:26:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.184+0000 7f4af0ff9640 1 -- 172.21.15.3:0/3171035356 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0116b0 con 0x7f4af4109e20 2024-09-21T06:26:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.184+0000 7f4afa17a640 1 -- 172.21.15.3:0/3171035356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af41a0b10 con 0x7f4af4109e20 2024-09-21T06:26:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.186+0000 7f4afa17a640 1 -- 172.21.15.3:0/3171035356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af4103fa0 con 0x7f4af4109e20 2024-09-21T06:26:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.187+0000 7f4af0ff9640 1 -- 172.21.15.3:0/3171035356 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f4aec0051d0 con 0x7f4af4109e20 2024-09-21T06:26:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.188+0000 7f4af0ff9640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4adc0b66b0 0x7f4adc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.188+0000 7f4af37fe640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4adc0b66b0 0x7f4adc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.188+0000 7f4af0ff9640 1 -- 172.21.15.3:0/3171035356 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8507+0+0 (secure 0 0 0) 0x7f4aec0e7870 con 0x7f4af4109e20 2024-09-21T06:26:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.188+0000 7f4af37fe640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4adc0b66b0 0x7f4adc0b8bc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400abe0 tx=0x7f4ae400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:24.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.192+0000 7f4af0ff9640 1 -- 172.21.15.3:0/3171035356 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4aec016030 con 0x7f4af4109e20 2024-09-21T06:26:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.419+0000 7f4afa17a640 1 -- 172.21.15.3:0/3171035356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4af402e640 con 0x7f4af4109e20 2024-09-21T06:26:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.420+0000 7f4af0ff9640 1 -- 172.21.15.3:0/3171035356 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11965 (secure 0 0 0) 0x7f4aec0addd0 con 0x7f4af4109e20 2024-09-21T06:26:24.423 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:26:24.423 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":39,"btime":"2024-09-21T06:25:45:903545+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14781,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/601337393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":601337393},{"type":"v1","addr":"172.21.15.3:6835","nonce":601337393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2476398034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2476398034},{"type":"v1","addr":"172.21.15.3:6837","nonce":2476398034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:13:11.205233+0000","modified":"2024-09-21T06:25:45.739295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24704,"mds_1":24691,"mds_2":24679,"mds_3":24674,"mds_4":24670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14844":{"gid":14844,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/1325295587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":1325295587},{"type":"v1","addr":"172.21.15.3:6839","nonce":1325295587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14856":{"gid":14856,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/2403205583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2403205583},{"type":"v1","addr":"172.21.15.3:6841","nonce":2403205583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/4270441897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4270441897},{"type":"v1","addr":"172.21.15.137:6833","nonce":4270441897}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2480313554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2480313554},{"type":"v1","addr":"172.21.15.204:6833","nonce":2480313554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6835/879721642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":879721642},{"type":"v1","addr":"172.21.15.204:6835","nonce":879721642}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6837/44063020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":44063020},{"type":"v1","addr":"172.21.15.137:6837","nonce":44063020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/3211759973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3211759973},{"type":"v1","addr":"172.21.15.204:6837","nonce":3211759973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/2228257050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2228257050},{"type":"v1","addr":"172.21.15.204:6839","nonce":2228257050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/2663570031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2663570031},{"type":"v1","addr":"172.21.15.137:6839","nonce":2663570031}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27344":{"gid":27344,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3457686047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3457686047},{"type":"v1","addr":"172.21.15.137:6835","nonce":3457686047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24704,"qdb_cluster":[24704,24691,24679,24674,24670]},"id":1}]} 2024-09-21T06:26:24.423 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 39 2024-09-21T06:26:24.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.423+0000 7f4afa17a640 1 -- 172.21.15.3:0/3171035356 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4adc0b66b0 msgr2=0x7f4adc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:24.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.423+0000 7f4afa17a640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4adc0b66b0 0x7f4adc0b8bc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400abe0 tx=0x7f4ae400a2d0 comp rx=0 tx=0).stop 2024-09-21T06:26:24.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.423+0000 7f4afa17a640 1 -- 172.21.15.3:0/3171035356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af4109e20 msgr2=0x7f4af419fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:24.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.423+0000 7f4afa17a640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af4109e20 0x7f4af419fcf0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00eaa0 tx=0x7f4aec00eea0 comp rx=0 tx=0).stop 2024-09-21T06:26:24.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.423+0000 7f4afa17a640 1 -- 172.21.15.3:0/3171035356 shutdown_connections 2024-09-21T06:26:24.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.423+0000 7f4afa17a640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f4adc0b66b0 0x7f4adc0b8bc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:24.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.423+0000 7f4afa17a640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af4109e20 0x7f4af419fcf0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:24.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.423+0000 7f4afa17a640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af41094d0 0x7f4af419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:24.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.423+0000 7f4afa17a640 1 --2- 172.21.15.3:0/3171035356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af4103b90 0x7f4af419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:24.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.423+0000 7f4afa17a640 1 -- 172.21.15.3:0/3171035356 >> 172.21.15.3:0/3171035356 conn(0x7f4af40ff620 msgr2=0x7f4af4101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:24.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.423+0000 7f4afa17a640 1 -- 172.21.15.3:0/3171035356 shutdown_connections 2024-09-21T06:26:24.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:24.423+0000 7f4afa17a640 1 -- 172.21.15.3:0/3171035356 wait complete. 2024-09-21T06:26:25.160 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:26:25.421 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:25.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:25 smithi003 ceph-mon[31227]: pgmap v679: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 818 B/s wr, 11 op/s 2024-09-21T06:26:25.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:25 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3171035356' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:26:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:25 smithi137 ceph-mon[34464]: pgmap v679: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 818 B/s wr, 11 op/s 2024-09-21T06:26:25.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:25 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3171035356' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:26:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:25 smithi204 ceph-mon[34659]: pgmap v679: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 818 B/s wr, 11 op/s 2024-09-21T06:26:25.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:25 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3171035356' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:26:25.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.934+0000 7f9895746640 1 -- 172.21.15.3:0/1867822984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98901094a0 msgr2=0x7f98901098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:25.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.934+0000 7f9895746640 1 --2- 172.21.15.3:0/1867822984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98901094a0 0x7f98901098b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f9884009900 tx=0x7f988402a410 comp rx=0 tx=0).stop 2024-09-21T06:26:25.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.935+0000 7f9895746640 1 -- 172.21.15.3:0/1867822984 shutdown_connections 2024-09-21T06:26:25.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.935+0000 7f9895746640 1 --2- 172.21.15.3:0/1867822984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9890109df0 0x7f9890110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.935+0000 7f9895746640 1 --2- 172.21.15.3:0/1867822984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98901094a0 0x7f98901098b0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.935+0000 7f9895746640 1 --2- 172.21.15.3:0/1867822984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9890103b60 0x7f9890103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.935+0000 7f9895746640 1 -- 172.21.15.3:0/1867822984 >> 172.21.15.3:0/1867822984 conn(0x7f989007b850 msgr2=0x7f989007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:25.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.935+0000 7f9895746640 1 -- 172.21.15.3:0/1867822984 shutdown_connections 2024-09-21T06:26:25.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.935+0000 7f9895746640 1 -- 172.21.15.3:0/1867822984 wait complete. 2024-09-21T06:26:25.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.935+0000 7f9895746640 1 Processor -- start 2024-09-21T06:26:25.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f9895746640 1 -- start start 2024-09-21T06:26:25.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f9895746640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9890103b60 0x7f989019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f9895746640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98901094a0 0x7f989019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f9895746640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9890109df0 0x7f989019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f9895746640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9890113b00 con 0x7f9890103b60 2024-09-21T06:26:25.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f9895746640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9890113980 con 0x7f98901094a0 2024-09-21T06:26:25.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f9895746640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9890113c80 con 0x7f9890109df0 2024-09-21T06:26:25.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f988effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9890103b60 0x7f989019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f988effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9890103b60 0x7f989019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43110/0 (socket says 172.21.15.3:43110) 2024-09-21T06:26:25.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f988effd640 1 -- 172.21.15.3:0/107661697 learned_addr learned my addr 172.21.15.3:0/107661697 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:26:25.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f988f7fe640 1 --2- 172.21.15.3:0/107661697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9890109df0 0x7f989019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f988effd640 1 -- 172.21.15.3:0/107661697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9890109df0 msgr2=0x7f989019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:25.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f988effd640 1 --2- 172.21.15.3:0/107661697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9890109df0 0x7f989019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f988effd640 1 -- 172.21.15.3:0/107661697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98901094a0 msgr2=0x7f989019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:25.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f988effd640 1 --2- 172.21.15.3:0/107661697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98901094a0 0x7f989019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.936+0000 7f988effd640 1 -- 172.21.15.3:0/107661697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98901a0540 con 0x7f9890103b60 2024-09-21T06:26:25.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.937+0000 7f988effd640 1 --2- 172.21.15.3:0/107661697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9890103b60 0x7f989019a5d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f988000d950 tx=0x7f988000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:25.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.937+0000 7f9877fff640 1 -- 172.21.15.3:0/107661697 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880017070 con 0x7f9890103b60 2024-09-21T06:26:25.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.937+0000 7f9877fff640 1 -- 172.21.15.3:0/107661697 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9880009c30 con 0x7f9890103b60 2024-09-21T06:26:25.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.937+0000 7f9895746640 1 -- 172.21.15.3:0/107661697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98901a0830 con 0x7f9890103b60 2024-09-21T06:26:25.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.937+0000 7f9877fff640 1 -- 172.21.15.3:0/107661697 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988000f620 con 0x7f9890103b60 2024-09-21T06:26:25.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.937+0000 7f9895746640 1 -- 172.21.15.3:0/107661697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98901a6f90 con 0x7f9890103b60 2024-09-21T06:26:25.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.938+0000 7f9895746640 1 -- 172.21.15.3:0/107661697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9890103f70 con 0x7f9890103b60 2024-09-21T06:26:25.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.939+0000 7f9877fff640 1 -- 172.21.15.3:0/107661697 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f9880005030 con 0x7f9890103b60 2024-09-21T06:26:25.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.940+0000 7f9877fff640 1 --2- 172.21.15.3:0/107661697 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f98780b6680 0x7f98780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.940+0000 7f9877fff640 1 -- 172.21.15.3:0/107661697 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8507+0+0 (secure 0 0 0) 0x7f98800e73b0 con 0x7f9890103b60 2024-09-21T06:26:25.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.940+0000 7f988e7fc640 1 --2- 172.21.15.3:0/107661697 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f98780b6680 0x7f98780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.941+0000 7f988e7fc640 1 --2- 172.21.15.3:0/107661697 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f98780b6680 0x7f98780b8b90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9884009230 tx=0x7f9884031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:25.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:25.944+0000 7f9877fff640 1 -- 172.21.15.3:0/107661697 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f98800ad910 con 0x7f9890103b60 2024-09-21T06:26:26.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.133+0000 7f9895746640 1 -- 172.21.15.3:0/107661697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f989002e640 con 0x7f9890103b60 2024-09-21T06:26:26.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.134+0000 7f9877fff640 1 -- 172.21.15.3:0/107661697 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23138 (secure 0 0 0) 0x7f9880009de0 con 0x7f9890103b60 2024-09-21T06:26:26.137 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-21T06:26:26.137 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"38d3c75c-77df-11ef-baf5-efdc52797490","created":"2024-09-21T06:04:24.293786+0000","modified":"2024-09-21T06:26:14.408049+0000","last_up_change":"2024-09-21T06:12:17.781243+0000","last_in_change":"2024-09-21T06:11:54.727435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:08:26.015803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:13:07.084406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:13:09.149891+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":90,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51736f81-f410-48ee-a8f3-121289607309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6803","nonce":615312328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6805","nonce":615312328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6809","nonce":615312328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":615312328},{"type":"v1","addr":"172.21.15.3:6807","nonce":615312328}]},"public_addr":"172.21.15.3:6803/615312328","cluster_addr":"172.21.15.3:6805/615312328","heartbeat_back_addr":"172.21.15.3:6809/615312328","heartbeat_front_addr":"172.21.15.3:6807/615312328","state":["exists","up"]},{"osd":1,"uuid":"55696a37-34c1-482a-8561-7d08f0aa29fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6801","nonce":4150747115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6803","nonce":4150747115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6807","nonce":4150747115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4150747115},{"type":"v1","addr":"172.21.15.137:6805","nonce":4150747115}]},"public_addr":"172.21.15.137:6801/4150747115","cluster_addr":"172.21.15.137:6803/4150747115","heartbeat_back_addr":"172.21.15.137:6807/4150747115","heartbeat_front_addr":"172.21.15.137:6805/4150747115","state":["exists","up"]},{"osd":2,"uuid":"8a81d34b-464b-46a2-8ef2-5223b5947004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6801","nonce":3392577077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6803","nonce":3392577077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6807","nonce":3392577077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3392577077},{"type":"v1","addr":"172.21.15.204:6805","nonce":3392577077}]},"public_addr":"172.21.15.204:6801/3392577077","cluster_addr":"172.21.15.204:6803/3392577077","heartbeat_back_addr":"172.21.15.204:6807/3392577077","heartbeat_front_addr":"172.21.15.204:6805/3392577077","state":["exists","up"]},{"osd":3,"uuid":"94ab3938-7a70-44cb-99bf-9d2dedc4eb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6811","nonce":4293741635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6813","nonce":4293741635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6817","nonce":4293741635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":4293741635},{"type":"v1","addr":"172.21.15.3:6815","nonce":4293741635}]},"public_addr":"172.21.15.3:6811/4293741635","cluster_addr":"172.21.15.3:6813/4293741635","heartbeat_back_addr":"172.21.15.3:6817/4293741635","heartbeat_front_addr":"172.21.15.3:6815/4293741635","state":["exists","up"]},{"osd":4,"uuid":"38551535-d692-401c-aad0-bee38f3026d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6809","nonce":2608988785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6811","nonce":2608988785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6815","nonce":2608988785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2608988785},{"type":"v1","addr":"172.21.15.137:6813","nonce":2608988785}]},"public_addr":"172.21.15.137:6809/2608988785","cluster_addr":"172.21.15.137:6811/2608988785","heartbeat_back_addr":"172.21.15.137:6815/2608988785","heartbeat_front_addr":"172.21.15.137:6813/2608988785","state":["exists","up"]},{"osd":5,"uuid":"3d4b785b-28d0-4654-a9b1-4c5c905f4bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6809","nonce":3709074044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6811","nonce":3709074044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6815","nonce":3709074044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3709074044},{"type":"v1","addr":"172.21.15.204:6813","nonce":3709074044}]},"public_addr":"172.21.15.204:6809/3709074044","cluster_addr":"172.21.15.204:6811/3709074044","heartbeat_back_addr":"172.21.15.204:6815/3709074044","heartbeat_front_addr":"172.21.15.204:6813/3709074044","state":["exists","up"]},{"osd":6,"uuid":"ba21dbe9-ab43-4b7e-9446-2929ee874c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6819","nonce":3862646594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6821","nonce":3862646594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6825","nonce":3862646594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3862646594},{"type":"v1","addr":"172.21.15.3:6823","nonce":3862646594}]},"public_addr":"172.21.15.3:6819/3862646594","cluster_addr":"172.21.15.3:6821/3862646594","heartbeat_back_addr":"172.21.15.3:6825/3862646594","heartbeat_front_addr":"172.21.15.3:6823/3862646594","state":["exists","up"]},{"osd":7,"uuid":"9610fe5c-93df-49f9-b8bd-5cb480e0871b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6817","nonce":2086822496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6819","nonce":2086822496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6823","nonce":2086822496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2086822496},{"type":"v1","addr":"172.21.15.137:6821","nonce":2086822496}]},"public_addr":"172.21.15.137:6817/2086822496","cluster_addr":"172.21.15.137:6819/2086822496","heartbeat_back_addr":"172.21.15.137:6823/2086822496","heartbeat_front_addr":"172.21.15.137:6821/2086822496","state":["exists","up"]},{"osd":8,"uuid":"49f6e9b4-6d14-434e-8b42-8b7ffeb2a072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6817","nonce":153476798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6819","nonce":153476798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6823","nonce":153476798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":153476798},{"type":"v1","addr":"172.21.15.204:6821","nonce":153476798}]},"public_addr":"172.21.15.204:6817/153476798","cluster_addr":"172.21.15.204:6819/153476798","heartbeat_back_addr":"172.21.15.204:6823/153476798","heartbeat_front_addr":"172.21.15.204:6821/153476798","state":["exists","up"]},{"osd":9,"uuid":"633f67a7-b759-4ea6-82ed-3c42b3fb70ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6827","nonce":1238491549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6829","nonce":1238491549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6833","nonce":1238491549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1238491549},{"type":"v1","addr":"172.21.15.3:6831","nonce":1238491549}]},"public_addr":"172.21.15.3:6827/1238491549","cluster_addr":"172.21.15.3:6829/1238491549","heartbeat_back_addr":"172.21.15.3:6833/1238491549","heartbeat_front_addr":"172.21.15.3:6831/1238491549","state":["exists","up"]},{"osd":10,"uuid":"94b65a48-ee87-4e59-9296-a72a22787f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6825","nonce":3486552683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6827","nonce":3486552683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6831","nonce":3486552683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3486552683},{"type":"v1","addr":"172.21.15.137:6829","nonce":3486552683}]},"public_addr":"172.21.15.137:6825/3486552683","cluster_addr":"172.21.15.137:6827/3486552683","heartbeat_back_addr":"172.21.15.137:6831/3486552683","heartbeat_front_addr":"172.21.15.137:6829/3486552683","state":["exists","up"]},{"osd":11,"uuid":"c5cc2850-9062-4b8e-beb3-bef0db30a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6825","nonce":1084308610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6827","nonce":1084308610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6831","nonce":1084308610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1084308610},{"type":"v1","addr":"172.21.15.204:6829","nonce":1084308610}]},"public_addr":"172.21.15.204:6825/1084308610","cluster_addr":"172.21.15.204:6827/1084308610","heartbeat_back_addr":"172.21.15.204:6831/1084308610","heartbeat_front_addr":"172.21.15.204:6829/1084308610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:31.898745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.089965+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:22.367464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:48.543248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:14.687331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:09:41.243579+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:07.290227+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:32.930046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:10:59.197638+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:24.315219+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:11:50.333639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:12:16.453640+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1573374138":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1595702358":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/547897695":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/3673347975":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/141972779":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6800/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:6801/1707771276":"2024-09-22T06:04:55.356595+0000","172.21.15.3:0/2363985440":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/2138752581":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/986982328":"2024-09-22T06:06:23.529208+0000","172.21.15.3:0/1820734047":"2024-09-22T06:05:21.689191+0000","172.21.15.3:6800/2929116087":"2024-09-22T06:05:21.689191+0000","172.21.15.3:0/45453677":"2024-09-22T06:06:23.529208+0000","172.21.15.3:6801/2929116087":"2024-09-22T06:05:21.689191+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:26:26.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.137+0000 7f9895746640 1 -- 172.21.15.3:0/107661697 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f98780b6680 msgr2=0x7f98780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:26.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.137+0000 7f9895746640 1 --2- 172.21.15.3:0/107661697 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f98780b6680 0x7f98780b8b90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9884009230 tx=0x7f9884031400 comp rx=0 tx=0).stop 2024-09-21T06:26:26.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.137+0000 7f9895746640 1 -- 172.21.15.3:0/107661697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9890103b60 msgr2=0x7f989019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:26.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.137+0000 7f9895746640 1 --2- 172.21.15.3:0/107661697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9890103b60 0x7f989019a5d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f988000d950 tx=0x7f988000de10 comp rx=0 tx=0).stop 2024-09-21T06:26:26.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.137+0000 7f9895746640 1 -- 172.21.15.3:0/107661697 shutdown_connections 2024-09-21T06:26:26.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.137+0000 7f9895746640 1 --2- 172.21.15.3:0/107661697 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f98780b6680 0x7f98780b8b90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:26.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.137+0000 7f9895746640 1 --2- 172.21.15.3:0/107661697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9890109df0 0x7f989019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:26.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.137+0000 7f9895746640 1 --2- 172.21.15.3:0/107661697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98901094a0 0x7f989019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:26.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.137+0000 7f9895746640 1 --2- 172.21.15.3:0/107661697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9890103b60 0x7f989019a5d0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:26.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.137+0000 7f9895746640 1 -- 172.21.15.3:0/107661697 >> 172.21.15.3:0/107661697 conn(0x7f989007b850 msgr2=0x7f9890107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:26.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.137+0000 7f9895746640 1 -- 172.21.15.3:0/107661697 shutdown_connections 2024-09-21T06:26:26.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:26.137+0000 7f9895746640 1 -- 172.21.15.3:0/107661697 wait complete. 2024-09-21T06:26:26.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:26 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/107661697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:26:26.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:26 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/107661697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:26:26.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:26 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/107661697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:26:26.954 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T06:26:27.212 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:27.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:27 smithi003 ceph-mon[31227]: pgmap v680: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.0 KiB/s wr, 11 op/s 2024-09-21T06:26:27.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:27 smithi204 ceph-mon[34659]: pgmap v680: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.0 KiB/s wr, 11 op/s 2024-09-21T06:26:27.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:27 smithi137 ceph-mon[34464]: pgmap v680: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.0 KiB/s wr, 11 op/s 2024-09-21T06:26:27.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.972+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3945310758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce58109e10 msgr2=0x7fce58110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:27.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.972+0000 7fce5e0dc640 1 --2- 172.21.15.3:0/3945310758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce58109e10 0x7fce58110640 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fce50009ca0 tx=0x7fce5002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:26:27.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.972+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3945310758 shutdown_connections 2024-09-21T06:26:27.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.972+0000 7fce5e0dc640 1 --2- 172.21.15.3:0/3945310758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce581094c0 0x7fce581098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:27.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.972+0000 7fce5e0dc640 1 --2- 172.21.15.3:0/3945310758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce58109e10 0x7fce58110640 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:27.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.972+0000 7fce5e0dc640 1 --2- 172.21.15.3:0/3945310758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce58103b80 0x7fce58103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:27.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.972+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3945310758 >> 172.21.15.3:0/3945310758 conn(0x7fce580ff5d0 msgr2=0x7fce58101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:27.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.972+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3945310758 shutdown_connections 2024-09-21T06:26:27.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.972+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3945310758 wait complete. 2024-09-21T06:26:27.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.972+0000 7fce5e0dc640 1 Processor -- start 2024-09-21T06:26:27.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce5e0dc640 1 -- start start 2024-09-21T06:26:27.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce5e0dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce58103b80 0x7fce5819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:27.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce5e0dc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce581094c0 0x7fce5819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:27.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce5e0dc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce58109e10 0x7fce5819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:27.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce5e0dc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fce58113ad0 con 0x7fce581094c0 2024-09-21T06:26:27.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce5e0dc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce58113950 con 0x7fce58103b80 2024-09-21T06:26:27.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce5e0dc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce58113c50 con 0x7fce58109e10 2024-09-21T06:26:27.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce56ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce581094c0 0x7fce5819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:27.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce56ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce581094c0 0x7fce5819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43130/0 (socket says 172.21.15.3:43130) 2024-09-21T06:26:27.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce56ffd640 1 -- 172.21.15.3:0/3034991813 learned_addr learned my addr 172.21.15.3:0/3034991813 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:26:27.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce57fff640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce58109e10 0x7fce5819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:27.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce577fe640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce58103b80 0x7fce5819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:27.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce56ffd640 1 -- 172.21.15.3:0/3034991813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce58109e10 msgr2=0x7fce5819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:27.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce56ffd640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce58109e10 0x7fce5819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:27.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce56ffd640 1 -- 172.21.15.3:0/3034991813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce58103b80 msgr2=0x7fce5819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:27.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce56ffd640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce58103b80 0x7fce5819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:27.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce56ffd640 1 -- 172.21.15.3:0/3034991813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce581a03d0 con 0x7fce581094c0 2024-09-21T06:26:27.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.973+0000 7fce57fff640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce58109e10 0x7fce5819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:26:27.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.974+0000 7fce56ffd640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce581094c0 0x7fce5819aa00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00d950 tx=0x7fce4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:27.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.974+0000 7fce54ff9640 1 -- 172.21.15.3:0/3034991813 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c017070 con 0x7fce581094c0 2024-09-21T06:26:27.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.974+0000 7fce54ff9640 1 -- 172.21.15.3:0/3034991813 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce4c009c30 con 0x7fce581094c0 2024-09-21T06:26:27.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.974+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3034991813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce581a06c0 con 0x7fce581094c0 2024-09-21T06:26:27.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.974+0000 7fce54ff9640 1 -- 172.21.15.3:0/3034991813 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c0047a0 con 0x7fce581094c0 2024-09-21T06:26:27.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.974+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3034991813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce581a6e20 con 0x7fce581094c0 2024-09-21T06:26:27.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.975+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3034991813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce58103f90 con 0x7fce581094c0 2024-09-21T06:26:27.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.977+0000 7fce54ff9640 1 -- 172.21.15.3:0/3034991813 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fce4c004020 con 0x7fce581094c0 2024-09-21T06:26:27.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.977+0000 7fce54ff9640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fce400b6680 0x7fce400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:27.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.978+0000 7fce577fe640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fce400b6680 0x7fce400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:27.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.978+0000 7fce54ff9640 1 -- 172.21.15.3:0/3034991813 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8507+0+0 (secure 0 0 0) 0x7fce4c0e7890 con 0x7fce581094c0 2024-09-21T06:26:27.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.978+0000 7fce577fe640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fce400b6680 0x7fce400b8b90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fce48009b60 tx=0x7fce48005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:27.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:27.981+0000 7fce54ff9640 1 -- 172.21.15.3:0/3034991813 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fce4c020020 con 0x7fce581094c0 2024-09-21T06:26:28.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.208+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3034991813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fce5819b590 con 0x7fce581094c0 2024-09-21T06:26:28.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.210+0000 7fce54ff9640 1 -- 172.21.15.3:0/3034991813 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) ==== 161+0+0 (secure 0 0 0) 0x7fce4c0addf0 con 0x7fce581094c0 2024-09-21T06:26:28.213 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:26:28.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.214+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3034991813 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fce400b6680 msgr2=0x7fce400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:28.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.214+0000 7fce5e0dc640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fce400b6680 0x7fce400b8b90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fce48009b60 tx=0x7fce48005f90 comp rx=0 tx=0).stop 2024-09-21T06:26:28.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.214+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3034991813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce581094c0 msgr2=0x7fce5819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:28.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.214+0000 7fce5e0dc640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce581094c0 0x7fce5819aa00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00d950 tx=0x7fce4c00de10 comp rx=0 tx=0).stop 2024-09-21T06:26:28.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.215+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3034991813 shutdown_connections 2024-09-21T06:26:28.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.215+0000 7fce5e0dc640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fce400b6680 0x7fce400b8b90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:28.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.215+0000 7fce5e0dc640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce58109e10 0x7fce5819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:28.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.215+0000 7fce5e0dc640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce581094c0 0x7fce5819aa00 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:28.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.215+0000 7fce5e0dc640 1 --2- 172.21.15.3:0/3034991813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce58103b80 0x7fce5819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:28.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.215+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3034991813 >> 172.21.15.3:0/3034991813 conn(0x7fce580ff5d0 msgr2=0x7fce581019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:28.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.215+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3034991813 shutdown_connections 2024-09-21T06:26:28.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:28.215+0000 7fce5e0dc640 1 -- 172.21.15.3:0/3034991813 wait complete. 2024-09-21T06:26:28.440 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b[63057]: ignoring --setuser ceph since I am not root 2024-09-21T06:26:28.440 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b[63057]: ignoring --setgroup ceph since I am not root 2024-09-21T06:26:28.440 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-mds[63061]: -- 172.21.15.137:0/1696107700 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bac8b31860 con 0x55bacccd9180 2024-09-21T06:26:28.440 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b[63057]: starting mds.b at 2024-09-21T06:26:28.441 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e[62522]: ignoring --setuser ceph since I am not root 2024-09-21T06:26:28.441 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e[62522]: ignoring --setgroup ceph since I am not root 2024-09-21T06:26:28.441 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-mds[62526]: -- 172.21.15.137:0/995370226 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558c782c1860 con 0x558c7c469600 2024-09-21T06:26:28.441 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e[62522]: starting mds.e at 2024-09-21T06:26:28.441 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-h[62791]: ignoring --setuser ceph since I am not root 2024-09-21T06:26:28.441 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-h[62791]: ignoring --setgroup ceph since I am not root 2024-09-21T06:26:28.442 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-mds[62795]: -- 172.21.15.137:0/3112052879 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5557c1bbf860 con 0x5557c5d67180 2024-09-21T06:26:28.442 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-h[62791]: starting mds.h at 2024-09-21T06:26:28.442 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-k[62239]: ignoring --setuser ceph since I am not root 2024-09-21T06:26:28.442 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-k[62239]: ignoring --setgroup ceph since I am not root 2024-09-21T06:26:28.442 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-mds[62243]: -- 172.21.15.137:0/859100924 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560625585860 con 0x56062972d180 2024-09-21T06:26:28.442 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-k[62239]: starting mds.k at 2024-09-21T06:26:28.590 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:26:28 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-g[73763]: ignoring --setuser ceph since I am not root 2024-09-21T06:26:28.590 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:26:28 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-g[73763]: ignoring --setgroup ceph since I am not root 2024-09-21T06:26:28.591 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:26:28 smithi003 ceph-mds[73767]: -- 172.21.15.3:0/3958672197 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a7d8199860 con 0x55a7dc341a80 2024-09-21T06:26:28.591 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:26:28 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-g[73763]: starting mds.g at 2024-09-21T06:26:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:28 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a[31223]: 2024-09-21T06:26:28.209+0000 7f4131099640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:26:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:28 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3034991813' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T06:26:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:28 smithi003 ceph-mon[31227]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T06:26:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:28 smithi003 ceph-mon[31227]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:26:28.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:28 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3034991813' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T06:26:28.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:28 smithi003 ceph-mon[31227]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T06:26:28.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:28 smithi003 ceph-mon[31227]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T06:26:28.654 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-f[63025]: ignoring --setuser ceph since I am not root 2024-09-21T06:26:28.654 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-f[63025]: ignoring --setgroup ceph since I am not root 2024-09-21T06:26:28.654 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-mds[63029]: -- 172.21.15.204:0/747438008 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a51b15860 con 0x557a55cbd600 2024-09-21T06:26:28.655 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-f[63025]: starting mds.f at 2024-09-21T06:26:28.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3034991813' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T06:26:28.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-mon[34659]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T06:26:28.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-mon[34659]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:26:28.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3034991813' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T06:26:28.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-mon[34659]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T06:26:28.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-mon[34659]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T06:26:28.656 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-i[63563]: ignoring --setuser ceph since I am not root 2024-09-21T06:26:28.656 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-i[63563]: ignoring --setgroup ceph since I am not root 2024-09-21T06:26:28.656 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-mds[63567]: -- 172.21.15.204:0/179502123 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c8213b5860 con 0x55c82555da80 2024-09-21T06:26:28.657 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-i[63563]: starting mds.i at 2024-09-21T06:26:28.657 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-l[62743]: ignoring --setuser ceph since I am not root 2024-09-21T06:26:28.657 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-l[62743]: ignoring --setgroup ceph since I am not root 2024-09-21T06:26:28.657 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-l[62743]: starting mds.l at 2024-09-21T06:26:28.658 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-c[63294]: ignoring --setuser ceph since I am not root 2024-09-21T06:26:28.658 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-c[63294]: ignoring --setgroup ceph since I am not root 2024-09-21T06:26:28.658 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-mds[63298]: -- 172.21.15.204:0/1428892217 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577968e1860 con 0x55779aa89a80 2024-09-21T06:26:28.658 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:26:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-c[63294]: starting mds.c at 2024-09-21T06:26:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3034991813' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T06:26:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-mon[34464]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T06:26:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-mon[34464]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:26:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3034991813' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T06:26:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-mon[34464]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T06:26:28.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:28 smithi137 ceph-mon[34464]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T06:26:28.978 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:26:29.240 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:29.770 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: pgmap v681: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-21T06:26:29.770 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.3:6838/1694407323,v1:172.21.15.3:6839/1694407323] up:boot 2024-09-21T06:26:29.770 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.137:6833/2175652648,v1:172.21.15.137:6834/2175652648] up:boot 2024-09-21T06:26:29.771 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.204:6832/1419998442,v1:172.21.15.204:6834/1419998442] up:boot 2024-09-21T06:26:29.771 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.137:6832/1293525677,v1:172.21.15.137:6835/1293525677] up:boot 2024-09-21T06:26:29.771 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.204:6833/2362336176,v1:172.21.15.204:6835/2362336176] up:boot 2024-09-21T06:26:29.771 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.3:6840/120296644,v1:172.21.15.3:6841/120296644] up:boot 2024-09-21T06:26:29.771 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.204:6837/3643424608,v1:172.21.15.204:6839/3643424608] up:boot 2024-09-21T06:26:29.771 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.204:6836/1867432808,v1:172.21.15.204:6838/1867432808] up:boot 2024-09-21T06:26:29.771 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.137:6836/1549335218,v1:172.21.15.137:6837/1549335218] up:boot 2024-09-21T06:26:29.771 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: mds.? [v2:172.21.15.137:6838/2117404298,v1:172.21.15.137:6839/2117404298] up:boot 2024-09-21T06:26:29.772 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T06:26:29.772 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:26:29.772 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:26:29.772 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:26:29.772 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:26:29.772 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:26:29.772 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:26:29.772 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:26:29.773 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:26:29.773 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:26:29.773 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:29 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:26:29.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.862+0000 7fde5accc640 1 -- 172.21.15.3:0/4007054536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde54105380 msgr2=0x7fde5410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:29.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.862+0000 7fde5accc640 1 --2- 172.21.15.3:0/4007054536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde54105380 0x7fde5410d8d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fde48009900 tx=0x7fde4802a410 comp rx=0 tx=0).stop 2024-09-21T06:26:29.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.862+0000 7fde5accc640 1 -- 172.21.15.3:0/4007054536 shutdown_connections 2024-09-21T06:26:29.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.862+0000 7fde5accc640 1 --2- 172.21.15.3:0/4007054536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde5410de60 0x7fde54110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:29.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.862+0000 7fde5accc640 1 --2- 172.21.15.3:0/4007054536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde54105380 0x7fde5410d8d0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:29.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.862+0000 7fde5accc640 1 --2- 172.21.15.3:0/4007054536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde54104a30 0x7fde54104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:29.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.862+0000 7fde5accc640 1 -- 172.21.15.3:0/4007054536 >> 172.21.15.3:0/4007054536 conn(0x7fde54079760 msgr2=0x7fde54079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:29.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.862+0000 7fde5accc640 1 -- 172.21.15.3:0/4007054536 shutdown_connections 2024-09-21T06:26:29.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.862+0000 7fde5accc640 1 -- 172.21.15.3:0/4007054536 wait complete. 2024-09-21T06:26:29.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.863+0000 7fde5accc640 1 Processor -- start 2024-09-21T06:26:29.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.863+0000 7fde5accc640 1 -- start start 2024-09-21T06:26:29.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.863+0000 7fde5accc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde54104a30 0x7fde54102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:29.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.863+0000 7fde5accc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde54105380 0x7fde54102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:29.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.863+0000 7fde5accc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5410de60 0x7fde541031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:29.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.863+0000 7fde5accc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fde54113aa0 con 0x7fde54105380 2024-09-21T06:26:29.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.863+0000 7fde5accc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fde54113920 con 0x7fde5410de60 2024-09-21T06:26:29.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.863+0000 7fde5accc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fde54113c20 con 0x7fde54104a30 2024-09-21T06:26:29.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.863+0000 7fde53fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde54105380 0x7fde54102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.863+0000 7fde53fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde54105380 0x7fde54102cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43190/0 (socket says 172.21.15.3:43190) 2024-09-21T06:26:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.863+0000 7fde53fff640 1 -- 172.21.15.3:0/3516857127 learned_addr learned my addr 172.21.15.3:0/3516857127 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:26:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.863+0000 7fde58a68640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde54104a30 0x7fde54102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.864+0000 7fde59269640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5410de60 0x7fde541031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.864+0000 7fde53fff640 1 -- 172.21.15.3:0/3516857127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde54104a30 msgr2=0x7fde54102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.864+0000 7fde53fff640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde54104a30 0x7fde54102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.864+0000 7fde53fff640 1 -- 172.21.15.3:0/3516857127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5410de60 msgr2=0x7fde541031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.864+0000 7fde53fff640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5410de60 0x7fde541031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.864+0000 7fde53fff640 1 -- 172.21.15.3:0/3516857127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde541a8770 con 0x7fde54105380 2024-09-21T06:26:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.864+0000 7fde59269640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5410de60 0x7fde541031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:26:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.864+0000 7fde53fff640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde54105380 0x7fde54102cb0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fde48006010 tx=0x7fde48004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.864+0000 7fde51ffb640 1 -- 172.21.15.3:0/3516857127 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4803f030 con 0x7fde54105380 2024-09-21T06:26:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.864+0000 7fde51ffb640 1 -- 172.21.15.3:0/3516857127 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde48003aa0 con 0x7fde54105380 2024-09-21T06:26:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.864+0000 7fde51ffb640 1 -- 172.21.15.3:0/3516857127 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48035070 con 0x7fde54105380 2024-09-21T06:26:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.864+0000 7fde5accc640 1 -- 172.21.15.3:0/3516857127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde541a8a00 con 0x7fde54105380 2024-09-21T06:26:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.864+0000 7fde5accc640 1 -- 172.21.15.3:0/3516857127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde541a8e90 con 0x7fde54105380 2024-09-21T06:26:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.865+0000 7fde5accc640 1 -- 172.21.15.3:0/3516857127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde5406c1f0 con 0x7fde54105380 2024-09-21T06:26:29.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.867+0000 7fde51ffb640 1 -- 172.21.15.3:0/3516857127 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7fde48042030 con 0x7fde54105380 2024-09-21T06:26:29.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.868+0000 7fde51ffb640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fde3c0b6680 0x7fde3c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:29.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.868+0000 7fde58a68640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fde3c0b6680 0x7fde3c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:29.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.868+0000 7fde51ffb640 1 -- 172.21.15.3:0/3516857127 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8893+0+0 (secure 0 0 0) 0x7fde481081d0 con 0x7fde54105380 2024-09-21T06:26:29.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.869+0000 7fde58a68640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fde3c0b6680 0x7fde3c0b8b90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fde541041c0 tx=0x7fde4400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:29.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:29.872+0000 7fde51ffb640 1 -- 172.21.15.3:0/3516857127 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fde480ce5b0 con 0x7fde54105380 2024-09-21T06:26:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: pgmap v681: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-21T06:26:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.3:6838/1694407323,v1:172.21.15.3:6839/1694407323] up:boot 2024-09-21T06:26:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.137:6833/2175652648,v1:172.21.15.137:6834/2175652648] up:boot 2024-09-21T06:26:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.204:6832/1419998442,v1:172.21.15.204:6834/1419998442] up:boot 2024-09-21T06:26:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.137:6832/1293525677,v1:172.21.15.137:6835/1293525677] up:boot 2024-09-21T06:26:29.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.204:6833/2362336176,v1:172.21.15.204:6835/2362336176] up:boot 2024-09-21T06:26:29.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.3:6840/120296644,v1:172.21.15.3:6841/120296644] up:boot 2024-09-21T06:26:29.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.204:6837/3643424608,v1:172.21.15.204:6839/3643424608] up:boot 2024-09-21T06:26:29.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.204:6836/1867432808,v1:172.21.15.204:6838/1867432808] up:boot 2024-09-21T06:26:29.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.137:6836/1549335218,v1:172.21.15.137:6837/1549335218] up:boot 2024-09-21T06:26:29.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: mds.? [v2:172.21.15.137:6838/2117404298,v1:172.21.15.137:6839/2117404298] up:boot 2024-09-21T06:26:29.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T06:26:29.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:26:29.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:26:29.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:26:29.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:26:29.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:26:29.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:26:29.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:26:29.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:26:29.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:26:29.897 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:29 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:26:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: pgmap v681: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-21T06:26:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.3:6838/1694407323,v1:172.21.15.3:6839/1694407323] up:boot 2024-09-21T06:26:29.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.137:6833/2175652648,v1:172.21.15.137:6834/2175652648] up:boot 2024-09-21T06:26:29.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.204:6832/1419998442,v1:172.21.15.204:6834/1419998442] up:boot 2024-09-21T06:26:29.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.137:6832/1293525677,v1:172.21.15.137:6835/1293525677] up:boot 2024-09-21T06:26:29.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.204:6833/2362336176,v1:172.21.15.204:6835/2362336176] up:boot 2024-09-21T06:26:29.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.3:6840/120296644,v1:172.21.15.3:6841/120296644] up:boot 2024-09-21T06:26:29.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.204:6837/3643424608,v1:172.21.15.204:6839/3643424608] up:boot 2024-09-21T06:26:29.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.204:6836/1867432808,v1:172.21.15.204:6838/1867432808] up:boot 2024-09-21T06:26:29.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.137:6836/1549335218,v1:172.21.15.137:6837/1549335218] up:boot 2024-09-21T06:26:29.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: mds.? [v2:172.21.15.137:6838/2117404298,v1:172.21.15.137:6839/2117404298] up:boot 2024-09-21T06:26:29.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T06:26:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:26:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:26:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:26:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:26:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:26:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:26:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:26:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:26:29.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:26:29.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:29 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:26:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.099+0000 7fde5accc640 1 -- 172.21.15.3:0/3516857127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fde54103a70 con 0x7fde54105380 2024-09-21T06:26:30.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.445+0000 7fde51ffb640 1 -- 172.21.15.3:0/3516857127 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) ==== 104+0+0 (secure 0 0 0) 0x7fde480d2990 con 0x7fde54105380 2024-09-21T06:26:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.449+0000 7fde5accc640 1 -- 172.21.15.3:0/3516857127 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fde3c0b6680 msgr2=0x7fde3c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.449+0000 7fde5accc640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fde3c0b6680 0x7fde3c0b8b90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fde541041c0 tx=0x7fde4400b390 comp rx=0 tx=0).stop 2024-09-21T06:26:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.449+0000 7fde5accc640 1 -- 172.21.15.3:0/3516857127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde54105380 msgr2=0x7fde54102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.449+0000 7fde5accc640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde54105380 0x7fde54102cb0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fde48006010 tx=0x7fde48004060 comp rx=0 tx=0).stop 2024-09-21T06:26:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.449+0000 7fde5accc640 1 -- 172.21.15.3:0/3516857127 shutdown_connections 2024-09-21T06:26:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.449+0000 7fde5accc640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7fde3c0b6680 0x7fde3c0b8b90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.449+0000 7fde5accc640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5410de60 0x7fde541031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.449+0000 7fde5accc640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde54105380 0x7fde54102cb0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.449+0000 7fde5accc640 1 --2- 172.21.15.3:0/3516857127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde54104a30 0x7fde54102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.449+0000 7fde5accc640 1 -- 172.21.15.3:0/3516857127 >> 172.21.15.3:0/3516857127 conn(0x7fde54079760 msgr2=0x7fde540ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.450+0000 7fde5accc640 1 -- 172.21.15.3:0/3516857127 shutdown_connections 2024-09-21T06:26:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:30.450+0000 7fde5accc640 1 -- 172.21.15.3:0/3516857127 wait complete. 2024-09-21T06:26:30.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:30 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3516857127' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:26:30.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:30 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:26:30.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:30 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3516857127' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:26:30.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:30 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:26:30.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:30 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3516857127' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:26:30.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:30 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:26:31.336 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:26:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:31 smithi003 ceph-mon[31227]: pgmap v683: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-21T06:26:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:31 smithi003 ceph-mon[31227]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T06:26:31.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:31 smithi003 ceph-mon[31227]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T06:26:31.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:31 smithi003 ceph-mon[31227]: Cluster is now healthy 2024-09-21T06:26:31.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:31 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/3516857127' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:26:31.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:31 smithi003 ceph-mon[31227]: fsmap 12 up:standby 2024-09-21T06:26:31.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.591 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:26:31.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:26:31.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:31 smithi003 ceph-mon[31227]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.594 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:31.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:31 smithi137 ceph-mon[34464]: pgmap v683: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-21T06:26:31.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:31 smithi137 ceph-mon[34464]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T06:26:31.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:31 smithi137 ceph-mon[34464]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T06:26:31.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:31 smithi137 ceph-mon[34464]: Cluster is now healthy 2024-09-21T06:26:31.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:31 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/3516857127' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:26:31.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:31 smithi137 ceph-mon[34464]: fsmap 12 up:standby 2024-09-21T06:26:31.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:26:31.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:26:31.896 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:31 smithi137 ceph-mon[34464]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:31 smithi204 ceph-mon[34659]: pgmap v683: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-21T06:26:31.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:31 smithi204 ceph-mon[34659]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T06:26:31.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:31 smithi204 ceph-mon[34659]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T06:26:31.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:31 smithi204 ceph-mon[34659]: Cluster is now healthy 2024-09-21T06:26:31.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:31 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/3516857127' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:26:31.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:31 smithi204 ceph-mon[34659]: fsmap 12 up:standby 2024-09-21T06:26:31.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:26:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:26:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:31 smithi204 ceph-mon[34659]: from='mgr.14223 172.21.15.3:0/2727846580' entity='mgr.x' 2024-09-21T06:26:32.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.090+0000 7f34a42a4640 1 -- 172.21.15.3:0/1595269660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349c1094a0 msgr2=0x7f349c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:32.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.090+0000 7f34a42a4640 1 --2- 172.21.15.3:0/1595269660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349c1094a0 0x7f349c1098b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f348c009900 tx=0x7f348c02a410 comp rx=0 tx=0).stop 2024-09-21T06:26:32.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.090+0000 7f34a42a4640 1 -- 172.21.15.3:0/1595269660 shutdown_connections 2024-09-21T06:26:32.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.090+0000 7f34a42a4640 1 --2- 172.21.15.3:0/1595269660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f349c109df0 0x7f349c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.090+0000 7f34a42a4640 1 --2- 172.21.15.3:0/1595269660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349c1094a0 0x7f349c1098b0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.090+0000 7f34a42a4640 1 --2- 172.21.15.3:0/1595269660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f349c103b60 0x7f349c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.090+0000 7f34a42a4640 1 -- 172.21.15.3:0/1595269660 >> 172.21.15.3:0/1595269660 conn(0x7f349c07b850 msgr2=0x7f349c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.090+0000 7f34a42a4640 1 -- 172.21.15.3:0/1595269660 shutdown_connections 2024-09-21T06:26:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.090+0000 7f34a42a4640 1 -- 172.21.15.3:0/1595269660 wait complete. 2024-09-21T06:26:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a42a4640 1 Processor -- start 2024-09-21T06:26:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a42a4640 1 -- start start 2024-09-21T06:26:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a42a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f349c103b60 0x7f349c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a42a4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349c1094a0 0x7f349c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a42a4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f349c109df0 0x7f349c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a42a4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f349c113ab0 con 0x7f349c1094a0 2024-09-21T06:26:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a42a4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f349c113930 con 0x7f349c103b60 2024-09-21T06:26:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a42a4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f349c113c30 con 0x7f349c109df0 2024-09-21T06:26:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a183f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349c1094a0 0x7f349c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a183f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349c1094a0 0x7f349c19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43202/0 (socket says 172.21.15.3:43202) 2024-09-21T06:26:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a183f640 1 -- 172.21.15.3:0/1977940828 learned_addr learned my addr 172.21.15.3:0/1977940828 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:26:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a2040640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f349c103b60 0x7f349c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:32.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a2841640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f349c109df0 0x7f349c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:32.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a183f640 1 -- 172.21.15.3:0/1977940828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f349c109df0 msgr2=0x7f349c19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:32.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a183f640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f349c109df0 0x7f349c19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a183f640 1 -- 172.21.15.3:0/1977940828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f349c103b60 msgr2=0x7f349c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:32.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a183f640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f349c103b60 0x7f349c19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a183f640 1 -- 172.21.15.3:0/1977940828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349c1a04f0 con 0x7f349c1094a0 2024-09-21T06:26:32.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a2040640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f349c103b60 0x7f349c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:26:32.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.091+0000 7f34a2841640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f349c109df0 0x7f349c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:26:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.092+0000 7f34a183f640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349c1094a0 0x7f349c19ab20 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f348c006010 tx=0x7f348c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.092+0000 7f34937fe640 1 -- 172.21.15.3:0/1977940828 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c03f030 con 0x7f349c1094a0 2024-09-21T06:26:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.092+0000 7f34a42a4640 1 -- 172.21.15.3:0/1977940828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349c1a0780 con 0x7f349c1094a0 2024-09-21T06:26:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.092+0000 7f34937fe640 1 -- 172.21.15.3:0/1977940828 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f348c003aa0 con 0x7f349c1094a0 2024-09-21T06:26:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.092+0000 7f34937fe640 1 -- 172.21.15.3:0/1977940828 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c035070 con 0x7f349c1094a0 2024-09-21T06:26:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.092+0000 7f34a42a4640 1 -- 172.21.15.3:0/1977940828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f349c1a6f40 con 0x7f349c1094a0 2024-09-21T06:26:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.093+0000 7f34a42a4640 1 -- 172.21.15.3:0/1977940828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f349c104de0 con 0x7f349c1094a0 2024-09-21T06:26:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.094+0000 7f34937fe640 1 -- 172.21.15.3:0/1977940828 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7f348c042030 con 0x7f349c1094a0 2024-09-21T06:26:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.095+0000 7f34937fe640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f34880b6680 0x7f34880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.095+0000 7f34a2040640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f34880b6680 0x7f34880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.095+0000 7f34937fe640 1 -- 172.21.15.3:0/1977940828 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8893+0+0 (secure 0 0 0) 0x7f348c1081d0 con 0x7f349c1094a0 2024-09-21T06:26:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.096+0000 7f34a2040640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f34880b6680 0x7f34880b8b90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3498009c20 tx=0x7f3498005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.098+0000 7f34937fe640 1 -- 172.21.15.3:0/1977940828 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f348c0ce5b0 con 0x7f349c1094a0 2024-09-21T06:26:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.288+0000 7f34a42a4640 1 -- 172.21.15.3:0/1977940828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f349c10bbc0 con 0x7f349c1094a0 2024-09-21T06:26:32.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.448+0000 7f34937fe640 1 -- 172.21.15.3:0/1977940828 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f348c0d2990 con 0x7f349c1094a0 2024-09-21T06:26:32.451 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:26:32.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.452+0000 7f34a42a4640 1 -- 172.21.15.3:0/1977940828 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f34880b6680 msgr2=0x7f34880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:32.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.452+0000 7f34a42a4640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f34880b6680 0x7f34880b8b90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3498009c20 tx=0x7f3498005f90 comp rx=0 tx=0).stop 2024-09-21T06:26:32.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.452+0000 7f34a42a4640 1 -- 172.21.15.3:0/1977940828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349c1094a0 msgr2=0x7f349c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:32.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.452+0000 7f34a42a4640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349c1094a0 0x7f349c19ab20 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f348c006010 tx=0x7f348c004060 comp rx=0 tx=0).stop 2024-09-21T06:26:32.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.452+0000 7f34a42a4640 1 -- 172.21.15.3:0/1977940828 shutdown_connections 2024-09-21T06:26:32.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.452+0000 7f34a42a4640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7f34880b6680 0x7f34880b8b90 secure :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3498009c20 tx=0x7f3498005f90 comp rx=0 tx=0).stop 2024-09-21T06:26:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.452+0000 7f34a42a4640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f349c109df0 0x7f349c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.452+0000 7f34a42a4640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349c1094a0 0x7f349c19ab20 secure :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f348c006010 tx=0x7f348c004060 comp rx=0 tx=0).stop 2024-09-21T06:26:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.452+0000 7f34a42a4640 1 --2- 172.21.15.3:0/1977940828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f349c103b60 0x7f349c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.452+0000 7f34a42a4640 1 -- 172.21.15.3:0/1977940828 >> 172.21.15.3:0/1977940828 conn(0x7f349c07b850 msgr2=0x7f349c105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.453+0000 7f34a42a4640 1 -- 172.21.15.3:0/1977940828 shutdown_connections 2024-09-21T06:26:32.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:32.453+0000 7f34a42a4640 1 -- 172.21.15.3:0/1977940828 wait complete. 2024-09-21T06:26:32.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:32 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1977940828' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:26:32.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:32 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1977940828' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:26:32.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:32 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1977940828' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:26:33.196 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:26:33.460 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:33.706 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:33 smithi003 ceph-mon[31227]: pgmap v684: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-21T06:26:33.706 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:33 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/1977940828' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:26:33.706 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:33 smithi003 ceph-mon[31227]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T06:26:33.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:33 smithi137 ceph-mon[34464]: pgmap v684: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-21T06:26:33.995 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:33 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/1977940828' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:26:33.995 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:33 smithi137 ceph-mon[34464]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T06:26:33.996 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:33 smithi204 ceph-mon[34659]: pgmap v684: 97 pgs: 97 active+clean; 133 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-21T06:26:33.996 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:33 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/1977940828' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:26:33.996 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:33 smithi204 ceph-mon[34659]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T06:26:33.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.948+0000 7ff4a5678640 1 -- 172.21.15.3:0/1069335140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a0102a60 msgr2=0x7ff4a0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:33.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.948+0000 7ff4a5678640 1 --2- 172.21.15.3:0/1069335140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a0102a60 0x7ff4a0102e70 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff490009900 tx=0x7ff49002a410 comp rx=0 tx=0).stop 2024-09-21T06:26:33.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.948+0000 7ff4a5678640 1 -- 172.21.15.3:0/1069335140 shutdown_connections 2024-09-21T06:26:33.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.948+0000 7ff4a5678640 1 --2- 172.21.15.3:0/1069335140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a01083a0 0x7ff4a01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:33.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.948+0000 7ff4a5678640 1 --2- 172.21.15.3:0/1069335140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a0108cf0 0x7ff4a010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.948+0000 7ff4a5678640 1 --2- 172.21.15.3:0/1069335140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a0102a60 0x7ff4a0102e70 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.948+0000 7ff4a5678640 1 -- 172.21.15.3:0/1069335140 >> 172.21.15.3:0/1069335140 conn(0x7ff4a0079730 msgr2=0x7ff4a0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.948+0000 7ff4a5678640 1 -- 172.21.15.3:0/1069335140 shutdown_connections 2024-09-21T06:26:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.948+0000 7ff4a5678640 1 -- 172.21.15.3:0/1069335140 wait complete. 2024-09-21T06:26:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.948+0000 7ff4a5678640 1 Processor -- start 2024-09-21T06:26:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff4a5678640 1 -- start start 2024-09-21T06:26:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff4a5678640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a0102a60 0x7ff4a010fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff4a5678640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a01083a0 0x7ff4a01102f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:33.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff4a5678640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a0108cf0 0x7ff4a0110830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:33.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff4a5678640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a0113a70 con 0x7ff4a0108cf0 2024-09-21T06:26:33.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff4a5678640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a01138f0 con 0x7ff4a01083a0 2024-09-21T06:26:33.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff4a5678640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a0113bf0 con 0x7ff4a0102a60 2024-09-21T06:26:33.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff49f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a0108cf0 0x7ff4a0110830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:33.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff49f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a0108cf0 0x7ff4a0110830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43226/0 (socket says 172.21.15.3:43226) 2024-09-21T06:26:33.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff49f7fe640 1 -- 172.21.15.3:0/2563500772 learned_addr learned my addr 172.21.15.3:0/2563500772 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-21T06:26:33.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff49effd640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a0102a60 0x7ff4a010fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff497fff640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a01083a0 0x7ff4a01102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff49f7fe640 1 -- 172.21.15.3:0/2563500772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a0102a60 msgr2=0x7ff4a010fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:34.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff49f7fe640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a0102a60 0x7ff4a010fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff49f7fe640 1 -- 172.21.15.3:0/2563500772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a01083a0 msgr2=0x7ff4a01102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:34.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff49f7fe640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a01083a0 0x7ff4a01102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff49f7fe640 1 -- 172.21.15.3:0/2563500772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a01a8700 con 0x7ff4a0108cf0 2024-09-21T06:26:34.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff497fff640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a01083a0 0x7ff4a01102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:26:34.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.949+0000 7ff49f7fe640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a0108cf0 0x7ff4a0110830 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff49800eaa0 tx=0x7ff49800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:34.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.950+0000 7ff49cff9640 1 -- 172.21.15.3:0/2563500772 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49800cdb0 con 0x7ff4a0108cf0 2024-09-21T06:26:34.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.950+0000 7ff49cff9640 1 -- 172.21.15.3:0/2563500772 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff498005020 con 0x7ff4a0108cf0 2024-09-21T06:26:34.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.950+0000 7ff4a5678640 1 -- 172.21.15.3:0/2563500772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a01a89f0 con 0x7ff4a0108cf0 2024-09-21T06:26:34.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.950+0000 7ff49cff9640 1 -- 172.21.15.3:0/2563500772 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4980116d0 con 0x7ff4a0108cf0 2024-09-21T06:26:34.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.950+0000 7ff4a5678640 1 -- 172.21.15.3:0/2563500772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a01a8ee0 con 0x7ff4a0108cf0 2024-09-21T06:26:34.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.951+0000 7ff4a5678640 1 -- 172.21.15.3:0/2563500772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a0102e70 con 0x7ff4a0108cf0 2024-09-21T06:26:34.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.953+0000 7ff49cff9640 1 -- 172.21.15.3:0/2563500772 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 27) ==== 154642+0+0 (secure 0 0 0) 0x7ff4980051d0 con 0x7ff4a0108cf0 2024-09-21T06:26:34.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.954+0000 7ff49cff9640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff48c0b6680 0x7ff48c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:34.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.954+0000 7ff49effd640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff48c0b6680 0x7ff48c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.955+0000 7ff49cff9640 1 -- 172.21.15.3:0/2563500772 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8339+0+0 (secure 0 0 0) 0x7ff4980e7be0 con 0x7ff4a0108cf0 2024-09-21T06:26:34.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.955+0000 7ff49effd640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff48c0b6680 0x7ff48c0b8b90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff490000c00 tx=0x7ff490031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:34.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:33.958+0000 7ff49cff9640 1 -- 172.21.15.3:0/2563500772 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff498016030 con 0x7ff4a0108cf0 2024-09-21T06:26:34.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.147+0000 7ff4a5678640 1 -- 172.21.15.3:0/2563500772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff4a01110b0 con 0x7ff4a0108cf0 2024-09-21T06:26:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.456+0000 7ff49cff9640 1 -- 172.21.15.3:0/2563500772 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7ff4a01110b0 con 0x7ff4a0108cf0 2024-09-21T06:26:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-21T06:26:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.460+0000 7ff4a5678640 1 -- 172.21.15.3:0/2563500772 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff48c0b6680 msgr2=0x7ff48c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.460+0000 7ff4a5678640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff48c0b6680 0x7ff48c0b8b90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff490000c00 tx=0x7ff490031400 comp rx=0 tx=0).stop 2024-09-21T06:26:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.460+0000 7ff4a5678640 1 -- 172.21.15.3:0/2563500772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a0108cf0 msgr2=0x7ff4a0110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.460+0000 7ff4a5678640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a0108cf0 0x7ff4a0110830 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff49800eaa0 tx=0x7ff49800ef60 comp rx=0 tx=0).stop 2024-09-21T06:26:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.461+0000 7ff4a5678640 1 -- 172.21.15.3:0/2563500772 shutdown_connections 2024-09-21T06:26:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.461+0000 7ff4a5678640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.3:6800/2227168690,v1:172.21.15.3:6801/2227168690] conn(0x7ff48c0b6680 0x7ff48c0b8b90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.461+0000 7ff4a5678640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4a0108cf0 0x7ff4a0110830 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.461+0000 7ff4a5678640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a01083a0 0x7ff4a01102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.461+0000 7ff4a5678640 1 --2- 172.21.15.3:0/2563500772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a0102a60 0x7ff4a010fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.461+0000 7ff4a5678640 1 -- 172.21.15.3:0/2563500772 >> 172.21.15.3:0/2563500772 conn(0x7ff4a0079730 msgr2=0x7ff4a0101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:34.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.461+0000 7ff4a5678640 1 -- 172.21.15.3:0/2563500772 shutdown_connections 2024-09-21T06:26:34.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-21T06:26:34.461+0000 7ff4a5678640 1 -- 172.21.15.3:0/2563500772 wait complete. 2024-09-21T06:26:34.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:34 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2563500772' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:26:34.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:34 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2563500772' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:26:34.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:34 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2563500772' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:26:35.139 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:26:35.166 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:26:35.201 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:26:35.229 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T06:26:35.230 DEBUG:teuthology.orchestra.run.smithi003:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:26:35.245 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:26:35.261 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:26:35.285 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T06:26:35.285 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T06:26:35.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.a 2024-09-21T06:26:35.460 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:35 smithi003 systemd[1]: Stopping Ceph mon.a for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:26:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:35 smithi204 ceph-mon[34659]: pgmap v686: 65 pgs: 65 active+clean; 54 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:26:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:35 smithi204 ceph-mon[34659]: from='client.? 172.21.15.3:0/2563500772' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:26:35.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:35 smithi204 ceph-mon[34659]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T06:26:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:35 smithi003 ceph-mon[31227]: pgmap v686: 65 pgs: 65 active+clean; 54 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:26:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:35 smithi003 ceph-mon[31227]: from='client.? 172.21.15.3:0/2563500772' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:26:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:35 smithi003 ceph-mon[31227]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T06:26:35.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:35 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a[31223]: 2024-09-21T06:26:35.578+0000 7f41368a4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:26:35.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:35 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a[31223]: 2024-09-21T06:26:35.578+0000 7f41368a4640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T06:26:35.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:35 smithi137 ceph-mon[34464]: pgmap v686: 65 pgs: 65 active+clean; 54 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:26:35.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:35 smithi137 ceph-mon[34464]: from='client.? 172.21.15.3:0/2563500772' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:26:35.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:35 smithi137 ceph-mon[34464]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T06:26:36.140 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:35 smithi003 podman[109866]: 2024-09-21 06:26:35.85518029 +0000 UTC m=+0.379569748 container died 6ceb763afa3c479746e0c332d336fa89a4ab399f9c9303235dfcf01e622ac614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T06:26:36.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:36 smithi003 podman[109866]: 2024-09-21 06:26:36.237217619 +0000 UTC m=+0.761607071 container cleanup 6ceb763afa3c479746e0c332d336fa89a4ab399f9c9303235dfcf01e622ac614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T06:26:36.590 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 21 06:26:36 smithi003 bash[109866]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-a 2024-09-21T06:26:36.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T06:26:36.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:26:36.913 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T06:26:36.913 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T06:26:36.913 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.b 2024-09-21T06:26:37.378 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:36 smithi137 systemd[1]: Stopping Ceph mon.b for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:26:37.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:37 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-b[34460]: 2024-09-21T06:26:37.376+0000 7f86bf7cf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:26:37.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:37 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-b[34460]: 2024-09-21T06:26:37.376+0000 7f86bf7cf640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T06:26:37.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:37 smithi137 podman[67066]: 2024-09-21 06:26:37.417476465 +0000 UTC m=+0.257992097 container died cc17861276f7c89b1bfe9c0a632db4f73036dd75c0a296c5d75f0a8b238246a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:26:38.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:37 smithi137 podman[67066]: 2024-09-21 06:26:37.927425375 +0000 UTC m=+0.767941002 container cleanup cc17861276f7c89b1bfe9c0a632db4f73036dd75c0a296c5d75f0a8b238246a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-b, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T06:26:38.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 21 06:26:37 smithi137 bash[67066]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-b 2024-09-21T06:26:38.377 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T06:26:38.425 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:26:38.426 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T06:26:38.426 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T06:26:38.426 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.c 2024-09-21T06:26:38.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:38 smithi204 systemd[1]: Stopping Ceph mon.c for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:26:39.116 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:38 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-c[34648]: 2024-09-21T06:26:38.755+0000 7f3706cec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:26:39.116 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:38 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-c[34648]: 2024-09-21T06:26:38.755+0000 7f3706cec640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T06:26:39.116 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:38 smithi204 podman[66989]: 2024-09-21 06:26:38.807184493 +0000 UTC m=+0.180417577 container died ee491905d6881487a7f21646a380aafcee8e61ed2edbc898b1101b9a0357a9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-c, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T06:26:39.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:39 smithi204 podman[66989]: 2024-09-21 06:26:39.231269923 +0000 UTC m=+0.604503005 container cleanup ee491905d6881487a7f21646a380aafcee8e61ed2edbc898b1101b9a0357a9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T06:26:39.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:39 smithi204 bash[66989]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-c 2024-09-21T06:26:39.710 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T06:26:39.732 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:39 smithi204 podman[67001]: 2024-09-21 06:26:39.466394785 +0000 UTC m=+0.658080921 container remove ee491905d6881487a7f21646a380aafcee8e61ed2edbc898b1101b9a0357a9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mon-c, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T06:26:39.733 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:39 smithi204 systemd[1]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.c.service: Deactivated successfully. 2024-09-21T06:26:39.733 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:39 smithi204 systemd[1]: Stopped Ceph mon.c for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:26:39.733 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 21 06:26:39 smithi204 systemd[1]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mon.c.service: Consumed 14.023s CPU time. 2024-09-21T06:26:39.757 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:26:39.757 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T06:26:39.757 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T06:26:39.758 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.x 2024-09-21T06:26:40.069 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:26:39 smithi003 systemd[1]: Stopping Ceph mgr.x for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:26:40.340 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:26:40 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x[31458]: 2024-09-21T06:26:40.066+0000 7fc572b55640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:26:40.340 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:26:40 smithi003 podman[109987]: 2024-09-21 06:26:40.108291718 +0000 UTC m=+0.153050327 container died 46bb2d2a398fad101e868f7080948ec841c9eefb9b0c50466bc06f13e0b4867a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T06:26:40.763 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:26:40 smithi003 podman[109987]: 2024-09-21 06:26:40.518140261 +0000 UTC m=+0.562898870 container cleanup 46bb2d2a398fad101e868f7080948ec841c9eefb9b0c50466bc06f13e0b4867a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T06:26:40.763 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:26:40 smithi003 bash[109987]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x 2024-09-21T06:26:41.014 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T06:26:41.037 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:26:40 smithi003 podman[110000]: 2024-09-21 06:26:40.761810324 +0000 UTC m=+0.656407142 container remove 46bb2d2a398fad101e868f7080948ec841c9eefb9b0c50466bc06f13e0b4867a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T06:26:41.037 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:26:41 smithi003 systemd[1]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.x.service: Deactivated successfully. 2024-09-21T06:26:41.037 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:26:41 smithi003 systemd[1]: Stopped Ceph mgr.x for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:26:41.037 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 21 06:26:41 smithi003 systemd[1]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.x.service: Consumed 56.886s CPU time. 2024-09-21T06:26:41.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:26:41.051 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T06:26:41.051 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T06:26:41.051 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.y 2024-09-21T06:26:41.395 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 21 06:26:41 smithi137 systemd[1]: Stopping Ceph mgr.y for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:26:41.760 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 21 06:26:41 smithi137 podman[67188]: 2024-09-21 06:26:41.411645867 +0000 UTC m=+0.158174926 container stop f23cce124baf2232f20577c596f5ac28908c9d2dc26d71669aa57e8dad6f53fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-y, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T06:26:41.760 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 21 06:26:41 smithi137 podman[67188]: 2024-09-21 06:26:41.437446156 +0000 UTC m=+0.183975226 container died f23cce124baf2232f20577c596f5ac28908c9d2dc26d71669aa57e8dad6f53fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T06:26:42.145 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 21 06:26:41 smithi137 podman[67188]: 2024-09-21 06:26:41.958584746 +0000 UTC m=+0.705113818 container cleanup f23cce124baf2232f20577c596f5ac28908c9d2dc26d71669aa57e8dad6f53fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-y, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T06:26:42.145 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 21 06:26:41 smithi137 bash[67188]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-y 2024-09-21T06:26:42.388 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T06:26:42.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:26:42.423 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T06:26:42.423 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T06:26:42.424 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.z 2024-09-21T06:26:42.767 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 21 06:26:42 smithi204 systemd[1]: Stopping Ceph mgr.z for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:26:43.072 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 21 06:26:42 smithi204 podman[67108]: 2024-09-21 06:26:42.766257117 +0000 UTC m=+0.142218054 container died 42ac69c4e6c161929373714e465800f7daa94918803f9205d999f98ec8a72477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T06:26:43.404 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 21 06:26:43 smithi204 podman[67108]: 2024-09-21 06:26:43.170787912 +0000 UTC m=+0.546748840 container cleanup 42ac69c4e6c161929373714e465800f7daa94918803f9205d999f98ec8a72477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-z, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T06:26:43.404 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 21 06:26:43 smithi204 bash[67108]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mgr-z 2024-09-21T06:26:43.599 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T06:26:43.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:26:43.635 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T06:26:43.635 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T06:26:43.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.0 2024-09-21T06:26:43.949 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:43 smithi003 systemd[1]: Stopping Ceph osd.0 for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:26:44.340 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:43 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0[48021]: 2024-09-21T06:26:43.947+0000 7f60cfdbf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:26:44.340 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:43 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0[48021]: 2024-09-21T06:26:43.947+0000 7f60cfdbf640 -1 osd.0 94 *** Got signal Terminated *** 2024-09-21T06:26:44.340 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:43 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0[48021]: 2024-09-21T06:26:43.947+0000 7f60cfdbf640 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:26:49.749 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:49 smithi003 podman[110109]: 2024-09-21 06:26:49.428031661 +0000 UTC m=+5.589992293 container died cf8ad698682fbc5745e7b1a78169d4d7d9a4b976874effa3b74f2515c1644a6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:26:50.059 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:49 smithi003 podman[110109]: 2024-09-21 06:26:49.848086917 +0000 UTC m=+6.010047535 container cleanup cf8ad698682fbc5745e7b1a78169d4d7d9a4b976874effa3b74f2515c1644a6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T06:26:50.059 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:49 smithi003 bash[110109]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0 2024-09-21T06:26:50.340 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:50 smithi003 podman[110126]: 2024-09-21 06:26:50.057957953 +0000 UTC m=+0.626515395 container remove cf8ad698682fbc5745e7b1a78169d4d7d9a4b976874effa3b74f2515c1644a6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T06:26:50.602 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:50 smithi003 podman[110214]: 2024-09-21 06:26:50.384333093 +0000 UTC m=+0.017321134 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:26:50.602 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:50 smithi003 podman[110214]: 2024-09-21 06:26:50.492036907 +0000 UTC m=+0.125024925 container create b15821f39f1b6720ffd56b6486fe862730a5303e471fe93b81fb3d29d4182e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T06:26:50.934 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:50 smithi003 podman[110214]: 2024-09-21 06:26:50.736393819 +0000 UTC m=+0.369381835 container init b15821f39f1b6720ffd56b6486fe862730a5303e471fe93b81fb3d29d4182e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:26:50.934 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:50 smithi003 podman[110214]: 2024-09-21 06:26:50.739814842 +0000 UTC m=+0.372802858 container start b15821f39f1b6720ffd56b6486fe862730a5303e471fe93b81fb3d29d4182e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T06:26:50.935 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:50 smithi003 podman[110214]: 2024-09-21 06:26:50.80292407 +0000 UTC m=+0.435912093 container attach b15821f39f1b6720ffd56b6486fe862730a5303e471fe93b81fb3d29d4182e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T06:26:51.256 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 21 06:26:50 smithi003 podman[110214]: 2024-09-21 06:26:50.933289373 +0000 UTC m=+0.566277400 container died b15821f39f1b6720ffd56b6486fe862730a5303e471fe93b81fb3d29d4182e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:26:51.595 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T06:26:51.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:26:51.638 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T06:26:51.639 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T06:26:51.639 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.1 2024-09-21T06:26:51.964 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:51 smithi137 systemd[1]: Stopping Ceph osd.1 for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:26:52.395 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:51 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1[39181]: 2024-09-21T06:26:51.962+0000 7f5d598d1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:26:52.395 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:51 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1[39181]: 2024-09-21T06:26:51.962+0000 7f5d598d1640 -1 osd.1 94 *** Got signal Terminated *** 2024-09-21T06:26:52.395 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:51 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1[39181]: 2024-09-21T06:26:51.962+0000 7f5d598d1640 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:26:57.645 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:57 smithi137 podman[67324]: 2024-09-21 06:26:57.340965996 +0000 UTC m=+5.511798401 container died c859ed03c62ad37167902eadd2422e08cdde684bb79a72f808beebe285d23553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T06:26:57.991 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:57 smithi137 podman[67324]: 2024-09-21 06:26:57.788327645 +0000 UTC m=+5.959160053 container cleanup c859ed03c62ad37167902eadd2422e08cdde684bb79a72f808beebe285d23553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T06:26:57.991 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:57 smithi137 bash[67324]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1 2024-09-21T06:26:58.395 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:57 smithi137 podman[67341]: 2024-09-21 06:26:57.989996144 +0000 UTC m=+0.643443796 container remove c859ed03c62ad37167902eadd2422e08cdde684bb79a72f808beebe285d23553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T06:26:58.687 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:58 smithi137 podman[67429]: 2024-09-21 06:26:58.315541216 +0000 UTC m=+0.016047217 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:26:58.687 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:58 smithi137 podman[67429]: 2024-09-21 06:26:58.443382964 +0000 UTC m=+0.143888955 container create c1234ac5d848aa7739604bf0c363693c2afd048e2d59e336eabdc688cd86b108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:26:58.687 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:58 smithi137 podman[67429]: 2024-09-21 06:26:58.685745869 +0000 UTC m=+0.386251880 container init c1234ac5d848aa7739604bf0c363693c2afd048e2d59e336eabdc688cd86b108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-21T06:26:59.145 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:58 smithi137 podman[67429]: 2024-09-21 06:26:58.68941584 +0000 UTC m=+0.389921839 container start c1234ac5d848aa7739604bf0c363693c2afd048e2d59e336eabdc688cd86b108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T06:26:59.146 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:58 smithi137 podman[67429]: 2024-09-21 06:26:58.802241553 +0000 UTC m=+0.502747560 container attach c1234ac5d848aa7739604bf0c363693c2afd048e2d59e336eabdc688cd86b108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:26:59.146 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 21 06:26:58 smithi137 podman[67429]: 2024-09-21 06:26:58.932793973 +0000 UTC m=+0.633299975 container died c1234ac5d848aa7739604bf0c363693c2afd048e2d59e336eabdc688cd86b108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T06:26:59.643 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T06:26:59.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:26:59.687 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T06:26:59.687 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T06:26:59.687 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.2 2024-09-21T06:27:00.002 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:26:59 smithi204 systemd[1]: Stopping Ceph osd.2 for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:27:00.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:27:00 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2[39755]: 2024-09-21T06:26:59.999+0000 7f1743afc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:27:00.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:27:00 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2[39755]: 2024-09-21T06:26:59.999+0000 7f1743afc640 -1 osd.2 94 *** Got signal Terminated *** 2024-09-21T06:27:00.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:27:00 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2[39755]: 2024-09-21T06:26:59.999+0000 7f1743afc640 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:27:05.878 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:27:05 smithi204 podman[67248]: 2024-09-21 06:27:05.471139405 +0000 UTC m=+5.590115181 container died 44dacea9245752e881635a62052e7f9f221c0b8432f498f0960a1ca3757b1350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T06:27:06.154 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:27:05 smithi204 podman[67248]: 2024-09-21 06:27:05.976064092 +0000 UTC m=+6.095039861 container cleanup 44dacea9245752e881635a62052e7f9f221c0b8432f498f0960a1ca3757b1350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T06:27:06.154 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:27:05 smithi204 bash[67248]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2 2024-09-21T06:27:06.573 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:27:06 smithi204 podman[67264]: 2024-09-21 06:27:06.260969824 +0000 UTC m=+0.791913865 container remove 44dacea9245752e881635a62052e7f9f221c0b8432f498f0960a1ca3757b1350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:27:06.891 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:27:06 smithi204 podman[67354]: 2024-09-21 06:27:06.475056159 +0000 UTC m=+0.016004514 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:27:06.891 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:27:06 smithi204 podman[67354]: 2024-09-21 06:27:06.612157869 +0000 UTC m=+0.153106254 container create 9c7e3ac7336d467105595a97cfd5dabb8c66fadaace8678a4109fb5f6940aa07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:27:07.154 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:27:06 smithi204 podman[67354]: 2024-09-21 06:27:06.890249436 +0000 UTC m=+0.431197773 container init 9c7e3ac7336d467105595a97cfd5dabb8c66fadaace8678a4109fb5f6940aa07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T06:27:07.155 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:27:06 smithi204 podman[67354]: 2024-09-21 06:27:06.893882527 +0000 UTC m=+0.434830863 container start 9c7e3ac7336d467105595a97cfd5dabb8c66fadaace8678a4109fb5f6940aa07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T06:27:07.155 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:27:06 smithi204 podman[67354]: 2024-09-21 06:27:06.998824775 +0000 UTC m=+0.539773127 container attach 9c7e3ac7336d467105595a97cfd5dabb8c66fadaace8678a4109fb5f6940aa07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-09-21T06:27:07.155 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 21 06:27:07 smithi204 podman[67354]: 2024-09-21 06:27:07.094334193 +0000 UTC m=+0.635282539 container died 9c7e3ac7336d467105595a97cfd5dabb8c66fadaace8678a4109fb5f6940aa07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T06:27:07.912 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T06:27:07.952 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:27:07.953 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T06:27:07.953 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T06:27:07.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.3 2024-09-21T06:27:08.261 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:07 smithi003 systemd[1]: Stopping Ceph osd.3 for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:27:08.590 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:08 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3[54593]: 2024-09-21T06:27:08.258+0000 7f6eb4c75640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:27:08.590 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:08 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3[54593]: 2024-09-21T06:27:08.258+0000 7f6eb4c75640 -1 osd.3 94 *** Got signal Terminated *** 2024-09-21T06:27:08.590 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:08 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3[54593]: 2024-09-21T06:27:08.258+0000 7f6eb4c75640 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:27:10.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:10 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:10.349+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:11.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:10 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:10.664+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:11.840 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:11 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3[54593]: 2024-09-21T06:27:11.346+0000 7f6ead258640 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:43.128436+0000 front 2024-09-21T06:26:43.128470+0000 (oldest deadline 2024-09-21T06:27:11.329166+0000) 2024-09-21T06:27:11.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:11 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:11.392+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:11.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:11 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:11.645+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:12.840 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:12 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3[54593]: 2024-09-21T06:27:12.350+0000 7f6ead258640 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:43.128436+0000 front 2024-09-21T06:26:43.128470+0000 (oldest deadline 2024-09-21T06:27:11.329166+0000) 2024-09-21T06:27:12.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:12 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:12.387+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:12.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:12 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:12.621+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:13.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:12 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:12.942+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:13.590 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:13 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:13.426+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:13.590 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:13 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:13.265+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:13.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:13 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:13.630+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:13.969 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:13 smithi003 podman[110317]: 2024-09-21 06:27:13.71368218 +0000 UTC m=+5.564117297 container died 6a67cbf73f842557cc243c4aa390297485b2076926323d51f18d2a79198ecea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:27:14.232 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:14 smithi003 podman[110317]: 2024-09-21 06:27:14.159556351 +0000 UTC m=+6.009991539 container cleanup 6a67cbf73f842557cc243c4aa390297485b2076926323d51f18d2a79198ecea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:27:14.233 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:14 smithi003 bash[110317]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3 2024-09-21T06:27:14.407 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:13 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:13.960+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:14.590 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:14 smithi003 podman[110334]: 2024-09-21 06:27:14.444726009 +0000 UTC m=+0.730984539 container remove 6a67cbf73f842557cc243c4aa390297485b2076926323d51f18d2a79198ecea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T06:27:14.590 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:14 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:14.230+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:14.591 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:14 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:14.460+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:14.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:14 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:14.169+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:15.006 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:14 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:14.674+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:15.057 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:14 smithi003 podman[110420]: 2024-09-21 06:27:14.787109791 +0000 UTC m=+0.111472639 container create 02fc579ddcbc11d3261cd20376a2e28e986cb96533a260ad1769b38629a4a43c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:27:15.057 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:14 smithi003 podman[110420]: 2024-09-21 06:27:14.692106085 +0000 UTC m=+0.016468900 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:27:15.057 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:15 smithi003 podman[110420]: 2024-09-21 06:27:15.056144216 +0000 UTC m=+0.380507026 container init 02fc579ddcbc11d3261cd20376a2e28e986cb96533a260ad1769b38629a4a43c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:27:15.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:15 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:15.203+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:15.340 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:15 smithi003 podman[110420]: 2024-09-21 06:27:15.059594066 +0000 UTC m=+0.383956877 container start 02fc579ddcbc11d3261cd20376a2e28e986cb96533a260ad1769b38629a4a43c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T06:27:15.341 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:15 smithi003 podman[110420]: 2024-09-21 06:27:15.156135448 +0000 UTC m=+0.480498269 container attach 02fc579ddcbc11d3261cd20376a2e28e986cb96533a260ad1769b38629a4a43c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-21T06:27:15.341 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 21 06:27:15 smithi003 podman[110420]: 2024-09-21 06:27:15.257105058 +0000 UTC m=+0.581467869 container died 02fc579ddcbc11d3261cd20376a2e28e986cb96533a260ad1769b38629a4a43c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:27:15.395 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:15 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:15.122+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:15.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:15 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:15.005+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:15.810 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:15 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:15.441+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:15.834 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T06:27:15.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:27:15.874 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T06:27:15.875 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T06:27:15.875 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.4 2024-09-21T06:27:16.043 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:15 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:15.695+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:16.395 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:16 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:16.148+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:16.395 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:15 smithi137 systemd[1]: Stopping Ceph osd.4 for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:27:16.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:16 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:16.041+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:16.431 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:16 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:16.157+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:16.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:16 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:16.429+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:16.895 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:16 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4[45318]: 2024-09-21T06:27:16.525+0000 7fef01bdd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:27:16.895 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:16 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4[45318]: 2024-09-21T06:27:16.525+0000 7fef01bdd640 -1 osd.4 94 *** Got signal Terminated *** 2024-09-21T06:27:16.895 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:16 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4[45318]: 2024-09-21T06:27:16.525+0000 7fef01bdd640 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:27:17.050 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:16 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:16.707+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:17.395 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:17 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:17.108+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:17.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:17 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:17.049+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:17.590 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:17 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:17.150+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:17.590 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:17 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:17.400+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:18.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:17 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:17.667+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:18.395 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:18 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:18.102+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:18.395 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:18 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:18.102+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:18.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:18 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:18.045+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:18.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:18 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:18.045+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:52.718100+0000 front 2024-09-21T06:26:52.718039+0000 (oldest deadline 2024-09-21T06:27:18.018036+0000) 2024-09-21T06:27:18.426 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:18 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:18.147+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:18.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:18 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:18.425+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:18.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:18 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:18.621+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:19.145 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:18 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4[45318]: 2024-09-21T06:27:18.813+0000 7feefe1c8640 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:53.926347+0000 front 2024-09-21T06:26:53.926373+0000 (oldest deadline 2024-09-21T06:27:18.026497+0000) 2024-09-21T06:27:19.145 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:19 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:19.061+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:19.145 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:19 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:19.061+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:19.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:19 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:19.074+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:19.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:19 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:19.074+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:52.718100+0000 front 2024-09-21T06:26:52.718039+0000 (oldest deadline 2024-09-21T06:27:18.018036+0000) 2024-09-21T06:27:19.445 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:19 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:19.098+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:19.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:19 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:19.444+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:20.041 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:19 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:19.652+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:20.146 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:19 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4[45318]: 2024-09-21T06:27:19.857+0000 7feefe1c8640 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:53.926347+0000 front 2024-09-21T06:26:53.926373+0000 (oldest deadline 2024-09-21T06:27:18.026497+0000) 2024-09-21T06:27:20.146 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:20 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:20.059+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:20.146 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:20 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:20.092+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:20.147 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:20 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:20.092+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:20.317 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:20 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:20.040+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:20.318 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:20 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:20.040+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:52.718100+0000 front 2024-09-21T06:26:52.718039+0000 (oldest deadline 2024-09-21T06:27:18.018036+0000) 2024-09-21T06:27:20.476 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:20 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:20.137+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:20.476 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:20 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:20.137+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:20.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:20 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:20.474+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:20.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:20 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:20.474+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:21.061 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:20 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:20.690+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:21.062 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:20 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:20.690+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:21.142 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:20 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4[45318]: 2024-09-21T06:27:20.854+0000 7feefe1c8640 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:53.926347+0000 front 2024-09-21T06:26:53.926373+0000 (oldest deadline 2024-09-21T06:27:18.026497+0000) 2024-09-21T06:27:21.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:21 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:21.067+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:21.324 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:21 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:21.059+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:21.324 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:21 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:21.059+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:52.718100+0000 front 2024-09-21T06:26:52.718039+0000 (oldest deadline 2024-09-21T06:27:18.018036+0000) 2024-09-21T06:27:21.395 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:21 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:21.140+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:21.395 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:21 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:21.140+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:21.478 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:21 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:21.128+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:21.478 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:21 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:21.128+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:21.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:21 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:21.322+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:21.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:21 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:21.477+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:21.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:21 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:21.477+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:22.060 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:21 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:21.661+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:22.061 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:21 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:21.661+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:22.272 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:22 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:22.034+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:22.272 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:21 smithi137 podman[67531]: 2024-09-21 06:27:21.953694738 +0000 UTC m=+5.878059126 container died 33a468151e932113a455f7b19ab20efcdaaf27f2ed7092fe900d1b9983dd4e11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T06:27:22.272 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:22 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:22.178+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:22.273 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:22 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:22.178+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:22.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:22 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:22.287+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:22.406 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:22 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:22.059+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:22.406 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:22 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:22.059+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:52.718100+0000 front 2024-09-21T06:26:52.718039+0000 (oldest deadline 2024-09-21T06:27:18.018036+0000) 2024-09-21T06:27:22.509 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:22 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:22.125+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:22.510 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:22 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:22.125+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:22.579 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:22 smithi137 podman[67531]: 2024-09-21 06:27:22.384497184 +0000 UTC m=+6.308861563 container cleanup 33a468151e932113a455f7b19ab20efcdaaf27f2ed7092fe900d1b9983dd4e11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T06:27:22.579 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:22 smithi137 bash[67531]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4 2024-09-21T06:27:22.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:22 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:22.508+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:22.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:22 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:22.508+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:22.895 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:22 smithi137 podman[67557]: 2024-09-21 06:27:22.577412018 +0000 UTC m=+0.618406211 container remove 33a468151e932113a455f7b19ab20efcdaaf27f2ed7092fe900d1b9983dd4e11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:27:22.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:22 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:22.631+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:22.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:22 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:22.631+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:23.282 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:22 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:22.993+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:23.283 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:23 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:23.139+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:23.283 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:23 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:23.139+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:23.283 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:22 smithi137 podman[67650]: 2024-09-21 06:27:22.895548973 +0000 UTC m=+0.015722222 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:27:23.284 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:23 smithi137 podman[67650]: 2024-09-21 06:27:23.020191417 +0000 UTC m=+0.140364699 container create 0f708700d607794820f9545534c973544b31404a332bf7267ba6d57e583f7fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T06:27:23.284 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:23 smithi137 podman[67650]: 2024-09-21 06:27:23.281478742 +0000 UTC m=+0.401651988 container init 0f708700d607794820f9545534c973544b31404a332bf7267ba6d57e583f7fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:27:23.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:23 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:23.253+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:23.405 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:23 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:23.103+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:23.405 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:23 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:23.103+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:52.718100+0000 front 2024-09-21T06:26:52.718039+0000 (oldest deadline 2024-09-21T06:27:18.018036+0000) 2024-09-21T06:27:23.506 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:23 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:23.136+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:23.506 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:23 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:23.136+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:23.645 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:23 smithi137 podman[67650]: 2024-09-21 06:27:23.285155086 +0000 UTC m=+0.405328334 container start 0f708700d607794820f9545534c973544b31404a332bf7267ba6d57e583f7fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T06:27:23.645 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:23 smithi137 podman[67650]: 2024-09-21 06:27:23.364305435 +0000 UTC m=+0.484478688 container attach 0f708700d607794820f9545534c973544b31404a332bf7267ba6d57e583f7fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:27:23.645 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 21 06:27:23 smithi137 podman[67650]: 2024-09-21 06:27:23.474992296 +0000 UTC m=+0.595165536 container died 0f708700d607794820f9545534c973544b31404a332bf7267ba6d57e583f7fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:27:23.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:23 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:23.505+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:23.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:23 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:23.505+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:23.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:23 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:23.586+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:23.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:23 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:23.586+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:24.077 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:23 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:23.957+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:24.095 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T06:27:24.136 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:27:24.137 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T06:27:24.137 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T06:27:24.137 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.5 2024-09-21T06:27:24.395 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:24 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:24.157+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:24.395 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:24 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:24.157+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:24.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:24 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:24.206+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:24.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:24 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:24.115+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:24.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:24 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:24.115+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:52.718100+0000 front 2024-09-21T06:26:52.718039+0000 (oldest deadline 2024-09-21T06:27:18.018036+0000) 2024-09-21T06:27:24.405 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:24 smithi204 systemd[1]: Stopping Ceph osd.5 for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:27:24.471 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:24.136+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:24.471 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:24.136+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:24.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:24.469+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:24.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:24 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:24.469+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:24.904 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:24 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:24.449+0000 7f86ff97e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:27:24.904 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:24 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:24.449+0000 7f86ff97e640 -1 osd.5 94 *** Got signal Terminated *** 2024-09-21T06:27:24.904 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:24 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:24.449+0000 7f86ff97e640 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:27:24.905 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:24 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:24.611+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:24.905 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:24 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:24.611+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:25.395 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:25 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:25.000+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:25.395 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:25 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:25.156+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:25.395 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:25 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:25.156+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:25.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:25 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:25.216+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:25.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:25 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:25.075+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:25.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:25 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:25.075+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:52.718100+0000 front 2024-09-21T06:26:52.718039+0000 (oldest deadline 2024-09-21T06:27:18.018036+0000) 2024-09-21T06:27:25.430 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:25.095+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:25.430 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:25.095+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:25.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:25.428+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:25.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:25.428+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:25.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:25 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:25.639+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:25.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:25 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:25.639+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:26.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:26 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:26.050+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:26.341 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:26 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:26.050+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:26.341 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:26 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:26.050+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.286277+0000 front 2024-09-21T06:27:03.286188+0000 (oldest deadline 2024-09-21T06:27:25.586366+0000) 2024-09-21T06:27:26.395 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:25 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:25.962+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:26.395 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:26 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:26.120+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:26.395 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:26 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:26.121+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:26.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:26 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:26.167+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:26.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:26 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:26.058+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:26.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:26 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:26.058+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:52.718100+0000 front 2024-09-21T06:26:52.718039+0000 (oldest deadline 2024-09-21T06:27:18.018036+0000) 2024-09-21T06:27:26.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:26 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:26.391+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:26.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:26 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:26.391+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:26.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:26 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:26.391+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:27.072 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:26 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:26.658+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:27.073 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:26 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:26.658+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:27.173 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:26 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:26.920+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:27.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:27 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:27.067+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:27.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:27 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:27.067+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:27.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:27 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:27.067+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.286277+0000 front 2024-09-21T06:27:03.286188+0000 (oldest deadline 2024-09-21T06:27:25.586366+0000) 2024-09-21T06:27:27.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:27 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:27.189+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:27.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:27 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:27.070+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:27.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:27 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:27.070+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:52.718100+0000 front 2024-09-21T06:26:52.718039+0000 (oldest deadline 2024-09-21T06:27:18.018036+0000) 2024-09-21T06:27:27.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:27 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:27.170+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:27.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:27 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:27.170+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:27.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:27 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:27.170+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:27.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:27 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:27.362+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:27.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:27 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:27.362+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:27.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:27 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:27.362+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:28.101 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:27 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:27.684+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:28.101 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:27 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:27.684+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:28.220 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:27 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:27.908+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:28.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:28 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:28.053+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:28.341 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:28 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:28.053+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:28.342 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:28 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:28.053+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.286277+0000 front 2024-09-21T06:27:03.286188+0000 (oldest deadline 2024-09-21T06:27:25.586366+0000) 2024-09-21T06:27:28.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:28.187+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:28.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:28.187+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:28.405 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:28.099+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:28.405 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:28.099+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:52.718100+0000 front 2024-09-21T06:26:52.718039+0000 (oldest deadline 2024-09-21T06:27:18.018036+0000) 2024-09-21T06:27:28.405 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:28.099+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.220002+0000 front 2024-09-21T06:27:03.220036+0000 (oldest deadline 2024-09-21T06:27:27.320140+0000) 2024-09-21T06:27:28.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:28.218+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:28.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:28.218+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:28.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:28.218+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:28.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:28 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:28.356+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:28.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:28 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:28.356+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:28.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:28 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:28.356+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:28.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:28.645+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:28.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:28 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:28.645+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:29.242 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:28.939+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:29.242 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:28 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:28.939+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:29.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:29 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:29.043+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:29.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:29 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:29.043+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:29.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:29 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:29.043+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.286277+0000 front 2024-09-21T06:27:03.286188+0000 (oldest deadline 2024-09-21T06:27:25.586366+0000) 2024-09-21T06:27:29.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:29 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:29.144+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:29.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:29 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:29.144+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:29.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:29 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:29.059+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:45.716523+0000 front 2024-09-21T06:26:45.716637+0000 (oldest deadline 2024-09-21T06:27:12.717318+0000) 2024-09-21T06:27:29.405 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:29 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:29.059+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:52.718100+0000 front 2024-09-21T06:26:52.718039+0000 (oldest deadline 2024-09-21T06:27:18.018036+0000) 2024-09-21T06:27:29.405 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:29 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5[45925]: 2024-09-21T06:27:29.059+0000 7f86fbf69640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.220002+0000 front 2024-09-21T06:27:03.220036+0000 (oldest deadline 2024-09-21T06:27:27.320140+0000) 2024-09-21T06:27:29.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:29 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:29.240+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:29.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:29 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:29.240+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:29.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:29 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:29.240+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:29.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:29 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:29.401+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:29.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:29 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:29.401+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:29.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:29 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:29.401+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:29.973 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:29 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:29.669+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:29.974 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:29 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:29.669+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:30.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:29 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:29.891+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:30.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:29 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:29.891+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:30.287 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:29 smithi204 podman[67479]: 2024-09-21 06:27:29.971184587 +0000 UTC m=+5.640100978 container died 9b33dcb743cfcffc89db78637d7c08928b00432327a541518ea1d0da05656f41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T06:27:30.287 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:30 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:30.128+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:30.287 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:30 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:30.128+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:30.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:30 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:30.019+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:30.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:30 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:30.019+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:30.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:30 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:30.019+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.286277+0000 front 2024-09-21T06:27:03.286188+0000 (oldest deadline 2024-09-21T06:27:25.586366+0000) 2024-09-21T06:27:30.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:30 smithi204 podman[67479]: 2024-09-21 06:27:30.383933802 +0000 UTC m=+6.052850192 container cleanup 9b33dcb743cfcffc89db78637d7c08928b00432327a541518ea1d0da05656f41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T06:27:30.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:30 smithi204 bash[67479]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5 2024-09-21T06:27:30.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:30 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:30.248+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:30.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:30 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:30.248+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:30.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:30 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:30.248+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:30.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:30 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:30.354+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:30.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:30 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:30.354+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:30.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:30 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:30.354+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:30.904 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:30 smithi204 podman[67494]: 2024-09-21 06:27:30.668475954 +0000 UTC m=+0.685840026 container remove 9b33dcb743cfcffc89db78637d7c08928b00432327a541518ea1d0da05656f41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T06:27:30.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:30 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:30.642+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:30.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:30 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:30.642+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:30.905 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:30 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:30.642+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:31.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:30 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:30.897+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:31.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:30 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:30.897+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:31.273 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:30 smithi204 podman[67585]: 2024-09-21 06:27:30.883910813 +0000 UTC m=+0.015896013 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:27:31.273 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:31 smithi204 podman[67585]: 2024-09-21 06:27:31.019629676 +0000 UTC m=+0.151614865 container create 260f2b384e100bcbb38432f8f8eb4cbc9b21ef38a2db7521fa23ab89e5b696e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T06:27:31.274 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:31 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:31.093+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:31.274 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:31 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:31.093+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:31.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:30 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:30.981+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:31.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:30 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:30.981+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:31.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:30 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:30.981+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.286277+0000 front 2024-09-21T06:27:03.286188+0000 (oldest deadline 2024-09-21T06:27:25.586366+0000) 2024-09-21T06:27:31.626 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:31 smithi204 podman[67585]: 2024-09-21 06:27:31.272367188 +0000 UTC m=+0.404352378 container init 260f2b384e100bcbb38432f8f8eb4cbc9b21ef38a2db7521fa23ab89e5b696e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T06:27:31.626 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:31 smithi204 podman[67585]: 2024-09-21 06:27:31.275812222 +0000 UTC m=+0.407797422 container start 260f2b384e100bcbb38432f8f8eb4cbc9b21ef38a2db7521fa23ab89e5b696e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:27:31.626 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:31 smithi204 podman[67585]: 2024-09-21 06:27:31.380934212 +0000 UTC m=+0.512919402 container attach 260f2b384e100bcbb38432f8f8eb4cbc9b21ef38a2db7521fa23ab89e5b696e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T06:27:31.626 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 21 06:27:31 smithi204 podman[67585]: 2024-09-21 06:27:31.472148755 +0000 UTC m=+0.604133955 container died 260f2b384e100bcbb38432f8f8eb4cbc9b21ef38a2db7521fa23ab89e5b696e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-5-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T06:27:31.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:31 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:31.295+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:31.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:31 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:31.295+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:31.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:31 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:31.295+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:31.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:31 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:31.356+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:31.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:31 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:31.356+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:31.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:31 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:31.356+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:31.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:31 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:31.624+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:31.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:31 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:31.624+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:31.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:31 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:31.624+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:32.159 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T06:27:32.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:27:32.201 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T06:27:32.201 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T06:27:32.201 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.6 2024-09-21T06:27:32.314 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:31 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:31.930+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:32.315 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:31 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:31.930+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:32.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:32.009+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:32.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:32.009+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:32.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:32.009+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.286277+0000 front 2024-09-21T06:27:03.286188+0000 (oldest deadline 2024-09-21T06:27:25.586366+0000) 2024-09-21T06:27:32.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:32 smithi003 systemd[1]: Stopping Ceph osd.6 for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:27:32.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:32 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:32.101+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:32.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:32 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:32.101+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:32.615 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:32.511+0000 7fd38ae04640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:27:32.615 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:32.511+0000 7fd38ae04640 -1 osd.6 94 *** Got signal Terminated *** 2024-09-21T06:27:32.615 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:32.511+0000 7fd38ae04640 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:27:32.616 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:32.382+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:32.616 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:32.382+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:32.616 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:32.382+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:32.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:32 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:32.312+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:32.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:32 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:32.312+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:32.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:32 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:32.312+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:32.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:32 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:32.635+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:32.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:32 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:32.635+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:32.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:32 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:32.635+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:33.303 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:32 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:32.909+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:33.303 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:32 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:32.909+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:33.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:32.974+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:33.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:32.974+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:33.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:32 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:32.974+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.286277+0000 front 2024-09-21T06:27:03.286188+0000 (oldest deadline 2024-09-21T06:27:25.586366+0000) 2024-09-21T06:27:33.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:33 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:33.091+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:33.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:33 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:33.091+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:33.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:33 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:33.301+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:33.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:33 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:33.301+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:33.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:33 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:33.301+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:33.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:33 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:33.424+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:33.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:33 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:33.424+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:33.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:33 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:33.424+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:33.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:33 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:33.608+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:33.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:33 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:33.608+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:33.905 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:33 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:33.608+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:34.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:33 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:33.935+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:34.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:33 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:33.935+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:34.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:33 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:33.935+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.286277+0000 front 2024-09-21T06:27:03.286188+0000 (oldest deadline 2024-09-21T06:27:25.586366+0000) 2024-09-21T06:27:34.351 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:33 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:33.943+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:34.351 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:33 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:33.943+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:34.352 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:33 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:33.943+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:12.228294+0000 front 2024-09-21T06:27:12.228175+0000 (oldest deadline 2024-09-21T06:27:33.928146+0000) 2024-09-21T06:27:34.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:34 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:34.109+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:34.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:34 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:34.109+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:34.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:34 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:34.349+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:34.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:34 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:34.349+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:34.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:34 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:34.349+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:34.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:34 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:34.438+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:34.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:34 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:34.438+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:34.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:34 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:34.438+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:34.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:34 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:34.438+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:34.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:34 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:34.628+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:34.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:34 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:34.628+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:34.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:34 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:34.628+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:35.305 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:34 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:34.925+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:35.306 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:34 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:34.925+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:35.306 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:34 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:34.925+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:12.228294+0000 front 2024-09-21T06:27:12.228175+0000 (oldest deadline 2024-09-21T06:27:33.928146+0000) 2024-09-21T06:27:35.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:34 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:34.895+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:35.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:34 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:34.895+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:35.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:34 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:34.895+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.286277+0000 front 2024-09-21T06:27:03.286188+0000 (oldest deadline 2024-09-21T06:27:25.586366+0000) 2024-09-21T06:27:35.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:35 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:35.094+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:35.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:35 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:35.094+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:35.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:35 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:35.303+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:35.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:35 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:35.303+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:35.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:35 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:35.303+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:35.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:35 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:35.406+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:35.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:35 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:35.406+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:35.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:35 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:35.406+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:35.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:35 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:35.406+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:35.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:35 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:35.649+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:35.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:35 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:35.649+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:35.905 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:35 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:35.649+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:36.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:35 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:35.878+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:36.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:35 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:35.878+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:36.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:35 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:35.878+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:12.228294+0000 front 2024-09-21T06:27:12.228175+0000 (oldest deadline 2024-09-21T06:27:33.928146+0000) 2024-09-21T06:27:36.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:35 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:35.878+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:36.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:35 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:35.878+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:36.340 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:35 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:35.878+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.286277+0000 front 2024-09-21T06:27:03.286188+0000 (oldest deadline 2024-09-21T06:27:25.586366+0000) 2024-09-21T06:27:36.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:36 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:36.079+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:36.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:36 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:36.079+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:36.405 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:36 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:36.079+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:36.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:36 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:36.278+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:36.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:36 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:36.278+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:36.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:36 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:36.278+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:36.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:36 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:36.278+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:36.837 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:36 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:36.414+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:36.837 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:36 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:36.414+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:36.837 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:36 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:36.414+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:36.838 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:36 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:36.414+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:36.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:36 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:36.619+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:36.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:36 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:36.619+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:36.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:36 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:36.619+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:37.090 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:36 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:36.835+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.683930+0000 front 2024-09-21T06:26:48.684012+0000 (oldest deadline 2024-09-21T06:27:12.784146+0000) 2024-09-21T06:27:37.090 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:36 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:36.835+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685439+0000 front 2024-09-21T06:26:55.685247+0000 (oldest deadline 2024-09-21T06:27:19.785148+0000) 2024-09-21T06:27:37.090 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:36 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6[61195]: 2024-09-21T06:27:36.835+0000 7fd3833e7640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:03.286277+0000 front 2024-09-21T06:27:03.286188+0000 (oldest deadline 2024-09-21T06:27:25.586366+0000) 2024-09-21T06:27:37.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:36 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:36.844+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:37.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:36 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:36.844+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:37.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:36 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:36.844+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:12.228294+0000 front 2024-09-21T06:27:12.228175+0000 (oldest deadline 2024-09-21T06:27:33.928146+0000) 2024-09-21T06:27:37.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:37 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:37.040+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:37.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:37 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:37.040+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:37.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:37 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:37.040+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:37.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:37 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:37.268+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:37.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:37 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:37.268+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:37.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:37 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:37.268+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:37.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:37 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:37.268+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:37.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:37 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:37.445+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:37.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:37 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:37.445+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:37.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:37 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:37.445+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:37.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:37 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:37.445+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:37.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:37 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:37.621+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:37.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:37 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:37.621+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:37.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:37 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:37.621+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:38.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:37 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:37.870+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:38.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:37 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:37.870+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:38.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:37 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:37.870+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:12.228294+0000 front 2024-09-21T06:27:12.228175+0000 (oldest deadline 2024-09-21T06:27:33.928146+0000) 2024-09-21T06:27:38.296 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:37 smithi003 podman[110522]: 2024-09-21 06:27:37.872803662 +0000 UTC m=+5.469467783 container died ba155ce0469acc0ab905fd4bb10557446387aa42886403d57840a257b217d223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T06:27:38.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:38 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:38.086+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:38.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:38 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:38.086+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:38.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:38 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:38.086+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:38.590 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:38 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:38.495+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:38.590 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:38 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:38.495+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:38.591 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:38 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:38.495+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:38.591 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:38 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:38.495+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:38.591 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:38 smithi003 podman[110522]: 2024-09-21 06:27:38.295007845 +0000 UTC m=+5.891671965 container cleanup ba155ce0469acc0ab905fd4bb10557446387aa42886403d57840a257b217d223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T06:27:38.591 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:38 smithi003 bash[110522]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6 2024-09-21T06:27:38.591 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:38 smithi003 podman[110550]: 2024-09-21 06:27:38.479644646 +0000 UTC m=+0.604230612 container remove ba155ce0469acc0ab905fd4bb10557446387aa42886403d57840a257b217d223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T06:27:38.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:38 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:38.265+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:38.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:38 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:38.265+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:38.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:38 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:38.265+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:38.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:38 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:38.265+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:39.069 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:38 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:38.657+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:39.070 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:38 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:38.657+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:39.070 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:38 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:38.657+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:39.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:38 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:38.834+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:39.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:38 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:38.834+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:39.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:38 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:38.834+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:12.228294+0000 front 2024-09-21T06:27:12.228175+0000 (oldest deadline 2024-09-21T06:27:33.928146+0000) 2024-09-21T06:27:39.176 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:38 smithi003 podman[110643]: 2024-09-21 06:27:38.823459217 +0000 UTC m=+0.016353847 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:27:39.176 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:38 smithi003 podman[110643]: 2024-09-21 06:27:38.95560314 +0000 UTC m=+0.148497759 container create 81146ff18ae4df0a6664291eed21d410e6e3d03e9a91672f724bcb43a41be4d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T06:27:39.176 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:39 smithi003 podman[110643]: 2024-09-21 06:27:39.175064473 +0000 UTC m=+0.367959104 container init 81146ff18ae4df0a6664291eed21d410e6e3d03e9a91672f724bcb43a41be4d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:27:39.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:39 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:39.067+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:39.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:39 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:39.067+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:39.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:39 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:39.067+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:39.504 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:39 smithi003 podman[110643]: 2024-09-21 06:27:39.178628284 +0000 UTC m=+0.371522905 container start 81146ff18ae4df0a6664291eed21d410e6e3d03e9a91672f724bcb43a41be4d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T06:27:39.504 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:39 smithi003 podman[110643]: 2024-09-21 06:27:39.233290911 +0000 UTC m=+0.426185525 container attach 81146ff18ae4df0a6664291eed21d410e6e3d03e9a91672f724bcb43a41be4d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T06:27:39.504 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 21 06:27:39 smithi003 podman[110643]: 2024-09-21 06:27:39.374039479 +0000 UTC m=+0.566934093 container died 81146ff18ae4df0a6664291eed21d410e6e3d03e9a91672f724bcb43a41be4d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-6-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T06:27:39.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:39 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:39.284+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:39.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:39 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:39.284+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:39.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:39 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:39.284+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:39.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:39 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:39.284+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:39.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:39 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:39.501+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:39.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:39 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:39.501+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:39.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:39 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:39.501+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:39.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:39 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:39.501+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:40.026 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:39 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:39.672+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:40.026 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:39 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:39.672+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:40.026 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:39 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:39.672+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:40.036 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T06:27:40.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:27:40.080 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T06:27:40.080 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T06:27:40.080 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.7 2024-09-21T06:27:40.127 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:39 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:39.860+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:40.127 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:39 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:39.860+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:40.127 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:39 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:39.860+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:12.228294+0000 front 2024-09-21T06:27:12.228175+0000 (oldest deadline 2024-09-21T06:27:33.928146+0000) 2024-09-21T06:27:40.395 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:40 smithi137 systemd[1]: Stopping Ceph osd.7 for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:27:40.397 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:40 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:40.315+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:40.397 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:40 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:40.315+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:40.398 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:40 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:40.315+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:40.398 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:40 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:40.315+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:40.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:40 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:40.024+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:40.496 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:40 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:40.024+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:40.496 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:40 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:40.024+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:40.826 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:40 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:40.395+0000 7f278b46f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:27:40.826 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:40 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:40.395+0000 7f278b46f640 -1 osd.7 94 *** Got signal Terminated *** 2024-09-21T06:27:40.827 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:40 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:40.395+0000 7f278b46f640 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:27:40.827 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:40 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:40.824+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:40.827 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:40 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:40.824+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:40.827 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:40 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:40.824+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:12.228294+0000 front 2024-09-21T06:27:12.228175+0000 (oldest deadline 2024-09-21T06:27:33.928146+0000) 2024-09-21T06:27:40.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:40 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:40.516+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:40.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:40 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:40.516+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:40.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:40 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:40.516+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:40.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:40 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:40.516+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:41.042 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:40 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:40.680+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:41.043 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:40 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:40.680+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:41.043 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:40 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:40.680+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:41.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:41 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:41.040+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:41.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:41 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:41.040+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:41.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:41 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:41.040+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:41.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:41 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:41.280+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:41.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:41 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:41.280+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:41.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:41 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:41.280+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:41.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:41 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:41.280+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:41.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:41 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:41.554+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:41.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:41 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:41.554+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:41.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:41 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:41.554+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:41.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:41 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:41.554+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:42.061 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:41 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:41.681+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:42.062 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:41 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:41.681+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:42.062 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:41 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:41.681+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:42.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:41 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:41.777+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:42.146 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:41 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:41.777+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:42.146 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:41 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:41.777+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:12.228294+0000 front 2024-09-21T06:27:12.228175+0000 (oldest deadline 2024-09-21T06:27:33.928146+0000) 2024-09-21T06:27:42.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:42 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:42.059+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:42.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:42 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:42.059+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:42.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:42 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:42.059+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:42.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:42 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:42.311+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:42.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:42 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:42.311+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:42.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:42 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:42.311+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:42.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:42 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:42.311+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:42.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:42 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:42.311+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:42.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:42 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:42.510+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:42.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:42 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:42.510+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:42.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:42 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:42.510+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:42.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:42 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:42.510+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:43.062 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:42 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:42.726+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:43.063 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:42 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:42.726+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:43.063 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:42 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:42.726+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:43.063 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:42 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:42.726+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:43.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:42 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:42.773+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:43.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:42 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:42.773+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:43.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:42 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:42.773+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:12.228294+0000 front 2024-09-21T06:27:12.228175+0000 (oldest deadline 2024-09-21T06:27:33.928146+0000) 2024-09-21T06:27:43.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:42 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:42.773+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:16.829073+0000 front 2024-09-21T06:27:16.829115+0000 (oldest deadline 2024-09-21T06:27:42.729184+0000) 2024-09-21T06:27:43.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:43 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:43.060+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:43.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:43 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:43.060+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:43.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:43 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:43.060+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:43.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:43 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:43.270+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:43.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:43 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:43.270+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:43.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:43 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:43.270+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:43.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:43 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:43.270+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:43.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:43 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:43.270+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:43.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:43 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:43.545+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:43.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:43 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:43.545+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:43.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:43 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:43.545+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:43.841 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:43 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:43.545+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:43.841 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:43 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:43.545+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:44.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:43 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:43.678+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:44.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:43 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:43.678+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:44.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:43 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:43.678+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:44.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:43 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:43.678+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:44.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:43 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:43.814+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:44.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:43 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:43.814+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:44.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:43 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:43.814+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:12.228294+0000 front 2024-09-21T06:27:12.228175+0000 (oldest deadline 2024-09-21T06:27:33.928146+0000) 2024-09-21T06:27:44.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:43 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:43.814+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:16.829073+0000 front 2024-09-21T06:27:16.829115+0000 (oldest deadline 2024-09-21T06:27:42.729184+0000) 2024-09-21T06:27:44.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:44 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:44.084+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:44.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:44 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:44.084+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:44.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:44 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:44.084+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:44.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:44 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:44.261+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:44.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:44 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:44.261+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:44.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:44 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:44.261+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:44.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:44 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:44.261+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:44.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:44 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:44.261+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:44.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:44 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:44.546+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:44.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:44 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:44.546+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:44.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:44 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:44.546+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:44.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:44 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:44.546+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:44.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:44 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:44.546+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:45.118 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:44 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:44.681+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:45.118 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:44 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:44.681+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:45.119 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:44 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:44.681+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:45.119 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:44 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:44.681+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:45.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:44 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:44.833+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.826007+0000 front 2024-09-21T06:26:55.825916+0000 (oldest deadline 2024-09-21T06:27:19.326134+0000) 2024-09-21T06:27:45.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:44 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:44.833+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:02.827162+0000 front 2024-09-21T06:27:02.826997+0000 (oldest deadline 2024-09-21T06:27:28.727044+0000) 2024-09-21T06:27:45.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:44 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:44.833+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:12.228294+0000 front 2024-09-21T06:27:12.228175+0000 (oldest deadline 2024-09-21T06:27:33.928146+0000) 2024-09-21T06:27:45.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:44 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7[51794]: 2024-09-21T06:27:44.833+0000 7f2783a52640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:16.829073+0000 front 2024-09-21T06:27:16.829115+0000 (oldest deadline 2024-09-21T06:27:42.729184+0000) 2024-09-21T06:27:45.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:45 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:45.116+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:45.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:45 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:45.116+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:45.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:45 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:45.116+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:45.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:45 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:45.116+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:20.288691+0000 front 2024-09-21T06:27:20.288657+0000 (oldest deadline 2024-09-21T06:27:44.388713+0000) 2024-09-21T06:27:45.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:45 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:45.243+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:45.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:45 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:45.243+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:45.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:45 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:45.243+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:45.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:45 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:45.243+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:45.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:45 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:45.243+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:45.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:45 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:45.559+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:45.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:45 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:45.559+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:45.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:45 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:45.559+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:45.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:45 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:45.559+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:45.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:45 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:45.559+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:45.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:45 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:45.649+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:45.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:45 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:45.649+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:45.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:45 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:45.649+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:45.905 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:45 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:45.649+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:46.145 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:45 smithi137 podman[67753]: 2024-09-21 06:27:45.879503007 +0000 UTC m=+5.608583630 container died 97fd481db6127d26f76e7b8f72c0cf9263f7d8878a5a93422439d20916e271c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T06:27:46.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:46 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:46.135+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:46.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:46 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:46.135+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:46.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:46 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:46.135+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:46.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:46 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:46.135+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:20.288691+0000 front 2024-09-21T06:27:20.288657+0000 (oldest deadline 2024-09-21T06:27:44.388713+0000) 2024-09-21T06:27:46.491 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:46 smithi137 podman[67753]: 2024-09-21 06:27:46.287185438 +0000 UTC m=+6.016266070 container cleanup 97fd481db6127d26f76e7b8f72c0cf9263f7d8878a5a93422439d20916e271c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T06:27:46.491 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:46 smithi137 bash[67753]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7 2024-09-21T06:27:46.491 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:46 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:46.233+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:46.491 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:46 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:46.233+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:46.491 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:46 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:46.233+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:46.492 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:46 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:46.233+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:46.492 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:46 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:46.233+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:46.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:46 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:46.520+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:46.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:46 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:46.520+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:46.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:46 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:46.520+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:46.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:46 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:46.520+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:46.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:46 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:46.520+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:46.895 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:46 smithi137 podman[67768]: 2024-09-21 06:27:46.488635055 +0000 UTC m=+0.601290643 container remove 97fd481db6127d26f76e7b8f72c0cf9263f7d8878a5a93422439d20916e271c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True) 2024-09-21T06:27:46.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:46 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:46.637+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:46.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:46 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:46.637+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:46.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:46 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:46.637+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:46.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:46 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:46.637+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:47.160 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:46 smithi137 podman[67853]: 2024-09-21 06:27:46.803296783 +0000 UTC m=+0.015727025 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:27:47.160 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:46 smithi137 podman[67853]: 2024-09-21 06:27:46.922871125 +0000 UTC m=+0.135301357 container create fff6a97ce16cd361ca121980bd5c8555d96fdd9c63f1f148f9871ed7bda39b25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:27:47.160 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:47 smithi137 podman[67853]: 2024-09-21 06:27:47.158900592 +0000 UTC m=+0.371330834 container init fff6a97ce16cd361ca121980bd5c8555d96fdd9c63f1f148f9871ed7bda39b25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T06:27:47.629 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:47 smithi137 podman[67853]: 2024-09-21 06:27:47.162329255 +0000 UTC m=+0.374759490 container start fff6a97ce16cd361ca121980bd5c8555d96fdd9c63f1f148f9871ed7bda39b25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:27:47.629 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:47 smithi137 podman[67853]: 2024-09-21 06:27:47.250652026 +0000 UTC m=+0.463082280 container attach fff6a97ce16cd361ca121980bd5c8555d96fdd9c63f1f148f9871ed7bda39b25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T06:27:47.630 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 21 06:27:47 smithi137 podman[67853]: 2024-09-21 06:27:47.348983195 +0000 UTC m=+0.561413429 container died fff6a97ce16cd361ca121980bd5c8555d96fdd9c63f1f148f9871ed7bda39b25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T06:27:47.630 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:47 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:47.192+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:47.630 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:47 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:47.192+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:47.630 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:47 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:47.192+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:47.630 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:47 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:47.192+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:47.630 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:47 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:47.192+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:47.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:47 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:47.169+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:47.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:47 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:47.169+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:47.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:47 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:47.169+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:47.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:47 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:47.169+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:20.288691+0000 front 2024-09-21T06:27:20.288657+0000 (oldest deadline 2024-09-21T06:27:44.388713+0000) 2024-09-21T06:27:47.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:47 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:47.561+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:47.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:47 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:47.561+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:47.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:47 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:47.561+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:47.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:47 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:47.561+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:47.840 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:47 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:47.561+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:47.970 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T06:27:48.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:27:48.010 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T06:27:48.010 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T06:27:48.010 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.8 2024-09-21T06:27:48.037 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:47 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:47.679+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:48.038 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:47 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:47.679+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:48.038 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:47 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:47.679+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:48.038 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:47 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:47.679+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:48.323 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:48 smithi204 systemd[1]: Stopping Ceph osd.8 for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:27:48.323 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:48 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:48.206+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:48.323 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:48 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:48.206+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:48.323 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:48 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:48.206+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:48.323 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:48 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:48.206+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:20.288691+0000 front 2024-09-21T06:27:20.288657+0000 (oldest deadline 2024-09-21T06:27:44.388713+0000) 2024-09-21T06:27:48.323 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:48 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:48.320+0000 7fdc1c223640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:27:48.324 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:48 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:48.320+0000 7fdc1c223640 -1 osd.8 94 *** Got signal Terminated *** 2024-09-21T06:27:48.324 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:48 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:48.320+0000 7fdc1c223640 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:27:48.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:48 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:48.231+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:48.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:48 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:48.232+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:48.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:48 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:48.232+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:48.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:48 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:48.232+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:48.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:48 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:48.232+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:49.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:48 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:48.597+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:49.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:48 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:48.597+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:49.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:48 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:48.597+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:49.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:48 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:48.597+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:49.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:48 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:48.597+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:49.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:48 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:48.702+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:49.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:48 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:48.702+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:49.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:48 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:48.702+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:49.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:48 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:48.702+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:49.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:49 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:49.188+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:49.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:49 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:49.188+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:49.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:49 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:49.188+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:49.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:49 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:49.188+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:49.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:49 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:49.188+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:49.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:49 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:49.233+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:49.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:49 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:49.233+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:49.655 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:49 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:49.233+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:49.655 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:49 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:49.233+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:20.288691+0000 front 2024-09-21T06:27:20.288657+0000 (oldest deadline 2024-09-21T06:27:44.388713+0000) 2024-09-21T06:27:50.087 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:49 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:49.647+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:50.087 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:49 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:49.647+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:50.088 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:49 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:49.647+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:50.088 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:49 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:49.647+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:50.088 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:49 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:49.647+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:50.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:49 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:49.654+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:50.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:49 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:49.654+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:50.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:49 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:49.654+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:50.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:49 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:49.654+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:50.546 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:50 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:50.189+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:50.547 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:50 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:50.189+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:50.547 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:50 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:50.189+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:50.547 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:50 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:50.189+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:50.547 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:50 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:50.189+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:50.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:50 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:50.194+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:50.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:50 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:50.194+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:50.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:50 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:50.194+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:50.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:50 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:50.194+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:20.288691+0000 front 2024-09-21T06:27:20.288657+0000 (oldest deadline 2024-09-21T06:27:44.388713+0000) 2024-09-21T06:27:51.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:50 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:50.693+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:51.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:50 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:50.693+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:51.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:50 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:50.693+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:51.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:50 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:50.693+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:51.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:50 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:50.693+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:51.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:50 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:50.696+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:51.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:50 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:50.696+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:51.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:50 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:50.696+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:51.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:50 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:50.696+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:51.432 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:51 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:51.220+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:51.433 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:51 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:51.220+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:51.433 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:51 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:51.220+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:51.433 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:51 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:51.220+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:20.288691+0000 front 2024-09-21T06:27:20.288657+0000 (oldest deadline 2024-09-21T06:27:44.388713+0000) 2024-09-21T06:27:51.559 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:51 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:51.207+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:51.559 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:51 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:51.207+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:51.559 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:51 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:51.207+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:51.559 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:51 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:51.207+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:51.559 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:51 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:51.207+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:52.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:51 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:51.655+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:52.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:51 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:51.655+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:52.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:51 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:51.655+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:52.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:51 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:51.655+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:52.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:51 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:51.655+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:52.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:51 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:51.705+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:52.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:51 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:51.705+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:52.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:51 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:51.705+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:52.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:51 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:51.705+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:52.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:52 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:52.241+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:52.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:52 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:52.241+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:52.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:52 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:52.241+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:52.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:52 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:52.241+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:52.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:52 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:52.241+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:52.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:52 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:52.189+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:52.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:52 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:52.189+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:52.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:52 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:52.189+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:52.655 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:52 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:52.189+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:20.288691+0000 front 2024-09-21T06:27:20.288657+0000 (oldest deadline 2024-09-21T06:27:44.388713+0000) 2024-09-21T06:27:52.655 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:52 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:52.189+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:27.190189+0000 front 2024-09-21T06:27:27.190164+0000 (oldest deadline 2024-09-21T06:27:51.790792+0000) 2024-09-21T06:27:53.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:52 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:52.669+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:53.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:52 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:52.669+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:53.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:52 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:52.669+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:53.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:52 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:52.669+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:53.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:52 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:52.669+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:53.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:52 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:52.669+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.932178+0000 front 2024-09-21T06:27:28.932032+0000 (oldest deadline 2024-09-21T06:27:51.831880+0000) 2024-09-21T06:27:53.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:52 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:52.694+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:53.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:52 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:52.694+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:53.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:52 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:52.694+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:53.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:52 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:52.694+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:53.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:53 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:53.235+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:53.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:53 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:53.235+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:53.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:53 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:53.235+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:53.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:53 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:53.235+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:53.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:53 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:53.235+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:53.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:53 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:53.191+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.685375+0000 front 2024-09-21T06:26:55.685489+0000 (oldest deadline 2024-09-21T06:27:20.985319+0000) 2024-09-21T06:27:53.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:53 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:53.191+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.486354+0000 front 2024-09-21T06:27:04.486305+0000 (oldest deadline 2024-09-21T06:27:27.986509+0000) 2024-09-21T06:27:53.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:53 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:53.191+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:09.687727+0000 front 2024-09-21T06:27:09.687749+0000 (oldest deadline 2024-09-21T06:27:35.587722+0000) 2024-09-21T06:27:53.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:53 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:53.191+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:20.288691+0000 front 2024-09-21T06:27:20.288657+0000 (oldest deadline 2024-09-21T06:27:44.388713+0000) 2024-09-21T06:27:53.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:53 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8[52239]: 2024-09-21T06:27:53.191+0000 7fdc14806640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:27.190189+0000 front 2024-09-21T06:27:27.190164+0000 (oldest deadline 2024-09-21T06:27:51.790792+0000) 2024-09-21T06:27:53.998 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:53 smithi204 podman[67688]: 2024-09-21 06:27:53.70569638 +0000 UTC m=+5.501453103 container died 7f21676818a5523c6a228429bd9d2dee41aebc9923ce757e6f61583b32d86f97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T06:27:53.998 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:53 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:53.654+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:53.999 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:53 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:53.654+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:53.999 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:53 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:53.654+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:53.999 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:53 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:53.654+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:53.999 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:53 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:53.654+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:27:54.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:53 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:53.666+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:54.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:53 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:53.666+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:54.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:53 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:53.666+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:54.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:53 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:53.666+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:54.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:53 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:53.666+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:54.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:53 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:53.666+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.932178+0000 front 2024-09-21T06:27:28.932032+0000 (oldest deadline 2024-09-21T06:27:51.831880+0000) 2024-09-21T06:27:54.390 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:54 smithi204 podman[67688]: 2024-09-21 06:27:54.096326256 +0000 UTC m=+5.892082971 container cleanup 7f21676818a5523c6a228429bd9d2dee41aebc9923ce757e6f61583b32d86f97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T06:27:54.391 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:54 smithi204 bash[67688]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8 2024-09-21T06:27:54.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:54 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:54.223+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:54.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:54 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:54.223+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:54.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:54 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:54.223+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:54.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:54 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:54.223+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:54.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:54 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:54.223+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:54.654 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:54 smithi204 podman[67717]: 2024-09-21 06:27:54.389327384 +0000 UTC m=+0.681295294 container remove 7f21676818a5523c6a228429bd9d2dee41aebc9923ce757e6f61583b32d86f97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:27:54.654 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:54 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:54.634+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:54.654 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:54 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:54.634+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:54.655 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:54 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:54.634+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:54.655 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:54 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:54.634+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:54.655 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:54 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:54.634+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:27:55.019 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:54 smithi204 podman[67806]: 2024-09-21 06:27:54.611377131 +0000 UTC m=+0.015811382 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:27:55.019 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:54 smithi204 podman[67806]: 2024-09-21 06:27:54.732141709 +0000 UTC m=+0.136576019 container create 41b95904ccacea8974b629278c974baf233c837e5a43a49e40cf300e4a41d75e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:27:55.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:54 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:54.620+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:55.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:54 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:54.620+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:55.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:54 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:54.620+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:55.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:54 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:54.620+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:55.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:54 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:54.620+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:55.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:54 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:54.620+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.932178+0000 front 2024-09-21T06:27:28.932032+0000 (oldest deadline 2024-09-21T06:27:51.831880+0000) 2024-09-21T06:27:55.404 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:55 smithi204 podman[67806]: 2024-09-21 06:27:55.018050421 +0000 UTC m=+0.422484663 container init 41b95904ccacea8974b629278c974baf233c837e5a43a49e40cf300e4a41d75e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T06:27:55.405 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:55 smithi204 podman[67806]: 2024-09-21 06:27:55.021460132 +0000 UTC m=+0.425894377 container start 41b95904ccacea8974b629278c974baf233c837e5a43a49e40cf300e4a41d75e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T06:27:55.405 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:55 smithi204 podman[67806]: 2024-09-21 06:27:55.101396193 +0000 UTC m=+0.505830435 container attach 41b95904ccacea8974b629278c974baf233c837e5a43a49e40cf300e4a41d75e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T06:27:55.405 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 21 06:27:55 smithi204 podman[67806]: 2024-09-21 06:27:55.219848047 +0000 UTC m=+0.624282290 container died 41b95904ccacea8974b629278c974baf233c837e5a43a49e40cf300e4a41d75e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-8-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:27:55.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:55 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:55.262+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:55.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:55 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:55.262+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:55.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:55 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:55.262+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:55.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:55 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:55.262+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:55.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:55 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:55.262+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:55.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:55 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:55.262+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:27:55.832 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:55 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:55.656+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:55.832 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:55 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:55.656+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:55.832 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:55 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:55.656+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:55.832 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:55 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:55.656+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:55.832 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:55 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:55.656+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:27:55.851 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T06:27:55.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:27:55.941 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T06:27:55.941 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T06:27:55.941 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.9 2024-09-21T06:27:55.970 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:55 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:55.642+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:55.970 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:55 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:55.642+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:55.970 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:55 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:55.642+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:55.970 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:55 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:55.642+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:55.971 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:55 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:55.642+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:55.971 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:55 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:55.642+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.932178+0000 front 2024-09-21T06:27:28.932032+0000 (oldest deadline 2024-09-21T06:27:51.831880+0000) 2024-09-21T06:27:56.265 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:55 smithi003 systemd[1]: Stopping Ceph osd.9 for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:27:56.590 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:56 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:56.263+0000 7f7819f5c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:27:56.590 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:56 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:56.263+0000 7f7819f5c640 -1 osd.9 94 *** Got signal Terminated *** 2024-09-21T06:27:56.590 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:56 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:56.263+0000 7f7819f5c640 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:27:56.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:56 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:56.221+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:56.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:56 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:56.221+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:56.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:56 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:56.221+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:56.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:56 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:56.221+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:56.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:56 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:56.221+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:56.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:56 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:56.221+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:27:57.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:56 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:56.609+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:57.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:56 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:56.609+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:57.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:56 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:56.609+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:57.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:56 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:56.609+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:57.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:56 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:56.609+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:57.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:56 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:56.609+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.932178+0000 front 2024-09-21T06:27:28.932032+0000 (oldest deadline 2024-09-21T06:27:51.831880+0000) 2024-09-21T06:27:57.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:56 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:56.660+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:57.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:56 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:56.660+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:57.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:56 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:56.660+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:57.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:56 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:56.660+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:57.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:56 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:56.660+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:27:57.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:57 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:57.181+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:57.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:57 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:57.181+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:57.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:57 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:57.181+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:57.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:57 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:57.181+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:57.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:57 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:57.181+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:57.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:57 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:57.181+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:27:58.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:57 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:57.640+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:58.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:57 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:57.640+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:58.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:57 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:57.640+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:58.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:57 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:57.640+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:58.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:57 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:57.640+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:58.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:57 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:57.640+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.932178+0000 front 2024-09-21T06:27:28.932032+0000 (oldest deadline 2024-09-21T06:27:51.831880+0000) 2024-09-21T06:27:58.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:57 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:57.668+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:58.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:57 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:57.668+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:58.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:57 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:57.668+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:58.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:57 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:57.668+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:58.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:57 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:57.668+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:27:58.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:58 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:58.178+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:58.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:58 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:58.178+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:58.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:58 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:58.178+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:58.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:58 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:58.178+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:58.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:58 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:58.178+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:58.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:58 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:58.178+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:27:59.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:58 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:58.669+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:27:59.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:58 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:58.669+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:27:59.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:58 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:58.669+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:27:59.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:58 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:58.669+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:27:59.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:58 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:58.669+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:27:59.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:58 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:58.669+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.932178+0000 front 2024-09-21T06:27:28.932032+0000 (oldest deadline 2024-09-21T06:27:51.831880+0000) 2024-09-21T06:27:59.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:58 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:58.672+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:27:59.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:58 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:58.672+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:27:59.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:58 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:58.672+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:27:59.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:58 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:58.672+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:27:59.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:58 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:58.672+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:27:59.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:59 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:59.220+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:27:59.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:59 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:59.220+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:27:59.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:59 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:59.220+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:27:59.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:59 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:59.220+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:27:59.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:59 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:59.220+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:27:59.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:27:59 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:27:59.220+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:28:00.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:59.645+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:28:00.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:59.645+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:28:00.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:59.645+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:28:00.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:59.645+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:28:00.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:59.645+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:28:00.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:27:59 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:27:59.645+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.932178+0000 front 2024-09-21T06:27:28.932032+0000 (oldest deadline 2024-09-21T06:27:51.831880+0000) 2024-09-21T06:28:00.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:59 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:59.688+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:00.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:59 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:59.688+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:00.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:59 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:59.688+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:00.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:59 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:59.688+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:00.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:59 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:59.688+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:00.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:27:59 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:27:59.688+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:00.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:00 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:00.173+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:28:00.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:00 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:00.173+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:28:00.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:00 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:00.173+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:28:00.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:00 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:00.173+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:28:00.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:00 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:00.173+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:28:00.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:00 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:00.173+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:28:01.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:00 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:28:00.621+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.624906+0000 front 2024-09-21T06:26:47.624949+0000 (oldest deadline 2024-09-21T06:27:09.925232+0000) 2024-09-21T06:28:01.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:00 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:28:00.621+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:55.727203+0000 front 2024-09-21T06:26:55.727268+0000 (oldest deadline 2024-09-21T06:27:20.427046+0000) 2024-09-21T06:28:01.090 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:00 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:28:00.621+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:00.427847+0000 front 2024-09-21T06:27:00.427702+0000 (oldest deadline 2024-09-21T06:27:25.727576+0000) 2024-09-21T06:28:01.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:00 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:28:00.621+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:08.528766+0000 front 2024-09-21T06:27:08.528791+0000 (oldest deadline 2024-09-21T06:27:33.828991+0000) 2024-09-21T06:28:01.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:00 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:28:00.621+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:19.730178+0000 front 2024-09-21T06:27:19.730137+0000 (oldest deadline 2024-09-21T06:27:43.230068+0000) 2024-09-21T06:28:01.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:00 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:28:00.621+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.932178+0000 front 2024-09-21T06:27:28.932032+0000 (oldest deadline 2024-09-21T06:27:51.831880+0000) 2024-09-21T06:28:01.091 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:00 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9[67412]: 2024-09-21T06:28:00.621+0000 7f781253f640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:35.233094+0000 front 2024-09-21T06:27:35.233051+0000 (oldest deadline 2024-09-21T06:27:59.933083+0000) 2024-09-21T06:28:01.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:00 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:00.729+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:01.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:00 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:00.729+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:01.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:00 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:00.729+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:01.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:00 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:00.729+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:01.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:00 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:00.729+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:01.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:00 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:00.729+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:01.601 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:01 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:01.215+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:28:01.601 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:01 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:01.215+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:28:01.601 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:01 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:01.215+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:28:01.601 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:01 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:01.215+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:28:01.601 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:01 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:01.215+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:28:01.602 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:01 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:01.215+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:28:02.064 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:01 smithi003 podman[110745]: 2024-09-21 06:28:01.690667017 +0000 UTC m=+5.545659147 container died 29941d79a932174732b1188a396adb658be6fb157d46cc87393bee9b8cdb2d0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T06:28:02.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:01 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:01.771+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:02.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:01 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:01.771+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:02.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:01 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:01.771+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:02.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:01 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:01.771+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:02.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:01 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:01.771+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:02.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:01 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:01.771+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:02.340 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:02 smithi003 podman[110745]: 2024-09-21 06:28:02.06305828 +0000 UTC m=+5.918050396 container cleanup 29941d79a932174732b1188a396adb658be6fb157d46cc87393bee9b8cdb2d0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T06:28:02.340 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:02 smithi003 bash[110745]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9 2024-09-21T06:28:02.340 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:02 smithi003 podman[110761]: 2024-09-21 06:28:02.222716336 +0000 UTC m=+0.527206851 container remove 29941d79a932174732b1188a396adb658be6fb157d46cc87393bee9b8cdb2d0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:28:02.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:02 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:02.234+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:28:02.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:02 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:02.234+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:28:02.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:02 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:02.234+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:28:02.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:02 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:02.234+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:28:02.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:02 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:02.234+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:28:02.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:02 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:02.234+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:28:02.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:02 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:02.234+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:34.963347+0000 front 2024-09-21T06:27:34.963388+0000 (oldest deadline 2024-09-21T06:28:01.363829+0000) 2024-09-21T06:28:02.968 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:02 smithi003 podman[110848]: 2024-09-21 06:28:02.542545614 +0000 UTC m=+0.016711772 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:28:02.968 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:02 smithi003 podman[110848]: 2024-09-21 06:28:02.639909573 +0000 UTC m=+0.114075713 container create 9d96026ebe52b79b30204cedc0874ab7a90f8bac59f57a9412879cab02efc055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T06:28:02.968 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:02 smithi003 podman[110848]: 2024-09-21 06:28:02.883866266 +0000 UTC m=+0.358032424 container init 9d96026ebe52b79b30204cedc0874ab7a90f8bac59f57a9412879cab02efc055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T06:28:02.969 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:02 smithi003 podman[110848]: 2024-09-21 06:28:02.887266694 +0000 UTC m=+0.361432841 container start 9d96026ebe52b79b30204cedc0874ab7a90f8bac59f57a9412879cab02efc055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T06:28:02.969 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:02 smithi003 podman[110848]: 2024-09-21 06:28:02.967094076 +0000 UTC m=+0.441260216 container attach 9d96026ebe52b79b30204cedc0874ab7a90f8bac59f57a9412879cab02efc055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T06:28:03.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:02 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:02.758+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:03.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:02 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:02.758+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:03.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:02 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:02.758+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:03.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:02 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:02.758+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:03.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:02 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:02.758+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:03.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:02 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:02.758+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:03.340 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:03 smithi003 podman[110848]: 2024-09-21 06:28:03.076296764 +0000 UTC m=+0.550462916 container died 9d96026ebe52b79b30204cedc0874ab7a90f8bac59f57a9412879cab02efc055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T06:28:03.583 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T06:28:03.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:03 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:03.253+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:28:03.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:03 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:03.253+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:28:03.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:03 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:03.253+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:28:03.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:03 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:03.253+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:28:03.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:03 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:03.253+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:28:03.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:03 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:03.253+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:28:03.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:03 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:03.253+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:34.963347+0000 front 2024-09-21T06:27:34.963388+0000 (oldest deadline 2024-09-21T06:28:01.363829+0000) 2024-09-21T06:28:03.648 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:03 smithi003 podman[110871]: 2024-09-21 06:28:03.570019702 +0000 UTC m=+0.487298118 container remove 9d96026ebe52b79b30204cedc0874ab7a90f8bac59f57a9412879cab02efc055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T06:28:03.648 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:03 smithi003 systemd[1]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.9.service: Deactivated successfully. 2024-09-21T06:28:03.648 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:03 smithi003 systemd[1]: Stopped Ceph osd.9 for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:28:03.648 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 21 06:28:03 smithi003 systemd[1]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.9.service: Consumed 1min 4.432s CPU time. 2024-09-21T06:28:03.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:03.664 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T06:28:03.664 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T06:28:03.665 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.10 2024-09-21T06:28:04.008 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:03 smithi137 systemd[1]: Stopping Ceph osd.10 for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:28:04.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:03 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:03.807+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:04.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:03 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:03.807+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:04.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:03 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:03.807+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:04.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:03 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:03.807+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:04.156 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:03 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:03.807+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:04.156 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:03 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:03.807+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:04.265 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:04 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:04.006+0000 7fccbfa50640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:04.265 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:04 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:04.006+0000 7fccbfa50640 -1 osd.10 94 *** Got signal Terminated *** 2024-09-21T06:28:04.266 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:04 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:04.006+0000 7fccbfa50640 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:28:04.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:04 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:04.264+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:28:04.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:04 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:04.264+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:28:04.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:04 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:04.264+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:28:04.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:04 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:04.264+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:28:04.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:04 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:04.264+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:28:04.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:04 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:04.264+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:28:04.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:04 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:04.264+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:34.963347+0000 front 2024-09-21T06:27:34.963388+0000 (oldest deadline 2024-09-21T06:28:01.363829+0000) 2024-09-21T06:28:05.102 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:04 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:04.830+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:05.102 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:04 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:04.830+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:05.102 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:04 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:04.830+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:05.102 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:04 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:04.830+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:05.103 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:04 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:04.830+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:05.103 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:04 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:04.830+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:05.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:05 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:05.311+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:28:05.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:05 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:05.311+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:28:05.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:05 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:05.311+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:28:05.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:05 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:05.311+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:28:05.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:05 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:05.311+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:28:05.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:05 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:05.311+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:28:05.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:05 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:05.311+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:34.963347+0000 front 2024-09-21T06:27:34.963388+0000 (oldest deadline 2024-09-21T06:28:01.363829+0000) 2024-09-21T06:28:06.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:05 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:05.823+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:06.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:05 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:05.823+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:06.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:05 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:05.823+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:06.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:05 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:05.823+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:06.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:05 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:05.823+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:06.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:05 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:05.823+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:06.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:06 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:06.340+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:28:06.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:06 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:06.340+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:28:06.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:06 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:06.340+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:28:06.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:06 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:06.340+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:28:06.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:06 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:06.340+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:28:06.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:06 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:06.340+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:28:06.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:06 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:06.340+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:34.963347+0000 front 2024-09-21T06:27:34.963388+0000 (oldest deadline 2024-09-21T06:28:01.363829+0000) 2024-09-21T06:28:07.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:06 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:06.865+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:07.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:06 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:06.865+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:07.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:06 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:06.865+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:07.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:06 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:06.865+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:07.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:06 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:06.865+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:07.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:06 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:06.865+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:07.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:06 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:06.865+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-21T06:27:45.082781+0000 front 2024-09-21T06:27:45.082899+0000 (oldest deadline 2024-09-21T06:28:06.782595+0000) 2024-09-21T06:28:07.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:07 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:07.339+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:28:07.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:07 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:07.339+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:28:07.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:07 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:07.339+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:28:07.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:07 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:07.339+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:28:07.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:07 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:07.339+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:28:07.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:07 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:07.339+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:28:07.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:07 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:07.339+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:34.963347+0000 front 2024-09-21T06:27:34.963388+0000 (oldest deadline 2024-09-21T06:28:01.363829+0000) 2024-09-21T06:28:08.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:07 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:07.852+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:08.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:07 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:07.852+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:08.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:07 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:07.852+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:08.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:07 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:07.852+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:08.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:07 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:07.852+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:08.156 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:07 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:07.852+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:08.156 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:07 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:07.852+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-21T06:27:45.082781+0000 front 2024-09-21T06:27:45.082899+0000 (oldest deadline 2024-09-21T06:28:06.782595+0000) 2024-09-21T06:28:08.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:08 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:08.379+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:48.838114+0000 front 2024-09-21T06:26:48.838158+0000 (oldest deadline 2024-09-21T06:27:14.038696+0000) 2024-09-21T06:28:08.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:08 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:08.379+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:56.939785+0000 front 2024-09-21T06:26:56.939607+0000 (oldest deadline 2024-09-21T06:27:17.439822+0000) 2024-09-21T06:28:08.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:08 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:08.379+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:01.540984+0000 front 2024-09-21T06:27:01.541112+0000 (oldest deadline 2024-09-21T06:27:26.240989+0000) 2024-09-21T06:28:08.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:08 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:08.379+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-09-21T06:27:11.542364+0000 front 2024-09-21T06:27:11.542227+0000 (oldest deadline 2024-09-21T06:27:35.642290+0000) 2024-09-21T06:28:08.645 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:08 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:08.379+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:18.543171+0000 front 2024-09-21T06:27:18.543204+0000 (oldest deadline 2024-09-21T06:27:42.043365+0000) 2024-09-21T06:28:08.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:08 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:08.379+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:29.062997+0000 front 2024-09-21T06:27:29.062883+0000 (oldest deadline 2024-09-21T06:27:54.962789+0000) 2024-09-21T06:28:08.646 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:08 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10[58185]: 2024-09-21T06:28:08.379+0000 7fccbc03b640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:34.963347+0000 front 2024-09-21T06:27:34.963388+0000 (oldest deadline 2024-09-21T06:28:01.363829+0000) 2024-09-21T06:28:09.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:08 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:08.889+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:09.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:08 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:08.889+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:09.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:08 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:08.889+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:09.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:08 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:08.889+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:09.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:08 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:08.889+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:09.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:08 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:08.889+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:09.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:08 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:08.889+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-21T06:27:45.082781+0000 front 2024-09-21T06:27:45.082899+0000 (oldest deadline 2024-09-21T06:28:06.782595+0000) 2024-09-21T06:28:09.850 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:09 smithi137 podman[67967]: 2024-09-21 06:28:09.524860717 +0000 UTC m=+5.654333100 container died b5881673a662445899fc063fd2f6a1c173d9399124c951e7fad6193c7c79e070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T06:28:09.850 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:09 smithi137 podman[67967]: 2024-09-21 06:28:09.847953283 +0000 UTC m=+5.977425663 container cleanup b5881673a662445899fc063fd2f6a1c173d9399124c951e7fad6193c7c79e070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:28:10.145 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:09 smithi137 bash[67967]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10 2024-09-21T06:28:10.145 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:10 smithi137 podman[67986]: 2024-09-21 06:28:10.090953151 +0000 UTC m=+0.557605356 container remove b5881673a662445899fc063fd2f6a1c173d9399124c951e7fad6193c7c79e070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:28:10.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:09 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:09.912+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:10.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:09 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:09.912+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:10.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:09 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:09.912+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:10.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:09 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:09.912+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:10.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:09 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:09.912+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:10.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:09 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:09.912+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:10.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:09 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:09.912+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-21T06:27:45.082781+0000 front 2024-09-21T06:27:45.082899+0000 (oldest deadline 2024-09-21T06:28:06.782595+0000) 2024-09-21T06:28:10.828 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:10 smithi137 podman[68076]: 2024-09-21 06:28:10.396398363 +0000 UTC m=+0.016296322 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:28:10.828 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:10 smithi137 podman[68076]: 2024-09-21 06:28:10.508492996 +0000 UTC m=+0.128390949 container create 309e75cdb839ceda5e44e42bc8fb3913c328712c73d14634f46755fbfbacd0c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:28:10.829 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:10 smithi137 podman[68076]: 2024-09-21 06:28:10.727378163 +0000 UTC m=+0.347276122 container init 309e75cdb839ceda5e44e42bc8fb3913c328712c73d14634f46755fbfbacd0c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T06:28:10.829 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:10 smithi137 podman[68076]: 2024-09-21 06:28:10.730547691 +0000 UTC m=+0.350445647 container start 309e75cdb839ceda5e44e42bc8fb3913c328712c73d14634f46755fbfbacd0c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T06:28:10.829 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:10 smithi137 podman[68076]: 2024-09-21 06:28:10.827381724 +0000 UTC m=+0.447279678 container attach 309e75cdb839ceda5e44e42bc8fb3913c328712c73d14634f46755fbfbacd0c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True) 2024-09-21T06:28:11.145 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 21 06:28:10 smithi137 podman[68076]: 2024-09-21 06:28:10.915737486 +0000 UTC m=+0.535635440 container died 309e75cdb839ceda5e44e42bc8fb3913c328712c73d14634f46755fbfbacd0c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:28:11.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:10 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:10.938+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:11.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:10 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:10.938+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:11.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:10 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:10.938+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:11.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:10 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:10.938+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:11.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:10 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:10.938+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:11.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:10 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:10.938+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:11.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:10 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:10.938+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-21T06:27:45.082781+0000 front 2024-09-21T06:27:45.082899+0000 (oldest deadline 2024-09-21T06:28:06.782595+0000) 2024-09-21T06:28:11.622 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T06:28:11.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:11.664 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T06:28:11.664 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T06:28:11.664 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.11 2024-09-21T06:28:11.967 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:11 smithi204 systemd[1]: Stopping Ceph osd.11 for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:28:12.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:11 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:11.965+0000 7f38cc34d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:12.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:11 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:11.965+0000 7f38cc34d640 -1 osd.11 94 *** Got signal Terminated *** 2024-09-21T06:28:12.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:11 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:11.965+0000 7f38cc34d640 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:28:12.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:11 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:11.980+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:12.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:11 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:11.980+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:12.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:11 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:11.980+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:12.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:11 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:11.980+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:12.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:11 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:11.980+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:12.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:11 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:11.980+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:12.406 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:11 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:11.980+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-21T06:27:45.082781+0000 front 2024-09-21T06:27:45.082899+0000 (oldest deadline 2024-09-21T06:28:06.782595+0000) 2024-09-21T06:28:13.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:13 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:13.020+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:13.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:13 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:13.020+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:13.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:13 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:13.020+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:13.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:13 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:13.020+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:13.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:13 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:13.020+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:13.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:13 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:13.020+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:13.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:13 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:13.020+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-21T06:27:45.082781+0000 front 2024-09-21T06:27:45.082899+0000 (oldest deadline 2024-09-21T06:28:06.782595+0000) 2024-09-21T06:28:14.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:14 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:14.008+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:14.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:14 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:14.008+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:14.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:14 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:14.008+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:14.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:14 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:14.008+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:14.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:14 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:14.008+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:14.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:14 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:14.008+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:14.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:14 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:14.008+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-21T06:27:45.082781+0000 front 2024-09-21T06:27:45.082899+0000 (oldest deadline 2024-09-21T06:28:06.782595+0000) 2024-09-21T06:28:15.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:15 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:15.042+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:15.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:15 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:15.042+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:15.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:15 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:15.042+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:15.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:15 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:15.042+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:15.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:15 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:15.042+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:15.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:15 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:15.042+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:15.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:15 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:15.042+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-21T06:27:45.082781+0000 front 2024-09-21T06:27:45.082899+0000 (oldest deadline 2024-09-21T06:28:06.782595+0000) 2024-09-21T06:28:16.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:16 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:16.032+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-21T06:26:47.072961+0000 front 2024-09-21T06:26:47.072768+0000 (oldest deadline 2024-09-21T06:27:09.972656+0000) 2024-09-21T06:28:16.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:16 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:16.032+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-21T06:26:54.074073+0000 front 2024-09-21T06:26:54.073949+0000 (oldest deadline 2024-09-21T06:27:20.474468+0000) 2024-09-21T06:28:16.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:16 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:16.032+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-21T06:27:04.575592+0000 front 2024-09-21T06:27:04.575429+0000 (oldest deadline 2024-09-21T06:27:30.475588+0000) 2024-09-21T06:28:16.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:16 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:16.032+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-21T06:27:21.378611+0000 front 2024-09-21T06:27:21.378666+0000 (oldest deadline 2024-09-21T06:27:42.478494+0000) 2024-09-21T06:28:16.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:16 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:16.032+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-21T06:27:28.879605+0000 front 2024-09-21T06:27:28.879648+0000 (oldest deadline 2024-09-21T06:27:53.579762+0000) 2024-09-21T06:28:16.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:16 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:16.032+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-09-21T06:27:33.580199+0000 front 2024-09-21T06:27:33.580268+0000 (oldest deadline 2024-09-21T06:27:59.480220+0000) 2024-09-21T06:28:16.405 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:16 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11[58607]: 2024-09-21T06:28:16.032+0000 7f38c8938640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-21T06:27:45.082781+0000 front 2024-09-21T06:27:45.082899+0000 (oldest deadline 2024-09-21T06:28:06.782595+0000) 2024-09-21T06:28:17.734 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:17 smithi204 podman[67909]: 2024-09-21 06:28:17.462470271 +0000 UTC m=+5.596801738 container died 897464b2b4c55d17154d43f6732c3b88ed42316f6a27663675c6bdeaf053d924 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T06:28:18.118 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:17 smithi204 podman[67909]: 2024-09-21 06:28:17.840528937 +0000 UTC m=+5.974860389 container cleanup 897464b2b4c55d17154d43f6732c3b88ed42316f6a27663675c6bdeaf053d924 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T06:28:18.118 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:17 smithi204 bash[67909]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11 2024-09-21T06:28:18.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:18 smithi204 podman[67925]: 2024-09-21 06:28:18.117062035 +0000 UTC m=+0.649657862 container remove 897464b2b4c55d17154d43f6732c3b88ed42316f6a27663675c6bdeaf053d924 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T06:28:18.721 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:18 smithi204 podman[68016]: 2024-09-21 06:28:18.365082778 +0000 UTC m=+0.015865927 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:28:18.721 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:18 smithi204 podman[68016]: 2024-09-21 06:28:18.492794376 +0000 UTC m=+0.143577516 container create c034be9e0e115feae5150491b84c7df1bfd15155c4e94d5b94cdffef94247b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T06:28:18.721 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:18 smithi204 podman[68016]: 2024-09-21 06:28:18.720210202 +0000 UTC m=+0.370993343 container init c034be9e0e115feae5150491b84c7df1bfd15155c4e94d5b94cdffef94247b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T06:28:19.154 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:18 smithi204 podman[68016]: 2024-09-21 06:28:18.723658047 +0000 UTC m=+0.374441197 container start c034be9e0e115feae5150491b84c7df1bfd15155c4e94d5b94cdffef94247b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T06:28:19.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:18 smithi204 podman[68016]: 2024-09-21 06:28:18.811789185 +0000 UTC m=+0.462572326 container attach c034be9e0e115feae5150491b84c7df1bfd15155c4e94d5b94cdffef94247b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:28:19.155 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 21 06:28:18 smithi204 podman[68016]: 2024-09-21 06:28:18.912480357 +0000 UTC m=+0.563263501 container died c034be9e0e115feae5150491b84c7df1bfd15155c4e94d5b94cdffef94247b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T06:28:19.513 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T06:28:19.552 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:19.552 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T06:28:19.552 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T06:28:19.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.a 2024-09-21T06:28:19.840 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 21 06:28:19 smithi003 systemd[1]: Stopping Ceph mds.a for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:28:20.340 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 21 06:28:19 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-a[74148]: 2024-09-21T06:28:19.855+0000 7f7d3df03640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:20.340 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 21 06:28:19 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-a[74148]: 2024-09-21T06:28:19.855+0000 7f7d3df03640 -1 mds.a *** got signal Terminated *** 2024-09-21T06:28:24.132 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 21 06:28:23 smithi003 podman[110962]: 2024-09-21 06:28:23.867642359 +0000 UTC m=+4.112630142 container died 5e6cd1bf61a6ff3ff49fef9f46b1c4da301864f36ada2b116bd86f4e2ea8d0f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T06:28:24.406 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 21 06:28:24 smithi003 podman[110962]: 2024-09-21 06:28:24.21287457 +0000 UTC m=+4.457862341 container cleanup 5e6cd1bf61a6ff3ff49fef9f46b1c4da301864f36ada2b116bd86f4e2ea8d0f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-a, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T06:28:24.407 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 21 06:28:24 smithi003 bash[110962]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-a 2024-09-21T06:28:24.629 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T06:28:24.665 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:24.665 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T06:28:24.665 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T06:28:24.666 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.d 2024-09-21T06:28:25.010 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 21 06:28:24 smithi003 systemd[1]: Stopping Ceph mds.d for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:28:25.340 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 21 06:28:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-d[73024]: 2024-09-21T06:28:25.007+0000 7f94e1109640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:25.340 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 21 06:28:25 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-d[73024]: 2024-09-21T06:28:25.007+0000 7f94e1109640 -1 mds.d *** got signal Terminated *** 2024-09-21T06:28:29.275 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 21 06:28:29 smithi003 podman[111084]: 2024-09-21 06:28:29.021301946 +0000 UTC m=+4.125003224 container died e72b103822bf05dc1c6d5af8bb555bb65b7ca9548807b199cd5d883405ad43ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T06:28:29.590 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 21 06:28:29 smithi003 podman[111084]: 2024-09-21 06:28:29.449196959 +0000 UTC m=+4.552898226 container cleanup e72b103822bf05dc1c6d5af8bb555bb65b7ca9548807b199cd5d883405ad43ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:28:29.590 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 21 06:28:29 smithi003 bash[111084]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-d 2024-09-21T06:28:29.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T06:28:29.886 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 21 06:28:29 smithi003 podman[111096]: 2024-09-21 06:28:29.625922529 +0000 UTC m=+0.605584291 container remove e72b103822bf05dc1c6d5af8bb555bb65b7ca9548807b199cd5d883405ad43ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T06:28:29.886 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 21 06:28:29 smithi003 systemd[1]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.d.service: Deactivated successfully. 2024-09-21T06:28:29.886 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 21 06:28:29 smithi003 systemd[1]: Stopped Ceph mds.d for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:28:29.886 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 21 06:28:29 smithi003 systemd[1]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.d.service: Consumed 1.094s CPU time. 2024-09-21T06:28:29.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:29.898 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T06:28:29.898 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T06:28:29.898 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.g 2024-09-21T06:28:30.221 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:28:29 smithi003 systemd[1]: Stopping Ceph mds.g for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:28:30.590 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:28:30 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-g[73763]: 2024-09-21T06:28:30.219+0000 7f34ea935640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:30.590 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:28:30 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-g[73763]: 2024-09-21T06:28:30.219+0000 7f34ea935640 -1 mds.g *** got signal Terminated *** 2024-09-21T06:28:32.572 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:28:32 smithi003 podman[111207]: 2024-09-21 06:28:32.304389183 +0000 UTC m=+2.176723129 container died 9257d07f9b96a8f80eef8d70f1c78fd4253797782cf9e9262094cf456ff11c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-g, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T06:28:33.156 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:28:32 smithi003 podman[111207]: 2024-09-21 06:28:32.895272759 +0000 UTC m=+2.767606703 container cleanup 9257d07f9b96a8f80eef8d70f1c78fd4253797782cf9e9262094cf456ff11c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T06:28:33.157 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 21 06:28:32 smithi003 bash[111207]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-g 2024-09-21T06:28:33.333 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T06:28:33.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:33.369 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T06:28:33.370 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T06:28:33.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.j 2024-09-21T06:28:33.692 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:28:33 smithi003 systemd[1]: Stopping Ceph mds.j for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:28:34.090 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:28:33 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-j[72660]: 2024-09-21T06:28:33.690+0000 7f1e4faeb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:34.090 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:28:33 smithi003 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-j[72660]: 2024-09-21T06:28:33.690+0000 7f1e4faeb640 -1 mds.j *** got signal Terminated *** 2024-09-21T06:28:35.090 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:28:34 smithi003 podman[111340]: 2024-09-21 06:28:34.80812418 +0000 UTC m=+1.202300516 container died 1b18b84989ff4d2faa9af7f1d7acc6ca484e5344455722becaede758af73192c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-j, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T06:28:35.510 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:28:35 smithi003 podman[111340]: 2024-09-21 06:28:35.307874207 +0000 UTC m=+1.702050586 container cleanup 1b18b84989ff4d2faa9af7f1d7acc6ca484e5344455722becaede758af73192c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-j, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:28:35.510 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 21 06:28:35 smithi003 bash[111340]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-j 2024-09-21T06:28:35.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T06:28:35.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:35.674 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T06:28:35.674 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T06:28:35.675 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.b 2024-09-21T06:28:35.983 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:28:35 smithi137 systemd[1]: Stopping Ceph mds.b for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:28:36.245 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:28:35 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b[63057]: 2024-09-21T06:28:35.981+0000 7f2e9f1f6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:36.245 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:28:35 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b[63057]: 2024-09-21T06:28:35.981+0000 7f2e9f1f6640 -1 mds.b *** got signal Terminated *** 2024-09-21T06:28:40.349 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:28:39 smithi137 podman[68179]: 2024-09-21 06:28:39.994543632 +0000 UTC m=+4.115648033 container died 35f23dd78deddf49f51a526bf343fd2d1dd644a9ab4d1d711cd959c8ee06e847 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T06:28:40.645 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:28:40 smithi137 podman[68179]: 2024-09-21 06:28:40.447193287 +0000 UTC m=+4.568297689 container cleanup 35f23dd78deddf49f51a526bf343fd2d1dd644a9ab4d1d711cd959c8ee06e847 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:28:40.645 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 21 06:28:40 smithi137 bash[68179]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-b 2024-09-21T06:28:40.890 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T06:28:40.926 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:40.927 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T06:28:40.927 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T06:28:40.927 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.e 2024-09-21T06:28:41.145 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:28:41 smithi137 systemd[1]: Stopping Ceph mds.e for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:28:41.645 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:28:41 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e[62522]: 2024-09-21T06:28:41.292+0000 7f5b71dd8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:41.645 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:28:41 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e[62522]: 2024-09-21T06:28:41.292+0000 7f5b71dd8640 -1 mds.e *** got signal Terminated *** 2024-09-21T06:28:44.645 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:28:44 smithi137 podman[68313]: 2024-09-21 06:28:44.325763635 +0000 UTC m=+3.182017961 container died 367b67e561674325194d629fc3fe477d637e6b2e2bf7d123370f77254b501ccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T06:28:44.969 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:28:44 smithi137 podman[68313]: 2024-09-21 06:28:44.725705323 +0000 UTC m=+3.581959654 container cleanup 367b67e561674325194d629fc3fe477d637e6b2e2bf7d123370f77254b501ccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T06:28:44.970 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 21 06:28:44 smithi137 bash[68313]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-e 2024-09-21T06:28:45.104 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T06:28:45.137 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:45.137 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T06:28:45.138 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T06:28:45.138 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.h 2024-09-21T06:28:45.395 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:28:45 smithi137 systemd[1]: Stopping Ceph mds.h for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:28:45.895 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:28:45 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-h[62791]: 2024-09-21T06:28:45.462+0000 7fa573b80640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:45.895 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:28:45 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-h[62791]: 2024-09-21T06:28:45.462+0000 7fa573b80640 -1 mds.h *** got signal Terminated *** 2024-09-21T06:28:48.598 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:28:48 smithi137 podman[68434]: 2024-09-21 06:28:48.317300339 +0000 UTC m=+2.964566080 container died 155436ebcdc74ac44d60e50a03293990a322f0217e8ac5ff26976550933f8235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-h, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T06:28:48.895 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:28:48 smithi137 podman[68434]: 2024-09-21 06:28:48.679246483 +0000 UTC m=+3.326512216 container cleanup 155436ebcdc74ac44d60e50a03293990a322f0217e8ac5ff26976550933f8235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-h, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:28:48.895 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 21 06:28:48 smithi137 bash[68434]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-h 2024-09-21T06:28:49.109 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T06:28:49.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:49.141 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T06:28:49.142 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T06:28:49.142 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.k 2024-09-21T06:28:49.645 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:28:49 smithi137 systemd[1]: Stopping Ceph mds.k for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:28:49.645 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:28:49 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-k[62239]: 2024-09-21T06:28:49.441+0000 7fc615001640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:49.645 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:28:49 smithi137 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-k[62239]: 2024-09-21T06:28:49.441+0000 7fc615001640 -1 mds.k *** got signal Terminated *** 2024-09-21T06:28:52.618 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:28:52 smithi137 podman[68556]: 2024-09-21 06:28:52.341572099 +0000 UTC m=+2.982123097 container died b1d6d9ac62cbf7c71259b402ab70c3f584fdad135bc81acc412433afbb9fe6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2) 2024-09-21T06:28:52.877 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:28:52 smithi137 podman[68556]: 2024-09-21 06:28:52.691649897 +0000 UTC m=+3.332200895 container cleanup b1d6d9ac62cbf7c71259b402ab70c3f584fdad135bc81acc412433afbb9fe6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-k, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T06:28:52.877 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:28:52 smithi137 bash[68556]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-k 2024-09-21T06:28:53.114 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T06:28:53.135 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:28:52 smithi137 podman[68568]: 2024-09-21 06:28:52.87582507 +0000 UTC m=+0.521562040 container remove b1d6d9ac62cbf7c71259b402ab70c3f584fdad135bc81acc412433afbb9fe6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T06:28:53.136 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:28:53 smithi137 systemd[1]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.k.service: Deactivated successfully. 2024-09-21T06:28:53.136 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:28:53 smithi137 systemd[1]: Stopped Ceph mds.k for 38d3c75c-77df-11ef-baf5-efdc52797490. 2024-09-21T06:28:53.136 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 21 06:28:53 smithi137 systemd[1]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.k.service: Consumed 2.114s CPU time. 2024-09-21T06:28:53.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:53.147 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T06:28:53.147 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T06:28:53.147 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.c 2024-09-21T06:28:53.522 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:28:53 smithi204 systemd[1]: Stopping Ceph mds.c for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:28:53.904 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:28:53 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-c[63294]: 2024-09-21T06:28:53.631+0000 7ff184c0b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:53.904 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:28:53 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-c[63294]: 2024-09-21T06:28:53.631+0000 7ff184c0b640 -1 mds.c *** got signal Terminated *** 2024-09-21T06:28:56.602 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:28:56 smithi204 podman[68118]: 2024-09-21 06:28:56.320390307 +0000 UTC m=+2.947663935 container died a9f07c06da34ceacca1d1f0560814156d44136883a1cdaaa4674836e93ac1d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-c, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T06:28:56.904 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:28:56 smithi204 podman[68118]: 2024-09-21 06:28:56.698609191 +0000 UTC m=+3.325882825 container cleanup a9f07c06da34ceacca1d1f0560814156d44136883a1cdaaa4674836e93ac1d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:28:56.904 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 21 06:28:56 smithi204 bash[68118]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-c 2024-09-21T06:28:57.086 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T06:28:57.122 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:57.122 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T06:28:57.123 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T06:28:57.123 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.f 2024-09-21T06:28:57.404 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:28:57 smithi204 systemd[1]: Stopping Ceph mds.f for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:28:57.904 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:28:57 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-f[63025]: 2024-09-21T06:28:57.451+0000 7fb403949640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:57.904 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:28:57 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-f[63025]: 2024-09-21T06:28:57.451+0000 7fb403949640 -1 mds.f *** got signal Terminated *** 2024-09-21T06:29:00.588 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:29:00 smithi204 podman[68249]: 2024-09-21 06:29:00.31617413 +0000 UTC m=+2.976809848 container died 682f556a8944104417fbb329ae0a187f43e3a285033d727dfddbf6cb5336f89a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-f, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:29:00.904 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:29:00 smithi204 podman[68249]: 2024-09-21 06:29:00.694463356 +0000 UTC m=+3.355099075 container cleanup 682f556a8944104417fbb329ae0a187f43e3a285033d727dfddbf6cb5336f89a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-f, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:29:00.904 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 21 06:29:00 smithi204 bash[68249]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-f 2024-09-21T06:29:01.140 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T06:29:01.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:29:01.174 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T06:29:01.174 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T06:29:01.174 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.i 2024-09-21T06:29:01.491 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:29:01 smithi204 systemd[1]: Stopping Ceph mds.i for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:29:01.904 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:29:01 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-i[63563]: 2024-09-21T06:29:01.488+0000 7f23cda43640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:29:01.904 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:29:01 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-i[63563]: 2024-09-21T06:29:01.488+0000 7f23cda43640 -1 mds.i *** got signal Terminated *** 2024-09-21T06:29:04.599 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:29:04 smithi204 podman[68369]: 2024-09-21 06:29:04.315451214 +0000 UTC m=+2.927462219 container died 648de31a88de9dfb31caa3cc319191790ca3bc0951f4b670d9427359df6af911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-21T06:29:04.904 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:29:04 smithi204 podman[68369]: 2024-09-21 06:29:04.706277636 +0000 UTC m=+3.318288639 container cleanup 648de31a88de9dfb31caa3cc319191790ca3bc0951f4b670d9427359df6af911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-i, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:29:04.904 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 21 06:29:04 smithi204 bash[68369]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-i 2024-09-21T06:29:05.081 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T06:29:05.113 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:29:05.114 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T06:29:05.114 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T06:29:05.114 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.l 2024-09-21T06:29:05.404 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:29:05 smithi204 systemd[1]: Stopping Ceph mds.l for 38d3c75c-77df-11ef-baf5-efdc52797490... 2024-09-21T06:29:05.904 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:29:05 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-l[62743]: 2024-09-21T06:29:05.443+0000 7f7838431640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:29:05.904 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:29:05 smithi204 ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-l[62743]: 2024-09-21T06:29:05.443+0000 7f7838431640 -1 mds.l *** got signal Terminated *** 2024-09-21T06:29:08.586 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:29:08 smithi204 podman[68489]: 2024-09-21 06:29:08.322115753 +0000 UTC m=+2.994862158 container died 3debaa0031e23eab5005b195055950b467971ba305501733450d872dc4235568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-l, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:29:08.904 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:29:08 smithi204 podman[68489]: 2024-09-21 06:29:08.677150273 +0000 UTC m=+3.349896713 container cleanup 3debaa0031e23eab5005b195055950b467971ba305501733450d872dc4235568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-l, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T06:29:08.904 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 21 06:29:08 smithi204 bash[68489]: ceph-38d3c75c-77df-11ef-baf5-efdc52797490-mds-l 2024-09-21T06:29:09.102 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-38d3c75c-77df-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T06:29:09.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:29:09.134 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T06:29:09.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T06:29:09.347 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:29:11.962 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T06:29:12.165 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:29:14.750 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T06:29:14.953 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:29:17.366 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:29:17.399 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:29:17.432 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:29:17.459 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T06:29:17.461 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913523/remote/smithi003/crash 2024-09-21T06:29:17.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/crash -- . 2024-09-21T06:29:17.498 INFO:teuthology.orchestra.run.smithi003.stderr:tar: /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T06:29:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:tar: Error is not recoverable: exiting now 2024-09-21T06:29:17.500 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913523/remote/smithi137/crash 2024-09-21T06:29:17.501 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/crash -- . 2024-09-21T06:29:17.542 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T06:29:17.543 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2024-09-21T06:29:17.544 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913523/remote/smithi204/crash 2024-09-21T06:29:17.545 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/crash -- . 2024-09-21T06:29:17.583 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T06:29:17.583 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2024-09-21T06:29:17.586 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T06:29:17.586 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T06:29:17.628 INFO:tasks.cephadm:Compressing logs... 2024-09-21T06:29:17.629 DEBUG:teuthology.orchestra.run.smithi003:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:29:17.671 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:29:17.674 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:29:17.696 INFO:teuthology.orchestra.run.smithi003.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T06:29:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:: No such file or directory 2024-09-21T06:29:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T06:29:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T06:29:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T06:29:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T06:29:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T06:29:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T06:29:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T06:29:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mon.a.log: 83.7% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T06:29:17.701 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T06:29:17.701 INFO:teuthology.orchestra.run.smithi204.stderr:: No such file or directory 2024-09-21T06:29:17.701 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T06:29:17.701 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T06:29:17.701 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T06:29:17.701 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T06:29:17.702 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T06:29:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.log 2024-09-21T06:29:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.6.log: 90.1% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T06:29:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T06:29:17.704 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T06:29:17.704 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mon.c.log: /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T06:29:17.704 INFO:teuthology.orchestra.run.smithi204.stderr: 80.8% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T06:29:17.704 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T06:29:17.704 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.log 2024-09-21T06:29:17.705 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.5.log: /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.log: 90.2% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T06:29:17.705 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T06:29:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.log: 86.7% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T06:29:17.706 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T06:29:17.707 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T06:29:17.707 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.l.log: 86.7% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T06:29:17.707 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T06:29:17.712 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.8.log: 91.0% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T06:29:17.714 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.d.log: 90.9% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T06:29:17.714 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose 95.0% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T06:29:17.714 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T06:29:17.714 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T06:29:17.718 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T06:29:17.721 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.j.log: 95.1% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T06:29:17.721 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T06:29:17.725 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mgr.z.log: 92.6% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T06:29:17.730 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T06:29:17.731 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T06:29:17.733 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T06:29:17.733 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T06:29:17.733 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T06:29:17.733 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.10.log: /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T06:29:17.733 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T06:29:17.736 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T06:29:17.736 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T06:29:17.736 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.b.log: /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T06:29:17.736 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T06:29:17.736 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T06:29:17.737 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.log 2024-09-21T06:29:17.737 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T06:29:17.737 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T06:29:17.737 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T06:29:17.737 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T06:29:17.743 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T06:29:17.751 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T06:29:17.759 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T06:29:17.789 INFO:teuthology.orchestra.run.smithi003.stderr: 94.3% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T06:29:17.790 INFO:teuthology.orchestra.run.smithi204.stderr: 94.5% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T06:29:17.798 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 80.8% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T06:29:17.812 INFO:teuthology.orchestra.run.smithi137.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T06:29:17.915 INFO:teuthology.orchestra.run.smithi137.stderr: 92.5% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T06:29:17.986 INFO:teuthology.orchestra.run.smithi137.stderr: 90.9% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T06:29:18.094 INFO:teuthology.orchestra.run.smithi003.stderr: 90.2% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T06:29:18.134 INFO:teuthology.orchestra.run.smithi137.stderr: 93.6% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T06:29:18.197 INFO:teuthology.orchestra.run.smithi204.stderr: 91.7% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T06:29:18.309 INFO:teuthology.orchestra.run.smithi137.stderr: 90.2% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T06:29:18.309 INFO:teuthology.orchestra.run.smithi137.stderr: 86.7% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T06:29:19.068 INFO:teuthology.orchestra.run.smithi003.stderr: 91.4% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T06:29:19.471 INFO:teuthology.orchestra.run.smithi137.stderr: 91.6% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T06:29:19.705 INFO:teuthology.orchestra.run.smithi003.stderr: 95.8% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T06:29:21.224 INFO:teuthology.orchestra.run.smithi204.stderr: 96.5% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T06:29:22.390 INFO:teuthology.orchestra.run.smithi204.stderr: 96.9% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T06:29:22.779 INFO:teuthology.orchestra.run.smithi204.stderr: 96.6% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T06:29:28.113 INFO:teuthology.orchestra.run.smithi003.stderr: 91.7% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T06:29:30.984 INFO:teuthology.orchestra.run.smithi003.stderr: 91.5% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T06:29:31.812 INFO:teuthology.orchestra.run.smithi204.stderr: 91.1% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T06:29:31.940 INFO:teuthology.orchestra.run.smithi003.stderr: 91.9% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T06:29:33.695 INFO:teuthology.orchestra.run.smithi204.stderr: 91.7% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T06:29:34.183 INFO:teuthology.orchestra.run.smithi003.stderr: 91.6% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T06:29:34.184 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-21T06:29:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:real 0m16.500s 2024-09-21T06:29:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:user 0m56.510s 2024-09-21T06:29:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:sys 0m1.694s 2024-09-21T06:29:35.648 INFO:teuthology.orchestra.run.smithi204.stderr: 91.2% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T06:29:36.068 INFO:teuthology.orchestra.run.smithi204.stderr: 91.2% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T06:29:36.071 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-21T06:29:36.071 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m18.382s 2024-09-21T06:29:36.071 INFO:teuthology.orchestra.run.smithi204.stderr:user 1m17.529s 2024-09-21T06:29:36.071 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m2.534s 2024-09-21T06:29:40.168 INFO:teuthology.orchestra.run.smithi137.stderr: 95.4% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T06:29:52.360 INFO:teuthology.orchestra.run.smithi137.stderr: 91.4% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T06:30:01.147 INFO:teuthology.orchestra.run.smithi137.stderr: 91.8% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T06:30:07.414 INFO:teuthology.orchestra.run.smithi137.stderr: 91.6% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T06:30:10.067 INFO:teuthology.orchestra.run.smithi137.stderr: 91.5% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T06:30:48.099 INFO:teuthology.orchestra.run.smithi137.stderr: 94.0% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T06:32:52.604 INFO:teuthology.orchestra.run.smithi137.stderr: 93.5% -- replaced with /var/log/ceph/38d3c75c-77df-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T06:32:52.607 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-09-21T06:32:52.607 INFO:teuthology.orchestra.run.smithi137.stderr:real 3m34.918s 2024-09-21T06:32:52.607 INFO:teuthology.orchestra.run.smithi137.stderr:user 4m0.387s 2024-09-21T06:32:52.607 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m8.647s 2024-09-21T06:32:52.608 INFO:tasks.cephadm:Archiving logs... 2024-09-21T06:32:52.609 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913523/remote/smithi003/log 2024-09-21T06:32:52.610 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:33:00.804 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913523/remote/smithi137/log 2024-09-21T06:33:00.805 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:33:36.375 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913523/remote/smithi204/log 2024-09-21T06:33:36.376 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:33:47.149 INFO:tasks.cephadm:Removing cluster... 2024-09-21T06:33:47.149 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 --force 2024-09-21T06:33:47.361 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:33:47.705 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 --force 2024-09-21T06:33:47.910 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:33:48.459 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 38d3c75c-77df-11ef-baf5-efdc52797490 --force 2024-09-21T06:33:48.664 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 38d3c75c-77df-11ef-baf5-efdc52797490 2024-09-21T06:33:49.052 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T06:33:49.052 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T06:33:49.072 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T06:33:49.089 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T06:33:49.106 INFO:tasks.cephadm:Teardown complete 2024-09-21T06:33:49.106 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T06:33:49.121 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T06:33:49.122 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:33:49.125 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:33:49.132 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:33:49.245 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T06:33:49.256 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T06:33:49.256 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:33:49.259 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:33:49.261 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:33:49.273 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:33:49.274 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:33:49.276 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:33:49.280 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:33:49.281 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-09-21T06:33:49.281 INFO:teuthology.orchestra.run.smithi003.stdout:^* hv01.front.sepia.ceph.com 3 8 377 39 -122us[ -136us] +/- 51ms 2024-09-21T06:33:49.281 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 43 +233us[ +219us] +/- 78ms 2024-09-21T06:33:49.281 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 171 -46us[ -59us] +/- 80ms 2024-09-21T06:33:49.281 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:33:49.281 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:33:49.282 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-09-21T06:33:49.282 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 -111us[ -86us] +/- 51ms 2024-09-21T06:33:49.282 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 +243us[ +268us] +/- 78ms 2024-09-21T06:33:49.282 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -27us[-2748ns] +/- 82ms 2024-09-21T06:33:49.282 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:33:49.290 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:33:49.291 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-09-21T06:33:49.291 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 -133us[ -129us] +/- 51ms 2024-09-21T06:33:49.291 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 +213us[ +217us] +/- 78ms 2024-09-21T06:33:49.291 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -49us[ -49us] +/- 82ms 2024-09-21T06:33:49.291 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:33:49.292 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T06:33:49.304 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T06:33:49.305 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T06:33:49.316 DEBUG:teuthology.orchestra.run.smithi003:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:33:49.409 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:33:49.500 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:33:49.565 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:33:49.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:33:49.597 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:33:49.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:33:49.626 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:33:49.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:33:49.653 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T06:33:49.665 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T06:33:49.676 INFO:teuthology.task.internal:Duration was 2180.987025 seconds 2024-09-21T06:33:49.676 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T06:33:49.687 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T06:33:49.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:33:49.690 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:33:49.693 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:33:49.731 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:33:49.732 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:33:49.750 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:33:50.200 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T06:33:50.200 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2024-09-21T06:33:50.201 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:33:50.226 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-09-21T06:33:50.226 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:33:50.251 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-09-21T06:33:50.251 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:33:50.282 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T06:33:50.283 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:33:50.285 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:33:50.294 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:33:50.350 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T06:33:50.351 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:33:50.356 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:33:50.362 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:33:51.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T06:33:51.600 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T06:33:51.601 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:33:51.626 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:33:51.679 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:33:51.710 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T06:33:51.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:33:51.726 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:33:51.728 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:33:51.748 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2024-09-21T06:33:51.765 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-09-21T06:33:51.777 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-09-21T06:33:51.804 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:33:51.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:33:51.819 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:33:51.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:33:51.851 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:33:51.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:33:51.867 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T06:33:51.879 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T06:33:51.880 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913523/remote/smithi003 2024-09-21T06:33:51.880 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:33:51.940 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913523/remote/smithi137 2024-09-21T06:33:51.940 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:33:51.979 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913523/remote/smithi204 2024-09-21T06:33:51.979 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:33:52.017 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T06:33:52.018 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:33:52.020 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:33:52.023 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:33:52.069 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T06:33:52.083 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T06:33:52.083 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T06:33:52.094 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T06:33:52.094 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:33:52.097 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:33:52.099 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:33:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:33 /home/ubuntu/cephtest 2024-09-21T06:33:52.113 INFO:teuthology.orchestra.run.smithi137.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:33 /home/ubuntu/cephtest 2024-09-21T06:33:52.125 INFO:teuthology.orchestra.run.smithi204.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:33 /home/ubuntu/cephtest 2024-09-21T06:33:52.127 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T06:33:52.139 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T06:33:52.166 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} duration: 2180.9870252609253 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi003 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=a3e885ac48dd4740a77456faadbfa1b6 status: fail success: false 2024-09-21T06:33:52.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T06:33:52.248 INFO:teuthology.run:FAIL